Using Lynx Version 2.8.3rel.1, I recently began experiencing difficulty in accessing the New York Times (NYT) site. I believe I have identified the cookie-related problem and, even though it appears to be the "fault" of that site and not of Lynx, I would like to describe it, since it may be informative, and also pose a couple of questions.
The main NYT page, <http://www.nytimes.com/>, is openly accessible, but actually reading the articles listed there requires a login (after a free registration process). The login problem I experience is this: When I sign-in with the "Save your ID and Password" option checked, it doesn't "take"; attempts to access articles, during that session, or later sessions, get redirected back to the sign-in page, <http://www.nytimes.com/auth/login>. If, however, I sign-in with the "Save your ID and Password" option *unchecked*, I can then successfully access articles during the current session; unsurprisingly, I must then sign-in again to access articles during a separate later session. (I will note that I run Lynx with persistent cookies enabled.) Here's why, I think: On initial access to <http://www.nytimes.com/>, the site sets a cookie (if it is not already present) called RMID via a request such as this: Set-cookie: RMID=<RMID>; expires=Friday, 22-Nov-2002 18:33:54 GMT; path=/; domain=.nytimes.com (For brevity and security, I'll used <Cookie-Name> to represent a value for Cookie-Name.) So far, so good: Lynx sets this cookie as requested. When I sign-in with the "Save your ID and Password" option checked, the server attempts to set two cookies, NYT-S and RDB, via requests such as these: Set-cookie: NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 13:31:54; path=/; domain=.nytimes.com Set-cookie: RDB=<RDB>; expires=Tuesday, 05-May-2002 14:31:54; path=/; domain=.nytimes.com Lynx evidently refuses to set either of these cookies. Why? Well, note what is missing in the "expires=" argument: there is no trailing " GMT"! Because of this, Lynx apparently views the Set-cookie requests as malformed and discards them. (FWIW, it also does this when ".nytimes.com" is listed in .lynxrc among the "cookie_loose_invalid_domains".) As readers of LYNX-DEV are undoubtedly aware, there never was a IETF RFC specification for "Netscape-style" ("version 1") cookies. The closest thing to an "official" specification is that found in the Netscape document "Persistent Client State HTTP Cookies," <http://www.netscape.com/newsref/std/cookie_spec.html>, which states, The date string [the argument for "expires="] is formatted as: Wdy, DD-Mon-YYYY HH:MM:SS GMT This is based on RFC 822, RFC 850, RFC 1036, and RFC 1123, with the variations that the only legal time zone is GMT and the separators between the elements of the date must be dashes. expires is an optional attribute. If not specified, the cookie will expire when the user's session ends. According to this, the trailing " GMT" is mandatory, and Lynx is justified (if perhaps a bit persnickety) in rejecting Netscape-style cookies which omit it. Alternatively, when I sign-in with the "Save your ID and Password" option unchecked, the NYT server sends Set-cookie requests like these: Set-cookie: NYT-S=<NYT-S>; path=/; domain=.nytimes.com Set-cookie: RDB=<RDB>; expires=Tuesday, 05-May-2002 14:33:56; path=/; domain=.nytimes.com Lynx still rejects the RDB cookie, but, in this case, it accepts the NYT-S cookie, which is apparently the important one for logging in, as a transient, current-session-only cookie; hence I am able to access articles during that session. I've notified the NYT site, <mailto:[EMAIL PROTECTED]>, about the missing "GMT". Perhaps they will fix it. In the meantime, here are two questions: 1) Since "GMT" is the "only legal time zone" for Netscape-style cookies, should otherwise well-formed Set-cookie requests be rejected simply for omitting it? 2) Lynx's trace log doesn't seem to flag the rejected cookies. Their rejection, so far as I can tell, can only be inferred by their omission from subsequent GET requests. For example, in the first scenario above, the log shows this: ------------------------------- Initial access of the main page ------------------------------- Set-cookie: RMID=<RMID>; expires=Friday, 22-Nov-2002 18:33:54 GMT; path=/; domain=.nytimes.com [...] HTMIME: PICKED UP Set-Cookie: 'RMID=<RMID>; expires=Friday, 22-Nov-2002 18:33:54 GMT; path=/; domain=.nytimes.com' [...] LYSetCookie called with host 'www.nytimes.com', path '/auth/login?URI=http://www.nytimes.com/aponline/national', and Set-Cookie: 'RMID=<RMID>; expires=Friday, 22-Nov-2002 18:33:54 GMT; path=/; domain=.nytimes.com' LYmktime: Parsing 'Friday, 22-Nov-2002 18:33:54 GMT' LYmktime: clock=1037990034, ctime=Fri Nov 22 13:33:54 2002 LYProcessSetCookie: attr=value pair: 'RMID=<RMID>' expires: 1037990034, Fri Nov 22 13:33:54 2002 --------------------------------------------------------------------------- Later in same session, after submitting username & password to sign-in page --------------------------------------------------------------------------- Set-cookie: NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 13:31:54; path=/; domain=.nytimes.com Set-cookie: RDB=<RDB>; expires=Tuesday, 05-May-2002 14:31:54; path=/; domain=.nytimes.com [...] HTMIME: PICKED UP Set-Cookie: 'NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 13:31:54; path=/; domain=.nytimes.com' [...] HTMIME: PICKED UP Set-Cookie: 'RDB=<RDB>; expires=Tuesday, 05-May-2002 14:31:54; path=/; domain=.nytimes.com' [...] LYSetCookie called with host 'www.nytimes.com', path '/auth', and Set-Cookie: 'NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 13:31:54; path=/; domain=.nytimes.com, RDB=<RDB>; expires=Tuesday, 05-May-2002 14:31:54; path=/; domain=.nytimes.com' LYmktime: Parsing 'Friday, 11-Nov-2002 13:31:54' LYmktime: Parsing 'Tuesday, 05-May-2002 14:31:54' LYProcessSetCookie: attr=value pair: 'NYT-S=<NYT-S>' LYProcessSetCookie: attr=value pair: 'RDB=<RDB>' --------------------------------------------------------------------------- Only later still, when attempting to access an article, is it now clear, to me anyway, that the NYT-S and RDB cookies weren't actually set. --------------------------------------------------------------------------- LYCookie: Searching for 'www.nytimes.com:80', '/auth/chk_login'. Checking cookie 1bdd38 RMID=<RMID> www.nytimes.com .nytimes.com 1 /auth/chk_login / 0 HTTP: Sending Cookie2: $Version ="1" HTTP: Sending Cookie: RMID=<RMID> [...] Writing: [...] User-Agent: Lynx/2.8.3rel.1 libwww-FM/2.14 SSL-MM/1.4.1 Cookie2: $Version="1" Cookie: RMID=<RMID> ---------------------------------- Sending HTTP request. HTTP: WRITE delivered OK 2) (cont.) Might it be possible to have the trace log flag rejected cookies in a more obvious manner? -- David Mosher <[EMAIL PROTECTED]> ; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to [EMAIL PROTECTED]
