I would say: if it adds no package dependencies, put it right in. Aristid Am 06.02.2012 22:09 schrieb "Myles C. Maxfield" <[email protected]>:
> After all these commits have been flying around, I have yet another > question: > > the 'HTTP' package defines Network.Browser which is a State monad which > keeps state about a "browser" (i.e. a cookie jar, a proxy, redirection > parameters, etc.) It would be pretty straightforward to implement this kind > of functionality on top of http-conduit. > > I was originally going to do it and release it as its own package, but it > may be beneficial to add such a module to the existing http-conduit > package. Should I add it in to the existing package, or release it as its > own package? > > --Myles > > On Mon, Feb 6, 2012 at 12:15 AM, Michael Snoyman <[email protected]>wrote: > >> Just an FYI for everyone: Myles sent an (incredibly thorough) pull >> request to handle cookies: >> >> https://github.com/snoyberg/http-conduit/pull/13 >> >> Thanks! >> >> On Sun, Feb 5, 2012 at 8:20 AM, Myles C. Maxfield >> <[email protected]> wrote: >> > 1. The spec defines a grammar for the attributes. They're in uppercase. >> > 2. Yes - 1.3 is the first version that lists DiffTime as an instance of >> > RealFrac (so I can use the 'floor' function to pull out the number of >> > seconds to render it) >> > 3. I'll see what I can do. >> > >> > --Myles >> > >> > >> > On Sat, Feb 4, 2012 at 9:06 PM, Michael Snoyman <[email protected]> >> wrote: >> >> >> >> Looks good, a few questions/requests: >> >> >> >> 1. Is there a reason to upper-case all the attributes? >> >> 2. Is the time >= 1.3 a requirements? Because that can cause a lot of >> >> trouble for people. >> >> 3. Can you send the patch as a Github pull request? It's easier to >> >> track that way. >> >> >> >> Michael >> >> >> >> On Sat, Feb 4, 2012 at 1:21 AM, Myles C. Maxfield >> >> <[email protected]> wrote: >> >> > Here is the patch to Web.Cookie. I didn't modify the tests at all >> >> > because >> >> > they were already broken - they looked like they hadn't been updated >> >> > since >> >> > SetCookie only had 5 parameters. I did verify by hand that the patch >> >> > works, >> >> > though. >> >> > >> >> > Thanks, >> >> > Myles >> >> > >> >> > >> >> > On Thu, Feb 2, 2012 at 11:26 PM, Myles C. Maxfield >> >> > <[email protected]> wrote: >> >> >> >> >> >> Alright, I'll make a small patch that adds 2 fields to SetCookie: >> >> >> setCookieMaxAge :: Maybe DiffTime >> >> >> setCookieSecureOnly :: Bool >> >> >> >> >> >> I've also gotten started on those cookie functions. I'm currently >> >> >> writing >> >> >> tests for them. >> >> >> >> >> >> @Chris: The best advice I can give is that Chrome (what I'm using >> as a >> >> >> source on all this) has the data baked into a .cc file. However, >> they >> >> >> have >> >> >> directions in a README and a script which will parse the list and >> >> >> generate >> >> >> that source file. I recommend doing this. That way, the Haskell >> module >> >> >> would >> >> >> have 2 source files: one file that reads the list and generates the >> >> >> second >> >> >> file, which is a very large source file that contains each element >> in >> >> >> the >> >> >> list. The list should export `elem`-type queries. I'm not quite sure >> >> >> how to >> >> >> handle wildcards that appear in the list - that part is up to you. >> >> >> Thanks >> >> >> for helping out with this :] >> >> >> >> >> >> --Myles >> >> >> >> >> >> >> >> >> On Thu, Feb 2, 2012 at 10:53 PM, Michael Snoyman < >> [email protected]> >> >> >> wrote: >> >> >>> >> >> >>> Looks good to me too. I agree with Aristid: let's make the change >> to >> >> >>> cookie itself. Do you want to send a pull request? I'm also >> >> >>> considering making the SetCookie constructor hidden like we have >> for >> >> >>> Request, so that if in the future we realize we need to add some >> other >> >> >>> settings, it doesn't break the API. >> >> >>> >> >> >>> Chris: I would recommend compiling it into the module. Best bet >> would >> >> >>> likely being converting the source file to Haskell source. >> >> >>> >> >> >>> Michael >> >> >>> >> >> >>> On Fri, Feb 3, 2012 at 6:32 AM, Myles C. Maxfield >> >> >>> <[email protected]> wrote: >> >> >>> > Alright. After reading the spec, I have these questions / >> concerns: >> >> >>> > >> >> >>> > The spec supports the "Max-Age" cookie attribute, which >> Web.Cookies >> >> >>> > doesn't. >> >> >>> > >> >> >>> > I see two possible solutions to this. The first is to have >> >> >>> > parseSetCookie >> >> >>> > take a UTCTime as an argument which will represent the current >> time >> >> >>> > so >> >> >>> > it >> >> >>> > can populate the setCookieExpires field by adding the Max-Age >> >> >>> > attribute >> >> >>> > to >> >> >>> > the current time. Alternatively, that function can return an IO >> >> >>> > SetCookie so >> >> >>> > it can ask for the current time by itself (which I think is >> inferior >> >> >>> > to >> >> >>> > taking the current time as an argument). Note that the spec says >> to >> >> >>> > prefer >> >> >>> > Max-Age over Expires. >> >> >>> > Add a field to SetCookie of type Maybe DiffTime which represents >> the >> >> >>> > Max-Age >> >> >>> > attribute >> >> >>> > >> >> >>> > Cookie code should be aware of the Public Suffix List as a part >> of >> >> >>> > its >> >> >>> > domain verification. The cookie code only needs to be able to >> tell >> >> >>> > if a >> >> >>> > specific string is in the list (W.Ascii -> Bool) >> >> >>> > >> >> >>> > I propose making an entirely unrelated package, >> public-suffix-list, >> >> >>> > with a >> >> >>> > module Network.PublicSuffixList, which will expose this >> function, as >> >> >>> > well as >> >> >>> > functions about parsing the list itself. Thoughts? >> >> >>> > >> >> >>> > Web.Cookie doesn't have a "secure-only" attribute. Adding one in >> is >> >> >>> > straightforward enough. >> >> >>> > The spec describes cookies as a property of HTTP, not of the >> World >> >> >>> > Wide >> >> >>> > Web. >> >> >>> > Perhaps "Web.Cookie" should be renamed? Just a thought; it >> doesn't >> >> >>> > really >> >> >>> > matter to me. >> >> >>> > >> >> >>> > As for Network.HTTP.Conduit.Cookie, the spec describes in section >> >> >>> > 5.3 >> >> >>> > "Storage Model" what fields a Cookie has. Here is my proposal for >> >> >>> > the >> >> >>> > functions it will expose: >> >> >>> > >> >> >>> > receiveSetCookie :: SetCookie -> Req.Request m -> UTCTime -> >> Bool -> >> >> >>> > CookieJar -> CookieJar >> >> >>> > >> >> >>> > Runs the algorithm described in section 5.3 "Storage Model" >> >> >>> > The UTCTime is the current-time, the Bool is whether or not the >> >> >>> > caller >> >> >>> > is an >> >> >>> > HTTP-based API (as opposed to JavaScript or anything else) >> >> >>> > >> >> >>> > updateCookieJar :: Res.Response a -> Req.Request m -> UTCTime -> >> >> >>> > CookieJar >> >> >>> > -> (CookieJar, Res.Response a) >> >> >>> > >> >> >>> > Applies "receiveSetCookie" to a Response. The output CookieJar is >> >> >>> > stripped >> >> >>> > of any Set-Cookie headers. >> >> >>> > Specifies "True" for the Bool in receiveSetCookie >> >> >>> > >> >> >>> > computeCookieString :: Req.Request m -> CookieJar -> UTCTime -> >> Bool >> >> >>> > -> >> >> >>> > (W.Ascii, CookieJar) >> >> >>> > >> >> >>> > Runs the algorithm described in section 5.4 "The Cookie Header" >> >> >>> > The UTCTime and Bool are the same as in receiveSetCookie >> >> >>> > >> >> >>> > insertCookiesIntoRequest :: Req.Request m -> CookieJar -> >> UTCTime -> >> >> >>> > (Req.Request m, CookieJar) >> >> >>> > >> >> >>> > Applies "computeCookieString" to a Request. The output cookie jar >> >> >>> > has >> >> >>> > updated last-accessed-times. >> >> >>> > Specifies "True" for the Bool in computeCookieString >> >> >>> > >> >> >>> > evictExpiredCookies :: CookieJar -> UTCTime -> CookieJar >> >> >>> > >> >> >>> > Runs the algorithm described in the last part of section 5.3 >> >> >>> > "Storage >> >> >>> > Model" >> >> >>> > >> >> >>> > This will make the relevant part of 'http' look like: >> >> >>> > >> >> >>> > go count req'' cookie_jar'' = do >> >> >>> > now <- liftIO $ getCurrentTime >> >> >>> > let (req', cookie_jar') = insertCookiesIntoRequest req'' >> >> >>> > (evictExpiredCookies cookie_jar'' now) now >> >> >>> > res' <- httpRaw req' manager >> >> >>> > let (cookie_jar, res) = updateCookieJar res' req' now >> >> >>> > cookie_jar' >> >> >>> > case getRedirectedRequest req' (responseHeaders res) >> >> >>> > (W.statusCode >> >> >>> > (statusCode res)) of >> >> >>> > Just req -> go (count - 1) req cookie_jar >> >> >>> > Nothing -> return res >> >> >>> > >> >> >>> > I plan to not allow for a user-supplied cookieFilter function. If >> >> >>> > they >> >> >>> > want >> >> >>> > that functionality, they can re-implement the >> redirection-following >> >> >>> > logic. >> >> >>> > >> >> >>> > Any thoughts on any of this? >> >> >>> > >> >> >>> > Thanks, >> >> >>> > Myles >> >> >>> > >> >> >>> > On Wed, Feb 1, 2012 at 5:19 PM, Myles C. Maxfield >> >> >>> > <[email protected]> >> >> >>> > wrote: >> >> >>> >> >> >> >>> >> Nope. I'm not. The RFC is very explicit about how to handle >> >> >>> >> cookies. >> >> >>> >> As >> >> >>> >> soon as I'm finished making sense of it (in terms of Haskell) >> I'll >> >> >>> >> send >> >> >>> >> another proposal email. >> >> >>> >> >> >> >>> >> On Feb 1, 2012 3:25 AM, "Michael Snoyman" <[email protected]> >> >> >>> >> wrote: >> >> >>> >>> >> >> >>> >>> You mean you're *not* making this proposal? >> >> >>> >>> >> >> >>> >>> On Wed, Feb 1, 2012 at 7:30 AM, Myles C. Maxfield >> >> >>> >>> <[email protected]> wrote: >> >> >>> >>> > Well, this is embarrassing. Please disregard my previous >> email. >> >> >>> >>> > I >> >> >>> >>> > should >> >> >>> >>> > learn to read the RFC *before* submitting proposals. >> >> >>> >>> > >> >> >>> >>> > --Myles >> >> >>> >>> > >> >> >>> >>> > >> >> >>> >>> > On Tue, Jan 31, 2012 at 6:37 PM, Myles C. Maxfield >> >> >>> >>> > <[email protected]> wrote: >> >> >>> >>> >> >> >> >>> >>> >> Here are my initial ideas about supporting cookies. Note >> that >> >> >>> >>> >> I'm >> >> >>> >>> >> using >> >> >>> >>> >> Chrome for ideas since it's open source. >> >> >>> >>> >> >> >> >>> >>> >> Network/HTTP/Conduit/Cookies.hs file >> >> >>> >>> >> Exporting the following symbols: >> >> >>> >>> >> >> >> >>> >>> >> type StuffedCookie = SetCookie >> >> >>> >>> >> >> >> >>> >>> >> A regular SetCookie can have Nothing for its Domain and Path >> >> >>> >>> >> attributes. A >> >> >>> >>> >> StuffedCookie has to have these fields set. >> >> >>> >>> >> >> >> >>> >>> >> type CookieJar = [StuffedCookie] >> >> >>> >>> >> >> >> >>> >>> >> Chrome's cookie jar is implemented as (the C++ equivalent >> of) >> >> >>> >>> >> Map >> >> >>> >>> >> W.Ascii >> >> >>> >>> >> StuffedCookie. The key is the "eTLD+1" of the domain, so >> >> >>> >>> >> lookups >> >> >>> >>> >> for >> >> >>> >>> >> all >> >> >>> >>> >> cookies for a given domain are fast. >> >> >>> >>> >> I think I'll stay with just a list of StuffedCookies just to >> >> >>> >>> >> keep >> >> >>> >>> >> it >> >> >>> >>> >> simple. Perhaps a later revision can implement the faster >> map. >> >> >>> >>> >> >> >> >>> >>> >> getRelevantCookies :: Request m -> CookieJar -> UTCTime -> >> >> >>> >>> >> (CookieJar, >> >> >>> >>> >> Cookies) >> >> >>> >>> >> >> >> >>> >>> >> Gets all the cookies from the cookie jar that should be set >> for >> >> >>> >>> >> the >> >> >>> >>> >> given >> >> >>> >>> >> Request. >> >> >>> >>> >> The time argument is whatever "now" is (it's pulled out of >> the >> >> >>> >>> >> function so >> >> >>> >>> >> the function can remain pure and easily testable) >> >> >>> >>> >> The function will also remove expired cookies from the >> cookie >> >> >>> >>> >> jar >> >> >>> >>> >> (given >> >> >>> >>> >> what "now" is) and return the filtered cookie jar >> >> >>> >>> >> >> >> >>> >>> >> putRelevantCookies :: Request m -> CookieJar -> >> [StuffedCookie] >> >> >>> >>> >> -> >> >> >>> >>> >> CookieJar >> >> >>> >>> >> >> >> >>> >>> >> Insert cookies from a server response into the cookie jar. >> >> >>> >>> >> The first argument is only used for checking to see which >> >> >>> >>> >> cookies >> >> >>> >>> >> are >> >> >>> >>> >> valid (which cookies match the requested domain, etc, so >> >> >>> >>> >> site1.com >> >> >>> >>> >> can't set >> >> >>> >>> >> a cookie for site2.com) >> >> >>> >>> >> >> >> >>> >>> >> stuffCookie :: Request m -> SetCookie -> StuffedCookie >> >> >>> >>> >> >> >> >>> >>> >> If the SetCookie's fields are Nothing, fill them in given >> the >> >> >>> >>> >> Request >> >> >>> >>> >> from >> >> >>> >>> >> which it originated >> >> >>> >>> >> >> >> >>> >>> >> getCookies :: Response a -> ([SetCookie], Response a) >> >> >>> >>> >> >> >> >>> >>> >> Pull cookies out of a server response. Return the response >> with >> >> >>> >>> >> the >> >> >>> >>> >> Set-Cookie headers filtered out >> >> >>> >>> >> >> >> >>> >>> >> putCookies :: Request a -> Cookies -> Request a >> >> >>> >>> >> >> >> >>> >>> >> A wrapper around renderCookies. Inserts some cookies into a >> >> >>> >>> >> request. >> >> >>> >>> >> Doesn't overwrite cookies that are already set in the >> request >> >> >>> >>> >> >> >> >>> >>> >> These functions will be exported from Network.HTTP.Conduit >> as >> >> >>> >>> >> well, so >> >> >>> >>> >> callers can use them to re-implement redirection chains >> >> >>> >>> >> I won't implement a cookie filtering function (like what >> >> >>> >>> >> Network.Browser >> >> >>> >>> >> has) >> >> >>> >>> >> >> >> >>> >>> >> If you want to have arbitrary handling of cookies, >> re-implement >> >> >>> >>> >> redirection following. It's not very difficult if you use >> the >> >> >>> >>> >> API >> >> >>> >>> >> provided, >> >> >>> >>> >> and the 'http' function is open source so you can use that >> as a >> >> >>> >>> >> reference. >> >> >>> >>> >> >> >> >>> >>> >> I will implement the functions according to RFC 6265 >> >> >>> >>> >> I will also need to write the following functions. Should >> they >> >> >>> >>> >> also be >> >> >>> >>> >> exported? >> >> >>> >>> >> >> >> >>> >>> >> canonicalizeDomain :: W.Ascii -> W.Ascii >> >> >>> >>> >> >> >> >>> >>> >> turns "..a.b.c..d.com..." to "a.b.c.d.com" >> >> >>> >>> >> Technically necessary for domain matching (Chrome does it) >> >> >>> >>> >> Perhaps unnecessary for a first pass? Perhaps we can trust >> >> >>> >>> >> users >> >> >>> >>> >> for >> >> >>> >>> >> now? >> >> >>> >>> >> >> >> >>> >>> >> domainMatches :: W.Ascii -> W.Ascii -> Maybe W.Ascii >> >> >>> >>> >> >> >> >>> >>> >> Does the first domain match against the second domain? >> >> >>> >>> >> If so, return the prefix of the first that isn't in the >> second >> >> >>> >>> >> >> >> >>> >>> >> pathMatches :: W.Ascii -> W.Ascii -> Bool >> >> >>> >>> >> >> >> >>> >>> >> Do the paths match? >> >> >>> >>> >> >> >> >>> >>> >> In order to implement domain matching, I have to have >> knowledge >> >> >>> >>> >> of >> >> >>> >>> >> the Public Suffix List so I know that >> sub1.sub2.pvt.k12.wy.us >> >> >>> >>> >> can >> >> >>> >>> >> set >> >> >>> >>> >> a >> >> >>> >>> >> cookie for sub2.pvt.k12.wy.us but not for k12.wy.us(because >> >> >>> >>> >> pvt.k12.wy.us >> >> >>> >>> >> is a "suffix"). There are a variety of ways to implement >> this. >> >> >>> >>> >> >> >> >>> >>> >> As far as I can tell, Chrome does it by using a script >> (which a >> >> >>> >>> >> human >> >> >>> >>> >> periodically runs) which parses the list at creates a .cc >> file >> >> >>> >>> >> that is >> >> >>> >>> >> included in the build. >> >> >>> >>> >> >> >> >>> >>> >> I might be wrong about the execution of the script; it >> might be >> >> >>> >>> >> a >> >> >>> >>> >> build >> >> >>> >>> >> step. If it is a build step, however, it is suspicious that >> a >> >> >>> >>> >> build >> >> >>> >>> >> target >> >> >>> >>> >> would try to download a file... >> >> >>> >>> >> >> >> >>> >>> >> Any more elegant ideas? >> >> >>> >>> >> >> >> >>> >>> >> Feedback on any/all of the above would be very helpful >> before I >> >> >>> >>> >> go >> >> >>> >>> >> off >> >> >>> >>> >> into the weeds on this project. >> >> >>> >>> >> >> >> >>> >>> >> Thanks, >> >> >>> >>> >> Myles C. Maxfield >> >> >>> >>> >> >> >> >>> >>> >> On Sat, Jan 28, 2012 at 8:17 PM, Michael Snoyman >> >> >>> >>> >> <[email protected]> >> >> >>> >>> >> wrote: >> >> >>> >>> >>> >> >> >>> >>> >>> Thanks, looks great! I've merged it into the Github tree. >> >> >>> >>> >>> >> >> >>> >>> >>> On Sat, Jan 28, 2012 at 8:36 PM, Myles C. Maxfield >> >> >>> >>> >>> <[email protected]> wrote: >> >> >>> >>> >>> > Ah, yes, you're completely right. I completely agree that >> >> >>> >>> >>> > moving >> >> >>> >>> >>> > the >> >> >>> >>> >>> > function into the Maybe monad increases readability. This >> >> >>> >>> >>> > kind >> >> >>> >>> >>> > of >> >> >>> >>> >>> > function >> >> >>> >>> >>> > is what the Maybe monad was designed for. >> >> >>> >>> >>> > >> >> >>> >>> >>> > Here is a revised patch. >> >> >>> >>> >>> > >> >> >>> >>> >>> > >> >> >>> >>> >>> > On Sat, Jan 28, 2012 at 8:28 AM, Michael Snoyman >> >> >>> >>> >>> > <[email protected]> >> >> >>> >>> >>> > wrote: >> >> >>> >>> >>> >> >> >> >>> >>> >>> >> On Sat, Jan 28, 2012 at 1:20 AM, Myles C. Maxfield >> >> >>> >>> >>> >> <[email protected]> wrote: >> >> >>> >>> >>> >> > the fromJust should never fail, beceause of the guard >> >> >>> >>> >>> >> > statement: >> >> >>> >>> >>> >> > >> >> >>> >>> >>> >> > | 300 <= code && code < 400 && isJust l'' && >> isJust >> >> >>> >>> >>> >> > l' = >> >> >>> >>> >>> >> > Just $ >> >> >>> >>> >>> >> > req >> >> >>> >>> >>> >> > >> >> >>> >>> >>> >> > Because of the order of the && operators, it will only >> >> >>> >>> >>> >> > evaluate >> >> >>> >>> >>> >> > fromJust >> >> >>> >>> >>> >> > after it makes sure that the argument isJust. That >> >> >>> >>> >>> >> > function >> >> >>> >>> >>> >> > in >> >> >>> >>> >>> >> > particular >> >> >>> >>> >>> >> > shouldn't throw any exceptions - it should only return >> >> >>> >>> >>> >> > Nothing. >> >> >>> >>> >>> >> > >> >> >>> >>> >>> >> > Knowing that, I don't quite think I understand what >> your >> >> >>> >>> >>> >> > concern >> >> >>> >>> >>> >> > is. >> >> >>> >>> >>> >> > Can >> >> >>> >>> >>> >> > you >> >> >>> >>> >>> >> > elaborate? >> >> >>> >>> >>> >> >> >> >>> >>> >>> >> You're right, but I had to squint really hard to prove >> to >> >> >>> >>> >>> >> myself >> >> >>> >>> >>> >> that >> >> >>> >>> >>> >> you're right. That's the kind of code that could easily >> be >> >> >>> >>> >>> >> broken >> >> >>> >>> >>> >> in >> >> >>> >>> >>> >> future updates by an unwitting maintainer (e.g., me). To >> >> >>> >>> >>> >> protect >> >> >>> >>> >>> >> the >> >> >>> >>> >>> >> world from me, I'd prefer if the code didn't have the >> >> >>> >>> >>> >> fromJust. >> >> >>> >>> >>> >> This >> >> >>> >>> >>> >> might be a good place to leverage the Monad instance of >> >> >>> >>> >>> >> Maybe. >> >> >>> >>> >>> >> >> >> >>> >>> >>> >> Michael >> >> >>> >>> >>> > >> >> >>> >>> >>> > >> >> >>> >>> >> >> >> >>> >>> >> >> >> >>> >>> > >> >> >>> > >> >> >>> > >> >> >> >> >> >> >> >> > >> > >> > >> > > > _______________________________________________ > Haskell-Cafe mailing list > [email protected] > http://www.haskell.org/mailman/listinfo/haskell-cafe > >
_______________________________________________ Haskell-Cafe mailing list [email protected] http://www.haskell.org/mailman/listinfo/haskell-cafe
