On Wed, Jan 5, 2011 at 4:40 PM, rainerh <[email protected]> wrote:
> Hello everybody,
>
> I'm trying to use clojure to parse web sites. Unfortunately there is
> some problem with following code (used library is Apache Commons HTTP
> Client):
>
> (defn request [url]
> (let [client (new org.apache.http.impl.client.DefaultHttpClient)]
> (with-open [rdr (reader
> (.getContent
> (.getEntity
> (.execute client (new
> org.apache.http.client.methods.HttpGet url)))))
> ]
Opens the stream.
> (let [content (line-seq rdr)]
Creates a lazy seq that will try to read from the stream to realize elements.
> (.shutdown (.getConnectionManager client))
Closes stream (doesn't with-open handle this automatically?)
> content
Returns unrealized lazy-seq.
Client tries to use lazy seq, lazy seq tries to realize item, stream
is closed => StreamClosedException.
Either process the seq inside the with-open (pass request a closure
that it calls with the seq) or realize it (loading the whole file in
memory) by changing the code to
(defn request [url]
(let [client (new org.apache.http.impl.client.DefaultHttpClient)]
(with-open [rdr (reader
(.getContent
(.getEntity
(.execute client
(new org.apache.http.client.methods.HttpGet url)))))]
(let [content (doall (line-seq rdr))]
(.shutdown (.getConnectionManager client))
content))))
The "doall" will force the lazy seq to realize all of its elements
while the connection is still open.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
