There aren't any problems with with-open/doseq/line-seq. The issue is
with with-open/line-seq. For example, it's instinctive (at least for
me anyway) to want to write a function like this:
(defn get-records [file-name]
(with-open [r (reader file-name)]
(line-seq r)))
Of course, the problem here is that the lazy sequence "escapes" the
with-open scope and you get an exception when you start consuming it.
And given a sequence there's no mechanism to say "I'm done with you,
clean up any resources you may be using".
That's the crux of it for me anyway. It seems like the simplest
approach is to just adjust my instincts and always make sure the
sequence is fully consumed within the with-open.
Cheers,
Dave
On Wed, Nov 7, 2012 at 10:56 AM, Jim - FooBar(); <jimpil1...@gmail.com> wrote:
> I know I'm coming a bit late in this thread but i did not have the chance to
> reply earlier...
>
> Can somebody elaborate briefly what is the problem with the combination of
> with-open/doseq/line-seq? In a project of mine I'm dealing with files larger
> than 200MB (which of course will not even open on a regular text-editor) and
> I've been happily using with-open/doseq/line-seq with great success...As
> Stuart pointed out as long as the entire processing happens within a
> 'with-open' all is good and this is the approach I'm always following with
> no memory issues...What am I missing? What exactly do you mean by "close the
> file"? Do you mean "close the clojure.java.io/reader"? which happens
> implicitly when a with-open exits? I've also not understood what the
> original problem is...Has anyone had problems with large files?
>
> sorry for the inconvenience...I read the entire thread and I'm still not
> understanding!
>
> Jim
>
>
>
> On 29/10/12 03:21, Dave Ray wrote:
>>
>> Stuart,
>>
>> Thanks for the link. It confirms the suspicions I had about a general
>> solution for this issue. For the particular code I'm working with,
>> I'll try pushing with-open further up and see if that gives me some of
>> the flexibility I'm looking for.
>>
>> Cheers,
>>
>> Dave
>>
>> On Sun, Oct 28, 2012 at 2:21 PM, Stuart Sierra
>> <the.stuart.sie...@gmail.com> wrote:
>>>
>>> On Friday, October 26, 2012 11:11:48 PM UTC-4, daveray wrote:
>>>>
>>>> I guess I looking for a magical line-seq that closes the file correctly
>>>> even if you consume part of the sequence, is resilient to exceptions,
>>>> etc, etc. I realize that it might be impossible, so I asked. :)
>>>
>>>
>>> It's been discussed extensively in the past, but no one has come up with
>>> a
>>> solution that adequately solves the general problem. See
>>> http://dev.clojure.org/display/design/Resource+Scopes for examples of
>>> some
>>> attempts.
>>>
>>> The best approach I've found is to manage resources like files farther up
>>> the stack. Instead of having a 'with-open' block that returns a sequence,
>>> put the 'with-open' block at a higher level, so that it encompasses the
>>> entire scope in which the file will be used.
>>>
>>> -S
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your
>>> first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
