Re: clojure.data.csv/write-csv isn't thread safe (should it be?)

[Matching Socks]

You give credit where none is due!  The awe is genuine.  Using higher-order 
functions to avoid writing a loop is old hat.  But you went further, and 
ran *100 threads* without writing a loop.  I was so amazed and impressed 
that an integer overflowed in my brain stem and vectored me off toward 
Huffman-encoding docstrings.  

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure.data.csv/write-csv isn't thread safe (should it be?)

[matt . t . grimm]

Well, snark and sarcasm aside, you've made me realize I was way too focused 
on making write-csv do what I wanted rather than come up with a better 
solution. So thanks for that, but do try to keep up the impression that 
this is a friendly, welcoming community.

On Thursday, May 2, 2019 at 6:07:53 PM UTC-6, Matching Socks wrote:
>
> Wow, that user.clj is impressive.  This is a top-notch application of 
> clojure core async!  Nonetheless, if one is going to the trouble of making 
> 100 threads, it would be clearer and farther-sighted (and not much more 
> work) to toss the results into a channel that a single thread drained into 
> an output file.  In that sense, a "fix" of the csv writer would not even be 
> in our best interest.  Moreover, isn't the default expectation that 
> multi-threaded side effects will inevitably produce a mud puddle?  If "this 
> is not thread-safe, of course" were added to one side-effecting function's 
> docstring, would the interests of fairness demand such a flag on many more 
> functions?!  Since most Clojure programs are not AOT-compiled, the extra 
> bloat of docstrings would make program startup take longer.  The delay 
> could be mitigated somewhat by Huffman-encoding the docstrings.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure.data.csv/write-csv isn't thread safe (should it be?)

[Michael Gardner]

Note that clojure.core/println is also not "thread-safe" in that sense: two 
threads doing `(println "foo" "bar")` may produce interleaved output.

> On May 2, 2019, at 05:59, matt.t.gr...@gmail.com wrote:
> 
> The write-csv function in clojure.data.csv isn't thread safe because even 
> though it uses the synchronized .write method on BufferedReader, it does so 
> at the cell level, not the row/line level. Is this expected or a bug? Is 
> there a reason it shouldn't be made thread safe?
> 
> Attached is a simple test showing the result of calling write-csv with a 
> single thread vs. 100 threads. With a single thread, the output is correct, 
> and with 100 threads the result is a jumbled mess. Conversely, if I use 
> clojure-csv and manually write at the row level using the same harness, it 
> works as expected with multiple threads.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure.data.csv/write-csv isn't thread safe (should it be?)

[Matching Socks]

Wow, that user.clj is impressive.  This is a top-notch application of 
clojure core async!  Nonetheless, if one is going to the trouble of making 
100 threads, it would be clearer and farther-sighted (and not much more 
work) to toss the results into a channel that a single thread drained into 
an output file.  In that sense, a "fix" of the csv writer would not even be 
in our best interest.  Moreover, isn't the default expectation that 
multi-threaded side effects will inevitably produce a mud puddle?  If "this 
is not thread-safe, of course" were added to one side-effecting function's 
docstring, would the interests of fairness demand such a flag on many more 
functions?!  Since most Clojure programs are not AOT-compiled, the extra 
bloat of docstrings would make program startup take longer.  The delay 
could be mitigated somewhat by Huffman-encoding the docstrings.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure.data.csv/write-csv isn't thread safe (should it be?)

[Alex Miller]

I think the vast majority of the time, people are writing a csv file in a 
single thread. Issue/patch welcome though...

On Thursday, May 2, 2019 at 7:59:05 AM UTC-5, matt@gmail.com wrote:
>
> The write-csv function in clojure.data.csv isn't thread safe because even 
> though it uses the synchronized .write method on BufferedReader, it does so 
> at 
> the cell level 
> ,
>  
> not the row/line level. Is this expected or a bug? Is there a reason it 
> shouldn't be made thread safe?
>
> Attached is a simple test showing the result of calling write-csv with a 
> single thread vs. 100 threads. With a single thread, the output is correct, 
> and with 100 threads the result is a jumbled mess. Conversely, if I use 
> clojure-csv  and manually 
> write at the row level using the same harness, it works as expected with 
> multiple threads.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

clojure.data.csv/write-csv isn't thread safe (should it be?)

[matt . t . grimm]

The write-csv function in clojure.data.csv isn't thread safe because even 
though it uses the synchronized .write method on BufferedReader, it does so at 
the cell level 
,
 
not the row/line level. Is this expected or a bug? Is there a reason it 
shouldn't be made thread safe?

Attached is a simple test showing the result of calling write-csv with a 
single thread vs. 100 threads. With a single thread, the output is correct, 
and with 100 threads the result is a jumbled mess. Conversely, if I use 
clojure-csv  and manually 
write at the row level using the same harness, it works as expected with 
multiple threads.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


user.clj
Description: Binary data