Re: [ANN] data.avl 0.0.14 – BUGFIX to splits, faster map/set reductions
All credit to Francis for the reduce patch! The perf win is really great. Re: IReduce – thanks, indeed, I forgot about the special-casing in reduce. I guess I'll tweak that for the next release. Incidentally, I notice I left out the final result in the after-patch 1.5.1 benchmark run in the previous email – repeated below in full. Cheers, Michał After patch, Clojure 1.5.1: user> (let [m1 (apply sorted-map (range 1)) m2 (apply avl/sorted-map (range 1))] (assert (= (reduce (fn [out kv] (+ out (key kv) (val kv))) 0 m1) (reduce (fn [out kv] (+ out (key kv) (val kv))) 0 m2))) (c/quick-bench (reduce (fn [out kv] (+ out (key kv) (val kv))) 0 m1)) (c/quick-bench (reduce (fn [out kv] (+ out (key kv) (val kv))) 0 m2))) Evaluation count : 990 in 6 samples of 165 calls. Execution time mean : 599.205620 µs Execution time std-deviation : 30.389791 µs Execution time lower quantile : 579.132582 µs ( 2.5%) Execution time upper quantile : 650.985417 µs (97.5%) Overhead used : 1.320107 ns Found 1 outliers in 6 samples (16.6667 %) low-severe 1 (16.6667 %) Variance from outliers : 13.8889 % Variance is moderately inflated by outliers Evaluation count : 3198 in 6 samples of 533 calls. Execution time mean : 191.906227 µs Execution time std-deviation : 3.447964 µs Execution time lower quantile : 188.550574 µs ( 2.5%) Execution time upper quantile : 197.006475 µs (97.5%) Overhead used : 1.320107 ns nil On 23 August 2016 at 02:59, Alex Miller wrote: > Nice work! Esp on the reduce stuff - great to see that. Any reason you > didn't go the IReduce path in Clojure too instead of CollReduce? Generally, > I'd say that's preferred when you control the data structure. > > > On Monday, August 22, 2016 at 7:43:51 PM UTC-5, Michał Marczyk wrote: >> >> Hi, >> >> I am pleased to announce the 0.0.14 release of data.avl, a Clojure >> Contrib library providing highly performant drop-in replacements for >> Clojure(Script)'s built-in sorted maps and sets that support O(log n) >> nth, rank-of, first-class submaps/subsets (like subseq, but preserving >> collection type; fully independent from the original for GC purposes) >> and splits by key and index. >> >> [org.clojure/data.avl "0.0.14"] >> >> >> org.clojure >> data.avl >> 0.0.14 >> >> >> org.clojure:data.avl:0.0.14 >> >> Changes in this release: >> >> 1. http://dev.clojure.org/jira/browse/DAVL-8 >> >> Fixed a bug in split-key that caused incorrect rank and size >> information to be stored in some collections resulting from splits >> using a key absent from the input collection. See the ticket for a >> more detailed post mortem and links to fixing commits. >> >> Many thanks to Darrick Wiebe for the report and test.check >> properties that exposed broken split-key return values! >> >> 2. http://dev.clojure.org/jira/browse/DAVL-7 >> >> data.avl maps and sets now implement CollReduce (in Clojure) / >> IReduce (in ClojureScript), leading to significantly improved >> reduce performance on data.avl inputs. See end of this email for >> some Criterium benchmarks. >> >> Many thanks to Francis Avila for providing the patch! >> >> 3. Seqs over data.avl collections can now be = to j.u.Lists. >> >> 4. data.avl collections now have their own print-dup implementations >> that correctly round-trip in Clojure. >> >> 5. Public Vars in the clojure.data.avl namespace now carry :added >> metadata. >> >> Cheers, >> Michał >> >> >> Reduce benchmarks: >> == >> >> Before patch: >> - >> >> user> (let [m1 (apply sorted-map (range 1)) >> m2 (apply avl/sorted-map (range 1))] >> (assert (= (reduce (fn [out kv] (+ out (key kv) (val kv))) >> 0 >> m1) >>(reduce (fn [out kv] (+ out (key kv) (val kv))) >> 0 >> m2))) >> (c/quick-bench >> (reduce (fn [out kv] (+ out (key kv) (val kv))) >> 0 >> m1)) >> (c/quick-bench >> (reduce (fn [out kv] (+ out (key kv) (val kv))) >> 0 >> m2))) >> Evaluation count : 120 in 6 samples of 20 calls. >> Execution time mean : 663.862017 µs >> Execution time std-deviation : 9.483652 µs >>Execution time lower quantile : 654.277800 µs ( 2.5%) >>Execution time upper quantile : 677.885431 µs (97.5%) >>Overhead used : 21.423458 ns >> Evaluation count : 468 in 6 samples of 78 calls. >> Execution time mean : 1.295972 ms >> Execution time std-deviation : 74.233890 µs >>Execution time lower quantile : 1.255853 ms (
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
In the data representing fragments of the user's code returned with these
macro errors, does it include metadata with :line and :column keys in it?
Perhaps that would be one way to give errors localized to particular places
in the user's code.
It isn't always available, e.g. keyword cannot have metadata, but symbols,
lists, vectors, and maps can.
Andy
On Mon, Aug 22, 2016 at 6:27 PM, Alex Miller wrote:
> Sorry, I missed this one in the thread somehow. This happens to be a case
> where you have *both* defn and destructuring specs in play, so it has even
> greater potential for confusion in generic errors.
>
>
> On Monday, August 22, 2016 at 11:23:33 AM UTC-5, Leon Grapenthin wrote:
>>
>> I welcome the strict checking over backwards compatibility for broken
>> syntax. E. g. allowing things like symbols in the ns decl would require
>> supporting that as a feature in future updates, analyzer code, other hosts
>> etc. The Clojure devs should not have to worry things with so little use.
>>
>> Still the error messages are simply far from good enough and that is what
>> appears to me as the main problem OP has. If the compiler had pointed him
>> to two fixes he could have done in a minute then he had not complained and
>> instead felt more happy with his new compiler.
>>
>> Here is my story when I loaded a large codebase today with 1.9-alpha11:
>>
>
> I'm going to reformat this a little (and formatting is certainly a thing
> that could change - since there are user values in here it's easy for the
> spacing to not be the best)...
>
> Call to clojure.core/defn did not conform to spec:
>
> You mentioned you knew exactly the location and the above gives you the
> form that is failing to match, so you've narrowed it down to a particular
> form in a particular file.
>
>
> In: [1 0]
> val: ({:keys [money/major money/minor money/currency], :or #:money{major
> 0, minor 0, currency :EUR}})
>
> here's the actual value that is failing - so this does include the part
> that's wrong (:or map with non-symbol keys)
>
>
> fails spec: :clojure.core.specs/arg-list
>
> here we name another spec - you can use this to (doc
> :clojure.core.specs/arg-list) to get more info (and so on). There will be
> probably be some kind of a way to do this in a "deep" way at some point.
>
>
> at: [:args :bs :arity-1 :args]
>
> the path through the specs - can be combined with the spec name above and
> the spec for defn to give you a lot more info - this is the kind of thing
> that a sideband tool in an IDE could do amazing things with.
>
>
> predicate: (cat :args (* :clojure.core.specs/binding-form) :varargs (?
> (cat :amp #{(quote &)} :form :clojure.core.specs/binding-form))),
>
> the predicate that is actually failing in the spec, probably not
> particularly helpful given the complexity (and recursiveness) of the
> destructuring specs
>
>
> Extra input
>
> this is the part of cat that I think could be made more explicit - could
> be saying here that the value it had (above) was expected to match the next
> part of the cat (binding-form). So that could say the equivalent of
> "Expected binding-form but had non-matching value ..." and could even find
> what parts of that value matched and maybe which didn't (the :or keys) such
> that you'd have a more precise description. There is some more stuff Rich
> and I have worked on around "hybrid maps" which is the case here with map
> destructuring - it's particularly challenging to get a good error out of
> that at the moment, but there's more that can be done.
>
>
> In: [1 0]
> val: {:keys [money/major money/minor money/currency], :or #:money{major 0,
> minor 0, currency :EUR}}
> fails spec: :clojure.core.specs/arg-list
> at: [:args :bs :arity-n :bodies :args]
> predicate: vector?
>
> This whole block is another (non-applicable) alternative that spec tried
> out (the multi-arity case). We know that's not what we were trying to do,
> but it's reported anyways. This is where fan-out kicks in and you can get a
> lot of noise that's not applicable. I think there's probably enough
> information to either be better about saying which case we thought you were
> matching or to hide some of them, etc.
>
>
> :clojure.spec/args (format-money [{:keys
>[money/major money/minor money/currency], :or #:money{major 0,
>minor 0, currency :EUR}}] (str major "," (if (zero? minor) "-"
>minor) " €"))
>
> And this is just reporting the actual passed args to the macro - I think
> this is confusing because it doesn't look like the call (because it's
> missing the macro name). As I said prior, I think this particular part of
> macro error reporting can do better as it knows more about what's going on
> - this should really be tied back up to the macro name at the top.
>
>
>
>
>>
>> I know where the problem is immediately because I looked at above error
>> and quickly jumped to the code that didn't work. Then I guessed it right
>> because I know what has been changed from Alex Release notes and because I
>> ha
Re: Generate a million file from a template
Abhinav Gogna writes: > (defn run-in-parallel > "run-in-parallel runs 500 different threads. > you can give each thread number of files you want to generate > Eg: run-in-parallel 100 will generate 500*100 = 5 files" > [y] > (dotimes [_ 500] > (future > (.start (Thread. (run-me y)) `future` alread spawns a thread (or at least dispatches into a thread pool) so you're actually spawning two threads here. You can either get rid of the `future` or just replace it with `(future (run-me y))`. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
ref strange behavior
Hello,
I had a problem when I run the following code:
(defn get-task [tasks]
(dosync
(let [task (first @tasks)]
(alter tasks rest)
task)))
(defn worker [& {:keys [tasks]}]
(agent {:tasks tasks}))
(defn worker-loop [{:keys [tasks] :as state}]
(loop [last-task nil]
(if-let [task (get-task tasks)]
(recur task)
(locking :out (println "Last task: " last-task
state)
(defn create-workers [count & options]
(->> (range 0 count)
(map (fn [_] (apply worker options)))
(into [])))
(defn start-workers [workers]
(doseq [worker workers] (send-off worker worker-loop)))
(def tasks (ref (range 1 1000)))
(def workers (create-workers 100 :tasks tasks))
(start-workers workers)
(apply await workers)
Description: I have several agents (100 in my case). Each agent running in
a separate thread. All agents share the one ref with the collection of
tasks (range of longs in my case). Each agent get tasks from the collection
(in transaction) one by one until the collection becomes empty and then
prints the last task which it handle. However, when I run this code it
looks like the collection of tasks suddenly becomes empty and workers
handle only portion of all tasks (average 25-40% of all number).
This code behave as I expected, when I create only one agent or use
explicit locking in get-task function:
(defn get-task [tasks]
(locking :lock
(dosync
(let [task (first @tasks)]
(alter tasks rest)
task
I run this code on the Clojure 1.8.0
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
Can anyone tell me, what am I doing wrong, or it really looks like a bug in
the clojure STM?
I already asked this question on stackoverflow.com (
http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
but so far nobody has been able to help me.
P.S. Sorry for my english skill.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: ref strange behavior
Taking a quick look at this, I *think* the problem crops up in your tasks
being a lazy sequence.
Modify this
(def tasks (ref (range 1 1000)))
to this
(def tasks (ref (doall (range 1 1000
Running that with your large endpoint was taking a lot of time, so I'd
suggest running it with 1e5 as that will finish much quicker.
On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>
> Hello,
>
> I had a problem when I run the following code:
>
> (defn get-task [tasks]
> (dosync
> (let [task (first @tasks)]
> (alter tasks rest)
> task)))
>
> (defn worker [& {:keys [tasks]}]
> (agent {:tasks tasks}))
>
> (defn worker-loop [{:keys [tasks] :as state}]
> (loop [last-task nil]
> (if-let [task (get-task tasks)]
> (recur task)
> (locking :out (println "Last task: " last-task
> state)
>
> (defn create-workers [count & options]
> (->> (range 0 count)
>(map (fn [_] (apply worker options)))
>(into [])))
>
> (defn start-workers [workers]
> (doseq [worker workers] (send-off worker worker-loop)))
>
> (def tasks (ref (range 1 1000)))
>
> (def workers (create-workers 100 :tasks tasks))
>
> (start-workers workers)
> (apply await workers)
>
> Description: I have several agents (100 in my case). Each agent running in
> a separate thread. All agents share the one ref with the collection of
> tasks (range of longs in my case). Each agent get tasks from the collection
> (in transaction) one by one until the collection becomes empty and then
> prints the last task which it handle. However, when I run this code it
> looks like the collection of tasks suddenly becomes empty and workers
> handle only portion of all tasks (average 25-40% of all number).
>
> This code behave as I expected, when I create only one agent or use
> explicit locking in get-task function:
>
> (defn get-task [tasks]
> (locking :lock
> (dosync
> (let [task (first @tasks)]
> (alter tasks rest)
> task
>
> I run this code on the Clojure 1.8.0
> java version "1.8.0_91"
> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>
> Can anyone tell me, what am I doing wrong, or it really looks like a bug
> in the clojure STM?
> I already asked this question on stackoverflow.com (
> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
> but so far nobody has been able to help me.
>
> P.S. Sorry for my english skill.
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: ref strange behavior
I haven't run your code yet, but it's bad form to use Clojure's reference
types inside other reference types. They should store persistent data
structures to really make any sense in a concurrent context.
On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>
> Hello,
>
> I had a problem when I run the following code:
>
> (defn get-task [tasks]
> (dosync
> (let [task (first @tasks)]
> (alter tasks rest)
> task)))
>
> (defn worker [& {:keys [tasks]}]
> (agent {:tasks tasks}))
>
> (defn worker-loop [{:keys [tasks] :as state}]
> (loop [last-task nil]
> (if-let [task (get-task tasks)]
> (recur task)
> (locking :out (println "Last task: " last-task
> state)
>
> (defn create-workers [count & options]
> (->> (range 0 count)
>(map (fn [_] (apply worker options)))
>(into [])))
>
> (defn start-workers [workers]
> (doseq [worker workers] (send-off worker worker-loop)))
>
> (def tasks (ref (range 1 1000)))
>
> (def workers (create-workers 100 :tasks tasks))
>
> (start-workers workers)
> (apply await workers)
>
> Description: I have several agents (100 in my case). Each agent running in
> a separate thread. All agents share the one ref with the collection of
> tasks (range of longs in my case). Each agent get tasks from the collection
> (in transaction) one by one until the collection becomes empty and then
> prints the last task which it handle. However, when I run this code it
> looks like the collection of tasks suddenly becomes empty and workers
> handle only portion of all tasks (average 25-40% of all number).
>
> This code behave as I expected, when I create only one agent or use
> explicit locking in get-task function:
>
> (defn get-task [tasks]
> (locking :lock
> (dosync
> (let [task (first @tasks)]
> (alter tasks rest)
> task
>
> I run this code on the Clojure 1.8.0
> java version "1.8.0_91"
> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>
> Can anyone tell me, what am I doing wrong, or it really looks like a bug
> in the clojure STM?
> I already asked this question on stackoverflow.com (
> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
> but so far nobody has been able to help me.
>
> P.S. Sorry for my english skill.
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
On Tuesday, August 23, 2016 at 2:58:42 AM UTC-5, Andy Fingerhut wrote:
>
> In the data representing fragments of the user's code returned with these
> macro errors, does it include metadata with :line and :column keys in it?
>
No, although the exception itself from macroexpansion will have the
file/line info for the beginning of the failing form and the "at" part of
the error gives the path in the data down to the value in question. So
there is a lot of info available.
>
> Perhaps that would be one way to give errors localized to particular
> places in the user's code.
>
> It isn't always available, e.g. keyword cannot have metadata, but symbols,
> lists, vectors, and maps can.
>
> Andy
>
>
> On Mon, Aug 22, 2016 at 6:27 PM, Alex Miller wrote:
>
>> Sorry, I missed this one in the thread somehow. This happens to be a case
>> where you have *both* defn and destructuring specs in play, so it has even
>> greater potential for confusion in generic errors.
>>
>>
>> On Monday, August 22, 2016 at 11:23:33 AM UTC-5, Leon Grapenthin wrote:
>>>
>>> I welcome the strict checking over backwards compatibility for broken
>>> syntax. E. g. allowing things like symbols in the ns decl would require
>>> supporting that as a feature in future updates, analyzer code, other hosts
>>> etc. The Clojure devs should not have to worry things with so little use.
>>>
>>> Still the error messages are simply far from good enough and that is
>>> what appears to me as the main problem OP has. If the compiler had pointed
>>> him to two fixes he could have done in a minute then he had not complained
>>> and instead felt more happy with his new compiler.
>>>
>>> Here is my story when I loaded a large codebase today with 1.9-alpha11:
>>>
>>
>> I'm going to reformat this a little (and formatting is certainly a thing
>> that could change - since there are user values in here it's easy for the
>> spacing to not be the best)...
>>
>> Call to clojure.core/defn did not conform to spec:
>>
>> You mentioned you knew exactly the location and the above gives you the
>> form that is failing to match, so you've narrowed it down to a particular
>> form in a particular file.
>>
>>
>> In: [1 0]
>> val: ({:keys [money/major money/minor money/currency], :or #:money{major
>> 0, minor 0, currency :EUR}})
>>
>> here's the actual value that is failing - so this does include the part
>> that's wrong (:or map with non-symbol keys)
>>
>>
>> fails spec: :clojure.core.specs/arg-list
>>
>> here we name another spec - you can use this to (doc
>> :clojure.core.specs/arg-list) to get more info (and so on). There will be
>> probably be some kind of a way to do this in a "deep" way at some point.
>>
>>
>> at: [:args :bs :arity-1 :args]
>>
>> the path through the specs - can be combined with the spec name above and
>> the spec for defn to give you a lot more info - this is the kind of thing
>> that a sideband tool in an IDE could do amazing things with.
>>
>>
>> predicate: (cat :args (* :clojure.core.specs/binding-form) :varargs (?
>> (cat :amp #{(quote &)} :form :clojure.core.specs/binding-form))),
>>
>> the predicate that is actually failing in the spec, probably not
>> particularly helpful given the complexity (and recursiveness) of the
>> destructuring specs
>>
>>
>> Extra input
>>
>> this is the part of cat that I think could be made more explicit - could
>> be saying here that the value it had (above) was expected to match the next
>> part of the cat (binding-form). So that could say the equivalent of
>> "Expected binding-form but had non-matching value ..." and could even find
>> what parts of that value matched and maybe which didn't (the :or keys) such
>> that you'd have a more precise description. There is some more stuff Rich
>> and I have worked on around "hybrid maps" which is the case here with map
>> destructuring - it's particularly challenging to get a good error out of
>> that at the moment, but there's more that can be done.
>>
>>
>> In: [1 0]
>> val: {:keys [money/major money/minor money/currency], :or #:money{major
>> 0, minor 0, currency :EUR}}
>> fails spec: :clojure.core.specs/arg-list
>> at: [:args :bs :arity-n :bodies :args]
>> predicate: vector?
>>
>> This whole block is another (non-applicable) alternative that spec tried
>> out (the multi-arity case). We know that's not what we were trying to do,
>> but it's reported anyways. This is where fan-out kicks in and you can get a
>> lot of noise that's not applicable. I think there's probably enough
>> information to either be better about saying which case we thought you were
>> matching or to hide some of them, etc.
>>
>>
>> :clojure.spec/args (format-money [{:keys
>>[money/major money/minor money/currency], :or #:money{major 0,
>>minor 0, currency :EUR}}] (str major "," (if (zero? minor) "-"
>>minor) " €"))
>>
>> And this is just reporting the actual passed args to the macro - I think
>> this is confusing because it doesn't lo
Re: ref strange behavior
This really seems like a good use case for Java's Executors:
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Executors.html#newFixedThreadPool(int)
You can specify the number of worker threads, then enqueue runnable jobs.
But wait! There's more! Clojure functions implement Runnable, so you can
just hand a zero argument function to the service once you create it:
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#submit(java.lang.Runnable)
On Tue, Aug 23, 2016 at 7:09 AM, wrote:
> I haven't run your code yet, but it's bad form to use Clojure's reference
> types inside other reference types. They should store persistent data
> structures to really make any sense in a concurrent context.
>
> On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>
>> Hello,
>>
>> I had a problem when I run the following code:
>>
>> (defn get-task [tasks]
>> (dosync
>> (let [task (first @tasks)]
>> (alter tasks rest)
>> task)))
>>
>> (defn worker [& {:keys [tasks]}]
>> (agent {:tasks tasks}))
>>
>> (defn worker-loop [{:keys [tasks] :as state}]
>> (loop [last-task nil]
>> (if-let [task (get-task tasks)]
>> (recur task)
>> (locking :out (println "Last task: " last-task
>> state)
>>
>> (defn create-workers [count & options]
>> (->> (range 0 count)
>>(map (fn [_] (apply worker options)))
>>(into [])))
>>
>> (defn start-workers [workers]
>> (doseq [worker workers] (send-off worker worker-loop)))
>>
>> (def tasks (ref (range 1 1000)))
>>
>> (def workers (create-workers 100 :tasks tasks))
>>
>> (start-workers workers)
>> (apply await workers)
>>
>> Description: I have several agents (100 in my case). Each agent running
>> in a separate thread. All agents share the one ref with the collection of
>> tasks (range of longs in my case). Each agent get tasks from the collection
>> (in transaction) one by one until the collection becomes empty and then
>> prints the last task which it handle. However, when I run this code it
>> looks like the collection of tasks suddenly becomes empty and workers
>> handle only portion of all tasks (average 25-40% of all number).
>>
>> This code behave as I expected, when I create only one agent or use
>> explicit locking in get-task function:
>>
>> (defn get-task [tasks]
>> (locking :lock
>> (dosync
>> (let [task (first @tasks)]
>> (alter tasks rest)
>> task
>>
>> I run this code on the Clojure 1.8.0
>> java version "1.8.0_91"
>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>>
>> Can anyone tell me, what am I doing wrong, or it really looks like a bug
>> in the clojure STM?
>> I already asked this question on stackoverflow.com (
>> http://stackoverflow.com/questions/39054911/strange-behavio
>> r-of-clojure-ref), but so far nobody has been able to help me.
>>
>> P.S. Sorry for my english skill.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: ref strange behavior
Yes, you are absolutely right. After i modify the code as you advised, it
worked correctly. Thank you very much!
Does it mean that is necessary to avoid the use of lazy data structures
within the STM?
вторник, 23 августа 2016 г., 19:57:58 UTC+7 пользователь hitesh написал:
>
> Taking a quick look at this, I *think* the problem crops up in your tasks
> being a lazy sequence.
>
> Modify this
>
> (def tasks (ref (range 1 1000)))
>
> to this
>
> (def tasks (ref (doall (range 1 1000
>
> Running that with your large endpoint was taking a lot of time, so I'd
> suggest running it with 1e5 as that will finish much quicker.
>
>
>
> On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>>
>> Hello,
>>
>> I had a problem when I run the following code:
>>
>> (defn get-task [tasks]
>> (dosync
>> (let [task (first @tasks)]
>> (alter tasks rest)
>> task)))
>>
>> (defn worker [& {:keys [tasks]}]
>> (agent {:tasks tasks}))
>>
>> (defn worker-loop [{:keys [tasks] :as state}]
>> (loop [last-task nil]
>> (if-let [task (get-task tasks)]
>> (recur task)
>> (locking :out (println "Last task: " last-task
>> state)
>>
>> (defn create-workers [count & options]
>> (->> (range 0 count)
>>(map (fn [_] (apply worker options)))
>>(into [])))
>>
>> (defn start-workers [workers]
>> (doseq [worker workers] (send-off worker worker-loop)))
>>
>> (def tasks (ref (range 1 1000)))
>>
>> (def workers (create-workers 100 :tasks tasks))
>>
>> (start-workers workers)
>> (apply await workers)
>>
>> Description: I have several agents (100 in my case). Each agent running
>> in a separate thread. All agents share the one ref with the collection of
>> tasks (range of longs in my case). Each agent get tasks from the collection
>> (in transaction) one by one until the collection becomes empty and then
>> prints the last task which it handle. However, when I run this code it
>> looks like the collection of tasks suddenly becomes empty and workers
>> handle only portion of all tasks (average 25-40% of all number).
>>
>> This code behave as I expected, when I create only one agent or use
>> explicit locking in get-task function:
>>
>> (defn get-task [tasks]
>> (locking :lock
>> (dosync
>> (let [task (first @tasks)]
>> (alter tasks rest)
>> task
>>
>> I run this code on the Clojure 1.8.0
>> java version "1.8.0_91"
>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>>
>> Can anyone tell me, what am I doing wrong, or it really looks like a bug
>> in the clojure STM?
>> I already asked this question on stackoverflow.com (
>> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
>>
>> but so far nobody has been able to help me.
>>
>> P.S. Sorry for my english skill.
>>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: ref strange behavior
Here is working example code demonstrating how you might do this without
agents:
(in-ns 'user)
(def tasks (ref (into clojure.lang.PersistentQueue/EMPTY (range 1 1000
(defn get-task
[tasks]
(dosync
(let [task (first @tasks)]
(alter tasks pop)
task)))
(defn worker-loop
[]
(loop [completed-tasks []]
(if-let [task (get-task tasks)]
(recur (conj completed-tasks task))
completed-tasks)))
(defn create-workers
[n & options]
(vec (repeatedly n (fn []
(future
(worker-loop))
(def workers
(create-workers 100))
(defn worker-test
[xs]
(= (set (mapcat deref workers))
(set xs)))
(worker-test (range 1 1000))
On Tuesday, August 23, 2016 at 9:09:31 AM UTC-4, adrian...@mail.yu.edu
wrote:
>
> I haven't run your code yet, but it's bad form to use Clojure's reference
> types inside other reference types. They should store persistent data
> structures to really make any sense in a concurrent context.
>
> On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>>
>> Hello,
>>
>> I had a problem when I run the following code:
>>
>> (defn get-task [tasks]
>> (dosync
>> (let [task (first @tasks)]
>> (alter tasks rest)
>> task)))
>>
>> (defn worker [& {:keys [tasks]}]
>> (agent {:tasks tasks}))
>>
>> (defn worker-loop [{:keys [tasks] :as state}]
>> (loop [last-task nil]
>> (if-let [task (get-task tasks)]
>> (recur task)
>> (locking :out (println "Last task: " last-task
>> state)
>>
>> (defn create-workers [count & options]
>> (->> (range 0 count)
>>(map (fn [_] (apply worker options)))
>>(into [])))
>>
>> (defn start-workers [workers]
>> (doseq [worker workers] (send-off worker worker-loop)))
>>
>> (def tasks (ref (range 1 1000)))
>>
>> (def workers (create-workers 100 :tasks tasks))
>>
>> (start-workers workers)
>> (apply await workers)
>>
>> Description: I have several agents (100 in my case). Each agent running
>> in a separate thread. All agents share the one ref with the collection of
>> tasks (range of longs in my case). Each agent get tasks from the collection
>> (in transaction) one by one until the collection becomes empty and then
>> prints the last task which it handle. However, when I run this code it
>> looks like the collection of tasks suddenly becomes empty and workers
>> handle only portion of all tasks (average 25-40% of all number).
>>
>> This code behave as I expected, when I create only one agent or use
>> explicit locking in get-task function:
>>
>> (defn get-task [tasks]
>> (locking :lock
>> (dosync
>> (let [task (first @tasks)]
>> (alter tasks rest)
>> task
>>
>> I run this code on the Clojure 1.8.0
>> java version "1.8.0_91"
>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>>
>> Can anyone tell me, what am I doing wrong, or it really looks like a bug
>> in the clojure STM?
>> I already asked this question on stackoverflow.com (
>> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
>>
>> but so far nobody has been able to help me.
>>
>> P.S. Sorry for my english skill.
>>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ANN] data.avl 0.0.14 – BUGFIX to splits, faster map/set reductions
I wasn't aware Clojure had an IReduce protocol! I thought CollReduce was just what IReduce is called in Clojure. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ref strange behavior
The lazy sequence works with the code I provided as well.
On Tuesday, August 23, 2016 at 9:29:08 AM UTC-4, Sergei Koledov wrote:
>
> Yes, you are absolutely right. After i modify the code as you advised, it
> worked correctly. Thank you very much!
> Does it mean that is necessary to avoid the use of lazy data structures
> within the STM?
>
> вторник, 23 августа 2016 г., 19:57:58 UTC+7 пользователь hitesh написал:
>>
>> Taking a quick look at this, I *think* the problem crops up in your tasks
>> being a lazy sequence.
>>
>> Modify this
>>
>> (def tasks (ref (range 1 1000)))
>>
>> to this
>>
>> (def tasks (ref (doall (range 1 1000
>>
>> Running that with your large endpoint was taking a lot of time, so I'd
>> suggest running it with 1e5 as that will finish much quicker.
>>
>>
>>
>> On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>>>
>>> Hello,
>>>
>>> I had a problem when I run the following code:
>>>
>>> (defn get-task [tasks]
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task)))
>>>
>>> (defn worker [& {:keys [tasks]}]
>>> (agent {:tasks tasks}))
>>>
>>> (defn worker-loop [{:keys [tasks] :as state}]
>>> (loop [last-task nil]
>>> (if-let [task (get-task tasks)]
>>> (recur task)
>>> (locking :out (println "Last task: " last-task
>>> state)
>>>
>>> (defn create-workers [count & options]
>>> (->> (range 0 count)
>>>(map (fn [_] (apply worker options)))
>>>(into [])))
>>>
>>> (defn start-workers [workers]
>>> (doseq [worker workers] (send-off worker worker-loop)))
>>>
>>> (def tasks (ref (range 1 1000)))
>>>
>>> (def workers (create-workers 100 :tasks tasks))
>>>
>>> (start-workers workers)
>>> (apply await workers)
>>>
>>> Description: I have several agents (100 in my case). Each agent running
>>> in a separate thread. All agents share the one ref with the collection of
>>> tasks (range of longs in my case). Each agent get tasks from the collection
>>> (in transaction) one by one until the collection becomes empty and then
>>> prints the last task which it handle. However, when I run this code it
>>> looks like the collection of tasks suddenly becomes empty and workers
>>> handle only portion of all tasks (average 25-40% of all number).
>>>
>>> This code behave as I expected, when I create only one agent or use
>>> explicit locking in get-task function:
>>>
>>> (defn get-task [tasks]
>>> (locking :lock
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task
>>>
>>> I run this code on the Clojure 1.8.0
>>> java version "1.8.0_91"
>>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
>>> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>>>
>>> Can anyone tell me, what am I doing wrong, or it really looks like a bug
>>> in the clojure STM?
>>> I already asked this question on stackoverflow.com (
>>> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
>>>
>>> but so far nobody has been able to help me.
>>>
>>> P.S. Sorry for my english skill.
>>>
>>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
> On Aug 22, 2016, at 7:50 PM, Alex Miller wrote: > You've complained in other channels about the "learning to read" error > messages part and I think you've taken it entirely the wrong way or maybe I > just disagree. There are benefits from reporting errors in a generic, > consistent way. […] Do there exist examples of what is desired for error messages in 1.9-final? Not promises, but a “this is what we’re shooting for”? What would you all like the specific error messages complained about in this thread to look like? Colin Fleming wrote: "The error message produced by the code I demoed at the conj last year would be: Unexpected symbol 'require' at while parsing namespace clauses. Expected :refer-clojure, :require, :use, :import, :load or :gen-class.” Is that the goal? I fear that the goal is that it should be my job to understand "(cat :attr-map (? map?) :clauses :clojure.core.specs/ns-clauses)”. For what little it’s worth, I consider that completely unacceptable. > - Getting the error data (specifically the explain-data output) to be both > sufficient and generically useful is the first priority. I think at this > point that's pretty close and unlikely to change significantly. My bias here is that I come from the learned-from-bitter-experience tradition that believes it’s very risky to (1) get the infrastructure right, and then (2) pop down the user-visible features on top of it. Very often, the infrastructure turns out to be a poor match for the actual needs of the features. But, since (1) is already done, the features - and consequently the users - suffer. Please understand I’m not being insulting when I say that everyone has weaknesses and blind spots, even undoubted geniuses. In Clojure, error messages and documentation (especially doc strings) have long been glaring weaknesses. So I am wishing to be helpful when I counsel *quickly* getting to worked examples of output, especially output that novices are likely to encounter. And exposing those messages to typical users, ones who are not familiar with core.spec. That seems prudent. I believe strongly enough in good error messages that I would be willing to do some of the scut work, if needed. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] data.avl 0.0.14 – BUGFIX to splits, faster map/set reductions
IReduce is a Java interface that can be used to allow a data structure to directly indicate participation in being self-reducible. In cases where you control the type (ie you are making the data structure), implementing IReduce (or really IReduceInit) is usually better as it is checked first. CollReduce is a protocol that provides a broader (open) extension mechanism to extend self-reducibility to other types. On Tuesday, August 23, 2016 at 8:38:27 AM UTC-5, Francis Avila wrote: > > I wasn't aware Clojure had an IReduce protocol! I thought CollReduce was > just what IReduce is called in Clojure. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
So personally, I don't want extremely accurate suggestions in the core of
Clojure. Why? Because I think they will never go far enough and I have a
ton of features I want to see that can't (shouldn't) be in the core of a
language.
I'll never forget the first undefined symbol I got in Clang after switching
from GCC. When I say "undefined local 'val' did you mean 'vals'?". I was
floored, what was this magic?
When I sit down and write a laundry list of things I would like to see in
error reporting for spec:
- Spelling suggestions: "did you mean :require instead of :requir?"
- Type suggestions: "you passed a symbol did you want a keyword?"
- Phonetic suggestions: "you passed :color, did you mean :colour?"
- Structural suggestions: "you passed [[42]] did you mean [42]?"
Most (all) of these are way too broad for inclusion in the core of a
language (imo), not to mention the problems you'd hit with multi-lingual
support. So I say to all that, give me data and let 3rd parties write the
libraries. I'd much rather have a 200MB "error reporter" lib in my
{:profiles {:dev ...}} than included in the clojure.jar.
Anyway, that's my 2cents. And if someone doesn't write that library for me
soon, I may have to :)
Timothy
On Tue, Aug 23, 2016 at 7:47 AM, Brian Marick
wrote:
>
> On Aug 22, 2016, at 7:50 PM, Alex Miller wrote:
>
>
> You've complained in other channels about the "learning to read" error
> messages part and I think you've taken it entirely the wrong way or maybe I
> just disagree. There are benefits from reporting errors in a generic,
> consistent way. […]
>
>
> Do there exist examples of what is desired for error messages in
> 1.9-final? Not promises, but a “this is what we’re shooting for”? What
> would you all like the specific error messages complained about in this
> thread to look like?
>
> Colin Fleming wrote: "The error message produced by the code I demoed at
> the conj last year would be:
>
> Unexpected symbol 'require' at while parsing
> namespace clauses. Expected :refer-clojure, :require, :use, :import, :load
> or :gen-class.”
>
> Is that the goal? I fear that the goal is that it should be my job to
> understand "(cat :attr-map (? map?) :clauses :clojure.core.specs/ns-clauses)”.
> For what little it’s worth, I consider that completely unacceptable.
>
> - Getting the error data (specifically the explain-data output) to be both
> sufficient and generically useful is the first priority. I think at this
> point that's pretty close and unlikely to change significantly.
>
>
> My bias here is that I come from the learned-from-bitter-experience
> tradition that believes it’s very risky to (1) get the infrastructure
> right, and then (2) pop down the user-visible features on top of it. Very
> often, the infrastructure turns out to be a poor match for the actual needs
> of the features. But, since (1) is already done, the features - and
> consequently the users - suffer.
>
> Please understand I’m not being insulting when I say that everyone has
> weaknesses and blind spots, even undoubted geniuses. In Clojure, error
> messages and documentation (especially doc strings) have long been glaring
> weaknesses. So I am wishing to be helpful when I counsel *quickly* getting
> to worked examples of output, especially output that novices are likely to
> encounter. And exposing those messages to typical users, ones who are not
> familiar with core.spec.
>
> That seems prudent.
>
> I believe strongly enough in good error messages that I would be willing
> to do some of the scut work, if needed.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and sto
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
I do not have an idea of what the final end point will look like exactly. I don't get the feeling that there is any answer that you will find satisfying, so I'm not sure what else I can do for you. We expect Clojure users to become familiar with spec and its output as it is (now) an essential part of the language. You will see specs in error messages. The focus in Clojure has always been biased towards building a powerful and expressive tool that is rewarding for experts vs optimizing for novices. Rich has talked at length about why that is (see https://www.infoq.com/presentations/design-composition-performance-keynote / https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/DesignCompositionPerformance.md in the section around languages as instruments). Pertinent bit (but you should watch the whole thing for context): So we need players. I would rant here, but I won't. But look at this guitar player with blisters. A harpist has blisters, a bass player with blisters. There's this barrier to overcome for every musician. Imagine if you downloaded something from GitHub and it gave you blisters. [Audience laughter] Right? The horrors! And yet how many people here play an instrument or have at one point in their lives? Yeah, a lot of programmers do. And for how many people did you just pick it up and it was awesome? How many wished, like, something could have made it more straightforward to get started with and, like, just made it easy? And how many would have believed after that that they could play it later? No, not at all. This is - it's actually quite important. The level of engagement that's required is quite important. So we shouldn't sell humanity short. Humans are incredible. In particular, they're incredible learners. One of the things that's really cool is you give a five-year-old or, I don't know, eight, maybe, a cello and some decent instruction, and they will learn how to play cello if they spend enough time doing it. In fact, humans will pretty much learn how to do anything that they spend enough time doing. We're incredibly good at it. And we're also really good teachers, in general. So I don't think we need to go to our tools and our instruments and make them oriented towards the first five seconds of people's experience because that's not going to serve them well. It's especially not going to serve anyone well who wants to achieve any kind of virtuosic ability with the tools. No one would become a virtuoso on the cello if they had red and green lights when they started. So neither of these two things is effort free, but we shouldn't be in a game to try to eliminate effort because we are novices, right? There's a sense in which we're only going to briefly be novices. You're only a complete beginning at something for an incredibly short period of time, and then you're over it. It's like we should not optimize for that. But, on the flipside, we're always learners no matter how much time you spend on the violin. Who sits there and says, "I'm done. I've completed learning violin. I finished it"? That's awesome. I personally don't play violin at all, but I don't think there would be a player on earth, no matter how great they are, who would say, "Yeah, I finished violin and I moved on to something else." We're constantly. It's just the human condition to do this. Things take effort. Just like we shouldn't target beginners, we shouldn't try to eliminate all effort. ...and there's more there - it's really worth reading/watching the whole thing. We are not apologetic about this bias. We expect you to engage and learn this tool that you're going to use for serious work because there's also deep payoff on the other side, just like learning to play the guitar or is more rewarding than learning to play the kazoo. I'm absolutely not talking about making something hard on purpose and I'm not saying that making things easy to learn is bad. I'm stating an ordering of priorities. It's more important to us to build a system of many parts that can be composed together into specifications that work as validators, and conformers, and sample generators, and error explainers, etc. We *also* want the automatic errors created from that to be useful and helpful and understandable thus this is a WIP. But creating error messages that are optimal for a user with no knowledge or Clojure or spec is just not the goal. Elena Machkasova has been doing research (supported in part by Cognitect) on figuring out what totally new users of Clojure need from error messages for her CS education classes and the answer there is just different from what an experienced user needs. That's ok. We care more about the latter. On Tuesday, August 23, 2016 at 8:49:38 AM UTC-5, Brian Marick wrote: > > > On Aug 22, 2016, at 7:50 PM, Alex Miller wrote: > > > You've complained in other channels about the "learning to read" error > messages part and I think yo
Re: ref strange behavior
Thank you very much! I suspected that I using the agents in wrong way. I
ran your code and it worked great.
Also I little changed your example to a worker keep only the last task and
run it on very large number of tasks (I had problem only on huge
collections):
(def tasks (ref (into [] (range 1 1000
(defn get-task
[tasks]
(dosync
(let [task (first @tasks)]
(alter tasks rest)
task)))
(defn worker-loop
[]
(loop [last-task nil]
(if-let [task (get-task tasks)]
(recur task)
last-task)))
(defn create-workers
[n & options]
(vec (repeatedly n (fn []
(future
(worker-loop))
(def workers
(create-workers 100))
(apply max (map deref workers))
This code worked as expected and returned 999.
But when I replace
(def tasks (ref (into [] (range 1 1000
to
(def tasks (ref (range 1 1000)))
the problem arose again and this code returnined 1734656.
So I think the problem is the lazy collection in the ref.
вторник, 23 августа 2016 г., 20:32:37 UTC+7 пользователь
adrian...@mail.yu.edu написал:
>
> Here is working example code demonstrating how you might do this without
> agents:
>
> (in-ns 'user)
>
> (def tasks (ref (into clojure.lang.PersistentQueue/EMPTY (range 1 1000
>
> (defn get-task
> [tasks]
> (dosync
> (let [task (first @tasks)]
> (alter tasks pop)
> task)))
>
> (defn worker-loop
> []
> (loop [completed-tasks []]
> (if-let [task (get-task tasks)]
> (recur (conj completed-tasks task))
> completed-tasks)))
>
> (defn create-workers
> [n & options]
> (vec (repeatedly n (fn []
>(future
> (worker-loop))
>
> (def workers
> (create-workers 100))
>
> (defn worker-test
> [xs]
> (= (set (mapcat deref workers))
> (set xs)))
>
> (worker-test (range 1 1000))
>
> On Tuesday, August 23, 2016 at 9:09:31 AM UTC-4, adrian...@mail.yu.edu
> wrote:
>>
>> I haven't run your code yet, but it's bad form to use Clojure's reference
>> types inside other reference types. They should store persistent data
>> structures to really make any sense in a concurrent context.
>>
>> On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>>>
>>> Hello,
>>>
>>> I had a problem when I run the following code:
>>>
>>> (defn get-task [tasks]
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task)))
>>>
>>> (defn worker [& {:keys [tasks]}]
>>> (agent {:tasks tasks}))
>>>
>>> (defn worker-loop [{:keys [tasks] :as state}]
>>> (loop [last-task nil]
>>> (if-let [task (get-task tasks)]
>>> (recur task)
>>> (locking :out (println "Last task: " last-task
>>> state)
>>>
>>> (defn create-workers [count & options]
>>> (->> (range 0 count)
>>>(map (fn [_] (apply worker options)))
>>>(into [])))
>>>
>>> (defn start-workers [workers]
>>> (doseq [worker workers] (send-off worker worker-loop)))
>>>
>>> (def tasks (ref (range 1 1000)))
>>>
>>> (def workers (create-workers 100 :tasks tasks))
>>>
>>> (start-workers workers)
>>> (apply await workers)
>>>
>>> Description: I have several agents (100 in my case). Each agent running
>>> in a separate thread. All agents share the one ref with the collection of
>>> tasks (range of longs in my case). Each agent get tasks from the collection
>>> (in transaction) one by one until the collection becomes empty and then
>>> prints the last task which it handle. However, when I run this code it
>>> looks like the collection of tasks suddenly becomes empty and workers
>>> handle only portion of all tasks (average 25-40% of all number).
>>>
>>> This code behave as I expected, when I create only one agent or use
>>> explicit locking in get-task function:
>>>
>>> (defn get-task [tasks]
>>> (locking :lock
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task
>>>
>>> I run this code on the Clojure 1.8.0
>>> java version "1.8.0_91"
>>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
>>> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>>>
>>> Can anyone tell me, what am I doing wrong, or it really looks like a bug
>>> in the clojure STM?
>>> I already asked this question on stackoverflow.com (
>>> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
>>>
>>> but so far nobody has been able to help me.
>>>
>>> P.S. Sorry for my english skill.
>>>
>>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscr
Re: ref strange behavior
Thank you! I know about Java's thread pools, but I want to do my job in
clojure-idiomatic way and possibly less using java interop.
вторник, 23 августа 2016 г., 20:21:23 UTC+7 пользователь tbc++ написал:
>
> This really seems like a good use case for Java's Executors:
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Executors.html#newFixedThreadPool(int)
>
> You can specify the number of worker threads, then enqueue runnable jobs.
> But wait! There's more! Clojure functions implement Runnable, so you can
> just hand a zero argument function to the service once you create it:
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#submit(java.lang.Runnable)
>
> On Tue, Aug 23, 2016 at 7:09 AM, >
> wrote:
>
>> I haven't run your code yet, but it's bad form to use Clojure's reference
>> types inside other reference types. They should store persistent data
>> structures to really make any sense in a concurrent context.
>>
>> On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>>
>>> Hello,
>>>
>>> I had a problem when I run the following code:
>>>
>>> (defn get-task [tasks]
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task)))
>>>
>>> (defn worker [& {:keys [tasks]}]
>>> (agent {:tasks tasks}))
>>>
>>> (defn worker-loop [{:keys [tasks] :as state}]
>>> (loop [last-task nil]
>>> (if-let [task (get-task tasks)]
>>> (recur task)
>>> (locking :out (println "Last task: " last-task
>>> state)
>>>
>>> (defn create-workers [count & options]
>>> (->> (range 0 count)
>>>(map (fn [_] (apply worker options)))
>>>(into [])))
>>>
>>> (defn start-workers [workers]
>>> (doseq [worker workers] (send-off worker worker-loop)))
>>>
>>> (def tasks (ref (range 1 1000)))
>>>
>>> (def workers (create-workers 100 :tasks tasks))
>>>
>>> (start-workers workers)
>>> (apply await workers)
>>>
>>> Description: I have several agents (100 in my case). Each agent running
>>> in a separate thread. All agents share the one ref with the collection of
>>> tasks (range of longs in my case). Each agent get tasks from the collection
>>> (in transaction) one by one until the collection becomes empty and then
>>> prints the last task which it handle. However, when I run this code it
>>> looks like the collection of tasks suddenly becomes empty and workers
>>> handle only portion of all tasks (average 25-40% of all number).
>>>
>>> This code behave as I expected, when I create only one agent or use
>>> explicit locking in get-task function:
>>>
>>> (defn get-task [tasks]
>>> (locking :lock
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task
>>>
>>> I run this code on the Clojure 1.8.0
>>> java version "1.8.0_91"
>>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
>>> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>>>
>>> Can anyone tell me, what am I doing wrong, or it really looks like a bug
>>> in the clojure STM?
>>> I already asked this question on stackoverflow.com (
>>> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
>>>
>>> but so far nobody has been able to help me.
>>>
>>> P.S. Sorry for my english skill.
>>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>>
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: ref strange behavior
On Tue, Aug 23, 2016 at 10:59 AM Sergei Koledov wrote: > Thank you! I know about Java's thread pools, but I want to do my job in > clojure-idiomatic way and possibly less using java interop. > > I think many of the use-cases for refs/agents are supplanted by core.async. In this case, the 'pipeline' family of functions can be used much like an executor pool. I'm not sure how idiomatic STM can be, when I've rarely used it. I think in most cases where you'd want to coordinate multiple refs within a transaction, a single atom is going to be much simpler and performant enough. I think clojure uses the 'toolkit' approach, in that it allows many things to be combined and encourages composition, but it doesn't prescribe or force you towards any particular shape of solution. That's a natural consequence of 'design by decoupling'. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
As some other people have stated: Its way, way premature to start worrying about the exact format of error messages. Given the facilities spec provides, its clear as day that vastly better messages can be built on top. Or even forget messages: syntax highlighting or source-code presentation can encode the same information without wading through text. Nothing is stopping people from attaching additional information to functions or keywords to help guide a reporting process. Its a great time for experimentation, if you have a better idea how to report the errors. On Tue, Aug 23, 2016 at 10:45 AM, Alex Miller wrote: > I do not have an idea of what the final end point will look like exactly. > I don't get the feeling that there is any answer that you will find > satisfying, so I'm not sure what else I can do for you. We expect Clojure > users to become familiar with spec and its output as it is (now) an > essential part of the language. You will see specs in error messages. > > The focus in Clojure has always been biased towards building a powerful > and expressive tool that is rewarding for experts vs optimizing for > novices. Rich has talked at length about why that is (see > https://www.infoq.com/presentations/design-composition-performance-keynote > / https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ > DesignCompositionPerformance.md in the section around languages as > instruments). Pertinent bit (but you should watch the whole thing for > context): > > So we need players. I would rant here, but I won't. But look at this > guitar player with blisters. A harpist has blisters, a bass player with > blisters. There's this barrier to overcome for every musician. Imagine if > you downloaded something from GitHub and it gave you blisters. > > [Audience laughter] > > Right? The horrors! And yet how many people here play an instrument or > have at one point in their lives? Yeah, a lot of programmers do. And for > how many people did you just pick it up and it was awesome? How many > wished, like, something could have made it more straightforward to get > started with and, like, just made it easy? And how many would have believed > after that that they could play it later? No, not at all. This is - it's > actually quite important. The level of engagement that's required is quite > important. > > So we shouldn't sell humanity short. Humans are incredible. In particular, > they're incredible learners. > > One of the things that's really cool is you give a five-year-old or, I > don't know, eight, maybe, a cello and some decent instruction, and they > will learn how to play cello if they spend enough time doing it. In fact, > humans will pretty much learn how to do anything that they spend enough > time doing. We're incredibly good at it. > > And we're also really good teachers, in general. So I don't think we need > to go to our tools and our instruments and make them oriented towards the > first five seconds of people's experience because that's not going to serve > them well. It's especially not going to serve anyone well who wants to > achieve any kind of virtuosic ability with the tools. No one would become a > virtuoso on the cello if they had red and green lights when they started. > > So neither of these two things is effort free, but we shouldn't be in a > game to try to eliminate effort because we are novices, right? > > There's a sense in which we're only going to briefly be novices. > > You're only a complete beginning at something for an incredibly short > period of time, and then you're over it. > > It's like we should not optimize for that. But, on the flipside, we're > always learners no matter how much time you spend on the violin. Who sits > there and says, "I'm done. I've completed learning violin. I finished it"? > That's awesome. I personally don't play violin at all, but I don't think > there would be a player on earth, no matter how great they are, who would > say, "Yeah, I finished violin and I moved on to something else." We're > constantly. It's just the human condition to do this. > > Things take effort. Just like we shouldn't target beginners, we shouldn't > try to eliminate all effort. > > > ...and there's more there - it's really worth reading/watching the whole > thing. We are not apologetic about this bias. We expect you to engage and > learn this tool that you're going to use for serious work because there's > also deep payoff on the other side, just like learning to play the guitar > or is more rewarding than learning to play the kazoo. > > I'm absolutely not talking about making something hard on purpose and I'm > not saying that making things easy to learn is bad. I'm stating an ordering > of priorities. It's more important to us to build a system of many parts > that can be composed together into specifications that work as validators, > and conformers, and sample generators, and error explainers, etc. We *also* > want the automatic errors created
[ANN] data.avl 0.0.15 – BUGFIX to subrange
Hi, I am pleased to announce the 0.0.15 release of data.avl, a Clojure Contrib library providing highly performant drop-in replacements for Clojure(Script)'s built-in sorted maps and sets that support O(log n) nth, rank-of, first-class submaps/subsets (like subseq, but preserving collection type; fully independent from the original for GC purposes) and splits by key and index. [org.clojure/data.avl "0.0.15"] org.clojure data.avl 0.0.15 org.clojure:data.avl:0.0.15 This is a bugfix release addressing a few issues in subrange: 1. subrange could sometimes misjudge whether its output collection should be empty and produce corrupt return values. 2. subrange incorrectly handled the single-test case where the specified limit fell past the extreme key actually present in the collection (e.g. (avl/subrange (avl/sorted-set 0 1 2) >= 3)). There are now multiple generative tests guarding against regressions on this front. Additionally, Darrick Wiebe, who reported DAVL-8 fixed in the previous release, provided generative tests that should guard against regressions in split-key. As a final tweak, the Clojure implementation of clojure.core.protocols/CollReduce is now replaced by an implementation of clojure.lang.IReduce. Thanks to Alex Miller for the suggestion! Cheers, Michał -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
On 8/23/16, 7:45 AM, "Alex Miller" wrote: > I'm absolutely not talking about making something hard on purpose > and I'm not saying that making things easy to learn is bad. I'm > stating an ordering of priorities. …and this is why I think it’s more important that Clojure is as “simple” and consistent as possible, so that when you are a beginner with Clojure, there are fewer surprises and the rules you need to learn are more straightforward. Despite clojure.spec messages not being “easy” to read at first – because they are unfamiliar – they are “simple” insofar as they are clearly structured and consistent and can be traced thru using just the ‘doc’ function. Sean -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
The error message I posted earlier comes right out of the parser with very
little infrastructure on top of it. It certainly doesn't require a 200MB
jar, more like 300 lines of code, including the parser itself.
On 24 August 2016 at 02:21, Timothy Baldridge wrote:
> So personally, I don't want extremely accurate suggestions in the core of
> Clojure. Why? Because I think they will never go far enough and I have a
> ton of features I want to see that can't (shouldn't) be in the core of a
> language.
>
> I'll never forget the first undefined symbol I got in Clang after
> switching from GCC. When I say "undefined local 'val' did you mean
> 'vals'?". I was floored, what was this magic?
>
> When I sit down and write a laundry list of things I would like to see in
> error reporting for spec:
> - Spelling suggestions: "did you mean :require instead of :requir?"
> - Type suggestions: "you passed a symbol did you want a keyword?"
> - Phonetic suggestions: "you passed :color, did you mean :colour?"
> - Structural suggestions: "you passed [[42]] did you mean [42]?"
>
> Most (all) of these are way too broad for inclusion in the core of a
> language (imo), not to mention the problems you'd hit with multi-lingual
> support. So I say to all that, give me data and let 3rd parties write the
> libraries. I'd much rather have a 200MB "error reporter" lib in my
> {:profiles {:dev ...}} than included in the clojure.jar.
>
> Anyway, that's my 2cents. And if someone doesn't write that library for me
> soon, I may have to :)
>
> Timothy
>
> On Tue, Aug 23, 2016 at 7:47 AM, Brian Marick
> wrote:
>
>>
>> On Aug 22, 2016, at 7:50 PM, Alex Miller wrote:
>>
>>
>> You've complained in other channels about the "learning to read" error
>> messages part and I think you've taken it entirely the wrong way or maybe I
>> just disagree. There are benefits from reporting errors in a generic,
>> consistent way. […]
>>
>>
>> Do there exist examples of what is desired for error messages in
>> 1.9-final? Not promises, but a “this is what we’re shooting for”? What
>> would you all like the specific error messages complained about in this
>> thread to look like?
>>
>> Colin Fleming wrote: "The error message produced by the code I demoed at
>> the conj last year would be:
>>
>> Unexpected symbol 'require' at while parsing
>> namespace clauses. Expected :refer-clojure, :require, :use, :import, :load
>> or :gen-class.”
>>
>> Is that the goal? I fear that the goal is that it should be my job to
>> understand "(cat :attr-map (? map?) :clauses
>> :clojure.core.specs/ns-clauses)”.
>> For what little it’s worth, I consider that completely unacceptable.
>>
>> - Getting the error data (specifically the explain-data output) to be
>> both sufficient and generically useful is the first priority. I think at
>> this point that's pretty close and unlikely to change significantly.
>>
>>
>> My bias here is that I come from the learned-from-bitter-experience
>> tradition that believes it’s very risky to (1) get the infrastructure
>> right, and then (2) pop down the user-visible features on top of it. Very
>> often, the infrastructure turns out to be a poor match for the actual needs
>> of the features. But, since (1) is already done, the features - and
>> consequently the users - suffer.
>>
>> Please understand I’m not being insulting when I say that everyone has
>> weaknesses and blind spots, even undoubted geniuses. In Clojure, error
>> messages and documentation (especially doc strings) have long been glaring
>> weaknesses. So I am wishing to be helpful when I counsel *quickly* getting
>> to worked examples of output, especially output that novices are likely to
>> encounter. And exposing those messages to typical users, ones who are not
>> familiar with core.spec.
>>
>> That seems prudent.
>>
>> I believe strongly enough in good error messages that I would be willing
>> to do some of the scut work, if needed.
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to cloj
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
>> It certainly doesn't require a 200MB jar, more like 300 lines of code,
including the parser itself.
I completely agree, what I'm saying is, if you start supporting my laundry
list of phonetic, spelling, and structural suggestions, your codesize will
quickly get much larger than 300 lines.
On Tue, Aug 23, 2016 at 1:16 PM, Colin Fleming
wrote:
> The error message I posted earlier comes right out of the parser with very
> little infrastructure on top of it. It certainly doesn't require a 200MB
> jar, more like 300 lines of code, including the parser itself.
>
> On 24 August 2016 at 02:21, Timothy Baldridge
> wrote:
>
>> So personally, I don't want extremely accurate suggestions in the core of
>> Clojure. Why? Because I think they will never go far enough and I have a
>> ton of features I want to see that can't (shouldn't) be in the core of a
>> language.
>>
>> I'll never forget the first undefined symbol I got in Clang after
>> switching from GCC. When I say "undefined local 'val' did you mean
>> 'vals'?". I was floored, what was this magic?
>>
>> When I sit down and write a laundry list of things I would like to see in
>> error reporting for spec:
>> - Spelling suggestions: "did you mean :require instead of :requir?"
>> - Type suggestions: "you passed a symbol did you want a keyword?"
>> - Phonetic suggestions: "you passed :color, did you mean :colour?"
>> - Structural suggestions: "you passed [[42]] did you mean [42]?"
>>
>> Most (all) of these are way too broad for inclusion in the core of a
>> language (imo), not to mention the problems you'd hit with multi-lingual
>> support. So I say to all that, give me data and let 3rd parties write the
>> libraries. I'd much rather have a 200MB "error reporter" lib in my
>> {:profiles {:dev ...}} than included in the clojure.jar.
>>
>> Anyway, that's my 2cents. And if someone doesn't write that library for
>> me soon, I may have to :)
>>
>> Timothy
>>
>> On Tue, Aug 23, 2016 at 7:47 AM, Brian Marick
>> wrote:
>>
>>>
>>> On Aug 22, 2016, at 7:50 PM, Alex Miller wrote:
>>>
>>>
>>> You've complained in other channels about the "learning to read" error
>>> messages part and I think you've taken it entirely the wrong way or maybe I
>>> just disagree. There are benefits from reporting errors in a generic,
>>> consistent way. […]
>>>
>>>
>>> Do there exist examples of what is desired for error messages in
>>> 1.9-final? Not promises, but a “this is what we’re shooting for”? What
>>> would you all like the specific error messages complained about in this
>>> thread to look like?
>>>
>>> Colin Fleming wrote: "The error message produced by the code I demoed at
>>> the conj last year would be:
>>>
>>> Unexpected symbol 'require' at while parsing
>>> namespace clauses. Expected :refer-clojure, :require, :use, :import, :load
>>> or :gen-class.”
>>>
>>> Is that the goal? I fear that the goal is that it should be my job to
>>> understand "(cat :attr-map (? map?) :clauses
>>> :clojure.core.specs/ns-clauses)”.
>>> For what little it’s worth, I consider that completely unacceptable.
>>>
>>> - Getting the error data (specifically the explain-data output) to be
>>> both sufficient and generically useful is the first priority. I think at
>>> this point that's pretty close and unlikely to change significantly.
>>>
>>>
>>> My bias here is that I come from the learned-from-bitter-experience
>>> tradition that believes it’s very risky to (1) get the infrastructure
>>> right, and then (2) pop down the user-visible features on top of it. Very
>>> often, the infrastructure turns out to be a poor match for the actual needs
>>> of the features. But, since (1) is already done, the features - and
>>> consequently the users - suffer.
>>>
>>> Please understand I’m not being insulting when I say that everyone has
>>> weaknesses and blind spots, even undoubted geniuses. In Clojure, error
>>> messages and documentation (especially doc strings) have long been glaring
>>> weaknesses. So I am wishing to be helpful when I counsel *quickly* getting
>>> to worked examples of output, especially output that novices are likely to
>>> encounter. And exposing those messages to typical users, ones who are not
>>> familiar with core.spec.
>>>
>>> That seems prudent.
>>>
>>> I believe strongly enough in good error messages that I would be willing
>>> to do some of the scut work, if needed.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
On Tue, Aug 23, 2016 at 7:45 AM, Alex Miller wrote: > We expect Clojure users to become familiar with spec and its output as it > is (now) an essential part of the language. You will see specs in error > messages. > Is there any documentation to help users understand how to interpret the error messages? For example, what is the "path" returned by spec errors? I've been through the spec guide, and don't recall seeing anything like that. I've been able to make a few deductions on my own, but without a better mental model, the messages are still pretty mystifying to me. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
The path is the series of tags you've traversed through the spec (when there are parts in :or :alt :cat etc). We will have more documentation on it but we've held off because it was changing pretty regularly in early alphas. A spec for the current explain-data is something like this (I'm just typing this in, so could be a bit off): ;; path is a stack of parts, which refer to tags in schemas (s/def ::part simple-keyword?) (s/def ::path (s/coll-of ::part :kind vector?)) ;; pred is a symbol, a set, or a function, but being lazy here (s/def ::pred any?) ;; the failing data value (s/def ::val any?) ;; via is a stack or schema identifiers (s/def ::schema-key qualified-ident?) (s/def ::via (s/coll-of ::schema-key :kind vector?)) ;; in is a stack of keys in nested data (s/def ::data-key any?) (s/def ::in (s/coll-of ::data-key :kind vector?)) ;; some of these may be required, I would have to analyze the code further (s/def ::problem (s/keys :opt-un [::path ::pred ::val ::via ::in])) (s/def ::problems (s/coll-of ::problem :min-count 1)) ;; added when macroexpand args check fails (s/def ::args (s/coll-of any?)) (s/def ::explain-data (s/keys :req [::problems] :opt [::args])) On Tuesday, August 23, 2016 at 2:33:34 PM UTC-5, puzzler wrote: > > On Tue, Aug 23, 2016 at 7:45 AM, Alex Miller wrote: > >> We expect Clojure users to become familiar with spec and its output as it >> is (now) an essential part of the language. You will see specs in error >> messages. >> > > Is there any documentation to help users understand how to interpret the > error messages? For example, what is the "path" returned by spec errors? > I've been through the spec guide, and don't recall seeing anything like > that. I've been able to make a few deductions on my own, but without a > better mental model, the messages are still pretty mystifying to me. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
While I think the spec errors are pretty unfriendly, it's probably worth remembering that most of the times you'll get one you would have got an inscrutable ClassCastException or incorrect behaviour from a totally different place in the codebase prior to spec. It's definitely a huge improvement on that. The :require issue was an unfortunate first encounter with a spec error because it caught a situation that wasn't actually causing a problem, I think most spec errors users see in the wild that won't be true for. On Tuesday, August 23, 2016 at 6:49:38 AM UTC-7, Brian Marick wrote: > > > On Aug 22, 2016, at 7:50 PM, Alex Miller > wrote: > > > You've complained in other channels about the "learning to read" error > messages part and I think you've taken it entirely the wrong way or maybe I > just disagree. There are benefits from reporting errors in a generic, > consistent way. […] > > > Do there exist examples of what is desired for error messages in > 1.9-final? Not promises, but a “this is what we’re shooting for”? What > would you all like the specific error messages complained about in this > thread to look like? > > Colin Fleming wrote: "The error message produced by the code I demoed at > the conj last year would be: > > Unexpected symbol 'require' at while parsing > namespace clauses. Expected :refer-clojure, :require, :use, :import, :load > or :gen-class.” > > Is that the goal? I fear that the goal is that it should be my job to > understand "(cat :attr-map (? map?) :clauses > :clojure.core.specs/ns-clauses)”. For what little it’s worth, I consider > that completely unacceptable. > > - Getting the error data (specifically the explain-data output) to be both > sufficient and generically useful is the first priority. I think at this > point that's pretty close and unlikely to change significantly. > > > My bias here is that I come from the learned-from-bitter-experience > tradition that believes it’s very risky to (1) get the infrastructure > right, and then (2) pop down the user-visible features on top of it. Very > often, the infrastructure turns out to be a poor match for the actual needs > of the features. But, since (1) is already done, the features - and > consequently the users - suffer. > > Please understand I’m not being insulting when I say that everyone has > weaknesses and blind spots, even undoubted geniuses. In Clojure, error > messages and documentation (especially doc strings) have long been glaring > weaknesses. So I am wishing to be helpful when I counsel *quickly* getting > to worked examples of output, especially output that novices are likely to > encounter. And exposing those messages to typical users, ones who are not > familiar with core.spec. > > That seems prudent. > > I believe strongly enough in good error messages that I would be willing > to do some of the scut work, if needed. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
> > But creating error messages that are optimal for a user with no knowledge > or Clojure or spec is just not the goal. This is a totally false dichotomy. No-one in this thread is asking for that. This thread has several examples of expert Clojure users for whom the error messages are incomprehensible. I am equally unapologetic about thinking that the musical instrument analogy is mostly bogus here. There are things that will always be difficult about learning Clojure because they're conceptual, such as functional programming. I think the analogy is fair there, they are just things that will require effort and practice to learn. But the error messages are about giving people the information they need *so that they can actually learn from their mistakes*. Clojure has historically been appallingly bad at that, and no-one should expect their users to flail around randomly trying things to see what works. I've spoken to various smart people who have described their experience of using Clojure as exactly that, even after a non-trivial amount of time using it. I hope spec can improve on that experience. On 24 August 2016 at 02:45, Alex Miller wrote: > I do not have an idea of what the final end point will look like exactly. > I don't get the feeling that there is any answer that you will find > satisfying, so I'm not sure what else I can do for you. We expect Clojure > users to become familiar with spec and its output as it is (now) an > essential part of the language. You will see specs in error messages. > > The focus in Clojure has always been biased towards building a powerful > and expressive tool that is rewarding for experts vs optimizing for > novices. Rich has talked at length about why that is (see > https://www.infoq.com/presentations/design-composition-performance-keynote > / https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ > DesignCompositionPerformance.md in the section around languages as > instruments). Pertinent bit (but you should watch the whole thing for > context): > > So we need players. I would rant here, but I won't. But look at this > guitar player with blisters. A harpist has blisters, a bass player with > blisters. There's this barrier to overcome for every musician. Imagine if > you downloaded something from GitHub and it gave you blisters. > > [Audience laughter] > > Right? The horrors! And yet how many people here play an instrument or > have at one point in their lives? Yeah, a lot of programmers do. And for > how many people did you just pick it up and it was awesome? How many > wished, like, something could have made it more straightforward to get > started with and, like, just made it easy? And how many would have believed > after that that they could play it later? No, not at all. This is - it's > actually quite important. The level of engagement that's required is quite > important. > > So we shouldn't sell humanity short. Humans are incredible. In particular, > they're incredible learners. > > One of the things that's really cool is you give a five-year-old or, I > don't know, eight, maybe, a cello and some decent instruction, and they > will learn how to play cello if they spend enough time doing it. In fact, > humans will pretty much learn how to do anything that they spend enough > time doing. We're incredibly good at it. > > And we're also really good teachers, in general. So I don't think we need > to go to our tools and our instruments and make them oriented towards the > first five seconds of people's experience because that's not going to serve > them well. It's especially not going to serve anyone well who wants to > achieve any kind of virtuosic ability with the tools. No one would become a > virtuoso on the cello if they had red and green lights when they started. > > So neither of these two things is effort free, but we shouldn't be in a > game to try to eliminate effort because we are novices, right? > > There's a sense in which we're only going to briefly be novices. > > You're only a complete beginning at something for an incredibly short > period of time, and then you're over it. > > It's like we should not optimize for that. But, on the flipside, we're > always learners no matter how much time you spend on the violin. Who sits > there and says, "I'm done. I've completed learning violin. I finished it"? > That's awesome. I personally don't play violin at all, but I don't think > there would be a player on earth, no matter how great they are, who would > say, "Yeah, I finished violin and I moved on to something else." We're > constantly. It's just the human condition to do this. > > Things take effort. Just like we shouldn't target beginners, we shouldn't > try to eliminate all effort. > > > ...and there's more there - it's really worth reading/watching the whole > thing. We are not apologetic about this bias. We expect you to engage and > learn this tool that you're going to use for serious work because there's > also deep payoff
[ANN] data.avl 0.0.16 – bugfix to one-sided subrange
Hi, data.avl 0.0.16 is now available with a bugfix to the "one-sided subrange past end of collection" case of subrange partly fixed in 0.0.15. [org.clojure/data.avl "0.0.16"] org.clojure data.avl 0.0.16 org.clojure:data.avl:0.0.16 Cheers, Michał -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Two suggestions re: core.spec, `ns`, and clojure 1.9alpha11
I'm not anywhere near as deep into spec as would be needed to fully understand this thread, but I will say that I agree with those who object to the guitar analogy. That argument would work just as well as a response to someone who complained about the difficulty of C++, or assembler, or APL. What's beautiful about Clojure is that it is, right now, already, a language for experts--*and* for novices (except for error reporting). I'm not in favor of an assumption that it's only a language for experts, and that therefore it's OK if error reporting is hard for novices. My standard for error reporting: modern Common Lisp implementations (e.g. SBCL, ABCL, CCL, LispWorks). Reading any stack trace requires learning, but the CL stacktraces are nowhere near as daunting as Java and Clojure. I don't expect Clojure's error reporting to ever be that good; I suspect that the overhead of doing that kind of thing on the JVM is not worth the cost. But it can aspire to that kind of ease. (In other respects, CL is more difficult than Clojure, imo.) If you read the introduction to Bjarne Soutstrup's C++ book 3rd ed (I don't know whether it's still there in the 4th), you'll see that at one point he says that his goal was to keep C++ a simple language. If you go on and read the rest of the book, you'll see how far from that goal the language ended up. I have (rare) moments when I worry that Clojure is great because it's young and hasn't had time to become a mess. I still trust Rich's judgement, though. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
