Re: comparator that compares collections similar to how strings are compared
Hm, google groups did not show my first reply to myself for a while,
and I thought it had gotten lost in the way, so I sent another post,
which can be safely ignored, in case anyone wonders what the
difference between the first and the second reply is - there is none,
fundamentally.
On Jul 10, 7:54 pm, Eugen Dück wrote:
> Well, turns out string-like-coll-comparator has a bug:
>
> x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
> (vec "n") 2) > (vec "a"))
> NullPointerException clojure.lang.AFunction.compare (AFunction.java:
> 59)
>
> It can be fixed though:
>
> (defn string-like-coll-comparator
> [coll1 coll2]
> (or (first (drop-while zero? (map compare coll1 coll2)))
> (- (count coll1) (count coll2
>
> Now, is this the way to get the sorting behavior I want or is anyone
> aware of something simpler or more performant?
>
> On Jul 10, 6:09 pm, Eugen Dück wrote:
>
>
>
>
>
>
>
> > Say I have this sorted map, using strings as keys:
>
> > x=> (sorted-map "ab" 1 "n" 2)
> > {"ab" 1, "n" 2}
>
> > When I do a subseq
>
> > x=> (subseq (sorted-map "ab" 1 "n" 2) > "aa")
> > (["ab" 1] ["n" 2])
>
> > I get back both entries.
>
> > Now if I do the same subseq on the same map, except that I turn all
> > strings into character collections, I only get the "ab" entry:
>
> > x=> (subseq (sorted-map (vec "ab") 1 (vec "n") 2) > (vec "aa"))
> > ([[\a \b] 1])
>
> > I guess that's fine, the length of collections is probably used first
> > and only if it differs would the actual entries be compared.
>
> > Using my own comparator to simulate the string comparison I can get
> > the string-like behavior:
>
> > (defn string-like-coll-comparator
> > [coll1 coll2]
> > (first (drop-while zero? (map compare coll1 coll2
>
> > x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
> > (vec "n") 2) > (vec "aa"))
> > ([[\a \b] 1] [[\n] 2])
>
> > Here's my question - is this the best way to get what I want?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: comparator that compares collections similar to how strings are compared
Well, turns out string-like-coll-comparator has a bug:
x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
(vec "n") 2) > (vec "a"))
NullPointerException clojure.lang.AFunction.compare (AFunction.java:
59)
It can be fixed though:
(defn string-like-coll-comparator
[coll1 coll2]
(or (first (drop-while zero? (map compare coll1 coll2)))
(- (count coll1) (count coll2
Now, is this the way to get the sorting behavior I want or is anyone
aware of something simpler or more performant?
On Jul 10, 6:09 pm, Eugen Dück wrote:
> Say I have this sorted map, using strings as keys:
>
> x=> (sorted-map "ab" 1 "n" 2)
> {"ab" 1, "n" 2}
>
> When I do a subseq
>
> x=> (subseq (sorted-map "ab" 1 "n" 2) > "aa")
> (["ab" 1] ["n" 2])
>
> I get back both entries.
>
> Now if I do the same subseq on the same map, except that I turn all
> strings into character collections, I only get the "ab" entry:
>
> x=> (subseq (sorted-map (vec "ab") 1 (vec "n") 2) > (vec "aa"))
> ([[\a \b] 1])
>
> I guess that's fine, the length of collections is probably used first
> and only if it differs would the actual entries be compared.
>
> Using my own comparator to simulate the string comparison I can get
> the string-like behavior:
>
> (defn string-like-coll-comparator
> [coll1 coll2]
> (first (drop-while zero? (map compare coll1 coll2
>
> x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
> (vec "n") 2) > (vec "aa"))
> ([[\a \b] 1] [[\n] 2])
>
> Here's my question - is this the best way to get what I want?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: comparator that compares collections similar to how strings are compared
Well, it can't be the best, because it has a bug. Calling it with "a"
instead of "aa" gives me a NPE:
x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
(vec "n") 2) > (vec "a"))
NullPointerException clojure.lang.AFunction.compare (AFunction.java:
59)
The following comparator should do the trick:
(defn string-like-coll-comparator
[coll1 coll2]
(or (first (drop-while zero? (map compare coll1 coll2)))
(- (count coll1) (count coll2
x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
(vec "n") 2) > (vec "a"))
([[\a \b] 1] [[\n] 2])
So is this then the best way to get the sorting behavior I need?
On Jul 10, 6:09 pm, Eugen Dück wrote:
> Say I have this sorted map, using strings as keys:
>
> x=> (sorted-map "ab" 1 "n" 2)
> {"ab" 1, "n" 2}
>
> When I do a subseq
>
> x=> (subseq (sorted-map "ab" 1 "n" 2) > "aa")
> (["ab" 1] ["n" 2])
>
> I get back both entries.
>
> Now if I do the same subseq on the same map, except that I turn all
> strings into character collections, I only get the "ab" entry:
>
> x=> (subseq (sorted-map (vec "ab") 1 (vec "n") 2) > (vec "aa"))
> ([[\a \b] 1])
>
> I guess that's fine, the length of collections is probably used first
> and only if it differs would the actual entries be compared.
>
> Using my own comparator to simulate the string comparison I can get
> the string-like behavior:
>
> (defn string-like-coll-comparator
> [coll1 coll2]
> (first (drop-while zero? (map compare coll1 coll2
>
> x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
> (vec "n") 2) > (vec "aa"))
> ([[\a \b] 1] [[\n] 2])
>
> Here's my question - is this the best way to get what I want?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
comparator that compares collections similar to how strings are compared
Say I have this sorted map, using strings as keys:
x=> (sorted-map "ab" 1 "n" 2)
{"ab" 1, "n" 2}
When I do a subseq
x=> (subseq (sorted-map "ab" 1 "n" 2) > "aa")
(["ab" 1] ["n" 2])
I get back both entries.
Now if I do the same subseq on the same map, except that I turn all
strings into character collections, I only get the "ab" entry:
x=> (subseq (sorted-map (vec "ab") 1 (vec "n") 2) > (vec "aa"))
([[\a \b] 1])
I guess that's fine, the length of collections is probably used first
and only if it differs would the actual entries be compared.
Using my own comparator to simulate the string comparison I can get
the string-like behavior:
(defn string-like-coll-comparator
[coll1 coll2]
(first (drop-while zero? (map compare coll1 coll2
x=> (subseq (sorted-map-by string-like-coll-comparator (vec "ab") 1
(vec "n") 2) > (vec "aa"))
([[\a \b] 1] [[\n] 2])
Here's my question - is this the best way to get what I want?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: deploying application updates to end users
Java Web Start is super easy to use for end users, if that is an option for you. All they have to do is click a link on a web page. The signing part is a bit of a pita to set up, unless you follow one of the quick howtos on web start for Java in general, and maybe by now there are some clojure (lein?) specific ones. I have used web start for a clojure pet project of mine a while back and the main problem to solve - apart from setting up the signing infrastructure - were security permissions so you can leave the Java sandbox. Clojure has its own classloader, which complicates these things a little bit. If you are going to take that route and run into any security permission issues, let me know and I'll try to dig up my solution to that problem. Haven't looked into updatingthough, but iirc web start has means for that, and you could also decide to let your app only run online, making it always up to date (the jars would be cached on the users disk, so no download every time). On Feb 19, 10:30 am, Seth wrote: > Has anyone had any experience in the best way to deploy updates to end > users who have no programming experience? The application basically > consists of a src directory, and a bat file which runs 'lein run' to > start the program. It also consists of some extraneous files that have > to be installed on the system under another program directory. > > The simplest method would be to take zip files and unzip them to the > appropriate place. But i was wondering about things like > 'uninstalling' updates, automatic updates from (free hosting?) > website, better 'patch' formats, etc. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
metadata example on http://clojure.org/evaluation
There's an example on metadata that resembles a repl session snippet
(although not exactly):
user=> (def x 1)
user=> (def y 2)
user=> ^{:x x} [x y 3]
^{:x 1} [1 2 3]
I interpret that as "the reader prints out meta data", which of course
is not true (by default). It could be changed, e.g. to
user=> (def x 1)
user=> (def y 2)
user> (meta ^{:x x} [x y 3])
{:x 1}
or to
user=> (def x 1)
user=> (def y 2)
user> (set! *print-meta* true)
true
user=> ^{:x x} [x y 3]
^{:x 1} [1 2 3]
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Any news on pull requests?
Not that size should matter when it comes to legalistics, but it appears the already mentioned KDE project, claiming that "the KDE community is the second largest Free Software community" by most measures, after the linux kernel, does not seem to have introduced a CA requirement by now. (From quickly glancing over a bunch of links on http://techbase.kde.org/Contribute and google, but maybe an active KDE contributor could verify this) Apart from them using different licenses than Clojure, which might have different requirements, if my last post's (completely unfounded) assumption is right, they would be more constrained in the future when it comes to changing things related to their licenses. On Feb 11, 3:00 pm, Mike Meyer wrote: > On Thu, 10 Feb 2011 21:57:01 -0800 > > Sean Corfield wrote: > > Agreement pretty much identical to the Clojure / Oracle CA (with the > > only exception to the process being that they will allow scanned, > > signed CAs to be emailed to the project team). > > That exception is the major issue, assuming you don't have issues with > the terms of the CA. > > > As noted elsewhere in this thread, written CAs are actually pretty > > common in large, successful open source projects... > > A far more interesting statistic would be how many of those large, > successful open source projects had CAs *before* they were large and > successful. > > -- > Mike Meyer http://www.mired.org/consulting.html > Independent Software developer/SCM consultant, email for more information. > > O< ascii ribbon campaign - stop html mail -www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Any news on pull requests?
Thanks guys for all the input. I finally read the CA, and I guess the principal difference between clojure contributions and wikipedia (which you wouldn't even know what the heck it was if they had asked everyone to sign a CA first) contributions - is not the type of content - code vs. prose (and the division is not that sharp either, as both contain both things, and more) - but the various transfers of rights of whom some are on the brink of granting exclusivity. And of course IANAL. That said, I've contributed to a number of open source projects, none of which required anything physical from me, and which wouldn't have gotten anything from me had they asked, just because I wouldn't have wanted to spend the time on red tape for every single one small patch (although important to me) for every one of those projects. Actually in the past I did also contribute to a bunch of different parts of the iirc biggest open source project - KDE - without ever signing anything, in fact mostly using some alias (so now you where all those pesky bugs originate ;). But I guess it all comes down to what Rich wants to be able to do with the code base and not get locked in by some guys he has to chase for permission later on if he wants to rearrange things, which I can understand. And for those posters telling us not to "waste your time, stop asking questions and just trust Rich": Why don't you stop wasting your time telling us what to do with ours? As this thread shows, it's still an open question, with some people curious for the answer. And even though I'll send the CA out this weekend, I am curious as to why signing is necessary for this project, when it isn't for others. And at least I have some ideas now. In terms of git pull requests by CA signers, I'm still hopeful though, as I don't see the legal issue there, and there's a hint in "please don't send pull requests via GitHub AT THIS TIME". Cheers Eugen On Feb 5, 11:45 am, Mike Meyer wrote: > On Fri, 4 Feb 2011 18:36:34 -0800 > > > > Sean Corfield wrote: > > On Fri, Feb 4, 2011 at 6:16 PM, Eugen Dück wrote: > > > Is it really necessary, though? We all agree to EULAs and make other > > > more significant legal commitments online all the time, and in some > > > cases without having proven who and where we are. > > > There are certainly some legal transactions that do not accept > > electronic "agreements" and require a physical signature. > > > IANAL so I looked up US copyright law and found this paragraph about > > transfers in Circular 1 (from here > >http://www.copyright.gov/help/faq/faq-assignment.html): > > > "Any or all of the copyright owner’s exclusive rights or any > > subdivision of those rights may be transferred, but the transfer of > > exclusive rights is not valid unless that transfer is in writing and > > signed by the owner of the rights conveyed or such owner’s duly > > authorized agent. Transfer of a right on a nonexclusive basis does not > > require a written agreement." > > > So that's why a written signature is required for the Clojure CA. > > Um, read the last line in the quote, about "nonexclusive basis". > > The first bullet of clause two in the CA (downloaded just now) grants > Rich Hickey a "... perpetual, irrevocable, non-exclusive, worldwide > ... license" > > Given that the license is nonexclusive transfer (and I have to wonder > if you'd get any contributors otherwise, or if any other OSS project > has an exclusive transfer), then according to that last line, it "does > not require a written agreement." > > IANAL either, but it sure seems like the current requirements exceeds > what the law requires. > > -- > Mike Meyer http://www.mired.org/consulting.html > Independent Network/Unix/Perforce consultant, email for more information. > > O< ascii ribbon campaign - stop html mail -www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Time/size bounded cache?
And if you don't have time to read the whole blog post - it's very
detailed - and just read code, you could scroll down to cgrand's
memoize8 function at https://gist.github.com/330644
On Feb 6, 8:32 pm, Eugen Dück wrote:
> A while back the discussion on the "bounded memoize" thread in this
> forum lead to Meikel writing up a nice summary of how to do caching,
> providing a pluggable strategy: lru, ttl, fifo, etc. Meikel's writeup
> can be found athttp://kotka.de/blog/2010/03/memoize_done_right.html
>
> It presents a bunch of different implementations, detailing whats good/
> bad about each of them.
>
> On Dec 31 2010, 3:35 am, Miki wrote:
>
> > Hello,
>
> > I'm wring a service that polls RSS feeds (using feedme). And I'd like to act
> > only on the new entries in each feed.
> > So far I have something like:
>
> > (defn get-new-entries [seen]
> > (let [seen? (complement seen)]
> > (filter #(seen? (:id %)) (get-entries
>
> > (defn poll [sleep-time]
> > (loop [seen #{}]
> > (let [entries (get-new-entries seen)]
> > (doall (map process-entry entries))
> > (println "")
> > (Thread/sleep sleep-time)
> > (recur (union seen (set (map :id entries)))
>
> > (Full demo code athttps://gist.github.com/760094)
>
> > The problem is that "seen" will grow without bounds.
> > Is there a built in way to have some sort of LRU cache or should I use
> > external libraries (like plru)?
>
> > Thanks,
> > --
> > Miki
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Time/size bounded cache?
A while back the discussion on the "bounded memoize" thread in this
forum lead to Meikel writing up a nice summary of how to do caching,
providing a pluggable strategy: lru, ttl, fifo, etc. Meikel's writeup
can be found at
http://kotka.de/blog/2010/03/memoize_done_right.html
It presents a bunch of different implementations, detailing whats good/
bad about each of them.
On Dec 31 2010, 3:35 am, Miki wrote:
> Hello,
>
> I'm wring a service that polls RSS feeds (using feedme). And I'd like to act
> only on the new entries in each feed.
> So far I have something like:
>
> (defn get-new-entries [seen]
> (let [seen? (complement seen)]
> (filter #(seen? (:id %)) (get-entries
>
> (defn poll [sleep-time]
> (loop [seen #{}]
> (let [entries (get-new-entries seen)]
> (doall (map process-entry entries))
> (println "")
> (Thread/sleep sleep-time)
> (recur (union seen (set (map :id entries)))
>
> (Full demo code athttps://gist.github.com/760094)
>
> The problem is that "seen" will grow without bounds.
> Is there a built in way to have some sort of LRU cache or should I use
> external libraries (like plru)?
>
> Thanks,
> --
> Miki
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Any news on pull requests?
scan -> email would be more convenient for me. @Chistopher > people keep citing it as some unique requirement foisted upon the community > by a power hungry dictator That's a generalization that lacks any relation to this thread, and I guess even the clojure community in general. I'm not aware of any criticism of our beloved dictator and I hope he'll serve many more terms. As long as he doesn't torture his people (like force them into signing CAs ;) Back to the git pull request question - nobody seems to know anything about it, but that issue should not be related to legal issues, if clojure maintainers pull only stuff from CA signers, right? On Feb 6, 10:42 am, Christopher Petrilli wrote: > On Sat, Feb 5, 2011 at 1:23 PM, Mike Meyer > > wrote: > > On Sat, 5 Feb 2011 00:09:41 -0500 > > Christopher Petrilli wrote: > >> For example, the following projects REQUIRE contributor agreements, in > >> writing, signed and either scanned or on paper, prior to accepting any > >> patches or commits: > > >> - Free Software Foundation > >> - Apache, and everything under it > >> - Python > > I'm sorry, I'm going to call foul on this. I've contributed to Python > > without ever signing a CA. And the current developers guide page > > doesn't have anything on it about needing to sign a CA. > > I do not know when you contributed, or whether it was before this > period, but > fromhttp://wiki.python.org/moin/PythonSoftwareFoundationLicenseFaq: > > > If your code is going to end up in Python or the standard library, the PSF > > will require you to: > > * License your code under an acceptable open source license. These > > currently include only the Academic Free License and the Apache License > > 2.0, although this list may be expanded in the future. (No, the PSF License > > is not acceptable; see below) > > * Fill out and submit a contributor agreement. > > Perhaps they exclude some small amount of code in the form of patches, > but it certainly applies to libraries, etc. You can find the > contributor agreement here:http://www.python.org/psf/contrib-form.htmlIt's > not that different > than the one for Clojure. Again, I'm not arguing whether it's a "good > thing" or not, simply that it's actually not as unheard of as people > think, nor is it poorly founded in US legal precedent. > > > Seriously, the "snail-mail" requirement is the only one that's really > > objectionable. Most places are quite happy with a scanned image of the > > signed document (i.e. - the Chickasaw nation for my citizenship > > papers). > > Then perhaps someone can offer to Rich to accept the scanned copies > and deal with them? One of the "joys" of an open source project is not > just contributing code, but helping out with the administrative > overhead of running a project. > > Chris > -- > | Chris Petrilli > | petri...@amber.org -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: force evaluation of macro parameter
I'm talking about '~name on https://github.com/clojure/clojure-contrib/blob/master/modules/dataflow/src/main/clojure/clojure/contrib/dataflow.clj#L338 and below. I want to be able to generate names, i.e. I don't have them as literals. On Feb 5, 2:29 pm, Ken Wesson wrote: > On Sat, Feb 5, 2011 at 12:08 AM, Eugen Dück wrote: > > yes, the code is more complex and it is in contrib. Changing contrib > > requires you to send an intercontinental mail first, as mentioned in > > my other post today... :) Reason enough to first checkout the other > > options and dig deeper into macros. > > > Will for now just try to solve it in my contrib fork. > > Which macro is it and what are you trying to do with it? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: force evaluation of macro parameter
On Feb 5, 1:52 pm, Eric Lavigne wrote: > This makes me think that the original macro needs some refactoring. > There should be a function that handles most of the work, and a macro > to make your code shorter in the common case. > > (defn unquoted-param [x] (println x)) > > (defmacro quoted-param [x] `(unquoted-param '~x)) > > Of course, this looks silly because unquoted-param is just println, > but I assume your real situation has a bit more to it. In general, you > should try to use functions more often macros. Even when a macro is > needed, it's still often best to let a function do most of the work. Thanks Eric, yes, the code is more complex and it is in contrib. Changing contrib requires you to send an intercontinental mail first, as mentioned in my other post today... :) Reason enough to first checkout the other options and dig deeper into macros. Will for now just try to solve it in my contrib fork. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
force evaluation of macro parameter
I'm using a macro that, stripped down to just expose my problem, looks like this: (defmacro quoted-param [x] `(println '~x)) It's all nice if I call it like (quoted-param 23) It will print the number 23. The following, however, will print "asdf", rather than 23: (def asdf 23) (quoted-param asdf) Which is of course what the quote is supposed to do. But is there any way to get that macro to expand to using the value of asdf, rather than the symbol itself? Or can only changing the macro fix this? I fear the latter, which would imply that using quotes like that in a macro should be done with great care, I guess. Adding a quote when 'calling' the macro is easy... Cheers Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Any news on pull requests?
On Feb 5, 11:51 am, Sean Corfield wrote: > off to Rich... If someone really feels signing and mailing an > agreement is "too much work" then they don't seem very committed to > contributing, IMO. It's really not much of a hardship is it? Things like github's pull requests are really great, as they are lowering the barriers for both the contributor to contribute as well as for the maintainer to review and merge in changes. And although the physical CA is 'orthogonal' to pull requests - btw. my real question here - it is setting up a barrier. If you'd have to sign and send mails for every open source project you want to contribute to, it would be pretty annoying. Not everyone is a main contributor to every project they have a small patch for. That said, clojure for me surely is a project that could convince me to send out that form, but I'm not that big a stamp donor these days. > "If you're not annoying somebody, you're not really alive." > -- Margaret Atwood So chances are I'm "really alive". -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Any news on pull requests?
On Feb 5, 11:00 am, Sean Corfield wrote: > On Fri, Feb 4, 2011 at 5:24 PM, Eugen Dück wrote: > > Furthermore, I was really surprised to find > > onhttp://clojure.org/contributing > > that I have to send a (non-e)mail around the world to be able to > > contribute > > Written acceptance of a contributor's agreement is fairly common on > large open source projects so that there is legal clearance for > inclusion of your contribution under the terms of the project license. Is it really necessary, though? We all agree to EULAs and make other more significant legal commitments online all the time, and in some cases without having proven who and where we are. Otoh, I guess Rich hasn't done this because he likes receiving mails... Or maybe he is indeed a stamp collector? And clojure CAs is just his way of attracting more stamps? ;) -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Any news on pull requests?
In June 2009, Rich wrote in "clojure goes git!": > Some items are still outstanding: > > Importation of existing issues > Placement of generated contrib documentation > Patch submission policy > > In particular, please don't send pull requests via GitHub at this > time. Any updates there? I really would like to contribute to clojure- contrib the github way. Furthermore, I was really surprised to find on http://clojure.org/contributing that I have to send a (non-e)mail around the world to be able to contribute (for the younger readers: http://simple.wikipedia.org/wiki/Mail has a pretty good explanation of what that was). Can't we agree to those terms by pushing a (pull request) button? Cheers Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: JSpinner SpinnerModel for numbers with different step sizes - beating Java in its own domain
To guard against NPEs, align-to-tick-size should be (defn align-to-tick-size [tick-map value] (if-let [tick-size (get-tick-size tick-map value false)] (let [remainder (mod value tick-size)] (if (zero? remainder) value (* tick-size ((if (>= remainder (/ tick-size 2)) inc identity) (quot value tick-size) value)) On Jan 23, 11:18 am, Eugen Dück wrote: > Anyone who has seen the subject implemented in Java/Swing might have > been stunned by the hundreds of lines of code that were deemed > necessary for it, with their appropriate share of bugs. (And an > implementation I've seen in C#/NetAdvantage was even worse, which was > less related to the language itself, as to the GUI toolkit) > > I knew this could be done way more elegant, but I was surprised it can > be done THIS nicely - if done in clojure. And if you think this can be > improved further or find any bugs, shoot! > > Example: Prices for Japanese stock have a step size of > 1 in the interval [0,3000], > 5 in the interval (3000,5000], > ... > > An easy way to model that would be a sorted map: > > (def japanese-stock-tick-map > (sorted-map > 0 1 > 3000 5 > 5000 10 > 3 50 > 5 100 > 30 500 > 50 1000 > 300 5000 > 500 1 > 3000 5 > 5000 10)) > > Figuring out the step size - which on the interval edges depends on > whether you go up or down - can be done like this: > > (defn get-tick-size > [tick-map value go-up?] > (second (first (rsubseq tick-map (if go-up? <= <) value > > Aligning an arbitrary value to the closest valid number (if > necessary): > > (defn align-to-tick-size > [tick-map value] > (let [tick-size (get-tick-size tick-map value false) > remainder (mod value tick-size)] > (if (zero? remainder) > value > (* tick-size ((if (>= remainder (/ tick-size 2)) inc identity) > (quot value tick-size)) > > Now implementing the SpinnerModel is trivial (which is why I will not > copy it here, but leave it onhttp://dueck.org/tick/tick-size-spinner.html > in a simple demo for anyone to try out). > > You're the man, Rich! -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
JSpinner SpinnerModel for numbers with different step sizes - beating Java in its own domain
Anyone who has seen the subject implemented in Java/Swing might have been stunned by the hundreds of lines of code that were deemed necessary for it, with their appropriate share of bugs. (And an implementation I've seen in C#/NetAdvantage was even worse, which was less related to the language itself, as to the GUI toolkit) I knew this could be done way more elegant, but I was surprised it can be done THIS nicely - if done in clojure. And if you think this can be improved further or find any bugs, shoot! Example: Prices for Japanese stock have a step size of 1 in the interval [0,3000], 5 in the interval (3000,5000], ... An easy way to model that would be a sorted map: (def japanese-stock-tick-map (sorted-map 0 1 3000 5 5000 10 3 50 5100 30500 50 1000 300 5000 500 1 3000 5 5000 10)) Figuring out the step size - which on the interval edges depends on whether you go up or down - can be done like this: (defn get-tick-size [tick-map value go-up?] (second (first (rsubseq tick-map (if go-up? <= <) value Aligning an arbitrary value to the closest valid number (if necessary): (defn align-to-tick-size [tick-map value] (let [tick-size (get-tick-size tick-map value false) remainder (mod value tick-size)] (if (zero? remainder) value (* tick-size ((if (>= remainder (/ tick-size 2)) inc identity) (quot value tick-size)) Now implementing the SpinnerModel is trivial (which is why I will not copy it here, but leave it on http://dueck.org/tick/tick-size-spinner.html in a simple demo for anyone to try out). You're the man, Rich! -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: compiling the instructions of a simple vm into a function
Thanks Nicolas, your first variant resembles the generated code much closer than my initial approach, which is great. I need the eval though, to be able to pass in non literals. In my real program I'm reading the instructions from a binary file. So if I want to be able to do something like this: (def three-instructions '([+ 2 3] [- 0 1] [+ 1 0])) (def compiled (compile-instructions three-instructions)) The macro would have to look like this: (defmacro compile-instructions [instructions] (let [memory (gensym "memory-")] `(fn [~memory] ~@(map (fn [[op m1 m2]] `(aset ~memory ~m1 (~op (aget ~memory ~m1) (aget ~memory ~m2 (eval instructions) But I like your suggestion to turn it into a function even better: (defn compile-instructions [instructions] (let [memory (gensym "memory-")] (eval `(fn [~memory] ~@(map (fn [[op m1 m2]] `(aset ~memory ~m1 (~op (aget ~memory ~m1) (aget ~memory ~m2 instructions) And (def compiled (compile-instructions three-instructions))) just works as before. So I guess macros don't add any value here. > eval is evil, but eval is not evil is a compiler (you have to evaluate the > code you read). > However eval is evil again in an "embedded compiler", when you use macro to > extend Clojure. What do you mean by "embedded compiler"? Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
compiling the instructions of a simple vm into a function
Recently, I implemented last year's ICFP problem. Part of it is writing a VM that consists of memory slots, some registers and input and input ports. It takes a list of instructions, works them off one after the other, reading from input or memory, calculating something and the changing registers or writing to memory or output. My implementation worked, but I wanted to make it faster, and thought compiling the instructions into a function using a macro could help. Indeed, my vm got 5 times faster, using arrays and an imperative style (generated by the compiler). But the macro I wrote is butt-ugly, so I thought I might get some advice on this list... To keep it simple I'll use an even simpler VM that only has memory and a simple instruction set. Example of an instruction: [+ 2 3] That means: read memory at address 2, read memory at address 3 and write the sum of both values back to address 2. This is how I use the compiler macro, feeding in 3 instructions: user=> (def compiled (compile-instructions '([+ 2 3] [- 0 1] [+ 1 0]))) #'user/compiled user=> (fn? compiled) true Now let's allocate an int array for the memory, using a size of 4 slots: user=> (def memory (into-array Integer/TYPE (range 4))) #'user/memory user=> (seq memory) (0 1 2 3) And now let's run the instructions on that memory: user=> (compiled memory) 0 user=> (seq memory) (-1 0 5 3) And finally, here comes the code for the macro - but be warned, I'm lacking formal macro education, which is why I'd appreciate any good comment: (defmacro compile-instructions [instructions] (let [memory (gensym "memory-")] (apply list 'fn [memory] (map (fn [[op m1 m2]] (list aset memory m1 (list op (list aget memory m1) (list aget memory m2 (eval instructions) I'm not using the backtick as it seemed simpler for what I was trying to do. I don't know if eval is evil and there's a better alternative? I'm actually not so sure whether it is good to roll out all instructions for performance. It might makes things actually slower, as the JIT can compile less, apart from the fact that a Java method is limited to 64k in size, so the number of instructions I can compile is limited: user=> (def compiled (compile-instructions (take 1150 (cycle '([+ 2 3] [- 0 1] [+ 1 0]) java.lang.ClassFormatError: Invalid method Code length 65581 in class file user$compiled__97 (NO_SOURCE_FILE:25) It blows up at just above 1k instructions. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: ICFP 2009 / ICFP 2010
OK, I'm done for this weekend. Here's what I got:
* the binary reader
* the vm ("interpreted" for now - would like to make a "compiled" vm.
still have to think about it, I guess macros would help?)
* a graphical visualizer
* a controller framework to solve the actual problems
The code is surprisingly modular and readable (if I may say so myself,
I was surprised at least).
All that's missing now is a binary writer to be able to hand in the
solution. But that is pretty straightforward, so I'm not sure if I'll
bother to do it in the next couple of days.
Way more interesting would be to speed this thing up, and the vm is a
crucial part, the instruction processor is iterated over A LOT. So
maybe here's my chance to finally learn macros... Will see, I guess
there are other things that I could do in the vm code to boost
performance, but macros are one thing I haven't really gotten around
to, except for some small and boring ones. Any suggestions on the
issue "compiled vm"?
And thanks guys for the answers to my "get name of fn" problem! Also
good to know is that 1.2 will improve this.
Cheers
Eugen
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: ICFP 2009 / ICFP 2010
So, I'm done for this weekend. What I have (and put on the blog) * a binary parser * the vm (interpreted, maybe I can turn it into a compiled vm in the coming week? any ideas? macros? have to think about it) * a graphical visualizer * a controller "framework", allows you to solve the actual problems, starting from the simplest, the Hohmann transfer What's missing now is only the binary writer, which should be straightforward to do now, maybe I'll take care of that the next couple of days, maybe not. Way more interesting would be to get this thing fast, and I guess compiling the vm should boost performance significantly, and it sounds like an interesting problem, too: Here's my chance to really get into macros (I guess?). But there are certainly other ways to improve performance. Will see. Thanks guys for the input on my "getting the fn name" problem, that's good to know, and great that 1.2 will change it, too. Cheers Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: ICFP 2009 / ICFP 2010
BTW, there will also be a 24h lightning round for those who can't
waste an extended weekend.
On Jun 12, 11:51 am, Eugen Dück wrote:
> I plan on doing ICFP 2010 next weekend in clojure (1.1), if the
> problem is interesting. In preparation, I'm currently doing the
> (implementation-wise) crucial bits of last year's contest.
>
> I put the first part of my implementation
> online:http://read-eval-puke.blogspot.com/2010/06/icfp-2009-orbit-binary-fil...
> so anyone who's interested do have a look.
>
> It would be great to see others participating in this year's contest!
> I found last year's problem really interesting - implementing a vm and
> a binary file parser and then using it to solve the actual problems.
> You can form teams of any size.
> And if I get any good suggestions (you can comment in my blog), I'll
> work them into the code, so it can be used as a cheat sheet in case we
> get similar problems this year.
>
> There's one question that came up when I implemented this. Is there a
> way to get the name of a clojure function, when all I have is the
> function object? Is it stored alongside the function? I didn't see any
> metadata or anything. Would I really have to reverse lookup the name
> in some namespace maps?
>
> The hack I'm currently using is a reverse-lookup map that I manually
> fill with all the functions that I need, which is only a few:
>
> user=> (def fn-to-name { > ">" < "<" = "=" >= ">=" <= "<=" })
> #'user/fn-to-name
> user=> (fn-to-name >)
> ">"
>
> Cheers
> Eugen
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
ICFP 2009 / ICFP 2010
I plan on doing ICFP 2010 next weekend in clojure (1.1), if the
problem is interesting. In preparation, I'm currently doing the
(implementation-wise) crucial bits of last year's contest.
I put the first part of my implementation online:
http://read-eval-puke.blogspot.com/2010/06/icfp-2009-orbit-binary-file-reader.html
so anyone who's interested do have a look.
It would be great to see others participating in this year's contest!
I found last year's problem really interesting - implementing a vm and
a binary file parser and then using it to solve the actual problems.
You can form teams of any size.
And if I get any good suggestions (you can comment in my blog), I'll
work them into the code, so it can be used as a cheat sheet in case we
get similar problems this year.
There's one question that came up when I implemented this. Is there a
way to get the name of a clojure function, when all I have is the
function object? Is it stored alongside the function? I didn't see any
metadata or anything. Would I really have to reverse lookup the name
in some namespace maps?
The hack I'm currently using is a reverse-lookup map that I manually
fill with all the functions that I need, which is only a few:
user=> (def fn-to-name { > ">" < "<" = "=" >= ">=" <= "<=" })
#'user/fn-to-name
user=> (fn-to-name >)
">"
Cheers
Eugen
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: mapping a collection and a "transposed" collection of collections
That's perfect - Thanks! On Jun 7, 11:43 am, ataggart wrote: > (map list x (apply map list y)) > > On Jun 6, 5:51 am, Eugen Dück wrote: > > > Suppose I have two collections: > > > (def x [1 2]) > > (def y [[\a \b] [\d \e] [\f \g]]) > > > And want to iterate over them in the following manner: > > > user=> (map list x (transpose y)) > > ((1 (\a \d \f)) > > (2 (\b \e \g))) > > > Where this is the transpose fn: > > > (defn transpose > > [in] > > (partition (count in) (apply interleave in))) > > > Is there any other nice way to do this without relying on some home- > > grown fn? > > > Cheers > > Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: mapping a collection and a "transposed" collection of collections
P.S. The transpose fn is one of the cases where I'd like to have a more general interleave fn, as lobbied for here: http://groups.google.com/group/clojure/t/c0366933a4333b69 Because with the current version of interleave: user=> (transpose [[1 2]]) java.lang.IllegalArgumentException: Wrong number of args passed to: core$interleave (NO_SOURCE_FILE:0) On Jun 6, 9:51 pm, Eugen Dück wrote: > Suppose I have two collections: > > (def x [1 2]) > (def y [[\a \b] [\d \e] [\f \g]]) > > And want to iterate over them in the following manner: > > user=> (map list x (transpose y)) > ((1 (\a \d \f)) > (2 (\b \e \g))) > > Where this is the transpose fn: > > (defn transpose > [in] > (partition (count in) (apply interleave in))) > > Is there any other nice way to do this without relying on some home- > grown fn? > > Cheers > Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
mapping a collection and a "transposed" collection of collections
Suppose I have two collections: (def x [1 2]) (def y [[\a \b] [\d \e] [\f \g]]) And want to iterate over them in the following manner: user=> (map list x (transpose y)) ((1 (\a \d \f)) (2 (\b \e \g))) Where this is the transpose fn: (defn transpose [in] (partition (count in) (apply interleave in))) Is there any other nice way to do this without relying on some home- grown fn? Cheers Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: (apply interleave [[1 2]])
And we could actually also add an no-arg version. My own version of interleave that I use looks like this: (defn interleav ([] nil) ([c] (seq c)) ([c1 c2] (interleave c1 c2)) ([c1 c2 & colls] (apply interleave c1 c2 colls))) I guess that's as generic as it gets. Does Rich read all threads and will silently ignore/implement this or is there any other good way to get his attention without nagging/ spamming him? I remember suggesting some other enhancement earlier with rather underwhelming success... On Jun 1, 5:12 am, Daniel Werner wrote: > On May 31, 10:07 pm, Daniel Werner > wrote: > > > quantitative logic > > That should have been "quantification logic". -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: (apply interleave [[1 2]])
Paul, I already gave a minimal example of the code it makes simpler, i.e. work in the first place: (apply interleave some-colls) I ran into this a couple of times, and wrote my own variant of interleave that handles the one-coll case. I'd rather see this case handled by interleave. How often do you do: (+ 5) or (* 3) ? But you might have used something like (apply + coll) or (reduce + coll) and under certain circumstances your coll might have had only one element. Still + works just fine and returns a value that makes sense (it even does if you call it with no argument). I'm basically asking to get the same case that clojure handles for a lot of other functions added to "interleave". Eugen On May 29, 7:07 pm, Paul Hobbs wrote: > What code would this make simpler? Are you constantly having to check this > > (apply interleave some-colls) > special case? If not, I don't see a reason to include it. > -- > Paul Hobbs > > On Sat, May 29, 2010 at 1:32 AM, Eugen Dück wrote: > > When I do > > > (apply interleave some-colls) > > > and some-colls is a sequence/collection of only one sequence/ > > collection, it will throw: > > > user=> (apply interleave [[1 2]]) > > java.lang.IllegalArgumentException: Wrong number of args passed to: > > core$interleave (NO_SOURCE_FILE:0) > > > (Of course I don't need the apply to cause that exception, but calling > > interleave directly with just one parameter doesn't make any sense. > > But in the case you use apply, having only one sequence in a sequence > > is a possible corner case that can arise "at run time") > > > In order to make interleave more general, I'd like to add a "one param > > overload" to interleave like > > > (defn interleave > > "Returns a lazy seq of the first item in each coll, then the second > > etc." > > ([c] (seq c)) > > ... > > > or even just > > > (defn interleave > > ([c] c) > > > but that would break the contract of interleave, in that it returns > > whatever you pass in, which might not be a sequence, as is the case in > > my example. > > > Any thoughts on this? > > > Eugen > > > -- > > You received this message because you are subscribed to the Google > > Groups "Clojure" group. > > To post to this group, send email to clojure@googlegroups.com > > Note that posts from new members are moderated - please be patient with > > your first post. > > To unsubscribe from this group, send email to > > clojure+unsubscr...@googlegroups.com > > For more options, visit this group at > >http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
(apply interleave [[1 2]])
When I do (apply interleave some-colls) and some-colls is a sequence/collection of only one sequence/ collection, it will throw: user=> (apply interleave [[1 2]]) java.lang.IllegalArgumentException: Wrong number of args passed to: core$interleave (NO_SOURCE_FILE:0) (Of course I don't need the apply to cause that exception, but calling interleave directly with just one parameter doesn't make any sense. But in the case you use apply, having only one sequence in a sequence is a possible corner case that can arise "at run time") In order to make interleave more general, I'd like to add a "one param overload" to interleave like (defn interleave "Returns a lazy seq of the first item in each coll, then the second etc." ([c] (seq c)) ... or even just (defn interleave ([c] c) but that would break the contract of interleave, in that it returns whatever you pass in, which might not be a sequence, as is the case in my example. Any thoughts on this? Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Can I GPL my Clojure project?
I was always assuming I can GPL my Clojure project, just as I can GPL projects using completely closed compilers and/or runtime environments (?). And the EPL itself is not restrictive when it comes to "non- derived work". But then I read a couple of threads in this group, and also got a response "you can't GPL it" when I announced my Kanji handwriting recognition the other day and mentioned I'm planning to GPL it. So I'm a bit FUDded now. Otoh, Stuart Sierra published his altlaw code under the (Affero) GPL: http://github.com/stuartsierra/altlaw-clojure-restlet/blob/master/LICENSE.txt He's doing law, so he must know what he's doing, right? :) In any case, I'd like to clarify this issue in this group. For me, and for everyone else thinking about GPL'ing their code, for whatever reasons. A couple of questions: 1) Can I GPL my Clojure project? 2) Can I GPL my Clojure project that also uses clojure-contrib? 3) Can I ship clojure.jar (and clojure-contrib.jar) with my GPL app? 4) Can I AOT clojure-contrib.jar (or parts of it) and ship the generated classes with my GPL app? And to all potential license zealots who think it's their time again: This is not a thread discussing what anybody thinks the license of clojure itself (or clojure-contrib) should be, it is about assessing the current situation for projects created in clojure, potentially depending on clojure-contrib. Also please don't try to convince me that GPL is viral and evil and so forth, just as I'm not trying to sell it to anyone here. I want to know if I can or cannot, nothing more, nothing less. Thanks Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
Re: ANN: Kanshiki Boom! - Japanese handwriting recognition
On Mar 27, 1:10 am, Jarkko Oranen wrote: > This looks neat. I probably won't find much use for it though, as my > input method already has this functionality, and even that doesn't get > much use due to the fact that I am horrible at writing kanji with the > mouse (I'm left-handed, but my mouse-hand is right.) I tried to get the recognition to a point where it doesn't matter how bad you type, although there's still room for improvement. It more or less finds the closest match in all Kanjis it knows, however different the Kanji might look form the drawing, but if there is none that is closer, it will be the first choice. So it does not try to force you to draw exact strokes. Still, the general order of strokes and direction will probably always be important, with more common variants to be added. > If you have no plans to try to make money with this thing, I you I'm not thinking of selling this as a product. > suggest simply put the thing on github so that interested people can > take a look. I don't think it's worth keeping things like this hidden. > Note that if you do so you should choose a licence other than GPL, > because it's known to be incompatible with EPL which Clojure uses. I just read through a couple of threads about the licensing issue in this group, and also the EPL and GPL licenses themselves and their FAQs. As I understand the EPL license, creating a modified version of an EPL product means creating derived work, and if derived work is distributed, the source code must be made available, which is exactly what I want. This should prevent others from benefiting from this open source software, without them letting me and others benefit from their changes by not open sourcing it. However, there might be another option, which I'm not 100% sure about: If Clojure was considered a "system library" as per GPL definition, I could release my software as GPL, just as other projects compiled with proprietary compilers are released as open source. I'm not sure whether I can bundle my software with clojure.jar and clojure- contrib.jar in that case though. > (EPL is a slightly weaker copyleft licence, more like LGPL). EPL is > the most common choice, but of course a BSD-style licence is fine as > well. At this point, my software is not a library, at least I'm not currently seeing it as one. That might change, and at that point in time I will decide what choices I want to give to the users of that now-library. For now, I just don't want this to be secretly included somewhere in a commercial/closed-source product - not that it's in that state anyway :), but I don't mind sharing the sources. In case this software actually gets to a state where commercial companies are interested in incorporating that into their own products, I'd like them to either use it open-source (so if they modify, they have to share modifications), or give me enough money so I don't mind them making money from it exclusively, without sharing (at least with me :). Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
ANN: Kanshiki Boom! - Japanese handwriting recognition
I'd like to announce 漢識 Boom!, a handwriting recognition software for Japanese Kanjis. It is available at http://dueck.org/kanshiki-boom/ Currently, it will be useful mainly for learners of the Japanese language, as * it requires the user to use the official stroke order and direction (although I added some variants that are often used by Japanese) * it is able to recognize only the 2000 basic Kanjis (I am using Todd Rubick's unistrok file, with minor changes). It is still beta quality, I guess, but do give it a try and let me know of any issues, preferably via the @-button in the program itself. Why do I announce it here? * It's written in Clojure - which rocks, as we all (except this one guy recently who threatened to unsubscribe from the group if someone responds to his post ;) know very well * If there's enough interest in the algorithm and the implementation of it in Clojure, I might also post/blog about it and GPL the source code And finally, this program works for me, so I am using it as-is and I stopped improving it. But real users (if any) giving feedback would probably motivate me to iron out any remaining issues and to turn this into something that is useful for a broader audience - and I do have a lot of ideas for improvement, just not the motivation to do it right now. Examples: * support kana * detect radicals / let user compose kanjis out of radicals * fine-tune the recognition algorithm * switch to a different data set, so recognition can be better, we get more kanjis and this can actually be useful for advanced and even native Japanese speakers Cheers Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
Re: web starting clojure apps without Java code
I fixed a couple of other issues, most of which show only on Windows. You should see Kanjis now on the right-hand side when drawing. Eugen On Mar 23, 5:30 pm, Zmitro Lapcjonak wrote: > On Mar 17, 4:56 pm, Eugen Dück wrote: > > > The complete jnlp can be found athttp://dueck.org/kanshiki-boom/. > > > I plan to introduce and document this beta-grade app soon, but if > > there's any Japanese learner out there interested in or in need of > > Kanji handwriting recognition, check it out, but please hold back with > > any bug reports etc. until I have introduced it. > > i started and tryed a little the app. > it downloaded (something from your site) then i was able to draw. > i drawed some basic kanji (even those found in unistrok.data) > pressed left button (the right works as "undo"). > but observed no reply from the app. > so, lets wait untill you anonce not-beta state of your app. > > good luck! > > -- > zmitro lapcionak -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
Re: web starting clojure apps without Java code
I just tried starting it from a Windows box, and for some reason, the kanjis are not displayed there. Maybe it's a font issue, not sure yet. Will investigate. On Mar 24, 9:06 pm, Eugen Dück wrote: > Hi Zmitro, > > you should see kanjis being detected as you draw the strokes. Maybe > the app didn't re-layout properly, that's why you didn't see the kanji > panel on the right hand side. I changed the jar slightly, and if the > re-layout thing was the reason, it could be fixed now. if you still > only see a white panel, try resizing the window. But it should re- > layout automatically now. > > And make sure to draw the strokes in the right order and direction. > > Cheers > Eugen > > On Mar 23, 5:30 pm, Zmitro Lapcjonak wrote: > > > On Mar 17, 4:56 pm, Eugen Dück wrote: > > > > The complete jnlp can be found athttp://dueck.org/kanshiki-boom/. > > > > I plan to introduce and document this beta-grade app soon, but if > > > there's any Japanese learner out there interested in or in need of > > > Kanji handwriting recognition, check it out, but please hold back with > > > any bug reports etc. until I have introduced it. > > > i started and tryed a little the app. > > it downloaded (something from your site) then i was able to draw. > > i drawed some basic kanji (even those found in unistrok.data) > > pressed left button (the right works as "undo"). > > but observed no reply from the app. > > so, lets wait untill you anonce not-beta state of your app. > > > good luck! > > > -- > > zmitro lapcionak -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
Re: web starting clojure apps without Java code
Hi Zmitro, you should see kanjis being detected as you draw the strokes. Maybe the app didn't re-layout properly, that's why you didn't see the kanji panel on the right hand side. I changed the jar slightly, and if the re-layout thing was the reason, it could be fixed now. if you still only see a white panel, try resizing the window. But it should re- layout automatically now. And make sure to draw the strokes in the right order and direction. Cheers Eugen On Mar 23, 5:30 pm, Zmitro Lapcjonak wrote: > On Mar 17, 4:56 pm, Eugen Dück wrote: > > > The complete jnlp can be found athttp://dueck.org/kanshiki-boom/. > > > I plan to introduce and document this beta-grade app soon, but if > > there's any Japanese learner out there interested in or in need of > > Kanji handwriting recognition, check it out, but please hold back with > > any bug reports etc. until I have introduced it. > > i started and tryed a little the app. > it downloaded (something from your site) then i was able to draw. > i drawed some basic kanji (even those found in unistrok.data) > pressed left button (the right works as "undo"). > but observed no reply from the app. > so, lets wait untill you anonce not-beta state of your app. > > good luck! > > -- > zmitro lapcionak -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
Re: web starting clojure apps without Java code
That would be great! Please post the link here when you're done. On Mar 18, 5:15 pm, LauJensen wrote: > Eugen, > > Fantastic insight - I cant wait to work that into a blogpost :) > > Lau > > On 17 Mar., 15:56, Eugen Dück wrote: > > > All, > > > Developing in clojure is a lot of fun, at least it was for me and a > > project of mine - except for one thing: Deploying the app as Java Web > > Start app, that took me a bit of time to figure out, and not only > > because Java Web Start is broken in debian squeeze (for a workaround, > > see bugs.debian.org/560056 ). > > > Java Web Start has been discussed in this group some time ago > > (http://groups.google.com/group/clojure/browse_thread/thread/f0c69735c... > > ), and the proposed solution at that time contained one Java class > > that did some static initialization (to propagate the necessary > > permissions to clojure's own classloader) and then went on to call RT > > to load a clj file, after fiddling around with PushBackReaders and so > > forth. > > > I would like to stay away from RT, as it can change, and I don't want > > to depend on RT staying the way it is. Now it turns out that Web Start > > is actually pretty easy if you just AOT your whole app and gen-class > > your main entry point. That way you don't need any Java code. > > > My clj file that contains the entry point starts like this: > > > (ns kanshiki.swing > > (:gen-class)) > > > Then I compile the app and create the jar file: > > mkdir classes > > java -cp clojure.jar:clojure-contrib-slim.jar:classes:. clojure.main - > > e "(compile 'kanshiki.swing)" > > (cd classes; jar cf ../kanshiki-boom.jar *) > > jarsigner kanshiki-boom.jar > > > And the jnlp contains these tags to make it work: > > ... > > > > > > > > > > > > > > ... > > > The complete jnlp can be found athttp://dueck.org/kanshiki-boom/. > > > I plan to introduce and document this beta-grade app soon, but if > > there's any Japanese learner out there interested in or in need of > > Kanji handwriting recognition, check it out, but please hold back with > > any bug reports etc. until I have introduced it. > > > Only one quick note: Kanjis you click will automatically be copied to > > the clipboard, so if you use it together with a kanji dictionary that > > can search the clipboard like kiten (yes, that's the KDE kanji > > dictionary with the hge memory leak, the other day it grew to 6GB > > before I killed it), it is actually useful to look up kanjis or words. > > > Oh, and did I mention lately that clojure is pure fun? Thanks again > > Rich! You've done (and are still doing) a terrific job! > > > Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
web starting clojure apps without Java code
All, Developing in clojure is a lot of fun, at least it was for me and a project of mine - except for one thing: Deploying the app as Java Web Start app, that took me a bit of time to figure out, and not only because Java Web Start is broken in debian squeeze (for a workaround, see bugs.debian.org/560056 ). Java Web Start has been discussed in this group some time ago ( http://groups.google.com/group/clojure/browse_thread/thread/f0c69735c5a9bd03/ ), and the proposed solution at that time contained one Java class that did some static initialization (to propagate the necessary permissions to clojure's own classloader) and then went on to call RT to load a clj file, after fiddling around with PushBackReaders and so forth. I would like to stay away from RT, as it can change, and I don't want to depend on RT staying the way it is. Now it turns out that Web Start is actually pretty easy if you just AOT your whole app and gen-class your main entry point. That way you don't need any Java code. My clj file that contains the entry point starts like this: (ns kanshiki.swing (:gen-class)) Then I compile the app and create the jar file: mkdir classes java -cp clojure.jar:clojure-contrib-slim.jar:classes:. clojure.main - e "(compile 'kanshiki.swing)" (cd classes; jar cf ../kanshiki-boom.jar *) jarsigner kanshiki-boom.jar And the jnlp contains these tags to make it work: ... ... The complete jnlp can be found at http://dueck.org/kanshiki-boom/ . I plan to introduce and document this beta-grade app soon, but if there's any Japanese learner out there interested in or in need of Kanji handwriting recognition, check it out, but please hold back with any bug reports etc. until I have introduced it. Only one quick note: Kanjis you click will automatically be copied to the clipboard, so if you use it together with a kanji dictionary that can search the clipboard like kiten (yes, that's the KDE kanji dictionary with the hge memory leak, the other day it grew to 6GB before I killed it), it is actually useful to look up kanjis or words. Oh, and did I mention lately that clojure is pure fun? Thanks again Rich! You've done (and are still doing) a terrific job! Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: bounded memoize
On Mar 14, 7:59 pm, Christophe Grand wrote: > Well the fn passed to swap! can be retried so in case of "bad luck" you'll > still create several delays. You're absolutely right! I wonder whether we can make any statement about the likeliness of this happening in memoize6/7 vs. memoize7-variant or your memoize8, although it's unlikely to occur in all scenarios so far that I've wanted to use memoize in. But nevertheless, it got me thinking: Does swap! always finish every retry it does? So it won't stop in the middle because it discovered that someone updated the atom in the meantime, correct? I guess It will only be discovered by the final compare-and-set. Answering myself, after checking Atom.java: Yes, that's how it works. So the likeliness of multiple delays being created might actually have increased in the latest memoize versions, as the "transaction" now can take longer, in case it has to do the delay :) As mentioned, this potential difference is probably of minor importance in most cases, more important is to keep the fact that retries can happen in mind... Thanks Christophe, and also Meikel, Michał, Laurent, André for all the input. That was very useful! Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: bounded memoize
On Mar 14, 5:42 pm, Meikel Brandmeyer wrote: > really shows, that concurrent programming is not trivial. Not even for > „trivial“ things like a memoised function. True. It's not too hard to be correct, but being correct and performant at the same time is a different issue... Thinking about your earlier comment: > discussion between Rich and Cliff Click: How to define the > borders of a transaction region?) That's what packing more than one value into a single atom kind of gives us, although in a limited way. This is one of my take-aways from this discussion. But you can't mix and match any arbitrary reference into such a "transaction", like with refs. Cheers Eugen -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: bounded memoize
Hi Christophe,
your fifo-strategy (the one that uses "identity" as the hit method)
does not work:
user=> (def g (memoize7 identity (fifo-strategy 3)))
#'user/g
user=> (g 1)
1
user=> (g 1)
java.lang.IllegalArgumentException: Wrong number of args passed to:
core$identity (NO_SOURCE_FILE:0)
You have to use something like #(first %&) instead of identity.
memoize7 looks fine to me in terms of avoiding extra computation. I'm
not sure though why memoize6 exists as a fn in its own right. I guess
you wouldn't use it without something like delays, as the whole point
of memoize is to avoid multiple costly computations of the same thing,
and memoize6 does not always do that, unless you give it a fn f that
takes care of it, as memoize7 does.
If you don't rely on memoize6, you can write memoize7 even more
compact than memoize6, like this:
(defn memoize7-variant
([f] (memoize7-variant f [{} identity (fn [mem args] mem) assoc]))
([f [init cached hit assoc]]
(let [mem (atom init)]
(fn [& args]
(deref ((cached
(swap! mem (fn [mem]
(if (contains? (cached mem) args)
(hit mem args)
(assoc mem args (delay (apply f args)))
args))
It's more compact, as we do the contains? (or in your original
version: "find") check only once inside the swap! function. This has
the (admittedly not really huge) added benefit that we don't create
multiple delays in the case of "bad luck", as mentioned in my earlier
post.
Any downsides to this?
Cheers
Eugen
On Mar 14, 9:26 am, Christophe Grand wrote:
> Hi Meikel,
>
> On Sat, Mar 13, 2010 at 10:51 PM, Meikel Brandmeyer wrote:
> > On Fri, Mar 12, 2010 at 08:27:15PM +0100, Christophe Grand wrote:
>
> > > See my memoize5: the call isn't computed inside the swap!s
>
> > That doesn't mean, that it is not computed several times!
>
> I agree: it can be concurrently computed several times (but a given thread
> would only compute it at most once).
> I'm sory: I read too quickly and didn't understand that you were concerned
> about the computation happening only once for all threads.
>
> > > Since you use a lock I think some clever combination of memoized
> > functions
> > > can create a deadlock.
>
> > No. In the protected area we simply create a promise and fire off a
> > thread, which does the computation. Then we return immediatelly. So no
> > deadlock possibility here. However we trade one lock for the „other“
> > (the promise). What is the possibility of a deadlock here? Well, the
> > computation of f never completes. But this is not a problem of memoize.
> > The only way memoize could cause a problem here is that the computation
> > of f somehow calls f again with the same arguments. Then it would
> > deadlock on the promise, but without memoize we would also have a
> > infinite loop here...
>
> You're right of course. I apologize.
> As a minor note: couldn't a future replace your promise? Or couldn't you get
> rid of the other thread and deliver the promise in the same thread but
> outside of the (locking ..) form?
> Hmm or even a delay!?
>
> Ok I tried to used delays. So I slightly modified my memoize5 (see memoize6)
> and I built on that a memoize7 which enforces (through delays) that a value
> is onlycomputed once (except if the strategy previously removed it from the
> cache of course).
>
> Do you see a problem in my latest
> implementation?http://gist.github.com/330644#LID153
>
> Thank you,
>
> Christophe
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: bounded memoize
On Mar 14, 11:57 am, Eugen Dück wrote: > This gap shrinks when using memoize7 thanks to the use of delays, but > it is not completely closed and can still lead to multiple delays of > the same computation. If we want to get rid off this gap and make it Actually, I take that back. The delay might be created multiple times, but only one of them will be deref'ed (multiple times). -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: bounded memoize
On Mar 13, 4:51 pm, Christophe Grand wrote:
> My variations on memoize use a single atom: your bounded-memoize id roughly
> equivalent to my memoize2 + fifo-strategy,
> seehttp://gist.github.com/330644#LID19.
I finally found the time to fully read your gist, and I see you are
indeed doing the same there. It's just a little less obvious due to
the indirection introduced by strategies.
> Two quick notes:
> * a PersistentQueue is a better fit there (conj at one end, pop/peek at the
> other) than a vector,
> * by padding it to the desired length (capacity) with dummy items, you are
> sure that the queue (or the vector) is always full, so you always have to
> remove an item: no more need for the (> (count v) capacity) test.
Agree, this simplifies the code a bit, and once we reach full
capacity, the "if" will always be true anyway.
> That's why Meikel tests twice if the value is already in the cache (the
> first time outside of a transactio, the second time inside) and why I
> introduce hit-or-assoc in memoize4/memoize5.
Still, with bad luck or just enough concurrency, simultaneous calls to
memoize6 with the same args can result in multiple computations, as
the computation (apply f args) is done outside of the swap:
(let [ret (if-let [e (find (cached @mem) args)] (val e) (apply f
args))
m (swap! mem hit-or-assoc args ret)]
This gap shrinks when using memoize7 thanks to the use of delays, but
it is not completely closed and can still lead to multiple delays of
the same computation. If we want to get rid off this gap and make it
truely atomic, we have to make the decision whether or not to compute
inside the swap! block.
My memoizer with the hard-coded fifo strategy thus turns into
something like this:
(defn bounded-memoize
[f capacity]
(let [mem (atom [{} []])]
(fn [& args]
(deref ((first
(swap! mem #(if (contains? (first %) args)
%
(let [new-cache (assoc (first %) args (delay
(apply f args)))
new-v (conj (second %) args)]
(if (> (count new-v) capacity)
[(dissoc new-cache (first new-v)) (subvec
new-v 1)]
[new-cache new-v])
args)
Using a padded queue (and getting rid of the anonymous % param names),
it can be (slightly) simplified to:
(defn bounded-memoize
[f capacity]
(let [mem (atom [{} (into clojure.lang.PersistentQueue/EMPTY (repeat
capacity :dummy))])]
(fn [& args]
(deref ((first
(swap! mem (fn [[m q :as cache]]
(if (contains? m args)
cache
[(dissoc (assoc m args (delay (apply f args)))
(peek q))
(-> q pop (conj args))]
args)
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: bounded memoize
Laurent, Meikel, Christophe,
I guess I must be missing something obvious, but can't we just put
more than one thing into an atom in order to get atomic behavior?
Using, say, a vector. Using the simple bounded memoizer as an example,
this looks to me like it works:
(defn bounded-memoize
[f capacity]
(let [mem (atom [{} []])]
(fn [& args]
(if-let [e (find (first @mem) args)]
(val e)
(let [ret (apply f args)]
(swap! mem #(let [cache (assoc (first %) args ret)
v (conj (second %) args)]
(if (> (count v) capacity)
[(dissoc cache (first v)) (subvec v 1)]
[cache v])))
ret)
And if this works, it should be applicable to the strategy-ed
memoizers, too.
Eugen
On Mar 13, 4:27 am, Christophe Grand wrote:
> Hi Meikel,
>
> Since Laurent dragged me into the discussion:
>
>
>
> On Thu, Mar 11, 2010 at 8:42 AM, Meikel Brandmeyer wrote:
> > Hello Laurent,
>
> > On Mar 10, 11:45 am, Laurent PETIT wrote:
>
> > > * usage of refs : I had a bad feeling, and cgrand confirmed this to
> > > me by pointing an even more interesting counter-argument. Me: using
> > > refs is not mandatory since you do not need to synchronize this change
> > > with anything else.
>
> > I don't think, that this is entirely true! You have to syncronise the
> > cache with the state in the strategy. This can be done with atoms only
> > if the cache and the strategy state are contained in the same atom and
> > all updates happen in a single call to swap! (or reset!). As soon as
> > you
> > have multiple calls, you need transactions again, because the atom
> > might
> > change between the two calls. And you can't swap! and return a result
> > at
> > the same time.
>
> > Say you have the LRU strategy. You deref the cache, find the result
> > cached. Now you swap! in the new access time into your state. But
> > between the deref and the swap! another call might trigger the removal
> > of the entry. So you always have to guard against the atom suddenly
> > changing underneath - even if it's only one atom.
>
> I agree.
>
> > Something what annoys me more is that the computation may be fired of
> > several times. Since the f is supposedly expensive, I'd rather avoid
> > that.
>
> See my memoize5: the call isn't computed inside the swap!s
>
>
>
> > I gave it another run and came up with another solution. Even more
> > hairy. :P
>
> > * a warm cache is fast
> > * a cache miss is potentially slow, but the missing value is computed
> > only
> > once, multiple requests for the same item wait for the initially
> > triggered
> > computation
> > * the cache can be updated (almost) in parallel for different items
>
> > (defn memoize
> > ([f] (memoize f naive-strategy))
> > ([f [cache-lookup cache-update]]
> > (let [cache-state (atom {})]
> > (fn [& args]
> > (if-let [e (cache-lookup cache-state args)]
> > @(val e)
> > @(locking cache-state
> > (if-let [e (cache-lookup cache-state args)]
> > (val e)
> > (let [result (promise)]
> > (.start (Thread. #(deliver result (apply f args
> > (swap! cache-state assoc-in [:cache args] result)
> > (cache-update cache-state args)
> > result
>
> > But with this you can't reliably implement strategies like LRU and the
> > like.
>
> Since you use a lock I think some clever combination of memoized functions
> can create a deadlock.
>
> > But this is really an interesting problem. I'll mull a little more
> > about it. :)
>
> I agree, it's interesting and here is my entry:http://gist.github.com/330644
>
> Christophe
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: bounded memoize
I totally agree, having different eviction strategies would be great
and having memoizers with max capacities in clojure.contrib would
probably be useful to a lot of developers.
Also, a memory sensitive memoizer (using soft references?) would be
nice. :)
On Mar 9, 4:51 pm, Meikel Brandmeyer wrote:
> Hi,
>
> On Mar 9, 4:41 am, Eugen Dück wrote:
>
>
>
> > Good points! Testing array-map briefly led me to believe they can be
> > used as the clojure equivalent of Java\s LinkedHashMaps.
>
> > Here's a version that uses a vector to remember order of insertion - I
> > guess I have to use refs and transactions now:
>
> > (defn bounded-memoize
> > [f bound]
> > (let [mem (ref {})
> > v (ref [])]
> > (fn [& args]
> > (if-let [e (find @mem args)]
> > (val e)
> > (let [ret (apply f args)]
> > (dosync
> > (when (= (count @v) bound)
> > (alter mem dissoc (first @v))
> > (alter v subvec 1))
> > (alter mem assoc args ret)
> > (alter v conj args))
> > ret)
>
> > Haven't looked at clojure's queues yet, they might make the code more
> > concise, but by looking at that other post, they don't seem to be
> > exposed in a clojurey way (using a java class name).
>
> How about generalising memoize to allow different strategies?
>
> (defn bound-cache-strategy
> "Implements a bound cache strategy for memoize. At most bound number
> of
> argument lists are kept in the cache. They are dropped in order of
> insertion."
> [bound]
> [find
> (let [values (ref clojure.lang.PersistentQueue/EMPTY)]
> (fn [cache args]
> (alter values conj args)
> (if (> (count @values) bound)
> (do
> (alter values pop)
> (dissoc cache args))
> cache)))])
>
> (defn lru-cache-strategy
> "Implements LRU cache strategy for memoize. At most bound number of
> argument lists are kept in the cache. They are dropped in LRU
> order."
> [bound]
> (let [values (ref {})]
> [(fn lru-cache-strategy-lookup
> [cache args]
> (when-let [e (find cache args)]
> (let [now (System/currentTimeMillis)]
> (dosync (alter values assoc args now))
> e)))
> (fn lru-cache-strategy-update
> [cache args]
> (let [now (System/currentTimeMillis)
> k (min-key @values (keys @values))]
> (alter values dissoc k)
> (alter values assoc args now)
> (dissoc cache k)))]))
>
> (defn most-used-cache-strategy
> "Implements MU cache strategy for memoize. At most bound number of
> argument lists are kept in the cache. They are dropped in LU order.
> In case elements have the same usage count, the order of drop is
> unspecified."
> [bound]
> (let [values (ref {})]
> [(fn most-used-cache-strategy-lookup
> [cache args]
> (when-let [e (find cache args)]
> (dosync (alter values update-in [args] inc))
> e))
> (fn most-used-cache-strategy-update
> [cache args]
> (let [k (min-key @values (keys @values))]
> (alter values dissoc k)
> (alter values assoc args 1)
> (dissoc cache k)))]))
>
> (defn memoize
> "Returns a memoized version of a referentially transparent function.
> The
> memoized version of the function keeps a cache of the mapping from
> arguments
> to results and, when calls with the same arguments are repeated
> often, has
> higher performance at the expense of higher memory use.
>
> Optionally a cache strategy might be supplied. A strategy is pair of
> functions:
> - one for accessing the cache, returns the map entry on success or
> nil
> (cf. find)
> - one, which takes the cache and the argument vector and might
> modify
> the cache.
> Possible implementation could be a bounded cache or a LRU strategy.
> Default is a naive strategy keeping all encountered argument lists
> forever.
> The cache update function is called in a transaction, the cache
> lookup
> function not necessarily so."
> ([f] (memoize f [find (fn [c _] c)]))
> ([f [cache-lookup cache-update]]
> (let [cache (ref {})]
> (fn [& args]
> (if-let [e (cache-lookup @cache args)]
> (val e)
> (dosync
> (if-let [e (cache-lookup @cache args)]
> (val e)
> (let [result (apply f args)]
> (alter cache assoc args result)
&
Re: bounded memoize
Good points! Testing array-map briefly led me to believe they can be
used as the clojure equivalent of Java\s LinkedHashMaps.
Here's a version that uses a vector to remember order of insertion - I
guess I have to use refs and transactions now:
(defn bounded-memoize
[f bound]
(let [mem (ref {})
v (ref [])]
(fn [& args]
(if-let [e (find @mem args)]
(val e)
(let [ret (apply f args)]
(dosync
(when (= (count @v) bound)
(alter mem dissoc (first @v))
(alter v subvec 1))
(alter mem assoc args ret)
(alter v conj args))
ret)
Haven't looked at clojure's queues yet, they might make the code more
concise, but by looking at that other post, they don't seem to be
exposed in a clojurey way (using a java class name).
On Mar 9, 11:22 am, Michał Marczyk wrote:
> On 8 March 2010 05:31, Eugen Dück wrote:
>
> > And here's a variant that evicts elements when the size of the cache
> > exceeds some limit. In that case, the first item that was put in it
> > will be dissoc'ed. I'm using an array-map to accomplish this:
>
> I don't think this will work as you expect it to. There are two reasons:
>
> (1) Array maps are transparently converted to hash maps when they grow
> beyond a certain size:
>
> user> (class (array-map :a 1 :b 2 :c 3 :d 4 :e 5 :f 6 :g 7 :h 8))
> clojure.lang.PersistentArrayMap
> user> (class (assoc (array-map :a 1 :b 2 :c 3 :d 4 :e 5 :f 6 :g 7 :h 8) :i 9))
> clojure.lang.PersistentArrayMap
> user> (class (assoc (array-map :a 1 :b 2 :c 3 :d 4 :e 5 :f 6 :g 7 :h
> 8) :i 9 :j 10))
> clojure.lang.PersistentHashMap
>
> (2) More importantly, if you dissoc the first key from an array map,
> then assoc a new key onto the array map, the newly assoc key will take
> the first slot:
>
> user> (first (assoc (dissoc (array-map :a 1 :b 2 :c 3) :a) :d 4))
> [:d 4]
>
> I'd suggest a vector instead; they're countable in constant time and
> you can use, say, conj and rest for add to end of queue / eject from
> front of queue.
>
> The idea is certainly a good one, though, so with the above mentioned
> issues fixed, this will be a nice utility function. Thanks for
> sharing!
>
> Sincerely,
> Michał
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
bounded memoize
In many real applications (I guess that rules out fibonacci), memoize will consume a lot of memory and create OutOfMemoryErrors sooner or later. Often you want to only keep the latest items in the cache and forget about older ones. I've seen a variant of memoize that evicts items based on a time-to- live bound in this group (http://groups.google.com/group/clojure/ browse_thread/thread/402c489f805ade94/ ). And here's a variant that evicts elements when the size of the cache exceeds some limit. In that case, the first item that was put in it will be dissoc'ed. I'm using an array-map to accomplish this: (defn bounded-memoize [f bound] (let [mem (atom (array-map))] (fn [& args] (if-let [e (find @mem args)] (val e) (let [ret (apply f args)] (swap! mem #(let [new-mem (assoc % args ret)] (if (> (count new-mem) bound) (dissoc new-mem (first (first new-mem))) new-mem))) ret) -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: generalize distinct
Agree, it might make sense to do that performance-wise. Although I'm hoping that the compiler would be able to inline that as well (optimizing away calls to identity). Btw, what does :inline-arities do? It sounds like this could do what we want. On Feb 24, 5:03 am, Sean Devlin wrote: > I guess I was getting at how to do the airity overloading. My first > instinct when developing is do something like this > > (def distinct > ([coll] (distinct identity coll)) > ([f coll] (lot-of-code))) > > However, if you look around core, you'll see that Rich "repeats > himself" a lot. This really obvious w/ fns like juxt & partial. It > turns out that this is results in faster code. So I'd implement > distinct like this. > > (def distinct > ([coll] (lots-of-code)) > ([f coll] (lot-of-similar-code))) > > That's all I was saying. > > Sean > > On Feb 23, 2:40 pm, Michał Marczyk wrote: > > > On 23 February 2010 20:26, Sean Devlin wrote: > > > > Take a look at sort/sort-by in core. That would be starting point for > > > fn style. > > > Not sure what you mean by that. > > > If anything, having an arity-overloaded distinct (optionally accepting > > a binary predicate to use in place of =) and a separate distinct-by > > (accepting an extra keyfn argument plus optionally the binary > > predicate) would make for a more uniform standard lib. > > > (I guess I suggested switching to the Haskell naming convention > > before, but there's no real reason to do that if the same > > functionality is available, plus I'd be confused by this myself, > > having gotten used to the Clojure names already... I retract that > > part.) > > > Sincerely, > > Michał -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: generalize distinct
I agree with you, Michal. But let me rephrase the question, maybe my initial long-winded post wasn't clear enough on that. Rather than having a separate fn 'distinct-by' in addition to the existing 'distinct', which, apart from the hard-coded keyfn would be EXACTLY the same, shouldn't we just generalize distinct to default to hard-coded if no keyfn is specified? (Btw I'm not considering my other suggestion to allow the set to be passed in here, which I personally like, but also understand can be seen as unorthogonal) And this is not only true for distinct-by, it's true for some of the other * / *-by pairs. It just seems this should be generalized in order not to duplicate code for these cases. If we copy-and-paste code, what's the justification? I'd say orthogonality is an argument against copy-and-paste. Do we copy and paste rather than generalize in order to have distinct a little bit faster due to it being hard-coded? Eugen On Feb 23, 5:13 am, Michał Marczyk wrote: > On 22 February 2010 20:28, Sean Devlin wrote: > > > Then is the seq (1 :a a) guaranteed? How do I know that I won't get > > (2 :b b), (1 :b c), etc? What if I want a specific combination > > instead? I've had to actually code this specific problem, and I found > > that using group-by & some secondary mapping operation was the only > > thing that gave me the flexibility I needed (manufacturing is fun!). > > The ordering guarantees distinct-by makes are exactly those that > distinct makes, because it uses the same code (as mentioned > previously, I lifted it all from clojure.core, then tweaked to take > the keyfn / eqfn into account). Basically this means that if your > collection has an intrinsic ordering, it will be preserved (the result > will include, for each equivalence class of items from the sequence > modulo the user-defined equivalence relation, the one earliest w.r.t. > that ordering). If it's a hash-map or a hash-set instead, you'll get > whatever ordering (seq coll) happens to produce. > > As for group-by giving you more flexibility -- well, it gives you a > lot of flexibility where it's appropriate to use it, but because of > its choice of data structure for the result, you can't use it to > reimplement distinct-by directly: > > user=> (group-by class [1 2 3 :a :b :c 'a 'b 'c]) > java.lang.ClassCastException: java.lang.Class cannot be cast to > java.lang.Comparable (NO_SOURCE_FILE:0) > > So no way to use non-Comparables as keys... > > And then there's the fact that you can't tell in which order the keys > discovered by group-by appeared in the original collection, which is > again because of its use of sorted-map, which has the consequence that > order is being mangled on purpose! E.g.: > > user=> (seq (group-by #(- %) [1 2 3 4 5])) > ([-5 [5]] [-4 [4]] [-3 [3]] [-2 [2]] [-1 [1]]) > > In other words: (seq (group-by f coll)) has an ordering possibly > completely unrelated to that of coll (so you'd have to make a separate > traversal through the coll to discover the original ordering of the > keys), whereas (distinct-by f coll), for either version of > distinct-by, preserves the ordering of coll. That's a desirable > property for when that's what you want to do, whereas group-by will, I > suppose, be more useful on other occasions. ;-) > > To sum it up, (1) distinct-by actually behaves in a very predictable > way (which may or may not be useful for any particular purpose), (2) > it cannot be implemented directly in terms of group-by. I'd say it's > pretty orthogonal to the existing library functions (that I know of) > actually... > > Sincerely, > Michał -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: generalize distinct
group-by does not filter, in contrast to distinct.
And I'm really only interested in one of the values considered equal
according to my keyfn. So if I sed group-by, I'd have to wrap it with
a call to map or similar. This is certainly an option, but I'd rather
not create all the intermediate vectors if I don't need to.
On Feb 22, 11:13 pm, Sean Devlin wrote:
> It sounds to me that you want to use c.c.seq-utils/group-by, not
> distinct.
>
> On Feb 22, 7:22 am, Eugen Dueck wrote:
>
> > Yes, for the reasons stated.
>
> > First, and most important, clojure.core/distinct does not let me pass
> > in a keyfn, it's hard-coded.
>
> > Second, and as mentioned that's debatable, distinct could make more of
> > the filtering functionality it already has available to the caller,
> > for free.
>
> > On Feb 22, 11:53 am, Wilson MacGyver wrote:
>
> > > Any reason why you can't use
> > > distinct?
>
> > >http://richhickey.github.com/clojure/clojure.core-api.html#clojure.co...
>
> > > On Feb 21, 2010 10:24 AM, "Eugen Dueck" wrote:
>
> > > Hi,
>
> > > Clojure is great! The gain in productivity from more low level
> > > languages like Java, but also more functional languages like Ruby and
> > > Common LISP etc. amazes me every day. Like how adding a simple "map"
> > > in front of the count here:
> > > (count colls)
> > > changes the code from counting the number of collections to
> > > enumerating the number of items in those collections. It's all these
> > > little things that take one minute or 5 in Java, but only 5 seconds in
> > > Clojure (in case you are a slow typer) that make you so much more
> > > efficient. With clojure, I spend most of my time thinking about
> > > problems at a conceptual level, rather than at the implementation
> > > level (Java: int[] vs. Iterable vs. List vs.
> > > Integer[] etc. - arrg!). Thanks Rich!
>
> > > That said, every now and then I come across a function in clojure.core
> > > that could be vastly improved in its range of applicability by just
> > > adding one or more optional parameters, defaulting to the currently
> > > hard-coded values (like the often implicit 'identity' function). Take
> > > 'distinct'. I'd like to be able to specify the keyfn, that's an
> > > important one for me, and while we're at it, I'd like to pass in the
> > > set that that function builds up incrementally, and normally starts
> > > with an empty set, so that I can pre-initialize it, say with #{ nil },
> > > so that it also filters out nils.
>
> > > This 2nd point is not that important, and I'm not sure if it is that
> > > great an idea in terms of "orthogonality" of the functions, as we have
> > > filter. But you'd get rid off one level of indirection. distinct
> > > basically gives us a filter for free.
>
> > > But the 1st point I think certainly makes sense, and we have a couple
> > > of other fns in clojure that have a variant with a keyfn parameter,
> > > like sort-by etc. I guess they normally get a different name.
>
> > > I'm not so sure about what the best order of parameters is in clojure,
> > > and named parameters would make this a no brainer, but this is what I
> > > currently use, the first parameter being coll, at least in the variant
> > > with only one parameter that makes it a drop-in replacement candidate
> > > for distinct:
>
> > > (defn distinkt
> > > "Returns a lazy sequence of the elements of coll with duplicates
> > > removed"
> > > ([coll] (distinkt coll identity))
> > > ([coll keyfn] (distinkt coll keyfn #{}))
> > > ([coll keyfn seen-items]
> > > (let [step (fn step [xs seen]
> > > (lazy-seq
> > > ((fn [[f :as xs] seen]
> > > (when-let [s (seq xs)]
> > > (let [key (keyfn f)]
> > > (if (contains? seen key)
> > > (recur (rest s) seen)
> > > (cons f (step (rest s) (conj seen
> > > key)))
> > > xs seen)))]
> > > (step coll seen-items
>
> > > I don't mind writing - or better - copy-and-pasting this code and
> > > keeping it in my project, but I think it could be useful for others,
> > > so I wouldn't mind at all if this makes it into clojure.core... :)
>
> > > Or is the reason for hard coding an (implicit) 'identity' performance?
> > > Or did I miss some other way to achieve the same goal?
>
> > > Eugen
>
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "Clojure" group.
> > > To post to this group, send email to clojure@googlegroups.com
> > > Note that posts from new members are moderated - please be patient with
> > > your
> > > first post.
> > > To unsubscribe from this group, send email to
> > > clojure+unsubscr...@googlegroups.com
> > > For more options, visit this group
> > > athttp://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post
