[tools.logging] Adding Null Logger Impl and Forcing It For Testing
Hi, I Midje test functions that log. I want to disable logging during testing. What's the easiest way to do that? I saw a message by Sean Corfield from a year and a half ago about altering the *logger-factory* root to ensure a particular logger implementation is chosen project wide. Obviously, this would conflict with the testing goal. I'd be happy to open a JIRA ticket and work on a patch. Thoughts? Thanks, Alyssa -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
gen-class for methods returning own class for chained calls
Hi everyone! I'm having trouble getting the following to compile from lein compile: (ns foo.Bar (:gen-class :methods [[chain [] foo.Bar]])) (defn -chain [this] this) My project.clj has foo.Bar declared as a :namespace. Perhaps I'm not understanding the compile error. Is there a way to declare that a method of a gen-class returns an instance of itself? Thanks! Alyssa -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: gen-class for methods returning own class for chained calls
N/m, this is a known issue. http://dev.clojure.org/jira/browse/CLJ-84 On Jan 8, 11:23 am, Alyssa Kwan alyssa.c.k...@gmail.com wrote: Hi everyone! I'm having trouble getting the following to compile from lein compile: (ns foo.Bar (:gen-class :methods [[chain [] foo.Bar]])) (defn -chain [this] this) My project.clj has foo.Bar declared as a :namespace. Perhaps I'm not understanding the compile error. Is there a way to declare that a method of a gen-class returns an instance of itself? Thanks! Alyssa -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: an object of class created using defrecord does not implement IFn .. while it behaves very similar to map otherwise ..
Generating readable code for IDEs is not a good reason. You should think carefully about variable capture and decide which you want. Usually, in a macro-generated defn, I do want to capture the parameters, so I would use ~'this. On Dec 30, 11:54 pm, André Thieme splendidl...@googlemail.com wrote: Am 31.12.2010 03:29, schrieb Alex Baranosky: I've been playing with making a macro to encapsulate Stuart's post, like this: (defmacro defrecord-ifn [name args] `(defrecord ~name ~...@args clojure.lang.IFn (invoke [this key] (get this key (defrecord-ifn Foo [a b c]) (def foo (Foo. A B C)) (prn (map foo [:a :c])) = (A, C) I get the error: No such var: user/this. I guess this is because it is expanding 'this' to 'user/this'. What is the proper way to get a macro like this to expand properly? Others have already pointed to this# . I just would like to add that you can as well use ~'this in some cases, where your macros generate defns. The advantage is that some editors (like emacs) will show you the parameter vector and that would show a useful name and not this_auto_foobarbaz123456 . -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Boston meetup Jan 11?
Hi! You would be more than welcome at the Boston Coding Dojo (http:// www.meetup.com/boston-coding-dojo/). We meet every other Thursday at First Church in Boston, in Boston's Back Bay on Marlborough St. In January, we are meeting on 1/6 and 1/20, so nothing on the week of 1/9, I'm afraid. What kind of meeting do you have in mind? I could certainly recommend some restaurants that are more conducive to conversation. If you are looking for a space to hack, I can certainly check with the church for availability; we meet in a lovely chapel with plenty of room. It would cost $75 for the space for the night. There's also the Workbar (http://www.workbarboston.com), which is pricier and a little more cramped... :). Everyone on the list should also check out Boston Software Craftsmanship (http://groups.google.com/group/boston-software- craftsmanship/). Their next meeting is on 1/24 and is on monads (http://gathers.us/events/jan-boston-software-craftsmanship-meeting). Thanks! Alyssa On Dec 30, 1:52 pm, dysinger t...@dysinger.net wrote: 10 of us from Sonian are going to converge on Boston the week of Jan 9. It's always awesome to meet other Clojure hackers. I propose we meet up somewhere central(ish) Jan 11 @ 7-9pm-ish. We have room at our company headquarters in Dedham but that might be a hike for some people. Any other places we could meet/greet hack for an hour or two central to Boston? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: an object of class created using defrecord does not implement IFn .. while it behaves very similar to map otherwise ..
Technically, there's no possibility of variable capture, but you should use gensyms: (defmacro defrecord-ifn [name args] `(defrecord ~name ~...@args clojure.lang.IFn (invoke [this# key#] (get this# key# Thanks, Alyssa On Dec 30, 9:29 pm, Alex Baranosky alexander.barano...@gmail.com wrote: I've been playing with making a macro to encapsulate Stuart's post, like this: (defmacro defrecord-ifn [name args] `(defrecord ~name ~...@args clojure.lang.IFn (invoke [this key] (get this key (defrecord-ifn Foo [a b c]) (def foo (Foo. A B C)) (prn (map foo [:a :c])) = (A, C) I get the error: No such var: user/this. I guess this is because it is expanding 'this' to 'user/this'. What is the proper way to get a macro like this to expand properly? Alex -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: dispatching on a resulting range
I completely disagree. If arbitrary load order were sufficient, there wouldn't be (prefer-method). (And CL wouldn't have a complex heuristic for ordering.) In reality, you may be extending someone else's library by calling (defmethod) on their (defmulti). And you could be using someone else's library that defines their own (defmethod)'s. You can't group stuff that's not yours. That being said, replacing an earlier method of the same priority falls in the same trap. It's dependent on load order, just in a way that has less potential for conflict because the footprint is smaller. I don't have an answer. To a certain extent, I question the usefulness. Dispatching belongs on the dispatching function, and I can't think of an example of range dispatching where the programmer doing the dispatching isn't also writing the method that is dispatched to (rendering my entire objection moot). They're certainly tightly coupled in a way that regular multimethods are not. Thanks, Alyssa On Dec 23, 1:37 pm, Jay Fields j...@jayfields.com wrote: 2010/12/23 Ken Wesson kwess...@gmail.com It most certainly is not. Yes, it is. Unlike cond clauses, methods might be scattered in different parts of a large code base _might_ be. But they don't need to be. I'd rather group my methods, know what I'm doing, and have to configure less. Anything that requires additional and unnecessary configuration is less desirable to me. And of course my code added a couple of other nice features as well. That's not really relevant, is it? My point was, and is, there's no reason to call anyone's code sucky. Especially when there's no correct answer. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Mocking multimethods
I'd like to discuss this design approach. (It's unrelated to the
testing issue.) I avoided this design because the undo-fn is
determined at do-patch! time. The use case is for a persistent system
like Git where patches may be undone long after being done - e.g. long
after the patch is written to the database. In the meantime, the
system may be upgraded and the undo-fn for a patch may change. If the
undo-fn and args is written to the database at do-patch! time, for
backwards compatibility, at undo-patch! time, the undo-fn would have
to be updated to whatever the new call is.
Contrast that with my current approach where only the fn is saved and
the undo-fn is determined at undo-patch! time. For backwards
compatibility, the undo-fn would have to support legacy fn formats,
which I conjecture are slightly less likely to change than the
corresponding undo-fn. This is a pure guess and is not based on real
requirements yet; I don't know that fns themselves are necessarily
more stable than their counterpart undo-fns. It's a vague sense that
undo-fns are more subject to bug fixes: fns themselves are processed
right away at do-patch! time, while undo-fns are like time bombs that
lay dormant until a patch has to be undone, long after the initial
patch.
In addition, in the current implementation I outlined, the undone
patch disappears. The real implementation will most likely be a patch
with undo-patch! as the :fn and the patch being undone as the :arg.
This is necessary to support merging with remote branches like in
Git. undo-patch!'s may themselves be undone, which would require 2
degrees of backwards compatibility: converting the undo-fn in the
first undo, and converting the fn in the second undo. Both approaches
have the same 2 degrees of backwards compatibility, but the command
approach still somehow seems more fragile.
I'd love to get your thoughts on this. Anyone with experience working
on a similar system?
Thanks!
Alyssa
On Dec 22, 4:35 am, Meikel Brandmeyer m...@kotka.de wrote:
Hi,
maybe a different approach could be to use a richer datatype than a function,
which carries both: the command and the undo command.
(deftype Command [action undo])
Then you could do something like:
(defn do-patch!
[command args]
(dosync
(let [patch {:command command :args (vec args)}]
(apply (.action command) args)
(alter patches- conj patch)
patch)))
(defn undo-patch!
[]
(dosync
((.undo (:command patch)))
(alter patches- pop)))
Then you could provide specially crafted Commands and there would not be the
need to stub anything. Also Commands wouldn't have to be global objects.
Sincerely
Meikel
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Mocking multimethods
The issue is where do I specify that: (undo-fn ...patch...) = (fn [] (reset! visible-evidence-of-a-side- effect :happened!)) undo-fn is a multimethod in my design, which requires a corresponding defmethod for each patch type. I need to create one for the scope of the test, but defmethod by definition alters the top-level state of the system, and doesn't automatically unroll like let or binding does. On Dec 22, 6:33 am, Brian Marick mar...@exampler.com wrote: I think I misunderstand the issue, because this works for me: (ns midje.util.git (:use [midje.sweet])) ;; Code except for undo-patch is the same as before. ;; ... ;; I pulled remove-patch out of undo-patch because I was ;; getting a screwy read error I didn't want to figure out ;; at 5 in the morning, but I'd probably do that anyway: ;;http://codebetter.com/jeremymiller/2006/12/03/composed-method-pattern/ (defn remove-patch [patch] (alter patches- #(remove (fn [p] (= patch p)) %))) (defn undo-patch [patch] (let [fn (undo-fn patch)] (dosync (fn) (remove-patch patch (fact The patch's undo-fn is called for its side effect and the patch is forgotten (let [visible-evidence-of-a-side-effect (atom nil)] (undo-patch ...patch...) = anything (provided (undo-fn ...patch...) = (fn [] (reset! visible-evidence-of-a-side-effect :happened!)) (remove-patch ...patch...) = :nothing-of-interest) @visible-evidence-of-a-side-effect = :happened!)) 1855 $ lein midje midje.util.git All claimed facts (2) have been confirmed. 1856 $ - Brian Marick, Artisanal Labrador Contract programming in Ruby and Clojure Author of /Ring/ (forthcoming; sample:http://bit.ly/hfdf9T)www.exampler.com,www.exampler.com/blog,www.twitter.com/marick -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Mocking multimethods
Thanks, Brian! I obviously didn't understand the nature of the provided form. That's really cool notation! This is exactly what I want. What are the cons of using midje? Any reason I shouldn't migrate all my unit testing to it? Thanks! Alyssa On Dec 22, 9:46 am, Brian Marick mar...@exampler.com wrote: On Dec 22, 2010, at 6:52 AM, Alyssa Kwan wrote: The issue is where do I specify that: (undo-fn ...patch...) = (fn [] (reset! visible-evidence-of-a-side- effect :happened!)) The code you quoted is that specification. It doesn't matter that undo-fn is a multimethod. Here's what the notation of the test says: When called with an arbitrary patch, undo-patch will produce a particular side effect. It does that because it uses undo-patch, which--when given that arbitrary patch--returns a function that produces that side effect. It also calls remove-patch with the given patch. undo-patch can return anything it wants. We don't care. (fact The patch's undo-fn is called for its side effect and the patch is forgotten (let [visible-evidence-of-a-side-effect (atom nil)] (undo-patch ...patch...) = anything (provided (undo-fn ...patch...) = (fn [] (reset! visible-evidence-of-a-side-effect :happened!)) (remove-patch ...patch...) = :nothing-of-interest) �...@visible-evidence-of-a-side-effect = :happened!)) - Brian Marick, Artisanal Labrador Contract programming in Ruby and Clojure Author of /Ring/ (forthcoming; sample:http://bit.ly/hfdf9T)www.exampler.com,www.exampler.com/blog,www.twitter.com/marick -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Automatically unmapping unit tests from namespaces
Hi everyone, My typical development workflow is to use leiningen to create a project stub, modify project.clj to add swank-clojure as a dev- dependency, and run lein-swank and connect from Emacs slime. As I create and modify files in the test and src namespaces/directory structures, I use C-c C-k to call slime-compile-and-load on the buffer that I'm currently in. And I use C-c M-p to switch to the test namespace I'm working on so that in the slime REPL I can call (run- tests). So far so good. What about when I need to delete a unit test? Reloading the test buffer doesn't remove it, and I need to either restart swank or reconnect slime, or manually remove those tests using (unmap-ns). Surely there's a better way... Thanks! Alyssa -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Automatically unmapping unit tests from namespaces
Awesome!!! This absolutely does the trick! On Dec 21, 7:16 pm, Phil Hagelberg p...@hagelb.org wrote: On Dec 21, 10:35 am, Alyssa Kwan alyssa.c.k...@gmail.com wrote: What about when I need to delete a unit test? Reloading the test buffer doesn't remove it, and I need to either restart swank or reconnect slime, or manually remove those tests using (unmap-ns). Surely there's a better way... If you're already using swank then you can try clojure-test-mode; it clears out all deftests in between test runs. https://github.com/technomancy/clojure-mode/blob/master/clojure-test-... It also highlights failures in the test buffer for better feedback. -Phil -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Mocking multimethods
Hi everyone,
Does anyone have any experience in mocking multimethods? I'm working
on a version control framework modeled after Git:
(def
^{:private true}
patches- (ref [])
(defn patches []
(seq @patches-))
(defn do-patch! [fn args]
(dosync
(apply fn args)
(let [patch {:fn fn
:args (vec args)}]
(alter patches- conj patch)
patch)))
I need to be able to undo recorded patches, which means at undo-time,
translating the recorded do-patch! fn and args into an undo-fn which
can be called. If an undo-fn cannot be found, it throws an
exception. This has to be extensible so that as I add operations, I
can add the counterpart undo-fn. Multimethods to the rescue!
(defmulti undo-fn :fn)
(defn undo-patch [patch]
(let [fn (undo-fn patch)]
(dosync
(fn)
(alter patches- #(remove #(= patch %) %)
So in another package, I add the operation create-table! to the
system, which has a counterpart drop-table!:
(defn create-table!- [name columns]
(... create table stuff ...))
(defn drop-table!- [name]
(... drop table stuff ...))
(defn create-table! [name columns]
(make-patch! create-table!- name columns))
(defmethod undo-fn create-table!- [patch]
#(drop-table!- (:name patch)))
So calling undo-patch! with a patch that was the result of create-
table! will call undo-fn, which will dispatch on the :fn member of the
patch, which is create-table!-, which will end up returning an
anonymous function that calls drop-table!- with the :name from
the :args member of the patch.
OK, this all works.
But from a unit testing perspective, how do I cleanly test undo-patch!
without any undo-fn methods declared? When testing, I just want to
know that undo-patch! attempts to resolve undo-fn, will throw an error
if it can't, and will call the anonymous function that undo-fn returns
if it can. In other words, how do I mock undo-fn?
Thanks,
Alyssa
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Ah-hah! Clojure is a Lisp
No, identifiers are names. Identity transcends names. For example, in a distributed shared object system, multiple machines on the same network will have different identifiers for the same identity. Ordinary usage isn't good enough for metaphysical discussions. There is a metaphysical discussion of identity which applies to this situation, and Rich Hickey has taken a particular position. His position is rigorous, internally consistent, and applicable to how most people in our culture model the world, e.g. it can be used to accomplish work by most of us. Most philosophical discussions of identity really mean equality, at least in Hickey nomenclature. On Dec 20, 12:56 pm, Ken Wesson kwess...@gmail.com wrote: On Mon, Dec 20, 2010 at 2:49 AM, Alex Osborne a...@meshy.org wrote: Ken Wesson kwess...@gmail.com writes: Ah. So, like the confused situations you get with Java's mutable collections. Two lists are equal if they have the same contents in the same order -- but then you use one as a key in a hashmap, and then add an item to it, and boom! Clojure separates this stuff out because the Clojure vector's immutability makes its value stable given its identity. Refs and atoms and agents can encapsulate mutable state, but their identity (as defined by = and hash) is fixed rather than changing with its state. Sort of. Identity (in the Clojure model) is not the same concept as equality. Nor is it reference equality (identical?). The overloading of terminology is somewhat unfortunate. By identity I mean a stable logical entity associated with a series of different values over time. -- clojure.org/state As Laurent mentioned the usual identities in Clojure are reference objects: vars, atoms, refs and so on. And some objects (keywords and symbols) exist to be almost pure identity, used to label other things. Symbols and keywords (and database IDs) aren't identities, they're identifiers (names). It seems you're using identity a little bit oddly here. In ordinary usage, identity would indeed be close to synonymous with identifier; the way you're using it here is actually closer to the usual comp-sci concept of a variable: a holder of mutable state, which can be pointed to a succession of different individual states. So part of this is a confusion arising from slightly odd or idiosyncratic terminology.- Hide quoted text - - Show quoted text - -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: HTML5 Validator
I hadn't considered using an online validator. Given that these are only unit tests, this is the simplest solution. Thanks! On Dec 18, 7:27 pm, Jeff Valk jv-li...@tx.rr.com wrote: On Saturday, December 18, 2010 at 02:10 pm, Alyssa Kwan wrote: I'd like to unit test my html output for well-formedness. What's an easy way to test it for HTML5 validity? Are there good Clojure libs for this? I only need to check for validity, not parse. I'm not aware of a native clojure html validator. That said, the first thing that comes to mind is to use something like clj-http [1] to post your markup to the w3c validator [2]. If you're doing this often, or offline, you could run the validator locally [3]. As a bonus, this method would get you validation for css, rss/atom, etc with miminal extra effort. Perhaps you've already considered this, but I figured I'd toss it out there anyway. Good luck! - Jeff [1]https://github.com/clj-sys/clj-http [2]http://validator.w3.org/#validate_by_input [3]http://validator.w3.org/docs/install.html (also in the debian/ubuntu repos as w3c-markup-validator) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ah-hah! Clojure is a Lisp
What things normally mean has no place in computer science. You have to embrace the jargon to be able to think rationally in the space. This in no way detracts from this discussion. When I say Hickey nomenclature, I mean vis a vis classical philosophy or Hegel. Lay nomenclature only muddies the water. On Dec 20, 1:41 pm, Ken Wesson kwess...@gmail.com wrote: On Mon, Dec 20, 2010 at 1:30 PM, Alyssa Kwan alyssa.c.k...@gmail.com wrote: No, identifiers are names. Identity transcends names. For example, in a distributed shared object system, multiple machines on the same network will have different identifiers for the same identity. Ordinary usage isn't good enough for metaphysical discussions. There is a metaphysical discussion of identity which applies to this situation, and Rich Hickey has taken a particular position. His position is rigorous, internally consistent, and applicable to how most people in our culture model the world, e.g. it can be used to accomplish work by most of us. Most philosophical discussions of identity really mean equality, at least in Hickey nomenclature. Thanks for making my point for me: identity normally means something other than what it means in Hickey nomenclature. :) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
HTML5 Validator
Hi! I'd like to unit test my html output for well-formedness. What's an easy way to test it for HTML5 validity? Are there good Clojure libs for this? I only need to check for validity, not parse. Thanks! Alyssa -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Lots of newbie clojure questions
Incidental mutability is the key. Functional programming doesn't eliminate mutability, it manages it: only the parts of the system that truly need to change state do so. Everything else is pure and easy to write and test. Contrast that with imperative programming where it's hard to tell which state changes are incidental and which are truly required. That's why Clojure provides identity objects: immutability is the default, and you have to ask for state change. That makes it a red flag that other programmers maintaining your code can observe. Idioms should be functions or macros. That's the beauty of symbolic programming: no boilerplate. Thanks, Alyssa On Dec 6, 9:19 pm, Robert McIntyre r...@mit.edu wrote: 1. What is the justification for using a map as a function? I find this to be very confusing. In math, a function is a mapping from one set to another, so from that perspective it makes good sense for a clojure-map to be a function from its set of keys to its set of values. The justification here is mathematical convention. Same thing with vectors being functions. 2. In practice, I find myself wincing when needing to decide whether or not to type an open-paren. Is that a common thing? Normally this isn't a problem for me because I always look at the bottom of my emacs window and am able to see the arguments whatever function I'm working with requires. 3. Is there a compendium of recursive idioms, ideally expressed in clojure? Including indentation conventions! (Is there an opportunity for a python inspired pre-processor that interprets indentation as parens?) You might try SICP (http://mitpress.mit.edu/sicp/) because Professor Sussman is awesome. Also try the clojure cookbook (http://en.wikibooks.org/wiki/Clojure_Programming/Examples/Cookbook) 4. Why don't long-running clojure programs monotonically increase in heap usage if all data-structures are immutable? If every reference to an immutable structure disappears than that structure can be garbage collected. Therefore, loops that create intermittent structures aren't that much of a memory hog since java garbage collects the structures every turn of the loop. ;;Memory : 0 (let [a (range 5)]) ;; Memory: big while the let is executing ;;Memory : 0 -- no reference to a anymore ! 5. In the REPL, isn't it redundant to always have to type the top- level parens? not for (let :) 6. Is there a place to get guidlines on macro idioms? The feature is so powerful that it seems like it would be easy to misuse it and crack a programmers head like a nut with a badly structured macro. Try Stuart Halloway's book, Programming Clojure. I liked it, at least. 1. Isn't the world actually imperative? And mutable? Collaboration *is* a messy proposition in real life. It's hard to fix your car, and even harder to have lots of people fix your car. I find the it models the real world better justification for functional programming rather confusing. (Indeed, the CPU and physical memory also have an imperative relationship!) The amount of mutability depends on your level of granularity. Let's say you want to add two numbers --- if you consider all the robotic actions the arm in your hard disk must do to load the memory into the CPU, the flow of electrons in the wires, etc, then there does seem to be mutable state everywhere. But, addition itself is a pure function, so the mutable state is incidental complexity that doesn't really matter in the long run. You can tell it doesn't matter because you can change all the mutable physical stuff and it won't affect the computation. You might use an abacus, a hydraulic computer, your brain, etc, but 2 + 2 is still 4 no matter how you get there. Just as Greens Theorem can transform a surface integral into a line integral and save a lot of work, it's important to circumscribe as much mutable state as possible with abstraction barriers to make our lives simpler. 2. 'Side-effects' are treated almost as a bad word by most functional programming advocates. And yet, aren't ALL programs executed for their side-effects? Does side effect perhaps then require a qualification or a tighter definition? Or perhaps side-effects aren't that bad? What is a side effect and what isn't depends again on your level of granularity. Too many side effects is a warning of badness because it means that you might be working at the wrong abstraction level. Incidental mutability should be removed because it just clutters everything up, but truly necessary mutability is just fine. 3. What is the relationship between the shape of a clojure program and the runtime call stack? (I ask because a clojure program looks strikingly similar to the callstack of an ordinary program when you 'pause' a thread.) 4. Is it possible (and/or advisable) to introduce a typing system on top of clojure? E.g. a macro-form along the lines of (fnif type_function function args) Try out
Re: Get sequence of values in arbitrarily nested collection
+1 Lazy is better. Personally, I would have used filter and map instead of for, but this is probably clearer. Thanks, Alyssa On Dec 6, 10:30 am, Justin Kramer jkkra...@gmail.com wrote: tree-seq makes this pretty simple: (defn nested-vals [key coll] (for [x (tree-seq coll? seq coll) :when (contains? x key)] (get x key))) This works with any type of key and all associative Clojure structures. It could be made compatible with Java structures by swapping out the 'coll?' predicate for something more general. Justin On Dec 5, 9:12 pm, Alex Baranosky alexander.barano...@gmail.com wrote: Hi guys, I would like a function to be able to take an arbitrarily nested collection and return a sequence of all values of a given key, such as :name, that appears anywhere in the nested collection. Does anything like this already exist? Thanks for the help, Alex- Hide quoted text - - Show quoted text - -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: There is no such thing as IAtom
+1 There is no STM integration with atoms. That's not a concern. Just write your own Clojure core with your change. I did for durable identities. shamelessPluggit://github.com/kwanalyssa/clojure.git/ shamelessPlug Seriously though, just use protocols. Thanks, Alyssa On Dec 6, 5:24 am, Benjamin Teuber bsteu...@googlemail.com wrote: I guess it was Rich's intention to have swap! be used for real atoms only so your code remains understandable - that's why it's called swap! for atoms, alter for refs and alter-var-root for vars. So why not define your own protocol for updating documents? If you really want (usually bad idea, I guess) you could still extend atoms or even refs to support this protocol, too. Regards, Benjamin On 5 Dez., 23:29, Pepijn de Vos pepijnde...@gmail.com wrote: tl;dr: Please add an interface to clojure.lang.Atom. kthxbye I had the brilliant idea of using CouchDB for something equally brilliant, and if possible implement a Clojure view server. Turns out Clutch fits the bill perfectly, except that I would like to use Aleph, and Couch is using something called MVCC. I haven't looked into the libs to deep, but I know the REST API, and what it does for updating is that you need to supply the current revision and if it doesn't match (i.e. there has been another update), the update fails. So then you need to get the new _rev and try again. Much the same way compare-and-set! works. I thought it would be perfect to implement IAtom and IDeref to get the latest value and to swap! documents in a spin loop with a side-effect-free function, like atoms do. But it turns out there is no interface for Atom and it is marked final. The same is true for Ref and Agent, but raek suggested this might be because they are more complex and integrated with the STM. Is there a good reason why I'm not allowed to implement my own atom? If not, can it be added? Thanks. Groeten, Pepijn de Vos -- Sent from my iPod Shufflehttp://pepijndevos.nl- Hide quoted text - - Show quoted text - -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: ANN: Durable Clojure - Functions and Closures
Yes, but why isn't persistence of emclosures/em generating more interest. ;) Persistence is solved, if you're OK with not being truly ACID... Seriously though, everyone has their own backends. I don't think anyone wants to be tied to BDB JE. Would there be interest in lazy-loading and -unloading data structures? On Nov 27, 7:35 pm, Ken Wesson kwess...@gmail.com wrote: On Sat, Nov 27, 2010 at 1:10 PM, Mark markaddle...@gmail.com wrote: Hi - I'm surprised your work doesn't generate more interest from folks. I wish I had more time, I would definitely jump in and help. Persistence doesn't seem to generate much interest in general. I posted my own stab at a way of persisting the ref world near-transparently a few weeks ago and it sank without a ripple. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: functional thinking
Hi Mike, TDD as if you meant it - http://gojko.net/2009/02/27/thought-provoking-tdd-exercise-at-the-software-craftsmanship-conference/ What you want is mocking and stubbing (these are different things!). http://s-expressions.com/2010/01/24/conjure-simple-mocking-and-stubbing-for-clojure-unit-tests/ Remember that unit testing is NOT integration testing... Thanks, Alyssa On Dec 1, 8:29 am, Laurent PETIT laurent.pe...@gmail.com wrote: Hi, 2010/12/1 Michael Ossareh ossa...@gmail.com Hi All, In the course of putting together my latest piece of work I decided to really embrace TDD. This is run of the mill for me in Java: - create some object that models your flow - create some object which contains your storage logic - create tests - dependency inject the correct storage logic depending on which scenario you're running in (prod / test / etc). Hum, this process does not seem to be eligible to be named TDD. In TDD, the tests are written first and shape the interface of your solution. Here what to do is more traditional: you write your domain objects, your logic, and you add tests. Not a critic of the methodology (I'm not advocating any methodology over another here, to be clear), but rather a thought on how things are named. Some more thoughts (not sure they will help, but who knows ?) : I've not been able to think about how to correctly achieve this same functionality in clojure. So far everything is pretty much pure functions, the storage functions being the only place where data is changed. Currently the storage is implemented with atomic maps. The production storage will be in Riak. Hmm, if by storage functions being the only place where data is changed you mean that in storage functions you do 2 things: change the value and store them, then IMHO you could split them in 2. I'm getting ready to build the Riak backend and now I'm faced with how to choose the correct backing implementation. I've a namespace, rah.test-storage, which implements the in-memory storage and I anticipate putting riak storage in rah.riak-storage - however I'm not sure what the best way to select the correct implementation at runtime or test time is. One solution I've come up with is to use defprotocol to define the functions for the storage layer (as you would an interface in java) and then have a defrecord for each implementation. Assume these to be in the namespace rah.storage, which would also house the functions which call the correct defrecord functions based on a property given at start time. This solution, however, feels like me trying to write Java in clojure - and I'm wondering how the lispers of the world would solve this same issue. Another solution would be to write the same set of functions in the rah.storage namespace which then look at the same property and then decide whether to call rah.riak-storage/store-user! or rah.test-storage/store-user!. The solution, as every solution, will have to be a trade-of. Here one axis for the tradeoff can be seen as how powerful you want your backend connectivity to be (singleton backend per app ? possibly several different backends at the same time ? pluggable backends during runtime ?) versus the ease of writing the app (the more probability you want power, the more probability there will be to have a backend object to be passed around : no backend object in case of a singleton backend for the app, several singleton backend objects). Note that if you know that each singleton backend will be of a different kind than the others, then simply a keyword for representing each backend may be sufficient. From what I can infer from what you described, if you would write the application without caring about programmatic testing, you would be fine by just having top level functions, and probably a top level configuration map with key/values for backend location, credentials, etc. If so, then it may be sufficient to leverage the possibility, in your testing framework (clojure.test ? anything else ...) to redefine the functions of the backend before the tests run. I'm pretty sure there are already such features allowing to temporarily redef (and restore at the end) the root value of global vars. HTH, -- Laurent- Hide quoted text - - Show quoted text - -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: why not change type compare functions do a compare on strings as well?
IMHO, any built-in string compare should support collations. I think this belongs in contrib in string. On Nov 29, 2:59 am, Tim Robinson tim.blacks...@gmail.com wrote: why not change type compare functions do a compare on strings as well? (defn ([x] true) ([x y](if (string? x) (. clojure.lang.Numbers (isPos (.compareTo x y))) (. clojure.lang.Numbers (gt x y ([x y more] (if ( x y) (if (next more) (recur y (first more) (next more)) ( y (first more))) false))) (defn ([x] true) ([x y](if (string? x) (. clojure.lang.Numbers (isNeg (.compareTo x y))) (. clojure.lang.Numbers (gt x y ([x y more] (if ( x y) (if (next more) (recur y (first more) (next more)) ( y (first more))) false))) It's just cleaner so we can do things like: user= ( 2010-06-11 2010-11-01) true user= ( Banana Apple) false make sense? Notes: * I ran a bunch of benchmarks, showing no real impact on performance. * probably would need to include = and = too. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
ANN: Durable Clojure - Functions and Closures
Extension of http://groups.google.com/group/clojure/browse_thread/thread/7c917e983f345723/a7ee771a7bcaaefc Hi everyone! I've extended the Clojure core to extend durability to functions and closures. 1. Functions with lexical and dynamic bindings are now supported. This includes functions that generate functions. 1.a. There is a slight overhead applied to all compiled functions; the bytecode is inaccessible at runtime except through a cache which maintains an entry for all classes regardless of whether they are eventually persisted or not. This doubles memory required per class, but should, in practice, be negligible. 2. Identities of identities are now supported. This means that persistent data structures of identities can be saved, i.e. you can have a ref of a hash-map of refs. 3. sorted-map and sorted-set are now supported. Get it here: git://github.com/kwanalyssa/clojure.git More testing, especially performance testing, is very welcome. However, at this point, I've satisfied my own requirements. I'd love to get this in scope eventually for enhancing Clojure core, if pain- free persistence is part of the grand vision. Discussion of API and implementation is also welcome. Thanks! Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ghost Vars?
On Nov 17, 10:35 pm, Ken Wesson kwess...@gmail.com wrote: On Wed, Nov 17, 2010 at 9:57 PM, Alyssa Kwan alyssa.c.k...@gmail.com wrote: How are you planning to persist a function? The clojure reader can't read functions output with spit or println. I'm not persisting functions at compile time. fns are first class objects, and I want to be able to persist them after creation like any other object. (def foo (fn [x y] (+ x y))) (dref foo :key store) Then later: (def bar (dref nil :key store)) bar is given the ref. Calling: (@bar 1 2) returns 3 Because it's not at compile time, I don't have access to the expr that generates the function. Stuart Halloway mentioned an invoke-time check for recompilation, which I assume requires the function to hang onto the expr and lexical environment which generates it. AFAICT, there is no such reference, and invoke-time lookups through vars are still being used; I'm probably not looking in the right place. Therefore, all I have at persist-time is the fn object, which is an instance of a subclass of AFunction. The framework does in instanceof check, then grabs the bytecode of the class. It saves the bytecode. It looks at the constructor and figures out if there are any instance variables, which are lexical bindings. It saves those objects. At deserialize/load-time, the framework grabs the bytecode. It renames the class to something guaranteed not to collide with any other class, existing or future/potential. It loads the class, then instantiates an object. If there were lexical bindings, it instantiates it with the instance variables that were saved at persist- time. Note that there's no official way to get the bytecode of a class object. If it's on disk, you can call getResourceAsStream on the ClassLoader. Otherwise, you're SOL. So at compile time I save a cache of WeakReference'd classes to their bytecode. All fn creation incurs this tax, but I hope it's light enough; it's certainly lighter than hanging onto the expr and lexical environment. I can think of at least three ways to build a framework for persisting functions but none run into problems with vars. All solutions run into problems with vars at deserialize/load time, because the var in the expr may not exist. You have to do *something* at that point, even if it's just throwing an exception for the missing var. Thanks, Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ghost Vars?
On Nov 18, 1:49 am, Alex Osborne a...@meshy.org wrote: Hi Alyssa, I don't think this situation is possible because Namespace.mapping (which maps symbols to vars and classes) is *not* thread-local. For a given namespace and symbol all threads will resolve the same var object. It's the binding of the Var to a value that can be thread-local. You shouldn't have to worry about it, as it happens in the dynamic environment when the fn is called, not when it is loaded. Perfect! That makes things so much easier! I assume that interning vars is synchronized then? This is the second big source code read FAIL in two days. Obviously I can't read. :) My preference would probably lean towards (2a). Anyone using drefs is going to have to deal explicitly with the issue of objects that aren't serializable (sockets, streams), so make them be very careful about what they put in them. Agreed. Only immutable objects may be persisted, unless they use one of the official identities (ref,atom,agent) for mutability. (2a) makes things easy. I'll have to play around to see if it's too restrictive. For the sake of practicality, I'd probably just be totally draconian about it and make fns and vars not durable at all. Almost any way you do it they'd make the program very brittle. They'd tie using the data very tightly to the structure of the source code, meaning that if you change the program (say rename a var) you're likely to not be able to read the data any more. All persistence requires dealing with migrations and compatibility. That's not a reason not to persist. What are needed are good tools/ idioms for dealing with it. ORM has succeeded on that front because the data is arbitrarily readable and writeable with standard tools, so the engineer can always manually modify stuff to migrate it. If a Clojure object data store were arbitrarily readable and writeable, then this is the first step to solving the problem. The second step is Ruby-style migrations. More on that later. ;) Thanks, Alyssa -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ghost Vars?
Hi Alex,
I understand exactly why this situation exists. I just think the
behavior is unexpected. When I create a function with a dynamic
binding, I expect the function to keep a reference to the *name*, not
the var that the name resolves to at compile/clinit time. Using with-
bindings* seems terribly unsupported as well.
I guess the question is: what do other people expect? Am I alone in
thinking that this is unexpected and undesirable?
Thanks,
Alyssa Kwan
On Nov 17, 3:04 am, Alex Osborne a...@meshy.org wrote:
Hi Alyssa,
Alyssa Kwan alyssa.c.k...@gmail.com writes:
ns-unmap isn't typically used. But for durability, we can think of
JVM shutdown/startup as unmapping everything and starting fresh.
Therefore, expected behavior for ns-unmap should be the same as
behavior for deserializing and loading an object after a new JVM
startup.
I think the point that you're missing is that vars are just plain old
first class objects. This includes being managed by the garbage
collector so they'll continue to exist until all references to them are
released. You can pass them around, put them in vectors etc.
Here's another issue I ran into:
= (def a 1)
#'user/a
= (defn b [] a)
#'user/b
When you compile this, a reference to the var #'user/a is embedded in
your b function. This reference is directly to the var object.
So far so good...
= (ns-unmap *ns* 'a)
nil
= a
java.lang.Exception: Unable to resolve symbol: a in this context
= (b)
2
The var has been unmapped from the namespace, but it still exists
because the function b has a reference to it. Vars don't need to live
in a namespace. For example, the with-local-vars macro creates a
local var which doesn't belong to a namespace.
= (binding [a 3]
(b))
java.lang.Exception: Unable to resolve var: a in this context
So what's the expected behavior here? I would think that after a is
unmapped, b should no longer work. Instead, it's hanging onto a with
the last value.
The var still exists, b holds a reference to it.
And the binding form doesn't work anymore, so there's
no way to dynamically bind over it. It's like a weird hybrid of
lexical and dynamic binding...
The only reason this is erroring is that the symbol 'a can no longer be
mapped to your var.
Actually there is a way to dynamically bind over a var which has been
unmapped, all you need is a reference to it:
user (def a 1)
#'user/a
user (def my-a #'user/a)
#'user/my-a
user (defn b [] a)
#'user/b
user (ns-unmap *ns* 'a)
nil
user (b)
1
user (with-bindings* {my-a 30} #(b))
30
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Dynamic Binding of Self-Referencing Functions Expected Behavior?
The issue here is not with b pointing to a. It's that b should point to b but doesn't. This *can't* be seen to be correct. Thanks, Alyssa Kwan On Nov 17, 9:22 am, Stuart Halloway stuart.hallo...@gmail.com wrote: In 1.2, functions were always looked up through their vars. While this is a low-cost operation, it does not allow maximum performance. In 1.3, function calls are compiled through their vars. If function a calls function c inside its body, there is no runtime lookup of the var c. However, each function makes a (very low cost) check on entry to see if anything has been recompiled. If so, the function is recompiled. This enables the dynamic repl interaction that you would expect from a lisp, with great performance. When, as in your example, a var b refers to var a, there is no function call, hence no hook point at which to make the check. If you want b to point to the new a, redef b. Stu Notice that when I redefined a, I only included one arity. If b were updated with the fn that a was redefined to, then (b 1 2) should have thrown an exception. Instead, it used the old definition of a but within that definition pointed to the new definition of a. This is internally inconsistent. I'm not proposing making all function definitions lexically bound. Yes, that would destroy interactive coding. But to be internally consistent, self-references should be lexical. In any case, I am using Github master and I thought I was using 1.2. 1.2 has self-references lexically bound, as David Sletten points out, which I agree is the correct behavior. But something has happened on 1.3 alpha that has changed that. I don't know if it's intentional or not. Thanks, Alyssa Kwan On Nov 16, 6:01 pm, David Nolen dnolen.li...@gmail.com wrote: But that would destroy one of the most useful features Lisp has to offer, interactive coding. Live coding would be impossible w/o this behavior as you would need to find and update all callers. Yuk. David On Tue, Nov 16, 2010 at 5:26 PM, Alyssa Kwan alyssa.c.k...@gmail.comwrote: I ran into this while working on making functions durable. Here's a contrived example: = (defn a ([x] x) ([x y] (+ (a x) (a y #'user/a = (a 1 2) 3 = (def b a) #'user/b = (b 1 2) 3 = (defn a [x] (- x)) #'user/a = (b 1 2) -3 Is this what people expect? I would think that the original definition of a, which is self-referencing, should point to itself no matter what it's named, not get resolved at invoke-time to see what the var is currently resolving to. Thanks, Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Dynamic Binding of Self-Referencing Functions Expected Behavior?
Thanks everyone! Alyssa Kwan On Nov 17, 11:10 am, Stuart Halloway stuart.hallo...@gmail.com wrote: I am wrong, there was a bug here, and Rich just fixed it. https://github.com/clojure/clojure/commit/8225407032ea643cbe3db7f35ef... Please retry against master, and sorry for inflicting more confusion. Stu Meikel, while a good description of how things work in 1.2, it's not accurate for 1.3, and my point was that Stu's description of how 1.3 works (by using words like the function is recompiled) does not match with my own knowledge of what had been done in 1.3 the days just before the conj. Since then, I've been away from #clojure (to my regret) and maybe I've missed new evolutions on the way it's handled in master. Anyway, this concept of the function is recompiled feels weird. My own comprehension of how things work in 1.3: Each function works through a cached set of var's values, and there's just a check to see if the cache is stale or not. If the cache is stale (some vars have been redef's, e.g. their root value has been changed), then the cache is recomputed. 2010/11/17 Meikel Brandmeyer m...@kotka.de Hi, On 17 Nov., 16:29, David Sletten da...@bosatsu.net wrote: = (defn a ([x] x) ([x y] (+ (a x) (a y #'user/a = (a 1 2) 3 = (def b a) #'user/b = (b 1 2) 3 = (defn a [x] (- x)) #'user/a = (b 1 2) -3 Let's call the original function assigned to 'a' a0 and the new one a1. After 'a' has been redefined to a1, 'b' still refers to a0. So the 2nd call to 'b' invokes a0 with two args (in fact, a1 only takes one arg now). But within a0 itself the references to 'a' are being resolved at runtime to a1 now, not as references to a0 as before. Are you saying that inside a0 Clojure detects that 'a' means something else now and recompiles a0 to point to a1? In any case, this behavior seems weird. Isn't that completely logical? Let's name things differently to keep functions and Vars apart. You define a function f and store it in Var a. f references Var a internally. Then you define Var b and store f in it by retrieving it from Var a. By executing (b 1 2) you basically retrieve f from b and call it with two arguments. Inside f, f itself is retrieved from the Var a and called with one argument. Now you create a new function g, which you store in Var a (assumption: re-defing an existing Var just replaces its value and does not unmap and map again a fresh Var). Again you execute (b 1 2). Same thing happens: f is retrieved and called with two arguments. Inside f however, now g is retrieved from Var a and called with one argument. This actually allows to define memoized recursive functions. Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en- Hide quoted text - - Show quoted text - -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ghost Vars?
Hi Alex,
OK, I agree. Dynamic vars are not the same as traditional dynamic
scoping.
For what I'm doing (making functions durable), it raises the
question: If you persist a function that points to a var, restart the
JVM, and deserialize/load the function from a data store, what should
happen?
1) In the loading thread, if a var exists with the same namespace and
symbol, set the internal reference to that var.
a) If the function is then passed to a different thread that has a
different var with that same namespace and symbol, the function will
still point to the one that was in the loader thread at the time the
function was deserialized/loaded.
2) In the loading thread, if a var does not exist with that same
namespace and symbol:
a) Throw an exception saying that the var doesn't exist.
b) Create the var with no namespace and symbol and no value. Wait
until the function is called to throw an exception saying that the var
is unbound.
c) Create the var with no namespace and symbol but with the value
that the var had in the persisting thread. There's no way to access
the var to modify it.
d) Create the var with no namespace and symbol but with the value
that the var had in the persisting thread. It's accessible somehow
(maybe the meta map), so the user can recover it and dynamically bind
over it.
e) Create the var with the namespace and symbol in the function,
modifying the RT var-space of the loading thread. Don't initialize
the var. This gives the user a chance to dynamically bind over it.
If the function is called before binding the var to something, throw
an exception saying that the var is unbound.
f) Create the var with the namespace and symbol in the function,
with the value that the var had in the persisting thread.
The normal case (1) is straightforward, and (1a) is what would happen
without se/des anyways. (2) is tricky. (2f) is most robust, but
really violates least surprise. I think Meikel's comments lean
towards (2a).
Also, the thread-local nature of vars raises the question of what
should happen when deserializing the same function from different
threads. If I create durable refs from different threads pointing to
the same store/key combo, does the thread that gets there first
control which var the deserialized function references? Or does each
thread get it's own ref? If so, how do you reconcile that there are
multiple refs being stored under the same store/key? That's
untenable, but it seems totally arbitrary to simply say that the first
thread that gets there determines which var the function is bound to
for the duration of the JVM instance.
What do you think?
Thanks,
Alyssa Kwan
On Nov 17, 8:20 pm, Alex Osborne a...@meshy.org wrote:
Alyssa Kwan alyssa.c.k...@gmail.com writes:
I understand exactly why this situation exists. I just think the
behavior is unexpected. When I create a function with a dynamic
binding, I expect the function to keep a reference to the *name*, not
the var that the name resolves to at compile/clinit time.
Oh, I see what you mean. I guess you're expecting something more like
Python's behaviour:
x = 5
def foo():
... return x
...
foo()
5
del x
foo()
Traceback (most recent call last):
File stdin, line 1, in module
File stdin, line 2, in foo
NameError: global name 'x' is not defined
In the case of Python, globals are a mutable map of names directly to
values and are presumably looked up at runtime. Python doesn't have
Clojure's concept of vars.
globals()
{..., 'x': 5, 'foo': function foo at 0xb775b7d4, ...}
I guess the question is: what do other people expect? Am I alone in
thinking that this is unexpected and undesirable?
It makes sense to me. I have a mental picture of functions closing over
the (lexical) environment as it existed when the function was defined
and that includes the dynamic vars as they were named at that moment.
Similarly in Python you can do this:
def foo():
return bar()
def bar():
return 5
Whereas in Clojure you would need to declare bar before foo.
That may mean dynamic vars are not exactly the same thing as traditional
dynamic scoping, but I don't see anything obviously unexpected or
undesirable about it. In fact quite the opposite, it's an intentional
design choice. It's consistent with the Clojure philosophy of
identities being first class. One of the major themes that distinguishes
Clojure's programming model from traditional languages is that names,
identities and values are distinct concepts.
A single var can be mapped into different namespaces under different
names (for example using the :rename argument to use). So the same
identity (var) may be referred to by multiple names (symbols), but it's
the identity you are dynamically binding a value to, not the name.
--
You received this message because you are subscribed to the Google
Groups
Dynamic Binding of Self-Referencing Functions Expected Behavior?
I ran into this while working on making functions durable. Here's a contrived example: = (defn a ([x] x) ([x y] (+ (a x) (a y #'user/a = (a 1 2) 3 = (def b a) #'user/b = (b 1 2) 3 = (defn a [x] (- x)) #'user/a = (b 1 2) -3 Is this what people expect? I would think that the original definition of a, which is self-referencing, should point to itself no matter what it's named, not get resolved at invoke-time to see what the var is currently resolving to. Thanks, Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Dynamic Binding of Self-Referencing Functions Expected Behavior?
Notice that when I redefined a, I only included one arity. If b were updated with the fn that a was redefined to, then (b 1 2) should have thrown an exception. Instead, it used the old definition of a but within that definition pointed to the new definition of a. This is internally inconsistent. I'm not proposing making all function definitions lexically bound. Yes, that would destroy interactive coding. But to be internally consistent, self-references should be lexical. In any case, I am using Github master and I thought I was using 1.2. 1.2 has self-references lexically bound, as David Sletten points out, which I agree is the correct behavior. But something has happened on 1.3 alpha that has changed that. I don't know if it's intentional or not. Thanks, Alyssa Kwan On Nov 16, 6:01 pm, David Nolen dnolen.li...@gmail.com wrote: But that would destroy one of the most useful features Lisp has to offer, interactive coding. Live coding would be impossible w/o this behavior as you would need to find and update all callers. Yuk. David On Tue, Nov 16, 2010 at 5:26 PM, Alyssa Kwan alyssa.c.k...@gmail.comwrote: I ran into this while working on making functions durable. Here's a contrived example: = (defn a ([x] x) ([x y] (+ (a x) (a y #'user/a = (a 1 2) 3 = (def b a) #'user/b = (b 1 2) 3 = (defn a [x] (- x)) #'user/a = (b 1 2) -3 Is this what people expect? I would think that the original definition of a, which is self-referencing, should point to itself no matter what it's named, not get resolved at invoke-time to see what the var is currently resolving to. Thanks, Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Ghost Vars?
ns-unmap isn't typically used. But for durability, we can think of JVM shutdown/startup as unmapping everything and starting fresh. Therefore, expected behavior for ns-unmap should be the same as behavior for deserializing and loading an object after a new JVM startup. Here's another issue I ran into: = (def a 1) #'user/a = (defn b [] a) #'user/b = (b) 1 = (def a 2) #'user/a = (b) 2 = (binding [a 3] (b)) 3 So far so good... = (ns-unmap *ns* 'a) nil = a java.lang.Exception: Unable to resolve symbol: a in this context = (b) 2 = (binding [a 3] (b)) java.lang.Exception: Unable to resolve var: a in this context So what's the expected behavior here? I would think that after a is unmapped, b should no longer work. Instead, it's hanging onto a with the last value. And the binding form doesn't work anymore, so there's no way to dynamically bind over it. It's like a weird hybrid of lexical and dynamic binding... Thanks, Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Function Design: sequence or argument?
Performance is part of it too. Allowing dispatch on arity leads to faster code. Many of the functions that operate on sequences are lazy so dispatch on arity doesn't apply. On Nov 15, 11:52 am, Chris christopher.ma...@gmail.com wrote: If you have a function that needs to treat multiple arguments as a group, what forces drive you to represent this as a single sequence argument vs. an argument? To give a concrete example, why does + work like (+ 1 2 3 4) instead of (+ [1 2 3 4]) Is it performance? Aesthetics? Composability concerns? Not having to call apply all the time? Thanks, Chris -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Incorrect behaviour for large s-expressions :(
I'm building an ETL app, so aggregate functions of arbitrarily large arity is a necessity. I've had to wrap a lot of core clojure functions with concrete arg lists to make them work with lazy sequences. In my limited experience, machine generated code of this nature should use lazy sequences that get realized at eval-time rather with arg lists that are realized compile-time. Keep in mind that currently, functions are subclasses of AFunction where constants are stored in the class file as static final class members. This may eventually get optimized, but I wouldn't hold my breath; it would take a LOT of static analysis to recognize closures, and this level of support for macros will probably never happen. Each function eats up your permgen. Garbage collection of classes vs. objects is also REALLY tricky. So you're probably better off using sequences that consume plain old heap and functions that don't close over things so you use less permgen space. So yes, I don't think this is worth getting worked up about. On Nov 14, 9:13 pm, David Nolen dnolen.li...@gmail.com wrote: On Sun, Nov 14, 2010 at 4:21 PM, Robert McIntyre r...@mit.edu wrote: That is not in fact an adequate workaround --- (eval `(apply + ~@(take 9001 (iterate inc 1 ;; OVER 9000!!! or, alternately (eval (cons 'apply (cons '+ (take 9001 (iterate inc 1) will fail just as in the addition examples. It's not true that you can just use an apply in your auto generated code, you would instead have to do something like a tree of function calls, so It may be worth increasing the limit for for the sake of enabling machine generated code. What are you peoples' thoughts on this? --Robert McIntyre Not worth getting worked up. David- Hide quoted text - - Show quoted text - -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Closures eat permgen?
If you look at the bytecode for the closures, you'll see that the Var that *ns*/a points to is resolved at clinit time, and the Java reference is stored as a static final class member. That's a small use of additional permgen. In your example, my-generator isn't the concern. It's the call to my- generator that creates functions, each of which creates bytecode, is loaded as a class, then is instantiated, and finally invoked. That temporarily uses permgen. There should be no problem collecting permgen in this trivial example. However, more complex scenarios will cause classes to live far beyond you would expect. If there is a leak or long-lived object, you'd rather it be in the regular heap instead. My point in the other thread was more of a question of using closures at all vs. not using closures. All functions lead to a loaded class, which by definition uses permgen. Unless there's a real win in terms of design or maintainability, you should use regular functions and just pass in your state/identity. The use case in the other thread was a bad one as far as using closures go; I was recommending against that. On Nov 15, 2:52 pm, Ken Wesson kwess...@gmail.com wrote: In another thread, someone just indicated that a closed-over variable chews up permgen. Is this true? I had been under the impression that keywords didn't get garbage-collected, so runtime generation with (keyword foo) ought to be used sparingly, but that was it. Perhaps the scenario was something along the lines of (defn make-generator [] (let [a (atom 0)] (fn [] (swap! a inc) @a))) (def my-generator (make-generator)) user= (my-generator) 1 user= (my-generator) 2 user= In this case, a closed-over atom is referenced by a function that's bound to a global var, my-generator. This won't be garbage-collected so long as my-generator isn't un/redef'd. Even so it shouldn't be permgen, technically speaking; just indirectly referenced by a loaded class and so ineligible for GC by more ordinary criteria. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Closures eat permgen?
I totally misunderstood the role of the EVAL context flag in the compile method of ObjExpr. Is there a general writeup anywhere of how the compiler works, especially the interaction of parse and emit? On Nov 15, 4:59 pm, Ken Wesson kwess...@gmail.com wrote: On Mon, Nov 15, 2010 at 4:24 PM, Alan a...@malloys.org wrote: On Nov 15, 12:12 pm, Alyssa Kwan alyssa.c.k...@gmail.com wrote: In your example, my-generator isn't the concern. It's the call to my- generator that creates functions, each of which creates bytecode, is loaded as a class, then is instantiated, and finally invoked. Not true. Compiling my-generator creates two classes, which at run time are simply instantiated as needed. I thought so. Now if your code has stuff like: (defn foo [x] (eval `(fn [quux] blah blah blah quux blah blah ~x blah))) then every call to foo generates new classes and loads them at runtime. Part of why eval should be used sparingly. :) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: (ab)using STM for longish calculations, network I/O
On Oct 15, 7:57 pm, peter veentjer alarmnum...@gmail.com wrote: On Oct 15, 2:51 pm, hobnob hob...@ml1.net wrote: Hi, I'm just starting to get my head wrapped around STM just by reading about it so I can make a decision on whether to port a Java project to Clojure. a) can STM transactions contain calculations that take a 'long' time, let's say computing the cryptographic hash of a plaintext. I'd 'ensure' the input parameters such as plain text, hash algo and bit- length, compute the hash (can be slow) and store the hash in a ref. What I'd need here is that the calculation is interrupted if the transaction is aborted for a retry. No need to complete the long calculation if we aren't going to store the result anyway. The existing convention in Java is to set the interrupt flag on a thread which is queried ever now and then by long-running calculations. This is a convention that many Java libraries adhere to. So how to do this interrupting? Also a tough question. If you have a non transactional flag, it doesn't need to provide the value you expect it to have. And using a transactional flag could also lead to strange problems like transactions always conflict on change of that flag. hobnob, I'm confused. What does the former (retry leading to immediate abort) have to do with the latter (interrupt by a monitoring process)? The former is actually really easy to implement, but it requires an enhancement to core: there's a RETRY_LIMIT of 10,000 for all transactions; let that be configurable per transaction and set the limit to 1 for the ones you want immediate abort for. b) With long-running calculations, nested dosyncs should really result in nested transactions. The point being if a transaction has several long-running calculations. E.g. first compute a hash, then encrypt the hash with a public key. In that case, if the 'ensured' parameters that the second encrypt part depends on are changed concurrently but not the parameters/result of the first part, then only the second calculation has to be repeated, not both. The second part could be enclosed in an inner dosync but currently Clojure will unnecessarily redo the whole thing. I guess you need a propagation level: RequiresNew. And at the end you need to be able to commit all the transactions as one. Afaik Clojure has no support for it, but I don't think it is very hard to add if clojure also exposes some kind of prepare methods to makes sure that the transaction is able to commit. For the Multiverse STM I have introduced CommitBarriers for this purpose, but I don't think they would be hard to add to the Clojure STM. peter, I'm confused. What does 2PC have to do with nested transactions? I'm new to transactional/locking programming, so I'm probably missing something blindingly obvious. hobnob, this seems to be a static analysis problem: given the body of a dosync, figure out what writes depend on ensured reads/previous writes, and only recalculate those on a retry. Again, this requires an enhancement to core, but I can certainly envision building out a dependency graph as the body of a dosync is walked down by LockingTransaction. This approach makes performance (even more) unpredictable, since not all locks would be released/reacquired on retry, but that should be more than compensated by the shorter transaction windows leading to less contention in the first place. Of course, building that graph out for small transactions is overkill, so it should be a setting passed into dosync. One key benefit of this approach is that it doesn't require the user to specify nested transactions. Transactions boundaries should be about correctness alone; performance should be kept orthogonal. This approach takes the load off of the user completely and lets the VM do it. c) Somewhat different: I'm not supposed to do I/O in a transaction because the transaction might be repeated and that will repeat the I/ O. But maybe that's what I want. The I/O could be a network output sending the computed value over the net to be stored on a remote machine instead of being stored in a local ref. I *do* want the computed value of each retry to be sent over the net. I guess my setup here could be considered an ad-hoc distributed TM, this touches the other discussion of STM with external transactions in this group. All communication with non transactional datasources from within a transaction is hard. You can do some stuff with the deferred and compensating actions if the transaction commits or aborts..But I haven't found a solution for this problem either. This was one of the main reasons for MS to drop their STM research project. hobnob, I'm genuinely curious what your use case is. Why do you want to keep tries? Is there some purpose other than logging or diagnosing system performance? If it's either of those, there should be ONE RIGHT WAY to look at Clojure
Re: Help Fixing Bug in Durable Reference Creation Inside DoSync Transaction
I've come up with a solution to the durable ref creation/declaration in dosync block bug. The creation/declaration exclusively locks the logical dref by key to force other transactions to retry. I welcome feedback and code review from anyone interested in out-of-the-box durability of STM identities. git://github.com/kwanalyssa/clojure.git Thanks! Alyssa On Oct 3, 12:54 am, Alyssa Kwan alyssa.c.k...@gmail.com wrote: I've noticed a bug in my implementation of the durable ref, or dref construct. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Help Fixing Bug in Durable Reference Creation Inside DoSync Transaction
I've noticed a bug in my implementation of the durable ref, or dref construct. This is an attempt to add durability to the STM. drefs have all of the ACI guarantees of refs, with the additional property of being durable, so changes to the state of the identity are durable. The API is that there exists a logical namespace keyed by store name - a string - and a key - an object, which then corresponds to a value. Calling (dref store key initVal) creates a durable ref with the state of initVal under the store name store and key key, which is immediately written to the underlying BDB JE database. Subsequent invocations of (dref) with the same store and key refer to the same logical object; initVal is ignored. This is also true if the subsequent (dref) call is in a later JVM instance; initVal is ignored and the durable value saved in the BDB JE database is used. (dref) is therefore a drop-in replacement for (ref). This enhancement to the Clojure core is accessible here: git://github.com/kwanalyssa/clojure.git. There's some slight enhancements since I last ANN'd this: the rest of the Clojure primitives are now supported: BigDecimals and Ratios; and any object type acceptable as values are also acceptable as keys. This opens up the possibility of easily chaining key-value pairs, and having built in, transactional, durable memoization. I'm currently working on adding the persistent data structures such that they're also persistent on disk, minimizing I/O; and making the persistent structures lazily-loaded and evicted. This means potentially having petabyte-size maps occupying a fraction of that in memory and seamlessing swapping sections of the keyset and attached entryset based on usage. The possibilities for multidimensional databases (the use case I'm enhancing Clojure to serve), or even simply as massive, transactional, memoized caches, is awesome. But there is a bug. It has to do with creating/declaring durable refs inside of dosync blocks. To wit: (dosync (def a (dref store key 0)) (ref-set a (/ 1 0))). This block will obviously fail (i.e. throw an exception). What then, is the var a referring to? It must refer to the dref that was declared/created. But what is the state of that dref identity? In the non-durable version: (dosync (def a (ref 0)) (ref-set a (/ 1 0))), the var a points to a ref with a state of 0. That's because the ref was implicitly created in it's own single-operation transaction (like a DB autocommit) that precedes the transaction declared by the (dosync). It is possible because the ref has 2 properties: it is local, nothing outside the dosync block can concurrently refer to the ref - the var a is ThreadLocal; and the ref is obviously being created, it is not pointing to a ref that exists before the dosync block. However, the durable version doesn't have those two properties. Because the dref namespace is global, another transaction concurrently can be referring to that dref: either a single-operation create/ declare - (dref store key aDifferentInitVal), or the same create/ declare in a different dosync block. Also, the (dref) statement may refer to a dref that already exists in the store: the (dref) isn't creating something new, but refers to something already in the DB, in which case the initVal is ignored. This makes the transaction semantics much harder. In my current implementation, I punt on some of these issues. Imagine a scenario where there are 2 vars pointing to the same logical dref. var a is pointing to (dref store key initVal) and so is var b. dosync block A containing an (alter) on var a is executing concurrently as dosync block B, which contains an (alter) on var b. Although these may be two different POJOs, they have the same store and key, so they are the same logical object. Maintaining ACI guarantees means that acquiring a read lock on the identity referred to by var a must acquire the same lock on the identity object referred to by var b. The same is true with write locks. If dosync block A already has a write lock on the logical dref under both var a and var b, dosync block B must back off and retry after dosync block A either finishes or is cancelled/errored out for good. This is a property of having multiple POJOs potentially pointing to the same logical identity because of a global namespace. I punt on this issue by ensuring that all drefs declared with store key have the same POJO by using a weak cache. Because they have the same POJO, a lock on one (dref) object is a lock on all. But this leads to a problem. To illustrate, imagine 3 scenarios: In the first scenario, the dref doesn't exist in the DB. The (dref) statement is creating a new dref. There are two dosync blocks concurrently executing: dosync block A and dosync block B. dosync block A just happens to begin executing before dosync block B. However, the (dref) statement in dosync A just happens to execute after the (dref) statement in dosync block B. Because the
ANN: Durable refs with ACID guarantees - Phase I
This is in reference to http://groups.google.com/group/clojure/browse_frm/thread/d6deac0c34d0ce28/5c1e11ec2bd52bde. Hi everyone! I have hacked the Clojure core to add durability to refs. The syntax to create these is (dref val key path), where key and path are strings. Then you use them just like refs. Creating a dref creates a global identity, such that subsequent dref calls to the same key and path will get the same dref. On subsequent dref calls, the val will be ignored and the persisted value used. This includes in subsequent VM instances. Get it here: git://github.com/kwanalyssa/clojure.git 1. path refers to BDB JE databases which get created in the data directory of your project. 2. BDB JE is used. I'm ignorant of IP and licensing issues. Making BDB JE core to Clojure is probably an issue. 3. Currently only a subset of Clojure primitives types is supported. No BigDecimal or Ratio yet. See the comprehensive list (and serialization mappings) at the bottom of src/jvm/clojure/lang/ DRef.java. BDB JE TupleBindings are used. Submissions welcome, especially for the persistent data structures. 4. How do we approach the problem of storing objects with lexical environments? 5. Unit tests welcome! I didn't do TDD since the work is in Java and there's no Java-level tests in the project. Please add your own in test/clojure/test_clojure/drefs.clj. I'm new to concurrency, so tests along those lines would be awesome! 6. The ACID part is not really guaranteed!!! STM is currently one- phase-commit. I inserted two-phase-commits to the data stores in the middle of the one-phase-commit. There's the remote possibility that STM in-memory changes fail AFTER writing to disk. It's REALLY remote, but it is possible. STM would have to be made 2PC to make this airtight. That's way beyond my current grasp of both concurrency and Clojure implementation. 7. I was aiming for an API where path is optional. However, I didn't want to stray from the ref API, which has variable arity. Suggestions on how to reconcile the two are welcome! 8. To maintain global identity, I use a static cache, which requires non-hard references to avoid OOM issues. This is my first time doing this, so please check my code to make sure that I'm doing it right. I'm using SoftReferences, though WeakReferences may be better for real- life usage patterns. Let me know! Please dig in! Feedback appreciated! Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: ANN: Durable refs with ACID guarantees - Phase I
It's probably possible to do it completely in Clojure, but you have to subclass Atom. There's no need for any transaction boundary; you just have to make sure that compareAndSet does a durable swap. My plan was to get durable refs done and then extend the other mutable identities, including atom. I'd love to work with you on it! On Sep 23, 8:27 am, Per Vognsen per.vogn...@gmail.com wrote: Cool! I'm getting back to Clojure after an extended absence. Just today I was pondering the design of a solution to a similar problem, though I suspect our requirements diverge on several points. My tentative conclusion was that it could be done entirely in Clojure and without modifying existing code. Maybe you can poke holes in my fledging plan since you've obviously been thinking about this sort of problem longer than me: There's a new pref reference type. It consists of a key and an atom containing nil for unloaded objects and an STM reference for loaded objects. When a pref is dereferenced, it checks its atom. If nil, it first loads the object from disk into a fresh STM reference (which has a metadata field pointing back to the pref) and mutates the atom so it points to it. In either case it finishes by dereferencing the STM reference. When a pref is mutated, it first goes through the same motions as for dereferencing. Then it simply forwards the mutation to the underlying STM reference. Watchers are installed on STM references backed by prefs. Thus we are notified when something is mutated. There is a pref-specific transaction boundary form called atomic, analogous to dosync. The watchers are used to determine which prefs were mutated during the transaction so as to flag them dirty for write-back or write-through caching; this is why we need the pref back reference in the metadata. Anyway, even assuming this all works, it will obviously be less computationally efficient than extending LockingTransaction.java with special support. -Per On Thu, Sep 23, 2010 at 10:21 AM, Alyssa Kwan alyssa.c.k...@gmail.com wrote: This is in reference tohttp://groups.google.com/group/clojure/browse_frm/thread/d6deac0c34d0 Hi everyone! I have hacked the Clojure core to add durability to refs. The syntax to create these is (dref val key path), where key and path are strings. Then you use them just like refs. Creating a dref creates a global identity, such that subsequent dref calls to the same key and path will get the same dref. On subsequent dref calls, the val will be ignored and the persisted value used. This includes in subsequent VM instances. Get it here: git://github.com/kwanalyssa/clojure.git 1. path refers to BDB JE databases which get created in the data directory of your project. 2. BDB JE is used. I'm ignorant of IP and licensing issues. Making BDB JE core to Clojure is probably an issue. 3. Currently only a subset of Clojure primitives types is supported. No BigDecimal or Ratio yet. See the comprehensive list (and serialization mappings) at the bottom of src/jvm/clojure/lang/ DRef.java. BDB JE TupleBindings are used. Submissions welcome, especially for the persistent data structures. 4. How do we approach the problem of storing objects with lexical environments? 5. Unit tests welcome! I didn't do TDD since the work is in Java and there's no Java-level tests in the project. Please add your own in test/clojure/test_clojure/drefs.clj. I'm new to concurrency, so tests along those lines would be awesome! 6. The ACID part is not really guaranteed!!! STM is currently one- phase-commit. I inserted two-phase-commits to the data stores in the middle of the one-phase-commit. There's the remote possibility that STM in-memory changes fail AFTER writing to disk. It's REALLY remote, but it is possible. STM would have to be made 2PC to make this airtight. That's way beyond my current grasp of both concurrency and Clojure implementation. 7. I was aiming for an API where path is optional. However, I didn't want to stray from the ref API, which has variable arity. Suggestions on how to reconcile the two are welcome! 8. To maintain global identity, I use a static cache, which requires non-hard references to avoid OOM issues. This is my first time doing this, so please check my code to make sure that I'm doing it right. I'm using SoftReferences, though WeakReferences may be better for real- life usage patterns. Let me know! Please dig in! Feedback appreciated! Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group
Re: ANN: Durable refs with ACID guarantees - Phase I
I haven't benchmarked... I don't have much experience with benchmarking. Assistance would be greatly appreciated! On Sep 23, 9:09 pm, Dragan Djuric draga...@gmail.com wrote: What is the performance penalty? On Sep 23, 5:21 am, Alyssa Kwan alyssa.c.k...@gmail.com wrote: This is in reference tohttp://groups.google.com/group/clojure/browse_frm/thread/d6deac0c34d0 Hi everyone! I have hacked the Clojure core to add durability to refs. The syntax to create these is (dref val key path), where key and path are strings. Then you use them just like refs. Creating a dref creates a global identity, such that subsequent dref calls to the same key and path will get the same dref. On subsequent dref calls, the val will be ignored and the persisted value used. This includes in subsequent VM instances. Get it here: git://github.com/kwanalyssa/clojure.git 1. path refers to BDB JE databases which get created in the data directory of your project. 2. BDB JE is used. I'm ignorant of IP and licensing issues. Making BDB JE core to Clojure is probably an issue. 3. Currently only a subset of Clojure primitives types is supported. No BigDecimal or Ratio yet. See the comprehensive list (and serialization mappings) at the bottom of src/jvm/clojure/lang/ DRef.java. BDB JE TupleBindings are used. Submissions welcome, especially for the persistent data structures. 4. How do we approach the problem of storing objects with lexical environments? 5. Unit tests welcome! I didn't do TDD since the work is in Java and there's no Java-level tests in the project. Please add your own in test/clojure/test_clojure/drefs.clj. I'm new to concurrency, so tests along those lines would be awesome! 6. The ACID part is not really guaranteed!!! STM is currently one- phase-commit. I inserted two-phase-commits to the data stores in the middle of the one-phase-commit. There's the remote possibility that STM in-memory changes fail AFTER writing to disk. It's REALLY remote, but it is possible. STM would have to be made 2PC to make this airtight. That's way beyond my current grasp of both concurrency and Clojure implementation. 7. I was aiming for an API where path is optional. However, I didn't want to stray from the ref API, which has variable arity. Suggestions on how to reconcile the two are welcome! 8. To maintain global identity, I use a static cache, which requires non-hard references to avoid OOM issues. This is my first time doing this, so please check my code to make sure that I'm doing it right. I'm using SoftReferences, though WeakReferences may be better for real- life usage patterns. Let me know! Please dig in! Feedback appreciated! Alyssa Kwan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: ANN: Durable refs with ACID guarantees - Phase I
Ah. I thought we were discussing prefs, or datoms (durable atoms), as I would call them. Because datoms are only synchronous but not coordinated, there's no transaction boundary. (More accurately, the swap! is the transaction boundary, much like auto-commit.) dosync has no effect on datoms. drefs, being coordinated, do require a transaction boundary. However, I don't think it's possible without modifying LockingTransaction. It's bad enough that the current implementation has 2PC against ACID resources wrapped inside of a 1PC STM transaction. To place the durable write outside of the 1PC would be much less safe. dosync enforces a global transaction order. If writes were outside LockingTransaction.run(), the order could (and probably would) be different between in-memory resources and durable resources. For ultimate safety, we need to be even more intrusive and add a prepare phase to the STM. On Sep 23, 11:05 pm, Per Vognsen per.vogn...@gmail.com wrote: On Thu, Sep 23, 2010 at 8:16 PM, Alyssa Kwan alyssa.c.k...@gmail.com wrote: There's no need for any transaction boundary; you just have to make sure that compareAndSet does a durable swap. I had the chance to read your code today. You have a transaction boundary in DRef.set() which is called by LockingTransaction.run() at commit time. My point was that if you weren't intrusively modifying LockingTransaction.java you would need to take care of that somewhere else, the most obvious place being a dosync wrapper form. All you would need is a seq of 'vals' returned on a commited run(). That would also be useful for application-side transaction logging, etc. -Per -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Extending Clojure's STM with external transactions
OK... This question is probably better directed at clojure-dev, but my membership is still pending. I'm having trouble interpreting LockingTransaction.run. Where exactly are read-locks for ensures set? And what precisely is in the commutes map and sets set? Why does membership in sets short-circuit the commutes loop, and what if there is a read lock on a member of the sets set? Everything is done with the exception of the actual compare and set operation with the BDB database. In other words, the real work. :) Thanks! Alyssa On Sep 1, 6:14 pm, Alyssa Kwan alyssa.c.k...@gmail.com wrote: I'll go one step further and say that we shouldn't have to call persist namespace. It should be automatic such that a change to the state of an identity is transactionally written. Let's start with refs. We can tackle the other identities later. The API is simple. Call (refp) instead of (ref) when creating a persisted ref. Passed into the call are a persistence address (file path, DB connection string, etc.) and a name that has to be unique to that persistence address. Not all refs end up being referred to by a top-level symbol in a package, and multi-process systems are hard... Ensuring uniqueness of name is up to the programmer. Upon creation, Clojure checks to see if the refp exists in the store; if so it instantiates in memory with that state, else it uses the default in the call. In a dosync block, the function runs as normal until commit time. Then Clojure acquires a transactional write lock on each refp that is alter-ed or ensure-d. It checks the value against memory. If it's the same, commit data store changes. If not, retry after refreshing memory with the current contents of the store. If the data store commit fails, retry a number of times. If the data store commit still can't proceed, roll back the whole thing. commute and refset are slightly different, but for an initial implementation, just treat commute as an alter, and ignore refset. Does this make sense? My intention is to cover the 80% case. The implementation would necessarily be chatty, since the API is chatty. That's OK for most systems. This API has the benefit of being able to be shared across Clojure instances. It's a nice bonus. A dosync block may contain symbols pointing to refp's spanning different data stores, which isn't too hard to handle. It simply requires that if this is the case, each data store must support two- phase commit or some other distributed transaction supporting protocol. For an initial implementation, I would just throw an exception. I've begun working on an implementation using BDB. What do people think? On Aug 30, 5:02 pm, nchubrich nchubr...@gmail.com wrote: I'm not aware of any, but +1 for seeing persistence handled as part of the language. A big project and a long-term one, to be sure, but could it not be considered a goal? In my student days, I was talking to a well-known Lisper (name suppressed for fear of Google indexing) about some data structures in MIT Scheme. When I asked about saving them to disk, he said in effect, You're on your ownthat's something that \should be handled, but never is. I think people are so used to this state of affairs they forget how ugly it really is. Programming languages are like Moses without Joshua: they lead your data in the wilderness, but when it comes to finding it a permanent home, you have to talk to someone else. And these someone elses (who seem to be as numberless as the sons of Abraham) each have their own habits and ways of talking. Persistence libraries always end up warping the entire codebase; I've never succeeded in keeping them at bay. Using data with Incanter is different from ClojureQL, which is different from just using contrib.sql, and all of it is different from dealing with just Clojure. (I've never even tried Clojure + Hibernate.) You might as well rewrite the program from scratch depending on what you use. Maybe other people have had better luck; but whatever luck they have, I'm sure it is a fight to keep programs abstracted from persistence. I'd like to be able to work with mere Clojure until my program is complete, and then work in a completely separate manner on how to read and write data. Or maybe there would be off-the-shelf solutions I could plug in for different needs: low latency, high read, high write, large stores, etc. On the Clojure side, you would simply call something like persist namespace, which would save the state of your current or given namespace (unless you pass it the names of variables as well, in which case it only saves those). And to read data, you would simply require or use it into your namespace: you could choose what granularity to give first-class status: just tables, or columns as well, etc. And you could do this equally well for XML, JSON, relational data, or a graph store
Re: Extending Clojure's STM with external transactions
How about introducing a second part to the api? (store) creates a wrapper for the persistent address, and refp then takes one of those wrappers and the name? I like that. I would go one step further and say refp should have a default data store that is used unless you specify anything else via 'store' or additional arguments to refp. This would go partway towards making persistence like garbage collection. Great suggestion! The reason I suggested persisting entire nampespaces was that sometimes (often) the need to persist data is outside of any usage of refs (or atoms, or agents). Maybe such a high-level facility could be built on top of refp. I disagree. There three kinds of identities - those that mutate in a multi-process environment, those that mutate in a guaranteed single- process environment, and those that don't mutate. If you can make outside changes to the underlying data store at runtime (DB or even properties file), through either another JVM or manually, then it's inherently multi-process. Multi-process mutation of identities belong in refs et al. The second case, mutation in a single-process, is synonymous with vars in Java/Clojure, where logical processes are VM threads. The problem is that vars are thread local; how in the heck do you name the thread- specific var for persistence? And read from them, given that the next time the number and nature of threads will be different; how do threads in a future VM map to threads in a previous one; threads are inherently volatile. I see the use case of long-running processes having a clear identity and needing to persist/recover state, but it's a much smaller use case than the first (multi-process mutation). Identities that don't mutate are like defonce, where the db state being read is locked for the duration of the VM. The only real use case is a properties database that can only be edited when the VM is shut down. I guess what I don't see is why you would need to OFTEN persist data outside of refs... The second case, though legitimate, seems minor, and can be easily modeled with the first. The third case (the properties database) is very often, but should easily be addressed by libs (though I agree it should be part of the language). I guess what I'm saying is that we should probably be using refs if persistence is required. Another thing I'd like to see (as I mentioned) is an extension of map syntax for the kinds of (e.g. SQLish or XMLish) data structures you get out of persistent stores. One of the brilliant things about Clojure is its unification of multiple sequence types. It was something that was crying out to be done. I think that, similarly, there could be a more unified and standard way of dealing with maps, maps-of-maps, maps containing lists, etc. There must be someone on this list who can develop a formally correct regular expression language for hierarchical data. The formal correctness part is not my strong suit. I'm not too familiar with XSL, but would a less verbose version of that work? Thanks! Alyssa -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Extending Clojure's STM with external transactions
Thanks, Constantine! Your work on cupboard is awesome! I'll take a look at the deadlock detection to see if I can help. Any thoughts on how to marshal functions? What about vars and dynamic binding? Thanks! Alyssa On Sep 5, 11:02 am, Constantine Vetoshev gepar...@gmail.com wrote: On Aug 30, 5:02 pm, nchubrich nchubr...@gmail.com wrote: Persistence libraries always end up warping the entire codebase; I've never succeeded in keeping them at bay. Using data with Incanter is different from ClojureQL, which is different from just using contrib.sql, and all of it is different from dealing with just Clojure. (I've never even tried Clojure + Hibernate.) You might as well rewrite the program from scratch depending on what you use. Maybe other people have had better luck; but whatever luck they have, I'm sure it is a fight to keep programs abstracted from persistence. I feel the same way. Late last year, I wrote a small BDB-based library for adding disk persistence to Clojure datatypes. It does require knowing in advance which types you want to write to disk. I haven't had time to work on it recently, but I'd like to pick it up again sometime soon. (And I welcome help — the wonders of open-source software and all that.) http://github.com/gcv/cupboard It has a few outstanding problems. 1. It needs to be updated for deftype and defrecord in Clojure 1.2. 2. Deadlock detection doesn't seem to work 100% of the time, and I haven't yet tracked down the reasons. Parts of the test suite currently fail as a result. 3. Reads are slow. Casual profiling hasn't revealed the reasons, so it's a bit tricky to track down. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Extending Clojure's STM with external transactions
Revision number is a great idea! I don't think I want to do copy-on-write within Clojure because it would require a separate thread for cleanup. The underlying database should take care of it anyways. Thanks! Alyssa On Sep 2, 8:47 am, Timothy Baldridge tbaldri...@gmail.com wrote: It checks the value against memory. If it's the same, commit data store changes. If not, retry after refreshing memory with the current contents of the store. May I suggest we take a page from the CouchDB book here? In addition to having a id each ref also has a revision id. Let's say the id for my ref is foo. Then the first time I modify that ref the revid becomes 1-foo. From there the 1- is incremented by one on each writing transaction. This allows for a single string load compare for each change, instead of reading the entire contents of the ref. That combined with CouchDB's copy-on-write means that you can kill a couchDB process mid-execution, and then restart it without any data corruption at all. If we find a way to do external persistence...that would be awesome, I can think of several use cases for it right now. Timothy -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Extending Clojure's STM with external transactions
ITA. I was planning on doing what clojure.contrib.sql does with a global var and with-connection block for dynamic binding. Unless people don't like that approach... On Sep 2, 12:56 pm, Mike Meyer mwm-keyword-googlegroups. 620...@mired.org wrote: On Wed, 1 Sep 2010 15:14:45 -0700 (PDT) Alyssa Kwan alyssa.c.k...@gmail.com wrote: I'll go one step further and say that we shouldn't have to call persist namespace. It should be automatic such that a change to the state of an identity is transactionally written. Let's start with refs. We can tackle the other identities later. The API is simple. Call (refp) instead of (ref) when creating a persisted ref. Passed into the call are a persistence address (file path, DB connection string, etc.) and a name that has to be unique to that persistence address. Not all refs end up being referred to by a top-level symbol in a package, and multi-process systems are hard... Ensuring uniqueness of name is up to the programmer. Upon creation, Clojure checks to see if the refp exists in the store; if so it instantiates in memory with that state, else it uses the default in the call. First, I like the idea. But I think it's a bit clunky. Putting the address information directly in the refp means you'll probably need it in multiple places, as the most common usage is probably to store more than one data item in each storage. This strikes me as a bad idea, violating DRY. How about introducing a second part to the api? (store) creates a wrapper for the persistent address, and refp then takes one of those wrappers and the name? mike -- Mike Meyer m...@mired.org http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O ascii ribbon campaign - stop html mail -www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Extending Clojure's STM with external transactions
I'll go one step further and say that we shouldn't have to call persist namespace. It should be automatic such that a change to the state of an identity is transactionally written. Let's start with refs. We can tackle the other identities later. The API is simple. Call (refp) instead of (ref) when creating a persisted ref. Passed into the call are a persistence address (file path, DB connection string, etc.) and a name that has to be unique to that persistence address. Not all refs end up being referred to by a top-level symbol in a package, and multi-process systems are hard... Ensuring uniqueness of name is up to the programmer. Upon creation, Clojure checks to see if the refp exists in the store; if so it instantiates in memory with that state, else it uses the default in the call. In a dosync block, the function runs as normal until commit time. Then Clojure acquires a transactional write lock on each refp that is alter-ed or ensure-d. It checks the value against memory. If it's the same, commit data store changes. If not, retry after refreshing memory with the current contents of the store. If the data store commit fails, retry a number of times. If the data store commit still can't proceed, roll back the whole thing. commute and refset are slightly different, but for an initial implementation, just treat commute as an alter, and ignore refset. Does this make sense? My intention is to cover the 80% case. The implementation would necessarily be chatty, since the API is chatty. That's OK for most systems. This API has the benefit of being able to be shared across Clojure instances. It's a nice bonus. A dosync block may contain symbols pointing to refp's spanning different data stores, which isn't too hard to handle. It simply requires that if this is the case, each data store must support two- phase commit or some other distributed transaction supporting protocol. For an initial implementation, I would just throw an exception. I've begun working on an implementation using BDB. What do people think? On Aug 30, 5:02 pm, nchubrich nchubr...@gmail.com wrote: I'm not aware of any, but +1 for seeing persistence handled as part of the language. A big project and a long-term one, to be sure, but could it not be considered a goal? In my student days, I was talking to a well-known Lisper (name suppressed for fear of Google indexing) about some data structures in MIT Scheme. When I asked about saving them to disk, he said in effect, You're on your ownthat's something that \should be handled, but never is. I think people are so used to this state of affairs they forget how ugly it really is. Programming languages are like Moses without Joshua: they lead your data in the wilderness, but when it comes to finding it a permanent home, you have to talk to someone else. And these someone elses (who seem to be as numberless as the sons of Abraham) each have their own habits and ways of talking. Persistence libraries always end up warping the entire codebase; I've never succeeded in keeping them at bay. Using data with Incanter is different from ClojureQL, which is different from just using contrib.sql, and all of it is different from dealing with just Clojure. (I've never even tried Clojure + Hibernate.) You might as well rewrite the program from scratch depending on what you use. Maybe other people have had better luck; but whatever luck they have, I'm sure it is a fight to keep programs abstracted from persistence. I'd like to be able to work with mere Clojure until my program is complete, and then work in a completely separate manner on how to read and write data. Or maybe there would be off-the-shelf solutions I could plug in for different needs: low latency, high read, high write, large stores, etc. On the Clojure side, you would simply call something like persist namespace, which would save the state of your current or given namespace (unless you pass it the names of variables as well, in which case it only saves those). And to read data, you would simply require or use it into your namespace: you could choose what granularity to give first-class status: just tables, or columns as well, etc. And you could do this equally well for XML, JSON, relational data, or a graph store; your choice. And the only difference between these and ordinary variables would beheaven forbid!a disk access might happen when you deal with them! To have such a system work well, you would need to enrich the way you query Clojure datastructures. I have some ideas on that, but I'd like to make sure I'm not shouting in the dark first. I'd like to see a day when programmers need to worry about persistence about as much as they worry about garbage collection now. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members
Re: Extending Clojure's STM with external transactions
I'm resurrecting this thread from quite a while ago... I'm very interested in being able to ensure that the state of a ref is persisted as part of the same transaction that updates the ref. Performance is not important; correctness is. Has any further work been done on this front? http://groups.google.com/group/clojure/browse_thread/thread/aa22a709501a64ac -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Why I have chosen not to employ clojure
This is definitely flamebait. However, he has a point. Perhaps we should make it clear on the Getting Started page that deploying Java is a prerequisite to deploying Clojure, and include links to resources on how to do that for the platform you are on. It's presently unclear to anyone without a Java background. However, everyone who has responded is right that developer experience notwithstanding, any sysadmin worth his/her salt will know how to install a JVM. On Mar 21, 5:08 pm, Marek Kubica ma...@xivilization.net wrote: On Sun, 21 Mar 2010 10:42:12 -0800 Tim Johnson t...@johnsons-web.com wrote: Here's how I installed the flash player on my system. 1)Downloaded install_flash_player_10_linux.tar.gz 2)Unzipped libflashplayer.so 3)Copied to /usr/lib/firefox-3.5.2/plugins/ Here's how I installed the Clojure REPL on my system. 1) Downloaded clojure-1.1.0.zip 2) Unzipped clojure.jar 3) Ran java -jar clojure.jar Hardly any different. FWIW, posting on a list and declaring immediately that one will unsubscribe afterwards, not even reading replies is near-flamebait quality. regards, Marek -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
