Re: Some basic guidance to designing functional vs. state parts of app
One aspect of your question that makes it difficult to answer is that you don't explain what the repercussions should be (in terms of synchronization) when a new applicant or job enters the pool or a new skill is attributed to an applicant. Should the job<->applicant matching function be restarted? Are you simply trying to isolate the matching algorithm from updates to the data structures? On Mar 2, 10:34 pm, Sophie wrote: > As a bit of a newbie to the functional + identity/state design space, > I'm struggling a bit with where to use identity constructs (refs) and > where to stay with pure functions, and could use some guidance. Pardon > me if some of my terms are a bit off. Here is a simple hypothetical > app for matching Applicants to Jobs. > > I have several Applicants, each with some set of Skills. There are > several Jobs, each with some Requirements. There is some Match > relation between Skills and the Jobs they can match. The set of > Applicants, each of their skills, and the set of Jobs can change with > time. There is a set of Jobs that each Applicant can fill, > functionally computed from applicant skills & job requirements & > Match. > > Ok, so jobs_that_an_applicant_can_fill is a pure function, I get > that. > > Do I design a single "World" ref whose state changes with time to > different worlds, so adding a new Applicant or even adding a new Skill > to an existing Applicant results in a new World value? Or is it better > to have an "Applicants" ref and a "Jobs" ref that refer to different > sets of those er.. things? > > Can each Applicant have a "skills" ref, whose state changes to > different sets of Skills? Should I design it this way? > > How do I choose? What are the trade-offs? > > Any and all guidance, insights, advice etc. welcome! > > Thanks! -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
On Mar 2, 8:34 pm, Sophie wrote: > Do I design a single "World" ref whose state changes with time to > different worlds, so adding a new Applicant or even adding a new Skill > to an existing Applicant results in a new World value? Or is it better > to have an "Applicants" ref and a "Jobs" ref that refer to different > sets of those er.. things? > It's simpler to use refs close to the object that will change; that makes the code to mutate the value simpler; if the world is the only mutable object, then the code to change something in there will increase with the depth of the mutation. > Can each Applicant have a "skills" ref, whose state changes to > different sets of Skills? Should I design it this way? Yes; and if the kills set in independent of other state it may be an atom and you avoid a dosync if the program is single-threaded. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
On Wed, 3 Mar 2010 14:20:56 -0800 (PST) Armando Blancas wrote: > > > On Mar 2, 8:34 pm, Sophie wrote: > > Do I design a single "World" ref whose state changes with time to > > different worlds, so adding a new Applicant or even adding a new Skill > > to an existing Applicant results in a new World value? Or is it better > > to have an "Applicants" ref and a "Jobs" ref that refer to different > > sets of those er.. things? > > > > It's simpler to use refs close to the object that will change; that > makes > the code to mutate the value simpler; if the world is the only mutable > object, > then the code to change something in there will increase with the > depth > of the mutation. I'm not sure it's simpler: update-in makes reaching deep into a structure to create a new structure with that one change in it easy, and is probably more efficient than trying to do the same thing by hand. More importantly, if you break your refs up into smaller objects, you should get better concurrency, as you can change two such refs in two different threads without a problem. If you put the two in a structure in the same ref, then you can't update one while the other is being worked on. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
On 3 March 2010 04:34, Sophie wrote: > As a bit of a newbie to the functional + identity/state design space, > I'm struggling a bit with where to use identity constructs (refs) and > where to stay with pure functions, and could use some guidance. Pardon > me if some of my terms are a bit off. Here is a simple hypothetical > app for matching Applicants to Jobs. > > I have several Applicants, each with some set of Skills. There are > several Jobs, each with some Requirements. There is some Match > relation between Skills and the Jobs they can match. The set of > Applicants, each of their skills, and the set of Jobs can change with > time. There is a set of Jobs that each Applicant can fill, > functionally computed from applicant skills & job requirements & > Match. > > Ok, so jobs_that_an_applicant_can_fill is a pure function, I get > that. > > Do I design a single "World" ref whose state changes with time to > different worlds, so adding a new Applicant or even adding a new Skill > to an existing Applicant results in a new World value? Or is it better > to have an "Applicants" ref and a "Jobs" ref that refer to different > sets of those er.. things? > > Can each Applicant have a "skills" ref, whose state changes to > different sets of Skills? Should I design it this way? > > How do I choose? What are the trade-offs? If you have a single value representing the whole world, then it seems that protecting it with an atom would be the simplest and most idiomatic solution. Refs are for co-ordinated change, i.e. where you want to update multiple values at the same time. For a single value there seems to be little reason to adopt refs, though I've often wondered why Stuart Halloway's book gives an example updating a single ref of messages for a chat application. If you partition your data more, into several values, then you should use refs to co-ordinate the change. The more granular you make the refs, the more you trade complexity of design for potential throughput under concurrent load, as finer grained refs are likely to mean less collisions during transactions (unless of course you update every ref in every transaction - which sounds like it'd be the worst case scenario). R. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
For a single value there seems to be little reason to adopt refs, though I've often wondered why Stuart Halloway's book gives an example updating a single ref of messages for a chat application. I seem to recall atoms being the last reference type to be introduced. They might not have existed when Stuart was writing. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
On Mar 2, 11:34 pm, Sophie wrote: > > How do I choose? What are the trade-offs? > > Any and all guidance, insights, advice etc. welcome! > > Thanks! To me, it seems like you have two orthogonal pieces of data, and a function that builds a report from that data. You have a set of job listings, and a set of applicants. I see no reason why these need to be coordinated (ref). The addition of a job listing is totally independent of an applicant updating her resume. So it's uncoordinated (atom). That is, the job listings are always in a consistent state no matter what I do to the applicant data. Said another way, there's nothing I can do to the applicant data that can put the job listings in an inconsistent state. As for the match function. Well it needs to get a snapshot of both pieces of data, and then determine what applicants match up with each job listing. While this function is running a job could be added, an applicant could update her resume, or maybe an applicant will remove her resume but your function won't see any of that, but who cares? It will see a consistent snapshot of both pieces of data, and from those it will build the result. As of time T1, here is your latest and greatest match report. Now, say you want the match report built every time a job listing or applicant data is changed. This way the 'latest' report is always in memory and ready to go. Well, in this case, you now need to coordinate the update of job listings or applicant data with the generation of the report. You never want your program to enter a state where the job listings are in state L101 and applicants are in state A54 and your report is based on the combination of (L101, A53). That's an inconsistent state because your report is not based on the latest data. In this case, you'd probably want to use refs to make sure the your job/applicant data is coordinated with your report data. However, if your data is constantly changing, and the match function takes a while to run then this may be very slow. Take what I say with a grain of salt as I'm also fairly new to this stuff. At the end of the day, experience is probably the best teacher. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
Thank you all, the replies so far and the questions have already deepened my understanding considerably! Looking forward to more. I think a bit more discussion like this (not necessarily my quite skimpy example) would be quite valuable to many like me. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
Someone could (and hopefully will) write a multi-hundred-page book exclusively on Clojure's approach to state. I simply didn't have enough space while overviewing the whole language. If you want an identity that is isolated from whatever else is (or might in the future) happen in your app, prefer atoms. However, if it does (or will) coordinate with other identities, use refs. I guess switching between these should be a feature of the Clojure Refactoring IDE. :-) Stu For a single value there seems to be little reason to adopt refs, though I've often wondered why Stuart Halloway's book gives an example updating a single ref of messages for a chat application. I seem to recall atoms being the last reference type to be introduced. They might not have existed when Stuart was writing. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
On Wed, Mar 3, 2010 at 5:20 PM, Rick Moynihan wrote: > > If you have a single value representing the whole world, then it seems > that protecting it with an atom would be the simplest and most > idiomatic solution. ... > For a single value > there seems to be little reason to adopt refs, ... I disagree. The problem with atoms is that updating an atom is a side-effect that cannot (typically) participate safely in a transaction. It might be retried, so if for example, you want to increment the contents of an atom, it might get incremented multiple times which is generally not okay (if you're storing something in a cache, then a retry might be okay). If you make a design decision to use an atom, you are effectively committing to never, ever being able to update that atom within a transaction. In my opinion, this has major implications and is a decision that should not be taken lightly. In a language like Clojure, we should strive to write things that are as side-effect free as possible, so they can participate in transactions. Using atoms runs the risk of creating headaches later, so unless you're 100% sure the semantics of your program won't change if they are retried, or if you're 100% sure you'll never want to change the contents of the atom as part of a sequence of logic involving setting other refs, don't use atoms. refs can do everything atoms can do and more. They are slightly slower, perhaps, but much safer. So unless you have an explicit performance reason to use atoms and have thought through the implications, the simplest design choice is to prefer refs. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
I don't follow your logic. Modifications of refs within a transaction retry too. On Mar 5, 11:56 am, Mark Engelberg wrote: > On Wed, Mar 3, 2010 at 5:20 PM, Rick Moynihan wrote: > > > > > If you have a single value representing the whole world, then it seems > > that protecting it with an atom would be the simplest and most > > idiomatic solution. > > ... > > > For a single value > > there seems to be little reason to adopt refs, ... > > I disagree. The problem with atoms is that updating an atom is a > side-effect that cannot (typically) participate safely in a transaction. It > might be retried, so if for example, you want to increment the contents of > an atom, it might get incremented multiple times which is generally not okay > (if you're storing something in a cache, then a retry might be okay). > > If you make a design decision to use an atom, you are effectively committing > to never, ever being able to update that atom within a transaction. > > In my opinion, this has major implications and is a decision that should not > be taken lightly. In a language like Clojure, we should strive to write > things that are as side-effect free as possible, so they can participate in > transactions. Using atoms runs the risk of creating headaches later, so > unless you're 100% sure the semantics of your program won't change if they > are retried, or if you're 100% sure you'll never want to change the contents > of the atom as part of a sequence of logic involving setting other refs, > don't use atoms. > > refs can do everything atoms can do and more. They are slightly slower, > perhaps, but much safer. So unless you have an explicit performance reason > to use atoms and have thought through the implications, the simplest design > choice is to prefer refs. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
On Fri, Mar 5, 2010 at 4:08 PM, eyeris wrote: > I don't follow your logic. Modifications of refs within a transaction > retry too. > He's saying that atoms don't participate in the STM. If an atom depends on the value of some refs, your program might be put into an inconsistent state if the transaction is retried and you're changing the value of the atom in the transaction. In general using an atom means you're sure that you'll never want to perform an atomic change of both it and some other mutable reference in your program. That's my understanding of it anyway. David -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
When you increment the contents of an atom, for example, it happens immediately. If it is inside a transaction, and the transaction retries, the contents will be incremented multiple times. Refs, on the other hand, retry from the initial state at the beginning of the transaction, so as long as there are no external side effects, you get the illusion that it only happened once. Essentially, changing an atom is like other side effects (e.g., display or file output), and therefore cannot be safely used inside a transaction. On Fri, Mar 5, 2010 at 1:08 PM, eyeris wrote: > I don't follow your logic. Modifications of refs within a transaction > retry too. > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
On 5 March 2010 17:56, Mark Engelberg wrote: > > On Wed, Mar 3, 2010 at 5:20 PM, Rick Moynihan > wrote: >> >> If you have a single value representing the whole world, then it seems >> that protecting it with an atom would be the simplest and most >> idiomatic solution. > >> For a single value there seems to be little reason to adopt refs, ... > > I disagree. The problem with atoms is that updating an atom is a > side-effect that cannot (typically) participate safely in a transaction. It > might be retried, so if for example, you want to increment the contents of > an atom, it might get incremented multiple times which is generally not okay > (if you're storing something in a cache, then a retry might be okay). > > If you make a design decision to use an atom, you are effectively committing > to never, ever being able to update that atom within a transaction. This is a good point, one that I hadn't considered and arguably also non-obvious! Though all is not necessarily lost when state you don't control is held in an atom, as you can wrap it inside an agent which will be co-ordinated with the STM, ensuring safe modification in the event of retries. > refs can do everything atoms can do and more. They are slightly slower, > perhaps, but much safer. So > unless you have an explicit performance reason to use atoms and have thought > through the > implications, the simplest design choice is to prefer refs. My simple (uncontended) tests seem to indicate that atoms are about 4x faster than refs (though I'd love to see a more comprehensive benchmark). Regardless I'll certainly be more cautious about using atoms in the future, as refs are without doubt more composable. R. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Some basic guidance to designing functional vs. state parts of app
> > If you make a design decision to use an atom, you are effectively committing > > to never, ever being able to update that atom within a transaction. > > refs can do everything atoms can do and more. They are slightly slower, > > perhaps, but much safer. So > > unless you have an explicit performance reason to use atoms and have > > thought through the > > implications, the simplest design choice is to prefer refs. I agree. At first I was always mixing and matching atoms and refs, but it seems like a case of premature optimization. If I understand correctly atoms don't give you any benefit over refs besides performance. As you grow a program it's virtually impossible to predict which variables will end up needing coordinated updates, so why not just use refs and count on the STM for everything by default? Once you get to the performance optimization phase you can start converting refs to atoms, add type hints where appropriate, and make use of transients in your inner loops. The ref->atom refactoring would be very cool, and if you start by using refs for everything it would be easy as a first pass to turn any ref that is always modified alone into an atom. -Jeff -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
