This is always tricky - the other dimension is that the transaction can span multiple components which should all be ignorant of each other, so a collecting fn (like James’ ‘do-things*’) isn’t feasible.
For me, I have the service create a tx and then either I pass in the tx instead of the db param (e.g. (defn my-comp [tx …])) or I bind the tx to a dynamic var but only allow access to it via a (defn in-tx [fn] (binding…. (fn *tx*))). Both of these styles are restrictive, but good enough. Another style I run into is when I have a Protocol which doesn’t care at all about persistence. For example, my event store has an in-memory implementation and a JDBC backed one. What can the signature look like: - everything takes in a db which some implementations ignore - yuck - the signature includes a magical ‘environment’ param - yuck - the signature is focused and the JDBC implementation uses (in-tx #(…)) This is definitely an area where pragmatism meets principle. I am away from the code at the moment, but is there any reason why the dynamic connection can’t be private? This goes some way to providing safety. > On 31 Jul 2015, at 09:54, J. Pablo Fernández <[email protected]> wrote: > > Hello James, > > Thanks for your answer. I do understand your point. Pure functions are easier > to reason about and my use of dynamic here breaks that purity. I'm not doing > it lightly. It already happened to me, that one of those functions that was > running inside the transaction, was not passed the transaction connection and > instead got the global one and the failure was silent and very hard to debug, > and this was with a project that has less than 200 lines of code. I'm trying > to find patterns that will work when this project has 200k lines of code. > > For me, the thing is, I have a traditional relational database here, this is > already far from pure. For example, calling (db/create-user > "[email protected] <mailto:[email protected]>") twice will not only not > return the same thing the second time, it'll actually raise an exception the > second time. Also, the database connection is *global state* unless each > function creates its own connection, which would be terrible. So, this global > state also breaks functional purity. > > The problem with the second aspect of breaking purity as far as I can see is > this: at some point, this global state has to be picked up and used, so at > some point a function will *not* get a database connection passed to it but > *will* access the database by using this global connection. I haven't > advanced this project enough to say this with 100% certainty, but, I think > there's going to be more than one function like that and at some point I'll > need to have one inside the other so I need them to be composable. Let me > show you a naive example: > > db/create-user is the low level database function that creates a record in > the user table > user/create is the function used to create a user, it takes care of, for > example, encrypting the password. > account/register is the function to register a new user, it takes care of > creating a user but also validation, sending a welcome email and so on. > > So each function calls the predecessor there and would pass the database > connection, account/register, being the entry point, would grab it from the > global state so it doesn't get a connection passed to it. So far, a lot of it > looks like pure functions (let's ignore the fact that a database breaks that > purity). The problem arises when I get another function, account/invite, that > is used to register a bunch of people one after the other, so that > account/invite would call account/register many times. The problem is that > account/invite *can't* start a transaction and have account/register and all > its inner functions use that transaction when that makes a lot of sense. > > To make account/register composable it needs to accept an optional database > connection and use that one if it's present, or the global one if it's not. > Every time a function does that there's a high risk of picking the wrong > database and account/invite and account/register shouldn't be dealing with > database connection management. That feels to me like lower level details > leaked into higher level abstractions. > > Now, I know this is a naive example and you could push the grabbing of the > global connection higher and higher, as long as the example is naive and > simple like this, but it does represent what in my experience is the reality > of web application development at least in another languages and I haven't > seen anything to make me think Clojure will be radically different here (at > least when using a patterns such as compojure). > > So yes, it's not purely function but with a database that's already > impossible and if I wanted purely functional I would probably be using > Haskell instead of Clojure. What I like about Clojure is this: > > Clojure is a practical language that recognizes the occasional need to > maintain a persistent reference to a changing value and provides 4 distinct > mechanisms for doing so in a controlled manner - Vars, Refs, Agents and Atoms. > > I'm just trying to be practical here. But I'm new and I'm not sure if an atom > that is a dynamic var has some hidden issues that I'm not seeing (other than > the fact of it being state that changes and that I have to manage explicitly > because the language is not protecting me from shooting myself in the foot > with it). > > Does it make sense? > > > > On 31 July 2015 at 03:16, James Reeves <[email protected] > <mailto:[email protected]>> wrote: > On 31 July 2015 at 01:44, J. Pablo Fernández <[email protected] > <mailto:[email protected]>> wrote: > I found passing around the database connection to each function that uses it > very error prone when you are using transactions as passing the wrong one > could mean a query runs outside the transaction when in the source code it is > inside the with-db-transaction function. So I ended up defining the db > namespace like this: > > (ns db) > > (defonce ^:dynamic conn (atom nil)) > > (defn connect! > (reset conn (generate-new-connection))) > > (defn run-query > [query] (run-query query @conn) > [query conn] (run-the-query-in-connection query conn)) > > This style of code is generally considered to be unidiomatic in Clojure. The > reason for this is that it significantly increases complexity, and Clojure is > about reducing complexity where possible. > > Consider a function like: > > (defn find-by-id [conn id] > (sql/query conn ["SELECT * FROM foo WHERE id = ?" id])) > > The output of this function is affected by its arguments (and by the state of > the database the connection is associated with), which is passed by its > caller. > > Now consider a function like: > > (defn find-by-id [id] > (sql/query @conn ["SELECT * FROM foo WHERE id = ?" id])) > > The output of this function is affected by its arguments... and by anything > that touches the global conn var, which could literally be anything in your > program, in any namespace, in any function, in any thread. > > The more ways in which a function has, the more "complex" it is. This is why > Clojure prefers immutable data over mutable data, and why function arguments > are generally preferred over dynamic vars. > > The problem of accidentally calling a database connection directly inside a > transaction is a difficult one, but I don't think the solution is to add more > complexity. An alternative solution would be to take the original database > connection out of scope, by moving your transaction code to a separate > function: > > (defn do-things* [tx] > (do-foo tx) > (do bar tx) > (do baz tx)) > > (defn do-things [db-spec] > (sql/with-db-transaction [tx db-spec] > (do-things* tx))) > > If this is still too prone to error, you could also automate this pattern > with a function: > > (defn wrap-transaction [f] > (fn [db-spec & args] > (sql/with-db-transaction [tx db-spec] > (apply f tx args)))) > > (def do-things > (wrap-transaction do-things*)) > > - James > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to [email protected] > <mailto:[email protected]> > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > [email protected] > <mailto:clojure%[email protected]> > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > <http://groups.google.com/group/clojure?hl=en> > --- > You received this message because you are subscribed to a topic in the Google > Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/fRi554wbPSk/unsubscribe > <https://groups.google.com/d/topic/clojure/fRi554wbPSk/unsubscribe>. > To unsubscribe from this group and all its topics, send an email to > [email protected] > <mailto:[email protected]>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. > > > > -- > J. Pablo Fernández <[email protected] <mailto:[email protected]>> > (http://pupeno.com <http://pupeno.com/>) > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to [email protected] > <mailto:[email protected]> > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > [email protected] > <mailto:[email protected]> > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > <http://groups.google.com/group/clojure?hl=en> > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to [email protected] Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
