Re: Style, Efficiency, and updating nested structure

[Erik Assum]

I would like to argue that your code has to be pretty inefficient if it's 
creating a bigger performance impact than fetching from the database. 

Erik. 
-- 
i farta

> Den 12. nov. 2015 kl. 15.36 skrev Dave Tenny :
> 
> Thanks for the replies, as always lots of interesting ways to do it in 
> clojure.  No zipper sugestions?
> 
> For my own take, I happen to like the last suggestion (Gareth's) the most I 
> think, of the ones offered so far.  It has a lispy elegance and is still 
> relatively efficient in traversals and memory allocation, compared to some 
> solutions (particular with a revision to the merge-roles implementation to 
> eliminate all the set creation, see next sentence).
> 
> Some of you rewrote the problem a bit.  For example, the 'merge-roles' 
> conversion to sets is very expensive, especially since I know that the input 
> in my original problem isn't going to offer duplicate project rules for a 
> given project ID.
> 
> On efficiency in general, it is perhaps a generational thing that people say 
> "don't prematurely optimize" so often and then cons up intermediate objects 
> with wild abandon.  And while a well crafted stop and copy garbage collector 
> can be have negligable GC time given sufficient memory free space, 
> allocations are still not free even in the best case, not to mention the time 
> to build the object being allocated (like hashing/comparing all the values in 
> a set).
> 
> I'd like to suggest that there is a difference between premature optimization 
> and avoiding flagrantly inefficient code.
> If you code so that you routinely copy sequences and nested data structures 
> without a care, then in a serious/large system
> your performance will have death by a thousand cuts.  The big O matters, at 
> least in some applications.
> 
> As indicated in this problem, the input is coming from a database, i.e. a 
> potentially large record source. My bias is that per-record processing is 
> always best kept to a minimum.  So inverting maps, copying sequences, 
> creating sets and such _for every record_ is just a big "nope" for me.And 
> that doesn't even consider what went into the retrieval of the records from 
> clojure.java.jdbc/query, where every record comes back as a map unless you do 
> something to prevent it.
> 
> Meanwhile there were good suggestions for both abstraction and efficiency 
> here, thanks.  I often forget about lazy-seq and have a tendency to lean 
> toward loop/recur, probably from too many years using Common Lisp.  And of 
> course elimination of the mutable change detection hack is good too.So 
> nice to see the different ways people tackle the problem.
> 
> 
>> On Wed, Nov 11, 2015 at 10:02 PM, Gareth Clapperton  
>> wrote:
>> Hey Daves
>> 
>> I too like the map idea (and using sets for roles for that matter). But 
>> sticking with collections, I might do something like this
>> 
>> (defn upsert 
>>   "Updates or inserts v into coll"
>>   [key-fn update-fn v coll]
>>   (let [k(key-fn v)]  
>> (letfn [(step [v [x & xs]]
>>   (lazy-seq
>> (cond
>>   (nil? x) (when v [v])
>>   (and (some? v) (= (key-fn x) k)) (cons (update-fn x v) 
>> (step nil xs))
>>   :default (consx
>> (step  v  xs)]
>>   (step v coll
>>   
>> (defn merge-roles [p1 {:keys [project_roles] :as p2}]
>>   (update-in p1 [:project_roles] #(vec (clojure.set/union (set %) (set 
>> project_roles)
>> 
>> (upsert :project_id merge-roles prj-role prj-roles)
>> 
>> Gareth
>> 
>> 
>>> On Wednesday, November 11, 2015 at 4:25:51 PM UTC-5, Dave Tenny wrote:
>>> A colleague and I are debating various things clojure as we were exploring 
>>> alternative ways to solve a problem.
>>> 
>>> Here's the description of the problem that a particular function is trying 
>>> to solve, and the first implementation of it.
>>> 
>>> (defn update-roles 
>>>   "Given a vector of maps of the form {:project_id N :project_name S 
>>> :project_roles [...roles...]}
>>>   if there is already a map for the indicated project id/name, add new-role 
>>> to it and returned
>>>   a copy the updated input vector, otherwise return a vector with a new map 
>>> entry for the newly found
>>>   project and initial role.  This function is basically aggregating tuples 
>>> from the database."
>>>   [projects project-id project-name new-role]
>>>   (let [updated? (atom nil)
>>> 
>>> projects* (mapv (fn [m] 
>>>   (if (= (:project_id m) project-id)
>>> (do (reset! updated? true)
>>> (assoc m :project_roles (conj 
>>> (:project_roles m) new-role)))
>>> m))
>>> projects)]
>>> (if @updated?
>>>   projects*
>>>   (conj projects {:project_id project-id :project_name project-name 
>>> :project_roles [new-role]})

Re: Style, Efficiency, and updating nested structure

[Dave Tenny]

Thanks for the replies, as always lots of interesting ways to do it in
clojure.  No zipper sugestions?

For my own take, I happen to like the last suggestion (Gareth's) the most I
think, of the ones offered so far.  It has a lispy elegance and is still
relatively efficient in traversals and memory allocation, compared to some
solutions (particular with a revision to the merge-roles implementation to
eliminate all the set creation, see next sentence).

Some of you rewrote the problem a bit.  For example, the 'merge-roles'
conversion to sets is very expensive, especially since I know that the
input in my original problem isn't going to offer duplicate project rules
for a given project ID.

On efficiency in general, it is perhaps a generational thing that people
say "don't prematurely optimize" so often and then cons up intermediate
objects with wild abandon.  And while a well crafted stop and copy garbage
collector can be have negligable GC time given sufficient memory free
space, allocations are still not free even in the best case, not to mention
the time to build the object being allocated (like hashing/comparing all
the values in a set).

I'd like to suggest that there is a difference between premature
optimization and avoiding flagrantly inefficient code.
If you code so that you routinely copy sequences and nested data structures
without a care, then in a serious/large system
your performance will have death by a thousand cuts.  The big O matters, at
least in some applications.

As indicated in this problem, the input is coming from a database, i.e. a
potentially large record source. My bias is that per-record processing is
always best kept to a minimum.  So inverting maps, copying sequences,
creating sets and such _*for every record*_ is just a big "nope" for me.
 And that doesn't even consider what went into the retrieval of the records
from clojure.java.jdbc/query, where every record comes back as a map unless
you do something to prevent it.

Meanwhile there were good suggestions for both abstraction and efficiency
here, thanks.  I often forget about lazy-seq and have a tendency to lean
toward loop/recur, probably from too many years using Common Lisp.  And of
course elimination of the mutable change detection hack is good too.So
nice to see the different ways people tackle the problem.


On Wed, Nov 11, 2015 at 10:02 PM, Gareth Clapperton 
wrote:

> Hey Daves
>
> I too like the map idea (and using sets for roles for that matter). But
> sticking with collections, I might do something like this
>
> (defn upsert
>   "Updates or inserts v into coll"
>   [key-fn update-fn v coll]
>   (let [k(key-fn v)]
> (letfn [(step [v [x & xs]]
>   (lazy-seq
> (cond
>   (nil? x) (when v [v])
>   (and (some? v) (= (key-fn x) k)) (cons (update-fn x v)
> (step nil xs))
>   :default (consx
>  (step  v  xs)]
>   (step v coll
>
> (defn merge-roles [p1 {:keys [project_roles] :as p2}]
>   (update-in p1 [:project_roles] #(vec (clojure.set/union (set %) (set
> project_roles)
>
> (upsert :project_id merge-roles prj-role prj-roles)
>
> Gareth
>
>
> On Wednesday, November 11, 2015 at 4:25:51 PM UTC-5, Dave Tenny wrote:
>>
>> A colleague and I are debating various things clojure as we were
>> exploring alternative ways to solve a problem.
>>
>> Here's the description of the problem that a particular function is
>> trying to solve, and the first implementation of it.
>>
>> (defn update-roles
>>   "Given a vector of maps of the form {:project_id N :project_name S
>> :project_roles [...roles...]}
>>   if there is already a map for the indicated project id/name, add
>> new-role to it and returned
>>   a copy the updated input vector, otherwise return a vector with a new
>> map entry for the newly found
>>   project and initial role.  This function is basically aggregating
>> tuples from the database."
>>   [projects project-id project-name new-role]
>>   (let [updated? (atom nil)
>>
>> projects* (mapv (fn [m]
>>   (if (= (:project_id m) project-id)
>> (do (reset! updated? true)
>> (assoc m :project_roles (conj (:project_roles
>> m) new-role)))
>> m))
>> projects)]
>> (if @updated?
>>   projects*
>>   (conj projects {:project_id project-id :project_name project-name 
>> :project_roles
>> [new-role]}
>>
>>
>> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
>> ]}] 2 "Two" :edit)
>> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
>> {:project_id
>> 2, :project_name "Two", :project_roles [:edit]}]
>> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
>> ]}] 1 "Two" :edit)
>> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]
>>
>>
>>
>>

Re: Style, Efficiency, and updating nested structure

[Gareth Clapperton]

Hey Daves

I too like the map idea (and using sets for roles for that matter). But 
sticking with collections, I might do something like this

(defn upsert 
  "Updates or inserts v into coll"
  [key-fn update-fn v coll]
  (let [k(key-fn v)]  
(letfn [(step [v [x & xs]]
  (lazy-seq
(cond
  (nil? x) (when v [v])
  (and (some? v) (= (key-fn x) k)) (cons (update-fn x v) 
(step nil xs))
  :default (consx   
 (step  v  xs)]
  (step v coll
  
(defn merge-roles [p1 {:keys [project_roles] :as p2}]
  (update-in p1 [:project_roles] #(vec (clojure.set/union (set %) (set 
project_roles)

(upsert :project_id merge-roles prj-role prj-roles)

Gareth

On Wednesday, November 11, 2015 at 4:25:51 PM UTC-5, Dave Tenny wrote:
>
> A colleague and I are debating various things clojure as we were exploring 
> alternative ways to solve a problem.
>
> Here's the description of the problem that a particular function is trying 
> to solve, and the first implementation of it.
>
> (defn update-roles 
>   "Given a vector of maps of the form {:project_id N :project_name S 
> :project_roles [...roles...]}
>   if there is already a map for the indicated project id/name, add 
> new-role to it and returned
>   a copy the updated input vector, otherwise return a vector with a new 
> map entry for the newly found
>   project and initial role.  This function is basically aggregating tuples 
> from the database."
>   [projects project-id project-name new-role]
>   (let [updated? (atom nil)
>
> projects* (mapv (fn [m] 
>   (if (= (:project_id m) project-id)
> (do (reset! updated? true)
> (assoc m :project_roles (conj (:project_roles 
> m) new-role)))
> m))
> projects)]
> (if @updated?
>   projects*
>   (conj projects {:project_id project-id :project_name project-name 
> :project_roles 
> [new-role]}
>
>
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 2 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 1 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]
>
>
>
> Now here's another implementation:
>
> (defn update-or-insert-project-role
>   [prj-roles prj-role]
>   (let [to-insert-prj-id (:project_id prj-role)
> by-pid   (group-by :project_id prj-roles)]
> (case (get by-pid to-insert-prj-id)
>   nil (conj prj-roles prj-role)
>   (->> (update-in by-pid [to-insert-prj-id 0 :project_roles] #(apply conj 
> % (:project_roles prj-role)))
>(mapcat second)
>(into [])
>
> ;; (def prj-roles [{:project_id 1, :project_name "One", :project_roles 
> [:own]} {:project_id 3 :project_name "Three" :project_roles [:edit]}])
> ;; (update-or-insert-project-role prj-roles {:project_id 2 :project_name 
> "Two" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]} {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-or-insert-project-role prj-roles {:project_id 1 :project_name 
> "One" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]}]
>
>
>
>
> The function is called in a loop to aggregate rows from a database, though 
> it isn't an overriding concern, we're not going millions of records in this 
> case.
>
> The first thing my colleague and I disagreed on was the calling sequence, 
> arguing over which is more readable.
> The second thing was whether efficiency in this context is really 
> important, or whether it's all moot in clojure.  
>
> Finally, I'm sure there's a better way, probably with Zippers or 
> something, but neither of us have used them. Suggestions for the stylistic 
> and operational epitome of clojure expression on this routine are welcome!
>
> Superficially, and probably incorrect in multiple ways, here is a poor 
> attempt at breaking down efficiency in terms of search/traversal and memory 
> allocations.  This was done by someone with no knowledge of clojure 
> internals (including the library implementations of the called functions).
>
> ;; Comparing the two routines per function call, for existing project case 
> (i.e. where roles are updated)
> ;;
> ;; Assuming each routine allocates new vector for new-role placement in 
> existing project
> ;; and MapEntry for assoc of new vector on project_roles, so they aren't 
> included in allocations 
> ;; below since both routines have to 

Re: Style, Efficiency, and updating nested structure

[James Reeves]

In general, it's not helpful to reach for optimisations like mutability
without a benchmarking tool to tell you how much of a difference you're
making.

I'd be tempted to write something like:

  (defn find-project-index [projects id]
(first (keep-indexed (fn [i p] (if (= (:project_id p) id) i))
projects)))

  (defn update-projects [projects project]
(if-let [idx (find-project-index projects (:project_id project))]
  (update-in projects [idx :project-roles] into (:project_roles
project))
  (conj projects project))

But it seems a better idea to keep the projects in a map referenced by id.
Then find-project-index could be replaced with a key lookup.

- James

On 11 November 2015 at 21:25, Dave Tenny  wrote:

> A colleague and I are debating various things clojure as we were exploring
> alternative ways to solve a problem.
>
> Here's the description of the problem that a particular function is trying
> to solve, and the first implementation of it.
>
> (defn update-roles
>   "Given a vector of maps of the form {:project_id N :project_name S
> :project_roles [...roles...]}
>   if there is already a map for the indicated project id/name, add
> new-role to it and returned
>   a copy the updated input vector, otherwise return a vector with a new
> map entry for the newly found
>   project and initial role.  This function is basically aggregating tuples
> from the database."
>   [projects project-id project-name new-role]
>   (let [updated? (atom nil)
>
> projects* (mapv (fn [m]
>   (if (= (:project_id m) project-id)
> (do (reset! updated? true)
> (assoc m :project_roles (conj (:project_roles
> m) new-role)))
> m))
> projects)]
> (if @updated?
>   projects*
>   (conj projects {:project_id project-id :project_name project-name 
> :project_roles
> [new-role]}
>
>
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 2 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 1 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]
>
>
>
> Now here's another implementation:
>
> (defn update-or-insert-project-role
>   [prj-roles prj-role]
>   (let [to-insert-prj-id (:project_id prj-role)
> by-pid   (group-by :project_id prj-roles)]
> (case (get by-pid to-insert-prj-id)
>   nil (conj prj-roles prj-role)
>   (->> (update-in by-pid [to-insert-prj-id 0 :project_roles] #(apply conj 
> % (:project_roles prj-role)))
>(mapcat second)
>(into [])
>
> ;; (def prj-roles [{:project_id 1, :project_name "One", :project_roles 
> [:own]} {:project_id 3 :project_name "Three" :project_roles [:edit]}])
> ;; (update-or-insert-project-role prj-roles {:project_id 2 :project_name 
> "Two" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]} {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-or-insert-project-role prj-roles {:project_id 1 :project_name 
> "One" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]}]
>
>
>
>
> The function is called in a loop to aggregate rows from a database, though
> it isn't an overriding concern, we're not going millions of records in this
> case.
>
> The first thing my colleague and I disagreed on was the calling sequence,
> arguing over which is more readable.
> The second thing was whether efficiency in this context is really
> important, or whether it's all moot in clojure.
>
> Finally, I'm sure there's a better way, probably with Zippers or
> something, but neither of us have used them. Suggestions for the stylistic
> and operational epitome of clojure expression on this routine are welcome!
>
> Superficially, and probably incorrect in multiple ways, here is a poor
> attempt at breaking down efficiency in terms of search/traversal and memory
> allocations.  This was done by someone with no knowledge of clojure
> internals (including the library implementations of the called functions).
>
> ;; Comparing the two routines per function call, for existing project case
> (i.e. where roles are updated)
> ;;
> ;; Assuming each routine allocates new vector for new-role placement in
> existing project
> ;; and MapEntry for assoc of new vector on project_roles, so they aren't
> included in allocations
> ;; below since both routines have to do it.
> ;;
> ;; Note that x-element map allocates storage for map and map-entries or
> clojure equivalent.
> ;; (and more expensive than an x-element vector, of course).
> ;;
>

Re: Style, Efficiency, and updating nested structure

[Rangel Spasov]

I have been using Specter https://github.com/nathanmarz/specter recently 
with great success for doing all kinds of transformation:

(let [project-id 10
  new-role :r3
  projects
  [{:project-id 1 :project-roles [:r1]}
   {:project-id 2 :project-roles [:r2]}]

  ^IPersistentVector project-exists? 
  (s/select [s/ALL :project-id #(= % project-id)] projects)]
  
  (if (not (empty? project-exists?))
;project exists, transform with specter
(s/transform
  [s/ALL
   (fn [x] (= project-id (get x :project-id)))
   :project-roles]
  (fn [x]
(conj x new-role))
  projects)
;doesn't exist, simply conj to the vector
(conj projects {:project-id project-id :project-roles [new-role]})))

On Wednesday, November 11, 2015 at 1:25:51 PM UTC-8, Dave Tenny wrote:
>
> A colleague and I are debating various things clojure as we were exploring 
> alternative ways to solve a problem.
>
> Here's the description of the problem that a particular function is trying 
> to solve, and the first implementation of it.
>
> (defn update-roles 
>   "Given a vector of maps of the form {:project_id N :project_name S 
> :project_roles [...roles...]}
>   if there is already a map for the indicated project id/name, add 
> new-role to it and returned
>   a copy the updated input vector, otherwise return a vector with a new 
> map entry for the newly found
>   project and initial role.  This function is basically aggregating tuples 
> from the database."
>   [projects project-id project-name new-role]
>   (let [updated? (atom nil)
>
> projects* (mapv (fn [m] 
>   (if (= (:project_id m) project-id)
> (do (reset! updated? true)
> (assoc m :project_roles (conj (:project_roles 
> m) new-role)))
> m))
> projects)]
> (if @updated?
>   projects*
>   (conj projects {:project_id project-id :project_name project-name 
> :project_roles 
> [new-role]}
>
>
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 2 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 1 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]
>
>
>
> Now here's another implementation:
>
> (defn update-or-insert-project-role
>   [prj-roles prj-role]
>   (let [to-insert-prj-id (:project_id prj-role)
> by-pid   (group-by :project_id prj-roles)]
> (case (get by-pid to-insert-prj-id)
>   nil (conj prj-roles prj-role)
>   (->> (update-in by-pid [to-insert-prj-id 0 :project_roles] #(apply conj 
> % (:project_roles prj-role)))
>(mapcat second)
>(into [])
>
> ;; (def prj-roles [{:project_id 1, :project_name "One", :project_roles 
> [:own]} {:project_id 3 :project_name "Three" :project_roles [:edit]}])
> ;; (update-or-insert-project-role prj-roles {:project_id 2 :project_name 
> "Two" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]} {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-or-insert-project-role prj-roles {:project_id 1 :project_name 
> "One" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]}]
>
>
>
>
> The function is called in a loop to aggregate rows from a database, though 
> it isn't an overriding concern, we're not going millions of records in this 
> case.
>
> The first thing my colleague and I disagreed on was the calling sequence, 
> arguing over which is more readable.
> The second thing was whether efficiency in this context is really 
> important, or whether it's all moot in clojure.  
>
> Finally, I'm sure there's a better way, probably with Zippers or 
> something, but neither of us have used them. Suggestions for the stylistic 
> and operational epitome of clojure expression on this routine are welcome!
>
> Superficially, and probably incorrect in multiple ways, here is a poor 
> attempt at breaking down efficiency in terms of search/traversal and memory 
> allocations.  This was done by someone with no knowledge of clojure 
> internals (including the library implementations of the called functions).
>
> ;; Comparing the two routines per function call, for existing project case 
> (i.e. where roles are updated)
> ;;
> ;; Assuming each routine allocates new vector for new-role placement in 
> existing project
> ;; and MapEntry for assoc of new vector on project_roles, so they aren't 
> included in allocations 
> ;; below since both routines have to do it.
> ;;
> ;; Note that x-element map allocates storage for

Re: Style, Efficiency, and updating nested structure

[Michael Gardner]

- Worrying about the performance of a small, pure function like this is almost 
certainly premature optimization.

- Avoid concurrency constructs like atoms if you don't need them.

- Have you considered using group-by?

> On Nov 11, 2015, at 13:25, Dave Tenny  wrote:
> 
> A colleague and I are debating various things clojure as we were exploring 
> alternative ways to solve a problem.
> 
> Here's the description of the problem that a particular function is trying to 
> solve, and the first implementation of it.
> 
> (defn update-roles 
>   "Given a vector of maps of the form {:project_id N :project_name S 
> :project_roles [...roles...]}
>   if there is already a map for the indicated project id/name, add new-role 
> to it and returned
>   a copy the updated input vector, otherwise return a vector with a new map 
> entry for the newly found
>   project and initial role.  This function is basically aggregating tuples 
> from the database."
>   [projects project-id project-name new-role]
>   (let [updated? (atom nil)
> 
> projects* (mapv (fn [m] 
>   (if (= (:project_id m) project-id)
> (do (reset! updated? true)
> (assoc m :project_roles (conj (:project_roles 
> m) new-role)))
> m))
> projects)]
> (if @updated?
>   projects*
>   (conj projects {:project_id project-id :project_name project-name 
> :project_roles [new-role]}
> 
> 
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own]}] 
> 2 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own]}] 
> 1 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]
> 
> 
> 
> Now here's another implementation:
> 
> (defn update-or-insert-project-role
>   [prj-roles prj-role]
>   (let [to-insert-prj-id (:project_id prj-role)
> by-pid   (group-by :project_id prj-roles)]
> (case (get by-pid to-insert-prj-id)
>   nil (conj prj-roles prj-role)
>   (->> (update-in by-pid [to-insert-prj-id 0 :project_roles] #(apply conj 
> % (:project_roles prj-role)))
>(mapcat second)
>(into [])
> 
> ;; (def prj-roles [{:project_id 1, :project_name "One", :project_roles 
> [:own]} {:project_id 3 :project_name "Three" :project_roles [:edit]}])
> ;; (update-or-insert-project-role prj-roles {:project_id 2 :project_name 
> "Two" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]} {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-or-insert-project-role prj-roles {:project_id 1 :project_name 
> "One" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]}]
> 
> 
> 
> The function is called in a loop to aggregate rows from a database, though it 
> isn't an overriding concern, we're not going millions of records in this case.
> 
> The first thing my colleague and I disagreed on was the calling sequence, 
> arguing over which is more readable.
> The second thing was whether efficiency in this context is really important, 
> or whether it's all moot in clojure.  
> 
> Finally, I'm sure there's a better way, probably with Zippers or something, 
> but neither of us have used them. Suggestions for the stylistic and 
> operational epitome of clojure expression on this routine are welcome!
> 
> Superficially, and probably incorrect in multiple ways, here is a poor 
> attempt at breaking down efficiency in terms of search/traversal and memory 
> allocations.  This was done by someone with no knowledge of clojure internals 
> (including the library implementations of the called functions).
> 
> ;; Comparing the two routines per function call, for existing project case 
> (i.e. where roles are updated)
> ;;
> ;; Assuming each routine allocates new vector for new-role placement in 
> existing project
> ;; and MapEntry for assoc of new vector on project_roles, so they aren't 
> included in allocations 
> ;; below since both routines have to do it.
> ;;
> ;; Note that x-element map allocates storage for map and map-entries or 
> clojure equivalent.
> ;; (and more expensive than an x-element vector, of course).
> ;;
> ;; n == length of input project list.
> ;; m == average length of input project list role vectors.
> ;; 
> ;; Object Allocations
> ;;   Function call:
> ;; update-roles: 
> ;;   1 atom
> ;;   1 O(n) vector for mapv 
> ;; update-or-insert-project-role: 
> ;;   1 3-entry map + 1 single-element vector for prj-role argument input.
> ;;   1 n-element map for group-by
> ;;   n vectors for group-by map 

Re: Style, Efficiency, and updating nested structure

[Atamert Ölçgen]

Hi Dave,

I would prefer functional approach (the latter, one without atoms and
reset!) over the one with mutable local state.

I would also refactor the update case as a separate function.

Both are from the style perspective.

I would try to optimize after I'm happy with how the code reads. And
definitely do that using data (benchmarks) instead of intuition.

As a side note, a set might be a better choice for :project_roles, vectors
allow duplicates.


On Wed, Nov 11, 2015 at 11:25 PM, Dave Tenny  wrote:

> A colleague and I are debating various things clojure as we were exploring
> alternative ways to solve a problem.
>
> Here's the description of the problem that a particular function is trying
> to solve, and the first implementation of it.
>
> (defn update-roles
>   "Given a vector of maps of the form {:project_id N :project_name S
> :project_roles [...roles...]}
>   if there is already a map for the indicated project id/name, add
> new-role to it and returned
>   a copy the updated input vector, otherwise return a vector with a new
> map entry for the newly found
>   project and initial role.  This function is basically aggregating tuples
> from the database."
>   [projects project-id project-name new-role]
>   (let [updated? (atom nil)
>
> projects* (mapv (fn [m]
>   (if (= (:project_id m) project-id)
> (do (reset! updated? true)
> (assoc m :project_roles (conj (:project_roles
> m) new-role)))
> m))
> projects)]
> (if @updated?
>   projects*
>   (conj projects {:project_id project-id :project_name project-name 
> :project_roles
> [new-role]}
>
>
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 2 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 1 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]
>
>
>
> Now here's another implementation:
>
> (defn update-or-insert-project-role
>   [prj-roles prj-role]
>   (let [to-insert-prj-id (:project_id prj-role)
> by-pid   (group-by :project_id prj-roles)]
> (case (get by-pid to-insert-prj-id)
>   nil (conj prj-roles prj-role)
>   (->> (update-in by-pid [to-insert-prj-id 0 :project_roles] #(apply conj 
> % (:project_roles prj-role)))
>(mapcat second)
>(into [])
>
> ;; (def prj-roles [{:project_id 1, :project_name "One", :project_roles 
> [:own]} {:project_id 3 :project_name "Three" :project_roles [:edit]}])
> ;; (update-or-insert-project-role prj-roles {:project_id 2 :project_name 
> "Two" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]} {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-or-insert-project-role prj-roles {:project_id 1 :project_name 
> "One" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]}]
>
>
>
>
> The function is called in a loop to aggregate rows from a database, though
> it isn't an overriding concern, we're not going millions of records in this
> case.
>
> The first thing my colleague and I disagreed on was the calling sequence,
> arguing over which is more readable.
> The second thing was whether efficiency in this context is really
> important, or whether it's all moot in clojure.
>
> Finally, I'm sure there's a better way, probably with Zippers or
> something, but neither of us have used them. Suggestions for the stylistic
> and operational epitome of clojure expression on this routine are welcome!
>
> Superficially, and probably incorrect in multiple ways, here is a poor
> attempt at breaking down efficiency in terms of search/traversal and memory
> allocations.  This was done by someone with no knowledge of clojure
> internals (including the library implementations of the called functions).
>
> ;; Comparing the two routines per function call, for existing project case
> (i.e. where roles are updated)
> ;;
> ;; Assuming each routine allocates new vector for new-role placement in
> existing project
> ;; and MapEntry for assoc of new vector on project_roles, so they aren't
> included in allocations
> ;; below since both routines have to do it.
> ;;
> ;; Note that x-element map allocates storage for map and map-entries or
> clojure equivalent.
> ;; (and more expensive than an x-element vector, of course).
> ;;
> ;; n == length of input project list.
> ;; m == average length of input project list role vectors.
> ;;
> ;; Object Allocations
> ;;   Function call:
> ;; update-roles:
> ;;   1 atom
> ;;   1 O(n) vector for

Re: Style, Efficiency, and updating nested structure

[Nelson Morris]

I can't speak much to the efficiency analysis, but if I was solving the
problem I would attempt to change the data structure. If the `proj-roles`
could be a map then everything could be quick map updates.  Given you run
this in a loop to aggregate everything perhaps converting to that form
before hand is an option. Then at the end the map could be expanded out
into the vector form the rest of the application wants. Something like:


(defn order-roles [roles]
  (filterv roles [:own :edit]))
;; add other roles here
;; if ordering doesn't matter then use (vec roles) or roles

(defn expand [permissions]
(mapv (fn [[[id name] roles]]
{:project_id id
 :project_name name
 :project_roles (filterv roles ordered-permissions)})
  permissions))

(defn add-role [permissions id name role]
(update-in permissions [id name] (fnil #(conj % role) #{})))

;; Examples
(-> {}
(add-role 1 "One" :own)
(add-role 2 "Two" :edit)
expand)
[{:project_id 1, :project_name "One", :project_roles [:own]}
 {:project_id 2, :project_name "Two", :project_roles [:edit]}]

(-> {}
(add-role 1 "One" :own)
(add-role 1 "One" :edit)
expand)
[{:project_id 1, :project_name "One", :project_roles [:own :edit]}]

On Wed, Nov 11, 2015 at 3:25 PM, Dave Tenny  wrote:

> A colleague and I are debating various things clojure as we were exploring
> alternative ways to solve a problem.
>
> Here's the description of the problem that a particular function is trying
> to solve, and the first implementation of it.
>
> (defn update-roles
>   "Given a vector of maps of the form {:project_id N :project_name S
> :project_roles [...roles...]}
>   if there is already a map for the indicated project id/name, add
> new-role to it and returned
>   a copy the updated input vector, otherwise return a vector with a new
> map entry for the newly found
>   project and initial role.  This function is basically aggregating tuples
> from the database."
>   [projects project-id project-name new-role]
>   (let [updated? (atom nil)
>
> projects* (mapv (fn [m]
>   (if (= (:project_id m) project-id)
> (do (reset! updated? true)
> (assoc m :project_roles (conj (:project_roles
> m) new-role)))
> m))
> projects)]
> (if @updated?
>   projects*
>   (conj projects {:project_id project-id :project_name project-name 
> :project_roles
> [new-role]}
>
>
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 2 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own
> ]}] 1 "Two" :edit)
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]
>
>
>
> Now here's another implementation:
>
> (defn update-or-insert-project-role
>   [prj-roles prj-role]
>   (let [to-insert-prj-id (:project_id prj-role)
> by-pid   (group-by :project_id prj-roles)]
> (case (get by-pid to-insert-prj-id)
>   nil (conj prj-roles prj-role)
>   (->> (update-in by-pid [to-insert-prj-id 0 :project_roles] #(apply conj 
> % (:project_roles prj-role)))
>(mapcat second)
>(into [])
>
> ;; (def prj-roles [{:project_id 1, :project_name "One", :project_roles 
> [:own]} {:project_id 3 :project_name "Three" :project_roles [:edit]}])
> ;; (update-or-insert-project-role prj-roles {:project_id 2 :project_name 
> "Two" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]} {:project_id 
> 2, :project_name "Two", :project_roles [:edit]}]
> ;; (update-or-insert-project-role prj-roles {:project_id 1 :project_name 
> "One" :project_roles [:edit]})
> ;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]} 
> {:project_id 3, :project_name "Three", :project_roles [:edit]}]
>
>
>
>
> The function is called in a loop to aggregate rows from a database, though
> it isn't an overriding concern, we're not going millions of records in this
> case.
>
> The first thing my colleague and I disagreed on was the calling sequence,
> arguing over which is more readable.
> The second thing was whether efficiency in this context is really
> important, or whether it's all moot in clojure.
>
> Finally, I'm sure there's a better way, probably with Zippers or
> something, but neither of us have used them. Suggestions for the stylistic
> and operational epitome of clojure expression on this routine are welcome!
>
> Superficially, and probably incorrect in multiple ways, here is a poor
> attempt at breaking down efficiency in terms of search/traversal and memory
> allocations.  This was done by someone with no knowledge of clojure
> 

Style, Efficiency, and updating nested structure

[Dave Tenny]

A colleague and I are debating various things clojure as we were exploring 
alternative ways to solve a problem.

Here's the description of the problem that a particular function is trying 
to solve, and the first implementation of it.

(defn update-roles 
  "Given a vector of maps of the form {:project_id N :project_name S 
:project_roles [...roles...]}
  if there is already a map for the indicated project id/name, add new-role 
to it and returned
  a copy the updated input vector, otherwise return a vector with a new map 
entry for the newly found
  project and initial role.  This function is basically aggregating tuples 
from the database."
  [projects project-id project-name new-role]
  (let [updated? (atom nil)

projects* (mapv (fn [m] 
  (if (= (:project_id m) project-id)
(do (reset! updated? true)
(assoc m :project_roles (conj (:project_roles 
m) new-role)))
m))
projects)]
(if @updated?
  projects*
  (conj projects {:project_id project-id :project_name project-name 
:project_roles 
[new-role]}


;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own]}] 
2 "Two" :edit)
;; => [{:project_id 1, :project_name "One", :project_roles [:own]} {:project_id 
2, :project_name "Two", :project_roles [:edit]}]
;; (update-roles [{:project_id 1 :project_name "One" :project_roles [:own]}] 
1 "Two" :edit)
;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]}]



Now here's another implementation:

(defn update-or-insert-project-role
  [prj-roles prj-role]
  (let [to-insert-prj-id (:project_id prj-role)
by-pid   (group-by :project_id prj-roles)]
(case (get by-pid to-insert-prj-id)
  nil (conj prj-roles prj-role)
  (->> (update-in by-pid [to-insert-prj-id 0 :project_roles] #(apply conj % 
(:project_roles prj-role)))
   (mapcat second)
   (into [])

;; (def prj-roles [{:project_id 1, :project_name "One", :project_roles [:own]} 
{:project_id 3 :project_name "Three" :project_roles [:edit]}])
;; (update-or-insert-project-role prj-roles {:project_id 2 :project_name "Two" 
:project_roles [:edit]})
;; => [{:project_id 1, :project_name "One", :project_roles [:own]} {:project_id 
3, :project_name "Three", :project_roles [:edit]} {:project_id 2, :project_name 
"Two", :project_roles [:edit]}]
;; (update-or-insert-project-role prj-roles {:project_id 1 :project_name "One" 
:project_roles [:edit]})
;; => [{:project_id 1, :project_name "One", :project_roles [:own :edit]} 
{:project_id 3, :project_name "Three", :project_roles [:edit]}]




The function is called in a loop to aggregate rows from a database, though 
it isn't an overriding concern, we're not going millions of records in this 
case.

The first thing my colleague and I disagreed on was the calling sequence, 
arguing over which is more readable.
The second thing was whether efficiency in this context is really 
important, or whether it's all moot in clojure.  

Finally, I'm sure there's a better way, probably with Zippers or something, 
but neither of us have used them. Suggestions for the stylistic and 
operational epitome of clojure expression on this routine are welcome!

Superficially, and probably incorrect in multiple ways, here is a poor 
attempt at breaking down efficiency in terms of search/traversal and memory 
allocations.  This was done by someone with no knowledge of clojure 
internals (including the library implementations of the called functions).

;; Comparing the two routines per function call, for existing project case 
(i.e. where roles are updated)
;;
;; Assuming each routine allocates new vector for new-role placement in 
existing project
;; and MapEntry for assoc of new vector on project_roles, so they aren't 
included in allocations 
;; below since both routines have to do it.
;;
;; Note that x-element map allocates storage for map and map-entries or 
clojure equivalent.
;; (and more expensive than an x-element vector, of course).
;;
;; n == length of input project list.
;; m == average length of input project list role vectors.
;; 
;; Object Allocations
;;   Function call:
;; update-roles: 
;;   1 atom
;;   1 O(n) vector for mapv 
;; update-or-insert-project-role: 
;;   1 3-entry map + 1 single-element vector for prj-role argument 
input.
;;   1 n-element map for group-by
;;   n vectors for group-by map values
;;   1 n-element map for update-in
;;   1 list/sequence for mapcat (+ n concat intermediaries?)
;;   1 vector for into
;;
;; If we discard the second 'into' and first 'mapv' allocations the 
update-or-insert-project-role routine allocates
;; 3 additional maps (two of which are O(n)), n additional vectors, and 1 
additional list/sequence.
;;   
;; Searches/traversals/copies
;;  update-roles: 
;;   O(n) - mapv
;;  update-or-insert-project-role: 
;;   O(n) - g

Re: updating nested structure

[Rich Hickey]

On Aug 28, 11:43 am, Parth Malwankar <[EMAIL PROTECTED]>
wrote:
> On Aug 28, 8:08 pm, Rich Hickey <[EMAIL PROTECTED]> wrote:
>
>
>
> > On Aug 27, 10:37 pm, Parth Malwankar <[EMAIL PROTECTED]>
> > wrote:
>
> > > On Aug 28, 12:10 am, Rich Hickey <[EMAIL PROTECTED]> wrote:
>
> > > > I posted a variant here:
>
> > > >http://paste.lisp.org/display/65964
>
> > > Rich,
>
> > > It works very nicely. Thanks.
>
> > > Just one thought in case the functions args are still being
> > > decided on. Could we consider taking access path as a
> > > vector rather than directly as function args.
>
> > > Here is the use case I have in mind.
>
> > > I think most access paths [:a :b :c] would be generated.
> > > Nested structures would be something like
>
> > > processor -> GPRs (general purpose regs) -> r0 r1 .. rN
> > >   ->  FPRs (floating point regs) -> f0 f1 .. fN
>
> > > fridge -> fruits -> apple mango ...
> > >-> veggies -> eggplant ...
> > >-> diary -> milk yoghurt ...
>
> > > So the developer may set up something like a
> > > (fridge-item-path (get-fruit)) => [:fruits :apple]
> > > (processor-reg-path (get-reg-arg-from-instruction)) => [:gpr-set :r0]
>
> > > With the current arg handling this is what we would need to do:
>
> > > user=> (item-path :mango)
> > > [:fruits :mango]
>
> > > user=> (apply mk-get my-fridge (conj (item-path :mango) :quantity))
> > > 30
>
> > > user=> (apply mk-assoc my-fridge (conj (item-path :mango) :quantity
> > > 40))
> > > {:fruits {:mango {:quantity 40, :color :yellow},
> > >   :apple {:quantity 20, :color :red}},
> > >   :diary-products {:milk {:quantity 1, :color :white,
> > >   :type :low-fat},
> > >:yoghurt {:quantity 10, :color :pink,
> > >  :type :strawberry}}}
>
> > > [formatting added above for readability]
>
> > > In case the access path were vectors the above could become:
>
> > > (mk-get my-fridge (item-path :mango) :quantity)
> > > (mk-assoc my-fridge (item-path :mango) :quantity new-quantity)
>
> > > Much less noise.
>
> > I don't see this (partial path + separate last key) as a general way
> > of doing things. The only other way I would consider is the entire
> > path as a sequence:
>
> > (defn mk-get [m ks]
> >   (reduce get m ks))
>
> > (defn mk-assoc [m [k & ks] v]
> >   (if ks
> > (assoc m k (mk-assoc (get m k) ks v))
> > (assoc m k v)))
>
> > ;usage
> > (def nx {:a {:b {:c {:content [1 10] :other 2)
>
> > (mk-get nx [:a :b :c :content])
> > -> [1 10]
>
> > (mk-assoc nx [:a :b :c :content 0] 42)
> > -> {:a {:b {:c {:other 2, :content [42 10]
>
> > (mk-assoc {} [:a :b :x] 42)
> > -> {:a {:b {:x 42}}}
>
> Thanks Rich.
> Sounds like a good idea to me. Works beautifully.
>
> user=> (load-file "fruits.clj")
> #'user/my-fridge
> user=> (mk-get my-fridge (item-quantity-path :mango))
> 30
> user=> (mk-assoc my-fridge (item-quantity-path :mango) 50)
> {:fruits {:mango {:quantity 50, :color :yellow}, :apple {:quantity
> 20, :color :red}}, :diary-products {:yoghurt {:quantity
> 10, :color :pink, :type :strawberry}, :milk {:quantity
> 1, :color :white, :type :low-fat}}}
> user=>
>


I've added get-in, assoc-in, and update-in to SVN rev 1010.

Rich
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Parth Malwankar]

On Aug 28, 8:08 pm, Rich Hickey <[EMAIL PROTECTED]> wrote:
> On Aug 27, 10:37 pm, Parth Malwankar <[EMAIL PROTECTED]>
> wrote:
>
>
>
> > On Aug 28, 12:10 am, Rich Hickey <[EMAIL PROTECTED]> wrote:
>
> > > I posted a variant here:
>
> > >http://paste.lisp.org/display/65964
>
> > Rich,
>
> > It works very nicely. Thanks.
>
> > Just one thought in case the functions args are still being
> > decided on. Could we consider taking access path as a
> > vector rather than directly as function args.
>
> > Here is the use case I have in mind.
>
> > I think most access paths [:a :b :c] would be generated.
> > Nested structures would be something like
>
> > processor -> GPRs (general purpose regs) -> r0 r1 .. rN
> >           ->  FPRs (floating point regs) -> f0 f1 .. fN
>
> > fridge -> fruits -> apple mango ...
> >        -> veggies -> eggplant ...
> >        -> diary -> milk yoghurt ...
>
> > So the developer may set up something like a
> > (fridge-item-path (get-fruit)) => [:fruits :apple]
> > (processor-reg-path (get-reg-arg-from-instruction)) => [:gpr-set :r0]
>
> > With the current arg handling this is what we would need to do:
>
> > user=> (item-path :mango)
> > [:fruits :mango]
>
> > user=> (apply mk-get my-fridge (conj (item-path :mango) :quantity))
> > 30
>
> > user=> (apply mk-assoc my-fridge (conj (item-path :mango) :quantity
> > 40))
> > {:fruits {:mango {:quantity 40, :color :yellow},
> >           :apple {:quantity 20, :color :red}},
> >   :diary-products {:milk {:quantity 1, :color :white,
> >                           :type :low-fat},
> >                    :yoghurt {:quantity 10, :color :pink,
> >                              :type :strawberry}}}
>
> > [formatting added above for readability]
>
> > In case the access path were vectors the above could become:
>
> > (mk-get my-fridge (item-path :mango) :quantity)
> > (mk-assoc my-fridge (item-path :mango) :quantity new-quantity)
>
> > Much less noise.
>
> I don't see this (partial path + separate last key) as a general way
> of doing things. The only other way I would consider is the entire
> path as a sequence:
>
> (defn mk-get [m ks]
>   (reduce get m ks))
>
> (defn mk-assoc [m [k & ks] v]
>   (if ks
>     (assoc m k (mk-assoc (get m k) ks v))
>     (assoc m k v)))
>
> ;usage
> (def nx {:a {:b {:c {:content [1 10] :other 2)
>
> (mk-get nx [:a :b :c :content])
> -> [1 10]
>
> (mk-assoc nx [:a :b :c :content 0] 42)
> -> {:a {:b {:c {:other 2, :content [42 10]
>
> (mk-assoc {} [:a :b :x] 42)
> -> {:a {:b {:x 42}}}
>

Thanks Rich.
Sounds like a good idea to me. Works beautifully.

user=> (load-file "fruits.clj")
#'user/my-fridge
user=> (mk-get my-fridge (item-quantity-path :mango))
30
user=> (mk-assoc my-fridge (item-quantity-path :mango) 50)
{:fruits {:mango {:quantity 50, :color :yellow}, :apple {:quantity
20, :color :red}}, :diary-products {:yoghurt {:quantity
10, :color :pink, :type :strawberry}, :milk {:quantity
1, :color :white, :type :low-fat}}}
user=>

Parth

> Rich
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Rich Hickey]

On Aug 27, 10:37 pm, Parth Malwankar <[EMAIL PROTECTED]>
wrote:
> On Aug 28, 12:10 am, Rich Hickey <[EMAIL PROTECTED]> wrote:
>
> > I posted a variant here:
>
> >http://paste.lisp.org/display/65964
>
> Rich,
>
> It works very nicely. Thanks.
>
> Just one thought in case the functions args are still being
> decided on. Could we consider taking access path as a
> vector rather than directly as function args.
>
> Here is the use case I have in mind.
>
> I think most access paths [:a :b :c] would be generated.
> Nested structures would be something like
>
> processor -> GPRs (general purpose regs) -> r0 r1 .. rN
>   ->  FPRs (floating point regs) -> f0 f1 .. fN
>
> fridge -> fruits -> apple mango ...
>-> veggies -> eggplant ...
>-> diary -> milk yoghurt ...
>
> So the developer may set up something like a
> (fridge-item-path (get-fruit)) => [:fruits :apple]
> (processor-reg-path (get-reg-arg-from-instruction)) => [:gpr-set :r0]
>
> With the current arg handling this is what we would need to do:
>
> user=> (item-path :mango)
> [:fruits :mango]
>
> user=> (apply mk-get my-fridge (conj (item-path :mango) :quantity))
> 30
>
> user=> (apply mk-assoc my-fridge (conj (item-path :mango) :quantity
> 40))
> {:fruits {:mango {:quantity 40, :color :yellow},
>   :apple {:quantity 20, :color :red}},
>   :diary-products {:milk {:quantity 1, :color :white,
>   :type :low-fat},
>:yoghurt {:quantity 10, :color :pink,
>  :type :strawberry}}}
>
> [formatting added above for readability]
>
> In case the access path were vectors the above could become:
>
> (mk-get my-fridge (item-path :mango) :quantity)
> (mk-assoc my-fridge (item-path :mango) :quantity new-quantity)
>
> Much less noise.
>

I don't see this (partial path + separate last key) as a general way
of doing things. The only other way I would consider is the entire
path as a sequence:

(defn mk-get [m ks]
  (reduce get m ks))

(defn mk-assoc [m [k & ks] v]
  (if ks
(assoc m k (mk-assoc (get m k) ks v))
(assoc m k v)))

;usage
(def nx {:a {:b {:c {:content [1 10] :other 2)

(mk-get nx [:a :b :c :content])
-> [1 10]

(mk-assoc nx [:a :b :c :content 0] 42)
-> {:a {:b {:c {:other 2, :content [42 10]

(mk-assoc {} [:a :b :x] 42)
-> {:a {:b {:x 42}}}

Rich


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Chouser]

On Thu, Aug 28, 2008 at 2:01 AM, Parth Malwankar
<[EMAIL PROTECTED]> wrote:
>
> On Aug 28, 8:13 am, Chouser <[EMAIL PROTECTED]> wrote:
>>
>> (apply mk-get my-fridge (item-path :mango) :quantity)
>
> I get an error with this.
>
> user=> (item-path :mango)
> [:fruits :mango]
> user=> (apply mk-get my-fridge (item-path :mango) :quantity)
> java.lang.IllegalArgumentException: Don't know how to create ISeq
> from: Keyword : :quantity

I'm sorry -- remind me not to post untested code after my bedtime.

You're right, that won't work.  Apply does allow you to provide
non-seq arguments followed by the single seq at the end.  This is what
I was thinking about:

user=> (apply + 1 2 3 [4 5 6])
21

But the apply expression I wrote above has a seq that you want to
expand early in the list, and non-seq at the end.  As you already
discovered, that just throws an exception.

So let me start over -- you want a function that takes a mix of keys
and vectors of keys, right?  You could of course wrap the ones Rich
provided for your own use cases.  For example if you always have one
vector followed by one plain key, you could do:

user=> (defn mk-get-1 [m v k] (apply mk-get m (concat v [k])))
#'user/mk-get-1
user=> (mk-get-1 my-fridge (item-path :mango) :quantity)
10

Or if you want to allow any kind of mixed vectors, seqs, and plain keys:

user=> (defn mk-get-2 [m & a] (apply mk-get m (mapcat #(if (or
(vector? %) (seq? %)) % [%]) a)))
#'user/mk-get-2
user=> (mk-get-2 my-fridge (item-path :mango) :quantity)
10
user=> (mk-get-2 my-fridge :fruits [:mango :quantity])
10
user=> (mk-get-2 my-fridge :fruits (list :mango :quantity))
10
user=> (mk-get-2 my-fridge :fruits [:mango] :quantity)
10

Neither of these strike me as very general.  I don't have my own use
cases (yet) for these functions, so I shouldn't speak too confidently,
but mk-get-1 seems like a pretty specific case (exactly one vector
followed by exactly one plain key).  And since vectors are perfectly
valid map keys, it's possible to build nested maps that can't be
accessed using mk-get-2:

user=> (mk-get {[:a :b] :c} [:a :b])
:c
user=> (mk-get-2 {[:a :b] :c} [:a :b])
nil

--Chouser

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Parth Malwankar]

On Aug 28, 8:13 am, Chouser <[EMAIL PROTECTED]> wrote:
> On Wed, Aug 27, 2008 at 10:37 PM, Parth Malwankar
>
> <[EMAIL PROTECTED]> wrote:
> > In case the access path were vectors the above could become:
>
> > (mk-get my-fridge (item-path :mango) :quantity)
> > (mk-assoc my-fridge (item-path :mango) :quantity new-quantity)
>
> > Much less noise.
>
> apply actually can do the conj'ing for you:
>
> (apply mk-get my-fridge (item-path :mango) :quantity)

I get an error with this.

user=> (item-path :mango)
[:fruits :mango]
user=> (apply mk-get my-fridge (item-path :mango) :quantity)
java.lang.IllegalArgumentException: Don't know how to create ISeq
from: Keyword : :quantity
java.lang.IllegalArgumentException: Don't know how to create ISeq
from: Keyword : :quantity
at clojure.lang.RT.seqFrom(RT.java:461)
at clojure.lang.RT.seq(RT.java:444)
at clojure.seq__28.invoke(boot.clj:92)
at clojure.spread__132.invoke(boot.clj:357)
at clojure.spread__132.invoke(boot.clj:358)
at clojure.spread__132.invoke(boot.clj:358)
at clojure.apply__135.doInvoke(boot.clj:364)
at clojure.lang.RestFn.invoke(RestFn.java:460)
at user.eval__2237.invoke(Unknown Source)
at clojure.lang.Compiler.eval(Compiler.java:3847)
at clojure.lang.Repl.main(Repl.java:75)
user=> (apply mk-get my-fridge (conj (item-path :mango) :quantity))
30

Is this supposed to work? If it does it will be very convenient.

Parth

>
> --Chouser
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Chouser]

On Wed, Aug 27, 2008 at 10:37 PM, Parth Malwankar
<[EMAIL PROTECTED]> wrote:
> In case the access path were vectors the above could become:
>
> (mk-get my-fridge (item-path :mango) :quantity)
> (mk-assoc my-fridge (item-path :mango) :quantity new-quantity)
>
> Much less noise.

apply actually can do the conj'ing for you:

(apply mk-get my-fridge (item-path :mango) :quantity)

--Chouser

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Parth Malwankar]

On Aug 28, 12:10 am, Rich Hickey <[EMAIL PROTECTED]> wrote:

> I posted a variant here:
>
> http://paste.lisp.org/display/65964
>

Rich,

It works very nicely. Thanks.

Just one thought in case the functions args are still being
decided on. Could we consider taking access path as a
vector rather than directly as function args.

Here is the use case I have in mind.

I think most access paths [:a :b :c] would be generated.
Nested structures would be something like

processor -> GPRs (general purpose regs) -> r0 r1 .. rN
  ->  FPRs (floating point regs) -> f0 f1 .. fN

fridge -> fruits -> apple mango ...
   -> veggies -> eggplant ...
   -> diary -> milk yoghurt ...

So the developer may set up something like a
(fridge-item-path (get-fruit)) => [:fruits :apple]
(processor-reg-path (get-reg-arg-from-instruction)) => [:gpr-set :r0]

With the current arg handling this is what we would need to do:

user=> (item-path :mango)
[:fruits :mango]

user=> (apply mk-get my-fridge (conj (item-path :mango) :quantity))
30

user=> (apply mk-assoc my-fridge (conj (item-path :mango) :quantity
40))
{:fruits {:mango {:quantity 40, :color :yellow},
  :apple {:quantity 20, :color :red}},
  :diary-products {:milk {:quantity 1, :color :white,
  :type :low-fat},
   :yoghurt {:quantity 10, :color :pink,
 :type :strawberry}}}

[formatting added above for readability]

In case the access path were vectors the above could become:

(mk-get my-fridge (item-path :mango) :quantity)
(mk-assoc my-fridge (item-path :mango) :quantity new-quantity)

Much less noise.

Its not a big deal as the user would probably be writing a layer
on top of mk-get/mk-assoc e.g. fridge-add-item fridge-check-item
but then with access-paths as vectors it usage will be cleaner.

If you think this is a common enough case it could be considered.

Also, thanks for the [m [k & ks] v] destructuring trick. I didn't know
we could do that. Very neat.

Thanks very much.
Parth

> Rich
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Rich Hickey]

On Aug 27, 4:39 am, Parth Malwankar <[EMAIL PROTECTED]> wrote:
> On Aug 26, 8:25 pm, Parth Malwankar <[EMAIL PROTECTED]> wrote:
>
>
>
> > Hello,
>
> > In order to update fields in nested structures/maps easily
> > I have created a macro 'field-write'.
> > (defmacro field-write [st k v access-spec]
> >   ; st=data, k=key to update, v=val to put, access-spec=access
> > vector
> >   ; ... code listing after interaction
>
> > user=> nx
> > {:a {:b {:c {:content 10
> > user=> (field-write nx :content 1000 [:a :b :c])
> > {:a {:b {:c {:content 1000
> > user=> (macroexpand '(field-write nx :content 1000 [:a :b :c]))
> > (clojure/assoc
> >   nx :a
> >   (clojure/assoc
> > (clojure/-> nx :a) :b
> > (clojure/assoc
> >   (clojure/-> nx :a :b) :c
> >   (clojure/assoc
> > (clojure/-> nx :a :b :c):content 1000
>
> > [formatting added above for readability]
>
> > It seems to be working fine but I thought it may be
> > good to get inputs from the Clojure experts here to
> > see if there is a better way to do this in Clojure.
>
> > (def nx {:a {:b {:c {:content 10)
>
> > (def field-write)
> > (defmacro field-write [st k v access-spec]
> >   ; st=data, k=key to update, v=val to put, access-spec=access
> > vector
> >   (if (pred/empty? access-spec)
> > `(assoc ~st ~k ~v)
> > (field-write st (last access-spec)
> >  `(assoc (-> ~st [EMAIL PROTECTED]) ~k ~v)
> >  (butlast access-spec
>
> After some more experimentation I found that the field-write
> macro didn't work with access-specs like (vector :a :b :c)
> ... I should have thought of that before.
>
> So I reimplemented field-read and field-write as functions
> which seem to work in all scenarios.
>
> (defn field-read [st as]
>   "user=> nx
>   {:a {:b {:c {:data 10
>   user=> (field-read nx [:a :b :c])
>   {:data 10}"
>   (eval (reduce (comp reverse list) st as)))
>
> (defn field-write [structure kwd value access-spec]
>   "user=> nx
>   {:a {:b {:c {:data 10
>   user=> (field-write nx :data 20 [:a :b :c])
>   {:a {:b {:c {:data 20
>   user=> (field-write nx :data (+ 2 2 2) (vector :a :b :c))
>   {:a {:b {:c {:data 6"
>   (loop [st structure
> k kwd
> v value
> as access-spec]
> (if (pred/empty? as)
>   (assoc st k v)
>   (recur st (last as)
>  (assoc (field-read st as) k v)
>  (butlast as)
>
> I suspect using eval in field-read is a bad idea (preformance?) but I
> couldn't come up with a better way to do it. access-spec
> needs to be a vector as its something thats returned by
> another function in my use case (otherwise I could have
> used it with ->).
>
> Can I do something better here? Comments?
>

I posted a variant here:

http://paste.lisp.org/display/65964

Rich

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Chouser]

On Tue, Aug 26, 2008 at 11:25 AM, Parth Malwankar
<[EMAIL PROTECTED]> wrote:
>
> In order to update fields in nested structures/maps easily
> I have created a macro 'field-write'.

It looks like this has come up on IRC a few weeks ago:
http://clojure-log.n01se.net/date/2008-07-15.html#09:22

I had forgotten that when I brought it up again today:
http://clojure-log.n01se.net/date/2008-08-27.html#10:00

It seems to me you're already pretty close to what Rich wants.

--Chouser

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Graham Fawcett]

On Wed, Aug 27, 2008 at 4:39 AM, Parth Malwankar
<[EMAIL PROTECTED]> wrote:
> After some more experimentation I found that the field-write
> macro didn't work with access-specs like (vector :a :b :c)
> ... I should have thought of that before.
>
> So I reimplemented field-read and field-write as functions
> which seem to work in all scenarios.
>
> (defn field-read [st as]
>  "user=> nx
>  {:a {:b {:c {:data 10
>  user=> (field-read nx [:a :b :c])
>  {:data 10}"
>  (eval (reduce (comp reverse list) st as)))

> I suspect using eval in field-read is a bad idea (preformance?) but I
> couldn't come up with a better way to do it. access-spec
> needs to be a vector as its something thats returned by
> another function in my use case (otherwise I could have
> used it with ->).

No need for eval here:

(defn field-read [st as]
  (loop [st st as as]
(if (first as)
  (recur (get st (first as))
 (rest as))
  st)))

Best,
Graham

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: updating nested structure

[Parth Malwankar]

On Aug 26, 8:25 pm, Parth Malwankar <[EMAIL PROTECTED]> wrote:
> Hello,
>
> In order to update fields in nested structures/maps easily
> I have created a macro 'field-write'.
>     (defmacro field-write [st k v access-spec]
>       ; st=data, k=key to update, v=val to put, access-spec=access
> vector
>       ; ... code listing after interaction
>
>     user=> nx
>     {:a {:b {:c {:content 10
>     user=> (field-write nx :content 1000 [:a :b :c])
>     {:a {:b {:c {:content 1000
>     user=> (macroexpand '(field-write nx :content 1000 [:a :b :c]))
>     (clojure/assoc
>       nx :a
>       (clojure/assoc
>         (clojure/-> nx :a) :b
>         (clojure/assoc
>           (clojure/-> nx :a :b) :c
>           (clojure/assoc
>             (clojure/-> nx :a :b :c):content 1000
>
> [formatting added above for readability]
>
> It seems to be working fine but I thought it may be
> good to get inputs from the Clojure experts here to
> see if there is a better way to do this in Clojure.
>
>     (def nx {:a {:b {:c {:content 10)
>
>     (def field-write)
>     (defmacro field-write [st k v access-spec]
>       ; st=data, k=key to update, v=val to put, access-spec=access
> vector
>       (if (pred/empty? access-spec)
>         `(assoc ~st ~k ~v)
>         (field-write st (last access-spec)
>                      `(assoc (-> ~st [EMAIL PROTECTED]) ~k ~v)
>                      (butlast access-spec
>

After some more experimentation I found that the field-write
macro didn't work with access-specs like (vector :a :b :c)
... I should have thought of that before.

So I reimplemented field-read and field-write as functions
which seem to work in all scenarios.

(defn field-read [st as]
  "user=> nx
  {:a {:b {:c {:data 10
  user=> (field-read nx [:a :b :c])
  {:data 10}"
  (eval (reduce (comp reverse list) st as)))

(defn field-write [structure kwd value access-spec]
  "user=> nx
  {:a {:b {:c {:data 10
  user=> (field-write nx :data 20 [:a :b :c])
  {:a {:b {:c {:data 20
  user=> (field-write nx :data (+ 2 2 2) (vector :a :b :c))
  {:a {:b {:c {:data 6"
  (loop [st structure
k kwd
v value
as access-spec]
(if (pred/empty? as)
  (assoc st k v)
  (recur st (last as)
 (assoc (field-read st as) k v)
 (butlast as)

I suspect using eval in field-read is a bad idea (preformance?) but I
couldn't come up with a better way to do it. access-spec
needs to be a vector as its something thats returned by
another function in my use case (otherwise I could have
used it with ->).

Can I do something better here? Comments?

Thanks very much.
Parth


> Thanks.
> Parth
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

updating nested structure

[Parth Malwankar]

Hello,

In order to update fields in nested structures/maps easily
I have created a macro 'field-write'.
(defmacro field-write [st k v access-spec]
  ; st=data, k=key to update, v=val to put, access-spec=access
vector
  ; ... code listing after interaction

user=> nx
{:a {:b {:c {:content 10
user=> (field-write nx :content 1000 [:a :b :c])
{:a {:b {:c {:content 1000
user=> (macroexpand '(field-write nx :content 1000 [:a :b :c]))
(clojure/assoc
  nx :a
  (clojure/assoc
(clojure/-> nx :a) :b
(clojure/assoc
  (clojure/-> nx :a :b) :c
  (clojure/assoc
(clojure/-> nx :a :b :c):content 1000

[formatting added above for readability]

It seems to be working fine but I thought it may be
good to get inputs from the Clojure experts here to
see if there is a better way to do this in Clojure.

(def nx {:a {:b {:c {:content 10)

(def field-write)
(defmacro field-write [st k v access-spec]
  ; st=data, k=key to update, v=val to put, access-spec=access
vector
  (if (pred/empty? access-spec)
`(assoc ~st ~k ~v)
(field-write st (last access-spec)
 `(assoc (-> ~st [EMAIL PROTECTED]) ~k ~v)
 (butlast access-spec

Thanks.
Parth

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---