RE: Merging two maps based on ids
Thanks Ted. Using your approach I could write the following function which will allow me to merge any number of data sets very easily: (defn merge-by-ids [& args] (reduce #(merge %1 %2) (map #(index % [:id]) args))) Example: (def data1 ; lowest priority '({:id 2 :a2 34 :a3 76 :a4 87}, ;<-- {:id 3 :a2 30 :a3 38 :a4 39}, {:id 4 :a2 10 :a3 73 :a4 38}, {:id 5 :a2 67 :a3 32 :a4 38}, {:id 7 :a2 84 :a3 86 :a4 63})) ;<-- (def data2 '({:id 3 :a2 5534 :a3 5576 :a4 5587}, ;<-- {:id 4 :a2 5584 :a3 5586 :a4 5563}, ;<-- {:id 12 :a2 5593 :a3 5512 :a4 5539}, {:id 13 :a2 5509 :a3 5539 :a4 5592})) ;<-- (def data3 ; highest priority '({:id 5 :a2 7734 :a3 7776 :a4 7787}, ;<-- {:id 12 :a2 7793 :a3 7712 :a4 5539}, ;<-- {:id 15 :a2 7709 :a3 7739 :a4 7792})) ;<-- (merge-by-ids data1 data2 data3) Result: { {:id 2} #{{:id 2, :a2 34, :a3 76, :a4 87}}, {:id 3} #{{:id 3, :a2 5534, :a3 5576, :a4 5587}}, {:id 4} #{{:id 4, :a2 5584, :a3 5586, :a4 5563}}, {:id 5} #{{:id 5, :a2 7734, :a3 7776, :a4 7787}}, {:id 7} #{{:id 7, :a2 84, :a3 86, :a4 63}}, {:id 12} #{{:id 12, :a2 7793, :a3 7712, :a4 5539}}, {:id 13} #{{:id 13, :a2 5509, :a3 5539, :a4 5592}}, {:id 15} #{{:id 15, :a2 7709, :a3 7739, :a4 7792}}, } -Original Message- From: clojure@googlegroups.com [mailto:clojure@googlegroups.com] On Behalf Of GrumpyLittleTed Sent: Thursday, April 28, 2011 6:56 PM To: Clojure Subject: Re: Merging two maps based on ids Sometimes it helps to change the structure of the data you are using to fit the job and simplify matters. In this case it appears that you are trying to represent a relation. However the tuples are being represented in a non-associative structure (a list). If you represent the relation using an outer map of inner maps then the job becomes as simple as (merge data-orig data-override) The simplest way to do this is to use the value of the key (in this case :id) as the key for the outer map. You also ensure the uniqueness of the key in the resulting outer map. If you need to keep the data in the list structure you can do the transform to a relation using (use 'clojure.set) (index data-orig [:id]) though this solution will give you keys in the form {:id 2} and each value as a set (for one to many relations whereas in this case you only need one-one). Using a proper relational approach the full solution is simply (def data-orig {2 {:id 2 :a2 34 :a3 76 :a4 87} 3 {:id 3 :a2 30 :a3 38 :a4 39} 5 {:id 5 :a2 67 :a3 32 :a4 38} 4 {:id 4 :a2 10 :a3 73 :a4 38} 7 {:id 7 :a2 84 :a3 86 :a4 63}}) (def data-override {2 {:id 2 :a2 5534 :a3 5576 :a4 5587} 3 {:id 3 :a2 5584 :a3 5586 :a4 5563} 12 {:id 12 :a2 5593 :a3 5512 :a4 5539} 13 {:id 13 :a2 5509 :a3 5539 :a4 5592}}) (merge data-orig data-override) On Apr 26, 5:23 am, "Bhinderwala, Shoeb" wrote: > Can someone help me write a merge function between two maps? My problem > is as follows: > > I have original data in a map: > > (def data-orig > '({:id 2 :a2 34 :a3 76 :a4 87}, > {:id 3 :a2 30 :a3 38 :a4 39}, > {:id 5 :a2 67 :a3 32 :a4 38}, > {:id 4 :a2 10 :a3 73 :a4 38}, > {:id 7 :a2 84 :a3 86 :a4 63})) > > Then I have override data: > > (def data-override > '({:id 2 :a2 5534 :a3 5576 :a4 5587}, > {:id 3 :a2 5584 :a3 5586 :a4 5563}, > {:id 12 :a2 5593 :a3 5512 :a4 5539}, > {:id 13 :a2 5509 :a3 5539 :a4 5592})) > > The result should be a merge of the two with the following conditions: > If the id is the same, should override the original data. If id is not > present in original then it should be added. The result should be: > > '({:id 2 :a2 5534 :a3 5576 :a4 5587}, ;overriden > {:id 3 :a2 5584 :a3 5586 :a4 5563}, ;overriden > {:id 5 :a2 67 :a3 32 :a4 38}, > {:id 4 :a2 10 :a3 73 :a4 38}, > {:id 7 :a2 84 :a3 86 :a4 63}, > {:id 12 :a2 5593 :a3 5512 :a4 5539}, ;added > {:id 13 :a2 5509 :a3 5539 :a4 5592}) ;added > > Thanks for your help. > -- Shoeb -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from
Re: Merging two maps based on ids
Sometimes it helps to change the structure of the data you are using to fit the job and simplify matters. In this case it appears that you are trying to represent a relation. However the tuples are being represented in a non-associative structure (a list). If you represent the relation using an outer map of inner maps then the job becomes as simple as (merge data-orig data-override) The simplest way to do this is to use the value of the key (in this case :id) as the key for the outer map. You also ensure the uniqueness of the key in the resulting outer map. If you need to keep the data in the list structure you can do the transform to a relation using (use 'clojure.set) (index data-orig [:id]) though this solution will give you keys in the form {:id 2} and each value as a set (for one to many relations whereas in this case you only need one-one). Using a proper relational approach the full solution is simply (def data-orig {2 {:id 2 :a2 34 :a3 76 :a4 87} 3 {:id 3 :a2 30 :a3 38 :a4 39} 5 {:id 5 :a2 67 :a3 32 :a4 38} 4 {:id 4 :a2 10 :a3 73 :a4 38} 7 {:id 7 :a2 84 :a3 86 :a4 63}}) (def data-override {2 {:id 2 :a2 5534 :a3 5576 :a4 5587} 3 {:id 3 :a2 5584 :a3 5586 :a4 5563} 12 {:id 12 :a2 5593 :a3 5512 :a4 5539} 13 {:id 13 :a2 5509 :a3 5539 :a4 5592}}) (merge data-orig data-override) On Apr 26, 5:23 am, "Bhinderwala, Shoeb" wrote: > Can someone help me write a merge function between two maps? My problem > is as follows: > > I have original data in a map: > > (def data-orig > '({:id 2 :a2 34 :a3 76 :a4 87}, > {:id 3 :a2 30 :a3 38 :a4 39}, > {:id 5 :a2 67 :a3 32 :a4 38}, > {:id 4 :a2 10 :a3 73 :a4 38}, > {:id 7 :a2 84 :a3 86 :a4 63})) > > Then I have override data: > > (def data-override > '({:id 2 :a2 5534 :a3 5576 :a4 5587}, > {:id 3 :a2 5584 :a3 5586 :a4 5563}, > {:id 12 :a2 5593 :a3 5512 :a4 5539}, > {:id 13 :a2 5509 :a3 5539 :a4 5592})) > > The result should be a merge of the two with the following conditions: > If the id is the same, should override the original data. If id is not > present in original then it should be added. The result should be: > > '({:id 2 :a2 5534 :a3 5576 :a4 5587}, ;overriden > {:id 3 :a2 5584 :a3 5586 :a4 5563}, ;overriden > {:id 5 :a2 67 :a3 32 :a4 38}, > {:id 4 :a2 10 :a3 73 :a4 38}, > {:id 7 :a2 84 :a3 86 :a4 63}, > {:id 12 :a2 5593 :a3 5512 :a4 5539}, ;added > {:id 13 :a2 5509 :a3 5539 :a4 5592}) ;added > > Thanks for your help. > -- Shoeb -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
RE: Merging two maps based on ids
Thanks Baishampayan, Meikel, pepijn, and Jonathan. The use of partition-by function does solve this very elegantly. Below is the list of solutions I have received: (defn merge-data [data1 data2] (map first (partition-by :id (sort-by :id (concat data2 data1) (defn merge-data2 [data1 data2] (let [override-ids (set (map :id data2))] (concat data2 (remove (comp override-ids :id) data1 (defn merge-override [merge-key data2 data1] (let [keyset (into #{} (map #(% merge-key) data2))] (apply merge data2 (remove #(keyset (% merge-key)) data1 From: clojure@googlegroups.com [mailto:clojure@googlegroups.com] On Behalf Of Jonathan Fischer Friberg Sent: Tuesday, April 26, 2011 12:58 PM To: clojure@googlegroups.com Subject: Re: Merging two maps based on ids Correction: (concat data1 data2) should be (concat data2 data1) On Tue, Apr 26, 2011 at 6:37 PM, Jonathan Fischer Friberg wrote: (defn merge-data [data1 data2] (map first (partition-by :id (sort-by :id (concat data1 data2) Since the sorting is stable (relative order is kept), we know that the first occurrence of each id is either the existing map from data1, or the new map from data2. On Tue, Apr 26, 2011 at 5:34 PM, pepijn (aka fliebel) wrote: Another option is using clojure.set, as is shown here: https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L9 9 On Apr 26, 10:10 am, Meikel Brandmeyer wrote: > Hi, > > you can construct the output sequence from your input sequences. > > (defn merge-data > [data-orig data-override] > (let [override-ids (set (map :id data-override))] > (concat data-override (remove (comp override-ids :id) data-orig > > If you need your output sorted you can also add a "(sord-by :id ...)" around > the concat. > > Sincerely > Meikel -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com <mailto:clojure%2bunsubscr...@googlegroups.com> For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
Correction: (concat data1 data2) should be (concat data2 data1) On Tue, Apr 26, 2011 at 6:37 PM, Jonathan Fischer Friberg < odysso...@gmail.com> wrote: > (defn merge-data [data1 data2] > (map first (partition-by :id (sort-by :id (concat data1 data2) > > Since the sorting is stable (relative order is kept), we know that the > first occurrence of each id is either the existing map from data1, or the > new map from data2. > > > On Tue, Apr 26, 2011 at 5:34 PM, pepijn (aka fliebel) < > pepijnde...@gmail.com> wrote: > >> Another option is using clojure.set, as is shown here: >> https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L99 >> >> On Apr 26, 10:10 am, Meikel Brandmeyer wrote: >> > Hi, >> > >> > you can construct the output sequence from your input sequences. >> > >> > (defn merge-data >> > [data-orig data-override] >> > (let [override-ids (set (map :id data-override))] >> > (concat data-override (remove (comp override-ids :id) data-orig >> > >> > If you need your output sorted you can also add a "(sord-by :id ...)" >> around >> > the concat. >> > >> > Sincerely >> > Meikel >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
(defn merge-data [data1 data2] (map first (partition-by :id (sort-by :id (concat data1 data2) Since the sorting is stable (relative order is kept), we know that the first occurrence of each id is either the existing map from data1, or the new map from data2. On Tue, Apr 26, 2011 at 5:34 PM, pepijn (aka fliebel) wrote: > Another option is using clojure.set, as is shown here: > https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L99 > > On Apr 26, 10:10 am, Meikel Brandmeyer wrote: > > Hi, > > > > you can construct the output sequence from your input sequences. > > > > (defn merge-data > > [data-orig data-override] > > (let [override-ids (set (map :id data-override))] > > (concat data-override (remove (comp override-ids :id) data-orig > > > > If you need your output sorted you can also add a "(sord-by :id ...)" > around > > the concat. > > > > Sincerely > > Meikel > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
Another option is using clojure.set, as is shown here: https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L99 On Apr 26, 10:10 am, Meikel Brandmeyer wrote: > Hi, > > you can construct the output sequence from your input sequences. > > (defn merge-data > [data-orig data-override] > (let [override-ids (set (map :id data-override))] > (concat data-override (remove (comp override-ids :id) data-orig > > If you need your output sorted you can also add a "(sord-by :id ...)" around > the concat. > > Sincerely > Meikel -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
> Can someone help me write a merge function between two maps? My problem is > as follows: > > I have original data in a map: > > (def data-orig > > '({:id 2 :a2 34 :a3 76 :a4 87}, > > {:id 3 :a2 30 :a3 38 :a4 39}, > > {:id 5 :a2 67 :a3 32 :a4 38}, > > {:id 4 :a2 10 :a3 73 :a4 38}, > > {:id 7 :a2 84 :a3 86 :a4 63})) > > Then I have override data: > > (def data-override > > '({:id 2 :a2 5534 :a3 5576 :a4 5587}, > > {:id 3 :a2 5584 :a3 5586 :a4 5563}, > > {:id 12 :a2 5593 :a3 5512 :a4 5539}, > > {:id 13 :a2 5509 :a3 5539 :a4 5592})) > > The result should be a merge of the two with the following conditions: If > the id is the same, should override the original data. If id is not present > in original then it should be added. The result should be: In the trivial case, you can use clojure.core/merge. Merge takes a variable number of maps and merges them together. If there is a conflict with keys, it will give preference to the latter map in left to right order. But in your case, you can write a trivial function that removes the common maps from data-orig first and then does a merge - (defn merge-override [merge-key override orig] (let [keyset (into #{} (map #(% merge-key) override))] (apply merge override (remove #(keyset (% merge-key)) orig (merge-override :id data-override data-orig) ; => ({:id 7, :a2 84, :a3 86, :a4 63} {:id 4, :a2 10, :a3 73, :a4 38} {:id 5, :a2 67, :a3 32, :a4 38} {:id 2, :a2 5534, :a3 5576, :a4 5587} {:id 3, :a2 5584, :a3 5586, :a4 5563} {:id 12, :a2 5593, :a3 5512, :a4 5539} {:id 13, :a2 5509, :a3 5539, :a4 5592}) ;; pprint - ;; ({:id 7, :a2 84, :a3 86, :a4 63} ;; {:id 4, :a2 10, :a3 73, :a4 38} ;; {:id 5, :a2 67, :a3 32, :a4 38} ;; {:id 2, :a2 5534, :a3 5576, :a4 5587} ;; {:id 3, :a2 5584, :a3 5586, :a4 5563} ;; {:id 12, :a2 5593, :a3 5512, :a4 5539} ;; {:id 13, :a2 5509, :a3 5539, :a4 5592}) Hope that helps. Regards, BG -- Baishampayan Ghose b.ghose at gmail.com -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en