Okay, I timed the difference with 1,400,000 or so record ids on a relatively old SUN box.
Union against an identical list 64930 milliseconds of CPU Union with a blank list 31890 ms. The result is the same. The timings were started after both lists were instantiated, and are for the MERGE.LIST operation only. -Rick -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Rick Nuckolls Sent: Thursday, May 17, 2012 2:22 PM To: 'U2 Users List' Subject: Re: [U2] SAVING UNIQUE ( multivalued) I do not know the guts of the MERGE.LIST command, but it may well sort and dedupe each list prior to performing the merge /union operation. But, for the sake of argument, suppose that, for a union, MERGE.LIST creates a single internal hashmap and makes an entry for each unique item in list1 and list2. Using a blank list for the either list in "MERGE.LIST list1 UNION list2" will produce the same result as using a duplicate list, but with just slightly more than half the work. -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin Sent: Thursday, May 17, 2012 2:15 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] SAVING UNIQUE ( multivalued) Rick, I'm not following you.. There isn't a 'second list'.. it's one list that is being manipulated (George's example only uses 1 list) which he calls 1 and UNIQUE (interchangeably) . Perhaps you can show us what you're referring to so we can see an example.. I thought George's solution was excellent. Chris > From: r...@lynden.com > To: u2-users@listserver.u2ug.org > Date: Thu, 17 May 2012 14:07:24 -0700 > Subject: Re: [U2] SAVING UNIQUE ( multivalued) > > UNION would definely be an alternative, but doing it against a needlessly > long second list offends my Scotch genes. It is possible to create an active, > zero length, selectlist programmatically, so perhaps I should write a little > command to do that and and a MERGE.LIST UNION against it. > > > > -Rick > > On May 17, 2012, at 1:47 PM, "George Gallen" <ggal...@wyanokegroup.com> wrote: > > > What about ... > > > > ----: P > > > > 0001: PA > > > > 0002: QSELECT <<C2,file>> * SAVING <<C3,field number>> > > > > 0003: SAVE.LIST UNIQUE > > 0004: GET.LIST UNIQUE TO 1 > > 0005: MERGE.LIST 1 UNION 1 > > 0006: SAVE.LIST UNIQUE > > 0007: GET.LIST UNIQUE > > > > (The MERGE.LIST will sort and dedupe) > > > > > > 0004: sh -c "/usr/bin/sort -uo '&SAVEDLISTS&/UNIQUE' '&SAVEDLISTS&/UNIQUE'" > > > > 0005: GET.LIST UNIQUE > > > > -----Original Message----- > > From: u2-users-boun...@listserver.u2ug.org > > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Rick Nuckolls > > Sent: Thursday, May 17, 2012 4:38 PM > > To: 'U2 Users List' > > Subject: Re: [U2] SAVING UNIQUE ( multivalued) > > > > This does present a challenge, > > > > The following (non-portable solution works) > > :ED VOC MUNIQUE > > > > 5 lines long. > > > > > > ----: P > > > > 0001: PA > > > > 0002: QSELECT <<C2,file>> * SAVING <<C3,field number>> > > > > 0003: SAVE.LIST UNIQUE > > > > 0004: sh -c "/usr/bin/sort -uo '&SAVEDLISTS&/UNIQUE' '&SAVEDLISTS&/UNIQUE'" > > > > 0005: GET.LIST UNIQUE > > > > Or, I suppose that one could also > > > > EDIT.LIST GARBAGE > > > > New record. > > > > > > ----: I > > > > 0001= KAJDF;LJASDLF;KJADSL;FKJASDL;FJKASLD;JFALS;DJFLASKDF > > > > 0002= > > > > Bottom at line 1. > > > > ----: > > > > Top. > > > > ----: FI > > > > "GARBAGE" filed in file "&SAVEDLISTS&". > > > > :GET.LIST GARBAGE TO 1 > > > > > > 1 record(s) selected to SELECT list #1. > > > > :QSELECT IU * SAVING 19 > > > > > > 32987 record(s) selected to SELECT list #0. > > > > ::MERGE.LIST 0 DIFF 1 > > > > > > 90 record(s) selected to SELECT list #0. > > > > :: > > > > ---------------- > > Be nice if there was MV support of 'SAVING UNIQUE', or a cleaner syntactic > > way to use MERGE.LIST to dedupe. > > > > -Rick > > > > > > > > -----Original Message----- > > From: u2-users-boun...@listserver.u2ug.org > > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin > > Sent: Thursday, May 17, 2012 1:20 PM > > To: u2-users@listserver.u2ug.org > > Subject: Re: [U2] SAVING UNIQUE ( multivalued) > > > > > > I usually get my exploded list and then DEDUPE using the following: > > > > LOCATE VALUE.TO.FIND IN UNIQUE.LIST BY "AL" SETTING POS ELSE > > UNIQUE.LIST = INSERT(UNIQUE.LIST,1,POS,0,VALUE.TO.FIND) > > END > > > > This puts everything in a nice ARRAY called UNIQUE.LIST, I then can use a > > FOR loop to loop through as needed. > > > > Chris > > > > > >> To: u2-users@listserver.u2ug.org > >> From: wjhon...@aol.com > >> Date: Thu, 17 May 2012 16:13:31 -0400 > >> Subject: Re: [U2] SAVING UNIQUE ( multivalued) > >> > >> > >> You have two issues. > >> One is to create a list from an exploded multi-value > >> The other issue, is to dedupe that list. > >> > >> Without using something like REFORMAT I don't think you can get the two > >> issues resolved at once. > >> Qselect does not allow Unique > >> So > >> QSELECT MYFILE SAVING 2 > >> will explode the multi-values, but it won't dedupe them. > >> > >> > >> > >> -----Original Message----- > >> From: Rick Nuckolls <r...@lynden.com> > >> To: 'U2 Users List' <u2-users@listserver.u2ug.org> > >> Sent: Thu, May 17, 2012 1:06 pm > >> Subject: [U2] SAVING UNIQUE ( multivalued) > >> > >> > >> Does anyone know a trick to achieve the equivalent of > >> SELECT filename SAVING UNIQUE fieldname > >> When "fieldname" is the name of a multivalued field, and I want a list of > >> all > >> he unique values for that field? > >> The syntax above does not explode the values, but just looks at the whole > >> field. > >> I suppose that I can do a LIST with a BY.EXP & a BREAK.ON, but perhaps I > >> am > >> issing something... > >> Thanks, > >> Rick Nuckolls > >> ynden Inc. > >> > >> ______________________________________________ > >> 2-Users mailing list > >> 2-us...@listserver.u2ug.org > >> ttp://listserver.u2ug.org/mailman/listinfo/u2-users > >> > >> _______________________________________________ > >> U2-Users mailing list > >> U2-Users@listserver.u2ug.org > >> http://listserver.u2ug.org/mailman/listinfo/u2-users > > > > _______________________________________________ > > U2-Users mailing list > > U2-Users@listserver.u2ug.org > > http://listserver.u2ug.org/mailman/listinfo/u2-users > > _______________________________________________ > > U2-Users mailing list > > U2-Users@listserver.u2ug.org > > http://listserver.u2ug.org/mailman/listinfo/u2-users > > _______________________________________________ > > U2-Users mailing list > > U2-Users@listserver.u2ug.org > > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
