[REBOL] Re: Trimming values from a series?
Hi, all, Just back from a week camping in the Great Smoky Mountains National Park, and my inbox prompts me to do a quick benchmark... Ed Dana wrote: > > Carl Read wrote: > > >On 25-Jul-02, Ed Dana wrote: > > > >>Maybe the algorithm is something similar to what's used for some > >>compression schemes? > >> > > > >Getting the unique values from a string would be simple. Just have an > >array of 256 flags and tick them as found based on the char values as > >you parse once through the string. Blocks of mixed datatypes > >wouldn't be quite so easy though... > > > > Of course, but somehow I think I'll need to sort more than 256 things, > some day. :) > For the test below, both BSU and BUS are initialized as copies of a series that contains 500 copies of the integers from 1 to 500 (making 25 integer values in all). (Console transcript wrapped for email transmission...) >> t0: now/time/precise rbus: unique sort bus t1: now/time/precise rbsu: sort unique bsu t2: now/time/precise print [to-decimal t1 - t0 to-decimal t2 - t1] 4.347 0.871 I got consistent results over multiple trials, so UNIQUEing first and then SORTing appears faster for blocks of integers than the other order. Independent tests showed that applying only UNIQUE to such a block is faster than applying SORT, so I conclude that UNIQUE is not simply sort-and-sweep. -jn- -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Carl Read wrote: >On 25-Jul-02, Ed Dana wrote: > >>Maybe the algorithm is something similar to what's used for some >>compression schemes? >> > >Getting the unique values from a string would be simple. Just have an >array of 256 flags and tick them as found based on the char values as >you parse once through the string. Blocks of mixed datatypes >wouldn't be quite so easy though... > Of course, but somehow I think I'll need to sort more than 256 things, some day. :) -- Sincerely, | Life is pain, Highness. Anyone who says Ed Dana| differently is selling something. Software Developer | -- The Princess Bride. 1Ghz Athlon Amiga | === http://members.cox.net/edanaii/Home/Default.html === -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Try this out. The datatype affects the performance greatly. b: make block! 1 h: make hash! 1 l: make list! 1 repeat i 1 [append b random i] repeat i 1 [append h random i] repeat i 1 [append l random i] t: now/time/precise loop 20 [unique b] loop 20 [unique b] loop 20 [unique b] print now/time/precise - t t: now/time/precise loop 20 [unique h] loop 20 [unique h] loop 20 [unique h] print now/time/precise - t t: now/time/precise loop 20 [unique l] loop 20 [unique l] loop 20 [unique l] print now/time/precise - t --Gregg -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Anton Rolls wrote: > I think it should be faster this way, too, > although it may not matter. Just for those interested ... >> a: [ a d e be e sn js am xmed dms d d a s e d s a s e s sa s d f de e s s dd fa s d d x cfas sd sd fa sd fas df asdf sd ] == [a d e be e sn js am xmed dms d d a s e d s a s e s sa s d f de e s s dd fa s d d x cfas sd sd fa sd fas df asdf sd] >> profiler/test [ unique sort copy a ] 1 == [0:00:04.271148] >> profiler/test [ unique sort copy a ] 1 == [0:00:04.794179] >> profiler/test [ unique sort copy a ] 1 == [0:00:04.29976] >> profiler/test [ sort unique copy a ] 1 == [0:00:03.604301] >> profiler/test [ sort unique copy a ] 1 == [0:00:03.57617] >> profiler/test [ sort unique copy a ] 1 == [0:00:03.555364] >> profiler/test [ sort unique copy a ] 1 == [0:00:03.681827] Of course this isn't really excessive testing, but it hints that you may be right. Kind regards, Ingo -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
unique v[sort v] -- heterogenous list chars Is split str unique chars[sort chars] > ** Original Subject: [REBOL] Re: Trimming values from a series? > ** Original Sender: Carl Read <[EMAIL PROTECTED]> > ** Original Date: Wed, 24 Jul 2002 23:56:16 -0500 > ** Original Message follows... > > On 25-Jul-02, Ed Dana wrote: > > > Sorting this string, for example, produces the following results: > >>> UNIQUE "Now is the time for all good men to come to the aid of > > their country" > > == "Now isthemfralgdcuy" > > > Which suggests that either UNIQUE uses some other algorithm, or it > > sorts it and then puts it back to its original form. The latter, > > would be very inefficient, of course, so I vote for the former. > > > Maybe the algorithm is something similar to what's used for some > > compression schemes? > > Getting the unique values from a string would be simple. Just have an > array of 256 flags and tick them as found based on the char values as > you parse once through the string. Blocks of mixed datatypes > wouldn't be quite so easy though... > > -- > Carl Read > > -- > To unsubscribe from this list, please send an email to > [EMAIL PROTECTED] with "unsubscribe" in the > subject, without the quotes. >** - End Original Message --- ** > Download NeoPlanet at http://www.neoplanet.com -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
On 25-Jul-02, Ed Dana wrote: > Sorting this string, for example, produces the following results: >>> UNIQUE "Now is the time for all good men to come to the aid of > their country" > == "Now isthemfralgdcuy" > Which suggests that either UNIQUE uses some other algorithm, or it > sorts it and then puts it back to its original form. The latter, > would be very inefficient, of course, so I vote for the former. > Maybe the algorithm is something similar to what's used for some > compression schemes? Getting the unique values from a string would be simple. Just have an array of 256 flags and tick them as found based on the char values as you parse once through the string. Blocks of mixed datatypes wouldn't be quite so easy though... -- Carl Read -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Carl Read wrote: >On 25-Jul-02, Ed Dana wrote: > >>>Hmmm. Are there cases where unique can change the order, as well as >>>stripping out duplicates? >>> > >>I would expect that UNIQUE first sorts the data in order to trim out >>the duplicates. It's the most efficient way to find and remove them. >>Which is why I went looking for it in SORT first. >> > >>I doubt that issuing SORT and UNIQUE in any combination is faster >>because of the redundancy. >> > >>I could be wrong, though... >> > >I think you are. Unique is much faster than sort, at least on Amiga. >Umm, well, at least on the block of 1s and 2s in a block I've just >tried. (: This suggests unique doesn't use sort, or at least not the >sort available to us. > I probably am. :) Sorting this string, for example, produces the following results: >> UNIQUE "Now is the time for all good men to come to the aid of their country" == "Now isthemfralgdcuy" Which suggests that either UNIQUE uses some other algorithm, or it sorts it and then puts it back to its original form. The latter, would be very inefficient, of course, so I vote for the former. Maybe the algorithm is something similar to what's used for some compression schemes? Just a thought... -- Sincerely, | Ed Dana| Courage is fear holding on a minute longer. Software Developer | -- General George S. Patton 1Ghz Athlon Amiga | === http://members.cox.net/edanaii/Home/Default.html === -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
On 25-Jul-02, Ed Dana wrote: >> Hmmm. Are there cases where unique can change the order, as well as >> stripping out duplicates? > I would expect that UNIQUE first sorts the data in order to trim out > the duplicates. It's the most efficient way to find and remove them. > Which is why I went looking for it in SORT first. > I doubt that issuing SORT and UNIQUE in any combination is faster > because of the redundancy. > I could be wrong, though... I think you are. Unique is much faster than sort, at least on Amiga. Umm, well, at least on the block of 1s and 2s in a block I've just tried. (: This suggests unique doesn't use sort, or at least not the sort available to us. -- Carl Read -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Carl Read wrote: >On 24-Jul-02, Brett Handley wrote: > >>Louis has already pointed out Unique >> > >>If you need your series sorted, it is probably better to sort after >>the Unique. This way you don't need to hope that Unique keeps the >>sorting intact. >> > >> sort unique [ "a" "b" "b" "c" "d" "e" "e" ] >> > >Hmmm. Are there cases where unique can change the order, as well as >stripping out duplicates? > I would expect that UNIQUE first sorts the data in order to trim out the duplicates. It's the most efficient way to find and remove them. Which is why I went looking for it in SORT first. I doubt that issuing SORT and UNIQUE in any combination is faster because of the redundancy. I could be wrong, though... -- Sincerely, | For long you live and high you fly. Ed Dana| And smiles you'll give and tears you'll cry. Software Developer | And all you touch and all you see, 1Ghz Athlon Amiga | Is all your life will ever be. | -- Pink Floyd, Breathe. === http://members.cox.net/edanaii/Home/Default.html === -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
I can't help you with fire, I am sorry. Too far away. There might be a fireman around your place somewhere in Georgia, though. Anton. > By the way, everybody, it is good to be back on my favorite list after a > fire (actually burned our cable in two) and other problems cut me off for > about 2 months. I have received more help on this list than on any other > list I have been on. The regulars on this list are not only > brilliant, but > also patient and quick to help those in need (often me :>)). > Thanks to you > all! > > Louis -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
I think it should be faster this way, too, although it may not matter. sort will have to rearrange less items after unique has removed some. I don't think unique would work any faster on data that's already sorted, though I could be wrong. Not a big issue for me at the moment.. :) Anton. > Hi Carl, > > > > If you need your series sorted, it is probably better to sort after > > > the Unique. This way you don't need to hope that Unique keeps the > > > sorting intact. > > > > >sort unique [ "a" "b" "b" "c" "d" "e" "e" ] > > > > Hmmm. Are there cases where unique can change the order, as well as > > stripping out duplicates? > > I don't really know, I haven't tested for it. But even if it is stable in > this version it could perhaps change in later versions and > because the help > for Unique does not suggest otherwise, I figure a bit of defensive > programming is warranted. > > My habit is to think: "if I need it like that - then I should > ensure it is > like that". > > Regards, > Brett. -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Hi Brett, I hope that these functions (unique intersect exclude) do not mess the list. Sometime, the order has value, but it is not the result of sort or of any other function, so can't be recreated. --- Ciao Romano -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Hi Carl, > > If you need your series sorted, it is probably better to sort after > > the Unique. This way you don't need to hope that Unique keeps the > > sorting intact. > > >sort unique [ "a" "b" "b" "c" "d" "e" "e" ] > > Hmmm. Are there cases where unique can change the order, as well as > stripping out duplicates? I don't really know, I haven't tested for it. But even if it is stable in this version it could perhaps change in later versions and because the help for Unique does not suggest otherwise, I figure a bit of defensive programming is warranted. My habit is to think: "if I need it like that - then I should ensure it is like that". Regards, Brett. -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
On 24-Jul-02, Brett Handley wrote: > Louis has already pointed out Unique > If you need your series sorted, it is probably better to sort after > the Unique. This way you don't need to hope that Unique keeps the > sorting intact. >sort unique [ "a" "b" "b" "c" "d" "e" "e" ] Hmmm. Are there cases where unique can change the order, as well as stripping out duplicates? -- Carl Read -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Brett and Ashley, Thanks. I have learned something new. For some reason I thought that the order had to be: unique sort x but a little experimentation proves you right. My misconception probably stems from the fact that I once wrote a c function similar to unique that wouldn't work unless the input was sorted first. By the way, everybody, it is good to be back on my favorite list after a fire (actually burned our cable in two) and other problems cut me off for about 2 months. I have received more help on this list than on any other list I have been on. The regulars on this list are not only brilliant, but also patient and quick to help those in need (often me :>)). Thanks to you all! Louis At 11:45 AM 7/24/2002 +1000, you wrote: > >If you need your series sorted, it is probably better to sort after the > >Unique. This way you don't need to hope that Unique keeps the sorting > >intact. > > > > sort unique [ "a" "b" "b" "c" "d" "e" "e" ] > > > >Regards, > >Brett. > >Also, at least conceptually, less for sort to do so more efficient . . . > >Regards, > > Ashley > >-- >To unsubscribe from this list, please send an email to >[EMAIL PROTECTED] with "unsubscribe" in the >subject, without the quotes. -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Ed Dana wrote: > How do I do it? > > I know I can use the sort command to put them in order, how can I trim > out the duplicates? > > Let's say I have a series, that I've sorted as follows: [ "a" "b" "b" > "c" "d" "e" "e" ]. > > Is there a command I can us to reduce it to: [ "a" "b" "c" "d" "e" ]? > OK, Nevermind. I found it. Looks like the UNIQUE command will do the job. So many functions, so little time... :) -- Sincerely, | The problems of two little people don't amount to Ed Dana| a hill of beans in this crazy mixed-up world! But Software Developer | this is OUR hill, and these are OUR beans! 1Ghz Athlon Amiga | -- Naked Gun via Casablanca. === http://members.cox.net/edanaii/Home/Default.html === -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
>If you need your series sorted, it is probably better to sort after the >Unique. This way you don't need to hope that Unique keeps the sorting >intact. > > sort unique [ "a" "b" "b" "c" "d" "e" "e" ] > >Regards, >Brett. Also, at least conceptually, less for sort to do so more efficient . . . Regards, Ashley -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Hi again Ed, Here is an example of use: >> x: [ "a" "b" "b" "c" "d" "e" "e" ] == ["a" "b" "b" "c" "d" "e" "e"] >> unique sort x == ["a" "b" "c" "d" "e"] >> Louis At 05:28 PM 7/23/2002 -0700, you wrote: >How do I do it? > >I know I can use the sort command to put them in order, how can I trim out >the duplicates? > >Let's say I have a series, that I've sorted as follows: [ "a" "b" "b" "c" >"d" "e" "e" ]. > >Is there a command I can us to reduce it to: [ "a" "b" "c" "d" "e" ]? > >-- >Sincerely, | Don't part with your illusions. When they are gone >Ed Dana| you may still exist, but you have ceased to live. >Software Developer | -- Mark Twain >1Ghz Athlon Amiga | === >http://members.cox.net/edanaii/Home/Default.html === > > > >-- >To unsubscribe from this list, please send an email to >[EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes. -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Louis has already pointed out Unique If you need your series sorted, it is probably better to sort after the Unique. This way you don't need to hope that Unique keeps the sorting intact. sort unique [ "a" "b" "b" "c" "d" "e" "e" ] Regards, Brett. -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
[REBOL] Re: Trimming values from a series?
Hi Ed, Type help unique at the console for the answer. Louis At 05:28 PM 7/23/2002 -0700, you wrote: >How do I do it? > >I know I can use the sort command to put them in order, how can I trim out >the duplicates? > >Let's say I have a series, that I've sorted as follows: [ "a" "b" "b" "c" >"d" "e" "e" ]. > >Is there a command I can us to reduce it to: [ "a" "b" "c" "d" "e" ]? > >-- >Sincerely, | Don't part with your illusions. When they are gone >Ed Dana| you may still exist, but you have ceased to live. >Software Developer | -- Mark Twain >1Ghz Athlon Amiga | === >http://members.cox.net/edanaii/Home/Default.html === > > > >-- >To unsubscribe from this list, please send an email to >[EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes. -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
