Re: [CMS-PIPELINES] swap two records
On 2019-08-18 9:47 AM, amphitr...@ok.de wrote: > RITA revealed that most of the time is lost (or spent ?) in > rearranging the records for later advantageous use. Instead of > inserting a tag to many lines I now tag _all_ and remove where > appropriate, Substr is so much faster than Insert. Right, INSERT has to allocate storage for a new record, while SUBSTR just keeps part of the existing one. CHOP similarly gives you two records for the price of none, and is good in place of FANOUT when you need just the presence of a record, not its contents, to control flow in another stream. CHOP 0 gives you a null record before the original, and CHOP NOT 00-FF gives you one after it. ¬R
Re: [CMS-PIPELINES] swap two records
Am 18.08.2019 um 21:44 schrieb Rob van der Heij: > Did I compete with the join | split approach? Can I also participate with > this? You cap yourself: now 60 ms only. To avoid the Not I bracketed Deal: ... literal | d: deal | drop ... alas there is no further gain by that. Ciao.Mike -- www.Ok.de - die kostenlose E-Mail Adresse
Re: [CMS-PIPELINES] swap two records
On Sun, 18 Aug 2019 at 15:51, wrote: > RITA showed, Glenn's solution is honourable second, while Rob's > suggestion only penultimate far behind ;) > Did I compete with the join | split approach? Can I also participate with this? pipe (end \) literal a b c a b 2 | split | x: if between /b/ 2 | d: not deal | g: gather| x: | cons \ d: | copy | g:
Re: [CMS-PIPELINES] swap two records
RITA showed, Glenn's solution is honourable second, while Rob's suggestion only penultimate far behind ;) The details: I tested only with one log file of 161'000 lines. RITA revealed that most of the time is lost (or spent ?) in rearranging the records for later advantageous use. Instead of inserting a tag to many lines I now tag _all_ and remove where appropriate, Substr is so much faster than Insert. While this preparation takes seconds, swapping of about 600 records is a matter of milliseconds not worth to shorten. But it's hobby and hours are worth a new insight. Glenns solution: 171 ms, with Pick 85 ms, Rob's suggestion: 69 ms. Ciao.Mike -- www.Ok.de - die kostenlose E-Mail Adresse
Re: [CMS-PIPELINES] swap two records
Am 17.08.2019 um 02:23 schrieb Glenn Knickerbocker: > [...] > A way that's a little tricky but avoids messing with combining records > is to use DROP LAST to delay a copy of the file by one record, [...] > > (end /) ... > | dup: fanout > | strfind '_' > | all: faninany > | ... > / dup: > | drop last 1 > | strnfind '_' > | all: > [...] Awesome tricky -- thank you for this idea. Once more I learned something from this list. (The 'if' Rob used in his suggestion was also new to me.) I will use RITA to find out the best solution. Even it's for hobby only I am not happy with the performance of my pipeline, for files of 50'000 lines and more I have to wait several seconds too much. > 0 * * * Top of File * * * > 1 /* RUSPIL EXEC: RedUce PIoneer trace Log */ > 2 "PIPE(end ?)<" arg(1), > 3 '! strlit conditional append',/* a broom for the runout cycle */ > 4 '! spec number 1 *-* nw', /* insert a line number */ > 5 '!a:lookup w2 autoadd master',/* stumbled we over it earlier? */ > 6 '! spec /see line/ 1 *-* 13', /* tag if YES */ > 7 '!b:faninany',/* gather with untagged */ > 8 '! notinside /see/ / /', /* keep only 1st line of rerun */ > 9 '!c:l w1 /see/', /* keep reference lines */ >10 '!d:faninany',/* gather with treated rest */ >11 '!e:if between /see/ 2', /* like Rob suggested: */ >12 '! join x05', /* make it 1 line of 2 columns */ >13 '! spec f2 1 write f1 1', /* write it swapped */ >14 '!e:',/* end if */ >15 '! >' arg(1), /* replace input file */ >16 '?a:!b:', /* short through*/ >17 '?c:',/* from LOCATE */ >18 '! spec stop anyeof a: w1 . select second b: w1 .',/* compute the */ >19'if ^eof() then print a-b 1 *-* nw fi',/* gap to next line */ >20 '!d:' /* to FANINANY */ >21 exit rc >22 * * * End of File * * * Result may look like this: > SQRT LOG C1 V 80 Trunc=80 Size=902 Line=878 Col=1 Alt=0 > > > !...+1+2+3+4+5+6+7... > 8761112 3867 00033C=C+1 A > > 877 see line 9 00035CPEX1 > > 878 29 4979 00312P= 8 > > 879 see line 1120 00314D1=(2) #46 > > 880 70 5008 004CAC=0 S > > 881 see line 1162 004CDP= 12 > > 882 1 5078 00D45RTI > > 8838909 5079 002BB?ST=1 14, GOYES +00026 [002E1] > > 884 see line 1344 002E1ST=014 > > 885 1 13988 002C0C=C+1 B > > 8862033 13989 002C3GONC-C [002B7] > > 887 see line 3855 002B7GOSUB +0018B [00442] > > 888 14 16022 002C6GOSUB +0037E [00644] > > 889 see line 3841 00644D1=(5) #20046 > > 890 1 16036 002CAGONC-00021 [002A9] > > 891 1 16037 002CDP= 12 > > 892 1 16038 002CFGOSUBL +003E3 [006B2] > > 893 1 16039 006B2ST=015 > > 894 1 16040 006B5D1=(5) #20047 > > 895 1 16041 006BCC=0 W > > 896 1 16042 006BFDAT1=C W > > 897 22 16043 006C3D1=(5) #0074F > > 898 see line 1164 006CAD0=(5) #2005D > > 899 5 16065 002D5GOSUB +0016D [00442] > > 900 see line 3856 00442GOLONG +008FB [00D3D] > > 901 1 16070 002D9?ST=1 14, GOYES
Re: [CMS-PIPELINES] swap two records
On 2019-08-14 12:45 PM, amphitr...@ok.de wrote: > In a file I have every now and then a record tagged at the > beginning, all other lines begin with blanks. I'd like to change > the sequence of tagged records with the one directly following it > (which is for sure not tagged). A way that's a little tricky but avoids messing with combining records is to use DROP LAST to delay a copy of the file by one record, and keep the tagged records from that stream. Note that I would normally add a new last record to drop with STRLITERAL APPEND, but if you know the last record won't be tagged we can skip that: (end /) ... | dup: fanout | strfind '_' | all: faninany | ... / dup: | drop last 1 | strnfind '_' | all: If the frequency of tagged records is very small, there might be some performance advantage to picking out and duplicating only the pairs--saves the cost of copying the other records, but adds a third stream for FANINANY to watch: (end /) ... | pair: pick from 1 ¬== ' ' count 2 | dup: fanout | strfind '_' | all: faninany | ... / dup: | drop last 1 | strnfind '_' | all: / pair: | all: ¬R
Re: [CMS-PIPELINES] swap two records
On 2019-08-14 5:11 PM, Rob van der Heij wrote: > So true. My dumb. I'm so used to BETWEEN and friends that I forget about > PICK for it. Now I need to revisit all my pipes and see what improvements > there would be (I already spot one where I used PICK to put the marker for > BETWEEN to find - sigh). For cases like this where the marker is in column 1 already, you could also enhance BETWEEN with syntax to allow multiple or negative selections like PICK: pick from 1 ¬== ' ' count 2 ==> between not ' ' 2 pick from 1 == 'a','b','c' count 2 ==> between 'a','b','c' 2 ¬R
Re: [CMS-PIPELINES] swap two records
Thank you both for all suggestions. I now use Rob's first one, just to keep it as a reminder that Between's "target" argument may also be a number of records. Am 14.08.2019 um 18:49 schrieb Rob van der Heij: > | x: if between /tag/ 2 > | spec 1-* c2x > | join / / > | spec w2 1 w1 nw > | split > | spec 1-* x2c > | x: > > Or close... Yes, very close. As there are for sure no tab chars in input I may do it in three steps: > !e:between /see/ 2 > ! join x05 > ! spec f2 1 write f1 1 In case you are interested in the context, I reduce trace logs from pocket calculator emulators (CPU level simulations) by omitting all repetitions like debouncing loops and so on. A simple lookup autoadd is all to do so, but I want to reference the first occurrence of the dropped block _and_ also show its size. The size is computed by the difference of line numbers using the second reading station of Specs, what delays by one record. So I need to swap two lines. Ciao.Mike -- www.Ok.de - die kostenlose E-Mail Adresse
Re: [CMS-PIPELINES] swap two records
On Wed, 14 Aug 2019 at 22:37, Michael Harding wrote: > Not so. Pick will pass the target and next record and fire off again the > next time the target is recognized. > So true. My dumb. I'm so used to BETWEEN and friends that I forget about PICK for it. Now I need to revisit all my pipes and see what improvements there would be (I already spot one where I used PICK to put the marker for BETWEEN to find - sigh). Rob
Re: [CMS-PIPELINES] swap two records
Not so. Pick will pass the target and next record and fire off again the next time the target is recognized. t pt data PT DATA A1 as of 08/14/19 11.02.42 record 1 record 2 tg tagged 1 record 4 record 5 record 6 tg tagged 2 record 7 record 8 record 9 Ready; T=0.01/0.01 13:35:53 pipe < pt data|rexx pt|cons record 2 record 1 record 4 tg tagged 1 record 6 record 5 record 7 tg tagged 2 record 9 record 8 Ready; T=0.01/0.01 13:36:27 -- Mike Harding z/VM System Support mhard...@us.ibm.com mikehard...@mindless.com (925) 672-3922 | (925) 672-4403 (h) (925) 323-2070 (m) /sp CMSTSO Pipelines Discussion List wrote on 08/14/2019 11:22:45 AM: > From: Rob van der Heij > To: CMS-PIPELINES@VM.MARIST.EDU > Date: 08/14/2019 11:23 AM > Subject: [EXTERNAL] Re: swap two records > Sent by: CMSTSO Pipelines Discussion List > > On Wed, 14 Aug 2019 at 20:18, Michael Harding wrote: > > > Perhaps simpler, as a rexx stage: > > > > '|a:Pick anycase from 1+ == /'tag'/ count 2', > > > Notice that PICK will do the trick only once for the entire stream. I read > Mike's "now and then" as multiple swapped records in a single stream, so > that's why I opted for BETWEEN instead. > > Sir Rob the Plumber > > > > > >
Re: [CMS-PIPELINES] swap two records
On Wed, 14 Aug 2019 at 20:18, Michael Harding wrote: > Perhaps simpler, as a rexx stage: > > '|a:Pick anycase from 1+ == /'tag'/ count 2', Notice that PICK will do the trick only once for the entire stream. I read Mike's "now and then" as multiple swapped records in a single stream, so that's why I opted for BETWEEN instead. Sir Rob the Plumber > >
Re: [CMS-PIPELINES] swap two records
> > I can also do a trick with not deal | gather if you want :-) >> >>
Re: [CMS-PIPELINES] swap two records
Perhaps simpler, as a rexx stage: Parse arg tag . 'CallPipe (End ?) *:', '|a:Pick anycase from 1+ == /'tag'/ count 2', '|Spec a: 1-* . set #0:=a read 1-* n write print #0 1', '|b:Faninany', '|*:', '?a:|b:' Exit Rc -- Mike Harding z/VM System Support mhard...@us.ibm.com mikehard...@mindless.com (925) 672-3922 | (925) 672-4403 (h) (925) 323-2070 (m) /sp CMSTSO Pipelines Discussion List wrote on 08/14/2019 09:49:43 AM: > From: Rob van der Heij > To: CMS-PIPELINES@VM.MARIST.EDU > Date: 08/14/2019 09:50 AM > Subject: [EXTERNAL] Re: swap two records > Sent by: CMSTSO Pipelines Discussion List > > | x: if between /tag/ 2 > | spec 1-* c2x > | join / / > | spec w2 1 w1 nw > | split > | spec 1-* x2c > | x: > > Or close... > > On Wed, 14 Aug 2019 at 18:45, wrote: > > > In a file I have every now and then a record tagged at the > > beginning, all other lines begin with blanks. I'd like to change > > the sequence of tagged records with the one directly following it > > (which is for sure not tagged). > > > > I tried with Fanintwo, but failed. I considered a mod of Throttle > > REXX (based on Dial), another idea with Synchronise, alas even in > > my imagination it gets rather "gordian/gordic(?)". May be Spec > > could help with its second reading station. > > > > Simpler solutions anyone? > > > > Ciao.Mike > > -- > > www.Ok.de - die kostenlose E-Mail Adresse > > >
Re: [CMS-PIPELINES] swap two records
| x: if between /tag/ 2 | spec 1-* c2x | join / / | spec w2 1 w1 nw | split | spec 1-* x2c | x: Or close... On Wed, 14 Aug 2019 at 18:45, wrote: > In a file I have every now and then a record tagged at the > beginning, all other lines begin with blanks. I'd like to change > the sequence of tagged records with the one directly following it > (which is for sure not tagged). > > I tried with Fanintwo, but failed. I considered a mod of Throttle > REXX (based on Dial), another idea with Synchronise, alas even in > my imagination it gets rather "gordian/gordic(?)". May be Spec > could help with its second reading station. > > Simpler solutions anyone? > > Ciao.Mike > -- > www.Ok.de - die kostenlose E-Mail Adresse >
