Re: [CMS-PIPELINES] deblocking with various possible linends

2023-03-14 Thread Rob van der Heij
On Tue, 14 Mar 2023 at 00:44, Donald Russell wrote: > —> ... | deblock linend 0a | split 0d | ... > > Could that cause extra lines? > Yes, it does. I agree removing a trailing x0d would have done. But the challenge we see is that a file with just CR will buffer the input before splitting. I've

Re: [CMS-PIPELINES] deblocking with various possible linends

2023-03-14 Thread John P. Hartmann
On 3/14/23 00:44, Donald Russell wrote: If you know the data has crlf or lfcr or just lf but never just cr then … deblock 0a | strip both 0d | … A more reliable way to remove just one cr from either end might be something like change (1 -1) x0d //

Re: [CMS-PIPELINES] deblocking with various possible linends

2023-03-14 Thread Rob van der Heij
On Tue, 14 Mar 2023 at 15:56, John P. Hartmann wrote: > A more reliable way to remove just one cr from either end might be > something like > > change (1 -1) x0d // > The repeating "range" doesn't really do that, but I just checked that strip x0d 1 will take one from either or both sides of

Re: [CMS-PIPELINES] deblocking with various possible linends

2023-03-14 Thread John P. Hartmann
On 3/14/23 16:05, Rob van der Heij wrote: The repeating "range" doesn't really do that, but I just checked that strip x0d 1 will take one from either or both sides of the record. You're right. It gives an error because it supports only ranges relative to the beginning of the record and fro

Re: [CMS-PIPELINES] deblocking with various possible linends

2023-03-14 Thread Rob van der Heij
On Tue, 14 Mar 2023 at 16:59, John P. Hartmann wrote: > On 3/14/23 16:05, Rob van der Heij wrote: > > The repeating "range" doesn't really do that, but I just checked that > > strip x0d 1 will take one from either or both sides of the record. > > You're right. It gives an error because it su

Re: [CMS-PIPELINES] deblocking with various possible linends

2023-03-14 Thread Glenn Knickerbocker
On Tue, 14 Mar 2023 08:07:15 +0100, Rob wrote: >On Tue, 14 Mar 2023 at 00:44, Donald Russell wrote: >> —> ... | deblock linend 0a | split 0d | ... >> Could that cause extra lines? >Yes, it does. I didn't find a case where SPLIT created any extra null records. It will *lose* null lines if ther

Re: [CMS-PIPELINES] deblocking with various possible linends

2023-03-14 Thread Rob van der Heij
Splitting a record with no words will pass the record (see example in usage note :-) On Wed, 15 Mar 2023 at 01:13, Glenn Knickerbocker wrote: > On Tue, 14 Mar 2023 08:07:15 +0100, Rob wrote: > >On Tue, 14 Mar 2023 at 00:44, Donald Russell > wrote: > >> —> ... | deblock linend 0a | split 0d |