Henry wrote:
> Testing things so short, test multiple runs.
Also, here you are including parsing times which might exceed executing
time.
I've run the following test plan with j904 Beta-d:
9!:56'cpu'
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
timex'(;:inv) z'
timex'([: ; '' '',each~ ])z'
on:
++ Mac MBA early 2014 (x86_64)
++ Mac mini M1 chip (native arm64)
++ Mac mini M1 chip (Rosetta x86_64 emulation)
Note the substantially longer sample LF-string I've used, viz the text of
stdlib.ijs .
I do have a somewhat smarter method of averaging repeated timings. But for
a forensic session I thought I'd use what everyone else has got.
Mac MBA early 2014 (x86_64)
9!:56'cpu'
x86_64
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
60421
timex'(;:inv) z'
0.015428
timex'(;:inv) z'
0.013476
timex'(;:inv) z'
0.013629
timex'(;:inv) z'
0.013494
timex'(;:inv) z'
0.014735
timex'(;:inv) z'
0.013335
timex'(;:inv) z'
0.013783
timex'(;:inv) z'
0.013353
(inserted by hand): mean value: 0.0139041
timex'([: ; '' '',each~ ])z'
0.013806
timex'([: ; '' '',each~ ])z'
0.013383
timex'([: ; '' '',each~ ])z'
0.013903
timex'([: ; '' '',each~ ])z'
0.013341
timex'([: ; '' '',each~ ])z'
0.013323
timex'([: ; '' '',each~ ])z'
0.013576
timex'([: ; '' '',each~ ])z'
0.011759
timex'([: ; '' '',each~ ])z'
0.013209
(inserted by hand): mean value: 0.0132875
Mac mini M1 chip (native arm64)
9!:56'cpu'
arm64
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
60421
timex'(;:inv) z'
0.012289
timex'(;:inv) z'
0.013109
timex'(;:inv) z'
0.01277
timex'(;:inv) z'
0.013593
timex'(;:inv) z'
0.014168
timex'(;:inv) z'
0.014498
timex'(;:inv) z'
0.011915
timex'(;:inv) z'
0.011848
(inserted by hand): mean value: 0.0130238
timex'([: ; '' '',each~ ])z'
0.011718
timex'([: ; '' '',each~ ])z'
0.013287
timex'([: ; '' '',each~ ])z'
0.011614
timex'([: ; '' '',each~ ])z'
0.01695
timex'([: ; '' '',each~ ])z'
0.015872
timex'([: ; '' '',each~ ])z'
0.013207
timex'([: ; '' '',each~ ])z'
0.012396
timex'([: ; '' '',each~ ])z'
0.014463
(inserted by hand): mean value: 0.0136884
Mac mini M1 chip (Rosetta x86_64 emulation)
9!:56'cpu'
x86_64
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
60421
timex'(;:inv) z'
0.014958
timex'(;:inv) z'
0.013211
timex'(;:inv) z'
0.014644
timex'(;:inv) z'
0.012561
timex'(;:inv) z'
0.012615
timex'(;:inv) z'
0.012795
timex'(;:inv) z'
0.012811
timex'(;:inv) z'
0.014273
(inserted by hand): mean value: 0.0134835
timex'([: ; '' '',each~ ])z'
0.010994
timex'([: ; '' '',each~ ])z'
0.011156
timex'([: ; '' '',each~ ])z'
0.011643
timex'([: ; '' '',each~ ])z'
0.011929
timex'([: ; '' '',each~ ])z'
0.0116
timex'([: ; '' '',each~ ])z'
0.012876
timex'([: ; '' '',each~ ])z'
0.011646
timex'([: ; '' '',each~ ])z'
0.01134
(inserted by hand): mean value: 0.011648
I conclude from this trial that with such a long string (z) there is
surprisingly little difference in execution time between j phrases,
architectures, or even Mac models.
Maybe z is too long? I wonder if just one non-CPU-dependent process isn't
dominating the transaction. Memory allocation? Both the MBA and Mac mini
use flash memory. Maybe that hasn't improved as much as CPUs since 2014,
the date of my MBA.
I typically use Zulu (the existing addon) with short strings: 10 to 100
bytes, so the parsing time is part of the deal for me. Though Zulu's
follow-on will get used routinely to process jscripts – though maybe only
line-by-line. So I'm back in the 10-100 byte bracket.
Ian Clark
On Mon, 30 May 2022 at 11:47, Henry Rich <[email protected]> wrote:
> Testing things so short, test multiple runs.
>
> Also, here you are including parsing times which might exceed executing
> time.
>
> Henry Rich
>
> On Sun, May 29, 2022, 11:16 PM Ian Clark <[email protected]> wrote:
>
> > > As well ask why you found it at all.
> >
> > I doubt I'd have found it at all, had not j904 crashed as soon as I gave
> it
> > my pet startup.ijs.
> > Which just happens to run a deeply paranoid test.
> >
> > > I am surprised that ;:inv is slower than ([: ; SP,each~ ]).
> >
> > So was I…
> >
> > z=: ;:'alpha bravo charlie '
> >
> > timex'(;:inv) z'
> >
> > 3.8e_5
> >
> > timex'([: ; '' '',each~ ])z'
> >
> > 3.1e_5
> >
> > JVERSION
> >
> > Engine: j904/j64/darwin
> >
> > Beta-d: commercial/2022-05-19T20:41:44
> >
> > Library: 9.04.01
> >
> > Qt IDE: 2.0.3/6.2.4(6.2.4)
> >
> > Platform: Darwin 64
> >
> > Installer: J904 install
> >
> > InstallPath: /applications/j904
> >
> > Contact: www.jsoftware.com
> >
> >
> > Even slower, in jconsole…
> >
> >
> > z=: ;:'alpha bravo charlie '
> > timex'(;:inv)z'
> > 4.2e_5
> > timex'([: ; '' '',each~ ])z'
> > 2.7e_5
> > JVERSION
> > Engine: j904/j64/darwin
> > Beta-d: commercial/2022-05-19T20:41:44
> > Library: 9.04.01
> > Platform: Darwin 64
> > Installer: J904 install
> > InstallPath: /applications/j904
> > Contact: www.jsoftware.com
> >
> >
> > But the more I use it, I sense the Apple M1 chip in my Mac mini is
> > lightning-fast at moving bytes around in memory, as opposed to doing
> > significant computation – when it doesn't perform all that much better
> than
> > the Intel chip Apple has discarded.
> >
> > Maybe not that surprising when you ponder the meaning of "reduced
> > instruction set".
> > M1 is an ARM architecture. (Which JVERSION doesn't show – maybe it
> should?)
> > Unless j904_mac64.zip is compiled for ARM, then j904 will be running
> under
> > Rosetta emulation of Intel architecture. I don't know how to discover if
> > that's happening.
> >
> >
> > On Sat, 28 May 2022 at 04:22, Henry Rich <[email protected]> wrote:
> >
> > > I am surprised that ;:inv is slower than ([: ; SP,each~ ]). That needs
> > > looking into. But ([: ; SP,each~ ]) uses a few features I have added
> > > (append in place inside boxes, and support for ;@:(f&.>) with fewer
> > > passes over the data) which is gratifying.
> > >
> > > When would you have found the beta-d bug you found? As well ask why
> you
> > > found it at all. AFAICS no intentional change in beta-d caused the
> > > error: a bit was being cleared erroneously and somehow in beta-d the
> bit
> > > was in an address.
> > >
> > > The older I get the more I realize there is no such thing as a working
> > > program. Even IEFBR14 had a bug.
> > >
> > > Henry Rich
> > >
> > >
> > >
> > >
> > > On 5/27/2022 10:53 PM, Ian Clark wrote:
> > > > Thanks for the ideas, Raul. Here's my grid:
> > > >
> > > > ┌───┬───────────────┬───────────────┬───────────────┬───────────────┐
> > > >
> > > > │ │ B(boxed) │ F(LFend) │ O(open) │ X(matrix) │
> > > >
> > > > ├───┼───────────────┼───────────────┼───────────────┼───────────────┤
> > > >
> > > > │B4*│] │cuT&aLF │cuT&aSP │[: dtb each <"1│
> > > >
> > > > ├───┼───────────────┼───────────────┼───────────────┼───────────────┤
> > > >
> > > > │F4*│[: ; LF,each~ ]│aLF │(SP,LF)&charsub│[: F4B B4X │
> > > >
> > > > ├───┼───────────────┼───────────────┼───────────────┼───────────────┤
> > > >
> > > > │O4*│[: ; SP,each~ ]│(LF,SP)&charsub│aSP │[: O4B B4X │
> > > >
> > > > ├───┼───────────────┼───────────────┼───────────────┼───────────────┤
> > > >
> > > > │X4*│> │>&(cuT&aLF) │>&(cuT&aSP) │] │
> > > >
> > > > └───┴───────────────┴───────────────┴───────────────┴───────────────┘
> > > >
> > > >
> > > > SP=: ' ' NB. (which IMO ought to be in stdlib)
> > > >
> > > > cuT=: <;._2
> > > >
> > > > aLF=: ] , LF #~ LF ~: {: NB. append LF if not there already
> > > >
> > > > aSP=: ] , SP #~ SP ~: {: NB. append ' ' if not there already
> > > >
> > > >
> > > > *Observations*:
> > > >
> > > >
> > > > I wish I'd known stdlib had cutLF, so thanks for that.
> > > >
> > > > Your use of (;:inv) is neat.
> > > > However timex tells me ([: ; SP,each~ ]) is faster on the Mac with M1
> > > chip,
> > > > so I'll go with that for now.
> > > >
> > > >
> > > > Providing verbs B4B, F4F… looks otiose, but simplifies the design of
> > > > consistency tests.
> > > >
> > > >
> > > > Personally I prefer the name O4X (say) to X2O. I used to employ the
> > > latter,
> > > > but saw Roger preferring names the other way round and found it
> > > > worked better for me too. Easier to check visually: see the last
> column
> > > > above.
> > > >
> > > > Accordingly I extended Zulu (~addons/format/zulu/*) with yet another
> > > > optional script, which defined o4x=:x2o etc. But that just made it
> even
> > > > more unwieldy.
> > > >
> > > >
> > > > This design, which I'm calling BFOX, corrects the wrong Zulu policy
> > > > regarding 'f' format by making a final LF mandatory in the new 'F'
> > > format.
> > > > This has several advantages. For a start it's what you get with:
> > > >
> > > > zuF=: noun define
> > > >
> > > > alpha
> > > >
> > > > bravo
> > > >
> > > > charlie
> > > >
> > > > )
> > > >
> > > > In Zulu this gives 4 rows, not 3 as under the current design.
> > > >
> > > >
> > > > It seems logical to me to extend the same policy to O format, since I
> > > mean
> > > > to implement O and F with identical code, merely switching SP <-->
> LF.
> > > >
> > > >
> > > > I want to avoid breaking existing code. So I think I'll create a new
> > > addon:
> > > > BFOX (~addons/format/bfox.ijs --all in a single script) and
> standardize
> > > on
> > > > *4* naming.
> > > >
> > > >
> > > > Instead of a mandatory test on loading the addon, I'll also provide a
> > > > separate test script (as addons are supposed to do), which checks
> > logical
> > > > consistency by converting via several routes back to the original,
> plus
> > > > addressing nouns with 0 or 1 "rows", plus all the other good things
> > Zulu
> > > > does. I've yet to do this, but I fancy it'll all be a lot neater,
> with
> > > > fewer edge cases to handle.
> > > >
> > > >
> > > > At least that was my plan until it struck me BFOX might make a better
> > > > Foreign in the (3!:) range, than yet another potentially klugey
> addon.
> > > But
> > > > this is all making work for Henry…!
> > > >
> > > >
> > > > Another thought: if Zulu hadn't done its mandatory suite of tests on
> > > > loading, when if ever would I have hit this bug in Beta-d?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, 26 May 2022 at 17:23, Raul Miller <[email protected]>
> > wrote:
> > > >
> > > >> (I sort of ignored the empty string case, obviously - in some cases
> > > >> these conversions would discard them. In others, it would preserve
> > > >> them. It's something of a judgement call ... somewhat analogous to
> > > >> forum choice, which I perhaps abused in my previous message.)
> > > >>
> > > >> --
> > > >> Raul
> > > >>
> > > >> On Thu, May 26, 2022 at 12:19 PM Raul Miller <[email protected]
> >
> > > >> wrote:
> > > >>> Here's how I think I'd tackle that "grid":
> > > >>>
> > > >>> b2f ;,&LF&.> b
> > > >>> b2o ;:inv b
> > > >>> b2x > b
> > > >>>
> > > >>> f2b cutLF f
> > > >>> f2o rplc&(LF,' ') f
> > > >>> f2x >cutLF f
> > > >>>
> > > >>> o2b cut o
> > > >>> o2f rplc&(' ',LF) o
> > > >>> o2x >cut o
> > > >>>
> > > >>> x2b <@dtb"1 x
> > > >>> x2f ;<@(,&LF)@dtb"1 x
> > > >>> x2o deb,' ',. x
> > > >>>
> > > >>> FYI,
> > > >>>
> > > >>> --
> > > >>> Raul
> > > >>>
> > > >>> On Thu, May 26, 2022 at 11:46 AM Ian Clark <[email protected]>
> > > >> wrote:
> > > >>>> Henry wrote:
> > > >>>>> The problem was with the form
> > > >>>> [&.>@:(<"0) 'ab'
> > > >>>> (used in x2b).
> > > >>>>
> > > >>>> x2b is defined as:
> > > >>>> [: (#~ ([: +./\. ' '&~:))&.> <"1
> > > >>>> so I guess that was a typo. But I get the idea.
> > > >>>>
> > > >>>> So… no need for me to make any changes to 'format/zulu' just yet.
> > > >>>>
> > > >>>> ASIDE:
> > > >>>> --but when I do, I'll throw away the entire implementation.
> > > >>>>
> > > >>>> Zulu addresses an enduring need for the J beginner wanting to
> > develop
> > > >> code
> > > >>>> for distribution. But it's deprecated for operational use.
> > > >>>>
> > > >>>> Zulu is an awful example of the edge-case 'tail' wagging the JAL
> > > 'dog'.
> > > >>>>
> > > >>>> If we ever get a *usable* addon, or a primitive (?), which
> > > >> comprehensively
> > > >>>> supports dictionaries, or even just collections of strings, then I
> > > >> hope &
> > > >>>> pray it eats Zulu for breakfast.
> > > >>>>
> > > >>>>
> > > >>>> Zulu's aims could be met by publishing a 4x4 grid of recommended
> > > >>>> *consistent* conversions between the 4 common "strings" formats:
> > > >>>>
> > > >>>> b - boxed
> > > >>>>
> > > >>>> f - LF-separated
> > > >>>>
> > > >>>> o - open
> > > >>>>
> > > >>>> x - matrix
> > > >>>>
> > > >>>> See the lab: Strings conversion package.
> > > >>>>
> > > >>>>
> > > >>>> WHAT'S MORE:
> > > >>>>
> > > >>>> A user-facing app doing smart things with LF-separated strings
> must
> > > >> adopt a
> > > >>>> policy towards final LF, especially for handling a collection of
> > > >> substrings
> > > >>>> containing 0 or 1 members.
> > > >>>>
> > > >>>> Zulu adopts the wrong policy. And pays for it with lots of slow
> > > >> edge-case
> > > >>>> code.
> > > >>>>
> > > >>>> On Wed, 25 May 2022 at 18:24, Henry Rich <[email protected]>
> > > wrote:
> > > >>>>
> > > >>>>> Fixed for next beta. The problem was with the form
> > > >>>>>
> > > >>>>> [&.>@:(<"0) 'ab'
> > > >>>>>
> > > >>>>> (used in x2b). The <"n created virtual blocks but erroneously
> > > >> flagged
> > > >>>>> them as inplaceable.
> > > >>>>>
> > > >>>>> Workaround: replace (<"n) with (<"<"n).
> > > >>>>>
> > > >>>>> Henry Rich
> > > >>>>>
> > > >>>>> On 5/24/2022 8:25 PM, Ian Clark wrote:
> > > >>>>>> Note: Addon 'format/zulu' does not crash 903 or earlier.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> JVERSION
> > > >>>>>>
> > > >>>>>> Engine: j904/j64/darwin
> > > >>>>>>
> > > >>>>>> Beta-d: commercial/2022-05-19T20:41:44
> > > >>>>>>
> > > >>>>>> Library: 9.04.01
> > > >>>>>>
> > > >>>>>> Qt IDE: 2.0.3/6.2.4(6.2.4)
> > > >>>>>>
> > > >>>>>> Platform: Darwin 64
> > > >>>>>>
> > > >>>>>> Installer: J904 install
> > > >>>>>>
> > > >>>>>> InstallPath: /applications/j904
> > > >>>>>>
> > > >>>>>> Contact: www.jsoftware.com
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Addon 'format/zulu' is required by various addons, notably
> > > >> math/tabula.
> > > >>>>>> If loaded directly or indirectly by your ~config/startup.ijs
> > > >>>>>>
> > > >>>>>> then JQt will terminate without showing output.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> jconsole will terminate also, but gives a briefer, clearer crash
> > > >> report:
> > > >>>>>>
> > > >>>>>> jconsole(2607,0x104ba4580) malloc: *** error for object
> > > >> 0x138091fc0:
> > > >>>>>> pointer being freed was not allocated
> > > >>>>>>
> > > >>>>>> jconsole(2607,0x104ba4580) malloc: *** set a breakpoint in
> > > >>>>>> malloc_error_break to debug
> > > >>>>>>
> > > >>>>>> zsh: abort /Applications/j904/bin/jconsole
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Saving session...
> > > >>>>>>
> > > >>>>>> ...copying shared history...
> > > >>>>>>
> > > >>>>>> ...saving history...truncating history files...
> > > >>>>>>
> > > >>>>>> ...completed.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> [Process completed]
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> The crash is caused by execution of a verb: zutest_zulu_ .
> > > >>>>>>
> > > >>>>>> This verb gets executed as the final step in loading addon
> > > >> 'format/zulu'
> > > >>>>> .
> > > >>>>>> It contributes nothing to the functionality of this suite of
> > string
> > > >>>>>> utilities, but gives all the working verbs a thorough test.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> I hope to find out why it crashes before the weekend. Meanwhile,
> > > >> here is
> > > >>>>> a
> > > >>>>>> workaround.
> > > >>>>>>
> > > >>>>>> It won't affect the operational behavior of the addon, but it's
> > > >> like
> > > >>>>>> turning off the fire alarms.
> > > >>>>>>
> > > >>>>>> Launch either jqt or jcon (any recent version)…
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> STEP 1
> > > >>>>>>
> > > >>>>>> open'~addons/format/zulu/zutest.ijs'
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> STEP 2
> > > >>>>>>
> > > >>>>>> Comment-out lines 64-66:
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> ok1=. ; 0 zutest each ;:'zu z1 z0'
> > > >>>>>>
> > > >>>>>> ok2=. ;0j1 zutest each ;:'zu z1 z0' NB. for conversions: a2*
> > > >>>>>>
> > > >>>>>> ZUTEST_z_=: ok1 , ok2
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> STEP 3
> > > >>>>>>
> > > >>>>>> Save the updated script.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Now you can triger the crash at will in either JQt or jcon like
> > > >> this:
> > > >>>>>>
> > > >>>>>> load 'format/zulu' NB. should not crash now
> > > >>>>>>
> > > >>>>>> 0 zutest 'zu' NB. crashes
> > > >>>>>>
> > > >>
> ----------------------------------------------------------------------
> > > >>>>>> For information about J forums see
> > > >> http://www.jsoftware.com/forums.htm
> > > >>>>>
> > > >>>>> --
> > > >>>>> This email has been checked for viruses by AVG.
> > > >>>>> https://www.avg.com
> > > >>>>>
> > > >>>>>
> > > >>
> ----------------------------------------------------------------------
> > > >>>>> For information about J forums see
> > > >> http://www.jsoftware.com/forums.htm
> > > >>>>
> > ----------------------------------------------------------------------
> > > >>>> For information about J forums see
> > > http://www.jsoftware.com/forums.htm
> > > >>
> ----------------------------------------------------------------------
> > > >> For information about J forums see
> > http://www.jsoftware.com/forums.htm
> > > >>
> > > >
> ----------------------------------------------------------------------
> > > > For information about J forums see
> http://www.jsoftware.com/forums.htm
> > >
> > >
> > > --
> > > This email has been checked for viruses by AVG.
> > > https://www.avg.com
> > >
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> > >
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
