1. An x argument to timex will give the average over x tries.
2. If z is short, you are mostly timing the execution of the adverbs and
conjunctions that make up the definition of the verb. For example, when
you write
;: inv
the system looks up inv as ^:_1 and then executes ;:^:_1 . That in turn
calls the parser (recursively) to parse the string
}:@;@(,&' '&.>"1) :. ;:
which is the actual inverse. The execution of all those @ &. and :. to
define the verb takes more time than running the verb on a short argument.
3. If you care about the performance on short y, you can either
* use ;:inv inside a tacit verb so that the parsing is done when the
verb is created;
* in an explicit definition, use (;:^._1) . This engages PPPP
(pre-parsed parenthesized primitives): ;:^:_1 will be parsed when the
explicit definition is created, and executed without even the overhead
of a name lookup.
4. For timing, you have to be careful using named arguments. ;:inv can
execute inplace on suitable y. 'Suitable' here means that all the boxed
contents are not being used anywhere else. In that case, adding the ' '
at the end can be done inplace (and the ; executes faster as well).
When you test on ';:^:_1 z', z is not a suitable y (because the value is
used in the name) and you will get slower results than if you timed
';:^:_1 expression&.>' Heisenberg! If you use a named z you get slower
code, and if you have the expression you aren't timing ;:^:_1 only. For
unboxed arguments you could use z_: to make the value of z inplaceable,
but I don't that that suffices for boxed z.
Henry Rich
On 5/30/2022 8:08 PM, Ian Clark wrote:
Henry wrote:
Testing things so short, test multiple runs.
Also, here you are including parsing times which might exceed executing
time.
I've run the following test plan with j904 Beta-d:
9!:56'cpu'
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
timex'(;:inv) z'
timex'([: ; '' '',each~ ])z'
on:
++ Mac MBA early 2014 (x86_64)
++ Mac mini M1 chip (native arm64)
++ Mac mini M1 chip (Rosetta x86_64 emulation)
Note the substantially longer sample LF-string I've used, viz the text of
stdlib.ijs .
I do have a somewhat smarter method of averaging repeated timings. But for
a forensic session I thought I'd use what everyone else has got.
Mac MBA early 2014 (x86_64)
9!:56'cpu'
x86_64
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
60421
timex'(;:inv) z'
0.015428
timex'(;:inv) z'
0.013476
timex'(;:inv) z'
0.013629
timex'(;:inv) z'
0.013494
timex'(;:inv) z'
0.014735
timex'(;:inv) z'
0.013335
timex'(;:inv) z'
0.013783
timex'(;:inv) z'
0.013353
(inserted by hand): mean value: 0.0139041
timex'([: ; '' '',each~ ])z'
0.013806
timex'([: ; '' '',each~ ])z'
0.013383
timex'([: ; '' '',each~ ])z'
0.013903
timex'([: ; '' '',each~ ])z'
0.013341
timex'([: ; '' '',each~ ])z'
0.013323
timex'([: ; '' '',each~ ])z'
0.013576
timex'([: ; '' '',each~ ])z'
0.011759
timex'([: ; '' '',each~ ])z'
0.013209
(inserted by hand): mean value: 0.0132875
Mac mini M1 chip (native arm64)
9!:56'cpu'
arm64
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
60421
timex'(;:inv) z'
0.012289
timex'(;:inv) z'
0.013109
timex'(;:inv) z'
0.01277
timex'(;:inv) z'
0.013593
timex'(;:inv) z'
0.014168
timex'(;:inv) z'
0.014498
timex'(;:inv) z'
0.011915
timex'(;:inv) z'
0.011848
(inserted by hand): mean value: 0.0130238
timex'([: ; '' '',each~ ])z'
0.011718
timex'([: ; '' '',each~ ])z'
0.013287
timex'([: ; '' '',each~ ])z'
0.011614
timex'([: ; '' '',each~ ])z'
0.01695
timex'([: ; '' '',each~ ])z'
0.015872
timex'([: ; '' '',each~ ])z'
0.013207
timex'([: ; '' '',each~ ])z'
0.012396
timex'([: ; '' '',each~ ])z'
0.014463
(inserted by hand): mean value: 0.0136884
Mac mini M1 chip (Rosetta x86_64 emulation)
9!:56'cpu'
x86_64
$z=: fread '/Applications/j904/system/main/stdlib.ijs'
60421
timex'(;:inv) z'
0.014958
timex'(;:inv) z'
0.013211
timex'(;:inv) z'
0.014644
timex'(;:inv) z'
0.012561
timex'(;:inv) z'
0.012615
timex'(;:inv) z'
0.012795
timex'(;:inv) z'
0.012811
timex'(;:inv) z'
0.014273
(inserted by hand): mean value: 0.0134835
timex'([: ; '' '',each~ ])z'
0.010994
timex'([: ; '' '',each~ ])z'
0.011156
timex'([: ; '' '',each~ ])z'
0.011643
timex'([: ; '' '',each~ ])z'
0.011929
timex'([: ; '' '',each~ ])z'
0.0116
timex'([: ; '' '',each~ ])z'
0.012876
timex'([: ; '' '',each~ ])z'
0.011646
timex'([: ; '' '',each~ ])z'
0.01134
(inserted by hand): mean value: 0.011648
I conclude from this trial that with such a long string (z) there is
surprisingly little difference in execution time between j phrases,
architectures, or even Mac models.
Maybe z is too long? I wonder if just one non-CPU-dependent process isn't
dominating the transaction. Memory allocation? Both the MBA and Mac mini
use flash memory. Maybe that hasn't improved as much as CPUs since 2014,
the date of my MBA.
I typically use Zulu (the existing addon) with short strings: 10 to 100
bytes, so the parsing time is part of the deal for me. Though Zulu's
follow-on will get used routinely to process jscripts – though maybe only
line-by-line. So I'm back in the 10-100 byte bracket.
Ian Clark
On Mon, 30 May 2022 at 11:47, Henry Rich <[email protected]> wrote:
Testing things so short, test multiple runs.
Also, here you are including parsing times which might exceed executing
time.
Henry Rich
On Sun, May 29, 2022, 11:16 PM Ian Clark <[email protected]> wrote:
As well ask why you found it at all.
I doubt I'd have found it at all, had not j904 crashed as soon as I gave
it
my pet startup.ijs.
Which just happens to run a deeply paranoid test.
I am surprised that ;:inv is slower than ([: ; SP,each~ ]).
So was I…
z=: ;:'alpha bravo charlie '
timex'(;:inv) z'
3.8e_5
timex'([: ; '' '',each~ ])z'
3.1e_5
JVERSION
Engine: j904/j64/darwin
Beta-d: commercial/2022-05-19T20:41:44
Library: 9.04.01
Qt IDE: 2.0.3/6.2.4(6.2.4)
Platform: Darwin 64
Installer: J904 install
InstallPath: /applications/j904
Contact: www.jsoftware.com
Even slower, in jconsole…
z=: ;:'alpha bravo charlie '
timex'(;:inv)z'
4.2e_5
timex'([: ; '' '',each~ ])z'
2.7e_5
JVERSION
Engine: j904/j64/darwin
Beta-d: commercial/2022-05-19T20:41:44
Library: 9.04.01
Platform: Darwin 64
Installer: J904 install
InstallPath: /applications/j904
Contact: www.jsoftware.com
But the more I use it, I sense the Apple M1 chip in my Mac mini is
lightning-fast at moving bytes around in memory, as opposed to doing
significant computation – when it doesn't perform all that much better
than
the Intel chip Apple has discarded.
Maybe not that surprising when you ponder the meaning of "reduced
instruction set".
M1 is an ARM architecture. (Which JVERSION doesn't show – maybe it
should?)
Unless j904_mac64.zip is compiled for ARM, then j904 will be running
under
Rosetta emulation of Intel architecture. I don't know how to discover if
that's happening.
On Sat, 28 May 2022 at 04:22, Henry Rich <[email protected]> wrote:
I am surprised that ;:inv is slower than ([: ; SP,each~ ]). That needs
looking into. But ([: ; SP,each~ ]) uses a few features I have added
(append in place inside boxes, and support for ;@:(f&.>) with fewer
passes over the data) which is gratifying.
When would you have found the beta-d bug you found? As well ask why
you
found it at all. AFAICS no intentional change in beta-d caused the
error: a bit was being cleared erroneously and somehow in beta-d the
bit
was in an address.
The older I get the more I realize there is no such thing as a working
program. Even IEFBR14 had a bug.
Henry Rich
On 5/27/2022 10:53 PM, Ian Clark wrote:
Thanks for the ideas, Raul. Here's my grid:
┌───┬───────────────┬───────────────┬───────────────┬───────────────┐
│ │ B(boxed) │ F(LFend) │ O(open) │ X(matrix) │
├───┼───────────────┼───────────────┼───────────────┼───────────────┤
│B4*│] │cuT&aLF │cuT&aSP │[: dtb each <"1│
├───┼───────────────┼───────────────┼───────────────┼───────────────┤
│F4*│[: ; LF,each~ ]│aLF │(SP,LF)&charsub│[: F4B B4X │
├───┼───────────────┼───────────────┼───────────────┼───────────────┤
│O4*│[: ; SP,each~ ]│(LF,SP)&charsub│aSP │[: O4B B4X │
├───┼───────────────┼───────────────┼───────────────┼───────────────┤
│X4*│> │>&(cuT&aLF) │>&(cuT&aSP) │] │
└───┴───────────────┴───────────────┴───────────────┴───────────────┘
SP=: ' ' NB. (which IMO ought to be in stdlib)
cuT=: <;._2
aLF=: ] , LF #~ LF ~: {: NB. append LF if not there already
aSP=: ] , SP #~ SP ~: {: NB. append ' ' if not there already
*Observations*:
I wish I'd known stdlib had cutLF, so thanks for that.
Your use of (;:inv) is neat.
However timex tells me ([: ; SP,each~ ]) is faster on the Mac with M1
chip,
so I'll go with that for now.
Providing verbs B4B, F4F… looks otiose, but simplifies the design of
consistency tests.
Personally I prefer the name O4X (say) to X2O. I used to employ the
latter,
but saw Roger preferring names the other way round and found it
worked better for me too. Easier to check visually: see the last
column
above.
Accordingly I extended Zulu (~addons/format/zulu/*) with yet another
optional script, which defined o4x=:x2o etc. But that just made it
even
more unwieldy.
This design, which I'm calling BFOX, corrects the wrong Zulu policy
regarding 'f' format by making a final LF mandatory in the new 'F'
format.
This has several advantages. For a start it's what you get with:
zuF=: noun define
alpha
bravo
charlie
)
In Zulu this gives 4 rows, not 3 as under the current design.
It seems logical to me to extend the same policy to O format, since I
mean
to implement O and F with identical code, merely switching SP <-->
LF.
I want to avoid breaking existing code. So I think I'll create a new
addon:
BFOX (~addons/format/bfox.ijs --all in a single script) and
standardize
on
*4* naming.
Instead of a mandatory test on loading the addon, I'll also provide a
separate test script (as addons are supposed to do), which checks
logical
consistency by converting via several routes back to the original,
plus
addressing nouns with 0 or 1 "rows", plus all the other good things
Zulu
does. I've yet to do this, but I fancy it'll all be a lot neater,
with
fewer edge cases to handle.
At least that was my plan until it struck me BFOX might make a better
Foreign in the (3!:) range, than yet another potentially klugey
addon.
But
this is all making work for Henry…!
Another thought: if Zulu hadn't done its mandatory suite of tests on
loading, when if ever would I have hit this bug in Beta-d?
On Thu, 26 May 2022 at 17:23, Raul Miller <[email protected]>
wrote:
(I sort of ignored the empty string case, obviously - in some cases
these conversions would discard them. In others, it would preserve
them. It's something of a judgement call ... somewhat analogous to
forum choice, which I perhaps abused in my previous message.)
--
Raul
On Thu, May 26, 2022 at 12:19 PM Raul Miller <[email protected]
wrote:
Here's how I think I'd tackle that "grid":
b2f ;,&LF&.> b
b2o ;:inv b
b2x > b
f2b cutLF f
f2o rplc&(LF,' ') f
f2x >cutLF f
o2b cut o
o2f rplc&(' ',LF) o
o2x >cut o
x2b <@dtb"1 x
x2f ;<@(,&LF)@dtb"1 x
x2o deb,' ',. x
FYI,
--
Raul
On Thu, May 26, 2022 at 11:46 AM Ian Clark <[email protected]>
wrote:
Henry wrote:
The problem was with the form
[&.>@:(<"0) 'ab'
(used in x2b).
x2b is defined as:
[: (#~ ([: +./\. ' '&~:))&.> <"1
so I guess that was a typo. But I get the idea.
So… no need for me to make any changes to 'format/zulu' just yet.
ASIDE:
--but when I do, I'll throw away the entire implementation.
Zulu addresses an enduring need for the J beginner wanting to
develop
code
for distribution. But it's deprecated for operational use.
Zulu is an awful example of the edge-case 'tail' wagging the JAL
'dog'.
If we ever get a *usable* addon, or a primitive (?), which
comprehensively
supports dictionaries, or even just collections of strings, then I
hope &
pray it eats Zulu for breakfast.
Zulu's aims could be met by publishing a 4x4 grid of recommended
*consistent* conversions between the 4 common "strings" formats:
b - boxed
f - LF-separated
o - open
x - matrix
See the lab: Strings conversion package.
WHAT'S MORE:
A user-facing app doing smart things with LF-separated strings
must
adopt a
policy towards final LF, especially for handling a collection of
substrings
containing 0 or 1 members.
Zulu adopts the wrong policy. And pays for it with lots of slow
edge-case
code.
On Wed, 25 May 2022 at 18:24, Henry Rich <[email protected]>
wrote:
Fixed for next beta. The problem was with the form
[&.>@:(<"0) 'ab'
(used in x2b). The <"n created virtual blocks but erroneously
flagged
them as inplaceable.
Workaround: replace (<"n) with (<"<"n).
Henry Rich
On 5/24/2022 8:25 PM, Ian Clark wrote:
Note: Addon 'format/zulu' does not crash 903 or earlier.
JVERSION
Engine: j904/j64/darwin
Beta-d: commercial/2022-05-19T20:41:44
Library: 9.04.01
Qt IDE: 2.0.3/6.2.4(6.2.4)
Platform: Darwin 64
Installer: J904 install
InstallPath: /applications/j904
Contact: www.jsoftware.com
Addon 'format/zulu' is required by various addons, notably
math/tabula.
If loaded directly or indirectly by your ~config/startup.ijs
then JQt will terminate without showing output.
jconsole will terminate also, but gives a briefer, clearer crash
report:
jconsole(2607,0x104ba4580) malloc: *** error for object
0x138091fc0:
pointer being freed was not allocated
jconsole(2607,0x104ba4580) malloc: *** set a breakpoint in
malloc_error_break to debug
zsh: abort /Applications/j904/bin/jconsole
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
[Process completed]
The crash is caused by execution of a verb: zutest_zulu_ .
This verb gets executed as the final step in loading addon
'format/zulu'
.
It contributes nothing to the functionality of this suite of
string
utilities, but gives all the working verbs a thorough test.
I hope to find out why it crashes before the weekend. Meanwhile,
here is
a
workaround.
It won't affect the operational behavior of the addon, but it's
like
turning off the fire alarms.
Launch either jqt or jcon (any recent version)…
STEP 1
open'~addons/format/zulu/zutest.ijs'
STEP 2
Comment-out lines 64-66:
ok1=. ; 0 zutest each ;:'zu z1 z0'
ok2=. ;0j1 zutest each ;:'zu z1 z0' NB. for conversions: a2*
ZUTEST_z_=: ok1 , ok2
STEP 3
Save the updated script.
Now you can triger the crash at will in either JQt or jcon like
this:
load 'format/zulu' NB. should not crash now
0 zutest 'zu' NB. crashes
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
