Re: [Jprogramming] Is is good idea to use J for reading large XML files ?

2021-09-07 Thread Raul Miller
Hmm I think I see at least a part of the issues you are talking about. Currently, J's api/expat expects you to provide the complete contents of the xml document when you call expat_parse_xml For large files, it would make sense to have a similarly structured expat_parse_file which takes a fi

Re: [Jprogramming] Viewmat palettes (and bitmaps)

2021-09-07 Thread Raul Miller
Understood. But, to better illustrate what I was trying to say, when I compare BGYPAL2 viewmat i.256 (((i.256)e.15*i.18)#BGYPAL2) viewmat i.256 The 18 entry palette looks slightly smoother, to me, than the 256 entry palette. (I see the same issue, to a lesser extent, with BYPAL3.) And, of

Re: [Jprogramming] Is is good idea to use J for reading large XML files ?

2021-09-07 Thread Mariusz Grasko
Raul, This large RAM usage happened for all files that I tested, there was nothing particurarly special about them. It is mostly products details from warehouses: ID SKU EAN NAME DESC QNT PRICE TAX IMAGE this type of elements. Some of them are attributes (price, tax, suggested price as attributes

Re: [Jprogramming] Viewmat palettes (and bitmaps)

2021-09-07 Thread Devon McCormick
I suspect there are duplicate entries in that one because it was the first one I generated. The other two are tweaked versions where I was more careful to avoid duplicates. On Tue, Sep 7, 2021 at 2:38 PM Raul Miller wrote: > Got it. > > And, these are all 256 row palettes. So, using them with v

Re: [Jprogramming] Fold experience

2021-09-07 Thread R.E. Boss
Your last (2) sentence(s) is what is/was confusing me. Now I know, I will try to remember. But (most) probably I will stick to my memo. Thanks, R.E. Boss -Original Message- From: Programming On Behalf Of Henry Rich Sent: dinsdag 7 september 2021 18:18 To: programm...@jsoftware.com Su

Re: [Jprogramming] Viewmat palettes (and bitmaps)

2021-09-07 Thread Raul Miller
Got it. And, these are all 256 row palettes. So, using them with viewmat i.256 shows their full details. That said, to my eye, these look identical: GRAYPAL viewmat i.256 (0 0 0,:255) viewmat i.256 I do see some subtle differences between BGYPAL and BGYPAL2 / BGYPAL3 I was noticing thing

Re: [Jprogramming] Viewmat palettes (and bitmaps)

2021-09-07 Thread Devon McCormick
The main results you might care about are the resulting palettes in the globals ADJPAL, BGYPAL, BGYPAL2, BGYPAL3, FULLSPECPAL, and GRAYPAL. The "BGY..." ones are limited to blue, green, and yellow, from dark to light. The others are as documented; all are an attempt to put colors in a logical, lin

Re: [Jprogramming] Fold experience

2021-09-07 Thread Henry Rich
Your diagram isn't right.  Rather than a column y{UV, you need a column y, the first entry in which would be 3. The first application of foo is 3 foo 0 0 0 which gives a result of shape 3 3 . The next application of foo is 2 foo (3x3 array) giving shape 3 3 3, etc. The y in foo is NOT the

Re: [Jprogramming] Is is good idea to use J for reading large XML files ?

2021-09-07 Thread Raul Miller
Looking at the implementation, it's literally setting up the expat parser to parse your file and then responding to your callbacks. If 'elm att val'=. x is creating a memory spike for you, that suggests that your test file contains extremely long xml elements. (Not text data, not nested xml elemen

Re: [Jprogramming] Fold experience

2021-09-07 Thread R.E. Boss
In my example UV=:0,(,-@|.)=i.3 foo=: {{ x+ y{UV }} and with the fold-diagram as guide, I thought the working of 0 0 0 ]F:. foo 3 2 _3 would be y{UV x= 0 0 0 v v 0 0 1 ---> 0 0 1 v v 0 1 0 ---> 0 1 1 v v 0 0 -1 --> 0 1 0 but it is not. The output produced

Re: [Jprogramming] Viewmat palettes (and bitmaps)

2021-09-07 Thread Raul Miller
Well, this line threw an error: NB* sombrero: the "sombrero" function (https://en.wikipedia.org/wiki/Sombrero_function) After fixing that NB, I got it to load. That said, it has some stuff in it that I can't use (for example, LD) and I'm not sure what all of it does. I'll have to find time to ex

Re: [Jprogramming] Fold experience

2021-09-07 Thread Henry Rich
Fold principles: 1. The items of y [and x] are applied to the x argument of v. This is analogousto u/ y and u/\. y.  The underlying J principle is that when a verb operates multiple times, it operates repeatedly on its y argument.  Thus the result of each application of v is the y argument to

Re: [Jprogramming] Viewmat palettes (and bitmaps)

2021-09-07 Thread Devon McCormick
Okay, I've added "stepsftn" - it was it a utility file that I have loaded by default. Please let me know if you notice any other missing routines. Thanks, Devon On Tue, Sep 7, 2021 at 12:43 AM Raul Miller wrote: > I thought that I had searched for roundNums on that page and not found > a defin

Re: [Jprogramming] Is is good idea to use J for reading large XML files ?

2021-09-07 Thread Mariusz Grasko
So after some break and experimenting with other stuff I have revisited api/expat. It seems that expat, the C library for stream-oriented XML parsing, should not load elements to RAM and only process as you go, yielding tokens like StartElements, chardata, attributes etc. In this J library it seems

Re: [Jprogramming] Fold experience

2021-09-07 Thread Raul Miller
But x is on the left of the verb, and y is on the right of the verb. The issue here, of course, is that the fold verb is the result of a conjunction, and the verbs used by that conjunction are different. Meanwhile, the reason that swapping parameters did not work was because the result you were l

Re: [Jprogramming] Fold experience

2021-09-07 Thread R.E. Boss
OK, I thought that x was always on the left of a verb, and y on the right. Did not expect the diagram would have priority over that J axiom. Never too old to learn, although it will be quite confusing. Now I have normal, old-fashioned verbs, and verbs which will be folded. Apart from that, why do

Re: [Jprogramming] Fold experience

2021-09-07 Thread Raul Miller
The diagram shows the y argument to the fold on the left and the x argument to the fold on the right. So that cannot be the issue you are talking about. The diagram shows y0 being used first, y1 second and so on, which matches what we saw happening in action. So that cannot be the issue you are ta

Re: [Jprogramming] Fold experience

2021-09-07 Thread R.E. Boss
The main point I was trying to make was that F:. showed behaviour not conform the diagram on https://code.jsoftware.com/wiki/Vocabulary/fcap. The role of x and y seem to be interchanged. However, then u F:.v~ should work, which does not. The advantage of fold is just that x can differ essentiall

Re: [Jprogramming] Fold experience

2021-09-07 Thread Raul Miller
Swapping the argument does not appear to influence the shape of the result. The results themselves are different, UV=:0,(,-@|.)=i.3 foo=: {{ x+ y{UV }} oof=: {{ y+ x{UV }} 0 0 0 (]F:.foo -: ]F:.foo~) 3 2 _3 0 0 0 0 (]F:.oof -: ]F:.oof~) 3 2 _3 0 That said, focusing on the variant which yo