correcting a typo/bug written earlier
(3 2 2 ,: 4) shape i.16
should be
(3 2 2 ,: ,4) shape i.16
Another way to specify/return ragged shapes would be in boxed form. I would
not have made the above typo if the input form for ragged shapes were boxes. So
(3 2 2;4) as a shape does not need to care about padding, and then some other
future meaning for table shapes could be looked into.
On Thursday, March 3, 2022, 11:50:25 a.m. EST, 'Pascal Jasmin' via Programming
<programm...@jsoftware.com> wrote:
I'm ok with the 0s, but another way to have 2 shapes (referring to 2 arrays) of
different sizes ie. one of 3 2 2 the other shape 4, would be that the "shape of
the shapes" is a "by 1 table":
3
1
so
(3 ,:1) $ 3 2 2 4 would be the shape of i. (3 ,:1) $ 3 2 2 4 or ((3 ,:1) $ 3
2 2 4) $ i.16
I'm ok with this as a formal definition for arrays of arrays, but I would
definitely create a verb named shape to ease with typing and even
processing/reshaping such that
((3 ,:1) $ 3 2 2 4) $ i.16
would be equivalent to
(3 2 2 ,: 4) shape i.16
ie. in table shapes, trailing 0s (fills) are equivalent to axes not being
present. I don't think this loses very much in expressibility, though I
understand that it creates an extra computation step to properly understand a
"shape item" with trailing 0s.
On Thursday, March 3, 2022, 11:01:57 a.m. EST, Henry Rich
<henryhr...@gmail.com> wrote:
I like this, except for the trailing-0 bit. I would say having data with
trailing 0 shape is 'common' rather than 'ultra-rare'.
Henry Rich
On 3/3/2022 10:15 AM, 'Pascal Jasmin' via Programming wrote:
> I don't have a good grasp of order, but a different practical interpretation
> of non-list shapes:
>
> 1. You mentioned distinguishing atoms from lists of 1 element. J already
> manages this well, and making an atomic shape result would not be of any
> help, IMO.
>
> 2. A table shape could be a code for ragged arrays, or arrays of arrays. The
> following shape:
>
> 3 2 2
> 4 0 0
>
> Would be 2 arrays, the first is shape 3 2 2, and the 2nd is shape 4.
> Trailing 0s would be fills. This precludes the ultra rare need of needing an
> "embedded" shape 4 0 or shape 4 0 0 in your data.
>
> A list of 3 strings/lists with individual lengths of 2 3 4 would have the
> shape:
>
> 2
> 3
> 4
>
> If there is an extention of the meaning of shape, I'd prefer it be in this
> direction rather than ultra high dimensional math that hurts my head.
>
>
> On Thursday, March 3, 2022, 05:13:27 a.m. EST, Elijah Stone
> <elro...@elronnd.net> wrote:
>
>
>
>
>
> A little thought exercise I embarked upon.
> Probably not very clearly expressed, and certainly of no practical use,
> but I think there are some interesting ideas:
>
> Why must the result of $y always be a vector? What would happen if we let
> it be a higher-ranked array? What would that _mean_?
>
> Well, say that 2 3 4-:$y. That means that y is a mapping from arrays x
> that have the same shape as 2 3 4 (and whose elements are natural and
> bounded by 2 3 4). In other words, if (x -:&$ 2 3 4) *. -. 0 e. x<2 3 4
> then (<x){y is well-defined as an atom. (I am intentially ignoring the
> possibility of sentences such as ((<1 1){y) ((<,1){y) ((<1){y) and so
> on; consider these sentences ill-formed for the time being.)
> Generalising: if (x -:&$ $y) *. -. 0 e. x<$y then (<x){y is well-defined
> as an atom.
>
> But this sentence makes no restrictions whatsoever on the rank of x! So
> it would seem that we've discovered a way to make a y whose shape is an
> array of any rank. But we must be careful. Really, all we've constructed
> is a strange entity y--call it an array-as-we-do-not-know-it--and, on
> arrays-as-we-do-not-know-them, we have defined that: $y is well-defined,
> and evaluates to an array-as-we-know-it. x{y is well-defined for some
> subset of arrays-as-we-know-them x, and evaluates to an
> atom-as-we-know-it.
>
> I will call this strange creature a 4th-order array. And all
> arrays-as-we-know-them are 3rd-order (this includes scalars, vectors, and
> higher-rank arrays). A 4th-order array has a shape which is a 3rd-order
> array, and maps from 3rd-order arrays to atoms. This suggests the
> existence of lower-order arrays, and higher-order ones.
>
> Following the same logic, then, a 3rd-order array would map 2nd-order
> arrays to atoms, and its shape would be a 2nd-order array. But--for
> 3rd-order y, the x in (<x){y for 3rd-order y is currently also a 3rd-order
> array, and $y is currently also 3rd-order.
>
> Or is it, really? We've already established that $y must always be a
> vector for ordinary, familiar, 3rd-order y; and that x must be too for
> (<x){y. (A vector is a 3rd-order array y which satisfies (,1)-:$$y.) I
> propose that there is another strange creature, which is a 2nd-order
> array, or _list_. This is no less strange than 4th-order array, and j
> does not have them. A list is very similar to a vector, though, in that
> it is an ordered sequence of atoms. And there is an obvious 1:1 mapping
> between vectors and lists. Therefore, when you write (<x){y, and x is a
> vector, it is first _converted_ to the corresponding list, and that list
> is used to index y. Similarly, when you write $y, and y is a third-order
> array, its shape is actually a list; that list is _converted_ to the
> corresponding vector before being returned.
>
> Then, fairly straightforwardly, we can define the 1st-order array, or
> _atom_, another strange creature. Atoms are used to index lists, and the
> shape of a list is an atom. And every atom (1st-order array) also has a
> counterpart in familiar 3rd-order land, as a _scalar_. (A scalar is a
> 3rd-order array y which satisfies ((,0)$0)-:$y.)
>
> The obvious difference between a vector and a list is that, if we want
> element 5 of y, and y is a list, we write (<5){y; but if y is a vector, we
> must write (<,5){y. But to make the distinction that way misses the
> point, I think. A 2nd-order array is a particular sort of array which is
> restricted in that it can only contain unidirectional sequences of atoms.
> A vector is of a different sort of array, which is less restricted, but it
> happens to belong to a particular subset which maps to lists. (The same
> relationship holds for scalars and atoms.)
>
> Now, imagine a hypothetical version of j which could represent arrays of
> orders other than 3. It is difficult to talk about such a j, because--for
> instance--there might be an atom value 3 and a scalar value 3, and these
> would be distinct; if I simply write '3', it is ambiguous which is
> referred to. So I will err on the side of verbosity rather than
> ambiguity.
>
> One thing immediately becomes clear: the result of $y in such a j should
> be the _actual_ shape of y; that is, it should have order one lower than
> y. What, then is the shape of an atom? It might be an error, but I
> should rather make it the _unit_. The unit is the singular value which is
> an order-0 array. The unit is also its own shape. Then, we can also say
> that an order n array is one which requires n applications of $ to reach a
> fixed point. Personally, I think a fixed point of a unit is nicer than a
> fixed point of ,1 (which is what we currently have, with all results
> squashed into 3rd-order arrays), but this is a point of style.
>
> Another nice consistency that results is that the idiom $$y evaluates to
> an atom when y is a 3rd-order array. That is, it gives you the rank of y,
> rather than a 1-element vector containing the rank.
>
> Another observation: this provides a way to relate j to k; the difference
> between them is one of degree, not kind. In k, everything is a list or a
> scalar; that is, all values have order 1 or order 2. In j, all values
> have order 3. (There is another difference, which is that k has nests
> where j has boxes, but this is orthogonal.)
>
> We can think of 3rd-order arrays as functions; for instance, an array of
> rank 4 is like a function of 4 parameters. If we index such an array
> using a list of length 4, that is like applying the corresponding
> function, where each element of the list is associated with the
> corresponding argument. By convention, these functions are curried.
> Even if y has rank 4, it is legal in j to write (<1 2 0){y, even though 1
> 2 0 has length 3. (Previously, I said I was considering this illegal;
> now, I am defining it again.) This works by currying y; producing an
> array which has its first 3 arguments fixed (to 1, 2, and 0,
> respectively), requiring only 1 more; because it only requires 1 more
> argument, we say it has rank 1. This works well enough for 3rd-order
> arrays, but what about higher-order ones? When is something a valid
> 'partial index', and what are the 'first arguments'?
>
> A better definition comes from , . If p is a valid index into y (as
> defined in the 3rd paragraph), then (<p){y applies the indexing procedure
> as usual. Otherwise, it is the array for which (<q){(<p){y is equivalent
> to (<p,q){y. (This only holds if there is q for which, given p and y, p,q
> is a valid index into y; otherwise it is an index error.) This is
> adequate to describe existing indexing behaviour for 3rd-order arrays .
> And it works without a hitch for 4th-order arrays. For higher-order
> arrays, it is simply necessary to extend , to work on higher-order arrays;
> given such an extension, that, too will work without a hitch (and probably
> (?) give sensible results).
>
> I think the foregoing definition goes most of the way towards explaining
> why we may index arrays using scalars; it is more than a convenience
> feature, and the scalar is not simply converted to a vector. (Going the
> rest of the way would require a complete and general model of , .)
>
> Now, some questions:
>
> 1. It there any practical value to this scheme? In some respects, an
> array with a non-list shape just seems like an array with a list shape,
> and bells on. But I am a low-dimensional creature, and perhaps if I had
> more use for very high-dimensional arrays, I would find it useful to
> structure my axes (and then structure the structure of my axes, and so on,
> leading to arrays of ever higher orders).
>
> The simplest 4th-order array which is 'interesting' has shape 2 2$2; that
> is, it corresponds roughly to a 4-dimensional array; that is rare enough.
> The simplest 5th-order array which is 'interesting' has shape (2 2$2)$2,
> that is 16-dimensional-ish. I generally have little use for arrays of
> such high dimension (4 is reasonable, 16 is quite unlikely), and have not
> found a linear organization stifling or problematic.
>
> The jump from the unit to the atom seems most significant, as it allows
> you to communicate any information at all; the unit carries no
> information. Next in significance is the jump from atoms to lists, which
> permit the communication of inherently _structured_ information. This is
> good enough for many applications (and k gets along just fine); moving to
> 3rd-order arrays simply increases the degree of structure available, which
> can be useful in some (ok, many) situations. So it would not be
> surprising if the added benefit of moving to higher orders continued to
> peter out, such that 3 hit a good sweet spot.
>
> 2. Is there any theoretical value to this scheme, or anything interesting
> which can be described only using it? I used it to explain a number of
> small inconsistencies, but that is marginal at best.
>
> Is there any existing mathematical precedent? I know that what j calls
> arrays the literature calls tensors. So I googled for 'higher-order
> tensors', but only found information on what we would call higher-_rank_
> arrays, not what I distinguished here as higher-_order_ arrays. I also
> searched for non-cartesian coordinate systems, but could not find anything
> where the components of the coordinates were not sequential and flat.
>
> 3. Are the different orders of arrays qualitatively different? E.G. a
> 3rd-order array can approximate both a 1st-order array (using a scalar)
> and a 2nd-order array (using a vector); but a 2nd-order array can not
> approximate a 1st-order array. (This is why, to my understanding, a
> scalar is an array in j, but it is not a list in k.) (On the other hand,
> 3rd-order arrays can not represent the unit.) I think all higher-order
> arrays can approximate any 3rd-order array; but do they gain any new
> powers?
>
> 4. I called arrays of order 0, 1, and 2 units, atoms, and lists,
> respectively. What should arrays of order 3 be called? Perhaps 'spaces'?
> Is there a sensible name for 4th- or 5th-order arrays?
>
> Is the name 'shape' still sensible if extended to different orders?
>
> 5. How would the j primitives work if extended to work on arrays of order
> other than 3? What are the rules of conformability? Can you add an atom
> to a scalar? A list to a matrix? A list to a vector? When can you
> catenate two 5th-order arrays? How does rank work on 4th-order arrays?
>
> 6. Is there a good intuition for higher-order arrays? I can more or less
> imagine what 4th-dimensional space is like (though I can't visualize it),
> but I have no idea what a 'space' of shape 2 2$y is like, even though it
> has the same number of 'degrees of freedom'.
>
> Is there a well-defined notion of distance defined in higher-order
> 'space'? If so, is it still scalar?
>
>
> Thoughts?
>
> -E
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
