Some further thoughts on this path,
&> and &.> would apply to each array in the array in identical functionality
for u as if data were boxed instead of ragged. Retaining outer shape of (same
number of total items before and after u&>).
"_1 would refer to the outter arrays as items. Rank inside each array for u"
would be written as u"n"_1. Unlike square arrays, "0 would have to be written
as "0"_1 for u to have the same result on ragged and square arrays. That
permits ragged arrays to be externally shaped as arrays of boxes can be.
These are the consistency decisions that everything else flows from.
ragged arrays would print/display like outter boxes. Perhaps with dashed lines
as vertical separators.
On Thursday, March 3, 2022, 11:50:25 a.m. EST, 'Pascal Jasmin' via Programming
<programm...@jsoftware.com> wrote:
I'm ok with the 0s, but another way to have 2 shapes (referring to 2 arrays) of
different sizes ie. one of 3 2 2 the other shape 4, would be that the "shape of
the shapes" is a "by 1 table":
3
1
so
(3 ,:1) $ 3 2 2 4 would be the shape of i. (3 ,:1) $ 3 2 2 4 or ((3 ,:1) $ 3
2 2 4) $ i.16
I'm ok with this as a formal definition for arrays of arrays, but I would
definitely create a verb named shape to ease with typing and even
processing/reshaping such that
((3 ,:1) $ 3 2 2 4) $ i.16
would be equivalent to
(3 2 2 ,: 4) shape i.16
ie. in table shapes, trailing 0s (fills) are equivalent to axes not being
present. I don't think this loses very much in expressibility, though I
understand that it creates an extra computation step to properly understand a
"shape item" with trailing 0s.
On Thursday, March 3, 2022, 11:01:57 a.m. EST, Henry Rich
<henryhr...@gmail.com> wrote:
I like this, except for the trailing-0 bit. I would say having data with
trailing 0 shape is 'common' rather than 'ultra-rare'.
Henry Rich
On 3/3/2022 10:15 AM, 'Pascal Jasmin' via Programming wrote:
> I don't have a good grasp of order, but a different practical interpretation
> of non-list shapes:
>
> 1. You mentioned distinguishing atoms from lists of 1 element. J already
> manages this well, and making an atomic shape result would not be of any
> help, IMO.
>
> 2. A table shape could be a code for ragged arrays, or arrays of arrays. The
> following shape:
>
> 3 2 2
> 4 0 0
>
> Would be 2 arrays, the first is shape 3 2 2, and the 2nd is shape 4.
> Trailing 0s would be fills. This precludes the ultra rare need of needing an
> "embedded" shape 4 0 or shape 4 0 0 in your data.
>
> A list of 3 strings/lists with individual lengths of 2 3 4 would have the
> shape:
>
> 2
> 3
> 4
>
> If there is an extention of the meaning of shape, I'd prefer it be in this
> direction rather than ultra high dimensional math that hurts my head.
>
>
> On Thursday, March 3, 2022, 05:13:27 a.m. EST, Elijah Stone
> <elro...@elronnd.net> wrote:
>
>
>
>
>
> A little thought exercise I embarked upon.
> Probably not very clearly expressed, and certainly of no practical use,
> but I think there are some interesting ideas:
>
> Why must the result of $y always be a vector? What would happen if we let
> it be a higher-ranked array? What would that _mean_?
>
> Well, say that 2 3 4-:$y. That means that y is a mapping from arrays x
> that have the same shape as 2 3 4 (and whose elements are natural and
> bounded by 2 3 4). In other words, if (x -:&$ 2 3 4) *. -. 0 e. x<2 3 4
> then (<x){y is well-defined as an atom. (I am intentially ignoring the
> possibility of sentences such as ((<1 1){y) ((<,1){y) ((<1){y) and so
> on; consider these sentences ill-formed for the time being.)
> Generalising: if (x -:&$ $y) *. -. 0 e. x<$y then (<x){y is well-defined
> as an atom.
>
> But this sentence makes no restrictions whatsoever on the rank of x! So
> it would seem that we've discovered a way to make a y whose shape is an
> array of any rank. But we must be careful. Really, all we've constructed
> is a strange entity y--call it an array-as-we-do-not-know-it--and, on
> arrays-as-we-do-not-know-them, we have defined that: $y is well-defined,
> and evaluates to an array-as-we-know-it. x{y is well-defined for some
> subset of arrays-as-we-know-them x, and evaluates to an
> atom-as-we-know-it.
>
> I will call this strange creature a 4th-order array. And all
> arrays-as-we-know-them are 3rd-order (this includes scalars, vectors, and
> higher-rank arrays). A 4th-order array has a shape which is a 3rd-order
> array, and maps from 3rd-order arrays to atoms. This suggests the
> existence of lower-order arrays, and higher-order ones.
>
> Following the same logic, then, a 3rd-order array would map 2nd-order
> arrays to atoms, and its shape would be a 2nd-order array. But--for
> 3rd-order y, the x in (<x){y for 3rd-order y is currently also a 3rd-order
> array, and $y is currently also 3rd-order.
>
> Or is it, really? We've already established that $y must always be a
> vector for ordinary, familiar, 3rd-order y; and that x must be too for
> (<x){y. (A vector is a 3rd-order array y which satisfies (,1)-:$$y.) I
> propose that there is another strange creature, which is a 2nd-order
> array, or _list_. This is no less strange than 4th-order array, and j
> does not have them. A list is very similar to a vector, though, in that
> it is an ordered sequence of atoms. And there is an obvious 1:1 mapping
> between vectors and lists. Therefore, when you write (<x){y, and x is a
> vector, it is first _converted_ to the corresponding list, and that list
> is used to index y. Similarly, when you write $y, and y is a third-order
> array, its shape is actually a list; that list is _converted_ to the
> corresponding vector before being returned.
>
> Then, fairly straightforwardly, we can define the 1st-order array, or
> _atom_, another strange creature. Atoms are used to index lists, and the
> shape of a list is an atom. And every atom (1st-order array) also has a
> counterpart in familiar 3rd-order land, as a _scalar_. (A scalar is a
> 3rd-order array y which satisfies ((,0)$0)-:$y.)
>
> The obvious difference between a vector and a list is that, if we want
> element 5 of y, and y is a list, we write (<5){y; but if y is a vector, we
> must write (<,5){y. But to make the distinction that way misses the
> point, I think. A 2nd-order array is a particular sort of array which is
> restricted in that it can only contain unidirectional sequences of atoms.
> A vector is of a different sort of array, which is less restricted, but it
> happens to belong to a particular subset which maps to lists. (The same
> relationship holds for scalars and atoms.)
>
> Now, imagine a hypothetical version of j which could represent arrays of
> orders other than 3. It is difficult to talk about such a j, because--for
> instance--there might be an atom value 3 and a scalar value 3, and these
> would be distinct; if I simply write '3', it is ambiguous which is
> referred to. So I will err on the side of verbosity rather than
> ambiguity.
>
> One thing immediately becomes clear: the result of $y in such a j should
> be the _actual_ shape of y; that is, it should have order one lower than
> y. What, then is the shape of an atom? It might be an error, but I
> should rather make it the _unit_. The unit is the singular value which is
> an order-0 array. The unit is also its own shape. Then, we can also say
> that an order n array is one which requires n applications of $ to reach a
> fixed point. Personally, I think a fixed point of a unit is nicer than a
> fixed point of ,1 (which is what we currently have, with all results
> squashed into 3rd-order arrays), but this is a point of style.
>
> Another nice consistency that results is that the idiom $$y evaluates to
> an atom when y is a 3rd-order array. That is, it gives you the rank of y,
> rather than a 1-element vector containing the rank.
>
> Another observation: this provides a way to relate j to k; the difference
> between them is one of degree, not kind. In k, everything is a list or a
> scalar; that is, all values have order 1 or order 2. In j, all values
> have order 3. (There is another difference, which is that k has nests
> where j has boxes, but this is orthogonal.)
>
> We can think of 3rd-order arrays as functions; for instance, an array of
> rank 4 is like a function of 4 parameters. If we index such an array
> using a list of length 4, that is like applying the corresponding
> function, where each element of the list is associated with the
> corresponding argument. By convention, these functions are curried.
> Even if y has rank 4, it is legal in j to write (<1 2 0){y, even though 1
> 2 0 has length 3. (Previously, I said I was considering this illegal;
> now, I am defining it again.) This works by currying y; producing an
> array which has its first 3 arguments fixed (to 1, 2, and 0,
> respectively), requiring only 1 more; because it only requires 1 more
> argument, we say it has rank 1. This works well enough for 3rd-order
> arrays, but what about higher-order ones? When is something a valid
> 'partial index', and what are the 'first arguments'?
>
> A better definition comes from , . If p is a valid index into y (as
> defined in the 3rd paragraph), then (<p){y applies the indexing procedure
> as usual. Otherwise, it is the array for which (<q){(<p){y is equivalent
> to (<p,q){y. (This only holds if there is q for which, given p and y, p,q
> is a valid index into y; otherwise it is an index error.) This is
> adequate to describe existing indexing behaviour for 3rd-order arrays .
> And it works without a hitch for 4th-order arrays. For higher-order
> arrays, it is simply necessary to extend , to work on higher-order arrays;
> given such an extension, that, too will work without a hitch (and probably
> (?) give sensible results).
>
> I think the foregoing definition goes most of the way towards explaining
> why we may index arrays using scalars; it is more than a convenience
> feature, and the scalar is not simply converted to a vector. (Going the
> rest of the way would require a complete and general model of , .)
>
> Now, some questions:
>
> 1. It there any practical value to this scheme? In some respects, an
> array with a non-list shape just seems like an array with a list shape,
> and bells on. But I am a low-dimensional creature, and perhaps if I had
> more use for very high-dimensional arrays, I would find it useful to
> structure my axes (and then structure the structure of my axes, and so on,
> leading to arrays of ever higher orders).
>
> The simplest 4th-order array which is 'interesting' has shape 2 2$2; that
> is, it corresponds roughly to a 4-dimensional array; that is rare enough.
> The simplest 5th-order array which is 'interesting' has shape (2 2$2)$2,
> that is 16-dimensional-ish. I generally have little use for arrays of
> such high dimension (4 is reasonable, 16 is quite unlikely), and have not
> found a linear organization stifling or problematic.
>
> The jump from the unit to the atom seems most significant, as it allows
> you to communicate any information at all; the unit carries no
> information. Next in significance is the jump from atoms to lists, which
> permit the communication of inherently _structured_ information. This is
> good enough for many applications (and k gets along just fine); moving to
> 3rd-order arrays simply increases the degree of structure available, which
> can be useful in some (ok, many) situations. So it would not be
> surprising if the added benefit of moving to higher orders continued to
> peter out, such that 3 hit a good sweet spot.
>
> 2. Is there any theoretical value to this scheme, or anything interesting
> which can be described only using it? I used it to explain a number of
> small inconsistencies, but that is marginal at best.
>
> Is there any existing mathematical precedent? I know that what j calls
> arrays the literature calls tensors. So I googled for 'higher-order
> tensors', but only found information on what we would call higher-_rank_
> arrays, not what I distinguished here as higher-_order_ arrays. I also
> searched for non-cartesian coordinate systems, but could not find anything
> where the components of the coordinates were not sequential and flat.
>
> 3. Are the different orders of arrays qualitatively different? E.G. a
> 3rd-order array can approximate both a 1st-order array (using a scalar)
> and a 2nd-order array (using a vector); but a 2nd-order array can not
> approximate a 1st-order array. (This is why, to my understanding, a
> scalar is an array in j, but it is not a list in k.) (On the other hand,
> 3rd-order arrays can not represent the unit.) I think all higher-order
> arrays can approximate any 3rd-order array; but do they gain any new
> powers?
>
> 4. I called arrays of order 0, 1, and 2 units, atoms, and lists,
> respectively. What should arrays of order 3 be called? Perhaps 'spaces'?
> Is there a sensible name for 4th- or 5th-order arrays?
>
> Is the name 'shape' still sensible if extended to different orders?
>
> 5. How would the j primitives work if extended to work on arrays of order
> other than 3? What are the rules of conformability? Can you add an atom
> to a scalar? A list to a matrix? A list to a vector? When can you
> catenate two 5th-order arrays? How does rank work on 4th-order arrays?
>
> 6. Is there a good intuition for higher-order arrays? I can more or less
> imagine what 4th-dimensional space is like (though I can't visualize it),
> but I have no idea what a 'space' of shape 2 2$y is like, even though it
> has the same number of 'degrees of freedom'.
>
> Is there a well-defined notion of distance defined in higher-order
> 'space'? If so, is it still scalar?
>
>
> Thoughts?
>
> -E
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
