To fill you in :) on a little history, J's older sister APL would give
errors in many of the cases where J automatically fills to pad out an array
result to make it regular. The automatic fill simplifies a lot of common
usages where one intends to "flatten" an array from boxed to unboxed.
So, we can do something like this
>'We';'want';'this';'to be a';'simple';'table'
instead of something like this
([:>]{.~&.>[:>./[:#&>]) 'We';'want';'this';'to be a';'simple';'table'
Also, fill is invaluable when we use overtake on an empty array to
distinguish between character and numeric arrays, e.g.
(' '=[:{.0$[:,]) 'hi'
1
(' '=[:{.0$[:,]) i. 2 3
0
On Fri, Jul 5, 2013 at 2:57 AM, Michal D. <[email protected]>wrote:
> Hi All,
>
> Raul: I like your idea of an adverb for specifying that fill should be an
> error. I've played around with a similar but opposite idea (error by
> default, and an adverb specifying fill).
>
> The two array approach is of course a possibility but I'm more interested
> in designing a language similar to J than solving a specific problem with
> J.
>
>
> Roger: Thanks for the example and discussion.
>
> Greg: Thanks as well for pointing out how things are different in K. It's
> probably worth taking a closer look at what happens there.
>
> Cheers,
>
> Mike
>
>
>
>
>
>
>
> On Sat, Jun 29, 2013 at 3:55 PM, Raul Miller <[email protected]>
> wrote:
>
> > Yes, but keep in mind that K is optimized for one thing: time series
> > databases. It provides a different set of abstractions from J. (J
> > gives flat namespaces, K gives you its tree. J arrays have rank, K
> > arrays do not. And... so on...)
> >
> > It's a good language though, and of course you should feel free to use
> > it, if you are so inclined.
> >
> > --
> > Raul
> >
> > On Sat, Jun 29, 2013 at 6:37 PM, greg heil <[email protected]> wrote:
> > >>A great example of purity of arrays (no admixture of types) allowing
> > transparency to the implementer (and speed of the result) is k (or kona
> > or...). Ragged arrays in arbitrary dimensions are perfectly transparent
> and
> > efficient there. Its implementation should be efficient too, as it clocks
> > in at about 100KB for the system.
> > >
> > > greg
> > > ~krsnadas.org
> > >
> > > --
> > >
> > > from: Roger Hui <[email protected]>
> > > to: Programming forum <[email protected]>
> > > date: 29 June 2013 11:14
> > > subject: Re: [Jprogramming] Implementation of fill
> > >
> > >>The benchmark I posted before in this thread illustrates a difference
> > between a different representation and the current boxed arrays with
> > special code support.
> > >
> > > c=: 7 + 1e6 ?@$ 13
> > > n=: +/c
> > > v=: ((i.n) e. 0,+/\ c) <;.1 a.{~97+n ?@$ 26
> > > m=: >v
> > >
> > > ts 'i.~ v'
> > > 0.35258 1.25836e7
> > > ts 'i.~ m'
> > > 0.235694 1.25837e7
> > >
> > >>i.~v illustrates the fact that the i. primitive scans a boxed left
> > argument to see whether all the boxes contain the same kind of stuff, and
> > uses a faster algorithm if they do.
> > >
> > > vx=: 3 1 4 1 5 9;v
> > >
> > > ts 'i.~ vx'
> > > 0.708579 1.25837e7
> > >
> > >>A slower algorithm is used for i.~vx because all the boxes do not
> > contain the same kind of stuff.
> > >
> > >>Usually, a negative result from the scan obtains pretty quickly. But
> you
> > can construct an argument where it finds out only at the end that it's
> not
> > all the same kind of stuff:
> > >
> > > vy=: 1|.vx
> > > ts 'i.~ vy'
> > > 0.734981 1.25837e7
> > >
> > >>Because J does not have a different representation for boxes of all the
> > same kind of stuff, it can pick and choose which primitives to optimize
> for
> > such boxes (while paying the price of doing the scan every time it needs
> to
> > know). If you have a different representation (as I said before) you have
> > to do work in every primitive to support that representation.
> > Alternatively, you can maintain a flag with an array recording various
> > properties that you are interested in, but then you'd have to do the work
> > to update that flag.
> > >
> > > --
> > >
> > > from: Roger Hui <[email protected]>
> > > to: Programming forum <[email protected]>
> > > date: 29 June 2013 10:24
> > > subject: Re: [Jprogramming] Implementation of fill
> > >
> > >>If you don't have to be compatible, you can do anything that is
> > imaginable. Disallowing fill is a possibility.
> > >
> > >>Having done both, I can tell you that designing a language is much
> > harder than implementing it. How do you debug a design, for example?
> > >
> > >>> On the other hand, how hard is it for the interpreter to recognized
> > boxed arrays where the boxed items have the same rank (# @: $) and
> > implement them as contiguous ragged arrays? (Perhaps the best solution if
> > possible).
> > >
> > >>Of course it is possible, and it may even be "easy" depending on how
> > much pain you are willing to endure. Keep in mind that if you have a
> > different representation you have to handle that representation in every
> > primitive. It is as much implementation work as having a new datatype.
> With
> > a new datatype you also have language design work.
> > >
> > >>Lest I sound too discouraging, let me say that you should go ahead and
> > do what you think is right. I myself did lots of stuff and succeeded
> > because I didn't know that it was supposed to be hard.
> > >
> > > --
> > >
> > > from: Michal D. <[email protected]>
> > > to: [email protected]
> > > date: 28 June 2013 21:17
> > > subject: Re: [Jprogramming] Implementation of fill
> > >
> > > Oops, ravelWFill should be appendWFill =/
> > >
> > > --
> > >
> > > from: Michal D. <[email protected]>
> > > to: [email protected]
> > > date: 28 June 2013 21:15
> > > subject: Re: [Jprogramming] Implementation of fill
> > >
> > >>So I have to clarify that this discussion is about a language similar
> to
> > J but not J itself, since obviously none of this stuff can be changed in
> J.
> > >
> > >>Maybe it makes sense to disallow fills altogether and instead result in
> > an error. How much expressive power is really lost since you usually want
> > to box the result anyways and might prevent errors (due to values being
> > equal to fills)?
> > >
> > > The problem with boxed arrays is that they are not contiguous in
> > > memory with all the corresponding performance concerns. Has any
> > > thought been given to a more fine grained shape representation for
> > > ragged arrays? Possibly compressed in some form like below:
> > >
> > > j=:1+i.
> > > ravelWFill =: ,&.<
> > > ]ragged=: (j 2 2) ravelWFill ((j 2) ravelWFill (j 3))
> > > 1 2 0
> > > 3 4 0
> > >
> > > 1 2 0
> > > 1 2 3
> > > ]shapeOfRagged=: (<2),(<(<2 2),(<(<2),(<2,3)))
> > > +-+-------------+
> > > |2|+---+-------+|
> > > | ||2 2|+-+---+||
> > > | || ||2|2 3|||
> > > | || |+-+---+||
> > > | |+---+-------+|
> > > +-+-------------+
> > >
> > >>On the other hand, how hard is it for the interpreter to recognized
> > boxed arrays where the boxed items have the same rank (# @: $) and
> > implement them as contiguous ragged arrays? (Perhaps the best solution if
> > possible).
> > >
> > > Cheers,
> > >
> > > Mike
> > >
> > >>Ps. I will be away for the weekend but will eagerly read all responses
> > when I get back.
> > >
> > > --
> > >
> > > from: Marshall Lochbaum <[email protected]>
> > > to: [email protected]
> > > date: 27 June 2013 08:21
> > > subject: Re: [Jprogramming] Implementation of fill
> > >
> > >>1) Fills need to be as unobtrusive as possible. This means they need to
> > keep the original array intact, and look distinct from the rest of the
> > array. Filling the front of an array would change which elements are at
> > which indices by moving each element up a few indices. This makes it
> > difficult to consistently index the array, so that's out. Cycling the
> array
> > is a bit more reasonable, but it has a few problems. First, the cycled
> > elements will look like data, making it hard to distinguish where the
> fill
> > has been added. Second, doing a cycle fill multiple times is not
> equivalent
> > to using one larger fill:
> > >
> > > ]a=.i.4
> > > 0 1 2 3
> > > 8$6$a
> > > 0 1 2 3 0 1 0 1
> > > 8$a
> > > 0 1 2 3 0 1 2 3
> > >
> > >>2) Yes, fills are a pain. In general, you should keep ragged data in
> > boxes rather than letting J concatenate it and fill out the shorter
> > elements. I can think of a few cases where fills are useful (working with
> > polynomials, which ideally are infinite lists where all but a finite
> number
> > of elements are zero, is one), but otherwise they will probably just
> damage
> > your data.
> > >
> > > Marshall
> > >
> > > --
> > >
> > > from: Michal D. <[email protected]>
> > > to: [email protected]
> > > date: 26 June 2013 23:00
> > > subject: [Jprogramming] Implementation of fill
> > >
> > > Hi All,
> > >
> > >>I was wondering if any J implementors have some insight to share.
> > >
> > >>(1) Is there a reason why fills are put in the places they are? Are
> > there any alternatives that were also explored? One might imagine filling
> > the front of an array instead of the back, or cycling through the
> elements
> > of the array instead of inserting fills.
> > >
> > >>(2) They seem a little bit more tricky and unwieldy than I originally
> > thought. Would others agree?
> > >
> > > Cheers,
> > > Mike
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
--
Devon McCormick, CFA
^me^ at acm.
org is my
preferred e-mail
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
