https://www.jsoftware.com/papers/array1.htm#5  relevant.

Permitting multidimensional dictionaries--or arrays some of whose axes are indexed by symbols--is interesting, and I had not thought of it (though it is somewhat obvious).

Obviously such arrays would have to be rectangular, that is homogenous. So for instance, if I have a rank 2 array where the first axis is integral and the second is symbolic, every row will have the same set of keys. So I think overtake makes perfect sense: retain all the keys, and use the fill element to generate the new values.

Negative dimensions seem inconsistent given that certain primitives (such as " and |:) already assign meaning to such. I think it makes most sense to make symbolic axes positive, same as normal axes.

A question is what the shape of a partly symbolic array should be. I propose a boxed array, where the symbolic axes are identified by a vector of keys. That seems to force an order on the keys; I am not sure of a good way around this. I note the linked 'What is an Array?' is careful not to show shape, only rank.

You bring up the issue of whether amend should have index error or not. There is an interesting issue there: what if you attempt to amend a nonexistent key in only some cells? Should we fill the remaining cells with the fill value, or err only then? The easier (and more consistent) answer is that it should be an error. Catenate offers an obvious way to add keys which does not have this problem; but update-or-add is a useful operation to have. Hmmm.

 -E

P.S. I was imagining catenate as effectively a union. This is not completely consistent, but good enough. But what should it do about duplicate keys? If we say that it always picks from one argument--the left, say--on conflict, then that provides a complete solution to the update problem.

P.P.S. Rank is fun to think about. Given a symbolic vector, you can get a vector of its keys from I. 1"0 y, which is rather cute.

On Mon, 31 Jan 2022, Raul Miller wrote:

That's a good point.

Conceptually, a "dictionary" is an implementation which supports indexing
along the lines of (ref i. names) { values

And, an array where at least one dimension is symbols instead of integers
would be in the right ballpark.

That said, this would share some aspects with sparse arrays, which is a bit
worrisome from an implementation perspective.

Also, some operations, like overtake (n {. m where n exceeds the size of m)
di not make sense.

So that leads into a lot of little issues -- like dimensions (should
dictionary dimensions be like regular dimensions, or should they be
something else? Infinite? Negative? Complex? Negative might make the most
sense... And, when updating, maybe index errors should not be
implemented...) Or, like representation--lots of critical details there,
all with costs and tradeoffs.

Anyways, looking at symbols and sparse arrays, we can see that significant
effort has
been made, already. And, that this will not be trivial.

Thanks,

--
Raul

On Mon, Jan 31, 2022 at 12:58 PM Elijah Stone <elro...@elronnd.net> wrote:

I agree with Eric regarding the challenges of adding dictionaries.

One issue: I think a necessary prerequisite is improved symbols.

On Mon, 31 Jan 2022, Alex Shroyer wrote:

> I agree with Raoul that competing with Python is not a good idea.
> But J can learn from Python's decisions (good and bad) to grow and
improve.
> In my opinion, numpy is Python's "killer app" because it brings
reasonable
> performance without much conceptual overhead.
> The feature of Python that enabled numpy is its extensibility, down to
the
> C layer.
>
> There are other good features of Python that J could copy, in particular
> the 'dictionary' data type.
> The array language K has dictionaries, so J might take some inspiration
> from there for integrating them into the language.
>
> Cheers,
> Alex
>
>
> On Mon, Jan 31, 2022 at 5:30 AM Raoul Schorer <raoul.scho...@gmail.com>
> wrote:
>
>> Just my 2c, but I think that competing with python in general is
somewhat
>> delusional. I think the key point for expanding J use to have a "killer
J
>> app". For example, an improved clone of or excellent plugin for
VisiData (
>> https://www.visidata.org/) is my idea of a killer app. But someone here
>> may
>> have a better idea?
>>
>> Cheers,
>> Raoul
>>
>> Le lun. 31 janv. 2022 à 03:49, Ric Sherlock <tikk...@gmail.com> a
écrit :
>>
>> > Yes from a data structure point of view, inverted tables get you a
lot of
>> > the way (note they're also available in the 'general/misc' addon -
load
>> > 'general/misc/inverted' ) and I've used them to good effect in my
>> > 'data/struct' addon (https://github.com/tikkanz/data_struct).
>> > I agree that J's arrays are more general, flexible & powerful. But
when
>> > you're dealing with a tabular data set there is an overhead to keeping
>> the
>> > fields in a table in sync that dataframes can help with. Perhaps it's
>> > something abstracting the structure so you don't have to deal so much
>> with
>> > the mechanics of manipulating it? At least for me :)
>> >
>> > On Mon, Jan 31, 2022 at 3:28 PM Elijah Stone <elro...@elronnd.net>
>> wrote:
>> >
>> > > https://code.jsoftware.com/wiki/Essays/Inverted_Table, perhaps?
>> > >
>> > > That said, I think a great strength of j is that data are _not_
>> > explicitly
>> > > tabular.  The associations are defined in an ad-hoc manner as
needed by
>> > > the programmer.  This is also an essential difference between the
array
>> > > paradigm and the relational paradigm (cf SQL): in the former,
pointers
>> > > come with implicit context; in the latter, that context must be
>> explicit.
>> > >
>> > >   -E
>> > >
>> > > P.S. regarding analysis/optimization: I would love to see it, but
for
>> > some
>> > > reason everybody is scared of building a compiler because of the
>> parsing
>> > > problem.
>> > >
>> > > On Mon, 31 Jan 2022, Ric Sherlock wrote:
>> > >
>> > > > Yes, I've been thinking that a Dataframes equivalent in J would be
>> > > useful.
>> > > > Most things are already possible with J's arrays, but conceptually
>> > > > DataFrames are well understood by many now, and they make it easy
to
>> > work
>> > > > with datasets as named fields.
>> > > > I've spent a reasonable amount of time working with Pandas, but
have
>> > > > recently been using Polars (Rust backend with Python bindings)
which
>> > > really
>> > > > shines for larger datasets. Performance (especially read/write) is
>> > > awesome,
>> > > > and the LazyFrames which optimise your query/analysis plan make a
big
>> > > > difference too.
>> > > > I haven't taken enough time to explore it, but maybe Jd is the
>> starting
>> > > > point in this space for J?
>> > > >
>> > > >
>> > > > On Mon, Jan 31, 2022 at 9:01 AM Michail L. Liarmakopoulos <
>> > > > m.l.liarm...@gmail.com> wrote:
>> > > >
>> > > >> Hello all,
>> > > >>
>> > > >> I find any parallels between python and J pretty interesting,
being
>> a
>> > > >> person with some python experience and an interest of the
>> applications
>> > > of
>> > > >> both python and J in mathematical modelling, analytics,
>> computational
>> > > math
>> > > >> and perhaps computational physics too.
>> > > >>
>> > > >> If you'd like to bring some features from the python
math/analytics
>> > > >> libraries/ecosystem in J, I'd suggest you to look at the
features of
>> > > three
>> > > >> libraries:
>> > > >>
>> > > >> - numpy (I believe most features are already covered from the
built
>> in
>> > > >> features of an array language such as J)
>> > > >>
>> > > >> - pandas ( a nice library for manipulating csv files within
python
>> as
>> > > >> dataframe objects -- see the dataframes from the R language)
>> > > >>
>> > > >> - scipy (a collection of methods and functions ranging from
solving
>> > > >> numerically: differential equations, evaluating definite
integrals,
>> > > >> constrained and unconstrained optimization, and I believe
statistics
>> > > too)
>> > > >>
>> > > >> There is also out there an amazing python library for symbolic
>> > > calculations
>> > > >> (like the ones you can do with Mathematica or WolframAlpha:
symbolic
>> > > >> evaluation of definite and indefinite integrals, symbolic
solutions
>> of
>> > > >> diff. equations, symbolic solutions of algebraic and Diophantine
>> > > equations
>> > > >> etc...).  It's called sympy.
>> > > >>
>> > > >> But I'm not sure if you'd like J to include symbolic computations
>> too
>> > > or if
>> > > >> the aim of the language is to excel only in numerics, data
>> analytics,
>> > > >> stats, whatever can be quantified pretty much.
>> > > >>
>> > > >> My few cents as food for thought.
>> > > >>
>> > > >>
>> > > >> Best regards,
>> > > >>
>> > > >> ---
>> > > >> Michail L. Liarmakopoulos, MSc
>> > > >>
>> > > >> On Sun, Jan 30, 2022, 20:39 R.E. Boss <r.e.b...@outlook.com>
wrote:
>> > > >>
>> > > >> >
>> > > >> >
>> > > >> > I copied the first chapter of the book A Journey to Core Python
>> (in
>> > > >> >
>> > > >>
>> > >
>> >
>>
https://drive.google.com/file/d/1p1uIANh-LFniNNRqjDeeWWd4_-ddEZmz/view?usp=sharing
>> > > >> )
>> > > >> > and have the question: do we want that J is competitive with
>> Python?
>> > > >> >
>> > > >> > If the answer is yes, the next question is: what is the to do
list
>> > to
>> > > be
>> > > >> > competitive and how long will it take?
>> > > >> >
>> > > >> > (And then the unavoidable question: WHY?)
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > Personally I think we must aim on the niches in the market, as
>> there
>> > > are
>> > > >> > the mathematical oriented people, e.g. the broad scientific
>> > community.
>> > > >> >
>> > > >> > Then all people experienced in Excel or other spreadsheets or
>> > > calculation
>> > > >> > tools.
>> > > >> >
>> > > >> > Schools and universities. Financial and statistical oriented
>> people.
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > What we should do, IMHO, is
>> > > >> >
>> > > >> > - to emphasize the strengths of J,
>> > > >> >
>> > > >> > - to improve (considerably) the error handling of J,
>> > > >> >
>> > > >> > - to admit the steep learning curve,
>> > > >> >
>> > > >> > - to facilitate the use of mnemonics instead of primitives (I
>> tried
>> > > this
>> > > >> > afternoon the primitives.ijs for half an hour, but was not
capable
>> > of
>> > > use
>> > > >> > any mnemonic, not even with
>> > > >> > https://code.jsoftware.com/wiki/Primitives_to_Mnemonics)
>> > > >> >
>> > > >> > - to decide which of the features, benefits or applications (of
>> > > Python)
>> > > >> we
>> > > >> > want J to have.
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > Just my 2 cents.
>> > > >> >
>> > > >> > R.E. Boss
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > ----------------------------------------------------------------------
>> > > >> > For information about J forums see
>> > > http://www.jsoftware.com/forums.htm
>> > > >> >
>> > > >>
>> ----------------------------------------------------------------------
>> > > >> For information about J forums see
>> > http://www.jsoftware.com/forums.htm
>> > > >>
>> > > >
>> ----------------------------------------------------------------------
>> > > > For information about J forums see
>> http://www.jsoftware.com/forums.htm
>> > >
----------------------------------------------------------------------
>> > > For information about J forums see
http://www.jsoftware.com/forums.htm
>> > >
>> > ----------------------------------------------------------------------
>> > For information about J forums see
http://www.jsoftware.com/forums.htm
>> >
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm

----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm

Reply via email to