The simplest dictionary is a table with 2 columns. First named k(ey), other
named v(alue)
A J locale is also a simple dictionary. Keys are limited to legal names, but
values are independently typed. It is difficult to find keys by value, whereas
it is simpler in the 2 column table concept. Retrieving multiple values by
key(s) maybe most valuable as returning a 2 column table with keys returned as
well to further subselect later.
Instead of limiting to 2 columns, and shoehorning all keys and record values
into a single variable/column, supporting multi column tables is still a
dictionary.
On Monday, January 31, 2022, 11:32:50 p.m. EST, Henry Rich
<henryhr...@gmail.com> wrote:
Many ideas have been aired. But please, every now and again address the
questions I need answers for:
1. What is a Dictionary, EXACTLY?
2. What operations can be performed on a Dictionary?
Henry Rich
On 1/31/2022 11:09 PM, 'Pascal Jasmin' via Programming wrote:
> I forgot to mention that the GET/SET interface can be applied to basic arrays
> and matrices and would mirror functionality of primitives such as { i.
>
> Independently, of whether data structures have a backend that syncs them to
> disk for permanence, they all have get/set/append/delete processing functions.
>
> Yes a typed multi key and multi value table is very much like a DBMS primary
> key'd table. Many dictionaries fit within those constraints, and compared to
> a coded key string and coded record value, including a simple boxed encoding
> of every field, a typed version will have performance benefits. A variant
> field within a dictionary or table is still possible and permissible, but
> shouldn't be a/the default implementation. A table/dictionary can be "self
> typing" = consistent with the first record value appended.
>
> Jd may be heavy for this. It would be an option to copy its interface, but I
> think I can improve it.
>
>
>
> On Monday, January 31, 2022, 09:20:47 p.m. EST, Don Guinn
> <dongu...@gmail.com> wrote:
>
>
>
>
>
> I hope you find me an idiot, but isn't that what a DBMS does? Maybe the
> DBMS in J could be extended a little to do all this.
>
> On Mon, Jan 31, 2022, 6:22 PM 'Pascal Jasmin' via Programming <
> programm...@jsoftware.com> wrote:
>
>> Beyond dictionaries is a "generic data structure" construct, and a large
>> extended family of data structures that can respond to the following
>> commands
>>
>> GET SET UPSERT APPEND DELETE INSERTafter(item) INSERTbefore(item) WHEREget
>> CONVERTtoJnativearray UPSERTfromJnativearray
>>
>> in addition to dictionaries, ragged nested arrays or generally any binary
>> encoding of datatype;shape/count;content
>>
>> for dictionaries, some applications (or at least language implementations)
>> permit variant/inconsistent type keys and/or values.
>>
>> and generally, a table is a dictionary with
>>
>> multiple keys and multiple associated values to each unique combination of
>> the multiple keys. With potential performance advantages to keeping it in
>> a sorted order if retrieval of items by query, or having multiple fields
>> indexed. Typed "tables"/dictionaries get inherent performance advantages
>> due to not requiring item boxing.
>>
>> as a generic "pattern framework" for data structures without using classes:
>>
>> DEFAULT =: ''
>>
>> data dataconstructor DEFAULT NB. dataconstructor is conjunction that will
>> return an adverb that is (conjunction data). example
>>
>> mydict = ('key1'; value1) dict DEFAULT NB. an adverb that is a
>> conjunction bound to specific data, and where the conjunction code responds
>> to commands in list above top.
>>
>> The list of commands near top of this message are actually numeric codes
>> that get passed to the bound datastructure adverb.
>>
>> GET mydict becomes a verb that will retrieve items.
>> SET mydict returns an conjunction with m as key, n as value that will
>> return the same mydict adverb but with updated "data"
>> (newkey; SET) mydict returns an adverb that is above conjunction with m
>> bound as newkey
>> (newkey: newdata; SET) mydict directly returns the updated mydict adverb.
>>
>> In the case of typed/consistent dictionaries, the internal data
>> representation is just an inverted table of "fields". And while the
>> underlying conjunction of mydict is bound to data to form an adverb, a
>> generic (consistent typed) dictaccess conjunction can operate on any
>> inverted table if prebound with GET/SET. And the GET/SET/OTHER "commands"
>> can form a verb as a result of paired with a generic access adverb, all
>> "command parameters" as x, and the inverted table/data structure as y.
>>
>> What makes this approach have more appeal to be over classes is besides
>> issues like reference management and it is hard to send methods to
>> class/objects in a "parameterized" way, is that arrays are J's
>> first/primary datastructure, and in some datastructures can be
>> transparently accessed by independent J code.
>>
>> Generally, putting the complexity in modifiers instead of classes is a
>> functional approach I would prefer. I can volunteer to start it off at
>> least if there is interest in this approach.
>>
>> It would be a datastructure access "language/dsl" on top of J.
>>
>>
>>
>>
>> On Monday, January 31, 2022, 01:43:30 p.m. EST, Henry Rich <
>> henryhr...@gmail.com> wrote:
>>
>>
>>
>>
>>
>> Quite right. Jan-Pieter has at least a good start on the model. What I
>> am suggesting is that the 'more integrated implementation' might not
>> need to be much more integrated than the J script. Searching and
>> sorting are already J primitives.
>>
>> The deficiency in J is that m&i. gives you a hashtable for searching m,
>> but if you modify m you have to recalculate the hash from scratch. This
>> makes m&i. a good read-mostly Dictionary but slow for frequent updates.
>> If we can refine the model to the point that such inefficiency is the
>> worst problem we have, a little integrated support could finish the
>> package.
>>
>> Henry Rich
>>
>> On 1/31/2022 1:37 PM, Eric Iverson wrote:
>>> A good and complete model is the first step. Then it is a matter of how
>>> much it is used and what drawbacks it might have that would be addressed
>> by
>>> a more integrated implementation.
>>>
>>> On Mon, Jan 31, 2022 at 1:27 PM Henry Rich <henryhr...@gmail.com> wrote:
>>>
>>>> I have looked into this quite a bit. I am not convinced that Dictionary
>>>> is a fundamental datatype like number or letter, or that the current
>>>> symbol support is deficient. That makes the first questions What is a
>>>> Dictionary? and Where can a Dictionary be used in J?
>>>>
>>>> The use case that would be important to me is a key-value store, aka
>>>> associative memory, where the Dictionary remember keys and values and
>>>> later returns a value when presented with the key. This feels to me like
>>>> a package written in J rather than a J primitive. A Dictionary would be
>>>> a numbered locale. The place where I could see added support inside J
>>>> would be in adding to and deleting from hashtables, specifically ones
>>>> created/used by m&i. .
>>>>
>>>> I invite proposals on what a Dictionary package needs to do. I have
>>>> done this before, and Jan-Pieter Jacobs responded with a package.
>>>> Quoting from his message of 20210327:
>>>>
>>>> *** QUOTE
>>>>
>>>> install'github:jpjacobs/types_dict@main'
>>>>
>>>> (except on (my) android, where github installs don't seem to work,
>> neither
>>>> in the Gui version JA nor on the commandline via termux)
>>>>
>>>> It is pretty simple in use, you just use:
>>>> d=: dict keys;vals
>>>> for creating the dictionary, which is a OOP object, having the following
>>>> methods:
>>>> get, set, map, sort, destroy, whose documentation is contained in
>>>> help_pdict_
>>>>
>>>> For performance, the create verb precomputes the lookup for the get
>> verb,
>>>> both forward get (key->value) and backward get inv (value->first key)
>> upon
>>>> object creation.
>>>>
>>>> *** END QUOTE
>>>>
>>>> I have not looked into this package, but it seems to me to have the
>>>> right entry points. Can we take that as a starting point to see what
>>>> needs to be added? The first step of course would be to put the addon
>>>> under Package Manager.
>>>>
>>>> Henry Rich
>>>>
>>>>
>>>>
>>>> On 1/31/2022 12:58 PM, Elijah Stone wrote:
>>>>> I agree with Eric regarding the challenges of adding dictionaries.
>>>>>
>>>>> One issue: I think a necessary prerequisite is improved symbols.
>>>>>
>>>>> On Mon, 31 Jan 2022, Alex Shroyer wrote:
>>>>>
>>>>>> I agree with Raoul that competing with Python is not a good idea.
>>>>>> But J can learn from Python's decisions (good and bad) to grow and
>>>>>> improve.
>>>>>> In my opinion, numpy is Python's "killer app" because it brings
>>>>>> reasonable
>>>>>> performance without much conceptual overhead.
>>>>>> The feature of Python that enabled numpy is its extensibility, down
>>>>>> to the
>>>>>> C layer.
>>>>>>
>>>>>> There are other good features of Python that J could copy, in
>> particular
>>>>>> the 'dictionary' data type.
>>>>>> The array language K has dictionaries, so J might take some
>> inspiration
>>>>>> from there for integrating them into the language.
>>>>>>
>>>>>> Cheers,
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>> On Mon, Jan 31, 2022 at 5:30 AM Raoul Schorer <
>> raoul.scho...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Just my 2c, but I think that competing with python in general is
>>>>>>> somewhat
>>>>>>> delusional. I think the key point for expanding J use to have a
>>>>>>> "killer J
>>>>>>> app". For example, an improved clone of or excellent plugin for
>>>>>>> VisiData (
>>>>>>> https://www.visidata.org/) is my idea of a killer app. But someone
>>>> here
>>>>>>> may
>>>>>>> have a better idea?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Raoul
>>>>>>>
>>>>>>> Le lun. 31 janv. 2022 à 03:49, Ric Sherlock <tikk...@gmail.com> a
>>>>>>> écrit :
>>>>>>>
>>>>>>>> Yes from a data structure point of view, inverted tables get you a
>>>>>>> lot of
>>>>>>>> the way (note they're also available in the 'general/misc' addon -
>>>>>>> load
>>>>>>>> 'general/misc/inverted' ) and I've used them to good effect in my
>>>>>>>> 'data/struct' addon (https://github.com/tikkanz/data_struct).
>>>>>>>> I agree that J's arrays are more general, flexible & powerful. But
>>>>>>> when
>>>>>>>> you're dealing with a tabular data set there is an overhead to
>>>>>>> keeping
>>>>>>> the
>>>>>>>> fields in a table in sync that dataframes can help with. Perhaps
>> it's
>>>>>>>> something abstracting the structure so you don't have to deal so
>> much
>>>>>>> with
>>>>>>>> the mechanics of manipulating it? At least for me :)
>>>>>>>>
>>>>>>>> On Mon, Jan 31, 2022 at 3:28 PM Elijah Stone <elro...@elronnd.net>
>>>>>>> wrote:
>>>>>>>>> https://code.jsoftware.com/wiki/Essays/Inverted_Table, perhaps?
>>>>>>>>>
>>>>>>>>> That said, I think a great strength of j is that data are _not_
>>>>>>>> explicitly
>>>>>>>>> tabular. The associations are defined in an ad-hoc manner as
>>>>>>> needed by
>>>>>>>>> the programmer. This is also an essential difference between
>>>>>>> the array
>>>>>>>>> paradigm and the relational paradigm (cf SQL): in the former,
>>>>>>> pointers
>>>>>>>>> come with implicit context; in the latter, that context must be
>>>>>>> explicit.
>>>>>>>>> -E
>>>>>>>>>
>>>>>>>>> P.S. regarding analysis/optimization: I would love to see it,
>>>>>>> but for
>>>>>>>> some
>>>>>>>>> reason everybody is scared of building a compiler because of the
>>>>>>> parsing
>>>>>>>>> problem.
>>>>>>>>>
>>>>>>>>> On Mon, 31 Jan 2022, Ric Sherlock wrote:
>>>>>>>>>
>>>>>>>>>> Yes, I've been thinking that a Dataframes equivalent in J
>>>>>>> would be
>>>>>>>>> useful.
>>>>>>>>>> Most things are already possible with J's arrays, but
>>>>>>> conceptually
>>>>>>>>>> DataFrames are well understood by many now, and they make it
>>>>>>> easy to
>>>>>>>> work
>>>>>>>>>> with datasets as named fields.
>>>>>>>>>> I've spent a reasonable amount of time working with Pandas,
>>>>>>> but have
>>>>>>>>>> recently been using Polars (Rust backend with Python bindings)
>>>>>>> which
>>>>>>>>> really
>>>>>>>>>> shines for larger datasets. Performance (especially
>>>>>>> read/write) is
>>>>>>>>> awesome,
>>>>>>>>>> and the LazyFrames which optimise your query/analysis plan
>>>>>>> make a big
>>>>>>>>>> difference too.
>>>>>>>>>> I haven't taken enough time to explore it, but maybe Jd is the
>>>>>>> starting
>>>>>>>>>> point in this space for J?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 31, 2022 at 9:01 AM Michail L. Liarmakopoulos <
>>>>>>>>>> m.l.liarm...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello all,
>>>>>>>>>>>
>>>>>>>>>>> I find any parallels between python and J pretty interesting,
>>>>>>> being
>>>>>>> a
>>>>>>>>>>> person with some python experience and an interest of the
>>>>>>> applications
>>>>>>>>> of
>>>>>>>>>>> both python and J in mathematical modelling, analytics,
>>>>>>> computational
>>>>>>>>> math
>>>>>>>>>>> and perhaps computational physics too.
>>>>>>>>>>>
>>>>>>>>>>> If you'd like to bring some features from the python
>>>>>>> math/analytics
>>>>>>>>>>> libraries/ecosystem in J, I'd suggest you to look at the
>>>>>>> features of
>>>>>>>>> three
>>>>>>>>>>> libraries:
>>>>>>>>>>>
>>>>>>>>>>> - numpy (I believe most features are already covered from the
>>>>>>> built
>>>>>>> in
>>>>>>>>>>> features of an array language such as J)
>>>>>>>>>>>
>>>>>>>>>>> - pandas ( a nice library for manipulating csv files within
>>>>>>> python
>>>>>>> as
>>>>>>>>>>> dataframe objects -- see the dataframes from the R language)
>>>>>>>>>>>
>>>>>>>>>>> - scipy (a collection of methods and functions ranging from
>>>>>>> solving
>>>>>>>>>>> numerically: differential equations, evaluating definite
>>>>>>> integrals,
>>>>>>>>>>> constrained and unconstrained optimization, and I believe
>>>>>>> statistics
>>>>>>>>> too)
>>>>>>>>>>> There is also out there an amazing python library for symbolic
>>>>>>>>> calculations
>>>>>>>>>>> (like the ones you can do with Mathematica or WolframAlpha:
>>>>>>> symbolic
>>>>>>>>>>> evaluation of definite and indefinite integrals, symbolic
>>>>>>> solutions
>>>>>>> of
>>>>>>>>>>> diff. equations, symbolic solutions of algebraic and Diophantine
>>>>>>>>> equations
>>>>>>>>>>> etc...). It's called sympy.
>>>>>>>>>>>
>>>>>>>>>>> But I'm not sure if you'd like J to include symbolic
>>>>>>> computations
>>>>>>> too
>>>>>>>>> or if
>>>>>>>>>>> the aim of the language is to excel only in numerics, data
>>>>>>> analytics,
>>>>>>>>>>> stats, whatever can be quantified pretty much.
>>>>>>>>>>>
>>>>>>>>>>> My few cents as food for thought.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best regards,
>>>>>>>>>>>
>>>>>>>>>>> ---
>>>>>>>>>>> Michail L. Liarmakopoulos, MSc
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Jan 30, 2022, 20:39 R.E. Boss <r.e.b...@outlook.com>
>>>>>>> wrote:
>>>>>>>>>>>> I copied the first chapter of the book A Journey to Core
>>>>>>> Python
>>>>>>> (in
>> https://drive.google.com/file/d/1p1uIANh-LFniNNRqjDeeWWd4_-ddEZmz/view?usp=sharing
>>>>>>>>>>> )
>>>>>>>>>>>> and have the question: do we want that J is competitive with
>>>>>>> Python?
>>>>>>>>>>>> If the answer is yes, the next question is: what is the to
>>>>>>> do list
>>>>>>>> to
>>>>>>>>> be
>>>>>>>>>>>> competitive and how long will it take?
>>>>>>>>>>>>
>>>>>>>>>>>> (And then the unavoidable question: WHY?)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Personally I think we must aim on the niches in the market, as
>>>>>>> there
>>>>>>>>> are
>>>>>>>>>>>> the mathematical oriented people, e.g. the broad scientific
>>>>>>>> community.
>>>>>>>>>>>> Then all people experienced in Excel or other spreadsheets or
>>>>>>>>> calculation
>>>>>>>>>>>> tools.
>>>>>>>>>>>>
>>>>>>>>>>>> Schools and universities. Financial and statistical oriented
>>>>>>> people.
>>>>>>>>>>>>
>>>>>>>>>>>> What we should do, IMHO, is
>>>>>>>>>>>>
>>>>>>>>>>>> - to emphasize the strengths of J,
>>>>>>>>>>>>
>>>>>>>>>>>> - to improve (considerably) the error handling of J,
>>>>>>>>>>>>
>>>>>>>>>>>> - to admit the steep learning curve,
>>>>>>>>>>>>
>>>>>>>>>>>> - to facilitate the use of mnemonics instead of primitives (I
>>>>>>> tried
>>>>>>>>> this
>>>>>>>>>>>> afternoon the primitives.ijs for half an hour, but was not
>>>>>>> capable
>>>>>>>> of
>>>>>>>>> use
>>>>>>>>>>>> any mnemonic, not even with
>>>>>>>>>>>> https://code.jsoftware.com/wiki/Primitives_to_Mnemonics)
>>>>>>>>>>>>
>>>>>>>>>>>> - to decide which of the features, benefits or applications
>>>>>>> (of
>>>>>>>>> Python)
>>>>>>>>>>> we
>>>>>>>>>>>> want J to have.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Just my 2 cents.
>>>>>>>>>>>>
>>>>>>>>>>>> R.E. Boss
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>> ----------------------------------------------------------------------
>>>>>>>>>>>> For information about J forums see
>>>>>>>>> http://www.jsoftware.com/forums.htm
>> ----------------------------------------------------------------------
>>>>>>>>>>> For information about J forums see
>>>>>>>> http://www.jsoftware.com/forums.htm
>> ----------------------------------------------------------------------
>>>>>>>>>> For information about J forums see
>>>>>>> http://www.jsoftware.com/forums.htm
>>>>>>>
>> ----------------------------------------------------------------------
>>>>>>>>> For information about J forums see
>>>>>>> http://www.jsoftware.com/forums.htm
>>>>>>>
>> ----------------------------------------------------------------------
>>>>>>>> For information about J forums see
>>>>>>> http://www.jsoftware.com/forums.htm
>>>>>>>
>> ----------------------------------------------------------------------
>>>>>>> For information about J forums see
>> http://www.jsoftware.com/forums.htm
>>>>>> ----------------------------------------------------------------------
>>>>>> For information about J forums see
>> http://www.jsoftware.com/forums.htm
>>>>> ----------------------------------------------------------------------
>>>>> For information about J forums see http://www.jsoftware.com/forums.htm
>>>> --
>>>> This email has been checked for viruses by AVG.
>>>> https://www.avg.com
>>>> ----------------------------------------------------------------------
>>>> For information about J forums see http://www.jsoftware.com/forums.htm
>>>>
>>> ----------------------------------------------------------------------
>>> For information about J forums see http://www.jsoftware.com/forums.htm
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
