Thanks Thomas, my comments are below: On Thu, Mar 10, 2016 at 10:35 AM, Thomas Costigliola <[email protected]> wrote: > > > Great list Joe, I'll respond to each below... > On Mar 9, 2016 7:26 AM, Joe Bogner <[email protected]> wrote: >> >> Some wishlist ideas (some are likely controversial): >> >> 1. Optimized CSV/TSV loading and writing > > I think some people like to keep the core language small and use the standard > library and addons for higher level tools. However, this is so useful and > prevalent in J that I think its a good fit and will help to modernize and > grow the language. Can you provide some details on how you think built in > support should work? >
I agree with keeping the core language small and adding to it under a set of guidelines which I won't attempt to enumerate here. The relevant criteria for this request would be "common" and "poor performance compared to competitive languages" I suppose native, compiled, addons could suffice (like CRAN) but let's assume we think it should be added to the core language. I would suggest either adding a foreign to 1!: or overloading 1!:1 and and 1!:2. The overload is debatable, I'm not sure how I feel about it so I'd want community input. Let's say we overloaded it, I would add an optional x to 1!:1 that could accept some boxed options and likewise 1!:2. HOWEVER, for consistency, it may make more sense to add as a optional boxed element on the y for both instead of x. For reading, we might consider using: 1. https://github.com/robertpostill/libcsv 2. or concepts from https://github.com/Rdatatable/data.table/blob/2448a0d9a7dd3ec0df60f7424528ed43317a9181/src/fread.c I like the idea of reading the data in as an inverted table. I think the individually boxed table should offered as a display representation (more below) The inverted table could then be written to a csv by using the 1!:2 overload (or other foreign). This will negate the need to convert to a string prior to writing Regarding the display of inverted tables as regularly boxed tables - it feels like there is an opportunity to add to 8!: (http://www.jsoftware.com/help/dictionary/dx008.htm) to add native support for inverted tables. As I worked with MicroJ, https://github.com/joebo/microj, I realized that I could have made tables a formatting option instead of a native type. The formatting options for inverted tables should allow things like: a. N rows to be output (e.g. output the top 5 rows) b. Range of rows to be output (e.g. after locating an item in a column, I want to output the corresponding rows) >> 2. JSON support > > Same as above. > I don't have a library recommendation, but I would suggest offering the ability to serialize into an inverted table as above I have JSON support in MicroJ: https://github.com/joebo/microj/blob/3f6d651607608bd7998a0eba225c317e5d36b865/tests.ijs#L848, https://github.com/joebo/microj/blob/4c123d956f66047573e54948d75a45aaa2f302cc/primitives.cs#L2747 I would suggest two possible features: 1. Select a string a JSON from a node path, e.g, extract the invoices node from this structure: { data: { invoices: [{nbr: 1, amt: 5}, {nbr: 2, amt: 5} ] } } 2. Convert a string of JSON to an inverted table I wouldn't worry so much about deeply nested structures for now except in the selection support from #1. Perhaps validate the idea by looking at some common JSON file formats - https://www.quandl.com/api/v3/datasets/WIKI/AAPL.json?start_date=1985-05-01 - http://api.worldbank.org/countries/all/indicators/PALM_OIL?date=2000M01:2020M12&per_page=1000&format=json >> 3. Parallel processing (big topic, but maybe something bite-sized for >> now - just to show some progress) > > I had the idea to use the c11 thread primitives to add asynchronous > operations. This will most likely be the next big project. > Excellent! >> 4. Sockets built-in (comparing to k/kona's simple way of retrieving >> web data or ipc) > > This is the same as 1 and 2 for me. I'd like to hear some ideas on how the > interface should be designed. > Let me give it some more thought, here's some ideas from K: http://code.kx.com/wiki/Cookbook/IPCInANutshell http://code.kx.com/wiki/Cookbook/MultithreadedInputMode http://code.kx.com/wiki/Cookbook/ProgrammingExamples >> 5. More interop: .NET/Java > > I have no personal interest in those specifically, but contributions are > always welcome. >> >> And then maybe a bit more tactical: >> 1. Method of getting memory address of current J instance: >> http://www.jsoftware.com/pipermail/source/2014-May/000581.html Great! This will be useful for C libraries called from J to be able to directly modify the J data >> 2. Method of cleaning up instance memory on linux: >> http://www.jsoftware.com/pipermail/source/2015-February/000663.html I'm not sure how helpful this will be in the real world except for when J is called from a server environment. In my usage, I was using a go based server to invoke J - https://github.com/joebo/lang-lab/tree/master/go/http-j > > Very useful, I'll implement these soon. > > > -------- Original message -------- > From: Joe Bogner <[email protected]> > Date: 3/9/2016 7:26 AM (GMT-05:00) > To: [email protected] > Subject: Re: [Jprogramming] Unbox request for requests > > Some wishlist ideas (some are likely controversial): > > 1. Optimized CSV/TSV loading and writing > 2. JSON support > 3. Parallel processing (big topic, but maybe something bite-sized for > now - just to show some progress) > 4. Sockets built-in (comparing to k/kona's simple way of retrieving > web data or ipc) > 5. More interop: .NET/Java > > And then maybe a bit more tactical: > 1. Method of getting memory address of current J instance: > http://www.jsoftware.com/pipermail/source/2014-May/000581.html > 2. Method of cleaning up instance memory on linux: > http://www.jsoftware.com/pipermail/source/2015-February/000663.html > > > On Tue, Mar 8, 2016 at 7:32 AM, Thomas Costigliola <[email protected]> wrote: >> >> >> So far Unbox has a couple of bug fixes and build system changes not in the >> standard source release. I have begun working on a re-implementation of >> gerunds that should fix some bugs and allow some new features. This is >> working out to be a somewhat long term project and I'd like to keep things >> fresh and exciting in the meantime. >> Please post your top BUG and/or FEATURE wishlists. >> >> >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
