On Thu, Nov 20, 2014 at 11:02 PM, Joshua M. Clulow <j...@sysmgr.org> wrote:
> On 20 November 2014 12:05, Gordon Ross <gordon.w.r...@gmail.com> wrote: > > Could one use some javascript code to convert the LDJSON form to the > > other proposed form? (with "stdout" and array wrapper stuff) > > Assuming that's true, I'd lean toward providing only the LDJSON form, > > which seems more "minimalist". > > Absolutely. If you're writing Javascript software that runs in a > Node.js environment, the "lstream" module will split input on any > stream (including "stdout" from a child process) into lines as they > are read into the process. You can then parse and process each line > independently. If you _need_ one big array, and your dataset fits in > entirely in the heap, you can assemble it trivially with Array.push() > as it arrives. > > If you're asking about Javascript in the browser, there are myriad > frameworks targeting the browser environment (e.g. jQuery, etc). > You'd need to do whatever your framework supports for streaming big > datasets from remote servers, but the basic principle is the same. Well not everybody love javascript or Node.js... especially for server side processing :) Regarding openzfs : with thousands of filesystems, command like "zfs list" are taking time (long time...) and JSON or LDJSON will not change anything on this point IMHO. REM: this will be very nice performance optimisation project for openzfs in general... Agree that when using LDJSON you can use less memory if you want to process output line by line but regarding JSON size and for openzfs usages which size are we dealing with : few MB I guess or please let us know the use case which will generate GB of JSON in a single shot ! Within Alyseo we do not care that much about using JSON or LDJSON outputs for this openzfs contrib cause we can process and manage both. But I do not have in mind any tools similar to openzfs which generate LDJSON, I'm certainly wrong here please give me some links. Anyway, the community need to decide and hope we can all move forward on this topic asap ;-) In order to make everybody "happy" we have proposed to implement both (JSON with "-J" and LDJSON using "-j") but fine if all want LDJSON only : shall we do a public survey or the core dev can take this decision ? Regards, Yacine
_______________________________________________ developer mailing list developer@open-zfs.org http://lists.open-zfs.org/mailman/listinfo/developer
