[jira] [Created] (ARROW-17979) [JS]: Update status docs
Dominik Moritz created ARROW-17979: -- Summary: [JS]: Update status docs Key: ARROW-17979 URL: https://issues.apache.org/jira/browse/ARROW-17979 Project: Apache Arrow Issue Type: Task Components: Documentation, JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz Arrow JS support nulls, sparse, and dense unions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17903) [JS] Update dependencies
Dominik Moritz created ARROW-17903: -- Summary: [JS] Update dependencies Key: ARROW-17903 URL: https://issues.apache.org/jira/browse/ARROW-17903 Project: Apache Arrow Issue Type: Task Components: JavaScript Affects Versions: 10.0.0 Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 10.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17321) Update dependencies
Dominik Moritz created ARROW-17321: -- Summary: Update dependencies Key: ARROW-17321 URL: https://issues.apache.org/jira/browse/ARROW-17321 Project: Apache Arrow Issue Type: Task Components: JavaScript Affects Versions: 9.0.0 Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 10.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-16693) [JavaScript] Upgrade to TS 4.7
Dominik Moritz created ARROW-16693: -- Summary: [JavaScript] Upgrade to TS 4.7 Key: ARROW-16693 URL: https://issues.apache.org/jira/browse/ARROW-16693 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16256) Document what spec version is supported.
Dominik Moritz created ARROW-16256: -- Summary: Document what spec version is supported. Key: ARROW-16256 URL: https://issues.apache.org/jira/browse/ARROW-16256 Project: Apache Arrow Issue Type: Task Components: Documentation Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 8.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16222) [JS] Allow appending null on sparse union and map builders
Dominik Moritz created ARROW-16222: -- Summary: [JS] Allow appending null on sparse union and map builders Key: ARROW-16222 URL: https://issues.apache.org/jira/browse/ARROW-16222 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz See https://github.com/apache/arrow/pull/12451 and in particular https://github.com/apache/arrow/pull/12451#pullrequestreview-887789954. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16210) [JS] Implement tableFromJSON
Dominik Moritz created ARROW-16210: -- Summary: [JS] Implement tableFromJSON Key: ARROW-16210 URL: https://issues.apache.org/jira/browse/ARROW-16210 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16209) [JS] Support setting values on Tables
Dominik Moritz created ARROW-16209: -- Summary: [JS] Support setting values on Tables Key: ARROW-16209 URL: https://issues.apache.org/jira/browse/ARROW-16209 Project: Apache Arrow Issue Type: Bug Components: JavaScript Affects Versions: 7.0.0 Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 8.0.0 See https://github.com/vega/vega-lite/issues/8105 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16208) [JS] Upgrade reps
Dominik Moritz created ARROW-16208: -- Summary: [JS] Upgrade reps Key: ARROW-16208 URL: https://issues.apache.org/jira/browse/ARROW-16208 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 8.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16167) [JS] Check for opportunities to optimize offsets
Dominik Moritz created ARROW-16167: -- Summary: [JS] Check for opportunities to optimize offsets Key: ARROW-16167 URL: https://issues.apache.org/jira/browse/ARROW-16167 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz Check for opportunities for https://github.com/apache/arrow/pull/12793 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16153) [JS] Consider implementing a tableFromArray
Dominik Moritz created ARROW-16153: -- Summary: [JS] Consider implementing a tableFromArray Key: ARROW-16153 URL: https://issues.apache.org/jira/browse/ARROW-16153 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz The idea here is to implement a function that creates a table from an array of objects using the struct builder. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16099) Warn about not supporting compression
Dominik Moritz created ARROW-16099: -- Summary: Warn about not supporting compression Key: ARROW-16099 URL: https://issues.apache.org/jira/browse/ARROW-16099 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz Fix For: 8.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16098) Support `Iterable>` for Table
Dominik Moritz created ARROW-16098: -- Summary: Support `Iterable>` for Table Key: ARROW-16098 URL: https://issues.apache.org/jira/browse/ARROW-16098 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Affects Versions: 7.0.0 Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 8.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-15821) Sourcemap paths don't work for files in directories
Dominik Moritz created ARROW-15821: -- Summary: Sourcemap paths don't work for files in directories Key: ARROW-15821 URL: https://issues.apache.org/jira/browse/ARROW-15821 Project: Apache Arrow Issue Type: Task Components: JavaScript Affects Versions: 7.0.0 Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 8.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-15379) Use a flywheel for struct row
Dominik Moritz created ARROW-15379: -- Summary: Use a flywheel for struct row Key: ARROW-15379 URL: https://issues.apache.org/jira/browse/ARROW-15379 Project: Apache Arrow Issue Type: Improvement Reporter: Dominik Moritz When we access a row from a table or a struct, we create a proxy for the struct. We could improve the performance of these accesses by creating a single instance of the proxy and store it on the vector or the data type and then reuse that instance. This should improve performance. See https://github.com/apache/arrow/blob/7029f90ea3b39e97f1a671227ca932cbcdbcee05/js/src/visitor/get.ts#L219 and https://github.com/apache/arrow/blob/7029f90ea3b39e97f1a671227ca932cbcdbcee05/js/src/vector/struct.ts#L27. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-14780) [JS]: add a clear method to AsyncByteQueue
Dominik Moritz created ARROW-14780: -- Summary: [JS]: add a clear method to AsyncByteQueue Key: ARROW-14780 URL: https://issues.apache.org/jira/browse/ARROW-14780 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-14775) [JS] Embrace ESM in main arrow package
Dominik Moritz created ARROW-14775: -- Summary: [JS] Embrace ESM in main arrow package Key: ARROW-14775 URL: https://issues.apache.org/jira/browse/ARROW-14775 Project: Apache Arrow Issue Type: Improvement Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 7.0.0 Instead of shipping both esm and commonjs, we could embrace esm as many other js packages do now and thereby clean up our bundles as well. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-14774) [JS] Correct package exports
Dominik Moritz created ARROW-14774: -- Summary: [JS] Correct package exports Key: ARROW-14774 URL: https://issues.apache.org/jira/browse/ARROW-14774 Project: Apache Arrow Issue Type: Bug Components: JavaScript Affects Versions: 6.0.1 Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-14773) [JS] Sourcemaps points to wrong file
Dominik Moritz created ARROW-14773: -- Summary: [JS] Sourcemaps points to wrong file Key: ARROW-14773 URL: https://issues.apache.org/jira/browse/ARROW-14773 Project: Apache Arrow Issue Type: Bug Components: JavaScript Affects Versions: 6.0.1 Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-14739) Docs point to wrong source
Dominik Moritz created ARROW-14739: -- Summary: Docs point to wrong source Key: ARROW-14739 URL: https://issues.apache.org/jira/browse/ARROW-14739 Project: Apache Arrow Issue Type: Bug Components: Documentation, JavaScript Affects Versions: 6.0.0 Reporter: Dominik Moritz Assignee: Krisztian Szucs If you click on the source link for example at https://arrow.apache.org/docs/js/enums/Arrow_dom.Type.html#DateMillisecond we get to https://github.com/kszucs/arrow/blob/f2faf5cbc/js/src/enum.ts#L106 instead of the official arrow repo. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-14508) Continuous releases of the JS library
Dominik Moritz created ARROW-14508: -- Summary: Continuous releases of the JS library Key: ARROW-14508 URL: https://issues.apache.org/jira/browse/ARROW-14508 Project: Apache Arrow Issue Type: Task Components: Continuous Integration, JavaScript Reporter: Dominik Moritz To help people test new features and detect bugs early, we should make regular releases of the JS package. This could be nightly or whenever we merge a pull request with a JS tag on it. I think the latter would be better since it would reduce the number of releases significantly. Releases should be tagged as canary. http://npm.github.io/publishing-pkgs-docs/updating/using-tags.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-14507) Make RC releases as tagged releases
Dominik Moritz created ARROW-14507: -- Summary: Make RC releases as tagged releases Key: ARROW-14507 URL: https://issues.apache.org/jira/browse/ARROW-14507 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz When we make RC releases, we should also make a tagged release on NPM instead of a full release. http://npm.github.io/publishing-pkgs-docs/updating/using-tags.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-14492) Browser bundles broken
Dominik Moritz created ARROW-14492: -- Summary: Browser bundles broken Key: ARROW-14492 URL: https://issues.apache.org/jira/browse/ARROW-14492 Project: Apache Arrow Issue Type: Bug Components: JavaScript Affects Versions: 6.0.0 Reporter: Dominik Moritz Assignee: Dominik Moritz Fix For: 6.0.1 The package.json file should use paths for the `browser` mapping instead of just the file names. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13790) Move ts files into a source directory in bundle
Dominik Moritz created ARROW-13790: -- Summary: Move ts files into a source directory in bundle Key: ARROW-13790 URL: https://issues.apache.org/jira/browse/ARROW-13790 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz RIght now, we publish the ts source files alongside the js, mjs, map, and .d.ts files. Because of how tsc works (https://github.com/microsoft/TypeScript/issues/10704), it will use the ts files instead of the d.ts files. That can lead to problems with compiling with never tsc versions and is more expensive than if tsc uses the d.ts files. To avoid this problem, we should move the sources into a sources directory. We need to make sure that declaration maps still work. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13514) [JS] Update flatbuffers
Dominik Moritz created ARROW-13514: -- Summary: [JS] Update flatbuffers Key: ARROW-13514 URL: https://issues.apache.org/jira/browse/ARROW-13514 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Fix For: 6.0.0 * Update the flatbuffers npm package to version 2 * Remove @types/flatbuffers since flatbuffers comes with its own typings * Update the generated flatbuffers -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13420) [JS] Update dependencies
Dominik Moritz created ARROW-13420: -- Summary: [JS] Update dependencies Key: ARROW-13420 URL: https://issues.apache.org/jira/browse/ARROW-13420 Project: Apache Arrow Issue Type: Task Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13419) [JS] Fix perf tests
Dominik Moritz created ARROW-13419: -- Summary: [JS] Fix perf tests Key: ARROW-13419 URL: https://issues.apache.org/jira/browse/ARROW-13419 Project: Apache Arrow Issue Type: Bug Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13303) [JS] Revise bundles
Dominik Moritz created ARROW-13303: -- Summary: [JS] Revise bundles Key: ARROW-13303 URL: https://issues.apache.org/jira/browse/ARROW-13303 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz * Use es2015 sources in the apache-arrow package since webpack 4 does not support esnext and many people still use it * Generate .cjs and .mjs files instead of just .js to make it clear what the files are. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13299) Upgrade ix and rxjs
Dominik Moritz created ARROW-13299: -- Summary: Upgrade ix and rxjs Key: ARROW-13299 URL: https://issues.apache.org/jira/browse/ARROW-13299 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13277) [JS] Add declaration maps
Dominik Moritz created ARROW-13277: -- Summary: [JS] Add declaration maps Key: ARROW-13277 URL: https://issues.apache.org/jira/browse/ARROW-13277 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Declaration maps link from .d.ts files to source files making it much easier to navigate code from TypeScript projects. We should add declaration maps to our bundles. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13275) [JS] Fix perf tests
Dominik Moritz created ARROW-13275: -- Summary: [JS] Fix perf tests Key: ARROW-13275 URL: https://issues.apache.org/jira/browse/ARROW-13275 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13274) [JS] Remove Webpack
Dominik Moritz created ARROW-13274: -- Summary: [JS] Remove Webpack Key: ARROW-13274 URL: https://issues.apache.org/jira/browse/ARROW-13274 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-13099) [JS] Easier way to work with decimals
Dominik Moritz created ARROW-13099:
--
Summary: [JS] Easier way to work with decimals
Key: ARROW-13099
URL: https://issues.apache.org/jira/browse/ARROW-13099
Project: Apache Arrow
Issue Type: Improvement
Components: JavaScript
Reporter: Dominik Moritz
I have table with a column, let’s call the column c. c.type returns Decimal
{scale: 3, precision: 12}. c.get(0) returns [519500, 0, 0, 0]. I know the value
should be 519.5. After much digging, I found that the value is actually an
arrow “big number”
(https://github.com/apache/arrow/blob/ccdbbe3b765d4df9b2b91270a69ad78a086f2d77/js/src/util/bn.ts),
so it has a valueOf function which returns 519500 (with the obvious caveat
that it’ll overflow if the number is more than 53-bits). I could divide that by
10^scale from c.type to get 519.5. Is there some easier way to do that?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-13031) JS: Support arm in closure compiler on macOS
Dominik Moritz created ARROW-13031: -- Summary: JS: Support arm in closure compiler on macOS Key: ARROW-13031 URL: https://issues.apache.org/jira/browse/ARROW-13031 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12875) [JS] Upgrade Jest and other minor updates
Dominik Moritz created ARROW-12875: -- Summary: [JS] Upgrade Jest and other minor updates Key: ARROW-12875 URL: https://issues.apache.org/jira/browse/ARROW-12875 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12833) [JS] Construct perf data in JS
Dominik Moritz created ARROW-12833: -- Summary: [JS] Construct perf data in JS Key: ARROW-12833 URL: https://issues.apache.org/jira/browse/ARROW-12833 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12832) [JS] Write benchmarks in TypeScript
Dominik Moritz created ARROW-12832: -- Summary: [JS] Write benchmarks in TypeScript Key: ARROW-12832 URL: https://issues.apache.org/jira/browse/ARROW-12832 Project: Apache Arrow Issue Type: Improvement Reporter: Dominik Moritz Assignee: Dominik Moritz This way we can run them against the sources directly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12800) [JS] Drop IE Support and remove text encoder and decoder polyfills
Dominik Moritz created ARROW-12800: -- Summary: [JS] Drop IE Support and remove text encoder and decoder polyfills Key: ARROW-12800 URL: https://issues.apache.org/jira/browse/ARROW-12800 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12799) [JS] Use Nullish Coalescing Operator (??) For Defaults
Dominik Moritz created ARROW-12799: -- Summary: [JS] Use Nullish Coalescing Operator (??) For Defaults Key: ARROW-12799 URL: https://issues.apache.org/jira/browse/ARROW-12799 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12798) [JS] Use == null Comparison
Dominik Moritz created ARROW-12798: -- Summary: [JS] Use == null Comparison Key: ARROW-12798 URL: https://issues.apache.org/jira/browse/ARROW-12798 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12797) [JS] Update readme
Dominik Moritz created ARROW-12797: -- Summary: [JS] Update readme Key: ARROW-12797 URL: https://issues.apache.org/jira/browse/ARROW-12797 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12796) [JS] Support JSON output from benchmarks
Dominik Moritz created ARROW-12796: -- Summary: [JS] Support JSON output from benchmarks Key: ARROW-12796 URL: https://issues.apache.org/jira/browse/ARROW-12796 Project: Apache Arrow Issue Type: Sub-task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12704) [JS] use optional chaining
Dominik Moritz created ARROW-12704: -- Summary: [JS] use optional chaining Key: ARROW-12704 URL: https://issues.apache.org/jira/browse/ARROW-12704 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12703) [JS] Separate Table from DataFrame
Dominik Moritz created ARROW-12703: -- Summary: [JS] Separate Table from DataFrame Key: ARROW-12703 URL: https://issues.apache.org/jira/browse/ARROW-12703 Project: Apache Arrow Issue Type: Improvement Reporter: Dominik Moritz Assignee: Dominik Moritz A DataFrame should be a valid table but not vice versa. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12702) [JS] Upgrade Webpack and terser
Dominik Moritz created ARROW-12702: -- Summary: [JS] Upgrade Webpack and terser Key: ARROW-12702 URL: https://issues.apache.org/jira/browse/ARROW-12702 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12690) [JS] Set up Conbench
Dominik Moritz created ARROW-12690: -- Summary: [JS] Set up Conbench Key: ARROW-12690 URL: https://issues.apache.org/jira/browse/ARROW-12690 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, JavaScript Reporter: Dominik Moritz Set up https://conbench.ursa.dev to continuously report benchmark results from JS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12580) [JS] Infer types for plain JS arrays
Dominik Moritz created ARROW-12580: -- Summary: [JS] Infer types for plain JS arrays Key: ARROW-12580 URL: https://issues.apache.org/jira/browse/ARROW-12580 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz It would be great if one could pass a normal js array to arrow and have some simple type inference. This could be similar to https://github.com/uwdata/arquero/blob/af843a4377254bab970d40559879f9191fe81079/src/arrow/encode/profiler.js#L15. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12562) [JS] Minimal predicate interface
Dominik Moritz created ARROW-12562:
--
Summary: [JS] Minimal predicate interface
Key: ARROW-12562
URL: https://issues.apache.org/jira/browse/ARROW-12562
Project: Apache Arrow
Issue Type: Improvement
Components: JavaScript
Reporter: Dominik Moritz
Assignee: Dominik Moritz
It would be great to have a minimal predicate interface that someone can
implement without having to support the full predicate interface with {{and}}
and {{or}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-12549) Table and RecordBatch should not extend Vector
Dominik Moritz created ARROW-12549: -- Summary: Table and RecordBatch should not extend Vector Key: ARROW-12549 URL: https://issues.apache.org/jira/browse/ARROW-12549 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Right now, Table are chunked vectors and Record Batches are struct vectors which means the classes are linked deeply. We should not extend and instead copy the binary search to Table and copy BaseVector.prototype.get etc to record batch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12548) [JS] Get rid of columns
Dominik Moritz created ARROW-12548: -- Summary: [JS] Get rid of columns Key: ARROW-12548 URL: https://issues.apache.org/jira/browse/ARROW-12548 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Just use the name Child (as we have for Vectors). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12538) Show Vectors in the docs
Dominik Moritz created ARROW-12538: -- Summary: Show Vectors in the docs Key: ARROW-12538 URL: https://issues.apache.org/jira/browse/ARROW-12538 Project: Apache Arrow Issue Type: Sub-task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12537) [JS] Docs build should not include test sources
Dominik Moritz created ARROW-12537: -- Summary: [JS] Docs build should not include test sources Key: ARROW-12537 URL: https://issues.apache.org/jira/browse/ARROW-12537 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12536) [JS] Construct tables from an iterable of objects or js array columns
Dominik Moritz created ARROW-12536:
--
Summary: [JS] Construct tables from an iterable of objects or js
array columns
Key: ARROW-12536
URL: https://issues.apache.org/jira/browse/ARROW-12536
Project: Apache Arrow
Issue Type: Improvement
Components: JavaScript
Reporter: Dominik Moritz
Right now, Arrow has no automatic type inference for JavaScript types, so I
think we would need to add that.
{code:javascript}
// Convert from JS types automatically
const t = Arrow.Table.from({
Country: ["USA", "Canada", "Mexico"],
GDP: [123, 234, 345],
})
// I'd also like Arrow to support other common JS table formats:
const t = Arrow.Table.from([
{Country: "USA", GDP: 123},
{Country: "Canada", GDP: 234},
{Country: "Mexico", GDP: 345},
])
const t = Arrow.Table.from([
["Country", "GDP"],
["USA", 123],
["Canada", 234],
["Mexico", 345],
])
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-12531) [JS] Make the docs more user friendly
Dominik Moritz created ARROW-12531: -- Summary: [JS] Make the docs more user friendly Key: ARROW-12531 URL: https://issues.apache.org/jira/browse/ARROW-12531 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Right now, the docs are very much just an API doc but they don't explain how to use certain functions. We should * add function comments * remove internal functions from the docs * add more examples * document Vector (https://arrow.apache.org/docs/js/modules/_vector_.vector.html is just a namespace) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12528) [JS] Support typed arrays in Vector.from and Table.new
Dominik Moritz created ARROW-12528: -- Summary: [JS] Support typed arrays in Vector.from and Table.new Key: ARROW-12528 URL: https://issues.apache.org/jira/browse/ARROW-12528 Project: Apache Arrow Issue Type: New Feature Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12525) Vector toJSON returns an array
Dominik Moritz created ARROW-12525: -- Summary: Vector toJSON returns an array Key: ARROW-12525 URL: https://issues.apache.org/jira/browse/ARROW-12525 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12423) Codecov badge in main Readme only applies to Rust
Dominik Moritz created ARROW-12423: -- Summary: Codecov badge in main Readme only applies to Rust Key: ARROW-12423 URL: https://issues.apache.org/jira/browse/ARROW-12423 Project: Apache Arrow Issue Type: Task Reporter: Dominik Moritz The badge in https://github.com/apache/arrow/blob/master/README.md links to https://app.codecov.io/gh/apache/arrow, which seems to only show the coverage for the Rust code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12393) Drop closure compiler
Dominik Moritz created ARROW-12393: -- Summary: Drop closure compiler Key: ARROW-12393 URL: https://issues.apache.org/jira/browse/ARROW-12393 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12384) [JS] Improve code style
Dominik Moritz created ARROW-12384: -- Summary: [JS] Improve code style Key: ARROW-12384 URL: https://issues.apache.org/jira/browse/ARROW-12384 Project: Apache Arrow Issue Type: Task Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12383) [JS] Update direct deps
Dominik Moritz created ARROW-12383: -- Summary: [JS] Update direct deps Key: ARROW-12383 URL: https://issues.apache.org/jira/browse/ARROW-12383 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12333) [JS] Remove jest-environment-node-debug and do not emit from typescript by default
Dominik Moritz created ARROW-12333: -- Summary: [JS] Remove jest-environment-node-debug and do not emit from typescript by default Key: ARROW-12333 URL: https://issues.apache.org/jira/browse/ARROW-12333 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12309) [JS] Make es2015 bundles the default
Dominik Moritz created ARROW-12309: -- Summary: [JS] Make es2015 bundles the default Key: ARROW-12309 URL: https://issues.apache.org/jira/browse/ARROW-12309 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz ES2015 is well supported now (https://caniuse.com/es6) so it makes sense to make it the default. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12308) [JS] Update dependencies
Dominik Moritz created ARROW-12308: -- Summary: [JS] Update dependencies Key: ARROW-12308 URL: https://issues.apache.org/jira/browse/ARROW-12308 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz We have a lot of outdated dependencies. Some we may be able to remove but others should at least be updated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12303) [JS] Use iterators instead of generators in critical code paths
Dominik Moritz created ARROW-12303: -- Summary: [JS] Use iterators instead of generators in critical code paths Key: ARROW-12303 URL: https://issues.apache.org/jira/browse/ARROW-12303 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Attachments: Screen Shot 2021-04-08 at 09.40.51.png I found that generators are generally slower than iterators in most browsers. See [https://observablehq.com/@domoritz/yield-vs-iterator-vs-callback.] I attached the results for Chrome. We should consider using iterators instead of generators for iterating over row proxies and vectors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12302) [JS] Arrow does not compile with Typescript 4.2
Dominik Moritz created ARROW-12302: -- Summary: [JS] Arrow does not compile with Typescript 4.2 Key: ARROW-12302 URL: https://issues.apache.org/jira/browse/ARROW-12302 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12281) [JS] Remove shx and rimraf
Dominik Moritz created ARROW-12281: -- Summary: [JS] Remove shx and rimraf Key: ARROW-12281 URL: https://issues.apache.org/jira/browse/ARROW-12281 Project: Apache Arrow Issue Type: Sub-task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz We can use del instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12276) [JS] Run build before tests
Dominik Moritz created ARROW-12276: -- Summary: [JS] Run build before tests Key: ARROW-12276 URL: https://issues.apache.org/jira/browse/ARROW-12276 Project: Apache Arrow Issue Type: Task Reporter: Dominik Moritz Assignee: Paul Taylor The tests exercise the compiled files but the test scripts do not automatically run the builds. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12275) [JS] Document how to run tests without building
Dominik Moritz created ARROW-12275: -- Summary: [JS] Document how to run tests without building Key: ARROW-12275 URL: https://issues.apache.org/jira/browse/ARROW-12275 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz https://github.com/apache/arrow/blob/master/js/DEVELOP.md does not document that one can run `npm run test -- -t src`. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12274) [JS] Document how to run tests without building
Dominik Moritz created ARROW-12274: -- Summary: [JS] Document how to run tests without building Key: ARROW-12274 URL: https://issues.apache.org/jira/browse/ARROW-12274 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz https://github.com/apache/arrow/blob/master/js/DEVELOP.md does not document that one can run `npm run test -- -t src`. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12273) [JS] Coveralls does not work anymore
Dominik Moritz created ARROW-12273: -- Summary: [JS] Coveralls does not work anymore Key: ARROW-12273 URL: https://issues.apache.org/jira/browse/ARROW-12273 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz Coveralls does not seem to be updated anymore. The last update was January 2019 (https://coveralls.io/github/apache/arrow). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12272) [JS] Remove npm-run-all
Dominik Moritz created ARROW-12272: -- Summary: [JS] Remove npm-run-all Key: ARROW-12272 URL: https://issues.apache.org/jira/browse/ARROW-12272 Project: Apache Arrow Issue Type: Sub-task Components: JavaScript Reporter: Dominik Moritz It doesn't seem to be used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12271) [JS] Make lerna optional
Dominik Moritz created ARROW-12271: -- Summary: [JS] Make lerna optional Key: ARROW-12271 URL: https://issues.apache.org/jira/browse/ARROW-12271 Project: Apache Arrow Issue Type: Sub-task Components: JavaScript Reporter: Dominik Moritz We use npx most of the time anyway so we could make lerna optional. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12270) [JS] remove rxjs dependency
Dominik Moritz created ARROW-12270: -- Summary: [JS] remove rxjs dependency Key: ARROW-12270 URL: https://issues.apache.org/jira/browse/ARROW-12270 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Paul Taylor -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12269) [JS] Move to eslint
Dominik Moritz created ARROW-12269: -- Summary: [JS] Move to eslint Key: ARROW-12269 URL: https://issues.apache.org/jira/browse/ARROW-12269 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Tslint is deprecated so we should switch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12268) [JS] Allow flexible versions of dependencies
Dominik Moritz created ARROW-12268: -- Summary: [JS] Allow flexible versions of dependencies Key: ARROW-12268 URL: https://issues.apache.org/jira/browse/ARROW-12268 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz https://github.com/apache/arrow/blob/master/js/.npmrc#L1 says that we should fix versions of dependencies but that means any user has to use the specific versions we use. It would be good to be more flexible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12238) [JS] Remove trailing spaces
Dominik Moritz created ARROW-12238: -- Summary: [JS] Remove trailing spaces Key: ARROW-12238 URL: https://issues.apache.org/jira/browse/ARROW-12238 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12239) [JS] Switch to yarn
Dominik Moritz created ARROW-12239: -- Summary: [JS] Switch to yarn Key: ARROW-12239 URL: https://issues.apache.org/jira/browse/ARROW-12239 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz Yarn is faster and has a more convenient API (e.g. no need to add `run`). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-12124) [Rust] Parquet writer creates invalid parquet files
Dominik Moritz created ARROW-12124:
--
Summary: [Rust] Parquet writer creates invalid parquet files
Key: ARROW-12124
URL: https://issues.apache.org/jira/browse/ARROW-12124
Project: Apache Arrow
Issue Type: Bug
Components: Rust
Reporter: Dominik Moritz
I wrote a simple CSV to Parquet converter at
https://github.com/domoritz/csv2parquet/blob/f53feb5bd995eab41dee09f2c4d722512052d7ca/src/main.rs.
Running it (`csv2parquet test.txt test.parquet`) with a simple file such as
```
a,b,c
0,1,hello world
0,1,hello world
0,1,hello world
0,1,hello world
0,1,hello world
0,1,hello world
0,1,hello world
```
And then trying to read in Python with
```
import pandas as pd
df = pd.read_parquet('test.parquet')
df.to_csv('test2.csv')
```
Results in this error
```
OSError: Could not open parquet input source '': Invalid: Parquet magic
bytes not found in footer. Either the file is corrupted or this is not a
parquet file.
```
The schema seems to be inferred correctly
```
Inferred Schema:
{
"fields": [
{
"name": "a",
"nullable": false,
"type": {
"name": "int",
"bitWidth": 64,
"isSigned": true
},
"children": []
},
{
"name": "b",
"nullable": false,
"type": {
"name": "int",
"bitWidth": 64,
"isSigned": true
},
"children": []
},
{
"name": "c",
"nullable": false,
"type": {
"name": "utf8"
},
"children": []
}
],
"metadata": {}
}
```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-11874) [Rust] implement FromStr for parquet::basic::Compression
Dominik Moritz created ARROW-11874: -- Summary: [Rust] implement FromStr for parquet::basic::Compression Key: ARROW-11874 URL: https://issues.apache.org/jira/browse/ARROW-11874 Project: Apache Arrow Issue Type: Bug Reporter: Dominik Moritz It would be nice to be able to create a compression from a string. We already support compression to string so I think it's only natural to support the inverse as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11873) [Rust] Implement From for ParquetError
Dominik Moritz created ARROW-11873: -- Summary: [Rust] Implement From for ParquetError Key: ARROW-11873 URL: https://issues.apache.org/jira/browse/ARROW-11873 Project: Apache Arrow Issue Type: Bug Reporter: Dominik Moritz It would be great to be able to raise an arrow error and have it automatically be converted to a parquet error. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11827) [Rust] Support JSON array of objects
Dominik Moritz created ARROW-11827:
--
Summary: [Rust] Support JSON array of objects
Key: ARROW-11827
URL: https://issues.apache.org/jira/browse/ARROW-11827
Project: Apache Arrow
Issue Type: Bug
Reporter: Dominik Moritz
The JSON parser should also support arrays of object, which are a common format
for data in JSON (albeit not very efficient).
```json
[{
"a": 42,
"b": true
}, {
"a": 12,
"b": false
}, {
"a": 7,
"b": true
}]
```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-11823) [Rust] JSON reader does not implement iterator
Dominik Moritz created ARROW-11823: -- Summary: [Rust] JSON reader does not implement iterator Key: ARROW-11823 URL: https://issues.apache.org/jira/browse/ARROW-11823 Project: Apache Arrow Issue Type: Bug Components: Rust Reporter: Dominik Moritz https://docs.rs/arrow/3.0.0/arrow/json/reader/struct.Reader.html does not implement the iterator trait even though it implements next. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11696) [Rust] Create record batches from in memory IPC
Dominik Moritz created ARROW-11696: -- Summary: [Rust] Create record batches from in memory IPC Key: ARROW-11696 URL: https://issues.apache.org/jira/browse/ARROW-11696 Project: Apache Arrow Issue Type: Improvement Components: Rust Reporter: Dominik Moritz I have the IPC format of Arrow record batches in memory (e.g. as `&[u8]`) and would like to create a vector of batches while avoiding as many memory copies as possible. It would be great if there was a way to create the vector without having to go through the file abstraction. I might be misunderstanding the way the file reader works and maybe it does not incur memory copies. I think it does, though, since creating arrow record batches from a larger arrow buffer takes much longer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11615) DataFusion does not support wasm32-unknown-unknown target
Dominik Moritz created ARROW-11615: -- Summary: DataFusion does not support wasm32-unknown-unknown target Key: ARROW-11615 URL: https://issues.apache.org/jira/browse/ARROW-11615 Project: Apache Arrow Issue Type: Improvement Components: Rust - DataFusion Reporter: Dominik Moritz The Arrow crate successfully compiles to WebAssembly (e.g. https://github.com/domoritz/arrow-wasm) but the DataFusion crate currently does not support the`wasm32-unknown-unknown` target. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11593) Parquet does not support wasm32-unknown-unknown target
Dominik Moritz created ARROW-11593: -- Summary: Parquet does not support wasm32-unknown-unknown target Key: ARROW-11593 URL: https://issues.apache.org/jira/browse/ARROW-11593 Project: Apache Arrow Issue Type: Improvement Components: Rust Reporter: Dominik Moritz The Arrow crate successfully compiles to WebAssembly (e.g. https://github.com/domoritz/arrow-wasm) but the Parquet crate currently does not support the`wasm32-unknown-unknown` target. Try out the repository at https://github.com/domoritz/parquet-wasm/commit/e877f9ad9c45c09f73d98fab2a8ad384a802b2e0. The problem seems to be in liblz4, even if I do not include lz4 in the feature flags. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11592) Typo in comment
Dominik Moritz created ARROW-11592: -- Summary: Typo in comment Key: ARROW-11592 URL: https://issues.apache.org/jira/browse/ARROW-11592 Project: Apache Arrow Issue Type: Task Components: Rust Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11408) Add window support to datafusion readme
Dominik Moritz created ARROW-11408: -- Summary: Add window support to datafusion readme Key: ARROW-11408 URL: https://issues.apache.org/jira/browse/ARROW-11408 Project: Apache Arrow Issue Type: Task Components: Rust - DataFusion Reporter: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11389) Inconsistent comments for datatypes
Dominik Moritz created ARROW-11389: -- Summary: Inconsistent comments for datatypes Key: ARROW-11389 URL: https://issues.apache.org/jira/browse/ARROW-11389 Project: Apache Arrow Issue Type: Task Components: Rust Reporter: Dominik Moritz Assignee: Dominik Moritz Comments should end with . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11352) Implementation status?
Dominik Moritz created ARROW-11352: -- Summary: Implementation status? Key: ARROW-11352 URL: https://issues.apache.org/jira/browse/ARROW-11352 Project: Apache Arrow Issue Type: Task Components: Rust Reporter: Dominik Moritz https://arrow.apache.org/docs/status.html says that the Rust implementation doesn't support anything except CSV R/W. Is that true? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11351) Reconsider proxy objects instead of defineProperty
Dominik Moritz created ARROW-11351: -- Summary: Reconsider proxy objects instead of defineProperty Key: ARROW-11351 URL: https://issues.apache.org/jira/browse/ARROW-11351 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz Related to https://issues.apache.org/jira/browse/ARROW-11347 I was wondering why Arrow uses Proxy objects instead of defineProperty, which was a bit faster in the experiments at https://observablehq.com/@jheer/from-apache-arrow-to-javascript-objects. I don't know whether a change makes sense but I would love to know the design rationale since I couldn't find anything in the issues or on GitHub about it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-11347) Consider Objects instead of Maps
Dominik Moritz created ARROW-11347: -- Summary: Consider Objects instead of Maps Key: ARROW-11347 URL: https://issues.apache.org/jira/browse/ARROW-11347 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz A quick experiment (https://observablehq.com/@domoritz/performance-of-maps-vs-objects) seems to show that object accesses are a lot faster than map accesses. Would it make sense to switch to objects in the row API to improve performance? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9982) IterableArrayLike should support map
Dominik Moritz created ARROW-9982: - Summary: IterableArrayLike should support map Key: ARROW-9982 URL: https://issues.apache.org/jira/browse/ARROW-9982 Project: Apache Arrow Issue Type: Improvement Components: JavaScript Reporter: Dominik Moritz `table.toArray()` returns an `IterableArrayLike` and I would like to be able to `map` a function to it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9580) Docs have superfluous ()
Dominik Moritz created ARROW-9580: - Summary: Docs have superfluous () Key: ARROW-9580 URL: https://issues.apache.org/jira/browse/ARROW-9580 Project: Apache Arrow Issue Type: Task Components: JavaScript Reporter: Dominik Moritz Assignee: Dominik Moritz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-6282) Support lossy compression
[ https://issues.apache.org/jira/browse/ARROW-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909946#comment-16909946 ] Dominik Moritz commented on ARROW-6282: --- Thank you for the support and comments. I won't be implementing this but I wanted to propose this as a feature since it would be incredibly useful for visualization. > Support lossy compression > - > > Key: ARROW-6282 > URL: https://issues.apache.org/jira/browse/ARROW-6282 > Project: Apache Arrow > Issue Type: New Feature >Reporter: Dominik Moritz >Priority: Major > > Arrow dataframes with large columns of integers or floats can be compressed > using gzip or brotli. However, in some cases it will be okay to compress the > data lossy to achieve even higher compression ratios. The main use case for > this is visualization where small inaccuracies matter less. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (ARROW-4588) [JS] add logging
[ https://issues.apache.org/jira/browse/ARROW-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909640#comment-16909640 ] Dominik Moritz commented on ARROW-4588: --- I don't think we have logging set up yet. > [JS] add logging > > > Key: ARROW-4588 > URL: https://issues.apache.org/jira/browse/ARROW-4588 > Project: Apache Arrow > Issue Type: New Feature > Components: JavaScript >Reporter: Dominik Moritz >Priority: Major > > As discussed in https://github.com/apache/arrow/pull/3634, the javascript > library will need some logging infrastructure. The goals for this > implementation are a lightweight logger that can be easily configured to not > write to console. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (ARROW-6282) Support lossy compression
Dominik Moritz created ARROW-6282: - Summary: Support lossy compression Key: ARROW-6282 URL: https://issues.apache.org/jira/browse/ARROW-6282 Project: Apache Arrow Issue Type: New Feature Reporter: Dominik Moritz Arrow dataframes with large columns of integers or floats can be compressed using gzip or brotli. However, in some cases it will be okay to compress the data lossy to achieve even higher compression ratios. The main use case for this is visualization where small inaccuracies matter less. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (ARROW-4588) [JS] add logging
Dominik Moritz created ARROW-4588: - Summary: [JS] add logging Key: ARROW-4588 URL: https://issues.apache.org/jira/browse/ARROW-4588 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz As discussed in https://github.com/apache/arrow/pull/3634, the javascript library will need some logging infrastructure. The goals for this implementation are a lightweight logger that can be easily configured to not write to console. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4588) [JS] add logging
[ https://issues.apache.org/jira/browse/ARROW-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominik Moritz updated ARROW-4588: -- Issue Type: New Feature (was: Bug) > [JS] add logging > > > Key: ARROW-4588 > URL: https://issues.apache.org/jira/browse/ARROW-4588 > Project: Apache Arrow > Issue Type: New Feature > Components: JavaScript >Reporter: Dominik Moritz >Priority: Major > > As discussed in https://github.com/apache/arrow/pull/3634, the javascript > library will need some logging infrastructure. The goals for this > implementation are a lightweight logger that can be easily configured to not > write to console. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4550) [JS] Fix AMD pattern
Dominik Moritz created ARROW-4550: - Summary: [JS] Fix AMD pattern Key: ARROW-4550 URL: https://issues.apache.org/jira/browse/ARROW-4550 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Dominik Moritz -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ARROW-4521) Improve performance of row proxy object
[ https://issues.apache.org/jira/browse/ARROW-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominik Moritz closed ARROW-4521. - Resolution: Duplicate > Improve performance of row proxy object > --- > > Key: ARROW-4521 > URL: https://issues.apache.org/jira/browse/ARROW-4521 > Project: Apache Arrow > Issue Type: Improvement > Components: JavaScript >Reporter: Dominik Moritz >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > As noted in > https://github.com/vega/vega-loader-arrow/commit/19c88e130aaeeae9d0166360db467121e5724352#r32253784, > there may be some inefficiencies with the row proxy that could be mitigated > by defining properties on a prototype object. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
