[jira] [Created] (ARROW-7445) [GLib] Add HadoopFileSystem support

2019-12-19 Thread Kenta Murata (Jira)
Kenta Murata created ARROW-7445: --- Summary: [GLib] Add HadoopFileSystem support Key: ARROW-7445 URL: https://issues.apache.org/jira/browse/ARROW-7445 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-7444) [GLib] Add LocalFileSystem support

2019-12-19 Thread Kenta Murata (Jira)
Kenta Murata created ARROW-7444: --- Summary: [GLib] Add LocalFileSystem support Key: ARROW-7444 URL: https://issues.apache.org/jira/browse/ARROW-7444 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-7443) [GLib] Add binding of arrow::fs

2019-12-19 Thread Kenta Murata (Jira)
Kenta Murata created ARROW-7443: --- Summary: [GLib] Add binding of arrow::fs Key: ARROW-7443 URL: https://issues.apache.org/jira/browse/ARROW-7443 Project: Apache Arrow Issue Type: New Feature

[C++][Compute] RFC: add SIMD support to C++ kernel

2019-12-19 Thread Yibo Cai
Hi, I'm investigating SIMD support to C++ compute kernel(not gandiva). A typical case is the sum kernel[1]. Below tight loop can be easily optimized with SIMD. for (int64_t i = 0; i < length; i++) { local.sum += values[i]; } Compiler already does loop vectorization. But it's done at

[jira] [Created] (ARROW-7442) [Ruby]Specifying column type as time causes segmentation fault

2019-12-19 Thread Kitaiti Makoto (Jira)
Kitaiti Makoto created ARROW-7442: - Summary: [Ruby]Specifying column type as time causes segmentation fault Key: ARROW-7442 URL: https://issues.apache.org/jira/browse/ARROW-7442 Project: Apache Arrow

Re: Trouble with building Arrow GLib

2019-12-19 Thread Sutou Kouhei
Hi, Arrow GLib can provide C API but it requires GObject Introspection for now. So you need to install GObject Introspection to use Arrow GLib for now. (I'll add a build option to build without GObject Introspection. Then you can build Arrow GLib without GObject Introspection in the next

Re: [C++] Validate UTF-8 in ValidateFull?

2019-12-19 Thread Wes McKinney
On Thu, Dec 19, 2019 at 1:11 PM Antoine Pitrou wrote: > > > > Would the Arrow team welcome a pull request that enhances > ValidateFull() to > > validate that utf8-column values are well-formed UTF-8 byte sequences? > > We already have a UTF-8 validation function, but it's not hooked into >

Re: [C++] Validate UTF-8 in ValidateFull?

2019-12-19 Thread Antoine Pitrou
> Would the Arrow team welcome a pull request that enhances ValidateFull() to > validate that utf8-column values are well-formed UTF-8 byte sequences? We already have a UTF-8 validation function, but it's not hooked into ValidateFull(). So, yes, that seems desirable to me. Can you open a JIRA

Re: [DISCUSS][C++] Pointer name aliasing

2019-12-19 Thread Francois Saint-Jacques
I created the following ticket (and sub-tasks) [1] to track François [1] https://jira.apache.org/jira/browse/ARROW-7438 On Tue, Nov 26, 2019 at 12:09 AM Micah Kornfield wrote: > > I would need to look at the other instances as well. I will try to so by > next week, but I think we can

[jira] [Created] (ARROW-7441) [C++] Remove compute pointer aliases

2019-12-19 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7441: - Summary: [C++] Remove compute pointer aliases Key: ARROW-7441 URL: https://issues.apache.org/jira/browse/ARROW-7441 Project: Apache Arrow

[jira] [Created] (ARROW-7439) [C++][Dataset] Remove dataset pointer aliases

2019-12-19 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7439: - Summary: [C++][Dataset] Remove dataset pointer aliases Key: ARROW-7439 URL: https://issues.apache.org/jira/browse/ARROW-7439 Project: Apache Arrow

[jira] [Created] (ARROW-7440) [C++][Gandiva] Remove gandiva pointer aliases

2019-12-19 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7440: - Summary: [C++][Gandiva] Remove gandiva pointer aliases Key: ARROW-7440 URL: https://issues.apache.org/jira/browse/ARROW-7440 Project: Apache Arrow

[jira] [Created] (ARROW-7438) [C++] Remove pointer aliases

2019-12-19 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7438: - Summary: [C++] Remove pointer aliases Key: ARROW-7438 URL: https://issues.apache.org/jira/browse/ARROW-7438 Project: Apache Arrow Issue

Re: Where can I report a security-related issue?

2019-12-19 Thread Adam Hooper
On Thu, Dec 19, 2019 at 8:43 AM Antoine Pitrou wrote: > > Security is a concern for us and we welcome any reports. That said, > Arrow is primarily concerned about efficiency (memory and processing > efficiency), which implies that implicit checks cannot often be > comprehensive. > Thank you

Re: Non-chunked large files / hdf5 support

2019-12-19 Thread Wes McKinney
On Tue, Dec 17, 2019 at 5:15 AM Maarten Breddels wrote: > > Hi, > > I had to catch up a bit with the arrow documentation before I could respond > properly. My fear was that Arrow demanded that the in-memory representation > was always 'packed', or 'flat'. After going through the docs, it seems

Re: Where can I report a security-related issue?

2019-12-19 Thread Antoine Pitrou
Hi Adam, Le 18/12/2019 à 20:00, Adam Hooper a écrit : > Thank you for the responses, Wes and Antoine. I've sent to private@ and > then filed https://issues.apache.org/jira/browse/ARROW-7435 > > @Antoine I'm glad to hear the IPC layer is a security boundary. Could you > please clarify what that

[NIGHTLY] Arrow Build Report for Job nightly-2019-12-19-0

2019-12-19 Thread Crossbow
Arrow Build Report for Job nightly-2019-12-19-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-12-19-0 Failed Tasks: - conda-osx-clang-py27: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-12-19-0-azure-conda-osx-clang-py27 -

Trouble with building Arrow GLib

2019-12-19 Thread Chengxin Ma
Hi All, I am building Arrow GLib on a system where I'm not the admin. In the installation instructions (https://github.com/apache/arrow/tree/master/c_glib) there are two options for building GLib: "How to build by users" and "How to build by developers". I followed the "by users" section and

[jira] [Created] (ARROW-7437) [Java] ReadChannel#readFully does not set writer index correctly

2019-12-19 Thread Liya Fan (Jira)
Liya Fan created ARROW-7437: --- Summary: [Java] ReadChannel#readFully does not set writer index correctly Key: ARROW-7437 URL: https://issues.apache.org/jira/browse/ARROW-7437 Project: Apache Arrow