[jira] [Created] (ARROW-6931) [Java] Consider starting to use Google Truth Fluent Assertions library

[Micah Kornfield (Jira)]

Micah Kornfield created ARROW-6931:
--

 Summary: [Java] Consider starting to use Google Truth Fluent 
Assertions library
 Key: ARROW-6931
 URL: https://issues.apache.org/jira/browse/ARROW-6931
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Java
Reporter: Micah Kornfield


This can offer more readable asserts than the limited JUnit assertions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ARROW-6930) [Java] Create static factory methods for common array types of testing.

[Micah Kornfield (Jira)]

Micah Kornfield created ARROW-6930:
--

 Summary: [Java] Create static factory methods for common array 
types of testing.
 Key: ARROW-6930
 URL: https://issues.apache.org/jira/browse/ARROW-6930
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Java
Reporter: Micah Kornfield


There is a lot of verbosity in the construction of Arrays for testing purposes 
(multiple lines of setSafe(...) or set(...).  We should start adding some 
static factory methods to make test setup clearer  and more concise.  A 
strawman proposal for BigIntVector might look like:

 

static BigIntVector create(String name, BufferAllocator allocator, Long... 
values).

 

Usage would be something like:

 

try (BigIntVector input = BigIntVectorCreate("sample_data", allocator, 1235L, 
null, 456L),

      BigIntVector expected = BigIntVectorCreate("sample_data", allocator, 1L, 
null, 0L),) {

   output = doSomethingWith(input);

   assertThat(output).isEqualTo(expected);

}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ARROW-6929) [C++] ValidateArray is out of sync with the ListArray IPC specification

[Micah Kornfield (Jira)]

Micah Kornfield created ARROW-6929:
--

 Summary: [C++] ValidateArray is out of sync with the ListArray IPC 
specification
 Key: ARROW-6929
 URL: https://issues.apache.org/jira/browse/ARROW-6929
 Project: Apache Arrow
  Issue Type: Bug
  Components: C++
Reporter: Micah Kornfield


* It appears to check that null values take zero space
 * It still checks for a begin offset of 0 if the array isn't sliced 
(technically this doesn't seem necessary and it could be non-zero even if the 
array wasn't sliced.)
 * I think it also fails if an array is sliced to truncate it since it should 
compare length to data_extent instead of last_offset.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

pyarrow and pyzmq no copy

[seshu yamajala]

I would like to use pyarrow with pyzmq no copy to send dicts of arrays
across the network without having to make copies of the arrays stored
in the dicts.

I've come up with an example, here: 
https://gist.github.com/syamajala/51d52f5e326ff719bf6231546091991d

However, I'm having trouble with deserializing and am seeing this:

Traceback (most recent call last):
  File "zmq_pyarrow.py", line 99, in 
col.run_collector()
  File "zmq_pyarrow.py", line 74, in run_collector
msg = pa.deserialize_components(comp)
  File "pyarrow/serialization.pxi", line 450, in
pyarrow.lib.deserialize_components
  File "pyarrow/serialization.pxi", line 307, in
pyarrow.lib.SerializedPyObject.from_components
  File "pyarrow/error.pxi", line 78, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Could not unwrap Buffer from the passed
Python object.

Does anyone have any tips?

Re: [ANNOUNCE] New Arrow committer: Eric Erhardt

[Fan Liya]

Congrats Eric!

Best,
Liya Fan

On Fri, Oct 18, 2019 at 3:06 AM paddy horan  wrote:

> Congrats Eric!
>
> 
> From: Micah Kornfield 
> Sent: Thursday, October 17, 2019 12:45:15 PM
> To: dev 
> Subject: Re: [ANNOUNCE] New Arrow committer: Eric Erhardt
>
> Congrats Eric!
>
> On Thu, Oct 17, 2019 at 6:58 AM Wes McKinney  wrote:
>
> > On behalf of the Arrow PMC, I'm happy to announce that Eric has
> > accepted an invitation to become a committer on Apache Arrow.
> >
> > Welcome, and thank you for your contributions!
> >
>

[jira] [Created] (ARROW-6928) [Rust] Add FixedSizeList type

[Neville Dipale (Jira)]

Neville Dipale created ARROW-6928:
-

 Summary: [Rust] Add FixedSizeList type
 Key: ARROW-6928
 URL: https://issues.apache.org/jira/browse/ARROW-6928
 Project: Apache Arrow
  Issue Type: Sub-task
  Components: Rust
Reporter: Neville Dipale


Support FixedSizeList, which is required for integration testing



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] [Rust] Adding support for Flight protocol

[Neville Dipale]

Thanks Andy,

Please see https://github.com/apache/arrow/pull/4167#issuecomment-543381089
for the status of the PR. We have a few missing data types (fixed list,
timezone to timestamp, etc.) that are currently stopping me from testing
the reading of files.

I'm trying out creating a fixed size list, and I'll open a PR for that if
my attempt works.

On Fri, 18 Oct 2019 at 01:10, Andy Grove  wrote:

> Thanks for all the updates. I'd like to get involved and help out with this
> effort as well. I don't have any major work planned for DataFusion for
> 1.0.0 now other than maybe moving to the new parquet ArrowReader, if it is
> ready in time.
>
> I have been chatting with the author of the Rust Flatbuffer project about
> some of the issues and I can take an action to follow up with that.
>
> I have also been talking with one of the authors of Tonic, and I believe
> they might be interested in helping here too.
>
> Let me know how else I can help out with this effort.
>
> Andy.
>
>
>
> On Thu, Oct 17, 2019 at 2:42 PM Neville Dipale 
> wrote:
>
> > Good evening
> >
> > With support for testing against integration files now done, I've resumed
> > work on the IPC reader. If I don't encounter trouble reading the existing
> > files, I expect to be done with this work by the end of the weekend. I
> had
> > taken the approach of one large PR to include all Rust-supported Arrow
> > types.
> >
> > I'm not sure of how long the writer would take, but I remain committed to
> > having this work completed by 1.0.
> >
> > I have to catch up on null-type roundtrip and the padding alignment work
> as
> > I haven't been able to keep abreast with development these last few
> months.
> > Also, the Rust flatbuffer issues that we've had haven't progressed in the
> > relevant repo, so it still makes ergonomics not great, but at least our
> > users don't have to worry about that.
> >
> > @Andy I have good experience with gRPC in Rust, and also want to see
> Flight
> > support landing soon.
> >
> > On Thu, 17 Oct 2019, 17:06 David Li,  wrote:
> >
> > > Just for reference, it's possible once the basic IPC support is
> merged; I
> > > had a proof of concept, though it needs to be updated to use Tonic over
> > > tower-grpc, actually implement the zero-copy optimizations, provide a
> > real
> > > API, etc.
> > >
> > > https://github.com/apache/arrow/pull/4167#issuecomment-529695811
> > >
> > > On Thu, Oct 17, 2019, 10:51 Wes McKinney  wrote:
> > >
> > > > I hope to see Flight in all the reference implementations eventually.
> > > >
> > > > Having hardened IPC support is a pre-requisite, it would be ideal to
> > > > have Rust as a participant in the integration tests
> > > >
> > > > On Thu, Oct 17, 2019 at 9:41 AM Andy Grove 
> > > wrote:
> > > > >
> > > > > I was approached directly about adding Flight support to the Rust
> > > > > implementation, and said I would start a discussion here on the
> > mailing
> > > > > list.
> > > > >
> > > > > There is ongoing work with IPC and integration and I believe that
> it
> > > > would
> > > > > make sense to start looking at adding Flight support.
> > > > >
> > > > > I'd like to hear what others think though.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Andy.
> > > >
> > >
> >
>

Re: [DISCUSS] [Rust] Adding support for Flight protocol

[Andy Grove]

Thanks for all the updates. I'd like to get involved and help out with this
effort as well. I don't have any major work planned for DataFusion for
1.0.0 now other than maybe moving to the new parquet ArrowReader, if it is
ready in time.

I have been chatting with the author of the Rust Flatbuffer project about
some of the issues and I can take an action to follow up with that.

I have also been talking with one of the authors of Tonic, and I believe
they might be interested in helping here too.

Let me know how else I can help out with this effort.

Andy.



On Thu, Oct 17, 2019 at 2:42 PM Neville Dipale 
wrote:

> Good evening
>
> With support for testing against integration files now done, I've resumed
> work on the IPC reader. If I don't encounter trouble reading the existing
> files, I expect to be done with this work by the end of the weekend. I had
> taken the approach of one large PR to include all Rust-supported Arrow
> types.
>
> I'm not sure of how long the writer would take, but I remain committed to
> having this work completed by 1.0.
>
> I have to catch up on null-type roundtrip and the padding alignment work as
> I haven't been able to keep abreast with development these last few months.
> Also, the Rust flatbuffer issues that we've had haven't progressed in the
> relevant repo, so it still makes ergonomics not great, but at least our
> users don't have to worry about that.
>
> @Andy I have good experience with gRPC in Rust, and also want to see Flight
> support landing soon.
>
> On Thu, 17 Oct 2019, 17:06 David Li,  wrote:
>
> > Just for reference, it's possible once the basic IPC support is merged; I
> > had a proof of concept, though it needs to be updated to use Tonic over
> > tower-grpc, actually implement the zero-copy optimizations, provide a
> real
> > API, etc.
> >
> > https://github.com/apache/arrow/pull/4167#issuecomment-529695811
> >
> > On Thu, Oct 17, 2019, 10:51 Wes McKinney  wrote:
> >
> > > I hope to see Flight in all the reference implementations eventually.
> > >
> > > Having hardened IPC support is a pre-requisite, it would be ideal to
> > > have Rust as a participant in the integration tests
> > >
> > > On Thu, Oct 17, 2019 at 9:41 AM Andy Grove 
> > wrote:
> > > >
> > > > I was approached directly about adding Flight support to the Rust
> > > > implementation, and said I would start a discussion here on the
> mailing
> > > > list.
> > > >
> > > > There is ongoing work with IPC and integration and I believe that it
> > > would
> > > > make sense to start looking at adding Flight support.
> > > >
> > > > I'd like to hear what others think though.
> > > >
> > > > Thanks,
> > > >
> > > > Andy.
> > >
> >
>

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-10-17-0

[Sutou Kouhei]

https://github.com/apache/arrow/pull/5689 will fix them:

> - debian-stretch:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-stretch
> - debian-buster:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-buster
> - ubuntu-disco:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-ubuntu-disco

In <5da86698.1c69fb81.db73f.2...@mx.google.com>
  "[NIGHTLY] Arrow Build Report for Job nightly-2019-10-17-0" on Thu, 17 Oct 
2019 06:03:20 -0700 (PDT),
  Crossbow  wrote:

> 
> Arrow Build Report for Job nightly-2019-10-17-0
> 
> All tasks: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0
> 
> Failed Tasks:
> - wheel-osx-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp36m
> - debian-stretch:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-stretch
> - wheel-osx-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp37m
> - wheel-osx-cp27m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp27m
> - wheel-osx-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp35m
> - docker-pandas-master:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-pandas-master
> - conda-linux-gcc-py37:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py37
> - docker-c_glib:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-c_glib
> - wheel-manylinux2010-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp37m
> - wheel-win-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-appveyor-wheel-win-cp37m
> - docker-clang-format:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-clang-format
> - wheel-win-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-appveyor-wheel-win-cp36m
> - gandiva-jar-osx:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-gandiva-jar-osx
> - debian-buster:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-buster
> - ubuntu-disco:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-ubuntu-disco
> - conda-linux-gcc-py36:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py36
> - conda-linux-gcc-py27:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py27
> - gandiva-jar-trusty:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-gandiva-jar-trusty
> 
> Succeeded Tasks:
> - docker-docs:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-docs
> - conda-win-vs2015-py37:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-win-vs2015-py37
> - wheel-manylinux1-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp37m
> - wheel-manylinux1-cp27mu:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp27mu
> - wheel-manylinux2010-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp36m
> - docker-turbodbc-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-turbodbc-integration
> - docker-cpp:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-cpp
> - docker-java:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-java
> - wheel-manylinux2010-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp35m
> - ubuntu-xenial:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-ubuntu-xenial
> - wheel-manylinux1-cp27m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp27m
> - docker-hdfs-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-hdfs-integration
> - 

[jira] [Created] (ARROW-6927) [C++] Add gRPC version check

[Kouhei Sutou (Jira)]

Kouhei Sutou created ARROW-6927:
---

 Summary: [C++] Add gRPC version check
 Key: ARROW-6927
 URL: https://issues.apache.org/jira/browse/ARROW-6927
 Project: Apache Arrow
  Issue Type: Improvement
  Components: C++
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou


We need gRPC++ 1.17.0 or later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ARROW-6925) Arrow fails to buld on MacOS 10.13.6 using brew gcc 7 and 8

[John Norris (Jira)]

John Norris created ARROW-6925:
--

 Summary: Arrow fails to buld on MacOS 10.13.6 using brew gcc 7 and 
8
 Key: ARROW-6925
 URL: https://issues.apache.org/jira/browse/ARROW-6925
 Project: Apache Arrow
  Issue Type: Bug
 Environment: MacOS 10.13.6 using both brew gcc 7 and 8.
Reporter: John Norris


Both SetupCxxFlags.cmake and ThirdpartyToolchain.cmake add -stdlib=libc++ to 
the compiler flags when APPLE is true, but if you're using GCC from brew (or 
presumably from anywhere other that Apple), this flag is not recognized and the 
build fails.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

C data interface: draft C++ implementation and Python <-> R bridge

[Antoine Pitrou]

Hi,

For the record, I've been working on a C++ implementation of exporting 
and importing data using the C data interface:

https://github.com/apache/arrow/pull/5608

(some datatypes are not handled yet, and metadata is currently not 
implemented)


Also Neal has used that PR to create a proof-of-concept of passing Arrow 
data between Python and R using zero-copy and between potentially 
different Arrow C++ runtimes:

https://github.com/pitrou/arrow/pull/5

Regards

Antoine.

Re: [ANNOUNCE] New Arrow committer: Eric Erhardt

[paddy horan]

Congrats Eric!


From: Micah Kornfield 
Sent: Thursday, October 17, 2019 12:45:15 PM
To: dev 
Subject: Re: [ANNOUNCE] New Arrow committer: Eric Erhardt

Congrats Eric!

On Thu, Oct 17, 2019 at 6:58 AM Wes McKinney  wrote:

> On behalf of the Arrow PMC, I'm happy to announce that Eric has
> accepted an invitation to become a committer on Apache Arrow.
>
> Welcome, and thank you for your contributions!
>

[jira] [Created] (ARROW-6923) [C++] Option for Filter kernel how to handle nulls in the selection vector

[Joris Van den Bossche (Jira)]

Joris Van den Bossche created ARROW-6923:


 Summary: [C++] Option for Filter kernel how to handle nulls in the 
selection vector
 Key: ARROW-6923
 URL: https://issues.apache.org/jira/browse/ARROW-6923
 Project: Apache Arrow
  Issue Type: Improvement
  Components: C++
Reporter: Joris Van den Bossche


How nulls are handled in the boolean mask (selection vector) in a filter kernel 
varies between languages / data analytics systems (e.g. base R propagates 
nulls, dplyr R skips (sees as False), SQL generally skips them as well I think, 
Julia raises an error).

Currently, in Arrow C++ we "propagate" nulls (null in the selection vector 
gives a null in the output):

{code}
In [7]: arr = pa.array([1, 2, 3]) 

In [8]: mask = pa.array([True, False, None]) 

In [9]: arr.filter(mask) 
Out[9]: 

[
  1,
  null
]
{code}

Given the different ways this could be done (propagate, skip, error), should we 
provide an option to control this behaviour?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ARROW-6922) [Python] Pandas master build is failing (MultiIndex.levels change)

[Joris Van den Bossche (Jira)]

Joris Van den Bossche created ARROW-6922:


 Summary: [Python] Pandas master build is failing 
(MultiIndex.levels change)
 Key: ARROW-6922
 URL: https://issues.apache.org/jira/browse/ARROW-6922
 Project: Apache Arrow
  Issue Type: Bug
  Components: Python
Reporter: Joris Van den Bossche
 Fix For: 0.15.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ARROW-6921) Regression: Cannot round-trip IPC files between PyArrow and Arrow JS

[Joe Quigley (Jira)]

Joe Quigley created ARROW-6921:
--

 Summary:  Regression: Cannot round-trip IPC files between PyArrow 
and Arrow JS
 Key: ARROW-6921
 URL: https://issues.apache.org/jira/browse/ARROW-6921
 Project: Apache Arrow
  Issue Type: Bug
  Components: JavaScript, Python
Affects Versions: 0.15.0, 0.15.1
 Environment: Broken PyArrow: conda-forge build 0.15.0-py37h8b68381_0
Working PyArrow: conda-forge build 0.14.1-py37h8b68381_2
Arrow JS versions tested: 0.13.0, 0.14.0 (apache-arrow@latest)
OS: Tested on Win7, Amazon Linux AMI 2018.03
Python platforms: Win7/Conda/Python 3.6, RHEL/Conda/Python 3.6
JS platforms: Node 10.15.0, Chrome 77.0.3865.120
Reporter: Joe Quigley


Originally raised by [Sarath|https://stackoverflow.com/users/403133/sarath] on 
[StackOverflow|https://stackoverflow.com/questions/58313254], reporting here as 
I've run into this issue as well.

When exporting an Arrow table using PyArrow, ArrowJS incorrectly imports it as 
a 0-row table, skipping any data in the table. The schema is imported 
correctly, including metadata, but the length of the table is 0.

{code:python}
import pyarrow as pa
table = pa.Table.from_pydict( {"a": [1, 2, 3], "b": [4, 5, 6]} )

with pa.RecordBatchFileWriter('file.arrow', table.schema) as writer:
writer.write_table(table)
{code}

If {{file.arrow}} was generated with PyArrow 0.15, the following JS snippet 
will fail. However, if you generated it with PyArrow 0.14, then the JS snippet 
will work as expected:

{code:javascript}
const { readFileSync } = require("fs");
const { Table } = require("apache-arrow");

const data = readFileSync("file.arrow");
const table = Table.from([ data ]);

console.assert(table.length === 3, "Table should have 3 rows");
console.assert(table.get(0) != null, "First row should not be null");
{code}

Tested with PyArrow 0.14.1, 0.15.0, and ArrowJS 0.13.0 and 0.14.1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New Arrow committer: Eric Erhardt

[Micah Kornfield]

Congrats Eric!

On Thu, Oct 17, 2019 at 6:58 AM Wes McKinney  wrote:

> On behalf of the Arrow PMC, I'm happy to announce that Eric has
> accepted an invitation to become a committer on Apache Arrow.
>
> Welcome, and thank you for your contributions!
>

[jira] [Created] (ARROW-6920) [python] create manylinux wheels for python3.8

[Simon Hewitt (Jira)]

Simon Hewitt created ARROW-6920:
---

 Summary: [python] create manylinux wheels for python3.8
 Key: ARROW-6920
 URL: https://issues.apache.org/jira/browse/ARROW-6920
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Python
Reporter: Simon Hewitt


There are currently no wheels available in pypi for python3.8. This means it's 
not possible to install later versions of arrow from pypi as the latest 
uploaded source release is 0.14.0.

It would also be useful to upload source releases to pypi...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ARROW-6919) [Python] Expose more builders in Cython

[Uwe Korn (Jira)]

Uwe Korn created ARROW-6919:
---

 Summary: [Python] Expose more builders in Cython
 Key: ARROW-6919
 URL: https://issues.apache.org/jira/browse/ARROW-6919
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Python
Reporter: Uwe Korn
Assignee: Uwe Korn






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] [Rust] Adding support for Flight protocol

[David Li]

Just for reference, it's possible once the basic IPC support is merged; I
had a proof of concept, though it needs to be updated to use Tonic over
tower-grpc, actually implement the zero-copy optimizations, provide a real
API, etc.

https://github.com/apache/arrow/pull/4167#issuecomment-529695811

On Thu, Oct 17, 2019, 10:51 Wes McKinney  wrote:

> I hope to see Flight in all the reference implementations eventually.
>
> Having hardened IPC support is a pre-requisite, it would be ideal to
> have Rust as a participant in the integration tests
>
> On Thu, Oct 17, 2019 at 9:41 AM Andy Grove  wrote:
> >
> > I was approached directly about adding Flight support to the Rust
> > implementation, and said I would start a discussion here on the mailing
> > list.
> >
> > There is ongoing work with IPC and integration and I believe that it
> would
> > make sense to start looking at adding Flight support.
> >
> > I'd like to hear what others think though.
> >
> > Thanks,
> >
> > Andy.
>

Re: [DISCUSS] [Rust] Adding support for Flight protocol

[Wes McKinney]

I hope to see Flight in all the reference implementations eventually.

Having hardened IPC support is a pre-requisite, it would be ideal to
have Rust as a participant in the integration tests

On Thu, Oct 17, 2019 at 9:41 AM Andy Grove  wrote:
>
> I was approached directly about adding Flight support to the Rust
> implementation, and said I would start a discussion here on the mailing
> list.
>
> There is ongoing work with IPC and integration and I believe that it would
> make sense to start looking at adding Flight support.
>
> I'd like to hear what others think though.
>
> Thanks,
>
> Andy.

Re: [Discuss] Streaming: Differentiate between length of RecordBatch and utilized portion-- common use-case?

[John Muehlhausen]

Micah, thanks very much for your input.  A few thoughts in response:

``Over the time horizon of desired latency if you aren't receiving enough
messages to take advantage of columnar analytics, a system probably has
enough time to compact batches after the fact for later analysis and
conversely if you are receiving many events you naturally get reasonable
batch sizes without having to do further work.''

To see my perspective it is necessary to stop thinking about business
analytics and latencies of minutes or seconds or significant fractions of
seconds.  In real-time financial trading systems we do things like kernel
bypass networking, core affinitization and spinning, fake workloads to keep
hot paths cached, almost complete avoidance of general purpose allocators
(preferring typed pools), all with the goal of shaving off one more
MICROsecond.

Therefore "probably has enough time" is never in our vocabulary.
Microbatching is not a thing.  It is either purely event driven or bust if
you are shooting for a dozen microseconds from packet-in to packet-out on
the wire.  This is also the reason that certain high-performance systems
will never use the reference implementations of Arrow, just as most other
financial tech standards are centered on the *protocol* and there end up
being dozens of implementations that compete on performance.

Arrow *as a protocol* has the potential to be a beautiful thing.  It is
simple and straightforward and brings needed standardization.  I would
encourage thinking of the reference implementations almost as second-class
citizens.  The reference implementations should not de-facto define the
protocol, but vice versa. (I know this is probably already the philosophy.)

Of course data eventually pops out of high performance systems and has a
second life in the hands of researchers, etc.  This is why I'd like the
"vanilla" Arrow to be able to deal with Arrow data as-constructed in the
higher performance systems.  Could there be special processes to refactor
all this data?  Of course, but why?  If I'm logging RecordBatches to disk
and some of them happen to have extra array elements (e.g. because of the
inevitable imperfect projection pre-allocation-- running out of variable
length string storage before running out of rows), why would I refactor
terabytes of data (and, more importantly, introduce a time window where
data is not available while this occurs) when I can just have pyarrow skip
the unused rows?  If I want compression/de-duplication I'll do it at the
storage media layer.

``the proposal is essentially changing the unit of exchange from
RecordBatches to a segment of a RecordBatch''

It is the recognition that a RecordBatch may need to be computationally
useful before it is completely received/constructed, and without adding
additional overhead to the reception/construction process.  I grab a canned
RecordBatch from a pool of them and start filling it in.  If an earlier
RecordBatch falls out of the computation window, I put it back in the
pool.  At any moment of time the batch advertises the truth:
RecordBatch.length is the immutable section of data at this moment in
time.  That section will not have changed in future moments.  It is also a
recognition that some RecordBatches so-constructed are not completely
fillable.  They are like a moving truck with a little empty space left.
Sure, we could custom-build trucks to exactly fit the cargo, but why?  It
takes too much time.  Grab a truck (or a RecordBatch) off the lot (pool)
and go...  when you've filled it as much as possible, but not perfectly,
grab another one that is exactly the same.

I think my perception of the situation is clearest if we think about
"frozen" or "sealed" RecordBatches that, during their construction stage,
use a one-size-fits-all set of arrays and variable length buffer storage.
I grab a RecordBatch off the lot that is "for two int columns and a
variable length string column where the average expected length of a string
is 10."  I fill it, then I'm done.  If my strings were slightly longer than
expected I have extra array elements and RecordBatch.length is less than
array length.

While it is true that I have other use-cases in mind regarding simultaneous
collection and computation, I'm hoping that the moving truck analogy by
itself demonstrates enough efficiency advantages (as compared to typical
batch construction) to warrant this change.  Put simply, it is Arrow
accommodating more space/time tradeoff options than it currently does.

When I was baking plasma into a system for the first time, I ran into the
example that I create a record batch to a mock stream in order to know how
much plasma memory to allocate.  That is the kind of "build the moving
truck to fit the cargo" that I just can't have!

-John

On Wed, Oct 16, 2019 at 10:15 PM Micah Kornfield 
wrote:

> Hi John and Wes,
>
> A few thoughts:
> One of the issues which we didn't get into in prior discussions, is the
> proposal is essentially 

[jira] [Created] (ARROW-6916) [Developer] Alphabetize task names in nightly Crossbow report

[Wes McKinney (Jira)]

Wes McKinney created ARROW-6916:
---

 Summary: [Developer] Alphabetize task names in nightly Crossbow 
report
 Key: ARROW-6916
 URL: https://issues.apache.org/jira/browse/ARROW-6916
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Developer Tools
Reporter: Wes McKinney
 Fix For: 1.0.0


It's a nuisance to have related tasks appearing in different places in the list 
when they could appear next to each other if places in alphabetical order



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-10-17-0

[Wes McKinney]

Holy moly lots of failures!

- docker-clang-format
https://issues.apache.org/jira/browse/ARROW-6914

- wheel-osx-cp36m:
- wheel-osx-cp37m:
- wheel-osx-cp27m:
- wheel-osx-cp35m:
- gandiva-jar-osx:
  - All seem to be caused by the Homebrew issues?

I'm looking at the others and will create JIRA issues as appropriate

On Thu, Oct 17, 2019 at 8:03 AM Crossbow  wrote:
>
>
> Arrow Build Report for Job nightly-2019-10-17-0
>
> All tasks: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0
>
> Failed Tasks:
> - wheel-osx-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp36m
> - debian-stretch:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-stretch
> - wheel-osx-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp37m
> - wheel-osx-cp27m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp27m
> - wheel-osx-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp35m
> - docker-pandas-master:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-pandas-master
> - conda-linux-gcc-py37:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py37
> - docker-c_glib:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-c_glib
> - wheel-manylinux2010-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp37m
> - wheel-win-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-appveyor-wheel-win-cp37m
> - docker-clang-format:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-clang-format
> - wheel-win-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-appveyor-wheel-win-cp36m
> - gandiva-jar-osx:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-gandiva-jar-osx
> - debian-buster:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-buster
> - ubuntu-disco:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-ubuntu-disco
> - conda-linux-gcc-py36:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py36
> - conda-linux-gcc-py27:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py27
> - gandiva-jar-trusty:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-gandiva-jar-trusty
>
> Succeeded Tasks:
> - docker-docs:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-docs
> - conda-win-vs2015-py37:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-win-vs2015-py37
> - wheel-manylinux1-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp37m
> - wheel-manylinux1-cp27mu:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp27mu
> - wheel-manylinux2010-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp36m
> - docker-turbodbc-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-turbodbc-integration
> - docker-cpp:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-cpp
> - docker-java:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-java
> - wheel-manylinux2010-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp35m
> - ubuntu-xenial:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-ubuntu-xenial
> - wheel-manylinux1-cp27m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp27m
> - docker-hdfs-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-hdfs-integration
> - wheel-manylinux1-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp35m
> - conda-osx-clang-py37:
>   URL: 
> 

[jira] [Created] (ARROW-6915) [Developer] Do not overwrite minor release version with merge script, even if not specified by committer

[Wes McKinney (Jira)]

Wes McKinney created ARROW-6915:
---

 Summary: [Developer] Do not overwrite minor release version with 
merge script, even if not specified by committer
 Key: ARROW-6915
 URL: https://issues.apache.org/jira/browse/ARROW-6915
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Developer Tools
Reporter: Wes McKinney
 Fix For: 1.0.0


Not every committer knows to write "$MAJOR_VERSION,$MINOR_VERSION" for the fix 
version when merging



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ARROW-6914) [CI] docker-clang-format nightly failing

[Wes McKinney (Jira)]

Wes McKinney created ARROW-6914:
---

 Summary: [CI] docker-clang-format nightly failing
 Key: ARROW-6914
 URL: https://issues.apache.org/jira/browse/ARROW-6914
 Project: Apache Arrow
  Issue Type: Bug
  Components: Continuous Integration
Reporter: Wes McKinney
 Fix For: 1.0.0


Don't know what happened but this needs to either be fixed or removed from the 
nightlies

{code}
Successfully built 86ece22bd823
Successfully tagged arrowdev/arrow-lint:latest
WARNING: The CI_ARROW_SHA variable is not set. Defaulting to a blank string.
WARNING: The CI_ARROW_BRANCH variable is not set. Defaulting to a blank string.
ERROR: No such service: clang-format
Exited with code 1
{code}

https://circleci.com/gh/ursa-labs/crossbow/3923?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[ANNOUNCE] New Arrow committer: Eric Erhardt

[Wes McKinney]

On behalf of the Arrow PMC, I'm happy to announce that Eric has
accepted an invitation to become a committer on Apache Arrow.

Welcome, and thank you for your contributions!

Re: Possible Arrow 0.15.1 release

[Wes McKinney]

Thanks. Yes, you have to type

1.0.0,0.15.1

when merging to set both fix versions

On Thu, Oct 17, 2019 at 8:52 AM Antoine Pitrou  wrote:
>
>
> I added 0.15.1 back to a couple of fixed issues where the merge script
> removed it :-/
>
>
>
> Le 17/10/2019 à 09:30, Wes McKinney a écrit :
> > Nearly all the fixes are in for 0.15.1.
> >
> > I think the only thing preventing us from making an RC soon is
> > (surprise) new problems with packaging (Homebrew-related, and others).
> >
> > Anything else that needs to go into 0.15.1 that is not marked as such?
> >
> > On Sat, Oct 12, 2019 at 1:52 AM Fan Liya  wrote:
> >>
> >> I have update ARROW-6738 
> >> to add 0.15.1.
> >> I hope this bug can be fixed in the next release.
> >>
> >> Best,
> >> Liya Fan
> >>
> >> On Sat, Oct 12, 2019 at 12:38 PM Micah Kornfield 
> >> wrote:
> >>
> >>> I updated the JIRA to add 0.15.1 but ARROW-6806 seems like it should be
> >>> included as well.
> >>>
> >>> On Fri, Oct 11, 2019 at 2:40 PM Wes McKinney  wrote:
> >>>
>  I just created a 0.15.1 version in JIRA. Please mark the issues you
>  want to include in the release. Perhaps we can cut an RC by mid next
>  week if bug fixes to the critical issues have been sorted out.
> 
>  On Fri, Oct 11, 2019 at 3:53 PM Neal Richardson
>   wrote:
> >
> > If 6844 isn't Python-only (I can't tell from the description) and
> > we're updating C++ libraries, I'd like to include
> > https://issues.apache.org/jira/browse/ARROW-3808 for R (not a bug fix
> > but would be helpful to get out). But if there's no C++ change
> > involved, then I probably wouldn't bother to update R on CRAN, so it's
> > moot.
> >
> > Neal
> >
> > On Fri, Oct 11, 2019 at 1:33 PM Sutou Kouhei 
> >>> wrote:
> >>
> >> Hi,
> >>
> >> If we release 0.15.1, I want to include
> >> https://issues.apache.org/jira/browse/ARROW-6777 into it.
> >>
> >>
> >> Thanks,
> >> --
> >> kou
> >>
> >> In  >>> gvr8cfnau09qs47dgauszfnvyquhasv8y...@mail.gmail.com
> >
> >>"Possible Arrow 0.15.1 release" on Fri, 11 Oct 2019 09:44:21 -0500,
> >>Wes McKinney  wrote:
> >>
> >>> hi folks,
> >>>
> >>> On account of a https://issues.apache.org/jira/browse/ARROW-6844
> >>> we
> >>> might want to make a 0.15.1 release before the next major release.
> >>> I
> >>> can look into a bug fix, but if we decide to go this route we
> >>> should
> >>> begin tracking other bug fixes that we'd like to include in a patch
> >>> release. Thoughts?
> >>>
> >>> - Wes
> 
> >>>

Re: Possible Arrow 0.15.1 release

[Antoine Pitrou]

I added 0.15.1 back to a couple of fixed issues where the merge script 
removed it :-/




Le 17/10/2019 à 09:30, Wes McKinney a écrit :

Nearly all the fixes are in for 0.15.1.

I think the only thing preventing us from making an RC soon is
(surprise) new problems with packaging (Homebrew-related, and others).

Anything else that needs to go into 0.15.1 that is not marked as such?

On Sat, Oct 12, 2019 at 1:52 AM Fan Liya  wrote:


I have update ARROW-6738 
to add 0.15.1.
I hope this bug can be fixed in the next release.

Best,
Liya Fan

On Sat, Oct 12, 2019 at 12:38 PM Micah Kornfield 
wrote:


I updated the JIRA to add 0.15.1 but ARROW-6806 seems like it should be
included as well.

On Fri, Oct 11, 2019 at 2:40 PM Wes McKinney  wrote:


I just created a 0.15.1 version in JIRA. Please mark the issues you
want to include in the release. Perhaps we can cut an RC by mid next
week if bug fixes to the critical issues have been sorted out.

On Fri, Oct 11, 2019 at 3:53 PM Neal Richardson
 wrote:


If 6844 isn't Python-only (I can't tell from the description) and
we're updating C++ libraries, I'd like to include
https://issues.apache.org/jira/browse/ARROW-3808 for R (not a bug fix
but would be helpful to get out). But if there's no C++ change
involved, then I probably wouldn't bother to update R on CRAN, so it's
moot.

Neal

On Fri, Oct 11, 2019 at 1:33 PM Sutou Kouhei 

wrote:


Hi,

If we release 0.15.1, I want to include
https://issues.apache.org/jira/browse/ARROW-6777 into it.


Thanks,
--
kou

In 
gvr8cfnau09qs47dgauszfnvyquhasv8y...@mail.gmail.com



   "Possible Arrow 0.15.1 release" on Fri, 11 Oct 2019 09:44:21 -0500,
   Wes McKinney  wrote:


hi folks,

On account of a https://issues.apache.org/jira/browse/ARROW-6844

we

might want to make a 0.15.1 release before the next major release.

I

can look into a bug fix, but if we decide to go this route we

should

begin tracking other bug fixes that we'd like to include in a patch
release. Thoughts?

- Wes

Re: Possible Arrow 0.15.1 release

[Wes McKinney]

Nearly all the fixes are in for 0.15.1.

I think the only thing preventing us from making an RC soon is
(surprise) new problems with packaging (Homebrew-related, and others).

Anything else that needs to go into 0.15.1 that is not marked as such?

On Sat, Oct 12, 2019 at 1:52 AM Fan Liya  wrote:
>
> I have update ARROW-6738 
> to add 0.15.1.
> I hope this bug can be fixed in the next release.
>
> Best,
> Liya Fan
>
> On Sat, Oct 12, 2019 at 12:38 PM Micah Kornfield 
> wrote:
>
> > I updated the JIRA to add 0.15.1 but ARROW-6806 seems like it should be
> > included as well.
> >
> > On Fri, Oct 11, 2019 at 2:40 PM Wes McKinney  wrote:
> >
> > > I just created a 0.15.1 version in JIRA. Please mark the issues you
> > > want to include in the release. Perhaps we can cut an RC by mid next
> > > week if bug fixes to the critical issues have been sorted out.
> > >
> > > On Fri, Oct 11, 2019 at 3:53 PM Neal Richardson
> > >  wrote:
> > > >
> > > > If 6844 isn't Python-only (I can't tell from the description) and
> > > > we're updating C++ libraries, I'd like to include
> > > > https://issues.apache.org/jira/browse/ARROW-3808 for R (not a bug fix
> > > > but would be helpful to get out). But if there's no C++ change
> > > > involved, then I probably wouldn't bother to update R on CRAN, so it's
> > > > moot.
> > > >
> > > > Neal
> > > >
> > > > On Fri, Oct 11, 2019 at 1:33 PM Sutou Kouhei 
> > wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > If we release 0.15.1, I want to include
> > > > > https://issues.apache.org/jira/browse/ARROW-6777 into it.
> > > > >
> > > > >
> > > > > Thanks,
> > > > > --
> > > > > kou
> > > > >
> > > > > In  > gvr8cfnau09qs47dgauszfnvyquhasv8y...@mail.gmail.com
> > > >
> > > > >   "Possible Arrow 0.15.1 release" on Fri, 11 Oct 2019 09:44:21 -0500,
> > > > >   Wes McKinney  wrote:
> > > > >
> > > > > > hi folks,
> > > > > >
> > > > > > On account of a https://issues.apache.org/jira/browse/ARROW-6844
> > we
> > > > > > might want to make a 0.15.1 release before the next major release.
> > I
> > > > > > can look into a bug fix, but if we decide to go this route we
> > should
> > > > > > begin tracking other bug fixes that we'd like to include in a patch
> > > > > > release. Thoughts?
> > > > > >
> > > > > > - Wes
> > >
> >

[NIGHTLY] Arrow Build Report for Job nightly-2019-10-17-0

[Crossbow]

Arrow Build Report for Job nightly-2019-10-17-0

All tasks: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0

Failed Tasks:
- wheel-osx-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp36m
- debian-stretch:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-stretch
- wheel-osx-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp37m
- wheel-osx-cp27m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp27m
- wheel-osx-cp35m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-osx-cp35m
- docker-pandas-master:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-pandas-master
- conda-linux-gcc-py37:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py37
- docker-c_glib:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-c_glib
- wheel-manylinux2010-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp37m
- wheel-win-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-appveyor-wheel-win-cp37m
- docker-clang-format:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-clang-format
- wheel-win-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-appveyor-wheel-win-cp36m
- gandiva-jar-osx:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-gandiva-jar-osx
- debian-buster:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-debian-buster
- ubuntu-disco:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-ubuntu-disco
- conda-linux-gcc-py36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py36
- conda-linux-gcc-py27:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-linux-gcc-py27
- gandiva-jar-trusty:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-gandiva-jar-trusty

Succeeded Tasks:
- docker-docs:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-docs
- conda-win-vs2015-py37:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-win-vs2015-py37
- wheel-manylinux1-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp37m
- wheel-manylinux1-cp27mu:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp27mu
- wheel-manylinux2010-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp36m
- docker-turbodbc-integration:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-turbodbc-integration
- docker-cpp:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-cpp
- docker-java:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-java
- wheel-manylinux2010-cp35m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp35m
- ubuntu-xenial:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-ubuntu-xenial
- wheel-manylinux1-cp27m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp27m
- docker-hdfs-integration:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-hdfs-integration
- wheel-manylinux1-cp35m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux1-cp35m
- conda-osx-clang-py37:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-conda-osx-clang-py37
- wheel-manylinux2010-cp27m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-travis-wheel-manylinux2010-cp27m
- docker-lint:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-lint
- docker-r:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-circle-docker-r
- centos-6:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-17-0-azure-centos-6
- docker-r-sanitizer:
  URL: 

[jira] [Created] (ARROW-6912) [Java] Extract a common base class for avro converter consumers

[Ji Liu (Jira)]

Ji Liu created ARROW-6912:
-

 Summary: [Java] Extract a common base class for avro converter 
consumers
 Key: ARROW-6912
 URL: https://issues.apache.org/jira/browse/ARROW-6912
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Java
Reporter: Ji Liu
Assignee: Ji Liu


Currently Avro converter consumers have some common variables and methods which 
could be eliminated by extracting a common class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)