[jira] [Created] (ARROW-7863) [C++][Python][CI] Ensure running HDFS related tests

2020-02-14 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-7863:
---

 Summary: [C++][Python][CI] Ensure running HDFS related tests
 Key: ARROW-7863
 URL: https://issues.apache.org/jira/browse/ARROW-7863
 Project: Apache Arrow
  Issue Type: Improvement
  Components: C++, Continuous Integration, Python
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Schemaless serialization

2020-02-14 Thread Tewfik Zeghmi
Hi Micah,

The primary language is Python.  I'm hoping the that the small overhead of
metadata is small compared to the schema information.

thank you!

On Fri, Feb 14, 2020 at 3:07 PM Micah Kornfield 
wrote:

> Hi Tewfik,
> What language?  it is possible to serialize them separately but the right
> hooks might not be exposed in all languages.
>
> There is still going to be a higher overhead for single row values in Arrow
> compared to Avro due to metadata requirements.
>
> Thanks,
> Micah
>
> On Fri, Feb 14, 2020 at 1:33 PM Tewfik Zeghmi  wrote:
>
> > Hi,
> >
> > I have a use case of creating a feature store to serve low latency
> traffic.
> > Given a key, we need the ability to save and read a feature vector in a
> low
> > latency Key Value store. Serializing an Arrow table with one row is takes
> > 1344 bytes, while the same singular row serialized with AVRO without the
> > schema uses 236 bytes.
> >
> > Is it possible to save serialize an Arrow table/RecordBatch independently
> > of the schema? Ideally, we'd like to serialize the schema once and not
> > along with every feature key, then be able to read the RecordBatch with
> the
> > schema.
> >
> > thank you!
> >
>


-- 
Taleb Tewfik Zeghmi


Re: Schemaless serialization

2020-02-14 Thread Micah Kornfield
Hi Tewfik,
What language?  it is possible to serialize them separately but the right
hooks might not be exposed in all languages.

There is still going to be a higher overhead for single row values in Arrow
compared to Avro due to metadata requirements.

Thanks,
Micah

On Fri, Feb 14, 2020 at 1:33 PM Tewfik Zeghmi  wrote:

> Hi,
>
> I have a use case of creating a feature store to serve low latency traffic.
> Given a key, we need the ability to save and read a feature vector in a low
> latency Key Value store. Serializing an Arrow table with one row is takes
> 1344 bytes, while the same singular row serialized with AVRO without the
> schema uses 236 bytes.
>
> Is it possible to save serialize an Arrow table/RecordBatch independently
> of the schema? Ideally, we'd like to serialize the schema once and not
> along with every feature key, then be able to read the RecordBatch with the
> schema.
>
> thank you!
>


[jira] [Created] (ARROW-7862) [R] Linux installation should run quieter by default

2020-02-14 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-7862:
--

 Summary: [R] Linux installation should run quieter by default
 Key: ARROW-7862
 URL: https://issues.apache.org/jira/browse/ARROW-7862
 Project: Apache Arrow
  Issue Type: Improvement
  Components: R
Reporter: Neal Richardson
Assignee: Neal Richardson
 Fix For: 1.0.0, 0.16.1


No need to blow up the console by default. Also this solves an {{R CMD check}} 
warning that surfaced on CRAN.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Schemaless serialization

2020-02-14 Thread Tewfik Zeghmi
Hi,

I have a use case of creating a feature store to serve low latency traffic.
Given a key, we need the ability to save and read a feature vector in a low
latency Key Value store. Serializing an Arrow table with one row is takes
1344 bytes, while the same singular row serialized with AVRO without the
schema uses 236 bytes.

Is it possible to save serialize an Arrow table/RecordBatch independently
of the schema? Ideally, we'd like to serialize the schema once and not
along with every feature key, then be able to read the RecordBatch with the
schema.

thank you!


Re: Thinking about 0.16.1 patch release

2020-02-14 Thread Neal Richardson
IIUC the difference in the verification script is that we construct the
wheel file name, download it, and install the file, rather than have pip
query a repo and download the latest:
https://github.com/apache/arrow/blob/master/dev/release/verify-release-candidate-wheels.bat#L69-L72

Verification failed first with a 404 on the wget, and then when the file
path was fixed to reflect the actual file name (the first time around, with
an extra m), python said that there was no package matching the ABI. If we
were testing pip installing from a repository, this would have failed as
soon as we started building wheels for py38.

Neal

On Fri, Feb 14, 2020 at 11:22 AM Krisztián Szűcs 
wrote:

> Actually we are verifying the wheels after building them, the manylinux
> wheels in fresh environments using docker.
> We should first investigate why didn't the windows wheel issue surface
> during its verification [1].
>
> [1]:
> https://github.com/apache/arrow/blob/master/dev/tasks/python-wheels/win-build.bat#L102
>
> On Fri, Feb 14, 2020 at 5:16 PM Neal Richardson
>  wrote:
> >
> > In terms of wheel verification, I added
> > https://issues.apache.org/jira/browse/ARROW-7853 to the 0.16.1 tag. It's
> > for CI using pip to install the wheels we build nightly. Obviously this
> is
> > not required for 0.16.1 but it would make the release verification
> simpler
> > because we would be running that logic every day and wouldn't have to
> wait
> > until release time to find issues.
> >
> > Neal
> >
> > On Fri, Feb 14, 2020 at 7:49 AM Wes McKinney 
> wrote:
> >
> > > It seems inevitable that a few annoying regressions will pop up as
> > > 0.16.0 becomes more widely deployed, e.g. ARROW-7841 was just reported
> > > and patched. I created a 0.16.1 Fix Version in JIRA so that we can tag
> > > issues that we may want to cherry pick into a maint-0.16.x branch. We
> > > probably would want to wait a couple of weeks before cutting the patch
> > > release, though.
> > >
> > > Also, as a matter of easing release management for patch releases, I
> > > wonder if it makes sense to focus RC verification efforts on the
> > > binary artifacts (especially the Python wheels). Should we write down
> > > a "reduced" set of tasks for release managers to track for patch
> > > releases?
> > >
> > > Thanks,
> > > Wes
> > >
>


Re: Thinking about 0.16.1 patch release

2020-02-14 Thread Krisztián Szűcs
Actually we are verifying the wheels after building them, the manylinux
wheels in fresh environments using docker.
We should first investigate why didn't the windows wheel issue surface
during its verification [1].

[1]: 
https://github.com/apache/arrow/blob/master/dev/tasks/python-wheels/win-build.bat#L102

On Fri, Feb 14, 2020 at 5:16 PM Neal Richardson
 wrote:
>
> In terms of wheel verification, I added
> https://issues.apache.org/jira/browse/ARROW-7853 to the 0.16.1 tag. It's
> for CI using pip to install the wheels we build nightly. Obviously this is
> not required for 0.16.1 but it would make the release verification simpler
> because we would be running that logic every day and wouldn't have to wait
> until release time to find issues.
>
> Neal
>
> On Fri, Feb 14, 2020 at 7:49 AM Wes McKinney  wrote:
>
> > It seems inevitable that a few annoying regressions will pop up as
> > 0.16.0 becomes more widely deployed, e.g. ARROW-7841 was just reported
> > and patched. I created a 0.16.1 Fix Version in JIRA so that we can tag
> > issues that we may want to cherry pick into a maint-0.16.x branch. We
> > probably would want to wait a couple of weeks before cutting the patch
> > release, though.
> >
> > Also, as a matter of easing release management for patch releases, I
> > wonder if it makes sense to focus RC verification efforts on the
> > binary artifacts (especially the Python wheels). Should we write down
> > a "reduced" set of tasks for release managers to track for patch
> > releases?
> >
> > Thanks,
> > Wes
> >


[jira] [Created] (ARROW-7861) [C++][Parquet] Add fuzz regression corpus for parquet reader

2020-02-14 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7861:
-

 Summary: [C++][Parquet] Add fuzz regression corpus for parquet 
reader
 Key: ARROW-7861
 URL: https://issues.apache.org/jira/browse/ARROW-7861
 Project: Apache Arrow
  Issue Type: Bug
Reporter: Francois Saint-Jacques






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ARROW-7860) [C++] Support cast to/from halffloat

2020-02-14 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-7860:
--

 Summary: [C++] Support cast to/from halffloat
 Key: ARROW-7860
 URL: https://issues.apache.org/jira/browse/ARROW-7860
 Project: Apache Arrow
  Issue Type: Improvement
  Components: C++, C++ - Compute
Reporter: Neal Richardson
 Fix For: 1.0.0


In trying to do ARROW-7753 I realized I couldn't make a halffloat. I tried 
creating a float64 (as R does naturally) and casting to float16, but it's not 
implemented. Looking at compute/kernels/cast.cc, and the associated source in 
compute/kernels/generated/codegen.py, {{FLOATING_TYPES = ['Float', 'Double']}}. 
Maybe halffloat just needs to be added there? 

Aside: searching through the code, it seems that this limitation of float types 
to float32 and float64 is the norm. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ARROW-7859) [R] Minor patches for CRAN submission 0.16.0.2

2020-02-14 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-7859:
--

 Summary: [R] Minor patches for CRAN submission 0.16.0.2
 Key: ARROW-7859
 URL: https://issues.apache.org/jira/browse/ARROW-7859
 Project: Apache Arrow
  Issue Type: Improvement
  Components: R
Reporter: Neal Richardson
Assignee: Neal Richardson
 Fix For: 1.0.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Thinking about 0.16.1 patch release

2020-02-14 Thread Neal Richardson
In terms of wheel verification, I added
https://issues.apache.org/jira/browse/ARROW-7853 to the 0.16.1 tag. It's
for CI using pip to install the wheels we build nightly. Obviously this is
not required for 0.16.1 but it would make the release verification simpler
because we would be running that logic every day and wouldn't have to wait
until release time to find issues.

Neal

On Fri, Feb 14, 2020 at 7:49 AM Wes McKinney  wrote:

> It seems inevitable that a few annoying regressions will pop up as
> 0.16.0 becomes more widely deployed, e.g. ARROW-7841 was just reported
> and patched. I created a 0.16.1 Fix Version in JIRA so that we can tag
> issues that we may want to cherry pick into a maint-0.16.x branch. We
> probably would want to wait a couple of weeks before cutting the patch
> release, though.
>
> Also, as a matter of easing release management for patch releases, I
> wonder if it makes sense to focus RC verification efforts on the
> binary artifacts (especially the Python wheels). Should we write down
> a "reduced" set of tasks for release managers to track for patch
> releases?
>
> Thanks,
> Wes
>


Thinking about 0.16.1 patch release

2020-02-14 Thread Wes McKinney
It seems inevitable that a few annoying regressions will pop up as
0.16.0 becomes more widely deployed, e.g. ARROW-7841 was just reported
and patched. I created a 0.16.1 Fix Version in JIRA so that we can tag
issues that we may want to cherry pick into a maint-0.16.x branch. We
probably would want to wait a couple of weeks before cutting the patch
release, though.

Also, as a matter of easing release management for patch releases, I
wonder if it makes sense to focus RC verification efforts on the
binary artifacts (especially the Python wheels). Should we write down
a "reduced" set of tasks for release managers to track for patch
releases?

Thanks,
Wes


[NIGHTLY] Arrow Build Report for Job nightly-2020-02-14-0

2020-02-14 Thread Crossbow


Arrow Build Report for Job nightly-2020-02-14-0

All tasks: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0

Failed Tasks:
- conda-linux-gcc-py27:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-linux-gcc-py27
- conda-osx-clang-py27:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-osx-clang-py27
- macos-r-autobrew:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-travis-macos-r-autobrew
- test-conda-python-2.7-pandas-latest:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-2.7-pandas-latest
- test-conda-python-2.7:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-2.7
- test-conda-python-3.7-pandas-master:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-3.7-pandas-master
- test-conda-python-3.7-turbodbc-latest:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-3.7-turbodbc-latest
- test-conda-python-3.7-turbodbc-master:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-3.7-turbodbc-master
- test-ubuntu-18.04-docs:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-ubuntu-18.04-docs
- wheel-manylinux2014-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-wheel-manylinux2014-cp36m
- wheel-manylinux2014-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-wheel-manylinux2014-cp37m

Succeeded Tasks:
- centos-6:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-centos-6
- centos-7:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-centos-7
- centos-8:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-centos-8
- conda-linux-gcc-py36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-linux-gcc-py36
- conda-linux-gcc-py37:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-linux-gcc-py37
- conda-linux-gcc-py38:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-linux-gcc-py38
- conda-osx-clang-py36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-osx-clang-py36
- conda-osx-clang-py37:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-osx-clang-py37
- conda-osx-clang-py38:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-osx-clang-py38
- conda-win-vs2015-py36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-win-vs2015-py36
- conda-win-vs2015-py37:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-win-vs2015-py37
- conda-win-vs2015-py38:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-conda-win-vs2015-py38
- debian-buster:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-debian-buster
- debian-stretch:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-azure-debian-stretch
- gandiva-jar-osx:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-travis-gandiva-jar-osx
- gandiva-jar-trusty:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-travis-gandiva-jar-trusty
- homebrew-cpp:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-travis-homebrew-cpp
- test-conda-cpp-valgrind:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-cpp-valgrind
- test-conda-cpp:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-cpp
- test-conda-python-3.6:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-3.6
- test-conda-python-3.7-dask-latest:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-3.7-dask-latest
- test-conda-python-3.7-hdfs-2.9.2:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-3.7-hdfs-2.9.2
- test-conda-python-3.7-pandas-latest:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-14-0-circle-test-conda-python-3.7-pandas-latest
- test-conda-python-3.7-spark-master:
  URL: 
https

Re: [VOTE] Adopt Arrow in-process C Data Interface specification

2020-02-14 Thread Wes McKinney
There is only 1 binding +1 vote so far, we should probably wait for
three before closing the vote (it's possible that lazy consensus could
be employed here but not much harm in waiting a few more days)

On Thu, Feb 13, 2020 at 8:15 PM Francois Saint-Jacques
 wrote:
>
> +1
>
> On Thu, Feb 13, 2020 at 9:08 PM Fan Liya  wrote:
> >
> > +1 (binding)
> >
> > On Thu, Feb 13, 2020 at 11:52 AM Wes McKinney  wrote:
> >
> > > +1 (binding)
> > >
> > > On Tue, Feb 11, 2020 at 4:29 PM Antoine Pitrou  wrote:
> > > >
> > > >
> > > > Ah, you're right, it's PR 6040:
> > > > https://github.com/apache/arrow/pull/6040
> > > >
> > > > Similarly, the C++ implementation is at PR 6026:
> > > > https://github.com/apache/arrow/pull/6026
> > > >
> > > > Regards
> > > >
> > > > Antoine.
> > > >
> > > >
> > > > Le 11/02/2020 à 23:17, Wes McKinney a écrit :
> > > > > hi Antoine, PR 5442 seems to no longer be the right one. Which open PR
> > > > > contains the specification now?
> > > > >
> > > > > On Tue, Feb 11, 2020 at 1:06 PM Antoine Pitrou 
> > > wrote:
> > > > >>
> > > > >>
> > > > >> Hello,
> > > > >>
> > > > >> We have been discussing the creation of a minimalist C-based data
> > > > >> interface for applications to exchange Arrow columnar data structures
> > > > >> with each other. Some notable features of this interface include:
> > > > >>
> > > > >> * A small amount of header-only C code can be copied independently
> > > into
> > > > >> third-party libraries and downstream applications, no dependencies 
> > > > >> are
> > > > >> needed even on Arrow C++ itself (notably, it is not required to use
> > > > >> Flatbuffers, though there are trade-offs resulting from this).
> > > > >>
> > > > >> * Low development investment (in other words: limited-scope use cases
> > > > >> can be accomplished with little code), so as to enable C or C++
> > > > >> libraries to export Arrow columnar data with minimal code.
> > > > >>
> > > > >> * Data lifetime management hooks so as to properly handle non-trivial
> > > > >> data sharing (for example passing Arrow columnar data to an async
> > > > >> processing consumer).
> > > > >>
> > > > >> This "C Data Interface" serves different use cases from the
> > > > >> language-independent IPC protocol and trades away a number of 
> > > > >> features
> > > > >> in the interest of minimalism / simplicity. It is not a replacement
> > > for
> > > > >> the IPC protocol and will only be used to interchange in-process data
> > > at
> > > > >> C or C++ call sites.
> > > > >>
> > > > >> The PR providing the specification is here:
> > > > >> https://github.com/apache/arrow/pull/5442
> > > > >>
> > > > >> In particular, you can read the spec document here:
> > > > >>
> > > https://github.com/pitrou/arrow/blob/doc-c-data-interface2/docs/source/format/CDataInterface.rst
> > > > >>
> > > > >> A fairly comprehensive C++ implementation of this demonstrating its
> > > > >> use is found here:
> > > > >> https://github.com/apache/arrow/pull/5608
> > > > >>
> > > > >> (note that other applications implementing the interface may choose 
> > > > >> to
> > > > >> only support a few features and thus have far less code to write)
> > > > >>
> > > > >> Please vote to adopt the SPECIFICATION (GitHub PR #5442).
> > > > >>
> > > > >> This vote will be open for at least 72 hours
> > > > >>
> > > > >> [ ] +1 Adopt C Data Interface specification
> > > > >> [ ] +0
> > > > >> [ ] -1 Do not adopt because...
> > > > >>
> > > > >> Thank you
> > > > >>
> > > > >> Regards
> > > > >>
> > > > >> Antoine.
> > > > >>
> > > > >>
> > > > >> (PS: yes, this is in large part a copy/paste of Wes's previous vote
> > > > >> email :-))
> > >


[jira] [Created] (ARROW-7858) [C++][Python] Support casting an Extension type to its storage type

2020-02-14 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7858:


 Summary: [C++][Python] Support casting an Extension type to its 
storage type
 Key: ARROW-7858
 URL: https://issues.apache.org/jira/browse/ARROW-7858
 Project: Apache Arrow
  Issue Type: Test
  Components: C++, Python
Reporter: Joris Van den Bossche


Currently, casting an extension type will always fail: "No cast implemented 
from extension to ...".

However, for casting, we could fall back to the storage array's casting rules?





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ARROW-7857) [Python] Failing test with pandas master for extension type conversion

2020-02-14 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7857:


 Summary: [Python] Failing test with pandas master for extension 
type conversion
 Key: ARROW-7857
 URL: https://issues.apache.org/jira/browse/ARROW-7857
 Project: Apache Arrow
  Issue Type: Test
  Components: Python
Reporter: Joris Van den Bossche
Assignee: Joris Van den Bossche


The pandas master test build has one failure


{code}
___ test_conversion_extensiontype_to_extensionarray 

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fcd6c580bd0>

def test_conversion_extensiontype_to_extensionarray(monkeypatch):
# converting extension type to linked pandas ExtensionDtype/Array
import pandas.core.internals as _int

storage = pa.array([1, 2, 3, 4], pa.int64())
arr = pa.ExtensionArray.from_storage(MyCustomIntegerType(), storage)
table = pa.table({'a': arr})

if LooseVersion(pd.__version__) < "0.26.0.dev":
# ensure pandas Int64Dtype has the protocol method (for older 
pandas)
monkeypatch.setattr(
pd.Int64Dtype, '__from_arrow__', _Int64Dtype__from_arrow__,
raising=False)

# extension type points to Int64Dtype, which knows how to create a
# pandas ExtensionArray
>   result = table.to_pandas()

opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py:3560:
 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyarrow/ipc.pxi:559: in pyarrow.lib.read_message
???
pyarrow/table.pxi:1369: in pyarrow.lib.Table._to_pandas
???
opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:764: 
in table_to_blockmanager
blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes)
opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:1102: 
in _table_to_blocks
for item in result]
opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:1102: 
in 
for item in result]
opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:723: 
in _reconstruct_block
pd_ext_arr = pandas_dtype.__from_arrow__(arr)
opt/conda/envs/arrow/lib/python3.7/site-packages/pandas/core/arrays/integer.py:108:
 in __from_arrow__
array = array.cast(pyarrow_type)
pyarrow/table.pxi:240: in pyarrow.lib.ChunkedArray.cast
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   pyarrow.lib.ArrowNotImplementedError: No cast implemented from 
extension to int64
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-02-13-0

2020-02-14 Thread Joris Van den Bossche
I opened an issue for the pandas-master failure:
https://issues.apache.org/jira/browse/ARROW-7857

On Thu, 13 Feb 2020 at 21:08, Crossbow  wrote:

>
> Arrow Build Report for Job nightly-2020-02-13-0
>
> All tasks:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0
>
> Failed Tasks:
> - conda-linux-gcc-py27:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-linux-gcc-py27
> - conda-osx-clang-py27:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-osx-clang-py27
> - macos-r-autobrew:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-travis-macos-r-autobrew
> - test-conda-python-2.7-pandas-latest:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-python-2.7-pandas-latest
> - test-conda-python-2.7:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-python-2.7
> - test-conda-python-3.7-pandas-master:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-python-3.7-pandas-master
> - test-conda-python-3.7-turbodbc-latest:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-python-3.7-turbodbc-latest
> - test-conda-python-3.7-turbodbc-master:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-python-3.7-turbodbc-master
> - test-r-rhub-debian-gcc-devel:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-test-r-rhub-debian-gcc-devel
> - test-ubuntu-18.04-docs:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-ubuntu-18.04-docs
> - wheel-manylinux2010-cp36m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-wheel-manylinux2010-cp36m
> - wheel-osx-cp35m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-travis-wheel-osx-cp35m
>
> Succeeded Tasks:
> - centos-6:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-centos-6
> - centos-7:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-centos-7
> - centos-8:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-centos-8
> - conda-linux-gcc-py36:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-linux-gcc-py36
> - conda-linux-gcc-py37:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-linux-gcc-py37
> - conda-linux-gcc-py38:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-linux-gcc-py38
> - conda-osx-clang-py36:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-osx-clang-py36
> - conda-osx-clang-py37:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-osx-clang-py37
> - conda-osx-clang-py38:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-osx-clang-py38
> - conda-win-vs2015-py36:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-win-vs2015-py36
> - conda-win-vs2015-py37:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-win-vs2015-py37
> - conda-win-vs2015-py38:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-conda-win-vs2015-py38
> - debian-buster:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-debian-buster
> - debian-stretch:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-azure-debian-stretch
> - gandiva-jar-osx:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-travis-gandiva-jar-osx
> - gandiva-jar-trusty:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-travis-gandiva-jar-trusty
> - homebrew-cpp:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-travis-homebrew-cpp
> - test-conda-cpp-valgrind:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-cpp-valgrind
> - test-conda-cpp:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-cpp
> - test-conda-python-3.6:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-13-0-circle-test-conda-python-3.6
> - test-conda-python-3.7-dask-latest:
>   URL:
> https://github.com/ursa-labs/crossbow/bran