Re: [C++] Compilation error when creating a Tuple with optional String

[Prudhvi Porandla]

Yes, explicitly creating std::string fixes the problem. This is not needed for 
newer gcc versions(>=6). 

Thanks a lot Ben and Omer.

> On 12-Sep-2019, at 10:57 PM, Omer F. Ozarslan  wrote:
> 
> I'm looking into it. Will submit PR today.
> 
> On Thu, Sep 12, 2019 at 11:32 AM Ben Kietzman  
> wrote:
>> 
>> It looks like a tuple constructor is choking on implicit conversion from 
>> string literal (char[6]) to boost::optional:
>> https://travis-ci.org/ursa-labs/crossbow/builds/582833123#L1161
>> 
>> That constructor in the 4.8.2 release of GCC requires that parameters be 
>> *convertible* to the corresponding element type (and not just that the 
>> element be constructible from the parameter)
>> https://github.com/gcc-mirror/gcc/blob/gcc-4_8_2-release/libstdc++-v3/include/std/tuple#L400-L405
>> 
>> Locally,
>>  // static_assert(std::is_convertible> boost::optional>::value, "FAILS");
>>  static_assert(std::is_convertible> boost::optional>::value, "fine");
>> So I think this can be resolved by wrapping string literal arguments to 
>> tuple constructors with std::string to ensure they are convertible.
>> 
>> I'll create a JIRA
>> 
>> On Thu, Sep 12, 2019 at 8:09 AM Prudhvi Porandla  wrote:
>>> 
>>> Hi,
>>> 
>>> Gandiva trusty 
>>> job(https://travis-ci.org/ursa-labs/crossbow/builds/582833123) is failing 
>>> with this commit (ARROW-6326: [C++] Nullable fields when converting 
>>> std::tuple to Table  
>>> ).
>>> The compilation is successful if the optional string column is removed from 
>>> boost_optional_types_tuple(see line 36 of cpp/src/arrow/stl_test.cc).
>>> 
>>> Could someone help me understand this error?
>>> 
>>> 
>>> Thanks,
>>> Prudhvi
>>> 
>>> 

Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte Flatbuffer alignment requirements (2nd vote)

[Wes McKinney]

Thanks all!

It looks like all the changes are in, so we need to see if the
integration tests pass before we can merge these changes into master

On Thu, Sep 12, 2019 at 7:19 AM Sebastien Binet  wrote:
>
> hi there,
>
> On Thu, Sep 12, 2019 at 12:45 AM Wes McKinney  wrote:
>
> > Thanks Bryan.
> >
> > I merged the Java patch with the EOS change and submitted a C++ patch
> > which also updates the specification
> >
> > https://github.com/apache/arrow/pull/5361
> >
> > Let me know when the JS or C# patches are ready to go and I can merge
> > those.
> >
> > I updated https://issues.apache.org/jira/browse/ARROW-6545 to track
> > the Go change corresponding to this
> >
>
> done. (I think)
>
> -s

Re: [Java] CI test failures

[Wes McKinney]

I was able to get the underlying errors to pop out on Travis CI

https://github.com/apache/arrow/pull/5370#issuecomment-531029711

This looks like a VM resource poverty issue. You may have to disable
these tests in Travis CI

On Thu, Sep 12, 2019 at 9:25 AM Wes McKinney  wrote:
>
> Does upgrading maven-surefire-plugin help?
>
> On Thu, Sep 12, 2019 at 8:29 AM Fan Liya  wrote:
> >
> > I have tried many times locally, with different values of forkCount, but
> > failed to reproduce the error.
> >
> > Best,
> > Liya Fan
> >
> > On Thu, Sep 12, 2019 at 4:21 PM Fan Liya  wrote:
> >
> > > It seems the problem occurred when the sure-fire plugin forks:
> > >
> > > [ERROR] Caused by:
> > > org.apache.maven.surefire.booter.SurefireBooterForkException: Error
> > > occurred in starting fork, check output in log
> > >
> > > So we need to adjust the forkCount parameter in the pom.xml file?
> > >
> > > Best,
> > > Liya Fan
> > >
> > > On Thu, Sep 12, 2019 at 3:10 PM Micah Kornfield 
> > > wrote:
> > >
> > >> I did not make much progress, I will spend some more time tomorrow, if no
> > >> one picks it up by then.
> > >>
> > >> On Wed, Sep 11, 2019 at 10:48 PM Micah Kornfield 
> > >> wrote:
> > >>
> > >> > I will try to take look the next couple of hours to see if I can fix it
> > >> > quickly.
> > >> >
> > >> > On Wed, Sep 11, 2019 at 4:54 AM Antoine Pitrou 
> > >> wrote:
> > >> >
> > >> >>
> > >> >> Hello,
> > >> >>
> > >> >> Some Travis-CI builds are failing because of Java issues.  It would be
> > >> >> good if a Java contributor or maintainer could take a look ASAP.
> > >> >> https://issues.apache.org/jira/browse/ARROW-6509
> > >> >>
> > >> >> Thanks
> > >> >>
> > >> >> Antoine.
> > >> >>
> > >> >
> > >>
> > >

[jira] [Created] (ARROW-6554) [C++] Implement "background write" option for BufferedOutputStream

[Wes McKinney (Jira)]

Wes McKinney created ARROW-6554:
---

 Summary: [C++] Implement "background write" option for 
BufferedOutputStream
 Key: ARROW-6554
 URL: https://issues.apache.org/jira/browse/ARROW-6554
 Project: Apache Arrow
  Issue Type: Improvement
  Components: C++
Reporter: Wes McKinney


In such cases we would have to allocate a new buffer on subsequent writes 
(while a background write is completing), but it could yield beneficial 
performance improvements for high latency filesystems



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (ARROW-6555) [C++] Implement benchmarks for BufferedInputStream / BufferedOutputStream

[Wes McKinney (Jira)]

Wes McKinney created ARROW-6555:
---

 Summary: [C++] Implement benchmarks for BufferedInputStream / 
BufferedOutputStream
 Key: ARROW-6555
 URL: https://issues.apache.org/jira/browse/ARROW-6555
 Project: Apache Arrow
  Issue Type: Improvement
  Components: C++
Reporter: Wes McKinney


See discussion in ARROW-4220



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [DISCUSS][C++] Rethinking our current C++ shared library (.so / .dll) approach

[Wes McKinney]

No.

On Thu, Sep 12, 2019 at 3:32 PM Neal Richardson
 wrote:
>
> Just to be clear: this is not part of the 0.15 release discussion, right?
>
> On Thu, Sep 12, 2019 at 1:29 PM Wes McKinney  wrote:
> >
> > On Thu, Sep 12, 2019 at 3:14 PM Antoine Pitrou  wrote:
> > >
> > >
> > > Le 12/09/2019 à 20:14, Wes McKinney a écrit :
> > > > hi folks,
> > > >
> > > > I wanted to share some concerns that I have about our current
> > > > trajectory with regards to producing shared libraries from the Arrow
> > > > build system.
> > > >
> > > > Currently, a comprehensive build produces many shared libraries:
> > > >
> > > > * libarrow
> > > > * libarrow_dataset
> > > > * libarrow_flight
> > > > * libarrow_python
> > > > * libgandiva
> > > > * libparquet
> > > > * libplasma
> > >
> > > Each library pulls its own set of non-trivial third-party dependencies.
> > >  For example, libarrow_flight includes gRPC, libcurl and OpenSSL as far
> > > as I remember.
> > >
> > > Worse, libarrow_cuda depends on the NVidia CUDA libraries.  That's a
> > > *huge* dependency (and may also have runtime costs in addition to
> > > installation / deployment annoyances).
> > >
> > > Some packagers may also want to distribute some sublibraries separately
> > > (Linux distros especially tend to like having many small, focussed
> > > shared libraries that they can distribute as separate, dependent
> > > packages) - but that's not my business and I'll let them speak up ;-)
> > >
> >
> > It's a fair point. Another possibility is to be more Boost-like in the
> > approach to the library structure. But there are costs involved with
> > this, e.g. each Boost component that produces a shared library carries
> > its own visiblity baggage
> >
> > https://github.com/boostorg/filesystem/blob/e268f557df657079be5ba4c19ab0d41ce66b6fb3/include/boost/filesystem/config.hpp
> >
> > My general point stands in any case, that we've encountered some
> > rottenness w.r.t. cross-DLL symbol management and we need to confront
> > the issues and decide how to deal with them.
> >
> > > So without deeper analysis I'm a bit worried about this proposal.
> > >
> > > Regards
> > >
> > > Antoine.

Re: [DISCUSS][C++] Rethinking our current C++ shared library (.so / .dll) approach

[Neal Richardson]

Just to be clear: this is not part of the 0.15 release discussion, right?

On Thu, Sep 12, 2019 at 1:29 PM Wes McKinney  wrote:
>
> On Thu, Sep 12, 2019 at 3:14 PM Antoine Pitrou  wrote:
> >
> >
> > Le 12/09/2019 à 20:14, Wes McKinney a écrit :
> > > hi folks,
> > >
> > > I wanted to share some concerns that I have about our current
> > > trajectory with regards to producing shared libraries from the Arrow
> > > build system.
> > >
> > > Currently, a comprehensive build produces many shared libraries:
> > >
> > > * libarrow
> > > * libarrow_dataset
> > > * libarrow_flight
> > > * libarrow_python
> > > * libgandiva
> > > * libparquet
> > > * libplasma
> >
> > Each library pulls its own set of non-trivial third-party dependencies.
> >  For example, libarrow_flight includes gRPC, libcurl and OpenSSL as far
> > as I remember.
> >
> > Worse, libarrow_cuda depends on the NVidia CUDA libraries.  That's a
> > *huge* dependency (and may also have runtime costs in addition to
> > installation / deployment annoyances).
> >
> > Some packagers may also want to distribute some sublibraries separately
> > (Linux distros especially tend to like having many small, focussed
> > shared libraries that they can distribute as separate, dependent
> > packages) - but that's not my business and I'll let them speak up ;-)
> >
>
> It's a fair point. Another possibility is to be more Boost-like in the
> approach to the library structure. But there are costs involved with
> this, e.g. each Boost component that produces a shared library carries
> its own visiblity baggage
>
> https://github.com/boostorg/filesystem/blob/e268f557df657079be5ba4c19ab0d41ce66b6fb3/include/boost/filesystem/config.hpp
>
> My general point stands in any case, that we've encountered some
> rottenness w.r.t. cross-DLL symbol management and we need to confront
> the issues and decide how to deal with them.
>
> > So without deeper analysis I'm a bit worried about this proposal.
> >
> > Regards
> >
> > Antoine.

Re: [DISCUSS][C++] Rethinking our current C++ shared library (.so / .dll) approach

[Wes McKinney]

On Thu, Sep 12, 2019 at 3:14 PM Antoine Pitrou  wrote:
>
>
> Le 12/09/2019 à 20:14, Wes McKinney a écrit :
> > hi folks,
> >
> > I wanted to share some concerns that I have about our current
> > trajectory with regards to producing shared libraries from the Arrow
> > build system.
> >
> > Currently, a comprehensive build produces many shared libraries:
> >
> > * libarrow
> > * libarrow_dataset
> > * libarrow_flight
> > * libarrow_python
> > * libgandiva
> > * libparquet
> > * libplasma
>
> Each library pulls its own set of non-trivial third-party dependencies.
>  For example, libarrow_flight includes gRPC, libcurl and OpenSSL as far
> as I remember.
>
> Worse, libarrow_cuda depends on the NVidia CUDA libraries.  That's a
> *huge* dependency (and may also have runtime costs in addition to
> installation / deployment annoyances).
>
> Some packagers may also want to distribute some sublibraries separately
> (Linux distros especially tend to like having many small, focussed
> shared libraries that they can distribute as separate, dependent
> packages) - but that's not my business and I'll let them speak up ;-)
>

It's a fair point. Another possibility is to be more Boost-like in the
approach to the library structure. But there are costs involved with
this, e.g. each Boost component that produces a shared library carries
its own visiblity baggage

https://github.com/boostorg/filesystem/blob/e268f557df657079be5ba4c19ab0d41ce66b6fb3/include/boost/filesystem/config.hpp

My general point stands in any case, that we've encountered some
rottenness w.r.t. cross-DLL symbol management and we need to confront
the issues and decide how to deal with them.

> So without deeper analysis I'm a bit worried about this proposal.
>
> Regards
>
> Antoine.

Re: [DISCUSS][C++] Rethinking our current C++ shared library (.so / .dll) approach

[Antoine Pitrou]

Le 12/09/2019 à 20:14, Wes McKinney a écrit :
> hi folks,
> 
> I wanted to share some concerns that I have about our current
> trajectory with regards to producing shared libraries from the Arrow
> build system.
> 
> Currently, a comprehensive build produces many shared libraries:
> 
> * libarrow
> * libarrow_dataset
> * libarrow_flight
> * libarrow_python
> * libgandiva
> * libparquet
> * libplasma

Each library pulls its own set of non-trivial third-party dependencies.
 For example, libarrow_flight includes gRPC, libcurl and OpenSSL as far
as I remember.

Worse, libarrow_cuda depends on the NVidia CUDA libraries.  That's a
*huge* dependency (and may also have runtime costs in addition to
installation / deployment annoyances).

Some packagers may also want to distribute some sublibraries separately
(Linux distros especially tend to like having many small, focussed
shared libraries that they can distribute as separate, dependent
packages) - but that's not my business and I'll let them speak up ;-)

So without deeper analysis I'm a bit worried about this proposal.

Regards

Antoine.

[jira] [Created] (ARROW-6553) [C#] Decide how to read message lengths - little-endian or machine dependent

[Eric Erhardt (Jira)]

Eric Erhardt created ARROW-6553:
---

 Summary: [C#] Decide how to read message lengths - little-endian 
or machine dependent
 Key: ARROW-6553
 URL: https://issues.apache.org/jira/browse/ARROW-6553
 Project: Apache Arrow
  Issue Type: Bug
  Components: C#
Reporter: Eric Erhardt


See the discussion 
[here|[https://github.com/apache/arrow/pull/5280#discussion_r323896532]]. We 
are currently reading message lengths using machine dependent endianness. 
Should this be changed to little-endian all the time?

It appears the C++ implementation does this same thing - use machine dependent 
endianness.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [C++] Compilation error when creating a Tuple with optional String

[Omer F. Ozarslan]

I'm looking into it. Will submit PR today.

On Thu, Sep 12, 2019 at 11:32 AM Ben Kietzman  wrote:
>
> It looks like a tuple constructor is choking on implicit conversion from 
> string literal (char[6]) to boost::optional:
> https://travis-ci.org/ursa-labs/crossbow/builds/582833123#L1161
>
> That constructor in the 4.8.2 release of GCC requires that parameters be 
> *convertible* to the corresponding element type (and not just that the 
> element be constructible from the parameter)
> https://github.com/gcc-mirror/gcc/blob/gcc-4_8_2-release/libstdc++-v3/include/std/tuple#L400-L405
>
> Locally,
>   // static_assert(std::is_convertible boost::optional>::value, "FAILS");
>   static_assert(std::is_convertible boost::optional>::value, "fine");
> So I think this can be resolved by wrapping string literal arguments to tuple 
> constructors with std::string to ensure they are convertible.
>
> I'll create a JIRA
>
> On Thu, Sep 12, 2019 at 8:09 AM Prudhvi Porandla  wrote:
>>
>> Hi,
>>
>> Gandiva trusty 
>> job(https://travis-ci.org/ursa-labs/crossbow/builds/582833123) is failing 
>> with this commit (ARROW-6326: [C++] Nullable fields when converting 
>> std::tuple to Table  
>> ).
>> The compilation is successful if the optional string column is removed from 
>> boost_optional_types_tuple(see line 36 of cpp/src/arrow/stl_test.cc).
>>
>> Could someone help me understand this error?
>>
>>
>> Thanks,
>> Prudhvi
>>
>>

Re: [DISCUSS][C++] Rethinking our current C++ shared library (.so / .dll) approach

[Wes McKinney]

One thing I forgot to mention:

One of the things driving the creation of new shared libraries is
interdependencies. For example:

libarrow -> libparquet
libarrow -> libarrow_dataset
libparquet -> libarrow_dataset

With the modular LLVM-like approach this issue goes away.

On Thu, Sep 12, 2019 at 1:16 PM Wes McKinney  wrote:
>
> I forgot to add the link to the LLVM library listing
>
> https://gist.github.com/wesm/d13c2844db0c19477e8ee5c95e36a0dc
>
> On Thu, Sep 12, 2019 at 1:14 PM Wes McKinney  wrote:
> >
> > hi folks,
> >
> > I wanted to share some concerns that I have about our current
> > trajectory with regards to producing shared libraries from the Arrow
> > build system.
> >
> > Currently, a comprehensive build produces many shared libraries:
> >
> > * libarrow
> > * libarrow_dataset
> > * libarrow_flight
> > * libarrow_python
> > * libgandiva
> > * libparquet
> > * libplasma
> >
> > There are some others. There are a number of problems with the current 
> > approach:
> >
> > * Each DLL needs its own set of "visibility" macros to control the use
> > of __declspec(dllimport/dllexport) on Windows, which is necessary to
> > instruct the import or export of symbols between DLLs on Windows. See
> > e.g. 
> > https://github.com/apache/arrow/blob/master/cpp/src/arrow/flight/visibility.h
> >
> > * Templates instantiated in one DLL may cause a violation of the One
> > Definition Rule during linking (we lost at least a day of work time
> > collectively to issues around this in ARROW-6244). It is good to be
> > able to share common template interfaces in general
> >
> > * Statically-linked dependencies in one shared lib may need to be
> > statically linked into another library. For example, libgandiva
> > statically links parts of LLVM, but we will likely have some other
> > code that makes use of LLVM for other purposes (it has been discussed
> > in the context of Avro parsing)
> >
> > Overall, my preferred solution to these issues is to move to a similar
> > approach to what the LLVM project does. To help understand, let me
> > have you first look at the libraries that come from the llvm-7-dev
> > package on Ubuntu
> >
> > Here we have a collection of static "module" libraries that implement
> > different parts of the LLVM platform. Finally, a _single_ shared
> > library libLLVM-7.so is produced.
> >
> > I think we should do the same thing in Apache Arrow. So we only ever
> > will produce a single shared library from the build. We can
> > additionally make the "name" of this shared library configurable to
> > suit different needs. For example, the default name could be simply
> > "libarrow.so" or something. But if someone wants to produce a
> > barebones Parquet shared library they can override the name to create
> > a "libparquet.so" that contains only the "libarrow_core.a" and
> > "libarrow_io.a" symbols needed for reading Parquet files.
> >
> > This would have additional benefits:
> >
> > * Use the same visibility macros for all exported C++ symbols, rather
> > than having to define DLL-specific visibility
> >
> > * Improved modularization of builds and linking for third party users,
> > similar to the way that LLVM's modular linking works, see the way that
> > Gandiva requests specific components from LLVM to use for static
> > linking 
> > https://github.com/apache/arrow/blob/master/cpp/cmake_modules/FindLLVM.cmake#L53
> >
> > * Net simpler linking and deployment. Only one shared library to deal with
> >
> > There are some drawbacks, however:
> >
> > * Our C++ Linux packaging approach would need to be changed to be more
> > LLVM-like (a single .deb/.yum package containing the C++ platform
> > rather than many packages as now)
> >
> > Interested to hear from other C++ developers.
> >
> > Thanks
> > Wes

Re: [DISCUSS][C++] Rethinking our current C++ shared library (.so / .dll) approach

[Wes McKinney]

I forgot to add the link to the LLVM library listing

https://gist.github.com/wesm/d13c2844db0c19477e8ee5c95e36a0dc

On Thu, Sep 12, 2019 at 1:14 PM Wes McKinney  wrote:
>
> hi folks,
>
> I wanted to share some concerns that I have about our current
> trajectory with regards to producing shared libraries from the Arrow
> build system.
>
> Currently, a comprehensive build produces many shared libraries:
>
> * libarrow
> * libarrow_dataset
> * libarrow_flight
> * libarrow_python
> * libgandiva
> * libparquet
> * libplasma
>
> There are some others. There are a number of problems with the current 
> approach:
>
> * Each DLL needs its own set of "visibility" macros to control the use
> of __declspec(dllimport/dllexport) on Windows, which is necessary to
> instruct the import or export of symbols between DLLs on Windows. See
> e.g. 
> https://github.com/apache/arrow/blob/master/cpp/src/arrow/flight/visibility.h
>
> * Templates instantiated in one DLL may cause a violation of the One
> Definition Rule during linking (we lost at least a day of work time
> collectively to issues around this in ARROW-6244). It is good to be
> able to share common template interfaces in general
>
> * Statically-linked dependencies in one shared lib may need to be
> statically linked into another library. For example, libgandiva
> statically links parts of LLVM, but we will likely have some other
> code that makes use of LLVM for other purposes (it has been discussed
> in the context of Avro parsing)
>
> Overall, my preferred solution to these issues is to move to a similar
> approach to what the LLVM project does. To help understand, let me
> have you first look at the libraries that come from the llvm-7-dev
> package on Ubuntu
>
> Here we have a collection of static "module" libraries that implement
> different parts of the LLVM platform. Finally, a _single_ shared
> library libLLVM-7.so is produced.
>
> I think we should do the same thing in Apache Arrow. So we only ever
> will produce a single shared library from the build. We can
> additionally make the "name" of this shared library configurable to
> suit different needs. For example, the default name could be simply
> "libarrow.so" or something. But if someone wants to produce a
> barebones Parquet shared library they can override the name to create
> a "libparquet.so" that contains only the "libarrow_core.a" and
> "libarrow_io.a" symbols needed for reading Parquet files.
>
> This would have additional benefits:
>
> * Use the same visibility macros for all exported C++ symbols, rather
> than having to define DLL-specific visibility
>
> * Improved modularization of builds and linking for third party users,
> similar to the way that LLVM's modular linking works, see the way that
> Gandiva requests specific components from LLVM to use for static
> linking 
> https://github.com/apache/arrow/blob/master/cpp/cmake_modules/FindLLVM.cmake#L53
>
> * Net simpler linking and deployment. Only one shared library to deal with
>
> There are some drawbacks, however:
>
> * Our C++ Linux packaging approach would need to be changed to be more
> LLVM-like (a single .deb/.yum package containing the C++ platform
> rather than many packages as now)
>
> Interested to hear from other C++ developers.
>
> Thanks
> Wes

[DISCUSS][C++] Rethinking our current C++ shared library (.so / .dll) approach

[Wes McKinney]

hi folks,

I wanted to share some concerns that I have about our current
trajectory with regards to producing shared libraries from the Arrow
build system.

Currently, a comprehensive build produces many shared libraries:

* libarrow
* libarrow_dataset
* libarrow_flight
* libarrow_python
* libgandiva
* libparquet
* libplasma

There are some others. There are a number of problems with the current approach:

* Each DLL needs its own set of "visibility" macros to control the use
of __declspec(dllimport/dllexport) on Windows, which is necessary to
instruct the import or export of symbols between DLLs on Windows. See
e.g. 
https://github.com/apache/arrow/blob/master/cpp/src/arrow/flight/visibility.h

* Templates instantiated in one DLL may cause a violation of the One
Definition Rule during linking (we lost at least a day of work time
collectively to issues around this in ARROW-6244). It is good to be
able to share common template interfaces in general

* Statically-linked dependencies in one shared lib may need to be
statically linked into another library. For example, libgandiva
statically links parts of LLVM, but we will likely have some other
code that makes use of LLVM for other purposes (it has been discussed
in the context of Avro parsing)

Overall, my preferred solution to these issues is to move to a similar
approach to what the LLVM project does. To help understand, let me
have you first look at the libraries that come from the llvm-7-dev
package on Ubuntu

Here we have a collection of static "module" libraries that implement
different parts of the LLVM platform. Finally, a _single_ shared
library libLLVM-7.so is produced.

I think we should do the same thing in Apache Arrow. So we only ever
will produce a single shared library from the build. We can
additionally make the "name" of this shared library configurable to
suit different needs. For example, the default name could be simply
"libarrow.so" or something. But if someone wants to produce a
barebones Parquet shared library they can override the name to create
a "libparquet.so" that contains only the "libarrow_core.a" and
"libarrow_io.a" symbols needed for reading Parquet files.

This would have additional benefits:

* Use the same visibility macros for all exported C++ symbols, rather
than having to define DLL-specific visibility

* Improved modularization of builds and linking for third party users,
similar to the way that LLVM's modular linking works, see the way that
Gandiva requests specific components from LLVM to use for static
linking 
https://github.com/apache/arrow/blob/master/cpp/cmake_modules/FindLLVM.cmake#L53

* Net simpler linking and deployment. Only one shared library to deal with

There are some drawbacks, however:

* Our C++ Linux packaging approach would need to be changed to be more
LLVM-like (a single .deb/.yum package containing the C++ platform
rather than many packages as now)

Interested to hear from other C++ developers.

Thanks
Wes

[jira] [Created] (ARROW-6552) [C++] boost::optional in STL test fails compiling in gcc 4.8.2

[Omer Ozarslan (Jira)]

Omer Ozarslan created ARROW-6552:


 Summary: [C++] boost::optional in STL test fails compiling in gcc 
4.8.2
 Key: ARROW-6552
 URL: https://issues.apache.org/jira/browse/ARROW-6552
 Project: Apache Arrow
  Issue Type: Bug
  Components: C++
Reporter: Omer Ozarslan


Quoting [~bkietz] from mailgroup:
{code:java}
a tuple constructor is choking on implicit conversion from
string literal (char[6]) to boost::optional{code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-09-12-0

[Krisztián Szűcs]

I was looking for the ticket, but Wes has sent it already.

`docker-r`: should be fixed by Neal's PR
https://github.com/apache/arrow/pull/5357

The two remaining ones are the spark integration and gandiva-jar.

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-09-12-0

[Wes McKinney]

I opened

https://github.com/dask/dask/issues/5398

The Dask integration test issue will have to be fixed upstream. I opened

https://issues.apache.org/jira/browse/ARROW-6551

so it's tracked on our end

On Thu, Sep 12, 2019 at 12:04 PM Wes McKinney  wrote:
>
> The C++ filter expressions PR seems to have broken some packaging builds
>
> https://issues.apache.org/jira/browse/ARROW-6550
>
> On Thu, Sep 12, 2019 at 12:01 PM Crossbow  wrote:
> >
> >
> > Arrow Build Report for Job nightly-2019-09-12-0
> >
> > All tasks: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0
> >
> > Failed Tasks:
> > - wheel-manylinux1-cp37m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp37m
> > - docker-dask-integration:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-dask-integration
> > - docker-spark-integration:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-spark-integration
> > - wheel-manylinux1-cp27m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27m
> > - docker-r:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-r
> > - gandiva-jar-trusty:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-trusty
> > - wheel-manylinux1-cp35m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp35m
> > - wheel-manylinux1-cp36m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp36m
> > - wheel-manylinux1-cp27mu:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27mu
> >
> > Pending Tasks:
> > - wheel-win-cp35m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp35m
> >
> > Succeeded Tasks:
> > - docker-java:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-java
> > - conda-osx-clang-py37:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-osx-clang-py37
> > - ubuntu-xenial:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-xenial
> > - wheel-manylinux2010-cp37m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp37m
> > - docker-cpp-static-only:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-cpp-static-only
> > - docker-hdfs-integration:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-hdfs-integration
> > - wheel-manylinux2010-cp27m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp27m
> > - docker-turbodbc-integration:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-turbodbc-integration
> > - ubuntu-bionic:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-bionic
> > - centos-6:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-centos-6
> > - wheel-win-cp36m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp36m
> > - wheel-osx-cp27m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp27m
> > - gandiva-jar-osx:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-osx
> > - docker-go:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-go
> > - docker-python-3.7:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-python-3.7
> > - docker-pandas-master:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-pandas-master
> > - docker-rust:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-rust
> > - wheel-osx-cp35m:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp35m
> > - conda-win-vs2015-py36:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-win-vs2015-py36
> > - docker-c_glib:
> >   URL: 
> > https://github.com/ursa-labs/crossbow/branches/all?

[jira] [Created] (ARROW-6551) [Python] Dask Parquet integration test failure

[Wes McKinney (Jira)]

Wes McKinney created ARROW-6551:
---

 Summary: [Python] Dask Parquet integration test failure
 Key: ARROW-6551
 URL: https://issues.apache.org/jira/browse/ARROW-6551
 Project: Apache Arrow
  Issue Type: Bug
  Components: Continuous Integration, Python
Reporter: Wes McKinney
 Fix For: 0.15.0


See upstream issue https://github.com/dask/dask/issues/5398




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-09-12-0

[Krisztián Szűcs]

Hi,

The pending wheel-win-cp35m task is actually passing, but the GitHub status
hasn't been updated by AppVeyor.
The manylinux failures are caused by a recent change.

On Thu, Sep 12, 2019 at 7:01 PM Crossbow  wrote:

>
> Arrow Build Report for Job nightly-2019-09-12-0
>
> All tasks:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0
>
> Failed Tasks:
> - wheel-manylinux1-cp37m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp37m
> - docker-dask-integration:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-dask-integration
> - docker-spark-integration:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-spark-integration
> - wheel-manylinux1-cp27m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27m
> - docker-r:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-r
> - gandiva-jar-trusty:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-trusty
> - wheel-manylinux1-cp35m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp35m
> - wheel-manylinux1-cp36m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp36m
> - wheel-manylinux1-cp27mu:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27mu
>
> Pending Tasks:
> - wheel-win-cp35m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp35m
>
> Succeeded Tasks:
> - docker-java:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-java
> - conda-osx-clang-py37:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-osx-clang-py37
> - ubuntu-xenial:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-xenial
> - wheel-manylinux2010-cp37m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp37m
> - docker-cpp-static-only:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-cpp-static-only
> - docker-hdfs-integration:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-hdfs-integration
> - wheel-manylinux2010-cp27m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp27m
> - docker-turbodbc-integration:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-turbodbc-integration
> - ubuntu-bionic:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-bionic
> - centos-6:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-centos-6
> - wheel-win-cp36m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp36m
> - wheel-osx-cp27m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp27m
> - gandiva-jar-osx:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-osx
> - docker-go:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-go
> - docker-python-3.7:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-python-3.7
> - docker-pandas-master:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-pandas-master
> - docker-rust:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-rust
> - wheel-osx-cp35m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp35m
> - conda-win-vs2015-py36:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-win-vs2015-py36
> - docker-c_glib:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-c_glib
> - conda-osx-clang-py36:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-osx-clang-py36
> - wheel-manylinux2010-cp35m:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp35m
> - wheel-manylinux2010-cp27mu:
>   URL:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-1

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-09-12-0

[Wes McKinney]

The C++ filter expressions PR seems to have broken some packaging builds

https://issues.apache.org/jira/browse/ARROW-6550

On Thu, Sep 12, 2019 at 12:01 PM Crossbow  wrote:
>
>
> Arrow Build Report for Job nightly-2019-09-12-0
>
> All tasks: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0
>
> Failed Tasks:
> - wheel-manylinux1-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp37m
> - docker-dask-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-dask-integration
> - docker-spark-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-spark-integration
> - wheel-manylinux1-cp27m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27m
> - docker-r:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-r
> - gandiva-jar-trusty:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-trusty
> - wheel-manylinux1-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp35m
> - wheel-manylinux1-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp36m
> - wheel-manylinux1-cp27mu:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27mu
>
> Pending Tasks:
> - wheel-win-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp35m
>
> Succeeded Tasks:
> - docker-java:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-java
> - conda-osx-clang-py37:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-osx-clang-py37
> - ubuntu-xenial:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-xenial
> - wheel-manylinux2010-cp37m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp37m
> - docker-cpp-static-only:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-cpp-static-only
> - docker-hdfs-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-hdfs-integration
> - wheel-manylinux2010-cp27m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp27m
> - docker-turbodbc-integration:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-turbodbc-integration
> - ubuntu-bionic:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-bionic
> - centos-6:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-centos-6
> - wheel-win-cp36m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp36m
> - wheel-osx-cp27m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp27m
> - gandiva-jar-osx:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-osx
> - docker-go:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-go
> - docker-python-3.7:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-python-3.7
> - docker-pandas-master:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-pandas-master
> - docker-rust:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-rust
> - wheel-osx-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp35m
> - conda-win-vs2015-py36:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-win-vs2015-py36
> - docker-c_glib:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-c_glib
> - conda-osx-clang-py36:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-osx-clang-py36
> - wheel-manylinux2010-cp35m:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp35m
> - wheel-manylinux2010-cp27mu:
>   URL: 
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travi

[jira] [Created] (ARROW-6550) [C++] Filter expressions PR failing manylinux package builds

[Wes McKinney (Jira)]

Wes McKinney created ARROW-6550:
---

 Summary: [C++] Filter expressions PR failing manylinux package 
builds
 Key: ARROW-6550
 URL: https://issues.apache.org/jira/browse/ARROW-6550
 Project: Apache Arrow
  Issue Type: Bug
  Components: C++
Reporter: Wes McKinney
Assignee: Benjamin Kietzman
 Fix For: 0.15.0


https://travis-ci.org/ursa-labs/crossbow/builds/584022060?utm_source=github_status&utm_medium=notification

{code}
In file included from /arrow/cpp/src/arrow/dataset/filter.cc:18:0:
/arrow/cpp/src/arrow/dataset/filter.h:398:24: error: missing space between ‘""’ 
and suffix identifier
 inline FieldExpression operator""_(const char* name, size_t name_length) {
^
/arrow/cpp/src/arrow/dataset/filter.cc:942:1: error: expected ‘}’ at end of 
input
 }  // namespace arrow
 ^
/arrow/cpp/src/arrow/dataset/filter.cc:942:1: error: expected ‘}’ at end of 
input
/arrow/cpp/src/arrow/dataset/filter.cc:942:1: error: expected ‘}’ at end of 
input
[23/271] Performing build step for 'jemalloc_ep'
{code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[NIGHTLY] Arrow Build Report for Job nightly-2019-09-12-0

[Crossbow]

Arrow Build Report for Job nightly-2019-09-12-0

All tasks: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0

Failed Tasks:
- wheel-manylinux1-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp37m
- docker-dask-integration:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-dask-integration
- docker-spark-integration:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-spark-integration
- wheel-manylinux1-cp27m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27m
- docker-r:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-r
- gandiva-jar-trusty:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-trusty
- wheel-manylinux1-cp35m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp35m
- wheel-manylinux1-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp36m
- wheel-manylinux1-cp27mu:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux1-cp27mu

Pending Tasks:
- wheel-win-cp35m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp35m

Succeeded Tasks:
- docker-java:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-java
- conda-osx-clang-py37:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-osx-clang-py37
- ubuntu-xenial:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-xenial
- wheel-manylinux2010-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp37m
- docker-cpp-static-only:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-cpp-static-only
- docker-hdfs-integration:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-hdfs-integration
- wheel-manylinux2010-cp27m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp27m
- docker-turbodbc-integration:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-turbodbc-integration
- ubuntu-bionic:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-ubuntu-bionic
- centos-6:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-centos-6
- wheel-win-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-appveyor-wheel-win-cp36m
- wheel-osx-cp27m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp27m
- gandiva-jar-osx:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-gandiva-jar-osx
- docker-go:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-go
- docker-python-3.7:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-python-3.7
- docker-pandas-master:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-pandas-master
- docker-rust:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-rust
- wheel-osx-cp35m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp35m
- conda-win-vs2015-py36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-win-vs2015-py36
- docker-c_glib:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-c_glib
- conda-osx-clang-py36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-azure-conda-osx-clang-py36
- wheel-manylinux2010-cp35m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp35m
- wheel-manylinux2010-cp27mu:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-manylinux2010-cp27mu
- wheel-osx-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-travis-wheel-osx-cp36m
- docker-lint:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circle-docker-lint
- docker-cpp-release:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-12-0-circl

Re: [C++] Compilation error when creating a Tuple with optional String

[Ben Kietzman]

It looks like a tuple constructor is choking on implicit conversion from
string literal (char[6]) to boost::optional:
https://travis-ci.org/ursa-labs/crossbow/builds/582833123#L1161

That constructor in the 4.8.2 release of GCC requires that parameters be
*convertible* to the corresponding element type (and not just that the
element be constructible from the parameter)
https://github.com/gcc-mirror/gcc/blob/gcc-4_8_2-release/libstdc++-v3/include/std/tuple#L400-L405

Locally,
  // static_assert(std::is_convertible>::value, "FAILS");
  static_assert(std::is_convertible>::value, "fine");
So I think this can be resolved by wrapping string literal arguments to
tuple constructors with std::string to ensure they are convertible.

I'll create a JIRA

On Thu, Sep 12, 2019 at 8:09 AM Prudhvi Porandla  wrote:

> Hi,
>
> Gandiva trusty job(
> https://travis-ci.org/ursa-labs/crossbow/builds/582833123) is failing
> with this commit (ARROW-6326: [C++] Nullable fields when converting
> std::tuple to Table  <
> https://github.com/apache/arrow/commit/d6bc0b6f5975304367000e3090e09d4cf8af1fb9
> >).
> The compilation is successful if the optional string column is removed
> from boost_optional_types_tuple(see line 36 of cpp/src/arrow/stl_test.cc).
>
> Could someone help me understand this error?
>
>
> Thanks,
> Prudhvi
>
>
>

Re: [Java] CI test failures

[Wes McKinney]

Does upgrading maven-surefire-plugin help?

On Thu, Sep 12, 2019 at 8:29 AM Fan Liya  wrote:
>
> I have tried many times locally, with different values of forkCount, but
> failed to reproduce the error.
>
> Best,
> Liya Fan
>
> On Thu, Sep 12, 2019 at 4:21 PM Fan Liya  wrote:
>
> > It seems the problem occurred when the sure-fire plugin forks:
> >
> > [ERROR] Caused by:
> > org.apache.maven.surefire.booter.SurefireBooterForkException: Error
> > occurred in starting fork, check output in log
> >
> > So we need to adjust the forkCount parameter in the pom.xml file?
> >
> > Best,
> > Liya Fan
> >
> > On Thu, Sep 12, 2019 at 3:10 PM Micah Kornfield 
> > wrote:
> >
> >> I did not make much progress, I will spend some more time tomorrow, if no
> >> one picks it up by then.
> >>
> >> On Wed, Sep 11, 2019 at 10:48 PM Micah Kornfield 
> >> wrote:
> >>
> >> > I will try to take look the next couple of hours to see if I can fix it
> >> > quickly.
> >> >
> >> > On Wed, Sep 11, 2019 at 4:54 AM Antoine Pitrou 
> >> wrote:
> >> >
> >> >>
> >> >> Hello,
> >> >>
> >> >> Some Travis-CI builds are failing because of Java issues.  It would be
> >> >> good if a Java contributor or maintainer could take a look ASAP.
> >> >> https://issues.apache.org/jira/browse/ARROW-6509
> >> >>
> >> >> Thanks
> >> >>
> >> >> Antoine.
> >> >>
> >> >
> >>
> >

[jira] [Created] (ARROW-6549) [C++] Switch back to latest jemalloc 5.x

[Antoine Pitrou (Jira)]

Antoine Pitrou created ARROW-6549:
-

 Summary: [C++] Switch back to latest jemalloc 5.x
 Key: ARROW-6549
 URL: https://issues.apache.org/jira/browse/ARROW-6549
 Project: Apache Arrow
  Issue Type: Wish
  Components: C++
Reporter: Antoine Pitrou


In ARROW-6478, we switched back to jemallox 4.x to fix a large performance 
regression in micro-benchmarks which appeared when bumping the vendored 
jemalloc version to 5.2.0.

Since then, the issue was 
[discussed|https://github.com/jemalloc/jemalloc/issues/1621] with the jemalloc 
maintainers and we found out that customizing some options allowed regaining 
all (or most) of the performance on micro-benchmarks with 5.2.x.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [Java] CI test failures

[Fan Liya]

I have tried many times locally, with different values of forkCount, but
failed to reproduce the error.

Best,
Liya Fan

On Thu, Sep 12, 2019 at 4:21 PM Fan Liya  wrote:

> It seems the problem occurred when the sure-fire plugin forks:
>
> [ERROR] Caused by:
> org.apache.maven.surefire.booter.SurefireBooterForkException: Error
> occurred in starting fork, check output in log
>
> So we need to adjust the forkCount parameter in the pom.xml file?
>
> Best,
> Liya Fan
>
> On Thu, Sep 12, 2019 at 3:10 PM Micah Kornfield 
> wrote:
>
>> I did not make much progress, I will spend some more time tomorrow, if no
>> one picks it up by then.
>>
>> On Wed, Sep 11, 2019 at 10:48 PM Micah Kornfield 
>> wrote:
>>
>> > I will try to take look the next couple of hours to see if I can fix it
>> > quickly.
>> >
>> > On Wed, Sep 11, 2019 at 4:54 AM Antoine Pitrou 
>> wrote:
>> >
>> >>
>> >> Hello,
>> >>
>> >> Some Travis-CI builds are failing because of Java issues.  It would be
>> >> good if a Java contributor or maintainer could take a look ASAP.
>> >> https://issues.apache.org/jira/browse/ARROW-6509
>> >>
>> >> Thanks
>> >>
>> >> Antoine.
>> >>
>> >
>>
>

Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte Flatbuffer alignment requirements (2nd vote)

[Sebastien Binet]

hi there,

On Thu, Sep 12, 2019 at 12:45 AM Wes McKinney  wrote:

> Thanks Bryan.
>
> I merged the Java patch with the EOS change and submitted a C++ patch
> which also updates the specification
>
> https://github.com/apache/arrow/pull/5361
>
> Let me know when the JS or C# patches are ready to go and I can merge
> those.
>
> I updated https://issues.apache.org/jira/browse/ARROW-6545 to track
> the Go change corresponding to this
>

done. (I think)

-s

[C++] Compilation error when creating a Tuple with optional String

[Prudhvi Porandla]

Hi,

Gandiva trusty job(https://travis-ci.org/ursa-labs/crossbow/builds/582833123) 
is failing with this commit (ARROW-6326: [C++] Nullable fields when converting 
std::tuple to Table  
).
The compilation is successful if the optional string column is removed from 
boost_optional_types_tuple(see line 36 of cpp/src/arrow/stl_test.cc).

Could someone help me understand this error?


Thanks,
Prudhvi

[jira] [Created] (ARROW-6548) [Python] consistently handle conversion of all-NaN arrays across types

[Joris Van den Bossche (Jira)]

Joris Van den Bossche created ARROW-6548:


 Summary: [Python] consistently handle conversion of all-NaN arrays 
across types
 Key: ARROW-6548
 URL: https://issues.apache.org/jira/browse/ARROW-6548
 Project: Apache Arrow
  Issue Type: Bug
  Components: Python
Reporter: Joris Van den Bossche


In ARROW-5682 (https://github.com/apache/arrow/pull/5333), next to fixing 
actual conversion bugs, I added the ability to convert all-NaN float arrays 
when converting to string type (and only with {{from_pandas=True}}). So this 
now works:

{code}
>>> pa.array(np.array([np.nan, np.nan], dtype=float), type=pa.string())

[
  null,
  null
]
{code}

However, I only added this for string type (and it already works for float and 
int types). If we are happy with this behaviour, we should also add it for 
other types.




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [Java] CI test failures

[Fan Liya]

It seems the problem occurred when the sure-fire plugin forks:

[ERROR] Caused by:
org.apache.maven.surefire.booter.SurefireBooterForkException: Error
occurred in starting fork, check output in log

So we need to adjust the forkCount parameter in the pom.xml file?

Best,
Liya Fan

On Thu, Sep 12, 2019 at 3:10 PM Micah Kornfield 
wrote:

> I did not make much progress, I will spend some more time tomorrow, if no
> one picks it up by then.
>
> On Wed, Sep 11, 2019 at 10:48 PM Micah Kornfield 
> wrote:
>
> > I will try to take look the next couple of hours to see if I can fix it
> > quickly.
> >
> > On Wed, Sep 11, 2019 at 4:54 AM Antoine Pitrou 
> wrote:
> >
> >>
> >> Hello,
> >>
> >> Some Travis-CI builds are failing because of Java issues.  It would be
> >> good if a Java contributor or maintainer could take a look ASAP.
> >> https://issues.apache.org/jira/browse/ARROW-6509
> >>
> >> Thanks
> >>
> >> Antoine.
> >>
> >
>

Re: [Java] CI test failures

[Micah Kornfield]

I did not make much progress, I will spend some more time tomorrow, if no
one picks it up by then.

On Wed, Sep 11, 2019 at 10:48 PM Micah Kornfield 
wrote:

> I will try to take look the next couple of hours to see if I can fix it
> quickly.
>
> On Wed, Sep 11, 2019 at 4:54 AM Antoine Pitrou  wrote:
>
>>
>> Hello,
>>
>> Some Travis-CI builds are failing because of Java issues.  It would be
>> good if a Java contributor or maintainer could take a look ASAP.
>> https://issues.apache.org/jira/browse/ARROW-6509
>>
>> Thanks
>>
>> Antoine.
>>
>