[jira] [Created] (ARROW-5919) [R] Add nightly tests for building r-arrow with dependencies from conda-forge
Uwe L. Korn created ARROW-5919: -- Summary: [R] Add nightly tests for building r-arrow with dependencies from conda-forge Key: ARROW-5919 URL: https://issues.apache.org/jira/browse/ARROW-5919 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 1.0.0 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (ARROW-5265) [Python/CI] Add integration test with kartothek
Uwe L. Korn created ARROW-5265: -- Summary: [Python/CI] Add integration test with kartothek Key: ARROW-5265 URL: https://issues.apache.org/jira/browse/ARROW-5265 Project: Apache Arrow Issue Type: Bug Components: Continuous Integration, Python Reporter: Uwe L. Korn Fix For: 0.15.0 https://github.com/JDASoftwareGroup/kartothek is a heavy user of Apache Arrow and thus a good indicator whether we have introduced some breakages in {{pyarrow}}. Thus we should run regular integration tests against it as we do with other libraries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-5074) [C++/Python] When installing into a SYSTEM prefix, RPATHs are not correctly set
Uwe L. Korn created ARROW-5074: -- Summary: [C++/Python] When installing into a SYSTEM prefix, RPATHs are not correctly set Key: ARROW-5074 URL: https://issues.apache.org/jira/browse/ARROW-5074 Project: Apache Arrow Issue Type: Bug Components: C++, Packaging, Python Reporter: Uwe L. Korn When installing the Arrow libraries into a system with a prefix (mostly a conda env), the RPATHs are not correctly set by CMake (there is no RPATH). Thus we need to use {{LD_LIBRARY_PATH}} in consumers. When packages are built using {{conda-build}}, this takes cares of that in its post-processing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4987) [C++] Use orc conda-package on Linux and OSX
Uwe L. Korn created ARROW-4987: -- Summary: [C++] Use orc conda-package on Linux and OSX Key: ARROW-4987 URL: https://issues.apache.org/jira/browse/ARROW-4987 Project: Apache Arrow Issue Type: Bug Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4985) [C++] arrow/testing headers are not installed
Uwe L. Korn created ARROW-4985: -- Summary: [C++] arrow/testing headers are not installed Key: ARROW-4985 URL: https://issues.apache.org/jira/browse/ARROW-4985 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4960) [R] Add crossbow task for r-arrow-feedstock
Uwe L. Korn created ARROW-4960: -- Summary: [R] Add crossbow task for r-arrow-feedstock Key: ARROW-4960 URL: https://issues.apache.org/jira/browse/ARROW-4960 Project: Apache Arrow Issue Type: Improvement Components: Packaging, R Reporter: Uwe L. Korn Fix For: 0.14.0 We also have an R package on conda-forge now: [https://github.com/conda-forge/r-arrow-feedstock] This should be tested using crossbow as we do with the other packages. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4948) [JS] Nightly test failing with "Cannot assign to read only property"
Uwe L. Korn created ARROW-4948: -- Summary: [JS] Nightly test failing with "Cannot assign to read only property" Key: ARROW-4948 URL: https://issues.apache.org/jira/browse/ARROW-4948 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Uwe L. Korn Fix For: JS-0.5.0 See [https://travis-ci.org/kszucs/crossbow/builds/507807857] This can be reproduced using {{docker-compose build js && docker-compose run js}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4946) [C++] Support detection of flatbuffers without FlatbuffersConfig.cmake
Uwe L. Korn created ARROW-4946: -- Summary: [C++] Support detection of flatbuffers without FlatbuffersConfig.cmake Key: ARROW-4946 URL: https://issues.apache.org/jira/browse/ARROW-4946 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 This is not always installed: https://github.com/ContinuumIO/anaconda-issues/issues/10731 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4944) [C++] Raise minimal required thrift-cpp to 0.11 in conda environment
Uwe L. Korn created ARROW-4944: -- Summary: [C++] Raise minimal required thrift-cpp to 0.11 in conda environment Key: ARROW-4944 URL: https://issues.apache.org/jira/browse/ARROW-4944 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 This is mainly for installing one the latest Thrift versions using conda and to speed up installation times a bit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4933) [R] Autodetect Parquet support using pkg-config
Uwe L. Korn created ARROW-4933: -- Summary: [R] Autodetect Parquet support using pkg-config Key: ARROW-4933 URL: https://issues.apache.org/jira/browse/ARROW-4933 Project: Apache Arrow Issue Type: Improvement Components: Packaging, R Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4931) [C++] CMake fails on gRPC ExternalProject
Uwe L. Korn created ARROW-4931: -- Summary: [C++] CMake fails on gRPC ExternalProject Key: ARROW-4931 URL: https://issues.apache.org/jira/browse/ARROW-4931 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 The detection for the {{grpc++}} header location is not working for the ExternalProject as the headers are not installed at configure time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4918) [C++] Add cmake-format to pre-commit
Uwe L. Korn created ARROW-4918: -- Summary: [C++] Add cmake-format to pre-commit Key: ARROW-4918 URL: https://issues.apache.org/jira/browse/ARROW-4918 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4917) [C++] orc_ep fails in cpp-alpine docker
Uwe L. Korn created ARROW-4917: -- Summary: [C++] orc_ep fails in cpp-alpine docker Key: ARROW-4917 URL: https://issues.apache.org/jira/browse/ARROW-4917 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Fix For: 0.13.0 Failure: {code:java} FAILED: c++/src/CMakeFiles/orc.dir/Timezone.cc.o /usr/bin/g++ -Ic++/include -I/build/cpp/orc_ep-prefix/src/orc_ep/c++/include -I/build/cpp/orc_ep-prefix/src/orc_ep/c++/src -Ic++/src -isystem /build/cpp/snappy_ep/src/snappy_ep-install/include -isystem c++/libs/thirdparty/zlib_ep-install/include -isystem c++/libs/thirdparty/lz4_ep-install/include -isystem /arrow/cpp/thirdparty/protobuf_ep-install/include -fdiagnostics-color=always -ggdb -O0 -g -fPIC -std=c++11 -Wall -Wno-unknown-pragmas -Wconversion -Werror -std=c++11 -Wall -Wno-unknown-pragmas -Wconversion -Werror -O0 -g -MD -MT c++/src/CMakeFiles/orc.dir/Timezone.cc.o -MF c++/src/CMakeFiles/orc.dir/Timezone.cc.o.d -o c++/src/CMakeFiles/orc.dir/Timezone.cc.o -c /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc: In member function 'void orc::TimezoneImpl::parseTimeVariants(const unsigned char*, uint64_t, uint64_t, uint64_t, uint64_t)': /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc:748:7: error: 'uint' was not declared in this scope uint nameStart = ptr[variantOffset + 6 * variant + 5]; ^~~~ /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc:748:7: note: suggested alternative: 'rint' uint nameStart = ptr[variantOffset + 6 * variant + 5]; ^~~~ rint /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc:749:11: error: 'nameStart' was not declared in this scope if (nameStart >= nameCount) { ^ /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc:749:11: note: suggested alternative: 'nameCount' if (nameStart >= nameCount) { ^ nameCount /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc:756:59: error: 'nameStart' was not declared in this scope + nameOffset + nameStart); ^ /build/cpp/orc_ep-prefix/src/orc_ep/c++/src/Timezone.cc:756:59: note: suggested alternative: 'nameCount' + nameOffset + nameStart); ^ nameCount{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4916) [C++] Add clang-format to pre-commit hook
Uwe L. Korn created ARROW-4916: -- Summary: [C++] Add clang-format to pre-commit hook Key: ARROW-4916 URL: https://issues.apache.org/jira/browse/ARROW-4916 Project: Apache Arrow Issue Type: Task Components: C++, Continuous Integration Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4909) [CI] Use hadolint to lint Dockerfiles
Uwe L. Korn created ARROW-4909: -- Summary: [CI] Use hadolint to lint Dockerfiles Key: ARROW-4909 URL: https://issues.apache.org/jira/browse/ARROW-4909 Project: Apache Arrow Issue Type: Task Components: Continuous Integration Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4907) [CI] Add docker container to inspect docker context
Uwe L. Korn created ARROW-4907: -- Summary: [CI] Add docker container to inspect docker context Key: ARROW-4907 URL: https://issues.apache.org/jira/browse/ARROW-4907 Project: Apache Arrow Issue Type: Task Components: Continuous Integration Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 The larger the docker context the slower the docker build. Adding a small container to inspect the context helps to curate a good {{.dockerignore}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4904) [C++] Move implementations in arrow/ipc/test-common.h into libarrow_testing
Uwe L. Korn created ARROW-4904: -- Summary: [C++] Move implementations in arrow/ipc/test-common.h into libarrow_testing Key: ARROW-4904 URL: https://issues.apache.org/jira/browse/ARROW-4904 Project: Apache Arrow Issue Type: Task Components: C++ Reporter: Uwe L. Korn Fix For: 0.14.0 Instead of having these all defined as {{static inline}} we should move the implementations into a testing support library. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4903) [C++] Building tests using only static libs not possible
Uwe L. Korn created ARROW-4903: -- Summary: [C++] Building tests using only static libs not possible Key: ARROW-4903 URL: https://issues.apache.org/jira/browse/ARROW-4903 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 We explicitly depend on shared libs in some tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4902) [C++/Flight] Compilation fails due to unreachable code
Uwe L. Korn created ARROW-4902: -- Summary: [C++/Flight] Compilation fails due to unreachable code Key: ARROW-4902 URL: https://issues.apache.org/jira/browse/ARROW-4902 Project: Apache Arrow Issue Type: Improvement Components: C++, FlightRPC Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 clang on OSX is quite picky, we can simply remove this code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4898) [C++] Old versions of FindProtobuf.cmake use ALL-CAPS for variables
Uwe L. Korn created ARROW-4898: -- Summary: [C++] Old versions of FindProtobuf.cmake use ALL-CAPS for variables Key: ARROW-4898 URL: https://issues.apache.org/jira/browse/ARROW-4898 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 We only need to handle {{PROTOBUF_PROTOC_LIBRARY}} vs {{Protobuf_PROTOC_LIBRARY}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4893) [C++] conda packages should use $PREFIX inside of conda-build
Uwe L. Korn created ARROW-4893: -- Summary: [C++] conda packages should use $PREFIX inside of conda-build Key: ARROW-4893 URL: https://issues.apache.org/jira/browse/ARROW-4893 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4891) [C++] ZLIB include directories not added
Uwe L. Korn created ARROW-4891: -- Summary: [C++] ZLIB include directories not added Key: ARROW-4891 URL: https://issues.apache.org/jira/browse/ARROW-4891 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 This causes a failing centos-7 build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4889) [C++] Add STATUS messages for Protobuf in CMake
Uwe L. Korn created ARROW-4889: -- Summary: [C++] Add STATUS messages for Protobuf in CMake Key: ARROW-4889 URL: https://issues.apache.org/jira/browse/ARROW-4889 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 With Protobuf it can easily happen that {{protoc}} and {{libprotobuf}} mismatch. We should have some output about this in CMake to better debug this when users report issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4888) [C++/Python] Test build with conda's defaults channel
Uwe L. Korn created ARROW-4888: -- Summary: [C++/Python] Test build with conda's defaults channel Key: ARROW-4888 URL: https://issues.apache.org/jira/browse/ARROW-4888 Project: Apache Arrow Issue Type: Bug Components: C++, Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.14.0 We mostly use {{conda-forge}} as the developers of Arrow but also have some users that would build with packages from {{defaults}}. As the versions of packages is a bit behind (and sometimes the contents are different), we should also have a docker-test for this channel. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4866) [C++] zstd ExternalProject failing on Windows
Uwe L. Korn created ARROW-4866: -- Summary: [C++] zstd ExternalProject failing on Windows Key: ARROW-4866 URL: https://issues.apache.org/jira/browse/ARROW-4866 Project: Apache Arrow Issue Type: Bug Components: C++, Packaging Reporter: Uwe L. Korn Fix For: 0.13.0 After [https://github.com/apache/arrow/pull/3885|https://github.com/apache/arrow/pull/3885,] the zstd ExternalProject is failing in the Windows builds, see [https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/23063072/job/bd0gom16atlkddtx] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4857) [C++/Python/CI] docker-compose in manylinux1 crossbow jobs too old
Uwe L. Korn created ARROW-4857: -- Summary: [C++/Python/CI] docker-compose in manylinux1 crossbow jobs too old Key: ARROW-4857 URL: https://issues.apache.org/jira/browse/ARROW-4857 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration, Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Nightlies are failing with {{Version in "./docker-compose.yml" is unsupported.}} https://travis-ci.org/kszucs/crossbow/builds/505676630?utm_source=github_status&utm_medium=notification -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4849) [C++] Add docker-compose entry for testing Ubuntu Bionic build with system packages
Uwe L. Korn created ARROW-4849: -- Summary: [C++] Add docker-compose entry for testing Ubuntu Bionic build with system packages Key: ARROW-4849 URL: https://issues.apache.org/jira/browse/ARROW-4849 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 To better support people on Ubuntu and also show the missing things to get Arrow packaged into Fedora, add an entry to the docker-compose.yml that builds on Ubuntu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4840) [C++] Persist CMake options in generated header
Uwe L. Korn created ARROW-4840: -- Summary: [C++] Persist CMake options in generated header Key: ARROW-4840 URL: https://issues.apache.org/jira/browse/ARROW-4840 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.14.0 (do this after we merged the CMake refactor) We should export all compile-time options like ARROW_WITH_ZSTD or ARROW_PARQUET in a header file so that other libraries depending on Arrow C++ can also respect that in their builds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4842) [C++] Persist CMake options in pkg-config files
Uwe L. Korn created ARROW-4842: -- Summary: [C++] Persist CMake options in pkg-config files Key: ARROW-4842 URL: https://issues.apache.org/jira/browse/ARROW-4842 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.14.0 Persist options like ARROW_WITH_ZSTD in {{arrow.pc}} so libraries can determine which features are available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4841) [C++] Persist CMake options in generated CMake config
Uwe L. Korn created ARROW-4841: -- Summary: [C++] Persist CMake options in generated CMake config Key: ARROW-4841 URL: https://issues.apache.org/jira/browse/ARROW-4841 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.14.0 (do this after we merged the CMake refactor) We should persist all options set during the CMake run also in {{arrowConfig.cmake}} so that CMake projects that depend on Arrow also can determine what they are able to provide. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4828) [Python] manylinux1 docker-compose context should be python/manylinux1
Uwe L. Korn created ARROW-4828: -- Summary: [Python] manylinux1 docker-compose context should be python/manylinux1 Key: ARROW-4828 URL: https://issues.apache.org/jira/browse/ARROW-4828 Project: Apache Arrow Issue Type: Bug Components: Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Currently it doesn't find the {{scripts}} folder on running {{docker-compose build python-manylinux1}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4786) [C++/Python] Support better parallelisation in manylinux1 base build
Uwe L. Korn created ARROW-4786: -- Summary: [C++/Python] Support better parallelisation in manylinux1 base build Key: ARROW-4786 URL: https://issues.apache.org/jira/browse/ARROW-4786 Project: Apache Arrow Issue Type: Task Components: C++, Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Currently we're building some dependencies single-threaded but could build them with much higher parallelisation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4778) [C++/Python] manylinux1: Update Thrift to 0.12.0
Uwe L. Korn created ARROW-4778: -- Summary: [C++/Python] manylinux1: Update Thrift to 0.12.0 Key: ARROW-4778 URL: https://issues.apache.org/jira/browse/ARROW-4778 Project: Apache Arrow Issue Type: Task Components: C++, Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4777) [C++/Python] manylinux1: Update lz4 to 1.8.3
Uwe L. Korn created ARROW-4777: -- Summary: [C++/Python] manylinux1: Update lz4 to 1.8.3 Key: ARROW-4777 URL: https://issues.apache.org/jira/browse/ARROW-4777 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4764) [C++/Java] conda-built libplasma_java doesn't work with system Java on Ubuntu Xenial
Uwe L. Korn created ARROW-4764: -- Summary: [C++/Java] conda-built libplasma_java doesn't work with system Java on Ubuntu Xenial Key: ARROW-4764 URL: https://issues.apache.org/jira/browse/ARROW-4764 Project: Apache Arrow Issue Type: Bug Components: C++, C++ - Plasma, Packaging, Python Reporter: Uwe L. Korn System Java will load also the system {{libstdc++}} which is older then the conda provided one. Thus a later loading of the plasma library will raise missing symbol errors: {code:java} TODO{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4762) [C++] Support RapidJSON<1.1.0
Uwe L. Korn created ARROW-4762: -- Summary: [C++] Support RapidJSON<1.1.0 Key: ARROW-4762 URL: https://issues.apache.org/jira/browse/ARROW-4762 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn To support building with as many system packages as possible, we should support RapidJSON<1.1.0. For example the distribution provided RapidJSON on Ubuntu Xenial is {{0.12~git20141031-3}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4763) [C++/Python] Cannot build Gandiva in conda on OSX due to package conflicts
Uwe L. Korn created ARROW-4763: -- Summary: [C++/Python] Cannot build Gandiva in conda on OSX due to package conflicts Key: ARROW-4763 URL: https://issues.apache.org/jira/browse/ARROW-4763 Project: Apache Arrow Issue Type: Improvement Components: C++, C++ - Gandiva, Python Reporter: Uwe L. Korn It is currently not reliably possible to build Gandiva with one of the conda toolchains on OSX as the packages {{llvm==4.0.1}} (pulled in by the compilers) and {{llvmdev==7.0.1}} conflict in some files: https://github.com/conda-forge/llvmdev-feedstock/issues/60 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4760) [C++] protobuf 3.7 defines EXPECT_OK that clashes with Arrow's macro
Uwe L. Korn created ARROW-4760: -- Summary: [C++] protobuf 3.7 defines EXPECT_OK that clashes with Arrow's macro Key: ARROW-4760 URL: https://issues.apache.org/jira/browse/ARROW-4760 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 This fails for me with the following error: {code:java} /home/travis/build/xhochy/arrow/cpp/src/arrow/flight/test-util.cc In file included from /home/travis/build/xhochy/arrow/cpp-toolchain/include/google/protobuf/util/type_resolver.h:39:0, from /home/travis/build/xhochy/arrow/cpp-toolchain/include/google/protobuf/util/json_util.h:37, from /home/travis/build/xhochy/arrow/cpp-toolchain/include/grpcpp/impl/codegen/config_protobuf.h:70, from /home/travis/build/xhochy/arrow/cpp/src/arrow/flight/customize_protobuf.h:23, from /home/travis/build/xhochy/arrow/cpp/src/arrow/flight/protocol-internal.h:20, from /home/travis/build/xhochy/arrow/cpp/src/arrow/flight/internal.h:23, from /home/travis/build/xhochy/arrow/cpp/src/arrow/flight/test-util.cc:35: /home/travis/build/xhochy/arrow/cpp-toolchain/include/google/protobuf/stubs/status.h:111:0: error: "EXPECT_OK" redefined [-Werror] #define EXPECT_OK(value) EXPECT_TRUE((value).ok()) In file included from /home/travis/build/xhochy/arrow/cpp/src/arrow/ipc/test-common.h:35:0, from /home/travis/build/xhochy/arrow/cpp/src/arrow/flight/test-util.cc:30: /home/travis/build/xhochy/arrow/cpp/src/arrow/testing/gtest_util.h:80:0: note: this is the location of the previous definition #define EXPECT_OK(expr) \ cc1plus: all warnings being treated as errors{code} I would workaround this by renaming our {{EXPECT_OK}} to {{ARROW_EXPECT_OK}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4761) [C++] Support zstandard<1
Uwe L. Korn created ARROW-4761: -- Summary: [C++] Support zstandard<1 Key: ARROW-4761 URL: https://issues.apache.org/jira/browse/ARROW-4761 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 To support building with as many system packages as possible on Ubuntu, we should support building with zstandard 0.5.1 which is the one available on Ubuntu Xenial. Given the size of our current code for Zstandard, this seems feasible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4751) [C++] Add pkg-config to conda_env_cpp.yml
Uwe L. Korn created ARROW-4751: -- Summary: [C++] Add pkg-config to conda_env_cpp.yml Key: ARROW-4751 URL: https://issues.apache.org/jira/browse/ARROW-4751 Project: Apache Arrow Issue Type: Task Components: C++, Continuous Integration Reporter: Uwe L. Korn Fix For: 0.13.0 Once the CMake refactor has been merged, we should add {{pkg-config}} to the dependencies as it should be also available for Windows now: https://github.com/conda-forge/pkg-config-feedstock/pull/27 This will simplify some packaging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4750) [C++] RapidJSON triggers Wclass-memaccess on GCC 8+
Uwe L. Korn created ARROW-4750: -- Summary: [C++] RapidJSON triggers Wclass-memaccess on GCC 8+ Key: ARROW-4750 URL: https://issues.apache.org/jira/browse/ARROW-4750 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 With GCC 8, we get build errors with RapidJSON due to {{-Wclass-memaccess}}. This is fixed in https://github.com/Tencent/rapidjson/pull/1323 . We should update our external project dependency to include this fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4747) [C++/PyPy] Add docker image to test against PyPy nightlies
Uwe L. Korn created ARROW-4747: -- Summary: [C++/PyPy] Add docker image to test against PyPy nightlies Key: ARROW-4747 URL: https://issues.apache.org/jira/browse/ARROW-4747 Project: Apache Arrow Issue Type: Task Components: C++, Python Reporter: Uwe L. Korn It seems (https://bitbucket.org/pypy/pypy/issues/2842/running-pyarrow-on-pypy-segfaults#comment-50670536) that we are close to being able to run with PyPy. At the moment, we don't actively work on supporting PyPy but this would be a good start on providing feedback at what is still missing. To have such an image, one would need to fork one of the current test setups in {{docker-compose.yml}} that build with system libraries (e.g. ubuntu-xenial or debian-testing) and exchange the system Python with the PyPy nightly builds from http://buildbot.pypy.org/nightly/trunk/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4746) [C++/Python] PyDataTime_Date wrongly casted to PyDataTime_DateTime
Uwe L. Korn created ARROW-4746: -- Summary: [C++/Python] PyDataTime_Date wrongly casted to PyDataTime_DateTime Key: ARROW-4746 URL: https://issues.apache.org/jira/browse/ARROW-4746 Project: Apache Arrow Issue Type: Bug Components: C++, Python Reporter: Uwe L. Korn As mentioned in https://bitbucket.org/pypy/pypy/issues/2842/running-pyarrow-on-pypy-segfaults#comment-50670536, we currently access a {{PyDataTime_Date}} object with a {{PyDataTime_DateTime}} cast in {{PyDateTime_DATE_GET_SECOND}} in our code in two instances. While CPython is able to deal with this wrong usage, PyPy is not able to do so. We should separate the path here into one that deals with dates and another that deals with datetimes. Reproducible code: {code:java} pa.array([datetime.date(2018, 5, 10)], type=pa.date64()){code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4733) [C++] Add CI entry that builds without the conda-forge toolchain but with system packages
Uwe L. Korn created ARROW-4733: -- Summary: [C++] Add CI entry that builds without the conda-forge toolchain but with system packages Key: ARROW-4733 URL: https://issues.apache.org/jira/browse/ARROW-4733 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Instead of using the conda-forge toolchain to provide parts of the dependencies but then compile with a system compiler, utilise the system packages to build and test the C++ implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4732) [C++] Add docker-compose entry for testing Debian Testing build with system packages
Uwe L. Korn created ARROW-4732: -- Summary: [C++] Add docker-compose entry for testing Debian Testing build with system packages Key: ARROW-4732 URL: https://issues.apache.org/jira/browse/ARROW-4732 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 To better support people on Fedora and also show the missing things to get Arrow packaged into Debian, add an entry to the docker-compose.yml that builds on Debian Testing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4731) [C++] Add docker-compose entry for testing Ubuntu Xenial build with system packages
Uwe L. Korn created ARROW-4731: -- Summary: [C++] Add docker-compose entry for testing Ubuntu Xenial build with system packages Key: ARROW-4731 URL: https://issues.apache.org/jira/browse/ARROW-4731 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 To better support people on Ubuntu and also show the missing things to get Arrow packaged into Fedora, add an entry to the docker-compose.yml that builds on Ubuntu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4730) [C++] Add docker-compose entry for testing Fedora build with system packages
Uwe L. Korn created ARROW-4730: -- Summary: [C++] Add docker-compose entry for testing Fedora build with system packages Key: ARROW-4730 URL: https://issues.apache.org/jira/browse/ARROW-4730 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 To better support people on Fedora and also show the missing things to get Arrow packaged into Fedora, add an entry to the docker-compose.yml that builds on Fedora -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4685) [C++] Update Boost to 1.69 in manylinux1 docker image
Uwe L. Korn created ARROW-4685: -- Summary: [C++] Update Boost to 1.69 in manylinux1 docker image Key: ARROW-4685 URL: https://issues.apache.org/jira/browse/ARROW-4685 Project: Apache Arrow Issue Type: Task Components: C++, Packaging, Python Reporter: Uwe L. Korn Fix For: 0.13.0 We currently use Boost 1.66 in the manylinux1 docker image but should use the latest version there to get all features, bugfixes, … The main difficulty in updating is that we want to use a namespaced Boost build and the build scripts have changed since the 1.66 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4672) [C++] clang-7 matrix entry is build using gcc
Uwe L. Korn created ARROW-4672: -- Summary: [C++] clang-7 matrix entry is build using gcc Key: ARROW-4672 URL: https://issues.apache.org/jira/browse/ARROW-4672 Project: Apache Arrow Issue Type: Bug Reporter: Uwe L. Korn Fix For: 0.13.0 Travis sets the following environment variables: {code} $ export CC="clang-7" $ export CXX="clang++-7" $ export TRAVIS_COMPILER=g++ $ export CXX=g++ $ export CXX_FOR_BUILD=g++ $ export CC=gcc $ export CC_FOR_BUILD=gcc $ export PATH=/usr/lib/ccache:$PATH {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4665) [C++] With glog activated, DCHECK macros are redefined
Uwe L. Korn created ARROW-4665: -- Summary: [C++] With glog activated, DCHECK macros are redefined Key: ARROW-4665 URL: https://issues.apache.org/jira/browse/ARROW-4665 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Fix For: 0.13.0 When building with {{glog}}, I get errors that the {{DCHECK_*}} macros are redefined: {code} In file included from /arrow/cpp/src/arrow/util/logging.cc:27: glog_ep-prefix/src/glog_ep/include/glog/logging.h:996: error: "DCHECK" redefined [-Werror] #define DCHECK(condition) CHECK(condition) In file included from /arrow/cpp/src/arrow/util/logging.cc:18: /arrow/cpp/src/arrow/util/logging.h:112: note: this is the location of the previous definition #define DCHECK(condition) ARROW_CHECK(condition) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4659) [CI] ubuntu/debian nightlies fail because of missing gandiva files
Uwe L. Korn created ARROW-4659: -- Summary: [CI] ubuntu/debian nightlies fail because of missing gandiva files Key: ARROW-4659 URL: https://issues.apache.org/jira/browse/ARROW-4659 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration, Packaging Reporter: Uwe L. Korn Fix For: 0.13.0 The nightly jobs fail with {code:java} dh_install dh_install: libgandiva13 missing files: usr/lib/*/gandiva/ dh_install: missing files, aborting debian/rules:14: recipe for target 'binary' failed make: *** [binary] Error 2 dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2{code} [~kou] [~kszucs] Is this because we now ship the precompiled code inside of the binary library? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4658) [C++] shared gflags is also a run-time conda requirement
Uwe L. Korn created ARROW-4658: -- Summary: [C++] shared gflags is also a run-time conda requirement Key: ARROW-4658 URL: https://issues.apache.org/jira/browse/ARROW-4658 Project: Apache Arrow Issue Type: Bug Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4657) [Release] gbenchmark should not be needed for verification
Uwe L. Korn created ARROW-4657: -- Summary: [Release] gbenchmark should not be needed for verification Key: ARROW-4657 URL: https://issues.apache.org/jira/browse/ARROW-4657 Project: Apache Arrow Issue Type: Bug Components: C++, Packaging Affects Versions: 0.12.0, 0.12.1 Reporter: Uwe L. Korn Fix For: 0.13.0 {{gbenchmark}} is built during verification and thus we require a minimal version of CMake 3.6. I would have guessed that we should not require it as we do not need to build the benchmarks during the verification. I guess that a recent fix from [~wesmckinn] may have fixed this, but we should verify this before doing the next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4649) [C++/CI/R] Add nightly job that builds `brew install apache-arrow --HEAD`
Uwe L. Korn created ARROW-4649: -- Summary: [C++/CI/R] Add nightly job that builds `brew install apache-arrow --HEAD` Key: ARROW-4649 URL: https://issues.apache.org/jira/browse/ARROW-4649 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration, R Reporter: Uwe L. Korn Fix For: 0.13.0 Now that we have an Arrow homebrew formula again and we may want to have it as a simple setup for R Arrow users, we should add a nightly crossbow task that checks whether this still builds fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4647) [Packaging] dev/release/00-prepare.sh fails for minor version changes
Uwe L. Korn created ARROW-4647: -- Summary: [Packaging] dev/release/00-prepare.sh fails for minor version changes Key: ARROW-4647 URL: https://issues.apache.org/jira/browse/ARROW-4647 Project: Apache Arrow Issue Type: Bug Components: Packaging Affects Versions: 0.12.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 When the next version is only on the patch level, we don't need to move the debian libraries to a different suffix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4636) [Python/Packaging] Crossbow builds for conda-osx fail on upload with Ruby linkage errors
Uwe L. Korn created ARROW-4636: -- Summary: [Python/Packaging] Crossbow builds for conda-osx fail on upload with Ruby linkage errors Key: ARROW-4636 URL: https://issues.apache.org/jira/browse/ARROW-4636 Project: Apache Arrow Issue Type: Bug Components: Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.1 The jobs are failing at the end with {code:java} 2.00s$ rvm $(travis_internal_ruby) --fuzzy do ruby -S gem install dpl Warning, new version of rvm available '1.29.7-next', you are using older version '1.29.3'. You can disable this warning with:echo rvm_autoupdate_flag=0 >> ~/.rvmrc You can enable auto-update with:echo rvm_autoupdate_flag=2 >> ~/.rvmrc Error loading RubyGems plugin "/Users/travis/.rvm/gems/ruby-2.4.3@global/gems/executable-hooks-1.4.2/lib/rubygems_plugin.rb": dlopen(/Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/openssl.bundle, 9): Library not loaded: /usr/local/opt/openssl@1.1/lib/libssl.1.1.dylib Referenced from: /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/openssl.bundle Reason: image not found - /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/openssl.bundle (LoadError) Error loading RubyGems plugin "/Users/travis/.rvm/gems/ruby-2.4.3@global/gems/gem-wrappers-1.3.2/lib/rubygems_plugin.rb": dlopen(/Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/openssl.bundle, 9): Library not loaded: /usr/local/opt/openssl@1.1/lib/libssl.1.1.dylib Referenced from: /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/openssl.bundle Reason: image not found - /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/openssl.bundle (LoadError) /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/yaml.rb:5:in `': It seems your ruby installation is missing psych (for YAML output). To eliminate this warning, please install libyaml and reinstall your ruby. /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': dlopen(/Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/psych.bundle, 9): Library not loaded: /usr/local/opt/libyaml/lib/libyaml-0.2.dylib (LoadError) Referenced from: /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/psych.bundle Reason: image not found - /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/x86_64-darwin17/psych.bundle from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/psych.rb:8:in `' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/yaml.rb:6:in `' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems.rb:701:in `load_yaml' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/config_file.rb:340:in `load_file' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/config_file.rb:202:in `initialize' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/gem_runner.rb:75:in `new' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/gem_runner.rb:75:in `do_configuration' from /Users/travis/.rvm/rubies/ruby-2.4.3/lib/ruby/2.4.0/rubygems/gem_runner.rb:40:in `run' from /Users/travis/.rvm/rubies/ruby-2.4.3/bin/gem:21:in `'{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4617) [C++] Support double-conversion<3.1
Uwe L. Korn created ARROW-4617: -- Summary: [C++] Support double-conversion<3.1 Key: ARROW-4617 URL: https://issues.apache.org/jira/browse/ARROW-4617 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn At the moment, we require {{double-conversion>=3.1}} but distributions are all mostly on 2.1 or 3.0. We should support them Nice side-effect is that {{double-conversion}} is not good in updating their version number. So thus 3.0 report itself as 2.0.1 and 3.1.x as 3.0 in the CMake definitions. Thus we cannot rely on their version number reporting for detecting the version. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4616) [C++] Log message in BuildUtils as STATUS
Uwe L. Korn created ARROW-4616: -- Summary: [C++] Log message in BuildUtils as STATUS Key: ARROW-4616 URL: https://issues.apache.org/jira/browse/ARROW-4616 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Without a logging level, they are considered as warning like and will be shown to the user in {{cmake}} after they run a configure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4615) [C++] Add checked_pointer_cast
Uwe L. Korn created ARROW-4615: -- Summary: [C++] Add checked_pointer_cast Key: ARROW-4615 URL: https://issues.apache.org/jira/browse/ARROW-4615 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 In the style of {{checked_cast}} but for the equivalent {{dynamic/static_pointer_cast}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4614) [C++/CI] Activate flight build in ci/docker_build_cpp.sh
Uwe L. Korn created ARROW-4614: -- Summary: [C++/CI] Activate flight build in ci/docker_build_cpp.sh Key: ARROW-4614 URL: https://issues.apache.org/jira/browse/ARROW-4614 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration, FlightRPC Reporter: Uwe L. Korn Assignee: Uwe L. Korn This is currently not build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4613) [C++] Alpine build failing as libgtestd.so is not found
Uwe L. Korn created ARROW-4613: -- Summary: [C++] Alpine build failing as libgtestd.so is not found Key: ARROW-4613 URL: https://issues.apache.org/jira/browse/ARROW-4613 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4612) [Python] Use cython from PyPI for windows wheels build
Uwe L. Korn created ARROW-4612: -- Summary: [Python] Use cython from PyPI for windows wheels build Key: ARROW-4612 URL: https://issues.apache.org/jira/browse/ARROW-4612 Project: Apache Arrow Issue Type: Improvement Components: Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4611) [C++] Rework CMake third-party logic
Uwe L. Korn created ARROW-4611: -- Summary: [C++] Rework CMake third-party logic Key: ARROW-4611 URL: https://issues.apache.org/jira/browse/ARROW-4611 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Instead of the current approach we are taking with the {{*_HOME}} variables, we should use more CMake features and also give users of Arrow a more high-level control. This is going to be a rather lengthy issue with a lot of subtasks. * Let the user decide on the top-level how dependencies should be handled. At the moment I can think of the following modes: ** AUTO: Guess the packaging system we're running in, use this where possible, otherwise build the dependencies through the {{ExternalProject}} logic. ** BUNDLED: Don't use any dependencies, build them all through {{ExternalProject}} ** SYSTEM: Use CMake's {{find_package}} and {{find_library}} without any custom paths. If packages are on non-default locations, let the user indicate it from the outside using the {{*_ROOT}} variables. ** CONDA: Same as SYSTEM but set all {{*_ROOT}} variables to {{ENV\{CONDA_PREFIX\}}}. ** BREW: This uses SYSTEM but asks {{brew}} for some dependencies for their installation prefix. * prefer dynamic linkage where possible * Use {{pkg-config}} and {{*Targets.cmake}} files in projects that publish these * Ensure that the necessary integration tests are in place (Fedora, Debian, Ubuntu, Alpine) * Integration tests that Arrow's {{*Targets.cmake}} and {{arrow.pc}} work. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4609) [C++] Use google benchmark from toolchain
Uwe L. Korn created ARROW-4609: -- Summary: [C++] Use google benchmark from toolchain Key: ARROW-4609 URL: https://issues.apache.org/jira/browse/ARROW-4609 Project: Apache Arrow Issue Type: Improvement Components: C++, Packaging Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4585) [C++] Dependency of Flight C++ sources on generated protobuf is not respected
Uwe L. Korn created ARROW-4585: -- Summary: [C++] Dependency of Flight C++ sources on generated protobuf is not respected Key: ARROW-4585 URL: https://issues.apache.org/jira/browse/ARROW-4585 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Fix For: 0.13.0 Seems like we have a race condition somewhere as we frequently run into {code:java} [82/273] Building CXX object src/arrow/flight/CMakeFiles/arrow_flight_testing_objlib.dir/test-util.cc.o FAILED: src/arrow/flight/CMakeFiles/arrow_flight_testing_objlib.dir/test-util.cc.o /usr/local/bin/ccache /Users/ukorn/miniconda3/envs/pyarrow-dev-2/bin/x86_64-apple-darwin13.4.0-clang++ -DARROW_JEMALLOC -DARROW_JEMALLOC_INCLUDE_DIR=/Users/ukorn/Development/arrow-repos-2/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist//include -DARROW_USE_GLOG -DARROW_USE_SIMD -DARROW_WITH_BROTLI -DARROW_WITH_LZ4 -DARROW_WITH_SNAPPY -DARROW_WITH_ZLIB -DARROW_WITH_ZSTD -Isrc -I../src -isystem /Users/ukorn/miniconda3/envs/pyarrow-dev-2/include -isystem gbenchmark_ep/src/gbenchmark_ep-install/include -isystem jemalloc_ep-prefix/src -isystem ../thirdparty/hadoop/include -isystem /Users/ukorn/miniconda3/envs/pyarrow-dev-2/include/thrift -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -Qunused-arguments -O3 -DNDEBUG -Wall -Wno-unknown-warning-option -msse4.2 -stdlib=libc++ -O3 -DNDEBUG -fPIC -std=gnu++11 -MD -MT src/arrow/flight/CMakeFiles/arrow_flight_testing_objlib.dir/test-util.cc.o -MF src/arrow/flight/CMakeFiles/arrow_flight_testing_objlib.dir/test-util.cc.o.d -o src/arrow/flight/CMakeFiles/arrow_flight_testing_objlib.dir/test-util.cc.o -c ../src/arrow/flight/test-util.cc In file included from ../src/arrow/flight/test-util.cc:35: In file included from ../src/arrow/flight/internal.h:29: ../src/arrow/flight/protocol-internal.h:22:10: fatal error: 'arrow/flight/Flight.grpc.pb.h' file not found #include "arrow/flight/Flight.grpc.pb.h" ^~~ 1 error generated. [87/273] Building CXX object src/arrow/python/CMakeFiles/arrow_python_objlib.dir/python_to_arrow.cc.o ninja: build stopped: subcommand failed. ninja 672,82s user 33,40s system 196% cpu 5:59,62 total{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4584) [Python] Add built wheel to manylinux1 dockerignore.
Uwe L. Korn created ARROW-4584: -- Summary: [Python] Add built wheel to manylinux1 dockerignore. Key: ARROW-4584 URL: https://issues.apache.org/jira/browse/ARROW-4584 Project: Apache Arrow Issue Type: Bug Components: Packaging, Python Affects Versions: 0.12.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Currently we add them to the docker context while we don't need them. This shrinks the context for me down to 55k instead of hundreds MiBs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4582) [C++/Python] Memory corruption on Pandas->Arrow conversion
Uwe L. Korn created ARROW-4582: -- Summary: [C++/Python] Memory corruption on Pandas->Arrow conversion Key: ARROW-4582 URL: https://issues.apache.org/jira/browse/ARROW-4582 Project: Apache Arrow Issue Type: Bug Components: C++, Python Affects Versions: 0.12.0, 0.11.1, 0.11.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 When converting DataFrames with numerical columns to Arrow tables we were seeing random segfaults in core Python code. This only happened in environments where we had a high level of parallelisation or slow code execution (e.g. in AddressSanitizer builds). The reason for these segfaults was that we were incrementing the reference count of the underlying NumPy buffer but were not holding the GIL while changing the reference count. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4577) [C++] Interface link libraries declared on arrow_shared target that are actually non-interface
Uwe L. Korn created ARROW-4577: -- Summary: [C++] Interface link libraries declared on arrow_shared target that are actually non-interface Key: ARROW-4577 URL: https://issues.apache.org/jira/browse/ARROW-4577 Project: Apache Arrow Issue Type: Bug Components: C++ Affects Versions: 0.12.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 We are pulling in {{jemalloc_static}} as an interface linking dependency in the {{arrowTargets.cmake}}. But as it is statically linked inside the shared library, consumers don't need to link against it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4528) [C++] Update lint docker container to LLVM-7
Uwe L. Korn created ARROW-4528: -- Summary: [C++] Update lint docker container to LLVM-7 Key: ARROW-4528 URL: https://issues.apache.org/jira/browse/ARROW-4528 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4494) [Java] arrow-jdbc JAR is not uploaded on release
Uwe L. Korn created ARROW-4494: -- Summary: [Java] arrow-jdbc JAR is not uploaded on release Key: ARROW-4494 URL: https://issues.apache.org/jira/browse/ARROW-4494 Project: Apache Arrow Issue Type: Bug Components: Java, Packaging Affects Versions: 0.12.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4471) [C++] Pass AR and RANLIB to all external projects
Uwe L. Korn created ARROW-4471: -- Summary: [C++] Pass AR and RANLIB to all external projects Key: ARROW-4471 URL: https://issues.apache.org/jira/browse/ARROW-4471 Project: Apache Arrow Issue Type: Bug Components: C++ Affects Versions: 0.12.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 With the latest updates I have problems linking to gbenchmark on OSX, this fixes it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4447) [C++] Investigate dynamic linking for libthift
Uwe L. Korn created ARROW-4447: -- Summary: [C++] Investigate dynamic linking for libthift Key: ARROW-4447 URL: https://issues.apache.org/jira/browse/ARROW-4447 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Uwe L. Korn Fix For: 0.13.0 We're currently only linking statically against {{libthrift}} . Distributions would often prefer a dynamic linkage to libraries where possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4389) [R] Installing clang-tools in CI is failing on trusty
Uwe L. Korn created ARROW-4389: -- Summary: [R] Installing clang-tools in CI is failing on trusty Key: ARROW-4389 URL: https://issues.apache.org/jira/browse/ARROW-4389 Project: Apache Arrow Issue Type: Bug Components: Continuous Integration, R Reporter: Uwe L. Korn Assignee: Uwe L. Korn CI fails with {code:java} W: The repository 'http://llvm.org/apt/trusty llvm-toolchain-trusty-6.0 Release' does not have a Release file. W: http://ppa.launchpad.net/couchdb/stable/ubuntu/dists/trusty/Release.gpg: Signature by key 15866BAFD9BCC4F3C1E0DFC7D69548E1C17EAB57 uses weak digest algorithm (SHA1) W: http://ppa.launchpad.net/kirillshkrogalev/ffmpeg-next/ubuntu/dists/trusty/Release.gpg: Signature by key 1EC6DBC9AA41BD34B32CC5A15C50E96D8EFE5982 uses weak digest algorithm (SHA1) W: GPG error: https://packagecloud.io/github/git-lfs/ubuntu trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6B05F25D762E3157 W: The repository 'https://packagecloud.io/github/git-lfs/ubuntu trusty InRelease' is not signed. W: There is no public key available for the following key IDs: 6B05F25D762E3157 E: Failed to fetch https://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-6.0/main/binary-amd64/Packages Protocol http not supported or disabled in libcurl E: Failed to fetch https://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-6.0/main/binary-i386/Packages Protocol http not supported or disabled in libcurl E: Some index files failed to download. They have been ignored, or old ones used instead. Reading package lists... Building dependency tree... Reading state information... E: Unable to locate package clang-6.0 E: Couldn't find any package by glob 'clang-6.0' E: Couldn't find any package by regex 'clang-6.0' E: Unable to locate package clang-format-6.0 E: Couldn't find any package by glob 'clang-format-6.0' E: Couldn't find any package by regex 'clang-format-6.0' E: Unable to locate package clang-tidy-6.0 E: Couldn't find any package by glob 'clang-tidy-6.0' E: Couldn't find any package by regex 'clang-tidy-6.0' The command "$TRAVIS_BUILD_DIR/ci/travis_install_clang_tools.sh" failed and exited with 100 during .{code} Seems like clang-tools are not used in the build, will remove them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4385) [Python] default_version of a release should not include SNAPSHOT
Uwe L. Korn created ARROW-4385: -- Summary: [Python] default_version of a release should not include SNAPSHOT Key: ARROW-4385 URL: https://issues.apache.org/jira/browse/ARROW-4385 Project: Apache Arrow Issue Type: Bug Components: Packaging, Python Affects Versions: 0.12.0 Reporter: Uwe L. Korn Fix For: 0.13.0 The 0.12 release had {{default_version = '0.12.0-SNAPSHOT'}} in the source thus triggering https://github.com/conda-forge/pyarrow-feedstock/issues/69. Instead, we should remove the SNAPSHOT suffix in our release scripts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4374) [C++] DictionaryBuilder does not correctly report length and null_count
Uwe L. Korn created ARROW-4374: -- Summary: [C++] DictionaryBuilder does not correctly report length and null_count Key: ARROW-4374 URL: https://issues.apache.org/jira/browse/ARROW-4374 Project: Apache Arrow Issue Type: Bug Affects Versions: 0.12.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 In comparison to most builders, they stay constantly at 0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4367) [C++] StringDictionaryBuilder segfaults on Finish with only null entries
Uwe L. Korn created ARROW-4367: -- Summary: [C++] StringDictionaryBuilder segfaults on Finish with only null entries Key: ARROW-4367 URL: https://issues.apache.org/jira/browse/ARROW-4367 Project: Apache Arrow Issue Type: Bug Components: C++ Affects Versions: 0.12.0 Reporter: Uwe L. Korn Fix For: 0.13.0 Sadly a regression from 0.11, detected during the turbodbc integration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4362) [Java] Test OpenJDK 11 in CI
Uwe L. Korn created ARROW-4362: -- Summary: [Java] Test OpenJDK 11 in CI Key: ARROW-4362 URL: https://issues.apache.org/jira/browse/ARROW-4362 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Java Reporter: Uwe L. Korn Assignee: Uwe L. Korn -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4360) [C++] Query homebrew for Thrift
Uwe L. Korn created ARROW-4360: -- Summary: [C++] Query homebrew for Thrift Key: ARROW-4360 URL: https://issues.apache.org/jira/browse/ARROW-4360 Project: Apache Arrow Issue Type: Bug Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Also search for LLVM with homebrew when on OSX and THRIFT_HOME is not set. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4356) [CI] Add integration (docker) test for turbodbc
Uwe L. Korn created ARROW-4356: -- Summary: [CI] Add integration (docker) test for turbodbc Key: ARROW-4356 URL: https://issues.apache.org/jira/browse/ARROW-4356 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration Reporter: Uwe L. Korn Fix For: 0.13.0 We regularly break our API so that {{turbodbc}} needs to make minor changes to support the new Arrow version. We should setup a small integration test to check before a release that {{turbodbc}} can easily upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4355) [C++] test-util functions are no longer part of libarrow
Uwe L. Korn created ARROW-4355: -- Summary: [C++] test-util functions are no longer part of libarrow Key: ARROW-4355 URL: https://issues.apache.org/jira/browse/ARROW-4355 Project: Apache Arrow Issue Type: Bug Components: C++ Affects Versions: 0.12.0 Reporter: Uwe L. Korn Assignee: Uwe L. Korn I have used these functions in other artifacts like {{turbodbc}}. I would like to have them back as part of libarrow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4322) [CI] docker nightlies fails after conda-forge compiler migration
Uwe L. Korn created ARROW-4322: -- Summary: [CI] docker nightlies fails after conda-forge compiler migration Key: ARROW-4322 URL: https://issues.apache.org/jira/browse/ARROW-4322 Project: Apache Arrow Issue Type: Bug Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4303) [Gandiva/Python] Build LLVM with RTTI in manylinux1 container
Uwe L. Korn created ARROW-4303: -- Summary: [Gandiva/Python] Build LLVM with RTTI in manylinux1 container Key: ARROW-4303 URL: https://issues.apache.org/jira/browse/ARROW-4303 Project: Apache Arrow Issue Type: Task Components: Gandiva, Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn We build with RTTI in Arrow and thus need the typeinfo symbols. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4298) [Java] Building Flight fails with OpenJDK 11
Uwe L. Korn created ARROW-4298: -- Summary: [Java] Building Flight fails with OpenJDK 11 Key: ARROW-4298 URL: https://issues.apache.org/jira/browse/ARROW-4298 Project: Apache Arrow Issue Type: New Feature Components: FlightRPC, Java Affects Versions: 0.12.0 Reporter: Uwe L. Korn Building flight fails with {code:java} [INFO] --- maven-compiler-plugin:3.6.2:compile (default-compile) @ arrow-flight --- [INFO] Compiling 39 source files to /Users/uwe/Development/arrow-repos-1/arrow/java/flight/target/classes [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /Users/uwe/Development/arrow-repos-1/arrow/java/flight/target/generated-sources/protobuf/org/apache/arrow/flight/impl/FlightServiceGrpc.java:[26,17] error: cannot find symbol symbol: class Generated location: package javax.annotation [INFO] 1 error{code} To fix this, I added the following dependency to {{flight/pom.xml}}: {code:java} javax.annotation javax.annotation-api 1.3.2 {code} This then passed the compile step but failed later with: {code:java} [INFO] --- maven-dependency-plugin:3.0.1:analyze-only (analyze) @ arrow-flight --- [WARNING] Unused declared dependencies found: [WARNING] javax.annotation:javax.annotation-api:jar:1.3.2:compile [ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (analyze) on project arrow-flight: Dependency problems found -> [Help 1]{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4291) [Dev] Support selecting features in release scripts
Uwe L. Korn created ARROW-4291: -- Summary: [Dev] Support selecting features in release scripts Key: ARROW-4291 URL: https://issues.apache.org/jira/browse/ARROW-4291 Project: Apache Arrow Issue Type: New Feature Components: Developer Tools, Packaging Reporter: Uwe L. Korn Sometimes not all components can be verified on a system. We should provide some environment variables to exclude them to proceed to the next step. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4290) [C++/Gandiva] Support detecting correct LLVM version in Homebrew
Uwe L. Korn created ARROW-4290: -- Summary: [C++/Gandiva] Support detecting correct LLVM version in Homebrew Key: ARROW-4290 URL: https://issues.apache.org/jira/browse/ARROW-4290 Project: Apache Arrow Issue Type: New Feature Components: C++, Gandiva Reporter: Uwe L. Korn We should also search in homebrew for the matching LLVM version for Gandiva on OSX. You can install it via {{brew install llvm@6}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4289) [C++] Forward AR and RANLIB to thirdparty builds
Uwe L. Korn created ARROW-4289: -- Summary: [C++] Forward AR and RANLIB to thirdparty builds Key: ARROW-4289 URL: https://issues.apache.org/jira/browse/ARROW-4289 Project: Apache Arrow Issue Type: New Feature Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn On OSX Mojave, it seems that there are many version of AR present. CMake seems to detect the right one whereas some thirdparty tooling picks up the wrong one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4287) [C++] Ensure minimal bison version on OSX for Thrift
Uwe L. Korn created ARROW-4287: -- Summary: [C++] Ensure minimal bison version on OSX for Thrift Key: ARROW-4287 URL: https://issues.apache.org/jira/browse/ARROW-4287 Project: Apache Arrow Issue Type: New Feature Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 Thrift currently just uses the first bison it finds but needs actually a newer one. We should look for the minimal version required and fall back explicitly to homebrew and use the newer version if it is available there. Note: I'll add a fix in our CMake toolchain but will also try to upstream this to Thrift. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4286) [C++/R] Namespace vendored Boost
Uwe L. Korn created ARROW-4286: -- Summary: [C++/R] Namespace vendored Boost Key: ARROW-4286 URL: https://issues.apache.org/jira/browse/ARROW-4286 Project: Apache Arrow Issue Type: New Feature Components: C++, Packaging, R Reporter: Uwe L. Korn Fix For: 0.13.0 For R, we vendor Boost and thus also include the symbols privately in our modules. While they are private, some things like virtual destructors can still interfere with other packages that vendor Boost. We should also namespace the vendored Boost as we do in the manylinux1 packaging: https://github.com/apache/arrow/blob/0f8bd747468dd28c909ef823bed77d8082a5b373/python/manylinux1/scripts/build_boost.sh#L28 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4265) [C++] Automatic conversion between Table and std::vector>
Uwe L. Korn created ARROW-4265: -- Summary: [C++] Automatic conversion between Table and std::vector> Key: ARROW-4265 URL: https://issues.apache.org/jira/browse/ARROW-4265 Project: Apache Arrow Issue Type: New Feature Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4210) [Python] Mention boost-cpp directly in the conda meta.yaml for pyarrow
Uwe L. Korn created ARROW-4210: -- Summary: [Python] Mention boost-cpp directly in the conda meta.yaml for pyarrow Key: ARROW-4210 URL: https://issues.apache.org/jira/browse/ARROW-4210 Project: Apache Arrow Issue Type: Task Components: Packaging, Python Reporter: Uwe L. Korn Fix For: 0.12.0 We seem to also link some boost symbols into pyarrow. This should guarantee through the new `run_exports` pinning mechanism that we only see installs of arrow-cpp and pyarrow with the same `boost-cpp` version. conda-forge PR: https://github.com/conda-forge/pyarrow-feedstock/pull/63 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4191) [C++] Use same CC and AR for jemalloc as for the main sources
Uwe L. Korn created ARROW-4191: -- Summary: [C++] Use same CC and AR for jemalloc as for the main sources Key: ARROW-4191 URL: https://issues.apache.org/jira/browse/ARROW-4191 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.0 jemalloc currently does its own CC and AR detection and sometimes comes to different conclusions. This let to linking errors for the static library for me. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4129) [Python] Fix syntax problem in benchmark docs
Uwe L. Korn created ARROW-4129: -- Summary: [Python] Fix syntax problem in benchmark docs Key: ARROW-4129 URL: https://issues.apache.org/jira/browse/ARROW-4129 Project: Apache Arrow Issue Type: Improvement Components: Documentation, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4107) [Python] Use ninja in pyarrow manylinux1 build
Uwe L. Korn created ARROW-4107: -- Summary: [Python] Use ninja in pyarrow manylinux1 build Key: ARROW-4107 URL: https://issues.apache.org/jira/browse/ARROW-4107 Project: Apache Arrow Issue Type: Improvement Components: Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn This should speed up the built slightly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4054) [Python] Update gtest, flatbuffers and OpenSSL in manylinux1 base image
Uwe L. Korn created ARROW-4054: -- Summary: [Python] Update gtest, flatbuffers and OpenSSL in manylinux1 base image Key: ARROW-4054 URL: https://issues.apache.org/jira/browse/ARROW-4054 Project: Apache Arrow Issue Type: Improvement Components: Packaging, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-3995) [CI] Use understandable names in Travis Matrix
Uwe L. Korn created ARROW-3995: -- Summary: [CI] Use understandable names in Travis Matrix Key: ARROW-3995 URL: https://issues.apache.org/jira/browse/ARROW-3995 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.0 Travis has a new feature to assign labels to the matrix entries making it much easier navigable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-3972) [Gandiva] Update to LLVM 7
Uwe L. Korn created ARROW-3972: -- Summary: [Gandiva] Update to LLVM 7 Key: ARROW-3972 URL: https://issues.apache.org/jira/browse/ARROW-3972 Project: Apache Arrow Issue Type: Improvement Components: C++, Gandiva Reporter: Uwe L. Korn As {{llvmlite}}, the other package in the Python ecosystem moved to LLVM 7, we should follow along to avoid problems when we use it in the same Python environment as Gandiva. Reference: https://github.com/numba/llvmlite/pull/412 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-3944) [Python] Build manylinux1 docker image directly in the CI
Uwe L. Korn created ARROW-3944: -- Summary: [Python] Build manylinux1 docker image directly in the CI Key: ARROW-3944 URL: https://issues.apache.org/jira/browse/ARROW-3944 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.0 Instead of always waiting for {{quay.io}} on PRs, we should built the docker image in the Travis job. We will still pull the current image from {{quay.io}} to profit from the layer caching. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-3932) [Python/Documentation] Include Benchmarks.md in Sphinx docs
Uwe L. Korn created ARROW-3932: -- Summary: [Python/Documentation] Include Benchmarks.md in Sphinx docs Key: ARROW-3932 URL: https://issues.apache.org/jira/browse/ARROW-3932 Project: Apache Arrow Issue Type: Improvement Components: Documentation, Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn https://github.com/apache/arrow/pull/2856#issuecomment-443711136 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-3834) [Doc] Merge Python & C++ and move to top-level
Uwe L. Korn created ARROW-3834: -- Summary: [Doc] Merge Python & C++ and move to top-level Key: ARROW-3834 URL: https://issues.apache.org/jira/browse/ARROW-3834 Project: Apache Arrow Issue Type: Improvement Components: Documentation Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.0 Merge the C++, Python and Format documentation and move it to the top-level folder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-3829) [Python] Support protocols to extract Arrow objects from third-party classes
Uwe L. Korn created ARROW-3829: -- Summary: [Python] Support protocols to extract Arrow objects from third-party classes Key: ARROW-3829 URL: https://issues.apache.org/jira/browse/ARROW-3829 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Uwe L. Korn Assignee: Uwe L. Korn Fix For: 0.12.0 In the style of NumPy's {{__array__}}, we should be able to ask inputs to {{pa.array}}, {{pa.Table.from_X}}, ... whether they can convert themselves to Arrow objects. This would allow for example to turn objects that hold an Arrow object internally to expose them directly instead of going a conversion path. My current use case involves Pandas {{ExtensionArray}} instances that internally have Arrow objects and should be reused when we pass the whole {{DataFrame}} to {{pa.Table.from_pandas}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)