Re: Plasma C++ error in Travis CI
Hi, It seems that this is related to system resource (memory) of CI machine not Plasma itself. Could you try the following change? It will reduce required resource: -- diff --git a/.travis.yml b/.travis.yml index ff2791b94..8c5420ede 100644 --- a/.travis.yml +++ b/.travis.yml @@ -54,6 +54,7 @@ jobs: -e ARROW_S3=OFF -e ARROW_USE_GLOG=OFF -e CMAKE_UNITY_BUILD=ON + -e CPP_MAKE_PARALLELISM=4 " # We need to use smaller build when cache doesn't exist # because Travis CI has "No output has been received in the -- Thanks, -- kou In <65afbf04-427a-4331-a845-98ae8841b...@gmail.com> "Plasma C++ error in Travis CI" on Sun, 24 Jan 2021 14:43:18 -0500, Ying Zhou wrote: > Hi, > > While refactoring my ORC writer so that Antoine and Uwe’s suggestions are > implemented I found this weird Travis CI error caused by Plasma. Since Plasma > is no longer maintained do we really need to have it in our Travis CI test? > Thanks! > > Ying > P.S. The job log is here: > https://travis-ci.com/github/apache/arrow/jobs/474756034 > The error message is contained below: > > [33/267] Building CXX object > src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o > 2021 <>FAILED: > src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o > 2022 <>/usr/bin/ccache /usr/bin/c++ -DARROW_EXPORTING -DARROW_HAVE_NEON > -DARROW_JEMALLOC -DARROW_JEMALLOC_INCLUDE_DIR="" -DARROW_MIMALLOC > -DARROW_NO_DEPRECATED_API -DARROW_WITH_RE2 -DARROW_WITH_TIMING_TESTS > -DARROW_WITH_UTF8PROC -DGTEST_LINKED_AS_SHARED_LIBRARY=1 -Isrc > -I/arrow/cpp/src -I/arrow/cpp/src/generated -isystem > /arrow/cpp/thirdparty/flatbuffers/include -isystem jemalloc_ep-prefix/src > -isystem mimalloc_ep/src/mimalloc_ep/lib/mimalloc-1.6/include -isystem > googletest_ep-prefix/include -isystem /arrow/cpp/thirdparty/hadoop/include > -Wno-noexcept-type -Wno-subobject-linkage -fdiagnostics-color=always -ggdb > -O0 -Wall -Wno-conversion -Wno-deprecated-declarations -Wno-sign-conversion > -Wno-unused-variable -Werror -fno-semantic-interposition -march=armv8-a > -fPIC -g -fPIC -std=c++11 -MD -MT > src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o -MF > src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o.d -o > src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o -c > src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx > 2023 <>c++: fatal error: Killed signal terminated program cc1plus > 2024 <>compilation terminated. > 2025 <> > 2026 <> > 2027 <>No output has been received in the last 10m0s, this potentially > indicates a stalled build or something wrong with the build itself. > 2028 <>Check the details on how to adjust your build configuration on: > https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received >
Re: [javascript] streaming IPC examples?
+Paul Taylor would your work with whatwg streams be relevant here? Are there any examples that would be useful for Ryan? Brian On Sat, Jan 23, 2021 at 4:52 PM Ryan McKinley wrote: > Hello- > > I am exploring options to support streaming in grafana. We have a golang > websocket server and am exploring options to send data to the browser. > > Are there any good examples of reading IPC data with callbacks for each > block? I see examples for mapd, and for reading whole tables -- but am > hoping for something that lets me read initial header data, then get each > record batch as a callback (rxjs) > https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format > > Thanks for any pointers > Ryan >
Plasma C++ error in Travis CI
Hi, While refactoring my ORC writer so that Antoine and Uwe’s suggestions are implemented I found this weird Travis CI error caused by Plasma. Since Plasma is no longer maintained do we really need to have it in our Travis CI test? Thanks! Ying P.S. The job log is here: https://travis-ci.com/github/apache/arrow/jobs/474756034 The error message is contained below: [33/267] Building CXX object src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o 2021 <>FAILED: src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o 2022 <>/usr/bin/ccache /usr/bin/c++ -DARROW_EXPORTING -DARROW_HAVE_NEON -DARROW_JEMALLOC -DARROW_JEMALLOC_INCLUDE_DIR="" -DARROW_MIMALLOC -DARROW_NO_DEPRECATED_API -DARROW_WITH_RE2 -DARROW_WITH_TIMING_TESTS -DARROW_WITH_UTF8PROC -DGTEST_LINKED_AS_SHARED_LIBRARY=1 -Isrc -I/arrow/cpp/src -I/arrow/cpp/src/generated -isystem /arrow/cpp/thirdparty/flatbuffers/include -isystem jemalloc_ep-prefix/src -isystem mimalloc_ep/src/mimalloc_ep/lib/mimalloc-1.6/include -isystem googletest_ep-prefix/include -isystem /arrow/cpp/thirdparty/hadoop/include -Wno-noexcept-type -Wno-subobject-linkage -fdiagnostics-color=always -ggdb -O0 -Wall -Wno-conversion -Wno-deprecated-declarations -Wno-sign-conversion -Wno-unused-variable -Werror -fno-semantic-interposition -march=armv8-a -fPIC -g -fPIC -std=c++11 -MD -MT src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o -MF src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o.d -o src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx.o -c src/plasma/CMakeFiles/plasma_objlib.dir/Unity/unity_0_cxx.cxx 2023 <>c++: fatal error: Killed signal terminated program cc1plus 2024 <>compilation terminated. 2025 <> 2026 <> 2027 <>No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself. 2028 <>Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received
Re: [NIGHTLY] Arrow Build Report for Job nightly-2021-01-24-0
The newly failing builds are due to a recent release of pygit2 crossbow dependency. The wheels were not available during the build, so it tried to compile without libgit2 installed. Seems like the pygit2 binary wheels have been uploaded to pypi [1], so it shouldn't fail from tomorrow on. [1]: https://pypi.org/project/pygit2/#files On Sun, Jan 24, 2021 at 11:45 AM Crossbow wrote: > > > Arrow Build Report for Job nightly-2021-01-24-0 > > All tasks: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0 > > Failed Tasks: > - centos-8-aarch64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-travis-centos-8-aarch64 > - conda-linux-gcc-py36-aarch64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py36-aarch64 > - conda-linux-gcc-py37-aarch64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py37-aarch64 > - conda-linux-gcc-py38-aarch64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py38-aarch64 > - conda-linux-gcc-py39-aarch64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py39-aarch64 > - test-conda-python-3.7-hdfs-3.2: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-test-conda-python-3.7-hdfs-3.2 > - test-conda-python-3.7-spark-branch-3.0: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-test-conda-python-3.7-spark-branch-3.0 > - test-conda-python-3.8-jpype: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-test-conda-python-3.8-jpype > - test-ubuntu-18.04-docs: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-test-ubuntu-18.04-docs > - wheel-osx-high-sierra-cp36m: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp36m > - wheel-osx-high-sierra-cp37m: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp37m > - wheel-osx-high-sierra-cp38: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp38 > - wheel-osx-high-sierra-cp39: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp39 > - wheel-osx-mavericks-cp36m: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-mavericks-cp36m > - wheel-osx-mavericks-cp37m: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-mavericks-cp37m > - wheel-osx-mavericks-cp39: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-mavericks-cp39 > - wheel-windows-cp36m: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp36m > - wheel-windows-cp37m: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp37m > - wheel-windows-cp38: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp38 > - wheel-windows-cp39: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp39 > > Succeeded Tasks: > - centos-7-aarch64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-travis-centos-7-aarch64 > - centos-7-amd64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-centos-7-amd64 > - centos-8-amd64: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-centos-8-amd64 > - conda-clean: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-clean > - conda-linux-gcc-py36-cpu-r36: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py36-cpu-r36 > - conda-linux-gcc-py36-cuda: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py36-cuda > - conda-linux-gcc-py37-cpu-r40: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py37-cpu-r40 > - conda-linux-gcc-py37-cuda: > URL: > https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py37-cuda > - conda-linux-gcc-py38-cpu: > URL: >
[Rust] Proposed discussion items for the Rust sync up meeting this Wednesday Jan 27, 2021
I would like to propose two items for discussion at the Rust sync up meeting on Wednesday, or comment via email if you would prefer: *Item 1: API Backwards compatibility expectations* What, if any, expectations should users of the Rust Arrow API have regarding having about needing to change their code when upgrading arrow? Should different parts of the library (eg. Arrow vs DataFusion, or certain Arrow modules) have different expectations? *Item 2: What are the expectations for merging a PR?* Do we have (or want to have) a more formal set of criteria for merging PRs? For example, Is a single PR approval enough? Is there any requirement on approvals (e.g. from a committer?) Is there some minimum time between approval and merge so others can comment, are there different people who should review / approve different parts of the code? I apologize if this is already written down somewhere, but I could not find it. Andrew
[NIGHTLY] Arrow Build Report for Job nightly-2021-01-24-0
Arrow Build Report for Job nightly-2021-01-24-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0 Failed Tasks: - centos-8-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-travis-centos-8-aarch64 - conda-linux-gcc-py36-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py36-aarch64 - conda-linux-gcc-py37-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py37-aarch64 - conda-linux-gcc-py38-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py38-aarch64 - conda-linux-gcc-py39-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-drone-conda-linux-gcc-py39-aarch64 - test-conda-python-3.7-hdfs-3.2: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-test-conda-python-3.7-hdfs-3.2 - test-conda-python-3.7-spark-branch-3.0: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-test-conda-python-3.7-spark-branch-3.0 - test-conda-python-3.8-jpype: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-test-conda-python-3.8-jpype - test-ubuntu-18.04-docs: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-test-ubuntu-18.04-docs - wheel-osx-high-sierra-cp36m: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp36m - wheel-osx-high-sierra-cp37m: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp37m - wheel-osx-high-sierra-cp38: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp38 - wheel-osx-high-sierra-cp39: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-high-sierra-cp39 - wheel-osx-mavericks-cp36m: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-mavericks-cp36m - wheel-osx-mavericks-cp37m: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-mavericks-cp37m - wheel-osx-mavericks-cp39: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-osx-mavericks-cp39 - wheel-windows-cp36m: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp36m - wheel-windows-cp37m: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp37m - wheel-windows-cp38: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp38 - wheel-windows-cp39: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-wheel-windows-cp39 Succeeded Tasks: - centos-7-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-travis-centos-7-aarch64 - centos-7-amd64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-centos-7-amd64 - centos-8-amd64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-github-centos-8-amd64 - conda-clean: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-clean - conda-linux-gcc-py36-cpu-r36: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py36-cpu-r36 - conda-linux-gcc-py36-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py36-cuda - conda-linux-gcc-py37-cpu-r40: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py37-cpu-r40 - conda-linux-gcc-py37-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py37-cuda - conda-linux-gcc-py38-cpu: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py38-cpu - conda-linux-gcc-py38-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py38-cuda - conda-linux-gcc-py39-cpu: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py39-cpu - conda-linux-gcc-py39-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-24-0-azure-conda-linux-gcc-py39-cuda - conda-osx-clang-py36-r36: URL:
