[jira] [Assigned] (ARROW-6859) [CI][Nightly] Disable docker layer caching for CircleCI tasks
[ https://issues.apache.org/jira/browse/ARROW-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs reassigned ARROW-6859: -- Assignee: Krisztian Szucs > [CI][Nightly] Disable docker layer caching for CircleCI tasks > - > > Key: ARROW-6859 > URL: https://issues.apache.org/jira/browse/ARROW-6859 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Fix For: 1.0.0 > > > CircleCI builds are failing because the layer caching is not available for > free plans. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-6859) [CI][Nightly] Disable docker layer caching for CircleCI tasks
Krisztian Szucs created ARROW-6859: -- Summary: [CI][Nightly] Disable docker layer caching for CircleCI tasks Key: ARROW-6859 URL: https://issues.apache.org/jira/browse/ARROW-6859 Project: Apache Arrow Issue Type: Bug Components: Continuous Integration Reporter: Krisztian Szucs Fix For: 1.0.0 CircleCI builds are failing because the layer caching is not available for free plans. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-6905) [Packaging][OSX] Nightly builds on MacOS are failing because of brew compile timeouts
Krisztian Szucs created ARROW-6905: -- Summary: [Packaging][OSX] Nightly builds on MacOS are failing because of brew compile timeouts Key: ARROW-6905 URL: https://issues.apache.org/jira/browse/ARROW-6905 Project: Apache Arrow Issue Type: Bug Components: Packaging Reporter: Krisztian Szucs Home-brew in our packaging builds has recently started to compile the dependencies instead of installing precompiled binaries. I'm not sure what's the issue, perhaps it is because the too old Xcode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6905) [Packaging][OSX] Nightly builds on MacOS are failing because of brew compile timeouts
[ https://issues.apache.org/jira/browse/ARROW-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-6905. Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 5681 [https://github.com/apache/arrow/pull/5681] > [Packaging][OSX] Nightly builds on MacOS are failing because of brew compile > timeouts > - > > Key: ARROW-6905 > URL: https://issues.apache.org/jira/browse/ARROW-6905 > Project: Apache Arrow > Issue Type: Bug > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Neal Richardson >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > Home-brew in our packaging builds has recently started to compile the > dependencies instead of installing precompiled binaries. I'm not sure what's > the issue, perhaps it is because the too old Xcode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-6937) [Packaging][Python] Fix conda linux and OSX wheel nightly builds
Krisztian Szucs created ARROW-6937: -- Summary: [Packaging][Python] Fix conda linux and OSX wheel nightly builds Key: ARROW-6937 URL: https://issues.apache.org/jira/browse/ARROW-6937 Project: Apache Arrow Issue Type: Bug Components: Packaging Reporter: Krisztian Szucs Assignee: Krisztian Szucs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6937) [Packaging][Python] Fix conda linux and OSX wheel nightly builds
[ https://issues.apache.org/jira/browse/ARROW-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-6937. Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 5647 [https://github.com/apache/arrow/pull/5647] > [Packaging][Python] Fix conda linux and OSX wheel nightly builds > > > Key: ARROW-6937 > URL: https://issues.apache.org/jira/browse/ARROW-6937 > Project: Apache Arrow > Issue Type: Bug > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6922) [Python] Pandas master build is failing (MultiIndex.levels change)
[ https://issues.apache.org/jira/browse/ARROW-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-6922. Fix Version/s: (was: 0.15.1) Resolution: Fixed Issue resolved by pull request 5695 [https://github.com/apache/arrow/pull/5695] > [Python] Pandas master build is failing (MultiIndex.levels change) > -- > > Key: ARROW-6922 > URL: https://issues.apache.org/jira/browse/ARROW-6922 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Joris Van den Bossche >Assignee: Joris Van den Bossche >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > https://circleci.com/gh/ursa-labs/crossbow/3933?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link > Caused by https://github.com/pandas-dev/pandas/pull/27242 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6922) [Python] Pandas master build is failing (MultiIndex.levels change)
[ https://issues.apache.org/jira/browse/ARROW-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6922: --- Fix Version/s: 0.15.1 > [Python] Pandas master build is failing (MultiIndex.levels change) > -- > > Key: ARROW-6922 > URL: https://issues.apache.org/jira/browse/ARROW-6922 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Joris Van den Bossche >Assignee: Joris Van den Bossche >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0, 0.15.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > https://circleci.com/gh/ursa-labs/crossbow/3933?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link > Caused by https://github.com/pandas-dev/pandas/pull/27242 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-2863) [Python] Add context manager APIs to RecordBatch*Writer/Reader classes
[
https://issues.apache.org/jira/browse/ARROW-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs resolved ARROW-2863.
Resolution: Fixed
Issue resolved by pull request 5563
[https://github.com/apache/arrow/pull/5563]
> [Python] Add context manager APIs to RecordBatch*Writer/Reader classes
> --
>
> Key: ARROW-2863
> URL: https://issues.apache.org/jira/browse/ARROW-2863
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Reporter: Wes McKinney
>Assignee: Krisztian Szucs
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> This would cause the {{close}} method to be called when the scope exits
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-6939) [Packaging][Crossbow] Always upload binary artifacts regardless of the test result
Krisztian Szucs created ARROW-6939: -- Summary: [Packaging][Crossbow] Always upload binary artifacts regardless of the test result Key: ARROW-6939 URL: https://issues.apache.org/jira/browse/ARROW-6939 Project: Apache Arrow Issue Type: Improvement Components: Packaging Reporter: Krisztian Szucs For debugging purposes the produced binaries are useful even if they are corrupted. We should upload the binaries to GitHub right after the binary is available, and run the tests afterwards. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6936) [Python] Improve error message when object of wrong type is given
[
https://issues.apache.org/jira/browse/ARROW-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs resolved ARROW-6936.
Resolution: Fixed
Issue resolved by pull request 5696
[https://github.com/apache/arrow/pull/5696]
> [Python] Improve error message when object of wrong type is given
> -
>
> Key: ARROW-6936
> URL: https://issues.apache.org/jira/browse/ARROW-6936
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Affects Versions: 0.15.0
>Reporter: Antoine Pitrou
>Assignee: Antoine Pitrou
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In the PyArrow {{unwrap_xxx}} functions, the error messages when an object of
> the wrong type is given are not very informative (e.g. "Could not unwrap
> Buffer from the passed Python object.").
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ARROW-6808) [Ruby] Ensure requiring suitable MSYS2 package
[
https://issues.apache.org/jira/browse/ARROW-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs resolved ARROW-6808.
Fix Version/s: 1.0.0
Resolution: Fixed
Issue resolved by pull request 5703
[https://github.com/apache/arrow/pull/5703]
> [Ruby] Ensure requiring suitable MSYS2 package
> --
>
> Key: ARROW-6808
> URL: https://issues.apache.org/jira/browse/ARROW-6808
> Project: Apache Arrow
> Issue Type: Bug
> Components: Ruby
> Environment: windows 10 msys2
>Reporter: Dominic Sisneros
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> C:\Users\Dominic E Sisneros>gem update red-arrow
> Updating installed gems
> Updating red-arrow
> Temporarily enhancing PATH for MSYS/MINGW...
> Installing required msys2 packages: mingw-w64-x86_64-arrow
> warning: mingw-w64-x86_64-arrow-0.14.0-2 is up to date -- skipping
> Building native extensions. This could take a while...
> ERROR: Error installing red-arrow:
> ERROR: Failed to build gem native extension.
> current directory:
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0/ext/arrow
> c:/Ruby26-x64/bin/ruby.exe -I c:/Ruby26-x64/lib/ruby/site_ruby/2.6.0 -r
> ./siteconf20191007-20416-1m6q3x3.rb extconf.rb
> checking --enable-debug-build option... no
> checking C++ compiler... x86_64-w64-mingw32-g++
> checking g++ version... 9.2 (gnu++14)
> checking for --enable-debug-build option... no
> checking for -Wall option to compiler... yes
> checking for -Waggregate-return option to compiler... yes
> checking for -Wcast-align option to compiler... yes
> checking for -Wextra option to compiler... no
> checking for -Wformat=2 option to compiler... yes
> checking for -Winit-self option to compiler... yes
> checking for -Wlarger-than-65500 option to compiler... yes
> checking for -Wmissing-declarations option to compiler... yes
> checking for -Wmissing-format-attribute option to compiler... yes
> checking for -Wmissing-include-dirs option to compiler... yes
> checking for -Wmissing-noreturn option to compiler... yes
> checking for -Wmissing-prototypes option to compiler... yes
> checking for -Wnested-externs option to compiler... no
> checking for -Wold-style-definition option to compiler... yes
> checking for -Wpacked option to compiler... yes
> checking for -Wp,-D_FORTIFY_SOURCE=2 option to compiler... yes
> checking for -Wpointer-arith option to compiler... yes
> checking for -Wswitch-default option to compiler... yes
> checking for -Wswitch-enum option to compiler... yes
> checking for -Wundef option to compiler... yes
> checking for -Wout-of-line-declaration option to compiler... no
> checking for -Wunsafe-loop-optimizations option to compiler... yes
> checking for -Wwrite-strings option to compiler... yes
> checking for Homebrew... no
> checking for arrow... yes
> checking for arrow-glib... yes
> creating Makefile
> current directory:
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0/ext/arrow
> make "DESTDIR=" clean
> current directory:
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0/ext/arrow
> make "DESTDIR="
> generating arrow-x64-mingw32.def
> compiling arrow.cpp
> compiling converters.cpp
> compiling raw-records.cpp
> raw-records.cpp: In lambda function:
> raw-records.cpp:61:52: error: 'class arrow::Column' has no member named
> 'chunks'
>61 | for (const auto array : chunked_array->chunks()) {
> |^~
> make: *** [Makefile:236: raw-records.o] Error 1
> make failed, exit code 2
> Gem files will remain installed in
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0 for inspection.
> Results logged to
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/extensions/x64-mingw32/2.6.0/red-arrow-0.15.0/gem_make.out
> Gems updated: red-arrow
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-6808) [Ruby] Ensure requiring suitable MSYS2 package
[
https://issues.apache.org/jira/browse/ARROW-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs updated ARROW-6808:
---
Summary: [Ruby] Ensure requiring suitable MSYS2 package (was: [ruby]
Doesn't build on windows msys2)
> [Ruby] Ensure requiring suitable MSYS2 package
> --
>
> Key: ARROW-6808
> URL: https://issues.apache.org/jira/browse/ARROW-6808
> Project: Apache Arrow
> Issue Type: Bug
> Components: Ruby
> Environment: windows 10 msys2
>Reporter: Dominic Sisneros
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> C:\Users\Dominic E Sisneros>gem update red-arrow
> Updating installed gems
> Updating red-arrow
> Temporarily enhancing PATH for MSYS/MINGW...
> Installing required msys2 packages: mingw-w64-x86_64-arrow
> warning: mingw-w64-x86_64-arrow-0.14.0-2 is up to date -- skipping
> Building native extensions. This could take a while...
> ERROR: Error installing red-arrow:
> ERROR: Failed to build gem native extension.
> current directory:
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0/ext/arrow
> c:/Ruby26-x64/bin/ruby.exe -I c:/Ruby26-x64/lib/ruby/site_ruby/2.6.0 -r
> ./siteconf20191007-20416-1m6q3x3.rb extconf.rb
> checking --enable-debug-build option... no
> checking C++ compiler... x86_64-w64-mingw32-g++
> checking g++ version... 9.2 (gnu++14)
> checking for --enable-debug-build option... no
> checking for -Wall option to compiler... yes
> checking for -Waggregate-return option to compiler... yes
> checking for -Wcast-align option to compiler... yes
> checking for -Wextra option to compiler... no
> checking for -Wformat=2 option to compiler... yes
> checking for -Winit-self option to compiler... yes
> checking for -Wlarger-than-65500 option to compiler... yes
> checking for -Wmissing-declarations option to compiler... yes
> checking for -Wmissing-format-attribute option to compiler... yes
> checking for -Wmissing-include-dirs option to compiler... yes
> checking for -Wmissing-noreturn option to compiler... yes
> checking for -Wmissing-prototypes option to compiler... yes
> checking for -Wnested-externs option to compiler... no
> checking for -Wold-style-definition option to compiler... yes
> checking for -Wpacked option to compiler... yes
> checking for -Wp,-D_FORTIFY_SOURCE=2 option to compiler... yes
> checking for -Wpointer-arith option to compiler... yes
> checking for -Wswitch-default option to compiler... yes
> checking for -Wswitch-enum option to compiler... yes
> checking for -Wundef option to compiler... yes
> checking for -Wout-of-line-declaration option to compiler... no
> checking for -Wunsafe-loop-optimizations option to compiler... yes
> checking for -Wwrite-strings option to compiler... yes
> checking for Homebrew... no
> checking for arrow... yes
> checking for arrow-glib... yes
> creating Makefile
> current directory:
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0/ext/arrow
> make "DESTDIR=" clean
> current directory:
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0/ext/arrow
> make "DESTDIR="
> generating arrow-x64-mingw32.def
> compiling arrow.cpp
> compiling converters.cpp
> compiling raw-records.cpp
> raw-records.cpp: In lambda function:
> raw-records.cpp:61:52: error: 'class arrow::Column' has no member named
> 'chunks'
>61 | for (const auto array : chunked_array->chunks()) {
> |^~
> make: *** [Makefile:236: raw-records.o] Error 1
> make failed, exit code 2
> Gem files will remain installed in
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/red-arrow-0.15.0 for inspection.
> Results logged to
> c:/Ruby26-x64/lib/ruby/gems/2.6.0/extensions/x64-mingw32/2.6.0/red-arrow-0.15.0/gem_make.out
> Gems updated: red-arrow
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-6855) [C++][Python][Flight] Implement Flight middleware
[ https://issues.apache.org/jira/browse/ARROW-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6855: --- Fix Version/s: 0.15.1 > [C++][Python][Flight] Implement Flight middleware > - > > Key: ARROW-6855 > URL: https://issues.apache.org/jira/browse/ARROW-6855 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Python >Reporter: Wes McKinney >Assignee: David Li >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0, 0.15.1 > > Time Spent: 20m > Remaining Estimate: 0h > > C++/Python side of ARROW-6074 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6610) [C++] Add ARROW_FILESYSTEM=ON/OFF CMake configuration flag
[ https://issues.apache.org/jira/browse/ARROW-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6610: --- Fix Version/s: (was: 0.15.0) 0.15.1 1.0.0 > [C++] Add ARROW_FILESYSTEM=ON/OFF CMake configuration flag > -- > > Key: ARROW-6610 > URL: https://issues.apache.org/jira/browse/ARROW-6610 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Antoine Pitrou >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0, 0.15.1 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Building this code should not be required in order to take advantage of the > columnar core (memory allocation, data structures, IPC) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6631) [C++] Do not build with any compression library dependencies by default
[ https://issues.apache.org/jira/browse/ARROW-6631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6631: --- Fix Version/s: (was: 0.15.1) > [C++] Do not build with any compression library dependencies by default > --- > > Key: ARROW-6631 > URL: https://issues.apache.org/jira/browse/ARROW-6631 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 3h > Remaining Estimate: 0h > > Numerous packaging scripts will have to be updated if we decide to do this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6831) [R] Update R macOS/Windows builds for change in cmake compression defaults
[ https://issues.apache.org/jira/browse/ARROW-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6831: --- Fix Version/s: (was: 0.15.1) > [R] Update R macOS/Windows builds for change in cmake compression defaults > -- > > Key: ARROW-6831 > URL: https://issues.apache.org/jira/browse/ARROW-6831 > Project: Apache Arrow > Issue Type: Improvement > Components: R >Reporter: Neal Richardson >Assignee: Neal Richardson >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > ARROW-6631 changed the defaults for including compressions but did not update > these build scripts. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6855) [C++][Python][Flight] Implement Flight middleware
[ https://issues.apache.org/jira/browse/ARROW-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6855: --- Fix Version/s: (was: 0.15.1) > [C++][Python][Flight] Implement Flight middleware > - > > Key: ARROW-6855 > URL: https://issues.apache.org/jira/browse/ARROW-6855 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Python >Reporter: Wes McKinney >Assignee: David Li >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > C++/Python side of ARROW-6074 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6864) [C++] bz2 / zstd tests not enabled
[
https://issues.apache.org/jira/browse/ARROW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs updated ARROW-6864:
---
Fix Version/s: (was: 0.15.1)
> [C++] bz2 / zstd tests not enabled
> --
>
> Key: ARROW-6864
> URL: https://issues.apache.org/jira/browse/ARROW-6864
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.15.0
>Reporter: Antoine Pitrou
>Assignee: Wes McKinney
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When passing {{-DARROW_WITH_ZSTD=on}} and {{-DARROW_WITH_BZ2=on}}, the
> relevant tests in {{arrow-compression-test}} and {{arrow-io-compressed-test}}
> are still not enabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-6963) [Packaging] Use crossbow's command to deploy artifacts from travis builds
Krisztian Szucs created ARROW-6963: -- Summary: [Packaging] Use crossbow's command to deploy artifacts from travis builds Key: ARROW-6963 URL: https://issues.apache.org/jira/browse/ARROW-6963 Project: Apache Arrow Issue Type: Improvement Components: Packaging Reporter: Krisztian Szucs Travis starts to fail more often during artefact deployment to GitHub releases. Crossbow has a builtin command to upload the artifacts which is more reliable. All of the travis builds should use the crossbow script instead of relying on travis's deployment feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6948) [Rust] [Parquet] Fix bool array support in arrow reader.
[ https://issues.apache.org/jira/browse/ARROW-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-6948. Resolution: Fixed Issue resolved by pull request 5705 [https://github.com/apache/arrow/pull/5705] > [Rust] [Parquet] Fix bool array support in arrow reader. > > > Key: ARROW-6948 > URL: https://issues.apache.org/jira/browse/ARROW-6948 > Project: Apache Arrow > Issue Type: Bug > Components: Rust >Reporter: Renjie Liu >Assignee: Renjie Liu >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6984) [C++] Update LZ4 to 1.9.2 for CVE-2019-17543
[ https://issues.apache.org/jira/browse/ARROW-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6984: --- Summary: [C++] Update LZ4 to 1.9.2 for CVE-2019-17543 (was: Update LZ4 to 1.9.2 for CVE-2019-17543) > [C++] Update LZ4 to 1.9.2 for CVE-2019-17543 > > > Key: ARROW-6984 > URL: https://issues.apache.org/jira/browse/ARROW-6984 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Affects Versions: 0.15.0 >Reporter: Sangeeth Keeriyadath >Priority: Major > Fix For: 0.15.1 > > > There is a reported CVE that LZ4 before 1.9.2 has a heap-based buffer > overflow in LZ4_write32 (More details in here - > [https://nvd.nist.gov/vuln/detail/CVE-2019-17543] ). I see that Apache Arrow > uses *v1.8.3* version ( > [https://github.com/apache/arrow/blob/47e5ecafa72b70112a64a1174b29b9db45f803ef/cpp/thirdparty/versions.txt#L38] > ). > We need to bump up the dependency version of LZ4 to *1.9.2* to get past the > reported CVE. Thank you! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-6984) [C++] Update LZ4 to 1.9.2 for CVE-2019-17543
[ https://issues.apache.org/jira/browse/ARROW-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs reassigned ARROW-6984: -- Assignee: Krisztian Szucs > [C++] Update LZ4 to 1.9.2 for CVE-2019-17543 > > > Key: ARROW-6984 > URL: https://issues.apache.org/jira/browse/ARROW-6984 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Affects Versions: 0.15.0 >Reporter: Sangeeth Keeriyadath >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 0.15.1 > > Time Spent: 20m > Remaining Estimate: 0h > > There is a reported CVE that LZ4 before 1.9.2 has a heap-based buffer > overflow in LZ4_write32 (More details in here - > [https://nvd.nist.gov/vuln/detail/CVE-2019-17543] ). I see that Apache Arrow > uses *v1.8.3* version ( > [https://github.com/apache/arrow/blob/47e5ecafa72b70112a64a1174b29b9db45f803ef/cpp/thirdparty/versions.txt#L38] > ). > We need to bump up the dependency version of LZ4 to *1.9.2* to get past the > reported CVE. Thank you! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6963) [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from travis builds
[ https://issues.apache.org/jira/browse/ARROW-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6963: --- Summary: [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from travis builds (was: [Packaging] Use crossbow's command to deploy artifacts from travis builds) > [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from > travis builds > - > > Key: ARROW-6963 > URL: https://issues.apache.org/jira/browse/ARROW-6963 > Project: Apache Arrow > Issue Type: Improvement > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Travis starts to fail more often during artefact deployment to GitHub > releases. > Crossbow has a builtin command to upload the artifacts which is more reliable. > All of the travis builds should use the crossbow script instead of relying on > travis's deployment feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-6963) [Packaging] Use crossbow's command to deploy artifacts from travis builds
[ https://issues.apache.org/jira/browse/ARROW-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs reassigned ARROW-6963: -- Assignee: Krisztian Szucs > [Packaging] Use crossbow's command to deploy artifacts from travis builds > - > > Key: ARROW-6963 > URL: https://issues.apache.org/jira/browse/ARROW-6963 > Project: Apache Arrow > Issue Type: Improvement > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Travis starts to fail more often during artefact deployment to GitHub > releases. > Crossbow has a builtin command to upload the artifacts which is more reliable. > All of the travis builds should use the crossbow script instead of relying on > travis's deployment feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6963) [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from travis builds
[ https://issues.apache.org/jira/browse/ARROW-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6963: --- Fix Version/s: 0.15.1 1.0.0 > [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from > travis builds > - > > Key: ARROW-6963 > URL: https://issues.apache.org/jira/browse/ARROW-6963 > Project: Apache Arrow > Issue Type: Improvement > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0, 0.15.1 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Travis starts to fail more often during artefact deployment to GitHub > releases. > Crossbow has a builtin command to upload the artifacts which is more reliable. > All of the travis builds should use the crossbow script instead of relying on > travis's deployment feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6963) [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from travis builds
[ https://issues.apache.org/jira/browse/ARROW-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-6963. Fix Version/s: (was: 0.15.1) Resolution: Fixed Issue resolved by pull request 5726 [https://github.com/apache/arrow/pull/5726] > [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from > travis builds > - > > Key: ARROW-6963 > URL: https://issues.apache.org/jira/browse/ARROW-6963 > Project: Apache Arrow > Issue Type: Improvement > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Travis starts to fail more often during artefact deployment to GitHub > releases. > Crossbow has a builtin command to upload the artifacts which is more reliable. > All of the travis builds should use the crossbow script instead of relying on > travis's deployment feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6963) [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from travis builds
[ https://issues.apache.org/jira/browse/ARROW-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6963: --- Fix Version/s: 0.15.1 > [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from > travis builds > - > > Key: ARROW-6963 > URL: https://issues.apache.org/jira/browse/ARROW-6963 > Project: Apache Arrow > Issue Type: Improvement > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0, 0.15.1 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Travis starts to fail more often during artefact deployment to GitHub > releases. > Crossbow has a builtin command to upload the artifacts which is more reliable. > All of the travis builds should use the crossbow script instead of relying on > travis's deployment feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6983) [C++] Threaded task group crashes sometimes
[ https://issues.apache.org/jira/browse/ARROW-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6983: --- Fix Version/s: 0.15.1 > [C++] Threaded task group crashes sometimes > --- > > Key: ARROW-6983 > URL: https://issues.apache.org/jira/browse/ARROW-6983 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Neal Richardson >Assignee: Antoine Pitrou >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0, 0.15.1 > > Time Spent: 2h > Remaining Estimate: 0h > > You can give this a more descriptive title :) > See discussion on ARROW-6977. > https://gist.github.com/pitrou/87f3091c226db3306c45b2c32dd9aea8 seems to fix > it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6984) [C++] Update LZ4 to 1.9.2 for CVE-2019-17543
[ https://issues.apache.org/jira/browse/ARROW-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6984: --- Fix Version/s: (was: 0.15.1) 1.0.0 > [C++] Update LZ4 to 1.9.2 for CVE-2019-17543 > > > Key: ARROW-6984 > URL: https://issues.apache.org/jira/browse/ARROW-6984 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Affects Versions: 0.15.0 >Reporter: Sangeeth Keeriyadath >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > There is a reported CVE that LZ4 before 1.9.2 has a heap-based buffer > overflow in LZ4_write32 (More details in here - > [https://nvd.nist.gov/vuln/detail/CVE-2019-17543] ). I see that Apache Arrow > uses *v1.8.3* version ( > [https://github.com/apache/arrow/blob/47e5ecafa72b70112a64a1174b29b9db45f803ef/cpp/thirdparty/versions.txt#L38] > ). > We need to bump up the dependency version of LZ4 to *1.9.2* to get past the > reported CVE. Thank you! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6962) [C++] [CI] Stop compiling with -Weverything
[
https://issues.apache.org/jira/browse/ARROW-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs updated ARROW-6962:
---
Fix Version/s: 0.15.1
> [C++] [CI] Stop compiling with -Weverything
> ---
>
> Key: ARROW-6962
> URL: https://issues.apache.org/jira/browse/ARROW-6962
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Assignee: Antoine Pitrou
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0, 0.15.1
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> We should simply use {{-Wall}} instead.
> [https://quuxplusone.github.io/blog/2018/12/06/dont-use-weverything/]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-6993) [CI] Pass -allowUntrasted flag during Macos SDK installation on Travis
Krisztian Szucs created ARROW-6993: -- Summary: [CI] Pass -allowUntrasted flag during Macos SDK installation on Travis Key: ARROW-6993 URL: https://issues.apache.org/jira/browse/ARROW-6993 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Krisztian Szucs See the failing build at https://travis-ci.org/apache/arrow/jobs/602560324 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6993) [CI] Pass -allowUntrasted flag during Macos SDK installation on Travis
[ https://issues.apache.org/jira/browse/ARROW-6993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6993: --- Issue Type: Bug (was: Improvement) > [CI] Pass -allowUntrasted flag during Macos SDK installation on Travis > -- > > Key: ARROW-6993 > URL: https://issues.apache.org/jira/browse/ARROW-6993 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration >Reporter: Krisztian Szucs >Priority: Major > > See the failing build at https://travis-ci.org/apache/arrow/jobs/602560324 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6993) [CI] Macos SDK installation fails on Travis
[ https://issues.apache.org/jira/browse/ARROW-6993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6993: --- Description: See the failing build at https://travis-ci.org/apache/arrow/jobs/602560324 Pass -allowUntrasted flag during the installation. was:See the failing build at https://travis-ci.org/apache/arrow/jobs/602560324 > [CI] Macos SDK installation fails on Travis > > > Key: ARROW-6993 > URL: https://issues.apache.org/jira/browse/ARROW-6993 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration >Reporter: Krisztian Szucs >Priority: Major > > See the failing build at https://travis-ci.org/apache/arrow/jobs/602560324 > Pass -allowUntrasted flag during the installation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-6993) [CI] Macos SDK installation fails on Travis
[ https://issues.apache.org/jira/browse/ARROW-6993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-6993: --- Summary: [CI] Macos SDK installation fails on Travis (was: [CI] Pass -allowUntrasted flag during Macos SDK installation on Travis) > [CI] Macos SDK installation fails on Travis > > > Key: ARROW-6993 > URL: https://issues.apache.org/jira/browse/ARROW-6993 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration >Reporter: Krisztian Szucs >Priority: Major > > See the failing build at https://travis-ci.org/apache/arrow/jobs/602560324 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-6995) [Packaging][Crossbow] The windows conda artifacts are not uploaded to GitHub releases
Krisztian Szucs created ARROW-6995: -- Summary: [Packaging][Crossbow] The windows conda artifacts are not uploaded to GitHub releases Key: ARROW-6995 URL: https://issues.apache.org/jira/browse/ARROW-6995 Project: Apache Arrow Issue Type: Improvement Components: Packaging Reporter: Krisztian Szucs The artifacts should be uploaded under the appropriate tag: https://github.com/ursa-labs/crossbow/releases/tag/ursabot-289-azure-conda-win-vs2015-py37 Most certainly the artifacts are produced in a different directory than previously, so the uploading script cannot find it https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=2180 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-6988) [CI][R] Buildbot's R Conda is failing
[
https://issues.apache.org/jira/browse/ARROW-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959967#comment-16959967
]
Krisztian Szucs commented on ARROW-6988:
I've managed to reproduce it locally, but I don't have time to investigate it
further right now, so I've turned it off.
> [CI][R] Buildbot's R Conda is failing
> -
>
> Key: ARROW-6988
> URL: https://issues.apache.org/jira/browse/ARROW-6988
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration, R
>Reporter: Francois Saint-Jacques
>Priority: Major
>
> {code:java}
> Running ‘testthat.R’
> ERROR
> Running the tests in ‘tests/testthat.R’ failed.
> Last 13 lines of output:
> 25: tryCatch(withCallingHandlers({eval(code, test_env)if (!handled
> && !is.null(test)) {skip_empty()}}, expectation =
> handle_expectation, skip = handle_skip, warning = handle_warning, message
> = handle_message, error = handle_error), error = handle_fatal, skip =
> function(e) {})
> 26: test_code(NULL, exprs, env)
> 27: source_file(path, new.env(parent = env), chdir = TRUE, wrap = wrap)
> 28: force(code)
> 29: with_reporter(reporter = reporter, start_end_reporter =
> start_end_reporter, {reporter$start_file(basename(path))
> lister$start_file(basename(path))source_file(path, new.env(parent =
> env), chdir = TRUE, wrap = wrap)reporter$.end_context()
> reporter$end_file()})
> 30: FUN(X[[i]], ...)
> 31: lapply(paths, test_file, env = env, reporter = current_reporter,
> start_end_reporter = FALSE, load_helpers = FALSE, wrap = wrap)
> 32: force(code)
> 33: with_reporter(reporter = current_reporter, results <- lapply(paths,
> test_file, env = env, reporter = current_reporter, start_end_reporter =
> FALSE, load_helpers = FALSE, wrap = wrap))
> 34: test_files(paths, reporter = reporter, env = env, stop_on_failure =
> stop_on_failure, stop_on_warning = stop_on_warning, wrap = wrap)
> 35: test_dir(path = test_path, reporter = reporter, env = env, filter =
> filter, ..., stop_on_failure = stop_on_failure, stop_on_warning =
> stop_on_warning, wrap = wrap)
> 36: test_package_dir(package = package, test_path = test_path, filter =
> filter, reporter = reporter, ..., stop_on_failure = stop_on_failure,
> stop_on_warning = stop_on_warning, wrap = wrap)
> 37: test_check("arrow")
> An irrecoverable exception occurred. R is aborting now ...
> Segmentation fault (core dumped)
> * checking for unstated dependencies in vignettes ... OK
> * checking package vignettes in ‘inst/doc’ ... OK
> * checking re-building of vignette outputs ... OK
> * DONE
> Status: 1 ERROR, 1 WARNING, 2 NOTEs
> See
> ‘/buildbot/AMD64_Conda_R/r/arrow.Rcheck/00check.log’
> for details.
> {code}
> [|https://ci.ursalabs.org/#/builders/95]
> [https://ci.ursalabs.org/#/builders/95/builds/2386]
> [https://ci.ursalabs.org/#/builders/95]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-6917) [Developer] Implement Python script to generate git cherry-pick commands needed to create patch build branch for maint releases
[ https://issues.apache.org/jira/browse/ARROW-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960668#comment-16960668 ] Krisztian Szucs commented on ARROW-6917: Here is an implementation https://gist.github.com/kszucs/b2743546044ccd3215e5bb34fa0d76a0 > [Developer] Implement Python script to generate git cherry-pick commands > needed to create patch build branch for maint releases > --- > > Key: ARROW-6917 > URL: https://issues.apache.org/jira/browse/ARROW-6917 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Wes McKinney >Priority: Major > Fix For: 1.0.0 > > > For 0.14.1, I maintained this script by hand. It would be less failure-prone > (maybe) to generate it based on the fix versions set in JIRA -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7032) [Release] Verify python wheels in the release verification script
Krisztian Szucs created ARROW-7032: -- Summary: [Release] Verify python wheels in the release verification script Key: ARROW-7032 URL: https://issues.apache.org/jira/browse/ARROW-7032 Project: Apache Arrow Issue Type: Improvement Components: Packaging Reporter: Krisztian Szucs Fix For: 1.0.0 For linux wheels use docker, otherwise setup a virtualenv and install the wheel supported on the host's platform. Testing should include the imports for the optional modules and perhaps running the unit tests, but the import testing should catch most of the wheel issues. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7039) [Python] Typecheck expects pandas to be installed
Krisztian Szucs created ARROW-7039:
--
Summary: [Python] Typecheck expects pandas to be installed
Key: ARROW-7039
URL: https://issues.apache.org/jira/browse/ARROW-7039
Project: Apache Arrow
Issue Type: Improvement
Components: Python
Reporter: Krisztian Szucs
Fix For: 1.0.0
See nightly build failure:
https://circleci.com/gh/ursa-labs/crossbow/4285?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
The following patch fixes it:
{code}
diff --git a/python/pyarrow/table.pxi b/python/pyarrow/table.pxi
index b65dac4cb..e0a82fd76 100644
--- a/python/pyarrow/table.pxi
+++ b/python/pyarrow/table.pxi
@@ -17,6 +17,7 @@
import warnings
+
cdef class ChunkedArray(_PandasConvertible):
"""
Array backed via one or more memory chunks.
@@ -1579,7 +1580,7 @@ def record_batch(data, names=None, schema=None,
metadata=None):
if isinstance(data, (list, tuple)):
return RecordBatch.from_arrays(data, names=names, schema=schema,
metadata=metadata)
-elif isinstance(data, _pandas_api.pd.DataFrame):
+elif _pandas_api.is_data_frame(data):
return RecordBatch.from_pandas(data, schema=schema)
else:
return TypeError("Expected pandas DataFrame or python dictionary")
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-7039) [Python] Typecheck expects pandas to be installed
[
https://issues.apache.org/jira/browse/ARROW-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963936#comment-16963936
]
Krisztian Szucs commented on ARROW-7039:
cc [~jorisvandenbossche]
> [Python] Typecheck expects pandas to be installed
> -
>
> Key: ARROW-7039
> URL: https://issues.apache.org/jira/browse/ARROW-7039
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Reporter: Krisztian Szucs
>Priority: Major
> Fix For: 1.0.0
>
>
> See nightly build failure:
> https://circleci.com/gh/ursa-labs/crossbow/4285?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
> The following patch fixes it:
> {code}
> diff --git a/python/pyarrow/table.pxi b/python/pyarrow/table.pxi
> index b65dac4cb..e0a82fd76 100644
> --- a/python/pyarrow/table.pxi
> +++ b/python/pyarrow/table.pxi
> @@ -17,6 +17,7 @@
> import warnings
> +
> cdef class ChunkedArray(_PandasConvertible):
> """
> Array backed via one or more memory chunks.
> @@ -1579,7 +1580,7 @@ def record_batch(data, names=None, schema=None,
> metadata=None):
> if isinstance(data, (list, tuple)):
> return RecordBatch.from_arrays(data, names=names, schema=schema,
> metadata=metadata)
> -elif isinstance(data, _pandas_api.pd.DataFrame):
> +elif _pandas_api.is_data_frame(data):
> return RecordBatch.from_pandas(data, schema=schema)
> else:
> return TypeError("Expected pandas DataFrame or python dictionary")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ARROW-7039) [Python] Typecheck expects pandas to be installed
[
https://issues.apache.org/jira/browse/ARROW-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs resolved ARROW-7039.
Resolution: Fixed
Issue resolved by pull request 5764
[https://github.com/apache/arrow/pull/5764]
> [Python] Typecheck expects pandas to be installed
> -
>
> Key: ARROW-7039
> URL: https://issues.apache.org/jira/browse/ARROW-7039
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Reporter: Krisztian Szucs
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> See nightly build failure:
> https://circleci.com/gh/ursa-labs/crossbow/4285?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
> The following patch fixes it:
> {code}
> diff --git a/python/pyarrow/table.pxi b/python/pyarrow/table.pxi
> index b65dac4cb..e0a82fd76 100644
> --- a/python/pyarrow/table.pxi
> +++ b/python/pyarrow/table.pxi
> @@ -17,6 +17,7 @@
> import warnings
> +
> cdef class ChunkedArray(_PandasConvertible):
> """
> Array backed via one or more memory chunks.
> @@ -1579,7 +1580,7 @@ def record_batch(data, names=None, schema=None,
> metadata=None):
> if isinstance(data, (list, tuple)):
> return RecordBatch.from_arrays(data, names=names, schema=schema,
> metadata=metadata)
> -elif isinstance(data, _pandas_api.pd.DataFrame):
> +elif _pandas_api.is_data_frame(data):
> return RecordBatch.from_pandas(data, schema=schema)
> else:
> return TypeError("Expected pandas DataFrame or python dictionary")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-7039) [Python] Typecheck expects pandas to be installed
[
https://issues.apache.org/jira/browse/ARROW-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs reassigned ARROW-7039:
--
Assignee: Joris Van den Bossche
> [Python] Typecheck expects pandas to be installed
> -
>
> Key: ARROW-7039
> URL: https://issues.apache.org/jira/browse/ARROW-7039
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Reporter: Krisztian Szucs
>Assignee: Joris Van den Bossche
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> See nightly build failure:
> https://circleci.com/gh/ursa-labs/crossbow/4285?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
> The following patch fixes it:
> {code}
> diff --git a/python/pyarrow/table.pxi b/python/pyarrow/table.pxi
> index b65dac4cb..e0a82fd76 100644
> --- a/python/pyarrow/table.pxi
> +++ b/python/pyarrow/table.pxi
> @@ -17,6 +17,7 @@
> import warnings
> +
> cdef class ChunkedArray(_PandasConvertible):
> """
> Array backed via one or more memory chunks.
> @@ -1579,7 +1580,7 @@ def record_batch(data, names=None, schema=None,
> metadata=None):
> if isinstance(data, (list, tuple)):
> return RecordBatch.from_arrays(data, names=names, schema=schema,
> metadata=metadata)
> -elif isinstance(data, _pandas_api.pd.DataFrame):
> +elif _pandas_api.is_data_frame(data):
> return RecordBatch.from_pandas(data, schema=schema)
> else:
> return TypeError("Expected pandas DataFrame or python dictionary")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ARROW-6881) [Rust] Remove "array_ops" in favor of the "compute" sub-module
[ https://issues.apache.org/jira/browse/ARROW-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-6881. Resolution: Fixed Issue resolved by pull request 5757 [https://github.com/apache/arrow/pull/5757] > [Rust] Remove "array_ops" in favor of the "compute" sub-module > -- > > Key: ARROW-6881 > URL: https://issues.apache.org/jira/browse/ARROW-6881 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.15.0 >Reporter: Paddy Horan >Assignee: Paddy Horan >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Once 4591 (https://issues.apache.org/jira/browse/ARROW-4591) is complete only > filter and limit will remain in the "array_ops" module and they can be moved > under the "compute" sub-crate. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6989) [Python][C++] Assert is triggered when decimal type inference occurs on a value with out of range precision
[ https://issues.apache.org/jira/browse/ARROW-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-6989. Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 5733 [https://github.com/apache/arrow/pull/5733] > [Python][C++] Assert is triggered when decimal type inference occurs on a > value with out of range precision > --- > > Key: ARROW-6989 > URL: https://issues.apache.org/jira/browse/ARROW-6989 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Micah Kornfield >Assignee: Joris Van den Bossche >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Example: > pa.array([decimal.Decimal(123.234)] ) > > The problem is that inference.cc calls the direct constructor for decimal > types instead using Make. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-7035) [R] Default arguments are unclear in write_parquet docs
[
https://issues.apache.org/jira/browse/ARROW-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs updated ARROW-7035:
---
Fix Version/s: (was: 0.15.1)
1.0.0
> [R] Default arguments are unclear in write_parquet docs
> ---
>
> Key: ARROW-7035
> URL: https://issues.apache.org/jira/browse/ARROW-7035
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
>Affects Versions: 0.15.0
> Environment: Ubuntu with libparquet-dev 0.15.0-1, R 3.6.1, and arrow
> 0.15.0.
>Reporter: Karl Dunkle Werner
>Priority: Minor
> Labels: documentation
> Fix For: 1.0.0
>
>
> Thank you so much for adding support for reading and writing parquet files in
> R! I have a few questions about the user interface and optional arguments,
> but I want to highlight how great it is to have this useful filetype to pass
> data back and forth.
> The defaults for the optional arguments in {{arrow::write_parquet}} aren't
> always clear. Here were my questions after reading the help docs from
> {{write_parquet}}:
> * What's the default {{version}}? Should a user prefer "2.0" for new
> projects?
> * What are acceptable values for {{compression}}? (Answer: {{uncompressed}},
> {{snappy}}, {{gzip}}, {{brotli}}, {{zstd}}, or {{lz4}}.)
> * What's the default for {{use_dictionary}}? Seems to be {{TRUE}}, at least
> some of the time.
> * What's the default for {{write_statistics}}? Should a user prefer {{TRUE}}?
> * Can I assume {{allow_truncated_timestamps}} is {{FALSE}} by default?
> As someone who works in both R and Python, I was a little surprised when
> pyarrow uses snappy compression by default, but R's default is uncompressed.
> My preference would be having the same default arguments, but that might be a
> fringe use-case.
> While I was digging into this, I was surprised that
> {{ParquetReaderProperties}} is exported and documented, but
> {{ParquetWriterProperties}} isn't. Is that intentional?
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7044) [Release] Create a post release script for the home-brew formulas
Krisztian Szucs created ARROW-7044: -- Summary: [Release] Create a post release script for the home-brew formulas Key: ARROW-7044 URL: https://issues.apache.org/jira/browse/ARROW-7044 Project: Apache Arrow Issue Type: Improvement Reporter: Krisztian Szucs Fix For: 1.0.0 The required steps are documented in the release management guide https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7046) [Release] Dockerize post-03-website.sh
Krisztian Szucs created ARROW-7046: -- Summary: [Release] Dockerize post-03-website.sh Key: ARROW-7046 URL: https://issues.apache.org/jira/browse/ARROW-7046 Project: Apache Arrow Issue Type: Improvement Reporter: Krisztian Szucs For example on OSX the BSD date function doesn't accept the same arguments so the script fails. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7056) [Python] test errors without S3
[
https://issues.apache.org/jira/browse/ARROW-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16966759#comment-16966759
]
Krisztian Szucs commented on ARROW-7056:
[~apitrou] the fixture is marked with
[S3|https://github.com/apache/arrow/blob/master/python/pyarrow/tests/test_fs.py#L54]
which is set
[False|https://github.com/apache/arrow/blob/master/python/pyarrow/tests/conftest.py#L77]
by default, so it should not try to run the S3 related tests.
Could you check what is the pytest option's value?
> [Python] test errors without S3
> ---
>
> Key: ARROW-7056
> URL: https://issues.apache.org/jira/browse/ARROW-7056
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.15.1
>Reporter: Antoine Pitrou
>Priority: Major
>
> When running the tests I get errors like this:
> {code}
> ___ ERROR at setup of
> test_non_path_like_input_raises[S3FileSystem]
>
> Traceback (most recent call last):
> File "/home/antoine/arrow/dev/python/pyarrow/tests/test_fs.py", line 57, in
> s3fs
> from pyarrow.s3fs import S3Options, S3FileSystem
> File "/home/antoine/arrow/dev/python/pyarrow/s3fs.py", line 20, in
> from pyarrow._s3fs import ( # noqa
> ModuleNotFoundError: No module named 'pyarrow._s3fs'
> {code}
> It seems that those tests should be automatically skipped if S3 support isn't
> enabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7101) [CI] Refactor docker-compose setup and use it with GitHub Actions
Krisztian Szucs created ARROW-7101: -- Summary: [CI] Refactor docker-compose setup and use it with GitHub Actions Key: ARROW-7101 URL: https://issues.apache.org/jira/browse/ARROW-7101 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Krisztian Szucs Fix For: 1.0.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-7101) [CI] Refactor docker-compose setup and use it with GitHub Actions
[ https://issues.apache.org/jira/browse/ARROW-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs reassigned ARROW-7101: -- Assignee: Krisztian Szucs > [CI] Refactor docker-compose setup and use it with GitHub Actions > - > > Key: ARROW-7101 > URL: https://issues.apache.org/jira/browse/ARROW-7101 > Project: Apache Arrow > Issue Type: Improvement > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-7101) [CI] Refactor docker-compose setup and use it with GitHub Actions
[ https://issues.apache.org/jira/browse/ARROW-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-7101. Resolution: Fixed Issue resolved by pull request 5589 [https://github.com/apache/arrow/pull/5589] > [CI] Refactor docker-compose setup and use it with GitHub Actions > - > > Key: ARROW-7101 > URL: https://issues.apache.org/jira/browse/ARROW-7101 > Project: Apache Arrow > Issue Type: Improvement > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-7114) [JS][CI] NodeJS build fails on Github Actions Windows node
[
https://issues.apache.org/jira/browse/ARROW-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs updated ARROW-7114:
---
Description:
I had an attempt to install
[cross-env|https://github.com/apache/arrow/blob/master/.github/workflows/js.yml#L108]
as suggested by but I guess it requires a bit more work.
{code:java}
> NODE_NO_WARNINGS=1 gulp build
# 'NODE_NO_WARNINGS' is not recognized as an internal or external command,
# operable program or batch file.
# npm ERR! code ELIFECYCLE
# npm ERR! errno 1
# npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build`
# npm ERR! Exit status 1
# npm ERR!
# npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script.
# npm ERR! This is probably not a problem with npm. There is likely additional
logging output above. {code}
was:
I had an attempt to install
[cross-env|https://github.com/apache/arrow/blob/master/.github/workflows/js.yml#L108]
as suggested by [~paultaylor] but I guess it requires a bit more work.
{code:java}
> NODE_NO_WARNINGS=1 gulp build
# 'NODE_NO_WARNINGS' is not recognized as an internal or external command,
# operable program or batch file.
# npm ERR! code ELIFECYCLE
# npm ERR! errno 1
# npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build`
# npm ERR! Exit status 1
# npm ERR!
# npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script.
# npm ERR! This is probably not a problem with npm. There is likely additional
logging output above. {code}
> [JS][CI] NodeJS build fails on Github Actions Windows node
> --
>
> Key: ARROW-7114
> URL: https://issues.apache.org/jira/browse/ARROW-7114
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration, JavaScript
>Reporter: Krisztian Szucs
>Priority: Major
> Fix For: 1.0.0
>
>
> I had an attempt to install
> [cross-env|https://github.com/apache/arrow/blob/master/.github/workflows/js.yml#L108]
> as suggested by but I guess it requires a bit more work.
> {code:java}
> > NODE_NO_WARNINGS=1 gulp build
> # 'NODE_NO_WARNINGS' is not recognized as an internal or external command,
> # operable program or batch file.
> # npm ERR! code ELIFECYCLE
> # npm ERR! errno 1
> # npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build`
> # npm ERR! Exit status 1
> # npm ERR!
> # npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script.
> # npm ERR! This is probably not a problem with npm. There is likely
> additional logging output above. {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7114) [JS][CI] NodeJS build fails on Github Actions Windows node
Krisztian Szucs created ARROW-7114:
--
Summary: [JS][CI] NodeJS build fails on Github Actions Windows node
Key: ARROW-7114
URL: https://issues.apache.org/jira/browse/ARROW-7114
Project: Apache Arrow
Issue Type: Improvement
Components: Continuous Integration, JavaScript
Reporter: Krisztian Szucs
Fix For: 1.0.0
I had an attempt to install
[cross-env|https://github.com/apache/arrow/blob/master/.github/workflows/js.yml#L108]
as suggested by [~paultaylor] but I guess it requires a bit more work.
{code:java}
> NODE_NO_WARNINGS=1 gulp build
# 'NODE_NO_WARNINGS' is not recognized as an internal or external command,
# operable program or batch file.
# npm ERR! code ELIFECYCLE
# npm ERR! errno 1
# npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build`
# npm ERR! Exit status 1
# npm ERR!
# npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script.
# npm ERR! This is probably not a problem with npm. There is likely additional
logging output above. {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-7114) [JS][CI] NodeJS build fails on Github Actions Windows node
[
https://issues.apache.org/jira/browse/ARROW-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs updated ARROW-7114:
---
Description:
I had an attempt to install
[cross-env|https://github.com/apache/arrow/blob/master/.github/workflows/js.yml#L108]
as suggested by [~paul.e.taylor] but I guess it requires a bit more work.
{code:java}
> NODE_NO_WARNINGS=1 gulp build
# 'NODE_NO_WARNINGS' is not recognized as an internal or external command,
# operable program or batch file.
# npm ERR! code ELIFECYCLE
# npm ERR! errno 1
# npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build`
# npm ERR! Exit status 1
# npm ERR!
# npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script.
# npm ERR! This is probably not a problem with npm. There is likely additional
logging output above. {code}
was:
I had an attempt to install
[cross-env|https://github.com/apache/arrow/blob/master/.github/workflows/js.yml#L108]
as suggested by but I guess it requires a bit more work.
{code:java}
> NODE_NO_WARNINGS=1 gulp build
# 'NODE_NO_WARNINGS' is not recognized as an internal or external command,
# operable program or batch file.
# npm ERR! code ELIFECYCLE
# npm ERR! errno 1
# npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build`
# npm ERR! Exit status 1
# npm ERR!
# npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script.
# npm ERR! This is probably not a problem with npm. There is likely additional
logging output above. {code}
> [JS][CI] NodeJS build fails on Github Actions Windows node
> --
>
> Key: ARROW-7114
> URL: https://issues.apache.org/jira/browse/ARROW-7114
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration, JavaScript
>Reporter: Krisztian Szucs
>Priority: Major
> Fix For: 1.0.0
>
>
> I had an attempt to install
> [cross-env|https://github.com/apache/arrow/blob/master/.github/workflows/js.yml#L108]
> as suggested by [~paul.e.taylor] but I guess it requires a bit more work.
> {code:java}
> > NODE_NO_WARNINGS=1 gulp build
> # 'NODE_NO_WARNINGS' is not recognized as an internal or external command,
> # operable program or batch file.
> # npm ERR! code ELIFECYCLE
> # npm ERR! errno 1
> # npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build`
> # npm ERR! Exit status 1
> # npm ERR!
> # npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script.
> # npm ERR! This is probably not a problem with npm. There is likely
> additional logging output above. {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7115) [C++][CI] Running Gandiva tests fails on Fedora 29
Krisztian Szucs created ARROW-7115:
--
Summary: [C++][CI] Running Gandiva tests fails on Fedora 29
Key: ARROW-7115
URL: https://issues.apache.org/jira/browse/ARROW-7115
Project: Apache Arrow
Issue Type: Improvement
Components: C++ - Gandiva, Continuous Integration
Reporter: Krisztian Szucs
Reproduce with {{docker-compose run -e ARROW_GANDIVA=ON fedora-cpp}}
{code}
Running gandiva-internals-test, redirecting output into
/build/cpp/build/test-logs/gandiva-internals-test.txt (attempt 1/1)
1364
: CommandLine Error: Option 'x86-experimental-vector-widening-legalization'
registered more than once!
1365
LLVM ERROR: inconsistency in registered CommandLine options
1366
/build/cpp/src/gandiva
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-7116) [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions
[ https://issues.apache.org/jira/browse/ARROW-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs reassigned ARROW-7116: -- Assignee: Krisztian Szucs > [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions > --- > > Key: ARROW-7116 > URL: https://issues.apache.org/jira/browse/ARROW-7116 > Project: Apache Arrow > Issue Type: Task > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > > This is required to push the outdated, but rebuilt images from GitHub actions > for reuse. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7116) [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions
Krisztian Szucs created ARROW-7116: -- Summary: [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions Key: ARROW-7116 URL: https://issues.apache.org/jira/browse/ARROW-7116 Project: Apache Arrow Issue Type: Task Components: Continuous Integration Reporter: Krisztian Szucs This is required to push the outdated, but rebuilt images from GitHub actions for reuse. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7117) [C++][CI] Fix the hanging C++ tests in Windows 2019
Krisztian Szucs created ARROW-7117: -- Summary: [C++][CI] Fix the hanging C++ tests in Windows 2019 Key: ARROW-7117 URL: https://issues.apache.org/jira/browse/ARROW-7117 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration Reporter: Krisztian Szucs Fix For: 1.0.0 C++ tests hang on windows-latest GitHub actions host, so the tests are disabled. We should fix it and re-enable the tests https://github.com/apache/arrow/blob/master/.github/workflows/cpp.yml#L171 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7119) [C++][CI] Use scripts/util_coredump.sh to show automatic backtraces
Krisztian Szucs created ARROW-7119: -- Summary: [C++][CI] Use scripts/util_coredump.sh to show automatic backtraces Key: ARROW-7119 URL: https://issues.apache.org/jira/browse/ARROW-7119 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration Reporter: Krisztian Szucs Fix For: 1.0.0 The script was previously used on Travis, we should enable it in docker and on GitHub actions to speed up the debugging process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7120) [C++][CI] Add .ccache to the docker-compose volume mounts
Krisztian Szucs created ARROW-7120: -- Summary: [C++][CI] Add .ccache to the docker-compose volume mounts Key: ARROW-7120 URL: https://issues.apache.org/jira/browse/ARROW-7120 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration Reporter: Krisztian Szucs Fix For: 1.0.0 This should speed up the compilation steps. Although I'm not sure whether we should define it as a "docker named volume" or a standard volume mounted from the host. The former one provides better isolation whereas the latter one makes the integration with GitHub actions and it's cache plugin easier. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7121) [C++][CI][Windows] Enable more features on the windows GHA build
Krisztian Szucs created ARROW-7121: -- Summary: [C++][CI][Windows] Enable more features on the windows GHA build Key: ARROW-7121 URL: https://issues.apache.org/jira/browse/ARROW-7121 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration Reporter: Krisztian Szucs Fix For: 1.0.0 Like `ARROW_GANDIVA: ON`, `ARROW_FLIGHT: ON`, `ARROW_PARQUET: ON` -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7122) [CI][Documenation] docker-compose developer guide in the sphinx documentation
Krisztian Szucs created ARROW-7122: -- Summary: [CI][Documenation] docker-compose developer guide in the sphinx documentation Key: ARROW-7122 URL: https://issues.apache.org/jira/browse/ARROW-7122 Project: Apache Arrow Issue Type: Improvement Components: Documentation Reporter: Krisztian Szucs Assignee: Krisztian Szucs Fix For: 1.0.0 We have a short guide in the sphinx documentation under integration.rst It needs to be updated with the recent docker-compose changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7123) [C#][CI] The docker container fails to run on Github Actions because of the ubuntu host version
Krisztian Szucs created ARROW-7123: -- Summary: [C#][CI] The docker container fails to run on Github Actions because of the ubuntu host version Key: ARROW-7123 URL: https://issues.apache.org/jira/browse/ARROW-7123 Project: Apache Arrow Issue Type: Improvement Components: C#, Continuous Integration Reporter: Krisztian Szucs So the current linux build runs directly on the host. See issue [dotnet/core#3509|https://github.com/dotnet/core/issues/3509] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7118) [CI] [Dev] "docker-compose run ubuntu-python" fails
[
https://issues.apache.org/jira/browse/ARROW-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972311#comment-16972311
]
Krisztian Szucs commented on ARROW-7118:
[~apitrou] it should work after evicting the cache. One of the image is using
the mounted named volume for the C++ artefacts, but that image has python 2.7
instead of 3.5
> [CI] [Dev] "docker-compose run ubuntu-python" fails
> ---
>
> Key: ARROW-7118
> URL: https://issues.apache.org/jira/browse/ARROW-7118
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Developer Tools
>Reporter: Antoine Pitrou
>Priority: Major
>
> It's taking up the wrong Python version:
> {code}
> -- Searching for Python libs in
> /usr/lib64;/usr/lib;/usr/lib/python2.7/config-x86_64-linux-gnu
> -- Looking for python2.7
> -- Found Python lib PYTHON_LIBRARY-NOTFOUND
> -- Found PythonLibs: PYTHON_LIBRARY-NOTFOUND
> CMake Error at cmake_modules/FindNumPy.cmake:62 (message):
> NumPy import failure:
> Traceback (most recent call last):
> File "", line 1, in
> ImportError: No module named numpy
> Call Stack (most recent call first):
> src/arrow/python/CMakeLists.txt:23 (find_package)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (ARROW-7118) [CI] [Dev] "docker-compose run ubuntu-python" fails
[
https://issues.apache.org/jira/browse/ARROW-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972311#comment-16972311
]
Krisztian Szucs edited comment on ARROW-7118 at 11/12/19 12:00 PM:
---
[~apitrou] it should work after evicting the cache (docker volume rm
arrow_amd64-ubuntu-18.04-cache). One of the image is using the mounted named
volume for the C++ artefacts, but that image has python 2.7 instead of 3.5
was (Author: kszucs):
[~apitrou] it should work after evicting the cache. One of the image is using
the mounted named volume for the C++ artefacts, but that image has python 2.7
instead of 3.5
> [CI] [Dev] "docker-compose run ubuntu-python" fails
> ---
>
> Key: ARROW-7118
> URL: https://issues.apache.org/jira/browse/ARROW-7118
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Developer Tools
>Reporter: Antoine Pitrou
>Priority: Major
>
> It's taking up the wrong Python version:
> {code}
> -- Searching for Python libs in
> /usr/lib64;/usr/lib;/usr/lib/python2.7/config-x86_64-linux-gnu
> -- Looking for python2.7
> -- Found Python lib PYTHON_LIBRARY-NOTFOUND
> -- Found PythonLibs: PYTHON_LIBRARY-NOTFOUND
> CMake Error at cmake_modules/FindNumPy.cmake:62 (message):
> NumPy import failure:
> Traceback (most recent call last):
> File "", line 1, in
> ImportError: No module named numpy
> Call Stack (most recent call first):
> src/arrow/python/CMakeLists.txt:23 (find_package)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7124) [Python][CI] Create a docker image for python ASV benchmarks
Krisztian Szucs created ARROW-7124: -- Summary: [Python][CI] Create a docker image for python ASV benchmarks Key: ARROW-7124 URL: https://issues.apache.org/jira/browse/ARROW-7124 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Python Reporter: Krisztian Szucs And fix the following script https://github.com/apache/arrow/blob/7bc2b0f3579b6fc2d1d6d9797f3a1e5e3726131f/ci/scripts/python_benchmark.sh -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7125) [CI] Find a short but related prefix for the env vars used for the docker-compose file
Krisztian Szucs created ARROW-7125: -- Summary: [CI] Find a short but related prefix for the env vars used for the docker-compose file Key: ARROW-7125 URL: https://issues.apache.org/jira/browse/ARROW-7125 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Krisztian Szucs To prevent collisions with the host's environment. The defaults are defined in the [.env|https://github.com/apache/arrow/blob/master/.env] file. These environment variables are used across the whole [docker-compose|https://github.com/apache/arrow/blob/master/docker-compose.yml] file, so every occurrences should be updates. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7126) [JS][CI][Archery] Move nodejs linting from js_build.sh to archery
Krisztian Szucs created ARROW-7126: -- Summary: [JS][CI][Archery] Move nodejs linting from js_build.sh to archery Key: ARROW-7126 URL: https://issues.apache.org/jira/browse/ARROW-7126 Project: Apache Arrow Issue Type: Improvement Components: Archery, Continuous Integration, JavaScript Reporter: Krisztian Szucs Currently all the linters are run by archery, except just a couple ones, like the [js linter|https://github.com/apache/arrow/blob/master/ci/scripts/js_build.sh#L29]. We should add support for it into archery and run in the lint build. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7127) [CI][Spark] Support specific Spark version in the integration tet including latest
Krisztian Szucs created ARROW-7127: -- Summary: [CI][Spark] Support specific Spark version in the integration tet including latest Key: ARROW-7127 URL: https://issues.apache.org/jira/browse/ARROW-7127 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Krisztian Szucs The current spark integration test only supports spark's master branch. I'd nice to have a way to test the latest and specific releases as well, similarly like the other integration tests [do|https://github.com/apache/arrow/blob/master/ci/docker/conda-python-pandas.dockerfile#L25] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7128) [CI] Fedora cron jobs are failing because of wrong fedora version
Krisztian Szucs created ARROW-7128: -- Summary: [CI] Fedora cron jobs are failing because of wrong fedora version Key: ARROW-7128 URL: https://issues.apache.org/jira/browse/ARROW-7128 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Krisztian Szucs Assignee: Krisztian Szucs The requested fedora version is 10 (Debian) instead of 29: https://github.com/apache/arrow/runs/299223601 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-7128) [CI] Fedora cron jobs are failing because of wrong fedora version
[ https://issues.apache.org/jira/browse/ARROW-7128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-7128: --- Issue Type: Bug (was: Improvement) > [CI] Fedora cron jobs are failing because of wrong fedora version > - > > Key: ARROW-7128 > URL: https://issues.apache.org/jira/browse/ARROW-7128 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > > The requested fedora version is 10 (Debian) instead of 29: > https://github.com/apache/arrow/runs/299223601 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7129) [C++][CI] Hiveserver2 instegration test fails to connect to impala container
Krisztian Szucs created ARROW-7129: -- Summary: [C++][CI] Hiveserver2 instegration test fails to connect to impala container Key: ARROW-7129 URL: https://issues.apache.org/jira/browse/ARROW-7129 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration Reporter: Krisztian Szucs Reproduce it with https://github.com/apache/arrow/blob/master/docker-compose.yml#L882-L885 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7130) [C++][CMake] Automatically set ARROW_GANDIVA_PC_CXX_FLAGS for conda and OSX sdk
Krisztian Szucs created ARROW-7130: -- Summary: [C++][CMake] Automatically set ARROW_GANDIVA_PC_CXX_FLAGS for conda and OSX sdk Key: ARROW-7130 URL: https://issues.apache.org/jira/browse/ARROW-7130 Project: Apache Arrow Issue Type: Improvement Components: C++ - Gandiva Reporter: Krisztian Szucs Fix For: 1.0.0 ARROW_GANDIVA_PC_CXX_FLAGS requires special treatment based on the platforms, see: - https://github.com/apache/arrow/blob/master/ci/scripts/cpp_build.sh#L27-L32 - https://github.com/conda-forge/arrow-cpp-feedstock/blob/master/recipe/build.sh#L12-L15 We should integrate this logic into CMake by default. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7131) [GLIB][CI] Fail to execute lua examples in the MacOS build
Krisztian Szucs created ARROW-7131: -- Summary: [GLIB][CI] Fail to execute lua examples in the MacOS build Key: ARROW-7131 URL: https://issues.apache.org/jira/browse/ARROW-7131 Project: Apache Arrow Issue Type: Improvement Components: CI, GLib Reporter: Krisztian Szucs Fails to load 'lgi.corelgilua51' despite that lgi is installed in the macOS build. References: - https://github.com/apache/arrow/blob/master/.github/workflows/ruby.yml#L77 - https://github.com/apache/arrow/blob/master/ci/scripts/c_glib_test.sh#L35 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-7128) [CI] Fedora cron jobs are failing because of wrong fedora version
[ https://issues.apache.org/jira/browse/ARROW-7128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs resolved ARROW-7128. Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 5808 [https://github.com/apache/arrow/pull/5808] > [CI] Fedora cron jobs are failing because of wrong fedora version > - > > Key: ARROW-7128 > URL: https://issues.apache.org/jira/browse/ARROW-7128 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The requested fedora version is 10 (Debian) instead of 29: > https://github.com/apache/arrow/runs/299223601 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7132) [C#][CI] Pre-install the C# dependencies in its dockerfile if possible
Krisztian Szucs created ARROW-7132: -- Summary: [C#][CI] Pre-install the C# dependencies in its dockerfile if possible Key: ARROW-7132 URL: https://issues.apache.org/jira/browse/ARROW-7132 Project: Apache Arrow Issue Type: Improvement Components: C#, Continuous Integration Reporter: Krisztian Szucs Similarly like we install the dependencies for C++, Python or Go. The Dockerfiles is here: https://github.com/apache/arrow/blob/master/ci/docker/ubuntu-18.04-csharp.dockerfile -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7134) [Ruby][CI] Pre-install the ruby dependencies in the dockerfile and remove it from the test script
Krisztian Szucs created ARROW-7134: -- Summary: [Ruby][CI] Pre-install the ruby dependencies in the dockerfile and remove it from the test script Key: ARROW-7134 URL: https://issues.apache.org/jira/browse/ARROW-7134 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Ruby Reporter: Krisztian Szucs See the comment https://github.com/apache/arrow/blob/master/ci/docker/linux-apt-ruby.dockerfile#L22-L24 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7133) [CI] Allow GH Actions to run on all branches
[ https://issues.apache.org/jira/browse/ARROW-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972382#comment-16972382 ] Krisztian Szucs commented on ARROW-7133: Do we want to run the same builds as for PRs or the same builds as on the master branch? > [CI] Allow GH Actions to run on all branches > > > Key: ARROW-7133 > URL: https://issues.apache.org/jira/browse/ARROW-7133 > Project: Apache Arrow > Issue Type: Bug >Reporter: Antoine Pitrou >Assignee: Antoine Pitrou >Priority: Major > > When working from feature branches, it is good to have CI results as well > (especially from a fork, before submitting a PR). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7135) [Java][CI] Pre-install the java dependencies in the dockerfile
Krisztian Szucs created ARROW-7135: -- Summary: [Java][CI] Pre-install the java dependencies in the dockerfile Key: ARROW-7135 URL: https://issues.apache.org/jira/browse/ARROW-7135 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Java Reporter: Krisztian Szucs See the comments: https://github.com/apache/arrow/blob/master/ci/docker/debian-9-java.dockerfile#L25-L28 Or perhaps just use the mounted maven m2 repository. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7116) [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions
[ https://issues.apache.org/jira/browse/ARROW-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972389#comment-16972389 ] Krisztian Szucs commented on ARROW-7116: INFRA ticket https://issues.apache.org/jira/browse/INFRA-19429 > [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions > --- > > Key: ARROW-7116 > URL: https://issues.apache.org/jira/browse/ARROW-7116 > Project: Apache Arrow > Issue Type: Task > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > > This is required to push the outdated, but rebuilt images from GitHub actions > for reuse. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7136) [Rust][CI] Pre-install the rust dependencies in the dockerfile
Krisztian Szucs created ARROW-7136: -- Summary: [Rust][CI] Pre-install the rust dependencies in the dockerfile Key: ARROW-7136 URL: https://issues.apache.org/jira/browse/ARROW-7136 Project: Apache Arrow Issue Type: Improvement Components: CI, Rust Reporter: Krisztian Szucs I'd be nice to pre-install the rust dependencies similarly like we does with go, see the comment https://github.com/apache/arrow/blob/master/ci/docker/debian-10-rust.dockerfile#L41-L44 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7137) [JS][CI] Pre-install the JS dependencies in the dockerfile
Krisztian Szucs created ARROW-7137: -- Summary: [JS][CI] Pre-install the JS dependencies in the dockerfile Key: ARROW-7137 URL: https://issues.apache.org/jira/browse/ARROW-7137 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, JavaScript Reporter: Krisztian Szucs If it is possible with npm, see comment https://github.com/apache/arrow/blob/master/ci/docker/debian-10-js.dockerfile#L24-L27 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7138) [Go][CI] Pre-install the go dependencies in the dockerfile using go get
Krisztian Szucs created ARROW-7138: -- Summary: [Go][CI] Pre-install the go dependencies in the dockerfile using go get Key: ARROW-7138 URL: https://issues.apache.org/jira/browse/ARROW-7138 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Go Reporter: Krisztian Szucs See comments https://github.com/apache/arrow/blob/master/ci/docker/debian-10-go.dockerfile#L22-L25 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7139) [CI] Consider re-adding the removed debian testing image
Krisztian Szucs created ARROW-7139: -- Summary: [CI] Consider re-adding the removed debian testing image Key: ARROW-7139 URL: https://issues.apache.org/jira/browse/ARROW-7139 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Krisztian Szucs I've removed the Debian-testing docker images, was not sure where did we run it. cc [~kou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7140) [CI] Turn off unnecessary features in the integration tests (spark/turbodbc/dask/hdfs)
Krisztian Szucs created ARROW-7140: -- Summary: [CI] Turn off unnecessary features in the integration tests (spark/turbodbc/dask/hdfs) Key: ARROW-7140 URL: https://issues.apache.org/jira/browse/ARROW-7140 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Krisztian Szucs We should reduce the built features to the minimal set required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7141) [C++][CI] Test the ported fuzzit integration image
Krisztian Szucs created ARROW-7141: -- Summary: [C++][CI] Test the ported fuzzit integration image Key: ARROW-7141 URL: https://issues.apache.org/jira/browse/ARROW-7141 Project: Apache Arrow Issue Type: Task Components: C++, Continuous Integration Reporter: Krisztian Szucs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7142) [C++] Compile error with GCC 5.4.0
Krisztian Szucs created ARROW-7142: -- Summary: [C++] Compile error with GCC 5.4.0 Key: ARROW-7142 URL: https://issues.apache.org/jira/browse/ARROW-7142 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Krisztian Szucs If I recall correctly it was template related. Reproduce it with: export UBUNTU=16.04 docker-compose build ubuntu-cpp docker-compose run --rm ubuntu-cpp cc [~fsaintjacques] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7143) [Archery] Create a wrapper script in archery for docker compose in order to run the containers with the host's user and group
Krisztian Szucs created ARROW-7143: -- Summary: [Archery] Create a wrapper script in archery for docker compose in order to run the containers with the host's user and group Key: ARROW-7143 URL: https://issues.apache.org/jira/browse/ARROW-7143 Project: Apache Arrow Issue Type: Bug Components: Archery, Continuous Integration Reporter: Krisztian Szucs Docker-compose has its limitations, one annoying one is that it writes the mounted directory with root user. Sadly there are builds which are contaminating the source directory, so we should have a wrapper script to handle the permissions properly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7144) [Archery] Consider to use archery with or instead of the pre-commit hooks
Krisztian Szucs created ARROW-7144: -- Summary: [Archery] Consider to use archery with or instead of the pre-commit hooks Key: ARROW-7144 URL: https://issues.apache.org/jira/browse/ARROW-7144 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Developer Tools Reporter: Krisztian Szucs Archery lint has most of the configured functionality in the pre-commit hooks except to check only the changed and relevant paths. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-7143) [Archery] Create a wrapper script in archery for docker compose in order to run the containers with the host's user and group
[ https://issues.apache.org/jira/browse/ARROW-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Szucs updated ARROW-7143: --- Issue Type: Improvement (was: Bug) > [Archery] Create a wrapper script in archery for docker compose in order to > run the containers with the host's user and group > - > > Key: ARROW-7143 > URL: https://issues.apache.org/jira/browse/ARROW-7143 > Project: Apache Arrow > Issue Type: Improvement > Components: Archery, Continuous Integration >Reporter: Krisztian Szucs >Priority: Major > > Docker-compose has its limitations, one annoying one is that it writes the > mounted directory with root user. Sadly there are builds which are > contaminating the source directory, so we should have a wrapper script to > handle the permissions properly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7145) [Archery] Consider porting the docker tool from ursabot to archery
Krisztian Szucs created ARROW-7145: -- Summary: [Archery] Consider porting the docker tool from ursabot to archery Key: ARROW-7145 URL: https://issues.apache.org/jira/browse/ARROW-7145 Project: Apache Arrow Issue Type: Improvement Components: Developer Tools Reporter: Krisztian Szucs Original implementation is here https://github.com/ursa-labs/ursabot/blob/master/ursabot/docker.py -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7118) [CI] [Dev] "docker-compose run ubuntu-python" fails
[
https://issues.apache.org/jira/browse/ARROW-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972416#comment-16972416
]
Krisztian Szucs commented on ARROW-7118:
We might need to cache less aggressively, mounting .ccache only and not the
build directories.
> [CI] [Dev] "docker-compose run ubuntu-python" fails
> ---
>
> Key: ARROW-7118
> URL: https://issues.apache.org/jira/browse/ARROW-7118
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Developer Tools
>Reporter: Antoine Pitrou
>Priority: Major
>
> It's taking up the wrong Python version:
> {code}
> -- Searching for Python libs in
> /usr/lib64;/usr/lib;/usr/lib/python2.7/config-x86_64-linux-gnu
> -- Looking for python2.7
> -- Found Python lib PYTHON_LIBRARY-NOTFOUND
> -- Found PythonLibs: PYTHON_LIBRARY-NOTFOUND
> CMake Error at cmake_modules/FindNumPy.cmake:62 (message):
> NumPy import failure:
> Traceback (most recent call last):
> File "", line 1, in
> ImportError: No module named numpy
> Call Stack (most recent call first):
> src/arrow/python/CMakeLists.txt:23 (find_package)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (ARROW-7118) [CI] [Dev] "docker-compose run ubuntu-python" fails
[
https://issues.apache.org/jira/browse/ARROW-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972416#comment-16972416
]
Krisztian Szucs edited comment on ARROW-7118 at 11/12/19 1:57 PM:
--
We might need to cache less aggressively, mounting .ccache only and not the
build directories. It matter only locally, but it should work locally too,
between multiple docker-compose runs.
was (Author: kszucs):
We might need to cache less aggressively, mounting .ccache only and not the
build directories. It matter only locally, but locally it should work.
> [CI] [Dev] "docker-compose run ubuntu-python" fails
> ---
>
> Key: ARROW-7118
> URL: https://issues.apache.org/jira/browse/ARROW-7118
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Developer Tools
>Reporter: Antoine Pitrou
>Priority: Major
>
> It's taking up the wrong Python version:
> {code}
> -- Searching for Python libs in
> /usr/lib64;/usr/lib;/usr/lib/python2.7/config-x86_64-linux-gnu
> -- Looking for python2.7
> -- Found Python lib PYTHON_LIBRARY-NOTFOUND
> -- Found PythonLibs: PYTHON_LIBRARY-NOTFOUND
> CMake Error at cmake_modules/FindNumPy.cmake:62 (message):
> NumPy import failure:
> Traceback (most recent call last):
> File "", line 1, in
> ImportError: No module named numpy
> Call Stack (most recent call first):
> src/arrow/python/CMakeLists.txt:23 (find_package)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (ARROW-7118) [CI] [Dev] "docker-compose run ubuntu-python" fails
[
https://issues.apache.org/jira/browse/ARROW-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972416#comment-16972416
]
Krisztian Szucs edited comment on ARROW-7118 at 11/12/19 1:57 PM:
--
We might need to cache less aggressively, mounting .ccache only and not the
build directories. It matter only locally, but locally it should work.
was (Author: kszucs):
We might need to cache less aggressively, mounting .ccache only and not the
build directories.
> [CI] [Dev] "docker-compose run ubuntu-python" fails
> ---
>
> Key: ARROW-7118
> URL: https://issues.apache.org/jira/browse/ARROW-7118
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Developer Tools
>Reporter: Antoine Pitrou
>Priority: Major
>
> It's taking up the wrong Python version:
> {code}
> -- Searching for Python libs in
> /usr/lib64;/usr/lib;/usr/lib/python2.7/config-x86_64-linux-gnu
> -- Looking for python2.7
> -- Found Python lib PYTHON_LIBRARY-NOTFOUND
> -- Found PythonLibs: PYTHON_LIBRARY-NOTFOUND
> CMake Error at cmake_modules/FindNumPy.cmake:62 (message):
> NumPy import failure:
> Traceback (most recent call last):
> File "", line 1, in
> ImportError: No module named numpy
> Call Stack (most recent call first):
> src/arrow/python/CMakeLists.txt:23 (find_package)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7146) [R][CI] Pre-install the R dependencies in the dockerfile
Krisztian Szucs created ARROW-7146: -- Summary: [R][CI] Pre-install the R dependencies in the dockerfile Key: ARROW-7146 URL: https://issues.apache.org/jira/browse/ARROW-7146 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Krisztian Szucs Here: https://github.com/apache/arrow/blob/master/ci/docker/linux-apt-r.dockerfile#L61 Using the https://github.com/apache/arrow/blob/master/ci/scripts/r_deps.sh script -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7116) [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions
[ https://issues.apache.org/jira/browse/ARROW-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973377#comment-16973377 ] Krisztian Szucs commented on ARROW-7116: [~kou] I've tried the GitHub package registry a couple weeks ago, but it was failing with errors indicating misconfigured docker registry. On long term I'd favour to switch, but simply put it wasn't working last time I've checked. > [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions > --- > > Key: ARROW-7116 > URL: https://issues.apache.org/jira/browse/ARROW-7116 > Project: Apache Arrow > Issue Type: Task > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > > This is required to push the outdated, but rebuilt images from GitHub actions > for reuse. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ARROW-7116) [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions
[ https://issues.apache.org/jira/browse/ARROW-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973377#comment-16973377 ] Krisztian Szucs edited comment on ARROW-7116 at 11/13/19 2:16 PM: -- [~kou] I've tried the GitHub package registry a couple weeks ago, but it was failing with errors indicating misconfigured docker registry. On long term I'd favour to switch, but simply put it wasn't working last time I checked. was (Author: kszucs): [~kou] I've tried the GitHub package registry a couple weeks ago, but it was failing with errors indicating misconfigured docker registry. On long term I'd favour to switch, but simply put it wasn't working last time I've checked. > [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions > --- > > Key: ARROW-7116 > URL: https://issues.apache.org/jira/browse/ARROW-7116 > Project: Apache Arrow > Issue Type: Task > Components: Continuous Integration >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > > This is required to push the outdated, but rebuilt images from GitHub actions > for reuse. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-6997) [Packaging] Add support for RHEL
[
https://issues.apache.org/jira/browse/ARROW-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Szucs resolved ARROW-6997.
Fix Version/s: 1.0.0
Resolution: Fixed
Issue resolved by pull request 5823
[https://github.com/apache/arrow/pull/5823]
> [Packaging] Add support for RHEL
>
>
> Key: ARROW-6997
> URL: https://issues.apache.org/jira/browse/ARROW-6997
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Packaging
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We need symbolic links to {{${VERSION}Server}} from {{${VERSION}}} such as
> {{7Server}} from {{7}}. (Is it available on BinTray?)
> We also need to update install information. We can't install {{epel-release}}
> by {{yum install -y epel-release}}. We need to specify URL explicitly: {{yum
> install
> https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm}}. See
> https://fedoraproject.org/wiki/EPEL for details.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7180) [CI] Java builds are not triggered on the master branch
Krisztian Szucs created ARROW-7180: -- Summary: [CI] Java builds are not triggered on the master branch Key: ARROW-7180 URL: https://issues.apache.org/jira/browse/ARROW-7180 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Java Reporter: Krisztian Szucs -- This message was sent by Atlassian Jira (v8.3.4#803005)
