[jira] [Created] (ARROW-7804) [C++][R] Compile error on macOS 10.11
Neal Richardson created ARROW-7804:
--
Summary: [C++][R] Compile error on macOS 10.11
Key: ARROW-7804
URL: https://issues.apache.org/jira/browse/ARROW-7804
Project: Apache Arrow
Issue Type: Improvement
Components: C++, Continuous Integration, R
Reporter: Neal Richardson
Fix For: 1.0.0
So, yeah, macOS 10.11 is from 2015, but that's what CRAN (R) builds on.
[~jeroenooms] found this error when building binaries for 0.16:
{code}
==> cmake ../cpp -DCMAKE_C_FLAGS_RELEASE=-DNDEBUG
-DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG
-DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/apache-arrow/
==> make
Last 15 lines from /Users/builder/Library/Logs/Homebrew/apache-arrow/02.make:
/tmp/apache-arrow-20200208-1254-r2avjm/apache-arrow-0.16.0/cpp/src/parquet/stream_writer.h:233:22:
error: default initialization of an object of const type 'const
parquet::EndRowType' without a user-provided default constructor
constexpr EndRowType EndRow;
^
{}
/tmp/apache-arrow-20200208-1254-r2avjm/apache-arrow-0.16.0/cpp/src/parquet/stream_writer.h:236:27:
error: default initialization of an object of const type 'const
parquet::EndRowGroupType' without a user-provided default constructor
constexpr EndRowGroupType EndRowGroup;
^
{}
2 errors generated.
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7803) [R][CI] Autobrew/homebrew tests always install from master
Neal Richardson created ARROW-7803:
--
Summary: [R][CI] Autobrew/homebrew tests always install from master
Key: ARROW-7803
URL: https://issues.apache.org/jira/browse/ARROW-7803
Project: Apache Arrow
Issue Type: Improvement
Components: Continuous Integration
Reporter: Neal Richardson
Assignee: Neal Richardson
Figure out how to get the formula to check out a branch when building {{--head}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] Release Apache Arrow 0.16.0 - RC2
Homebrew PR is up: https://github.com/Homebrew/homebrew-core/pull/49908 On Fri, Feb 7, 2020 at 3:44 PM Neal Richardson wrote: > I'm working (with Jeroen) on the R package stuff, and I'll put together > the homebrew formula PR now. > > Neal > > On Fri, Feb 7, 2020 at 6:42 AM Andy Grove wrote: > >> Rust crates are published. Filed >> https://issues.apache.org/jira/browse/ARROW-7794 for the arrow-flight >> issue. >> >> On Fri, Feb 7, 2020 at 7:05 AM Andy Grove wrote: >> >> > I'll look at the Rust issue now. >> > >> > On Fri, Feb 7, 2020 at 3:27 AM Krisztián Szűcs < >> szucs.kriszt...@gmail.com> >> > wrote: >> > >> >> Status of the post release tasks: >> >> >> >> - [DONE] marking the released version as "RELEASED" on JIRA >> >> - [DONE] rebase the master branch on top of the release branch >> >> We have failing builds on the master because of the bumped version >> >> numbers, those must be fixed. >> >> - [DONE] rebase the pull requests >> >> - [DONE] uploading source release artifacts to SVN >> >> - [DONE] uploading binary release artifacts to Bintray >> >> - [DONE] updating the Arrow website >> >> - [DONE] python source distribution >> >> - [DONE] ruby gems >> >> - [DONE] javascript npm packages >> >> - [DONE] .NET nuget packages >> >> - [DONE] java maven artifacts >> >>I had to re-stage the maven artifacts >> >> - [kszucs] python wheels >> >> Partially done, the manylinux2010 and manylinux2014 wheels have hit >> >> the 60MB limit of pypi, requested to increase it >> >> https://github.com/pypa/pypi-support/issues/186 >> >> - [xhochy] conda packages >> >> Uwe works on it, arrow-cpp is already merged. >> >> - [andygrove] rust packages >> >>Arrow-flight misses the format directory, didn't dig deeper. Andy >> >> could you take a look? >> >> - [nealrichardson] R packages >> >> Neal should handle this. >> >> - [ ] homebrew packages >> >> - [ ] updating website with new API documentation >> >> - [ ] announcing release >> >> >> >> On Fri, Feb 7, 2020 at 8:18 AM Krisztián Szűcs >> >> wrote: >> >> > >> >> > The VOTE carries with 4 binding +1 votes, 3 non-binding +1 votes and >> >> > one binding +0 vote. >> >> > >> >> > I'm starting the post-release tasks, if anyone wants to help please >> let >> >> me know. >> >> > >> >> > On Fri, Feb 7, 2020 at 12:25 AM Krisztián Szűcs >> >> > wrote: >> >> > > >> >> > > So far we have the following votes: >> >> > > >> >> > > +0 (binding) >> >> > > +1 (binding) >> >> > > +1 (non-binding) >> >> > > +1 (binding) >> >> > > +1 (non-binding) >> >> > > +1 (binding) >> >> > > +1 (non-binding) >> >> > > +1 (binding) >> >> > > >> >> > > 4 +1 (binding) >> >> > > 3 +1 (non-binding) >> >> > > >> >> > > I'm waiting for votes until tomorrow morning (UTC), then I'm >> closing >> >> the VOTE. >> >> > > >> >> > > Thanks everyone! >> >> > > >> >> > > - Krisztian >> >> > > >> >> > > On Fri, Feb 7, 2020 at 12:06 AM Krisztián Szűcs >> >> > > wrote: >> >> > > > >> >> > > > Testing on macOS Catalina >> >> > > > >> >> > > > Binaries: OK >> >> > > > >> >> > > > Wheels: OK >> >> > > > Verified on macOS and on Linux. >> >> > > > On linux the verification script has failed for python 3.5 and >> >> manylinux2010 >> >> > > > and manylinux2014 with unsupported platform tag. I've manually >> >> checked >> >> > > > these wheels in the python:3.5 docker image, and the wheels were >> >> good >> >> > > > (this is automatically checked by crossbow too [1]). All other >> >> wheels were >> >> > > > passing using the verification script. >> >> > > > >> >> > > > Source: OK >> >> > > > I had to revert the nvm path [2] to pass the js and integration >> >> tests and >> >> > > > force the glib test to use my system python instead of the conda >> >> one. >> >> > > > >> >> > > > I vote with +1 (binding) >> >> > > > >> >> > > > [1]: >> >> https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L568 >> >> > > > [2]: >> >> >> https://github.com/apache/arrow/commit/37434fb34a1f2cd5273092ed3e1c61db90bb4dd2 >> >> > > > >> >> > > > >> >> > > > On Thu, Feb 6, 2020 at 7:42 PM Neal Richardson >> >> > > > wrote: >> >> > > > > >> >> > > > > I re-verified the macOS wheels and they worked but I had to >> >> hard-code >> >> > > > > `MACOSX_DEPLOYMENT_TARGET="10.14"` to get past the cython >> error I >> >> reported >> >> > > > > previously. I tried to set that env var dynamically based on >> your >> >> current >> >> > > > > OS version but didn't succeed in getting it passed through to >> >> pytest, >> >> > > > > despite many attempts to `export` it; someone with better bash >> >> skills than >> >> > > > > I should probably add that to the script. FTR `defaults read >> >> loginwindow >> >> > > > > SystemVersionStampAsString | sed s/\.[0-9]$//` returns the >> right >> >> macOS >> >> > > > > version. >> >> > > > > >> >> > > > > Neal >> >> > > > > >> >> > > > > On Wed, Feb 5, 2020 at 1:29 PM Wes McKinney < >> wesmck...@gmail.com> >> >> wrote: >> >> > > > > >> >> > > > > > +1 (binding) >> >> > > > > > >>
[jira] [Created] (ARROW-7802) [C++] Support for LargeBinary and LargeString in the hash kernel
Zhuo Peng created ARROW-7802: Summary: [C++] Support for LargeBinary and LargeString in the hash kernel Key: ARROW-7802 URL: https://issues.apache.org/jira/browse/ARROW-7802 Project: Apache Arrow Issue Type: Bug Reporter: Zhuo Peng Currently they are not supported: https://github.com/apache/arrow/blob/a76e277213e166dbeb148260498995ba053566fb/cpp/src/arrow/compute/kernels/hash.cc#L456 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Release Apache Arrow 0.16.0 - RC2
I'm working (with Jeroen) on the R package stuff, and I'll put together the homebrew formula PR now. Neal On Fri, Feb 7, 2020 at 6:42 AM Andy Grove wrote: > Rust crates are published. Filed > https://issues.apache.org/jira/browse/ARROW-7794 for the arrow-flight > issue. > > On Fri, Feb 7, 2020 at 7:05 AM Andy Grove wrote: > > > I'll look at the Rust issue now. > > > > On Fri, Feb 7, 2020 at 3:27 AM Krisztián Szűcs < > szucs.kriszt...@gmail.com> > > wrote: > > > >> Status of the post release tasks: > >> > >> - [DONE] marking the released version as "RELEASED" on JIRA > >> - [DONE] rebase the master branch on top of the release branch > >> We have failing builds on the master because of the bumped version > >> numbers, those must be fixed. > >> - [DONE] rebase the pull requests > >> - [DONE] uploading source release artifacts to SVN > >> - [DONE] uploading binary release artifacts to Bintray > >> - [DONE] updating the Arrow website > >> - [DONE] python source distribution > >> - [DONE] ruby gems > >> - [DONE] javascript npm packages > >> - [DONE] .NET nuget packages > >> - [DONE] java maven artifacts > >>I had to re-stage the maven artifacts > >> - [kszucs] python wheels > >> Partially done, the manylinux2010 and manylinux2014 wheels have hit > >> the 60MB limit of pypi, requested to increase it > >> https://github.com/pypa/pypi-support/issues/186 > >> - [xhochy] conda packages > >> Uwe works on it, arrow-cpp is already merged. > >> - [andygrove] rust packages > >>Arrow-flight misses the format directory, didn't dig deeper. Andy > >> could you take a look? > >> - [nealrichardson] R packages > >> Neal should handle this. > >> - [ ] homebrew packages > >> - [ ] updating website with new API documentation > >> - [ ] announcing release > >> > >> On Fri, Feb 7, 2020 at 8:18 AM Krisztián Szűcs > >> wrote: > >> > > >> > The VOTE carries with 4 binding +1 votes, 3 non-binding +1 votes and > >> > one binding +0 vote. > >> > > >> > I'm starting the post-release tasks, if anyone wants to help please > let > >> me know. > >> > > >> > On Fri, Feb 7, 2020 at 12:25 AM Krisztián Szűcs > >> > wrote: > >> > > > >> > > So far we have the following votes: > >> > > > >> > > +0 (binding) > >> > > +1 (binding) > >> > > +1 (non-binding) > >> > > +1 (binding) > >> > > +1 (non-binding) > >> > > +1 (binding) > >> > > +1 (non-binding) > >> > > +1 (binding) > >> > > > >> > > 4 +1 (binding) > >> > > 3 +1 (non-binding) > >> > > > >> > > I'm waiting for votes until tomorrow morning (UTC), then I'm closing > >> the VOTE. > >> > > > >> > > Thanks everyone! > >> > > > >> > > - Krisztian > >> > > > >> > > On Fri, Feb 7, 2020 at 12:06 AM Krisztián Szűcs > >> > > wrote: > >> > > > > >> > > > Testing on macOS Catalina > >> > > > > >> > > > Binaries: OK > >> > > > > >> > > > Wheels: OK > >> > > > Verified on macOS and on Linux. > >> > > > On linux the verification script has failed for python 3.5 and > >> manylinux2010 > >> > > > and manylinux2014 with unsupported platform tag. I've manually > >> checked > >> > > > these wheels in the python:3.5 docker image, and the wheels were > >> good > >> > > > (this is automatically checked by crossbow too [1]). All other > >> wheels were > >> > > > passing using the verification script. > >> > > > > >> > > > Source: OK > >> > > > I had to revert the nvm path [2] to pass the js and integration > >> tests and > >> > > > force the glib test to use my system python instead of the conda > >> one. > >> > > > > >> > > > I vote with +1 (binding) > >> > > > > >> > > > [1]: > >> https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L568 > >> > > > [2]: > >> > https://github.com/apache/arrow/commit/37434fb34a1f2cd5273092ed3e1c61db90bb4dd2 > >> > > > > >> > > > > >> > > > On Thu, Feb 6, 2020 at 7:42 PM Neal Richardson > >> > > > wrote: > >> > > > > > >> > > > > I re-verified the macOS wheels and they worked but I had to > >> hard-code > >> > > > > `MACOSX_DEPLOYMENT_TARGET="10.14"` to get past the cython error > I > >> reported > >> > > > > previously. I tried to set that env var dynamically based on > your > >> current > >> > > > > OS version but didn't succeed in getting it passed through to > >> pytest, > >> > > > > despite many attempts to `export` it; someone with better bash > >> skills than > >> > > > > I should probably add that to the script. FTR `defaults read > >> loginwindow > >> > > > > SystemVersionStampAsString | sed s/\.[0-9]$//` returns the right > >> macOS > >> > > > > version. > >> > > > > > >> > > > > Neal > >> > > > > > >> > > > > On Wed, Feb 5, 2020 at 1:29 PM Wes McKinney < > wesmck...@gmail.com> > >> wrote: > >> > > > > > >> > > > > > +1 (binding) > >> > > > > > > >> > > > > > I was able to verify the Windows wheels with the following > >> patch applied > >> > > > > > > >> > > > > > https://github.com/apache/arrow/pull/6364 > >> > > > > > > >> > > > > > On Wed, Feb 5, 2020 at 1:09 PM Krisztián Szűcs > >> > > > > > wrote: > >> > > > > > > > >> > >
[jira] [Created] (ARROW-7801) [R][CI] Add lint and doc GitHub Action workflows
Neal Richardson created ARROW-7801: -- Summary: [R][CI] Add lint and doc GitHub Action workflows Key: ARROW-7801 URL: https://issues.apache.org/jira/browse/ARROW-7801 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Neal Richardson Assignee: Neal Richardson Fix For: 1.0.0 Like https://github.com/r-lib/actions/tree/master/examples#render-readme. * If changes to r/README.Rmd, render readme * If changes to r/R, render docs * If changes to r/src, lint.sh --fix -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7800) [Python] Expose GetRecordBatchReader API in PyArrow
Rollo Konig-brock created ARROW-7800:
Summary: [Python] Expose GetRecordBatchReader API in PyArrow
Key: ARROW-7800
URL: https://issues.apache.org/jira/browse/ARROW-7800
Project: Apache Arrow
Issue Type: Bug
Components: Python
Reporter: Rollo Konig-brock
Fix For: 1.0.0
The GetRecordBatchReader API is really useful for streaming ParquetFiles with
lots of RLE.
I propose exposing this API in PyArrow in the following manner:
{code}
file_ = ParquetFile('file/path.parquet', batch_size=100)
for batch in file_.get_batches():
pass
{code}
(If anyone has any better ideas hit me up, I'm not 100% sold on exposing it
this way.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7799) [R][CI] Remove flatbuffers from homebrew formulae
Neal Richardson created ARROW-7799: -- Summary: [R][CI] Remove flatbuffers from homebrew formulae Key: ARROW-7799 URL: https://issues.apache.org/jira/browse/ARROW-7799 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Packaging, R Reporter: Neal Richardson Assignee: Neal Richardson Fix For: 1.0.0 ARROW-6634 vendored flatbuffers so these shouldn't be needed anymore. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7798) [R] Refactor vector to Array conversion
Francois Saint-Jacques created ARROW-7798: - Summary: [R] Refactor vector to Array conversion Key: ARROW-7798 URL: https://issues.apache.org/jira/browse/ARROW-7798 Project: Apache Arrow Issue Type: Improvement Components: R Reporter: Francois Saint-Jacques There's a bit of technical debt accumulated in this file: * Mix of conversion *and* casting, ideally we'd move casting out of there (at the cost of more memory copy). The rationale is that the conversion logic will differ from the CastKernels, e.g. when to raise errors, benefits from complex conversions like timezone... The current implementation is fast, e.g. it fuses the conversion and casting in a single loop at the cost of code clarity and divergence. * There should be 2 paths, zero-copy, non zero-copy. The non-zero copy should use the newly introduced VectorToArrayConverter which will work with complex nested types. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7797) [Release][Rust] Fix arrow-flight's version in datafusion crate
Neal Richardson created ARROW-7797: -- Summary: [Release][Rust] Fix arrow-flight's version in datafusion crate Key: ARROW-7797 URL: https://issues.apache.org/jira/browse/ARROW-7797 Project: Apache Arrow Issue Type: Bug Components: Developer Tools, Rust - DataFusion Reporter: Neal Richardson Assignee: Neal Richardson -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7796) arrow::write_* functions should return their inputs
Sam Albers created ARROW-7796:
-
Summary: arrow::write_* functions should return their inputs
Key: ARROW-7796
URL: https://issues.apache.org/jira/browse/ARROW-7796
Project: Apache Arrow
Issue Type: Improvement
Components: R
Affects Versions: 0.15.1
Environment: Windows 10, R 3.6.2
Reporter: Sam Albers
I am wondering if you'd consider a slight change to what is returned by the
write_* functions. In \{readr} the write functions return its input which is
very useful for saving intermediate objects within a pipeline. I'd be happy to
take this on and submit as a pull request.
{code:java}
library(arrow)
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp
library(readr)
#>
#> Attaching package: 'readr'
#> The following object is masked from 'package:arrow':
#>
#> read_table
iris_arrow <- write_parquet(iris, "iris.parquet")
iris_arrow
#> NULL
iris_readr <- write_csv(iris, "iris.csv")
head(iris_readr)
#> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1 5.1 3.5 1.4 0.2 setosa
#> 2 4.9 3.0 1.4 0.2 setosa
#> 3 4.7 3.2 1.3 0.2 setosa
#> 4 4.6 3.1 1.5 0.2 setosa
#> 5 5.0 3.6 1.4 0.2 setosa
#> 6 5.4 3.9 1.7 0.4 setosa
devtools::session_info()
#> - Session info
---
#> setting value
#> version R version 3.6.2 (2019-12-12)
#> os Windows 10 x64
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_Canada.1252
#> ctype English_Canada.1252
#> tz America/Los_Angeles
#> date 2020-02-07
#>
#> - Packages
---
#> package * version date lib source
#> arrow * 0.15.1.20200207 2020-02-07 [1] local
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.0)
#> backports 1.1.5 2019-10-02 [1] CRAN (R 3.6.1)
#> bit 1.1-15.1 2020-01-14 [1] CRAN (R 3.6.2)
#> bit64 0.9-7 2017-05-08 [1] CRAN (R 3.6.0)
#> callr 3.4.1 2020-01-24 [1] CRAN (R 3.6.2)
#> cli 2.0.1 2020-01-08 [1] CRAN (R 3.6.2)
#> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.0)
#> desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.0)
#> devtools 2.2.1 2019-09-24 [1] CRAN (R 3.6.2)
#> digest 0.6.23 2019-11-23 [1] CRAN (R 3.6.1)
#> ellipsis 0.3.0 2019-09-20 [1] CRAN (R 3.6.1)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.0)
#> fansi 0.4.1 2020-01-08 [1] CRAN (R 3.6.2)
#> fs 1.3.1 2019-05-06 [1] CRAN (R 3.6.0)
#> glue 1.3.1 2019-03-12 [1] CRAN (R 3.6.0)
#> highr 0.8 2019-03-20 [1] CRAN (R 3.6.0)
#> hms 0.5.3 2020-01-08 [1] CRAN (R 3.6.2)
#> htmltools 0.4.0 2019-10-04 [1] CRAN (R 3.6.1)
#> knitr 1.27 2020-01-16 [1] CRAN (R 3.6.2)
#> magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.0)
#> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.0)
#> pillar 1.4.3 2019-12-20 [1] CRAN (R 3.6.2)
#> pkgbuild 1.0.6 2019-10-09 [1] CRAN (R 3.6.1)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 3.6.1)
#> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.0)
#> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 3.6.2)
#> processx 3.4.1 2019-07-18 [1] CRAN (R 3.6.1)
#> ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.0)
#> purrr 0.3.3 2019-10-18 [1] CRAN (R 3.6.1)
#> R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.1)
#> Rcpp 1.0.3 2019-11-08 [1] CRAN (R 3.6.1)
#> readr * 1.3.1 2018-12-21 [1] CRAN (R 3.6.1)
#> remotes 2.1.0 2019-06-24 [1] CRAN (R 3.6.1)
#> rlang 0.4.3 2020-01-24 [1] CRAN (R 3.6.2)
#> rmarkdown 2.1 2020-01-20 [1] CRAN (R 3.6.2)
#> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.0)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.0)
#> stringi 1.4.4 2020-01-09 [1] CRAN (R 3.6.2)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.2)
#> testthat 2.3.1 2019-12-01 [1] CRAN (R 3.6.1)
#> tibble 2.1.3 2019-06-06 [1] CRAN (R 3.6.2)
#> tidyselect 0.2.5 2018-10-11 [1] CRAN (R 3.6.2)
#> usethis 1.5.1.9000 2020-01-31 [1] Github (r-lib/usethis@c31336d)
#> vctrs 0.2.2 2020-01-24 [1] CRAN (R 3.6.2)
#> withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.0)
#> xfun 0.12 2020-01-13 [1] CRAN (R 3.6.2)
#> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.6.2)
#>
#> [1] C:/Users/salbers/R/win-library/3.6
#> [2] C:/Program Files/R/R-3.6.2/library
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7795) [Rust - DataFusion] Support boolean negation (NOT)
Jorge created ARROW-7795: Summary: [Rust - DataFusion] Support boolean negation (NOT) Key: ARROW-7795 URL: https://issues.apache.org/jira/browse/ARROW-7795 Project: Apache Arrow Issue Type: Improvement Components: Rust - DataFusion Reporter: Jorge This is a proposal to support the negation operation NOT. The user should be able to write something like ```SELECT a, b FROM t WHERE NOT a``` -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Release Apache Arrow 0.16.0 - RC2
Rust crates are published. Filed https://issues.apache.org/jira/browse/ARROW-7794 for the arrow-flight issue. On Fri, Feb 7, 2020 at 7:05 AM Andy Grove wrote: > I'll look at the Rust issue now. > > On Fri, Feb 7, 2020 at 3:27 AM Krisztián Szűcs > wrote: > >> Status of the post release tasks: >> >> - [DONE] marking the released version as "RELEASED" on JIRA >> - [DONE] rebase the master branch on top of the release branch >> We have failing builds on the master because of the bumped version >> numbers, those must be fixed. >> - [DONE] rebase the pull requests >> - [DONE] uploading source release artifacts to SVN >> - [DONE] uploading binary release artifacts to Bintray >> - [DONE] updating the Arrow website >> - [DONE] python source distribution >> - [DONE] ruby gems >> - [DONE] javascript npm packages >> - [DONE] .NET nuget packages >> - [DONE] java maven artifacts >>I had to re-stage the maven artifacts >> - [kszucs] python wheels >> Partially done, the manylinux2010 and manylinux2014 wheels have hit >> the 60MB limit of pypi, requested to increase it >> https://github.com/pypa/pypi-support/issues/186 >> - [xhochy] conda packages >> Uwe works on it, arrow-cpp is already merged. >> - [andygrove] rust packages >>Arrow-flight misses the format directory, didn't dig deeper. Andy >> could you take a look? >> - [nealrichardson] R packages >> Neal should handle this. >> - [ ] homebrew packages >> - [ ] updating website with new API documentation >> - [ ] announcing release >> >> On Fri, Feb 7, 2020 at 8:18 AM Krisztián Szűcs >> wrote: >> > >> > The VOTE carries with 4 binding +1 votes, 3 non-binding +1 votes and >> > one binding +0 vote. >> > >> > I'm starting the post-release tasks, if anyone wants to help please let >> me know. >> > >> > On Fri, Feb 7, 2020 at 12:25 AM Krisztián Szűcs >> > wrote: >> > > >> > > So far we have the following votes: >> > > >> > > +0 (binding) >> > > +1 (binding) >> > > +1 (non-binding) >> > > +1 (binding) >> > > +1 (non-binding) >> > > +1 (binding) >> > > +1 (non-binding) >> > > +1 (binding) >> > > >> > > 4 +1 (binding) >> > > 3 +1 (non-binding) >> > > >> > > I'm waiting for votes until tomorrow morning (UTC), then I'm closing >> the VOTE. >> > > >> > > Thanks everyone! >> > > >> > > - Krisztian >> > > >> > > On Fri, Feb 7, 2020 at 12:06 AM Krisztián Szűcs >> > > wrote: >> > > > >> > > > Testing on macOS Catalina >> > > > >> > > > Binaries: OK >> > > > >> > > > Wheels: OK >> > > > Verified on macOS and on Linux. >> > > > On linux the verification script has failed for python 3.5 and >> manylinux2010 >> > > > and manylinux2014 with unsupported platform tag. I've manually >> checked >> > > > these wheels in the python:3.5 docker image, and the wheels were >> good >> > > > (this is automatically checked by crossbow too [1]). All other >> wheels were >> > > > passing using the verification script. >> > > > >> > > > Source: OK >> > > > I had to revert the nvm path [2] to pass the js and integration >> tests and >> > > > force the glib test to use my system python instead of the conda >> one. >> > > > >> > > > I vote with +1 (binding) >> > > > >> > > > [1]: >> https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L568 >> > > > [2]: >> https://github.com/apache/arrow/commit/37434fb34a1f2cd5273092ed3e1c61db90bb4dd2 >> > > > >> > > > >> > > > On Thu, Feb 6, 2020 at 7:42 PM Neal Richardson >> > > > wrote: >> > > > > >> > > > > I re-verified the macOS wheels and they worked but I had to >> hard-code >> > > > > `MACOSX_DEPLOYMENT_TARGET="10.14"` to get past the cython error I >> reported >> > > > > previously. I tried to set that env var dynamically based on your >> current >> > > > > OS version but didn't succeed in getting it passed through to >> pytest, >> > > > > despite many attempts to `export` it; someone with better bash >> skills than >> > > > > I should probably add that to the script. FTR `defaults read >> loginwindow >> > > > > SystemVersionStampAsString | sed s/\.[0-9]$//` returns the right >> macOS >> > > > > version. >> > > > > >> > > > > Neal >> > > > > >> > > > > On Wed, Feb 5, 2020 at 1:29 PM Wes McKinney >> wrote: >> > > > > >> > > > > > +1 (binding) >> > > > > > >> > > > > > I was able to verify the Windows wheels with the following >> patch applied >> > > > > > >> > > > > > https://github.com/apache/arrow/pull/6364 >> > > > > > >> > > > > > On Wed, Feb 5, 2020 at 1:09 PM Krisztián Szűcs >> > > > > > wrote: >> > > > > > > >> > > > > > > There were binary naming issues with the macosx and the >> win-cp38 >> > > > > > > wheels. >> > > > > > > I've uploaded them, all of the wheels should be available now >> [1] >> > > > > > > >> > > > > > > Note that the newly built macosx wheels have 10_9 platform tag >> > > > > > > instead of 10_6, so the verification script must be updated >> [2] to >> > > > > > > verify the macosx wheels. >> > > > > > > >> > > > > > > [1] >> https://bintray.com/apache/arrow/python-rc/0.16.0-rc2#files >> >
[jira] [Created] (ARROW-7794) [Rust] cargo publish fails for arrow-flight due to relative path to Flight.proto
Andy Grove created ARROW-7794:
-
Summary: [Rust] cargo publish fails for arrow-flight due to
relative path to Flight.proto
Key: ARROW-7794
URL: https://issues.apache.org/jira/browse/ARROW-7794
Project: Apache Arrow
Issue Type: Bug
Components: Rust
Affects Versions: 0.16.0
Reporter: Andy Grove
Assignee: Andy Grove
Fix For: 1.0.0
Running "cargo publish" for the arrow-flight crate resulted in this error:
{code:java}
error: failed to run custom build command for `arrow-flight v0.16.0
(/home/andy/apache-arrow-0.16.0/rust/target/package/arrow-flight-0.16.0)`Caused
by:
process didn't exit successfully:
`/home/andy/apache-arrow-0.16.0/rust/target/package/arrow-flight-0.16.0/target/debug/build/arrow-flight-1b2906a3933d2832/build-script-build`
(exit code: 1)
--- stderr
Error: Custom { kind: Other, error: "protoc failed: ../../format: warning:
directory does not exist.\nCould not make proto path relative:
../../format/Flight.proto: No such file or directory\n" }
{code}
The workaround was to edit the build.rs and make the path absolute and then run
"cargo publish --allow-dirty", but we should find a better solution before the
next release.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] Release Apache Arrow 0.16.0 - RC2
I'll look at the Rust issue now. On Fri, Feb 7, 2020 at 3:27 AM Krisztián Szűcs wrote: > Status of the post release tasks: > > - [DONE] marking the released version as "RELEASED" on JIRA > - [DONE] rebase the master branch on top of the release branch > We have failing builds on the master because of the bumped version > numbers, those must be fixed. > - [DONE] rebase the pull requests > - [DONE] uploading source release artifacts to SVN > - [DONE] uploading binary release artifacts to Bintray > - [DONE] updating the Arrow website > - [DONE] python source distribution > - [DONE] ruby gems > - [DONE] javascript npm packages > - [DONE] .NET nuget packages > - [DONE] java maven artifacts >I had to re-stage the maven artifacts > - [kszucs] python wheels > Partially done, the manylinux2010 and manylinux2014 wheels have hit > the 60MB limit of pypi, requested to increase it > https://github.com/pypa/pypi-support/issues/186 > - [xhochy] conda packages > Uwe works on it, arrow-cpp is already merged. > - [andygrove] rust packages >Arrow-flight misses the format directory, didn't dig deeper. Andy > could you take a look? > - [nealrichardson] R packages > Neal should handle this. > - [ ] homebrew packages > - [ ] updating website with new API documentation > - [ ] announcing release > > On Fri, Feb 7, 2020 at 8:18 AM Krisztián Szűcs > wrote: > > > > The VOTE carries with 4 binding +1 votes, 3 non-binding +1 votes and > > one binding +0 vote. > > > > I'm starting the post-release tasks, if anyone wants to help please let > me know. > > > > On Fri, Feb 7, 2020 at 12:25 AM Krisztián Szűcs > > wrote: > > > > > > So far we have the following votes: > > > > > > +0 (binding) > > > +1 (binding) > > > +1 (non-binding) > > > +1 (binding) > > > +1 (non-binding) > > > +1 (binding) > > > +1 (non-binding) > > > +1 (binding) > > > > > > 4 +1 (binding) > > > 3 +1 (non-binding) > > > > > > I'm waiting for votes until tomorrow morning (UTC), then I'm closing > the VOTE. > > > > > > Thanks everyone! > > > > > > - Krisztian > > > > > > On Fri, Feb 7, 2020 at 12:06 AM Krisztián Szűcs > > > wrote: > > > > > > > > Testing on macOS Catalina > > > > > > > > Binaries: OK > > > > > > > > Wheels: OK > > > > Verified on macOS and on Linux. > > > > On linux the verification script has failed for python 3.5 and > manylinux2010 > > > > and manylinux2014 with unsupported platform tag. I've manually > checked > > > > these wheels in the python:3.5 docker image, and the wheels were good > > > > (this is automatically checked by crossbow too [1]). All other > wheels were > > > > passing using the verification script. > > > > > > > > Source: OK > > > > I had to revert the nvm path [2] to pass the js and integration > tests and > > > > force the glib test to use my system python instead of the conda one. > > > > > > > > I vote with +1 (binding) > > > > > > > > [1]: > https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L568 > > > > [2]: > https://github.com/apache/arrow/commit/37434fb34a1f2cd5273092ed3e1c61db90bb4dd2 > > > > > > > > > > > > On Thu, Feb 6, 2020 at 7:42 PM Neal Richardson > > > > wrote: > > > > > > > > > > I re-verified the macOS wheels and they worked but I had to > hard-code > > > > > `MACOSX_DEPLOYMENT_TARGET="10.14"` to get past the cython error I > reported > > > > > previously. I tried to set that env var dynamically based on your > current > > > > > OS version but didn't succeed in getting it passed through to > pytest, > > > > > despite many attempts to `export` it; someone with better bash > skills than > > > > > I should probably add that to the script. FTR `defaults read > loginwindow > > > > > SystemVersionStampAsString | sed s/\.[0-9]$//` returns the right > macOS > > > > > version. > > > > > > > > > > Neal > > > > > > > > > > On Wed, Feb 5, 2020 at 1:29 PM Wes McKinney > wrote: > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > I was able to verify the Windows wheels with the following patch > applied > > > > > > > > > > > > https://github.com/apache/arrow/pull/6364 > > > > > > > > > > > > On Wed, Feb 5, 2020 at 1:09 PM Krisztián Szűcs > > > > > > wrote: > > > > > > > > > > > > > > There were binary naming issues with the macosx and the > win-cp38 > > > > > > > wheels. > > > > > > > I've uploaded them, all of the wheels should be available now > [1] > > > > > > > > > > > > > > Note that the newly built macosx wheels have 10_9 platform tag > > > > > > > instead of 10_6, so the verification script must be updated > [2] to > > > > > > > verify the macosx wheels. > > > > > > > > > > > > > > [1] > https://bintray.com/apache/arrow/python-rc/0.16.0-rc2#files > > > > > > > [2] > > > > > > > https://github.com/apache/arrow/pull/6362/files#diff-8cc7fa3ae5de30b356c17d7a4b59fe09R658 > > > > > > > > > > > > > > On Wed, Feb 5, 2020 at 6:30 PM Krisztián Szűcs > > > > > > > wrote: > > > > > > > > > > > > > > > > The wheel was built successfully and available under the > crossbow > >
[NIGHTLY] Arrow Build Report for Job nightly-2020-02-07-0
Arrow Build Report for Job nightly-2020-02-07-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0 Failed Tasks: - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-travis-gandiva-jar-osx - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-travis-homebrew-cpp - macos-r-autobrew: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-travis-macos-r-autobrew - test-conda-python-3.7-turbodbc-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7-turbodbc-latest - test-conda-python-3.7-turbodbc-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7-turbodbc-master - wheel-manylinux1-cp36m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-wheel-manylinux1-cp36m - wheel-manylinux2014-cp38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-wheel-manylinux2014-cp38 Succeeded Tasks: - centos-6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-centos-6 - centos-7: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-centos-7 - centos-8: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-centos-8 - conda-linux-gcc-py27: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-linux-gcc-py27 - conda-linux-gcc-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-linux-gcc-py36 - conda-linux-gcc-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-linux-gcc-py37 - conda-linux-gcc-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-linux-gcc-py38 - conda-osx-clang-py27: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-osx-clang-py27 - conda-osx-clang-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-osx-clang-py36 - conda-osx-clang-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-osx-clang-py37 - conda-osx-clang-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-osx-clang-py38 - conda-win-vs2015-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-win-vs2015-py36 - conda-win-vs2015-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-win-vs2015-py37 - conda-win-vs2015-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-conda-win-vs2015-py38 - debian-buster: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-debian-buster - debian-stretch: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-azure-debian-stretch - gandiva-jar-trusty: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-travis-gandiva-jar-trusty - test-conda-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-cpp - test-conda-python-2.7-pandas-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-2.7-pandas-latest - test-conda-python-2.7: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-2.7 - test-conda-python-3.6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.6 - test-conda-python-3.7-dask-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7-dask-latest - test-conda-python-3.7-hdfs-2.9.2: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7-hdfs-2.9.2 - test-conda-python-3.7-pandas-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7-pandas-latest - test-conda-python-3.7-pandas-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7-pandas-master - test-conda-python-3.7-spark-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7-spark-master - test-conda-python-3.7: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-07-0-circle-test-conda-python-3.7 - test-conda-python-3.8-dask-master:
Re: Arrow doesn't have a MapType
Arrow does have a Map type [1][2][3]. It is represented as a list of pairs. François [1] https://github.com/apache/arrow/blob/762202418541e843923b8cae640d15b4952a0af6/format/Schema.fbs#L60-L87 [2] https://github.com/apache/arrow/blob/762202418541e843923b8cae640d15b4952a0af6/cpp/src/arrow/type.h#L691-L719 [3] https://github.com/apache/arrow/blob/762202418541e843923b8cae640d15b4952a0af6/java/vector/src/main/java/org/apache/arrow/vector/complex/MapVector.java#L36-L47 On Fri, Feb 7, 2020 at 3:55 AM Shawn Yang wrote: > > Hi guys, > I'm writing an cross-language row-oriented serialization framework mainly > for java/python for now. I detained many data types and schema, field, such > as Byte, short, int, long, double, float, map, array, struct,. But then I > find using Arrow schema is a better choice. Since my framework need to > support conversion between my row-format and arrow columnar format. If I do > all it by myself, I need to support schema conversion and schema > serialization. Which is not necessary if I use arrow schema. > > But I find that arrow doesn't have a map data type, which is I needed > exactly. I know I can use struct to mock it or ExtensionType for it. But > it's not very convenient. So I want to know whether will Map type be > supported by arrow? > > Thanks. Regards
[jira] [Created] (ARROW-7793) [Java] If the child allocators leak memory the base allocator doesn't release its memory before throwing exception
Projjal Chanda created ARROW-7793: - Summary: [Java] If the child allocators leak memory the base allocator doesn't release its memory before throwing exception Key: ARROW-7793 URL: https://issues.apache.org/jira/browse/ARROW-7793 Project: Apache Arrow Issue Type: Bug Reporter: Projjal Chanda Assignee: Projjal Chanda Here [1] it should call super.close() before throwing exception [1] https://github.com/apache/arrow/blob/762202418541e843923b8cae640d15b4952a0af6/java/memory/src/main/java/org/apache/arrow/memory/BaseAllocator.java#L481 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Release Apache Arrow 0.16.0 - RC2
Status of the post release tasks: - [DONE] marking the released version as "RELEASED" on JIRA - [DONE] rebase the master branch on top of the release branch We have failing builds on the master because of the bumped version numbers, those must be fixed. - [DONE] rebase the pull requests - [DONE] uploading source release artifacts to SVN - [DONE] uploading binary release artifacts to Bintray - [DONE] updating the Arrow website - [DONE] python source distribution - [DONE] ruby gems - [DONE] javascript npm packages - [DONE] .NET nuget packages - [DONE] java maven artifacts I had to re-stage the maven artifacts - [kszucs] python wheels Partially done, the manylinux2010 and manylinux2014 wheels have hit the 60MB limit of pypi, requested to increase it https://github.com/pypa/pypi-support/issues/186 - [xhochy] conda packages Uwe works on it, arrow-cpp is already merged. - [andygrove] rust packages Arrow-flight misses the format directory, didn't dig deeper. Andy could you take a look? - [nealrichardson] R packages Neal should handle this. - [ ] homebrew packages - [ ] updating website with new API documentation - [ ] announcing release On Fri, Feb 7, 2020 at 8:18 AM Krisztián Szűcs wrote: > > The VOTE carries with 4 binding +1 votes, 3 non-binding +1 votes and > one binding +0 vote. > > I'm starting the post-release tasks, if anyone wants to help please let me > know. > > On Fri, Feb 7, 2020 at 12:25 AM Krisztián Szűcs > wrote: > > > > So far we have the following votes: > > > > +0 (binding) > > +1 (binding) > > +1 (non-binding) > > +1 (binding) > > +1 (non-binding) > > +1 (binding) > > +1 (non-binding) > > +1 (binding) > > > > 4 +1 (binding) > > 3 +1 (non-binding) > > > > I'm waiting for votes until tomorrow morning (UTC), then I'm closing the > > VOTE. > > > > Thanks everyone! > > > > - Krisztian > > > > On Fri, Feb 7, 2020 at 12:06 AM Krisztián Szűcs > > wrote: > > > > > > Testing on macOS Catalina > > > > > > Binaries: OK > > > > > > Wheels: OK > > > Verified on macOS and on Linux. > > > On linux the verification script has failed for python 3.5 and > > > manylinux2010 > > > and manylinux2014 with unsupported platform tag. I've manually checked > > > these wheels in the python:3.5 docker image, and the wheels were good > > > (this is automatically checked by crossbow too [1]). All other wheels were > > > passing using the verification script. > > > > > > Source: OK > > > I had to revert the nvm path [2] to pass the js and integration tests and > > > force the glib test to use my system python instead of the conda one. > > > > > > I vote with +1 (binding) > > > > > > [1]: https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L568 > > > [2]: > > > https://github.com/apache/arrow/commit/37434fb34a1f2cd5273092ed3e1c61db90bb4dd2 > > > > > > > > > On Thu, Feb 6, 2020 at 7:42 PM Neal Richardson > > > wrote: > > > > > > > > I re-verified the macOS wheels and they worked but I had to hard-code > > > > `MACOSX_DEPLOYMENT_TARGET="10.14"` to get past the cython error I > > > > reported > > > > previously. I tried to set that env var dynamically based on your > > > > current > > > > OS version but didn't succeed in getting it passed through to pytest, > > > > despite many attempts to `export` it; someone with better bash skills > > > > than > > > > I should probably add that to the script. FTR `defaults read loginwindow > > > > SystemVersionStampAsString | sed s/\.[0-9]$//` returns the right macOS > > > > version. > > > > > > > > Neal > > > > > > > > On Wed, Feb 5, 2020 at 1:29 PM Wes McKinney wrote: > > > > > > > > > +1 (binding) > > > > > > > > > > I was able to verify the Windows wheels with the following patch > > > > > applied > > > > > > > > > > https://github.com/apache/arrow/pull/6364 > > > > > > > > > > On Wed, Feb 5, 2020 at 1:09 PM Krisztián Szűcs > > > > > wrote: > > > > > > > > > > > > There were binary naming issues with the macosx and the win-cp38 > > > > > > wheels. > > > > > > I've uploaded them, all of the wheels should be available now [1] > > > > > > > > > > > > Note that the newly built macosx wheels have 10_9 platform tag > > > > > > instead of 10_6, so the verification script must be updated [2] to > > > > > > verify the macosx wheels. > > > > > > > > > > > > [1] https://bintray.com/apache/arrow/python-rc/0.16.0-rc2#files > > > > > > [2] > > > > > https://github.com/apache/arrow/pull/6362/files#diff-8cc7fa3ae5de30b356c17d7a4b59fe09R658 > > > > > > > > > > > > On Wed, Feb 5, 2020 at 6:30 PM Krisztián Szűcs > > > > > > wrote: > > > > > > > > > > > > > > The wheel was built successfully and available under the crossbow > > > > > > > releases. Something must have gone wrong during download/upload > > > > > > > to bintray. I'm re-uploading the wheels again, waiting for the > > > > > > > network. > > > > > > > > > > > > > > On Wed, Feb 5, 2020 at 6:14 PM Wes McKinney > > > > > wrote: > > > > > > > > > > > > > > > > The Windows wheel RC script is
[jira] [Created] (ARROW-7792) arrow::read_feather does not close connection to file
Martin created ARROW-7792: - Summary: arrow::read_feather does not close connection to file Key: ARROW-7792 URL: https://issues.apache.org/jira/browse/ARROW-7792 Project: Apache Arrow Issue Type: Bug Components: R Reporter: Martin x = as.data.frame(seq(1:100)) pbFilename <- file.path(getwd(), "reproduceBug.feather") arrow::write_feather(x = x, sink = pbFilename) file.exists(pbFilename) file.remove(pbFilename) arrow::write_feather(x = x, sink = pbFilename) tempDX <- arrow::read_feather(file = pbFilename, as_data_frame = T) file.exists(pbFilename) file.remove(pbFilename) >Warning message: >In file.remove(pbFilename) : >cannot remove file 'C:/Martin/Repo/ReinforcementLearner/reproduceBug.feather', >reason > 'Permission denied' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7791) [C++] [Test] Fix building error "cannot bind lvalue"
Jiajia Li created ARROW-7791: Summary: [C++] [Test] Fix building error "cannot bind lvalue" Key: ARROW-7791 URL: https://issues.apache.org/jira/browse/ARROW-7791 Project: Apache Arrow Issue Type: Test Components: C++ Reporter: Jiajia Li arrow/cpp/src/parquet/arrow/arrow_reader_writer_test.cc:2900:12: error: cannot bind 'std::unique_ptr' lvalue to 'std::unique_ptr&&' -- This message was sent by Atlassian Jira (v8.3.4#803005)
Arrow doesn't have a MapType
Hi guys, I'm writing an cross-language row-oriented serialization framework mainly for java/python for now. I detained many data types and schema, field, such as Byte, short, int, long, double, float, map, array, struct,. But then I find using Arrow schema is a better choice. Since my framework need to support conversion between my row-format and arrow columnar format. If I do all it by myself, I need to support schema conversion and schema serialization. Which is not necessary if I use arrow schema. But I find that arrow doesn't have a map data type, which is I needed exactly. I know I can use struct to mock it or ExtensionType for it. But it's not very convenient. So I want to know whether will Map type be supported by arrow? Thanks. Regards
[jira] [Created] (ARROW-7790) [Website] Update how to install Linux packages
Kouhei Sutou created ARROW-7790: --- Summary: [Website] Update how to install Linux packages Key: ARROW-7790 URL: https://issues.apache.org/jira/browse/ARROW-7790 Project: Apache Arrow Issue Type: Improvement Components: Website Reporter: Kouhei Sutou Assignee: Kouhei Sutou -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7789) Unknown error when using arrow::write_feather() in R 3.5.3
Martin created ARROW-7789: - Summary: Unknown error when using arrow::write_feather() in R 3.5.3 Key: ARROW-7789 URL: https://issues.apache.org/jira/browse/ARROW-7789 Project: Apache Arrow Issue Type: Bug Reporter: Martin Unknown error when using arrow::write_feather() in R 3.5.3 pb = as.data.frame(seq(1:100)) pbFilename <- file.path(getwd(), "reproduceBug.feather") arrow::write_feather(x = pb, sink = pbFilename) >Error in exists(name, envir = envir, inherits = FALSE) : > use of NULL environment is defunct -- This message was sent by Atlassian Jira (v8.3.4#803005)
