Re: Coordinating / scheduling C++ Parquet-Arrow nested data work (ARROW-1644 and others)

2020-02-04 Thread Micah Kornfield
> > Glad to hear about the progress. As I mentioned on #2, what do you > think about setting up a feature branch for you to merge PRs into? > Then the branch can be iterated on and we can merge it back when it's > feature complete and does not have perf regressions for the flat > read/write path.

Re: [DISCUSS] Improving GitHub Actions configurations + tooling

2020-02-04 Thread Wes McKinney
hi Kou, Thanks for your thoughts. My preference would be to generate _some_ configurations but not necessarily _all_. The more copy-and-pasting is going on, the harder it will be to refactor and make general improvements to some configurations (since you may have to change N things instead of 1

Re: [Gandiva] LLVM version

2020-02-04 Thread Sutou Kouhei
Thanks! In <3ab45aeb-d789-4876-8a1e-ac8424257...@www.fastmail.com> "Re: [Gandiva] LLVM version" on Tue, 04 Feb 2020 17:36:12 +0530, "Projjal Chanda" wrote: > Hi Kou, > Sure. I will let you know. > > Regards, > Projjal > > On Tue, Feb 4, 2020, at 2:20 AM, Sutou Kouhei wrote: >> Hi Projjal,

Re: [DISCUSS] Improving GitHub Actions configurations + tooling

2020-02-04 Thread Sutou Kouhei
Hi, -0 for this idea. I don't strongly against this idea but I don't like this idea. The current raw GitHub Actions configurations + raw Docker tools (+ raw shell scripts) are simple and easy to understand for many developers because there are no magics. If we create our tools, they will

Re: [Java] Issues with IntelliJ + errorprone + OpenJDK

2020-02-04 Thread Fan Liya
I could not reproduce the problem, and I did not see "central maven org" from my environment. 1. If the problem occurred when installing the plugin, maybe we can download the plugin (from https://plugins.jetbrains.com/plugin/7349-error-prone-compiler/versions) and install it locally. 2. If the

Re: [VOTE] Release Apache Arrow 0.16.0 - RC2

2020-02-04 Thread Krisztián Szűcs
Hi, I've cherry-picked the wheel fix [1] on top of the 0.16 release tag, re-built the wheels using crossbow [2], and uploaded them to bintray [3] (also removed win-py38m). Anyone who has voted after verifying the wheels, please re-run the verification script again for the wheels and re-vote.

Re: Coordinating / scheduling C++ Parquet-Arrow nested data work (ARROW-1644 and others)

2020-02-04 Thread Wes McKinney
hi Micah On Mon, Feb 3, 2020 at 12:01 AM Micah Kornfield wrote: > > Just to give an update. I've been a little bit delayed, but my progress is > as follows: > 1. Had 1 PR merged that will exercise basic end-to-end tests. > 2. Have another PR open that allows a configuration option in C++ to >

[jira] [Created] (ARROW-7772) Unable to dplyr::filter on date32 object when using open_dataset()

2020-02-04 Thread Stephanie Hazlitt (Jira)
Stephanie Hazlitt created ARROW-7772: Summary: Unable to dplyr::filter on date32 object when using open_dataset() Key: ARROW-7772 URL: https://issues.apache.org/jira/browse/ARROW-7772 Project:

[jira] [Created] (ARROW-7771) [Release] Use ARROW_TMPDIR environment variable in the verification scripts instead of TMPDIR

2020-02-04 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-7771: -- Summary: [Release] Use ARROW_TMPDIR environment variable in the verification scripts instead of TMPDIR Key: ARROW-7771 URL: https://issues.apache.org/jira/browse/ARROW-7771

Re: [DISCUSS] Improving GitHub Actions configurations + tooling

2020-02-04 Thread Krisztián Szűcs
On Tue, Feb 4, 2020 at 6:31 PM Wes McKinney wrote: > > I'm personally not too concerned with the details as long as people > generally agree that the solution is maintainable (Kou's and Antoine's > feedback here would be helpful) and there is not an abundant odor of > code duplication I'm not

Re: [VOTE] Release Apache Arrow 0.16.0 - RC2

2020-02-04 Thread Bryan Cutler
+1 I had some trouble due to ARROW-7760 at first, but applied the same patch and passed. I ran the command: TMPDIR=/tmp/arrow TEST_DEFAULT=0 TEST_SOURCE=1 TEST_CPP=1 TEST_PYTHON=1 TEST_JAVA=1 TEST_INTEGRATION_CPP=1 TEST_INTEGRATION_JAVA=1 dev/release/verify-release-candidate.sh source 0.16.0 2

[jira] [Created] (ARROW-7770) [Release] Archery does not use correct integration test args

2020-02-04 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-7770: --- Summary: [Release] Archery does not use correct integration test args Key: ARROW-7770 URL: https://issues.apache.org/jira/browse/ARROW-7770 Project: Apache Arrow

Arrow sync call February 5 at 12:00 US/Eastern, 17:00 UTC

2020-02-04 Thread Neal Richardson
Hi all, Reminder that our biweekly call is tomorrow at https://meet.google.com/vtm-teks-phx. All are welcome to join. Notes will be sent out to the mailing list afterwards (like, right after, I promise). Neal

Re: Arrow sync call January 22 at 12:00 US/Eastern, 17:00 UTC

2020-02-04 Thread Neal Richardson
Oops, forgot to send the notes for the call from two weeks ago: Attendees: Projjal Chanda Todd Hendricks Ben Kietzman Micah Kornfield Rok Mihevc Antoine Pitrou Neal Richardson François Saint-Jacques Krisztián Szucs Aaron Luke Discussion: * 0.16 * C++ Result class refactor: we've stalled, still a

Re: [Java] Issues with IntelliJ + errorprone + OpenJDK

2020-02-04 Thread Bryan Cutler
Here is where it came up at, looks to be installed in the m2 repository bryan@lm-P50 ~ $ find ~/ -name "failureaccess-*.jar" -type f /home/bryan/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar

[jira] [Created] (ARROW-7769) [Python] Default PYARROW_CMAKE_GENERATOR can yield broken libraries with MSVC if the C++ toolchain is different

2020-02-04 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7769: --- Summary: [Python] Default PYARROW_CMAKE_GENERATOR can yield broken libraries with MSVC if the C++ toolchain is different Key: ARROW-7769 URL:

[jira] [Created] (ARROW-7768) Implement Length and TryClone traits for Cursor> in reader.rs

2020-02-04 Thread David Kegley (Jira)
David Kegley created ARROW-7768: --- Summary: Implement Length and TryClone traits for Cursor> in reader.rs Key: ARROW-7768 URL: https://issues.apache.org/jira/browse/ARROW-7768 Project: Apache Arrow

[jira] [Created] (ARROW-7767) [C++] Add a facility to create a Bitmap buffer from an data pointer with a specified sentinel

2020-02-04 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7767: - Summary: [C++] Add a facility to create a Bitmap buffer from an data pointer with a specified sentinel Key: ARROW-7767 URL:

[jira] [Created] (ARROW-7766) [Python][Packaging] Windows py38 wheels are built with wrong ABI tag

2020-02-04 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-7766: -- Summary: [Python][Packaging] Windows py38 wheels are built with wrong ABI tag Key: ARROW-7766 URL: https://issues.apache.org/jira/browse/ARROW-7766 Project:

Re: [VOTE] Release Apache Arrow 0.16.0 - RC2

2020-02-04 Thread Wes McKinney
+1 (binding) Some patches were required to the verification scripts but I have run: * Full source verification on Ubuntu 18.04 * Linux binary verification * Source verification on Windows 10 (needed ARROW-6757) * Windows binary verification. Note that Python 3.8 wheel is broken (see ARROW-7755).

Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-02-04-0

2020-02-04 Thread Wes McKinney
congratulations! =) On Tue, Feb 4, 2020 at 7:55 AM Francois Saint-Jacques wrote: > > This is a first! > > On Tue, Feb 4, 2020 at 8:47 AM Crossbow wrote: > > > > > > Arrow Build Report for Job nightly-2020-02-04-0 > > > > All tasks: > >

Re: [DISCUSS] Improving GitHub Actions configurations + tooling

2020-02-04 Thread Wes McKinney
I'm personally not too concerned with the details as long as people generally agree that the solution is maintainable (Kou's and Antoine's feedback here would be helpful) and there is not an abundant odor of code duplication On Tue, Feb 4, 2020 at 9:48 AM Neal Richardson wrote: > > What if we

[jira] [Created] (ARROW-7765) [C++] Add Result to the Visitor pattern

2020-02-04 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7765: - Summary: [C++] Add Result to the Visitor pattern Key: ARROW-7765 URL: https://issues.apache.org/jira/browse/ARROW-7765 Project: Apache Arrow

[jira] [Created] (ARROW-7764) [C++] Builders allocate a null bitmap buffer even if there is no nulls

2020-02-04 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7764: - Summary: [C++] Builders allocate a null bitmap buffer even if there is no nulls Key: ARROW-7764 URL: https://issues.apache.org/jira/browse/ARROW-7764

Re: [DISCUSS] Improving GitHub Actions configurations + tooling

2020-02-04 Thread Neal Richardson
What if we wrote our own action(s) to wrap up some of the boilerplate? It doesn't seem that there are any off-the-shelf actions we could use to drive docker-compose: https://github.com/marketplace?utf8=%E2%9C%93=actions=docker-compose but I don't think it would be that difficult to wrap

Re: [Java] Issues with IntelliJ + errorprone + OpenJDK

2020-02-04 Thread Andy Grove
Actually, central.maven.org doesn't even exist ... On Tue, Feb 4, 2020 at 8:28 AM Andy Grove wrote: > Thanks for the help but I followed the same instructions and get this > error: > > Error:Failed to download error-prone compiler JARs: Failed to download ' >

Re: [Java] Issues with IntelliJ + errorprone + OpenJDK

2020-02-04 Thread Andy Grove
Thanks for the help but I followed the same instructions and get this error: Error:Failed to download error-prone compiler JARs: Failed to download ' http://central.maven.org/maven2/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar ': central.maven.org The issue is that this maven

[jira] [Created] (ARROW-7763) [Java] Update README with instructions for IntelliJ users

2020-02-04 Thread Andy Grove (Jira)
Andy Grove created ARROW-7763: - Summary: [Java] Update README with instructions for IntelliJ users Key: ARROW-7763 URL: https://issues.apache.org/jira/browse/ARROW-7763 Project: Apache Arrow

Re: [DISCUSS] Improving GitHub Actions configurations + tooling

2020-02-04 Thread Krisztián Szűcs
Hi, On Mon, Feb 3, 2020 at 5:25 PM Wes McKinney wrote: > > hi folks, > > I have noticed that many of our GitHub Actions configurations are very > similar to each other They are indeed. > > https://www.diffchecker.com/eF4tHdzo > > Aside from the "copy-paste" issue, some work would have to be done

[jira] [Created] (ARROW-7762) [Python] Exceptions in ParquetWriter get ignored

2020-02-04 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7762: Summary: [Python] Exceptions in ParquetWriter get ignored Key: ARROW-7762 URL: https://issues.apache.org/jira/browse/ARROW-7762 Project: Apache Arrow

Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-02-04-0

2020-02-04 Thread Francois Saint-Jacques
This is a first! On Tue, Feb 4, 2020 at 8:47 AM Crossbow wrote: > > > Arrow Build Report for Job nightly-2020-02-04-0 > > All tasks: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-04-0 > > Succeeded Tasks: > - centos-6: > URL: >

[NIGHTLY] Arrow Build Report for Job nightly-2020-02-04-0

2020-02-04 Thread Crossbow
Arrow Build Report for Job nightly-2020-02-04-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-04-0 Succeeded Tasks: - centos-6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-04-0-azure-centos-6 - centos-7: URL:

[jira] [Created] (ARROW-7761) [C++] Add S3 support to fs::FileSystemFromUri

2020-02-04 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7761: - Summary: [C++] Add S3 support to fs::FileSystemFromUri Key: ARROW-7761 URL: https://issues.apache.org/jira/browse/ARROW-7761 Project: Apache Arrow

[jira] [Created] (ARROW-7760) [Release] Fix verify-release-candidate.sh since pip3 seems to no longer be in miniconda

2020-02-04 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-7760: -- Summary: [Release] Fix verify-release-candidate.sh since pip3 seems to no longer be in miniconda Key: ARROW-7760 URL: https://issues.apache.org/jira/browse/ARROW-7760

Re: [Gandiva] LLVM version

2020-02-04 Thread Projjal Chanda
Hi Kou, Sure. I will let you know. Regards, Projjal On Tue, Feb 4, 2020, at 2:20 AM, Sutou Kouhei wrote: > Hi Projjal, > > > Let me test the change by running it with Dremio. > > Thanks! > > > Will update if there are any issues. > > It means that we can move forward if we don't get any >