Re: Ask for ARM CI for spark

2019-06-25 Thread Tianhua huang
Thanks Shane :) This sounds good, and yes I agree that it's best to keep the test/build infrastructure in one place. If you can't find the ARM resource we are willing to support the ARM instance :) Our goal is to make more open source software to be more compatible for aarch64 platform, so let's

Re: Ask for ARM CI for spark

2019-06-25 Thread shane knapp
...or via VM as you mentioned earlier. :) shane (who will file a JIRA tomorrow) On Tue, Jun 25, 2019 at 6:44 PM shane knapp wrote: > i'd much prefer that we keep the test/build infrastructure in one place. > > we don't have ARM hardware, but there's a slim possibility i can scare > something u

Re: Ask for ARM CI for spark

2019-06-25 Thread shane knapp
i'd much prefer that we keep the test/build infrastructure in one place. we don't have ARM hardware, but there's a slim possibility i can scare something up in our older research stock... another option would be to run the build in a arm-based docker container, which (according to the intarwebs)

Re: Ask for ARM CI for spark

2019-06-25 Thread Tianhua huang
I forked apache/spark project and propose a job( https://github.com/theopenlab/spark/pull/1) for spark building in OpenLab ARM instance, this is the first step to build spark on ARM, I can enable a periodic job for arm building for apache/spark master if you guys like. Later I will run tests for

DSV2 API Question

2019-06-25 Thread Andrew Melo
Hello, I've (nearly) implemented a DSV2-reader interface to read particle physics data stored in the ROOT (https://root.cern.ch/) file format. You can think of these ROOT files as roughly parquet-like: column-wise and nested (i.e. a column can be of type "float[]", meaning each row in the column i