...or via VM as you mentioned earlier. :) shane (who will file a JIRA tomorrow)
On Tue, Jun 25, 2019 at 6:44 PM shane knapp <skn...@berkeley.edu> wrote: > i'd much prefer that we keep the test/build infrastructure in one place. > > we don't have ARM hardware, but there's a slim possibility i can scare > something up in our older research stock... > > another option would be to run the build in a arm-based docker container, > which (according to the intarwebs) is possible. > > shane > > On Tue, Jun 25, 2019 at 6:35 PM Tianhua huang <huangtianhua...@gmail.com> > wrote: > >> I forked apache/spark project and propose a job( >> https://github.com/theopenlab/spark/pull/1) for spark building in >> OpenLab ARM instance, this is the first step to build spark on ARM, I can >> enable a periodic job for arm building for apache/spark master if you >> guys like. Later I will run tests for spark. I also willing to be the >> maintainer of the arm ci of spark. >> >> Thanks for you attention. >> >> On Thu, Jun 20, 2019 at 10:17 AM Tianhua huang <huangtianhua...@gmail.com> >> wrote: >> >>> Thanks Sean. >>> >>> I am very happy to hear that the community will put effort to fix the >>> ARM-related issues. I'd be happy to help if you like. And could you give >>> the trace link of this issue, then I can check it is fixed or not, thank >>> you. >>> As far as I know the old versions of spark support ARM, and now the new >>> versions don't, this just shows that we need a CI to check whether the >>> spark support ARM and whether some modification break it. >>> I will add a demo job in OpenLab to build spark on ARM and do a simple >>> UT test. Later I will give the job link. >>> >>> Let me know what you think. >>> >>> Thank you all! >>> >>> >>> On Wed, Jun 19, 2019 at 8:47 PM Sean Owen <sro...@gmail.com> wrote: >>> >>>> I'd begin by reporting and fixing ARM-related issues in the build. If >>>> they're small, of course we should do them. If it requires significant >>>> modifications, we can discuss how much Spark can support ARM. I don't >>>> think it's yet necessary for the Spark project to run these CI builds >>>> until that point, but it's always welcome if people are testing that >>>> separately. >>>> >>>> On Wed, Jun 19, 2019 at 7:41 AM Holden Karau <hol...@pigscanfly.ca> >>>> wrote: >>>> > >>>> > Moving to dev@ for increased visibility among the developers. >>>> > >>>> > On Wed, Jun 19, 2019 at 1:24 AM Tianhua huang < >>>> huangtianhua...@gmail.com> wrote: >>>> >> >>>> >> Thanks for your reply. >>>> >> >>>> >> As I said before, I met some problem of build or test for spark on >>>> aarch64 server, so it will be better to have the ARM CI to make sure the >>>> spark is compatible for AArch64 platforms. >>>> >> >>>> >> I’m from OpenLab team(https://openlabtesting.org/ ,a community to >>>> do open source project testing. And we can support some Arm virtual >>>> machines to AMPLab Jenkins, and also we have a developer team that willing >>>> to work on this, we willing to maintain build CI jobs and address the CI >>>> issues. What do you think? >>>> >> >>>> >> >>>> >> Thanks for your attention. >>>> >> >>>> >> >>>> >> On Wed, Jun 19, 2019 at 6:39 AM shane knapp <skn...@berkeley.edu> >>>> wrote: >>>> >>> >>>> >>> yeah, we don't have any aarch64 systems for testing... this has >>>> been asked before but is currently pretty low on our priority list as we >>>> don't have the hardware. >>>> >>> >>>> >>> sorry, >>>> >>> >>>> >>> shane >>>> >>> >>>> >>> On Mon, Jun 10, 2019 at 7:08 PM Tianhua huang < >>>> huangtianhua...@gmail.com> wrote: >>>> >>>> >>>> >>>> Hi, sorry to disturb you. >>>> >>>> The CI testing for apache spark is supported by AMPLab Jenkins, >>>> and I find there are some computers(most of them are Linux (amd64) arch) >>>> for the CI development, but seems there is no Aarch64 computer for spark CI >>>> testing. Recently, I build and run test for spark(master and branch-2.4) on >>>> my arm server, and unfortunately there are some problems, for example, ut >>>> test is failed due to a LEVELDBJNI native package, the details for java >>>> test see http://paste.openstack.org/show/752063/ and python test see >>>> http://paste.openstack.org/show/752709/ >>>> >>>> So I have a question about the ARM CI testing for spark, is there >>>> any plan to support it? Thank you very much and I will wait for your reply! >>>> >>> >>>> >>> >>>> >>> >>>> >>> -- >>>> >>> Shane Knapp >>>> >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>> >>> https://rise.cs.berkeley.edu >>>> > >>>> > >>>> > >>>> > -- >>>> > Twitter: https://twitter.com/holdenkarau >>>> > Books (Learning Spark, High Performance Spark, etc.): >>>> https://amzn.to/2MaRAG9 >>>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>> >>> > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
