Thanks Sean. I am very happy to hear that the community will put effort to fix the ARM-related issues. I'd be happy to help if you like. And could you give the trace link of this issue, then I can check it is fixed or not, thank you. As far as I know the old versions of spark support ARM, and now the new versions don't, this just shows that we need a CI to check whether the spark support ARM and whether some modification break it. I will add a demo job in OpenLab to build spark on ARM and do a simple UT test. Later I will give the job link.
Let me know what you think. Thank you all! On Wed, Jun 19, 2019 at 8:47 PM Sean Owen <sro...@gmail.com> wrote: > I'd begin by reporting and fixing ARM-related issues in the build. If > they're small, of course we should do them. If it requires significant > modifications, we can discuss how much Spark can support ARM. I don't > think it's yet necessary for the Spark project to run these CI builds > until that point, but it's always welcome if people are testing that > separately. > > On Wed, Jun 19, 2019 at 7:41 AM Holden Karau <hol...@pigscanfly.ca> wrote: > > > > Moving to dev@ for increased visibility among the developers. > > > > On Wed, Jun 19, 2019 at 1:24 AM Tianhua huang <huangtianhua...@gmail.com> > wrote: > >> > >> Thanks for your reply. > >> > >> As I said before, I met some problem of build or test for spark on > aarch64 server, so it will be better to have the ARM CI to make sure the > spark is compatible for AArch64 platforms. > >> > >> I’m from OpenLab team(https://openlabtesting.org/ ,a community to do > open source project testing. And we can support some Arm virtual machines > to AMPLab Jenkins, and also we have a developer team that willing to work > on this, we willing to maintain build CI jobs and address the CI issues. > What do you think? > >> > >> > >> Thanks for your attention. > >> > >> > >> On Wed, Jun 19, 2019 at 6:39 AM shane knapp <skn...@berkeley.edu> > wrote: > >>> > >>> yeah, we don't have any aarch64 systems for testing... this has been > asked before but is currently pretty low on our priority list as we don't > have the hardware. > >>> > >>> sorry, > >>> > >>> shane > >>> > >>> On Mon, Jun 10, 2019 at 7:08 PM Tianhua huang < > huangtianhua...@gmail.com> wrote: > >>>> > >>>> Hi, sorry to disturb you. > >>>> The CI testing for apache spark is supported by AMPLab Jenkins, and I > find there are some computers(most of them are Linux (amd64) arch) for the > CI development, but seems there is no Aarch64 computer for spark CI > testing. Recently, I build and run test for spark(master and branch-2.4) on > my arm server, and unfortunately there are some problems, for example, ut > test is failed due to a LEVELDBJNI native package, the details for java > test see http://paste.openstack.org/show/752063/ and python test see > http://paste.openstack.org/show/752709/ > >>>> So I have a question about the ARM CI testing for spark, is there any > plan to support it? Thank you very much and I will wait for your reply! > >>> > >>> > >>> > >>> -- > >>> Shane Knapp > >>> UC Berkeley EECS Research / RISELab Staff Technical Lead > >>> https://rise.cs.berkeley.edu > > > > > > > > -- > > Twitter: https://twitter.com/holdenkarau > > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 > > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >
