Can you begin by testing yourself? I think the first step is to make sure the build and tests work on ARM. If you find problems you can isolate them and try to fix them, or at least report them. It's only worth getting CI in place when we think builds will work.
On Tue, Jun 25, 2019 at 9:26 PM Tianhua huang <huangtianhua...@gmail.com> wrote: > > Thanks Shane :) > > This sounds good, and yes I agree that it's best to keep the test/build > infrastructure in one place. If you can't find the ARM resource we are > willing to support the ARM instance :) Our goal is to make more open source > software to be more compatible for aarch64 platform, so let's to do it. I > will be happy if I can give some help for the goal. > > Waiting for you good news :) > > On Wed, Jun 26, 2019 at 9:47 AM shane knapp <skn...@berkeley.edu> wrote: >> >> ...or via VM as you mentioned earlier. :) >> >> shane (who will file a JIRA tomorrow) >> >> On Tue, Jun 25, 2019 at 6:44 PM shane knapp <skn...@berkeley.edu> wrote: >>> >>> i'd much prefer that we keep the test/build infrastructure in one place. >>> >>> we don't have ARM hardware, but there's a slim possibility i can scare >>> something up in our older research stock... >>> >>> another option would be to run the build in a arm-based docker container, >>> which (according to the intarwebs) is possible. >>> >>> shane >>> >>> On Tue, Jun 25, 2019 at 6:35 PM Tianhua huang <huangtianhua...@gmail.com> >>> wrote: >>>> >>>> I forked apache/spark project and propose a >>>> job(https://github.com/theopenlab/spark/pull/1) for spark building in >>>> OpenLab ARM instance, this is the first step to build spark on ARM, I can >>>> enable a periodic job for arm building for apache/spark master if you guys >>>> like. Later I will run tests for spark. I also willing to be the >>>> maintainer of the arm ci of spark. >>>> >>>> Thanks for you attention. >>>> >>>> On Thu, Jun 20, 2019 at 10:17 AM Tianhua huang <huangtianhua...@gmail.com> >>>> wrote: >>>>> >>>>> Thanks Sean. >>>>> >>>>> I am very happy to hear that the community will put effort to fix the >>>>> ARM-related issues. I'd be happy to help if you like. And could you give >>>>> the trace link of this issue, then I can check it is fixed or not, thank >>>>> you. >>>>> As far as I know the old versions of spark support ARM, and now the new >>>>> versions don't, this just shows that we need a CI to check whether the >>>>> spark support ARM and whether some modification break it. >>>>> I will add a demo job in OpenLab to build spark on ARM and do a simple UT >>>>> test. Later I will give the job link. >>>>> >>>>> Let me know what you think. >>>>> >>>>> Thank you all! >>>>> >>>>> >>>>> On Wed, Jun 19, 2019 at 8:47 PM Sean Owen <sro...@gmail.com> wrote: >>>>>> >>>>>> I'd begin by reporting and fixing ARM-related issues in the build. If >>>>>> they're small, of course we should do them. If it requires significant >>>>>> modifications, we can discuss how much Spark can support ARM. I don't >>>>>> think it's yet necessary for the Spark project to run these CI builds >>>>>> until that point, but it's always welcome if people are testing that >>>>>> separately. >>>>>> >>>>>> On Wed, Jun 19, 2019 at 7:41 AM Holden Karau <hol...@pigscanfly.ca> >>>>>> wrote: >>>>>> > >>>>>> > Moving to dev@ for increased visibility among the developers. >>>>>> > >>>>>> > On Wed, Jun 19, 2019 at 1:24 AM Tianhua huang >>>>>> > <huangtianhua...@gmail.com> wrote: >>>>>> >> >>>>>> >> Thanks for your reply. >>>>>> >> >>>>>> >> As I said before, I met some problem of build or test for spark on >>>>>> >> aarch64 server, so it will be better to have the ARM CI to make sure >>>>>> >> the spark is compatible for AArch64 platforms. >>>>>> >> >>>>>> >> I’m from OpenLab team(https://openlabtesting.org/ ,a community to do >>>>>> >> open source project testing. And we can support some Arm virtual >>>>>> >> machines to AMPLab Jenkins, and also we have a developer team that >>>>>> >> willing to work on this, we willing to maintain build CI jobs and >>>>>> >> address the CI issues. What do you think? >>>>>> >> >>>>>> >> >>>>>> >> Thanks for your attention. >>>>>> >> >>>>>> >> >>>>>> >> On Wed, Jun 19, 2019 at 6:39 AM shane knapp <skn...@berkeley.edu> >>>>>> >> wrote: >>>>>> >>> >>>>>> >>> yeah, we don't have any aarch64 systems for testing... this has >>>>>> >>> been asked before but is currently pretty low on our priority list >>>>>> >>> as we don't have the hardware. >>>>>> >>> >>>>>> >>> sorry, >>>>>> >>> >>>>>> >>> shane >>>>>> >>> >>>>>> >>> On Mon, Jun 10, 2019 at 7:08 PM Tianhua huang >>>>>> >>> <huangtianhua...@gmail.com> wrote: >>>>>> >>>> >>>>>> >>>> Hi, sorry to disturb you. >>>>>> >>>> The CI testing for apache spark is supported by AMPLab Jenkins, and >>>>>> >>>> I find there are some computers(most of them are Linux (amd64) >>>>>> >>>> arch) for the CI development, but seems there is no Aarch64 >>>>>> >>>> computer for spark CI testing. Recently, I build and run test for >>>>>> >>>> spark(master and branch-2.4) on my arm server, and unfortunately >>>>>> >>>> there are some problems, for example, ut test is failed due to a >>>>>> >>>> LEVELDBJNI native package, the details for java test see >>>>>> >>>> http://paste.openstack.org/show/752063/ and python test see >>>>>> >>>> http://paste.openstack.org/show/752709/ >>>>>> >>>> So I have a question about the ARM CI testing for spark, is there >>>>>> >>>> any plan to support it? Thank you very much and I will wait for >>>>>> >>>> your reply! >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> -- >>>>>> >>> Shane Knapp >>>>>> >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>>>> >>> https://rise.cs.berkeley.edu >>>>>> > >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > Twitter: https://twitter.com/holdenkarau >>>>>> > Books (Learning Spark, High Performance Spark, etc.): >>>>>> > https://amzn.to/2MaRAG9 >>>>>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> >>> >>> >>> -- >>> Shane Knapp >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >> >> >> >> -- >> Shane Knapp >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
