пт, 17 янв. 2020 г. в 19:33, Martin Grigorov <[email protected]>:
> > > On Fri, Jan 17, 2020 at 4:13 PM Martin Grigorov <[email protected]> > wrote: > >> Hi all, >> >> Today's build consistently fails on my ARM64 VM: >> >> ########################## Starting vtest ########################## >> Testing with haproxy version: 2.2-dev0-70c5b0-123 >> # top TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (3.004) exit=2 >> 1 tests failed, 0 tests skipped, 34 tests passed >> ########################## Gathering results ########################## >> ###### Test case: reg-tests/mcli/mcli_start_progs.vtc ###### >> ## test results in: >> "/tmp/haregtests-2020-01-17_14-01-45.SGkYcJ/vtc.12807.6adfff44" >> ---- h1 haproxy h1 PID file check failed: >> ---- h1 Bad exit status: 0x0100 exit 0x1 signal 0 core 0 >> Makefile:964: recipe for target 'reg-tests' failed >> make: *** [reg-tests] Error 1 >> > > git bisect blaims this commit: > > 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit > commit 25b569302167e71b32e569a2366027e8e320e80a > Author: William Lallemand <[email protected]> > Date: Tue Jan 14 15:38:43 2020 +0100 > > REGTEST: mcli/mcli_start_progs: start 2 programs > > This regtest tests the issue #446 by starting 2 programs and checking > if > they exist in the "show proc" of the master CLI. > > Should be backported as far as 2.0. > > > https://travis-ci.com/haproxy/haproxy is green > https://cirrus-ci.com/github/haproxy/haproxy is green > and what is even more interesting is that > https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled ARM64 > testing on TravisCI) also just passed (after few failures due to timing > issues (I guess) > timing out might happen because of https://github.com/haproxy/haproxy/commit/ac8147446c7a3d1aa607042bc782095b03bc8dc4 your fork is 16 commits behind current master. try to rebase to master > > >> Regards, >> Martin >> >> On Fri, Jan 17, 2020 at 9:22 AM Илья Шипицин <[email protected]> >> wrote: >> >>> привет! >>> >>> пт, 17 янв. 2020 г. в 11:42, Martin Grigorov <[email protected] >>> >: >>> >>>> Привет Илья, >>>> >>>> On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин <[email protected]> >>>> wrote: >>>> >>>>> >>>>> >>>>> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov < >>>>> [email protected]>: >>>>> >>>>>> Hi Илья, >>>>>> >>>>>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> Hello, Martin! >>>>>>> >>>>>>> btw, just curious, how is Apache Foundation (or you personally) >>>>>>> interested in all that ? >>>>>>> please do not blame me, I really like to know. >>>>>>> >>>>>> >>>>> ok, so you work in some company that is interested in haproxy on >>>>> ARM64. >>>>> you do not want to tell it's name, at least is it legal ? is it >>>>> related to some government ? >>>>> if "no" and "no", I guess most people won't ask any more questions :) >>>>> >>>> >>>> It is legal and I do not work for a government of any country! >>>> >>>> >>>>> >>>>> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I >>>>> contribute to it. >>>>> Please do not consider me as an "official representative". >>>>> >>>>> >>>>> I'm interested in testing haproxy on ARM64, I planned to do so. I can >>>>> priorierize it according to your interest to it. >>>>> and yes, I can accept hardware donation (even considering I'm not part >>>>> of Haproxy Inc). >>>>> >>>>> also, from my point of view, what would be really useful in your case >>>>> is testing not just official reg-tests, but also >>>>> your configs. reg-tests cover only partially. If you enable clang asan >>>>> in your own workload there's a chance to catch >>>>> something interesting (or, to make sure your own workload is ok). I >>>>> can help with that as well. >>>>> >>>> >>>> Thanks for the offer! >>>> I've discussed it with my managers. >>>> Our offer is to donate a VM that could be used as an official CI agent >>>> for the HAProxy project long term. >>>> >>> >>> I'd split this into short term and long term approach. >>> >>> if you need to start any time soon, I'd focus on your own workload >>> testing first. I would build stand which emulates >>> your production workload, compile haproxy using clang address sanitizer >>> and give it a try (functional testing, load testing, ...) >>> >>> I can help with that. >>> >>> As for long term solution, currently haproxy simply cannot attach any >>> dedicated build agent to their CI. travis-ci does not allow >>> attaching dedicated agents. And haproxy team is very conservative in >>> adding new CI servers. >>> >>> I think, I will add arm64 (most probably openssl-1.1.1 only for now) >>> soon. Also, I'm going to investigate your libressl failures. >>> >>> so, dedicated vm definetly will help in troubleshooting issues, for >>> manual builds. It would save bunch of time. I do not mind if you will >>> add my ssh to that vm. >>> >>> >>> also, I requested access to Linaro. >>> >>> >>>> >>>> You can use Linaro for short term testing though. >>>> https://www.linaro.cloud/ >>>> Here you can request free VM for short periods: >>>> https://servicedesk.linaro.org/servicedesk/customer/portal/19/create/265 >>>> P.S. Linaro is not my employer! >>>> >>>> >>>> Regards, >>>> Martin >>>> >>>> >>>>> >>>>>> >>>>>>> >>>>>> @apache.org is just one of my several emails. And it is the default >>>>>> one in my email client. >>>>>> ASF is not related anyhow to my participation here. >>>>>> If I used my work email then it might look like some kind of >>>>>> advertisement. I'd like to avoid that! >>>>>> Next time I will use my @gmail.com one, as more neutral. Actually >>>>>> I've used the GMail one when registering to this mailing list, so >>>>>> probably >>>>>> the post from @apache has been moderated. I'll be more careful next time! >>>>>> >>>>>> Thanks, Илья! >>>>>> >>>>>> Regards, >>>>>> Martin >>>>>> >>>>>> >>>>>>> >>>>>>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov <[email protected]>: >>>>>>> >>>>>>>> Hello HAProxy developers, >>>>>>>> >>>>>>>> At work we are going to use more and more ARM64 based servers and >>>>>>>> HAProxy is one of the main products we rely on. >>>>>>>> So I went checking whether HAProxy project has a CI environment for >>>>>>>> testing on ARM architecture. >>>>>>>> >>>>>>> >>>>>>> we are looking towards >>>>>>> https://docs.travis-ci.com/user/multi-cpu-architectures >>>>>>> >>>>>>> >>>>>>>> I've found this recent discussion: >>>>>>>> https://www.mail-archive.com/[email protected]/msg35302.html >>>>>>>> (I didn't find a way how to continue on the same mail thread, so I'm >>>>>>>> starting a new one. Apologies for that!). >>>>>>>> >>>>>>> >>>>>>> I played with arm64 for a while, the issue I encountered is >>>>>>> travis-ci cache key, i.e. we cache openssl builds between our builds. >>>>>>> so travis used the same cache key for both amd64 and arm64 builds >>>>>>> (this might have changed recently, I did not check yet) >>>>>>> >>>>>>> arm64 is in my queue (as well as recent s390x arch from travis), >>>>>>> hope to get back to it within month or so. >>>>>>> >>>>>>> >>>>>>>> From this discussion and from >>>>>>>> https://github.com/haproxy/haproxy/blob/master/.travis.yml I >>>>>>>> understand that there is no public CI in use (i.e. TravisCI or >>>>>>>> CirrusCI) >>>>>>>> but some of the developers run some tests locally regularly. >>>>>>>> >>>>>>> >>>>>>> it is not completely true. >>>>>>> there's public CI. we do not use github PR machinery, so sometimes >>>>>>> tests fail after push to master branch. it is considered as ok, failures >>>>>>> are fixed pretty fast. >>>>>>> for example, see >>>>>>> https://www.mail-archive.com/[email protected]/msg35910.html >>>>>>> it was just perfect, failure detected using CI and fixed within few >>>>>>> days. no customers affected. >>>>>>> >>>>>>> >>>>>>>> I've forked the project and tested on TravisCI ( >>>>>>>> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately >>>>>>>> the builds were not very stable: >>>>>>>> 1) some tests fail sometimes. I guess it is because of some timing >>>>>>>> issues >>>>>>>> For example: >>>>>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636745241 >>>>>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636750676 >>>>>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636763346 >>>>>>>> >>>>>>> >>>>>>> that's very interesting. I'll have a look. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> 2) There was some weird issue on testing with LibreSSL >>>>>>>> The output redirect at >>>>>>>> https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96 >>>>>>>> for >>>>>>>> some reason got stuck the build. I've removed temporarily the output >>>>>>>> redirects and then it passed. So, it looks like some issue with >>>>>>>> TravisCI >>>>>>>> environment. >>>>>>>> >>>>>>> >>>>>>> arm64 is slower, I guess we should add "*travis*_*wait* *30" *to >>>>>>> build-ssl.sh script >>>>>>> thank for the hint >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> In addition I've run the build and tests on one of our machines and >>>>>>>> all was OK! >>>>>>>> >>>>>>>> My question to you is: Are you happy with your current way of >>>>>>>> testing ARM architectures or you want to add more ? >>>>>>>> Here are some options: >>>>>>>> 1) enable TravisCI >>>>>>>> >>>>>>> >>>>>>> already done >>>>>>> >>>>>>> https://travis-ci.com/haproxy/haproxy >>>>>>> >>>>>>> >>>>>>>> 2) my company is willing to donate an ARM64 based VM, if you are >>>>>>>> interested. >>>>>>>> >>>>>>> >>>>>>> I do not work at Haproxy Inc :) >>>>>>> Willy ? >>>>>>> >>>>>>> >>>>>>>> You will have a SSH access and a user with sudo permissions to >>>>>>>> install anything that is needed. >>>>>>>> The spec is aarch64 8 cores CPU 2GHz (Kunpeng), 16GB RAM, 500G disk >>>>>>>> space and 5M network bandwidth. The OS could be any of CentOS >>>>>>>> 7.4/7.5/7.6, >>>>>>>> EulerOS 2.8, Fedora 29, OpenSuse 15.0 and Ubuntu 16.04/18.04. >>>>>>>> >>>>>>>> In both cases it will be ARM64. From the earlier mail discussion I >>>>>>>> understand you would prefer ARM32. >>>>>>>> >>>>>>> >>>>>>> as for myself, I prefer both arm64 and arm32. >>>>>>> however, both AMD64 and ARM64 are the same 64 bits. both of them are >>>>>>> little-endian. but you mentioned at least 3 builds with ARM64 failing >>>>>>> (we >>>>>>> have those tests passing on AMD64)! >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> Martin >>>>>>>> >>>>>>>
