No I'm saying look, potatoes or cherries let's do something, anything, other than nothing and here is a place to discuss with summarizations.
On Tue, Mar 6, 2018, 10:52 AM Carsten Haitzler <ras...@rasterman.com> wrote: > On Tue, 06 Mar 2018 16:21:48 +0000 Stephen Houston <smhousto...@gmail.com> > said: > > > Sigh. You are missing the point... > > i'm not... you're saying "hey look he wants cherries. check out this box of > potatoes we have!". > > i'm not saying that one or the other is invalid or wrong, but you're > linking > things that are not the same thing. not like that ticket solves anything > for > stefan and CI. that's my point. so why link them? > > > On Tue, Mar 6, 2018, 10:16 AM Carsten Haitzler <ras...@rasterman.com> > wrote: > > > > > On Tue, 06 Mar 2018 15:27:06 +0000 Stephen Houston < > smhousto...@gmail.com> > > > said: > > > > > > > We have developers leaving or severely cutting back their work, and > this > > > > includes developers who carry a large work load. Now we see Stefan > has > > > > lost faith and interest in QA/CI and is going to step back from > that... I > > > > think at some point we need to agree to stop arguing the merits of > > > getting > > > > better structure and better organization and agree that SOMETHING > needs > > > to > > > > be done and start putting together a plan. So I think indefini put > > > > together a phab ticket https://phab.enlightenment.org/T6740 we > should > > > > really work there to put together a plan to help. > > > > > > not sure why you said the above... that exact ticket is entirely > unrelated > > > to > > > what stefan is talking about. in fact they disagree. > > > > > > > On Tue, Mar 6, 2018 at 8:46 AM Carsten Haitzler < > ras...@rasterman.com> > > > > wrote: > > > > > > > > > On Tue, 6 Mar 2018 13:43:09 +0100 Stefan Schmidt < > > > ste...@osg.samsung.com> > > > > > said: > > > > > > > > > > > Hello. > > > > > > > > > > > > > > > > > > On 03/06/2018 12:34 PM, Carsten Haitzler (The Rasterman) wrote: > > > > > > > On Tue, 6 Mar 2018 09:54:50 +0100 Stefan Schmidt < > > > > > ste...@osg.samsung.com> > > > > > > > said: > > > > > > > > > > > > > >> Hello. > > > > > > >> > > > > > > >> > > > > > > >> On 03/06/2018 07:44 AM, Carsten Haitzler (The Rasterman) > wrote: > > > > > > >>> tbh platforms is an issue. windows is kind of big as setting > up a > > > > > > >>> cross-build environment is a fair bit of work. setting up a > > > windows > > > > > vm > > > > > > >>> costs money to test (need licenses, or if you can scrounge > one > > > off > > > > > another > > > > > > >>> pc you have/had). osx is worse in that you have to go buy > > > hardware > > > > > to run > > > > > > >>> osx to test. > > > > > > >>> > > > > > > >>> i think making it a requirement that every commit work on > every > > > > > platform > > > > > > >>> is not viable UNLESS: > > > > > > >>> > > > > > > >>> 1. a windows cross-compile env is available to developers > (e.g. > > > ssh > > > > > into a > > > > > > >>> vm set up for this with clear documentation and > scripts/tools to > > > do > > > > > the > > > > > > >>> builds. i have one set up for me at home). > > > > > > >>> 2. a vm with remote display that developers can use to > run/test > > > > > changes > > > > > > >>> easily. > > > > > > >>> 3. an actual osx box that developers can ssh into and > compile and > > > > > runa nd > > > > > > >>> test and remotely view/interact with like the windows vm. > > > > > > >>> 4. same for freebsd etc. > > > > > > >> We do not have this and I am pretty sure we never will (I can > only > > > > > hope the > > > > > > >> future proofs me wrong). Maybe we should be more honest and > state > > > > > that any > > > > > > > then i can't see how we could make it a requirement for > committing > > > > > that they > > > > > > > build/run there. > > > > > > > > > > > > Having them running a build is very different from having full > remote > > > > > shell > > > > > > access. E.g. osx build slaves available on TravisCi do not have > any > > > > > option to > > > > > > get shell access. > > > > > > > > > > > > Detecting a problem is the first step. A build test for these > > > *before* > > > > > > entering master would do this. It would still need fixing, but > that > > > is > > > > > the > > > > > > same we have right now. The difference really is where we detect > the > > > > > break > > > > > > and if we have it sitting in master or not. > > > > > > > > > > ok - i wasn't thinking github's travis but more an actual osx > machine > > > > > plugged > > > > > in somewhere we can remotely manage and use. :) i know travis will > be > > > very > > > > > limited. > > > > > > > > > > > > when people report issues with specific builds/platforms then > > > > > > > address them as needed. > > > > > > > > > > > > I can report the build being broken on osx, on aarch64, with the > > > debug > > > > > > profile enabled and also make check with cxx enabled right now. > That > > > are > > > > > 4 > > > > > > things broken in master just what i see today. > > > > > > > > > > arm64 - known. and this is a bit of a surprise from luajit. it's > > > > > technically > > > > > been broken like this forever. it wasn't some change we made. > somehow > > > > > luajit > > > > > started breaking at some point. the "we only allow you to use 27 > bits > > > out > > > > > of 64 > > > > > of a pointer" thing. > > > > > > > > > > so this is not something CI would magically stop as changes to the > > > > > dependencies > > > > > started making things crash there. at least i have personally > compiled > > > > > using a > > > > > luajit compiled myself on arm64 and it worked. this was a while > back. > > > > > > > > > > cxx bindings. indeed these are a problem to the point where i just > > > disable > > > > > them > > > > > in my build. i am not sure if this is good or bad though. it's > kind of > > > > > tired > > > > > with the eo and bindings work and as things change multiple > languages > > > need > > > > > to > > > > > adapt. > > > > > > > > > > as for osx - i have no info as to what is going on there or why. > :( i > > > don't > > > > > know even where the build breaks/logs etc. are. i do know > > > > > build.enlightenment.org and that basically seems ok. > > > > > > > > > > > >> platform we support (besides Linux on x86_64) has only been > > > tested at > > > > > some > > > > > > >> point in the past. > > > > > > > i actually cross-build for windows maybe every few weeks or > so, and > > > > > freebsd > > > > > > > maybe similarly too. i build on rpi3 (32bit) too regularly > enough. > > > > > > > > > > > > > > we haven't tested on every linux distro either... so should we > only > > > > > claim a > > > > > > > few distributions? i don't think we're being dishonest really. > > > > > releases do > > > > > > > get a lot more testing to see they work across os's. master > may not > > > > > get as > > > > > > > much until a release cycle. > > > > > > > > > > > > Yeah, and as I had the pleasure of handling these releases I can > > > tell you > > > > > > that dealing with them so late in the cycle is a nightmare. > Having a > > > > > smoother > > > > > > release cycle is actually one of my main motivations to have this > > > early > > > > > > detection and avoidance of breaks. > > > > > > > > > > indeed you are right - catching things early is far better. i > really > > > was > > > > > more > > > > > about us being honest about the releases working on those > platforms or > > > > > not. :) > > > > > that's all. you are right there and i totally agree with that. > > > > > > > > > > > >>> if a platform is on EASILY accessible and able to be EASILY > > > worked > > > > > with, > > > > > > >>> then making this a requirement to pass a build/test on that > > > platform > > > > > is > > > > > > >>> silly. > > > > > > >>> > > > > > > >>> developers have to be able to do incremental builds. not a > "wait > > > 10 > > > > > mins > > > > > > >>> for the next build cycle to happen then wait another 10 for > the > > > log > > > > > to > > > > > > >>> see if it worked this time". that's fine for reports. it's > not > > > fine > > > > > for > > > > > > >>> actual development. > > > > > > >>> > > > > > > >>> if this infra existed and worked well, THEN i think it might > be > > > sane > > > > > to > > > > > > >>> start adding "it has to pass a build test". then deciding on > what > > > > > > >>> platforms have to be supported is another step. this has to > be > > > > > pain-free > > > > > > >>> or as close as possible to that. > > > > > > >> Good luck to finding somehow setting this all up and keep it > > > working. > > > > > > >> Definitely not me. :-) If I look back how badly the idea of > > > having a > > > > > > >> windows vm, a mac and a arm device hooked up to Jenkins turned > > > out. I > > > > > > >> simply gave up on that part. > > > > > > > well then i just can't see us ever making it a requirement they > > > build > > > > > across > > > > > > > these os's on every single commit if it can't be automated and > made > > > > > > > available to developers to actually look into and see what is > > > working > > > > > or > > > > > > > not and why. :( > > > > > > > > > > > > > >>> not to mention... the test suites need to actually be > reliable. i > > > > > just > > > > > > >>> found one of the ecore_file tests was grabbing a page from > > > sf.net > > > > > ... and > > > > > > >>> now sf.net is refusing to servie it anymore thus test suites > > > keep > > > > > failing. > > > > > > >>> tests that are fragile like this should not be some > gatekeeper > > > as to > > > > > if > > > > > > >>> code goes in or not. > > > > > > >>> > > > > > > >>> if a test suite is to be a gatekeeper it has to be done > right. > > > that > > > > > means > > > > > > >>> it has to work reliably on the build host. run very quickly. > > > things > > > > > like > > > > > > >>> testing network fetches has to not rely on anything outside > of > > > that > > > > > > >>> vm/box/chroot etc. etc. ... and we don't have that situation. > > > this > > > > > > >>> probably needs to be fixed first and foremost. not to mention > > > just > > > > > > >>> dealing with check and our tests to find what went wrong is a > > > > > nightmare. > > > > > > >>> finding the test that goes wrong in a sea of output ... is > bad. > > > > > > >>> > > > > > > >>> so my take iis this: first work on the steps needed to get > the > > > final > > > > > > >>> outcome. better test infra. easier to write tests. easier to > run > > > and > > > > > find > > > > > > >>> just the test that failed and run it by itself easily etc. it > > > should > > > > > be as > > > > > > >>> simple as: > > > > > > >>> > > > > > > >>> make check > > > > > > >>> ... > > > > > > >>> FAIL: src/tests/some_test_binary > > > > > > >>> > > > > > > >>> and to test it i just copy & paste that binary/line and > nothing > > > more > > > > > and i > > > > > > >>> get exactly that one test that failed. i don't have to set > env > > > vars, > > > > > read > > > > > > >>> src code to find the test name and so on. ... it currently > is not > > > > > this > > > > > > >>> simple by far. ;( > > > > > > >> Yes, our tests are not as reliable as they should be. > > > > > > >> Yes, they would need to run in an controlled env. > > > > > > >> Yes, we might need so look at alternatives to libcheck. > > > > > > > i'm just saying these need to not randomly reject commits from > devs > > > > > when the > > > > > > > issue has nothing to do with what the dev is doing. it can't > > > become an > > > > > > > automated requirement unless its reliably correct. :( > > > > > > > > > > > > > >> But even with me agreeing to the three things above the core > > > question > > > > > stays > > > > > > >> still open. > > > > > > >> > > > > > > >> Is this developer community willing to accept a working test > > > suite as > > > > > a > > > > > > >> gatekeeper? I don't think this is the case. > > > > > > > i think it's best to make it an expectation that devs run make > > > check > > > > > and > > > > > > > compile against efl before a push > > > > > > > > > > > > That is an expectation that is failing in my reality for many > years > > > now. > > > > > > Getting people to run make check or even distcheck is a fight > against > > > > > > windmills. Before I normally can do the first beta release from > > > master > > > > > for a > > > > > > new cycle I need to do onion bug fixing by peeling off one bug > after > > > > > another > > > > > > to finally get some tarballs produced. > > > > > > > > > > i know that at least i run make check a few times per week and i am > > > doing > > > > > multiple builds from scratch per day including building against > efl. > > > > > > > > > > > > before even considering making it the > > > > > > > gatekeeper. we aren't even there yet with enough tooling, let > alone > > > > > talking > > > > > > > of an automated gatekeeper. just a reliable, easy to use and > > > complete > > > > > test > > > > > > > suite would be a big step forward. i think it's putting the > cart > > > > > before the > > > > > > > horse to talk automated gatekeepers + jenkins ... etc. without > > > getting > > > > > > > these first things right. > > > > > > > > > > > > The fun fact here is that most bogus reports actually come from > > > another > > > > > bugs > > > > > > that have been introduced before and are now shadowing new ones. > > > > > > > > > > > > We also have bogus reports due to our homebrewn infrastructure > and > > > > > Jenkins > > > > > > misconfigurations. > > > > > > > > > > that's not too great. :( > > > > > > > > > > > >> My personal motivation to work on QA and CI has gone down to > zero > > > > > over the > > > > > > >> years. It just feels like a Sisyphus task to look at master > again > > > and > > > > > again > > > > > > >> why it is broken now. Dig through the commits, bisect them, > point > > > > > fingers > > > > > > >> and constantly poke people top get it fixed. All long after > the > > > > > problem > > > > > > >> have entered master. > > > > > > > what you want is gerrit. and i've seen how that works. i'm not > a > > > fan. > > > > > it > > > > > > > ends up either: > > > > > > > > > > > > > > 1. it's ignored and patches sit in review for weeks, months or > > > years > > > > > and > > > > > > > vanish or > > > > > > > 2. it's gamed because everything has to go through it, it's > > > minimized > > > > > to try > > > > > > > and remove the impact. people just vote things up without real > > > review > > > > > etc. > > > > > > > > > > > > > > if you automated the voting to a build check instead of humans, > > > you'd > > > > > need > > > > > > > to find some way to do that and have a test build bot vote and > do > > > it > > > > > FAST. > > > > > > > that means you need enough infra for a build per commit and it > has > > > to > > > > > be > > > > > > > totally reliable. the build env and the tests. is that really > > > possible? > > > > > > > then if you don't do them in strict order you end up with > > > conflicts, > > > > > and if > > > > > > > some are rejected from a push with multiple commits, you have > > > dependent > > > > > > > commits... > > > > > > > > > > > > > > i don't see how this ends up better. it ends up worse i think. > > > > > > > > > > > > > >> I willing to admit that the approach I used to reach my goals > > > might > > > > > have > > > > > > >> been flawed and simply failed. Someone else might want to > pick up > > > the > > > > > > >> slack on it. > > > > > > > i really don't think we have a lot of break issues given size > and > > > > > > > complexity. not build breaks anyway. > > > > > > > > > > > > See the list above just for the four I see today. > > > > > > > > > > well i guess i disabled cxx bindings so i stop seeing those, but i > do > > > see > > > > > things buildiong cross-compile for windows and on freebsd as well > as > > > arm32 > > > > > regularly and don't run into build issues there. > > > > > > > > > > indeed your examples are for systems i don't have builds or build > infra > > > > > for. > > > > > > > > > > > > right now if jenkins detects a build break... how does > > > > > > > a developer know? it can take hours before it detects > something. > > > > > should they > > > > > > > sit hitting reload on the browser for the next hour hoping one > of > > > the > > > > > builds > > > > > > > going contains their commit? we have a broken feedback cycle. > > > jenkins > > > > > should > > > > > > > mail the mailing list with "commit X broke build: log etc. link > > > here". > > > > > if a > > > > > > > build breaks, jenkins should go back commits until it finds a > > > working > > > > > one > > > > > > > > > > > > Oh, now we even expect automated git bisecting from it? :-) > > > > > > > > > > i was even thinking a simple "roll back 1 at a time brute force > until > > > > > found" :) > > > > > nothing as fancy as a bisect! :) assuming > > > > > > > > > > > > then > > > > > > > report the commit that broke. at least devs would get a > > > notification. > > > > > > > > > > > > These mails are getting sent to the e-b0rk mailing list as I have > > > been > > > > > asked > > > > > > to not sent them to the main development list. > > > > > > > > > > i totally missed this list. i guess that's why i don't know. :) i > > > should > > > > > subscribe. i imagine the lack of knowing about this might be part > of > > > the > > > > > problem. > > > > > > > > > > > > unless i > > > > > > > they sit staring at build.e.org with reloads or someone tells > me, > > > i > > > > > just > > > > > > > have no idea a build would have broken. > > > > > > > > > > > > > > i think we've talked about this many times before... :) > > > > > > > > > > > > > > > > > > > We did and we both had many of the same arguments before. :-) > > > > > > > > > > > > I reached the point where I have no motivation left for doing > CI/QA > > > work, > > > > > > though. Thus I am going to drop it and hope someone else picks > it up. > > > > > > > > > > :( > > > > > > > > > > -- > > > > > ------------- Codito, ergo sum - "I code, therefore I am" > > > -------------- > > > > > Carsten Haitzler - ras...@rasterman.com > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > > Check out the vibrant tech community on one of the world's most > > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > > _______________________________________________ > > > > > enlightenment-devel mailing list > > > > > enlightenment-devel@lists.sourceforge.net > > > > > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > Check out the vibrant tech community on one of the world's most > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > _______________________________________________ > > > > enlightenment-devel mailing list > > > > enlightenment-devel@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > > > > > > > > > > > > > -- > > > ------------- Codito, ergo sum - "I code, therefore I am" > -------------- > > > Carsten Haitzler - ras...@rasterman.com > > > > > > > > > -- > ------------- Codito, ergo sum - "I code, therefore I am" -------------- > Carsten Haitzler - ras...@rasterman.com > > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
