On Thu, 2 Apr 2020 at 20:49, Adam Williamson <adamw...@fedoraproject.org> wrote:
> On Thu, 2020-04-02 at 11:52 +0100, Leigh Griffin wrote: > > [up front: apologies for any weird formatting in this email, Evolution > is crashing on me like crazy while I try to edit it, so I'm sending it > from Roundcube which I don't normally do] > > > I'm also not entirely sure about Adam Saleh, but he does not have any > > > infra activity I can find since the end of January and lists himself as > > > working for Exponea on Github and his personal blog. > > > > > > > Adam is working in CPE. CPE isn't entirely about Infra, he is working > > on CI > > improvements at the moment alongside some others > > Okay, well, next time I see him I'll mention he might want to update the > Exponea references :P > > But "CPE isn't entirely about Infra" is sort of the point here: my > initial assertion was that there are fewer people working on > *Fedora-relevant app development* than there were before. I'm happy to > accept that there may be more people in the CPE team than there were at > some point in its past, but that's not what I'm actually interested in > here. CI is great (whichever CI you mean, I lose track sometimes :>) but > someone working on CI is not someone working on the Fedora app > infrastructure, or on sourcing/deploying/integrating/maintaining > outsourced replacements for it, which is the actual problem space here. > > > > If I'm missing anyone, please do correct me. > > > > Other developers in our pool right now are: > > - Ryan Lerch > > I mentioned already that Ryan could go in either list or neither - he > was around at both times but I wasn't sure whether to count him as a > developer or not. But he can't count to one list but not the other, as > he's been here all along. :) > > > - Michal Konecny > > - Mohan Boddu > > - Tomas Hrcka > > - Nils Philippsen > > - Vipul Siddharth > > - James Richardson > > > > We additionally have 2 new Ops folks joining us over the next 2 weeks. > > > > Across them, the majority are working on the Fedora aspects (Infra, > > Dev, > > Releng) of the CPE remit. > > So yeah, let's discount the releng folks first, because releng has > existed all along, and - as I said - my original statement was not about > "people who are organizationally in the same team as the people who work > on Fedora app stuff" but "people who work on Fedora app stuff". So that > lets out Mohan and Tomas. > I don't think this is fair at all, Tomas and Mohan are doing a lot of development and trying to improve a lot of the tooling around releng so that we move from a manual heavy process to a more automated or at least tool assisted process. If you consider that releng tooling is not application work, then please explain to me what is bodhi, release-monitoring, anitya, mdapi etc ... It seems to me that these services are 100% release engineering focused. > > As for the others: so, I didn't count Michal at first as I could not > find any infra-related activity for him, but since you included him I > looked closer, and found he's just hiding himself really well - his > github account name, for some inaccountable reason, is "Zlopez", and his > profile doesn't have his real name in it. So I finally got his logs: > > https://github.com/Zlopez > https://pagure.io/user/zlopez > > ...and yeah, there's some infra activity there, so add him to the list. > > Initially I was just looking at Github logs, as all the infra stuff I > could find was hosted there, and Nils' list is: > > https://github.com/nphilipp > > which was busy up to the end of December but looked extremely empty all > of this year, so I figured he'd switched track or role or something and > didn't count him. But now I look closer, it seems what happened is he's > been working almost exclusively on a thing called rpmautospec, which is > hosted on Pagure: > > https://pagure.io/Fedora-Infra/rpmautospec > > so, he's clearly working hard on something, although I'm not sure it's > directly a part of Fedora app/infra stuff - it's an automatic packaging > thingy, it looks like, I'm not sure what it's being used for. In fact, > it seems like pingou and Adam Saleh are doing quite a bit of work on > this thing too, so it's clearly sucking up quite a lot of developer > time. It's also a fairly new project, which seems interesting given that > "we don't have time to maintain all the projects we have already" is the > recurring refrain here. > This is an effort to improve the packager workflow in order to automate the generation of spec file changelogs and release field. Indeed it is a new project but we considered that improving and automating the packager workflow is something worth investing time in and creating new projects. Here's Vipul's lists: > > https://github.com/siddharthvipul > https://pagure.io/user/siddharthvipul1 > > he seems to work exclusively on CentOS CI. Okay, Fedora *uses* CentOS > CI, but presumably back in the 2018 timeframe, someone (whether that's > Vipul or someone else) was working on CentOS CI who wasn't included in > my list, because I only listed people working on Fedora stuff. So this > still seems like a wash. > Vipul and I have done extensive work to add the OSBS aarch64 cluster in staging, and it might comes as a surprise but yes we are working as a team and even sometimes pair programing and sharing knowledge. But you will find some of Vipul's contribution on the ansible repo git logs. Also this work is directly coming as a request from the council to support the IOT objective, so I think it is fair to count it as "development" work even if this was mostly operation work and deploying a new OpenShift cluster for OSBS, since this time could have been spend on other application if that was asked by the council. > > James I didn't count as he's listed as an intern. But here's his Github > log (I can't find him on Pagure): > > https://github.com/james02135 > > so I don't mean this as a knock at all, but he's obviously not > equivalent to one regular full-time dev, nor would we (I hope) expect > him to be. > > So if we include Ryan on both lists and add Michal, we get to this. > Previous: > > puiterwijk > Randy (bowlofeggs) > pingou > jcline > Ralph > jflory > abompard > rlerch > > Current: > > abompard > lrossetti/odra > pingou > scoady > cverna > mkonecny (Zlopez) > > If we also count rpmautospec, we add Nils and Adam to the current list: > > abompard > lrossetti/odra > pingou > scoady > cverna > mkonecny (Zlopez) > nphilipp > asaleh > > and it looks like it's about even. But that's counting rpmautospec, > which seems to be an odd counterpoint to the overall CPE narrative that > "we have too many projects and we're trying to get rid of the > maintenance burden" - you already don't have resources to maintain the > things that Fedora very definitely needs and is using right now, but you > *do* have resources to dedicate some or all of the time of three > engineers to a brand new project which certainly looks cool but is > definitely a new invented-here thing that isn't absolutely essential to > Fedora's needs and isn't replacing an existing project? > > Yeah we are even, and we have 2 new persons joining the team next week with a more sysadmin/operation profile because we really need to support nirik and smooge in that area. I think what you are failing to see is that for roughly the same team, there is much more thing to do. The project keeps adding new things, we now have containers, flatpaks, IoT, silverblue, CoreOS and this is good thing but it adds more work on the team for example the releng work that was needed 5 years ago has now triple, same thing on the infra side. On the application development, tools and application have to be adapted to take care of these new deliverables. I am pretty sure you know this very well because that must impact you also on the QA side. Also application development requires more effort as the time passes, working on legacy code base is really time consuming, for example after completing the work on Bodhi for rawhide gating not much time went into Bodhi's development for a few months, when I started to look back at the project to prepare a new release I easily spent a week just fixing the integration tests,. > To be really clear: I don't want the takeaway from this to be "Adam is > very mad and doesn't want CPE to be allowed to work on cool new projects > any more". I like cool new projects! Cool new projects are great! I > write them myself sometimes! I'm just having trouble joining up the dots > here in terms of high-level strategy. > We are really starting with prioritization and trying to be better at how we organize our work, so yeah maybe not every is great, but I did not know that perfection was the minimum required. We also share a weekly status update of every that is going on and welcome feedback and comments. Also if you think that we are really on the wrong tracks please feel free to reach out to the council or directly to Aoife which is working really hard to try to make sense of all the requests and needs that arrives at our door. I think it is important to note that a lot of people in the team are quite new to being part of a community. Honestly I don't think we are giving them a really good experience. But what strike me the most is the little trust there is in our team. When a group of people is working as hard as the folks in the CPE team, (some being around during the weekends or waking up in the middle of the night to restart a service) is telling that they cannot sustain the amount of work they have, there is so little trust in that team that we have to go and check the commit logs of these people to see on what they are working and if this time is well invested. Honestly I have no words, and no motivation to go and fix any of the tickets that are waiting to be fixed. > > The number of active developers on Fedora initiatives has gone up > > drastically since I joined the team in 2019. You are possibly not > > seeing > > that as the team have moved from a model of siloed work on multiple > > apps, > > swimming against the tid working 16 hour days, to working on team > > oriented > > initiatives to add real value to the ecosystem. So the noise of working > > on > > multiple small things at once is not as loud as it was in 2018 which is > > giving that illusion. > > What I'm looking at is the commit logs. That's all that ultimately > matters. But see above revisions, of course > . > > [snip the stuff about whether we need elections apps and blahblah > because I don't have anything to add there] > > > > > Now when the CPE team goes and ask for more people because we > struggle > > > with > > > > current situation, I can only guess that these non critical > applications > > > > are mentioned. If I was putting my own money to sponsor a team to > help > > > > building a Linux distribution I would be asking why do we have to > > > develop a > > > > calendar application or why do we need a custom git forge. I > personally > > > > find really great that the different use cases and requirements for > the > > > use > > > > of Pagure were gathered and I am convinced that people working on > this > > > did > > > > their very best to find a use case to justify the investment needed > in > > > > Pagure but it seems that we don't have such a thing. > > > > > > I think other people are following up on this, but from following the > > > discussion, it appears to me that there seems to have been a large slip > > > somewhere along the line from Ben submitting a list of ~50 Fedora > > > submissions, to Leigh suggesting that (after those were, mysteriously, > > > "summarized" somehow) > > > > Can you help me understand what the mystery is about? We took in 300+ > > requirements that we distilled down into the generic list that we came > > up > > with, many of which are buckets / Epics. Every single requirement was > > analysed. I have said this multiple times but please, if this is still > > a > > mystery to you, let me know how I can help clarify it. > > The specific gap I am talking about is the gap between this list > submitted by Ben Cotton on behalf of Fedora: > > > https://lists.fedoraproject.org/archives/list/council-disc...@lists.fedoraproject.org/thread/OEPDGVKYAJIQ6YYZU5J3XT3LHXFROFI5/ > > and the 'summarized' list you have pointed to here: > > https://hackmd.io/@My419-DISUGgo4gcmO1D6A/HJB74lcL8 > > Now, right off the bat, we have a huge problem. The "summarized" list > claims this: > > "after duplications and similar in theme requirements were merged > together, we were left with the following unique User Story list:" > > you've also phrased the same thing slightly differently on the mailing > list: > > "As a team we evaluated every single requirement (over 300 of them) and > the presentation in the combined User Story list is an amalgamation of > like for like User Stories to capture at a high level the spirit of the > requests." > > However, the top three points on the Fedora list relate to F/OSS and > self-hosting principles. These points are entirely missing from the > "summarized" list. They have not been "merged" with "duplications" or > "similar in theme requirements". The "spirit" of them has not been > "captured" in a "User Story". They are just *not there*. They have not > been summarized. They were dropped. So the claim is false, and there was > not just a summarization process here, there was some sort of filtering > process. Things from the stakeholder lists were outright left out of the > "summarization". I'll pick this up again later. > > Of the other 44 points on the Fedora list, 23 mention dist-git > specifically. The term "dist-git" is mentioned 40 times. Some of these > aren't truly "dist-git" specific, and are really just generic git or > forge requirements, but many of them *are* specific to dist-git and the > packager workflow integration; I'd count at least points 9, 17, 18, 19, > 20, 22, 23, 24, 25, 26, 28, 29, and maybe 37 in this bucket. In the > "summarized" list, the string "dist-git" does not appear once. I can buy > that a few of the requirements are *possibly* "summarized" into the > points about access control, but generic access control requirements do > not at all cover all the details of dist-git integration. Effectively, > all the details relating to dist-git and packager workflow interaction - > which constitute the large majority of significant requirements on the > Fedora list that wouldn't be satisfied by all of the candidates - are > lost in the summarized list. > > You say that all the (original 300+) requirements were analyzed, but > when you have been asked for specific details on any of the issues > relating to dist-git / packager workflow integration, these are the > kinds of answers you have given: > > "The Fedora specific requirements (as I am on the Fedora lists here) had > very few unique needs such that both Gitlab and Pagure would have > satisfied their desire." > > If you read the two lists above and my notes, I do not see how you can > possibly claim that Fedora "had very few unique needs". This was the > quote that first got me really worried that Fedora's needs had not been > properly considered and understood. (We can also note, of course, that > while both Gitlab CE and Pagure can potentially satisfy the "open > source" and "self-hosted" requirements, Gitlab Ultimate - to which the > decision appears to be weighted, a suggestion no-one has yet denied, > beyond a very mealy-mouth and deniable "no option is off the table yet" > - does not satisfy either). You later made more or less the same claim > again: > > "The needs of Fedora, CentOS, RHEL and the CPE team were weighed equally > in our decision. Fedora had very few specific needs in their > requirements where as some stakeholder groups had." > > which only makes me worry more. > > [to a question about Bugzilla package workflow integration] "I don't > have an answer to this as we haven't done that deep level of tooling > analysis and integration analysis yet." > > How can you have considered Fedora's requirements with regard to tooling > and integration if you haven't "done a deep level of tooling analysis > and integration analysis yet"? > > [when Till attempted to make similar points about the dist-git > requirements] "The User Stories are deliberately vague and that > represents around 10 unique requirements that boil down to having group > membership and management capabilities." > > "Having group memembership and management capabilities" is nowhere close > to covering the dist-git integration requirements. Pagure had these more > or less from day one, but we still had to spend months on engineering > the dist-git integration. You cannot count generic group membership and > ACL capabilities as "covering" these requirements, and if you did, that > was a fundamental misunderstanding of what Fedora was requesting. > > [when asked if you know how many existing integrations with Pagure there > are] "No. Our analysis will tell us that in due course" > > Once again - if you don't even know that, how can you possibly have > fully evaluated Fedora's detailed requirements regarding dist-git / > packager workflow integration? You defended this by saying "I do know of > many important integrations that I come into contact with during my day > to day job in the team, that doesn't mean I know every single one of > them. We analysed the critical path needs via the requirements and any > reference to tooling integrations (e.g. fedora messaging was reference > in multiple conversations) will be brought forward with us for deeper > technical conversations", but the extent to which the detail in Fedora's > list is lost in the "summarized" list gives me (and I suspect others) no > confidence at all that this analysis was done correctly. > > To put it slightly more generally: I think Fedora as a whole would have > expected that CPE was *already* deeply familiar with the importance of > dist-git / packager workflow integration. That CPE would *already* > understand that this - along with F/OSS principles - would be the meat > of Fedora's "unique" requirements in this process. And that the rolling > of these into "user stories" would be a useful technique for > facilitating the process, but not the Holy Grail of the process outside > of which nothing would matter at all; I would have expected the > knowledge that (for instance) pingou already has about this stuff to > have just been included directly in the decision-making process. > > However, as you've described it, that doesn't seem to have been what's > happened. Instead, this deep base of existing knowledge within your team > seems to have been sidelined in the decision process, and instead the > decision has been made based mostly or entirely on this apparently > somewhat wobbly process of bundling up and "summarizing" requirements > (through three levels of the telephone game, from Fedora lists to the > Council to this "summarized" list) as "User Stories". Or at least, that > is how you are presenting the process as having worked and the decision > as having been made, and the focus on generic forge-type requirements > and the repeated contention that Fedora had "very few" "unique" > requirements reinforces that impression. > > On a general note, I'm really having trouble understanding the overall > logic behind this "requirements" process at all. You've mentioned there > were over 300 requirements from the stakeholders and you summarized them > into this list. But when people have pressed you on individual items in > the summarized list, you've said stuff like this: > > [on private PR comments] "It is a valid requirement from a stakeholder > that we had to take at face value." > > OK, but why did *this* requirement make the "summarized" list when > others - as I've mentioned above - didn't? I am discounting the claim > that the list somehow summarizes all the given requirements, because it > very clearly and inarguably does not. You had to "take it at face > value", but there were 300 other requirements that had to be taken "at > face value" as well, and not all of them made the list. As that's true, > you can't duck Neal's question: why did *this specific request*, which > Neal gives various reasons for not considering a great priority, make > the summarized list? You can say "we think you're wrong that it's not a > very important request, we think it is an important request for reason > X", but you can't just duck the question by saying there was no choice > made that can be questioned. Clearly there *was*. > > [on the mobile app requirement] "It's a requirement that came from the > Fedora Community and one we could not satisfy as soon as Github was > ruled out. I do agree the experience is not great on it." > > Again, given the context that there clearly was some sort of filtering > process here, this reads as very bizarre. A bells-and-whistles feature > requirement that you acknowledge only Github barely fulfils, badly, > makes the summarized list, but "we want it to be F/OSS" and multiple > very specific fundamental dist-git integration requirements somehow > don't make the summarized list or are garbled beyond recognition? > > [on gists] "It's not up to me to gauge the merit of that as a use case > but it's a valid requirement which we considered." > > But the question is why *this* item from the list of 300+ made the > "summarized" list rather than being smooshed into something very vaguely > related, or dropped entirely. Did someone *else* gauge the merit of that > as a use case? Or did you make the decision on some other grounds? > > You have also implied there was some kind of "weighting" involved, when > asked if a requirement that "Github" be at the top of the UI would have > been accepted: > > "Would it be weighed equally with a more core functional requirement? > The answer is of course no but we would > have respected that request." > > but you have not provided any further detail on exactly how this > "weighting" worked and how Fedora's requirements (those which were not, > apparently, entirely dropped) were "weighted". How was Fedora's "we want > it to be open source" "weighed" against requests that cannot currently > be satisfied by an open source choice, if it was considered at all? > > I will also make a side note: it was claimed earlier in this thread that > the "mobile app" requirement "came from Fedora", but there is a rather > interesting discrepancy there. The Fedora requirement reads: > > "As a Fedora contributor, I can perform issue and pull request actions > on mobile devices via a native mobile app or a mobile-friendly webapp so > that I can contribute while away from my desk." > > The "summarized" version reads: > > "As a General User > I want a mobile native app > To allow me contribute while away from my desk" > > somehow the "or a mobile-friendly webapp" bit got lost. (And the > specific actions). > -- > Adam Williamson > Fedora QA Community Monkey > IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net > http://www.happyassassin.net > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org >
_______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
