>
> - Ticket for: remove -h, have -f and -p (free and paid)
+1 to this, probably there isn't anyone using -h. There are some jobs that
can't pass with the free option. Maybe we should remove them from the
workflow when the free option is used. Perhaps that could save new
contributors some confusion. Or should we leave them because a subset of
the tests inside those jobs can still pass even with the free tier?
By the way, the generate.sh script already accepts a -f flag. It's used to
stop checking that the specified environment variables are known. It was
meant to be a kind of general "--force" flag.
On Mon, 24 Oct 2022 at 20:07, Ekaterina Dimitrova <e.dimitr...@gmail.com>
wrote:
> Seems like my email crashed with Andres’ one.
> My understanding is we will use the ticket CASSANDRA-17113 as
> placeholder, the work there will be rebased/reworked etc depending on what
> we agree with.
> I also agree with the other points he made. Sounds reasonable to me
>
> On Mon, 24 Oct 2022 at 15:03, Ekaterina Dimitrova <e.dimitr...@gmail.com>
> wrote:
>
>> Thank you Josh
>>
>> So about push with/without a single click, I guess you mean to
>> parameterize whether the step build needs approval or not? Pre-commit the
>> new flag will use the “no-approval” version, but during development we
>> still will be able to push the tests without immediately starting all
>> tests, right?
>> - parallelism + -h being removed - just to confirm, that means we will
>> not use xlarge containers. As David confirmed, this is not needed for all
>> jibs and it is important as otherwise whoever uses paid account will burn
>> their credits time faster for very similar duration runs.
>>
>> CASSANDRA-17930 - I will use the opportunity also to mention that many of
>> the identified missing jobs in CircleCI will be soon there - Andres is
>> working on all variations unit tests, I am doing final testing on fixing
>> the Python upgrade tests (we weren’t using the right parameters and running
>> way more jobs then we should) and Derek is looking into the rest of the
>> Python test. I still need to check whether we need something regarding
>> in-jvm etc, the simulator ones are running only for jdk8 for now,
>> confirmed. All this should unblock us to be able to do next releases based
>> on CircleCI as we agreed. Then we move to do some
>> changes/additions/improvements to Jenkins. And of course, the future
>> improvements we agreed on.
>>
>> On Mon, 24 Oct 2022 at 14:10, Josh McKenzie <jmcken...@apache.org> wrote:
>>
>>> Auto-run on push? Can you elaborate?
>>>
>>> Yep - instead of having to go to circle and click, when you push your
>>> branch the circle hook picks it up and kicks off the top level job
>>> automatically. I tend to be paranoid and push a lot of incremental work
>>> that's not ready for CI remotely so it's not great for me, but I think
>>> having it be optional is the Right Thing.
>>>
>>> So here's the outstanding work I've distilled from this thread:
>>> - Create an epic for circleci improvement work (we have a lot of little
>>> augments to do here; keep it organized and try and avoid redundancy)
>>> - Include CASSANDRA-17600 in epic umbrella
>>> - Include CASSANDRA-17930 in epic umbrella
>>> - Ticket to tune parallelism per job
>>> -
>>> > def java_parallelism(src_dir, kind, num_file_in_worker, include =
>>> lambda a, b: True):
>>> > d = os.path.join(src_dir, 'test', kind)
>>> > num_files = 0
>>> > for root, dirs, files in os.walk(d):
>>> > for f in files:
>>> > if f.endswith('Test.java') and
>>> include(os.path.join(root, f), f):
>>> > num_files += 1
>>> > return math.floor(num_files / num_file_in_worker)
>>> >
>>> > def fix_parallelism(args, contents):
>>> > jobs = contents['jobs']
>>> >
>>> > unit_parallelism = java_parallelism(args.src,
>>> 'unit', 20)
>>> > jvm_dtest_parallelism = java_parallelism(args.src,
>>> 'distributed', 4, lambda full, name: 'upgrade' not in full)
>>> > jvm_dtest_upgrade_parallelism = java_parallelism(args.src,
>>> 'distributed', 2, lambda full, name: 'upgrade' in full)
>>> - `TL;DR - I find all test files we are going to run, and based off
>>> a pre-defined variable that says “idea” number of files per worker, I then
>>> calculate how many workers we need. So unit tests are num_files / 20 ~= 35
>>> workers. Can I be “smarter” by knowing which files have higher cost?
>>> Sure… but the “perfect” and the “average” are too similar that it wasn’t
>>> worth it...`
>>> - Ticket to combine pre-commit jobs into 1 pipeline for all JDK's
>>> - Path to activate all supported JDK's for pre-commit at root
>>> (one-click pre-merge full validation)
>>> - Path to activate per JDK below that (interim work partial
>>> validation)
>>> - Ticket to rename jobs in circleci
>>> - Reference comment:
>>> https://issues.apache.org/jira/browse/CASSANDRA-17939?focusedCommentId=17617016&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17617016
>>> - (buildjdk)_(runjdk)_(testsuite) format:
>>> - j8_j8_jvm_dtests
>>> - j8_j11_jvm_dtests
>>> - j11_j11_jvm_dtest_vnode
>>> etc
>>> - Ticket for flag in generate.sh to support auto run on push (see
>>> response above)
>>> - Ticket for: remove -h, have -f and -p (free and paid) (probably
>>> intersects with https://issues.apache.org/jira/browse/CASSANDRA-17600)
>>>
>>> Anything wrong w/the above or anything missed? If not, I'll go do some
>>> JIRA'ing.
>>>
>>>
>>> ~Josh
>>>
>>>
>>> On Fri, Oct 21, 2022, at 3:50 PM, Josh McKenzie wrote:
>>>
>>> I am cool with removing circle if apache CI is stable and works, we do
>>> need to solve the non-committer issue but would argue that partially exists
>>> in circle today (you can be a non-commuter with a paid account, but you
>>> can’t be a non-committer with a free account)
>>>
>>> There's a few threads here:
>>> 1. non-committers should be able to run ci
>>> 2. People that have resources and want to run ci faster should be able
>>> to do so (assuming the ci of record could serve to be faster)
>>> 3. ci should be stable
>>>
>>> Thus far we haven't landed on 1 system that satisfies all 3. There's
>>> some background discussions brainstorming how to get there; when / if
>>> things come from that they'll as always be brought to the list for
>>> discussion.
>>>
>>> On Fri, Oct 21, 2022, at 1:44 PM, Ekaterina Dimitrova wrote:
>>>
>>> I agree with David with one caveat - last time I checked only some
>>> Python tests lack enough resources with the free tier. The rest run slower
>>> than with a paid account, but they do fine. In fact I use the free tier if
>>> I want to test only unit or in-jvm tests sometimes. I guess that is what he
>>> meant by partially but even being able to run the non-Python tests is a win
>>> IMHO. If we find a solution for all tests though… even better.
>>> @Derek your idea sounds interesting, I will be happy to see a proposal.
>>> Thank you
>>>
>>> On Fri, 21 Oct 2022 at 13:39, David Capwell <dcapw...@apple.com> wrote:
>>>
>>> I am cool with removing circle if apache CI is stable and works, we do
>>> need to solve the non-committer issue but would argue that partially exists
>>> in circle today (you can be a non-commuter with a paid account, but you
>>> can’t be a non-committer with a free account)
>>>
>>>
>>>
>>> On Oct 20, 2022, at 2:20 PM, Josh McKenzie <jmcken...@apache.org> wrote:
>>>
>>> I believe it's original intention to be just about CircleCI.
>>>
>>> It was but fwiw I'm good w/us exploring adjacent things regarding CI
>>> here. I'm planning on deep diving on the thread tomorrow and distilling a
>>> snapshot of the work we have a consensus on for circle and summarizing here
>>> so we don't lose that. Seems like it's fairly non-controversial.
>>>
>>> On Thu, Oct 20, 2022, at 5:14 PM, Mick Semb Wever wrote:
>>>
>>>
>>>
>>> On Thu, 20 Oct 2022 at 22:07, Derek Chen-Becker <de...@chen-becker.org>
>>> wrote:
>>>
>>> Would the preclusion of non-committers also prevent us from configuring
>>> Jenkins to auto-test on PR independent of who opens it?
>>>
>>> One of my current concerns is that we're maintaining 2x the CI for 1x
>>> the benefit, and I don't currently see an easy way to unify them (perhaps a
>>> lack of imagination?). I know there's a long history behind the choice of
>>> CircleCI, so I'm not trying to be hand-wavy about all of the thought that
>>> went into that decision, but that decision has costs beyond just a paid
>>> CircleCI account. My long term, probably naive, goals for CI would be to:
>>>
>>> 1. Have a CI system that is *fully* available to *any* contributor,
>>> modulo safeguards to prevent abuse
>>>
>>>
>>>
>>> This thread is going off-topic, as I believe it's original intention to
>>> be just about CircleCI.
>>>
>>> But on your point… our community CI won't be allowed (by ASF), nor have
>>> capacity (limited donated resources), to run pre-commit testing by anyone
>>> and everyone.
>>>
>>> Today, trusted contributors can be handed tokens to ci-cassandra.a.o
>>> (make sure to label them so they can be revoked easily), but we still face
>>> the issue that too many pre-commit runs impacts the throughput and quality
>>> of the post-commit runs (though this has improved recently).
>>>
>>> It's on my wishlist to be able to: with a single command line; spin up
>>> the ci-cassandra.a.o stack on any k8s cluster, run any git sha through it
>>> and collect results, and tear it down. Variations on this would solve
>>> non-committers being able to repeat, use, and work on their own (or a
>>> separately donated) CI system, and folk/companies with money to be able to
>>> run their own ci-cassandra.a.o stacks for faster pre-commit turnaround
>>> time. Having this reproducibility of the CI system would make testing
>>> changes to it easier as well, so I'd expect a positive feedback loop here.
>>>
>>> I have some rough ideas on how to get started on this, if anyone would
>>> like to buddy up on it.
>>>
>>>
>>>
>>>
