Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Berenguer Blasi
The problem with using the first approach is that it fixes the current situation but it doesn't prevent it from happening again. The second proposal prevents it from happening again but at the cost of a bigger rename I'd volunteer to if needed. Regards On 24/10/22 20:38, Miklosovic, Stefan

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-24 Thread Derek Chen-Becker
This could also be a pipeline parameter instead of hacking it in generate.sh. I promise I'll have a proposal before the end of the week. Derek On Mon, Oct 24, 2022 at 2:13 PM Josh McKenzie wrote: > @Ekaterina: I recall us going back and forth on whether default should be > require approval or

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-24 Thread Josh McKenzie
@Ekaterina: I recall us going back and forth on whether default should be require approval or not and there not being a consensus. I'm fine not changing the status quo and just parameterizing that in generate.sh so folks can locally script how they want to setup when they alias up generate.sh.

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-24 Thread Andrés de la Peña
> > - Ticket for: remove -h, have -f and -p (free and paid) +1 to this, probably there isn't anyone using -h. There are some jobs that can't pass with the free option. Maybe we should remove them from the workflow when the free option is used. Perhaps that could save new contributors some

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-24 Thread Ekaterina Dimitrova
Seems like my email crashed with Andres’ one. My understanding is we will use the ticket CASSANDRA-17113 as placeholder, the work there will be rebased/reworked etc depending on what we agree with. I also agree with the other points he made. Sounds reasonable to me On Mon, 24 Oct 2022 at 15:03,

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-24 Thread Ekaterina Dimitrova
Thank you Josh So about push with/without a single click, I guess you mean to parameterize whether the step build needs approval or not? Pre-commit the new flag will use the “no-approval” version, but during development we still will be able to push the tests without immediately starting all

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-24 Thread Andrés de la Peña
> > Yep - instead of having to go to circle and click, when you push your > branch the circle hook picks it up and kicks off the top level job > automatically. I tend to be paranoid and push a lot of incremental work > that's not ready for CI remotely so it's not great for me, but I think > having

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Miklosovic, Stefan
Yeah, that is what the branch in my original email actually already solved. I mean ... CassandraAuthorizerTruncatingTests BatchTests UUIDTests these are ending on Tests which is illegal and they are fixed there. Other cases are either "TestBase" or "Tester" which should be legal. I think

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-24 Thread Josh McKenzie
> Auto-run on push? Can you elaborate? Yep - instead of having to go to circle and click, when you push your branch the circle hook picks it up and kicks off the top level job automatically. I tend to be paranoid and push a lot of incremental work that's not ready for CI remotely so it's not

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Derek Chen-Becker
OK, I think the approach you're proposing is fine. As an orthogonal idea, is there any way to do a one-time audit to narrow down files that we need to inspect, or any way to automate confirmation of a file containing tests? For example: * Start with all of the files under the `test` directory (is

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Miklosovic, Stefan
One more point as I was reading your email in a hurry. The ultimate goal is to run all tests. So even if we apply the proposed regexp in checkstyle and there is an unfortunate case that MyTestSplit1 would slip through on a review and it would not be executed in CI, if we do not figure out some

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Miklosovic, Stefan
We do not want to allow MyTestSplit1. That would be forbidden but current regexp would not catch it. There is a gap here I was talking about. This is the case of "test" in the middle which should be possible (DistributedTestBaseImpl) but then we need to fix cases like these and detect this

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Derek Chen-Becker
OK, that makes sense, and I missed the rename of "TestBaseImpl" (which I agree is in need of fixing). When you say: > What we are trying to achieve by that is to have a test which ends on "Test" and nothing else and it may contain "Test" only in case it does not start with it. Should the regex

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Miklosovic, Stefan
I meant MyFeatureTest -> DistributedTestBaseImpl -> DistributedTestBase TestBaseImpl would be renamed to DistributedTestBaseImpl DistributedTestBase is in dtest package (separate project / dependency) From: Miklosovic, Stefan Sent: Monday, October 24,

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Miklosovic, Stefan
Hi Derek, yes, the regexp is ^(?!(?:Test))(?!.*?(?:Tests)).*$ What we are trying to achieve by that is to have a test which ends on "Test" and nothing else and it may contain "Test" only in case it does not start with it. To your second paragraph, I do not think we have any *Base files which

Re: Some tests are never executed in CI due to their name

2022-10-24 Thread Derek Chen-Becker
For clarity, what is the final regex you've landed on for the first approach? Is it "^(?!(?:Test))(?!.*?(?:Tests)).*$" ? Are we just trying to reject anything that *starts* with "Test" or contains "Tests" somewhere in the name? It might be more clear to state what we think a valid test name should

Some tests are never executed in CI due to their name

2022-10-24 Thread Miklosovic, Stefan
Hi list, it was reported that some tests are not executed as part of CI because of their name (1). There is (2) and (3) so right now it runs only "*Test.java" files. Except of us renaming the affected classes, we should fix this in a more robust way - via checkstyle - so we enforce how test