[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17778234#comment-17778234 ] ASF subversion and git services commented on NIFI-12178: Commit ed3a70c9859f7cccdf03c6133f05d42ab108cd25 in nifi's branch refs/heads/main from Chris Sampson [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ed3a70c985 ] NIFI-12178 Add integration-tests and docker-tests GitHub Workflows Use Java Zulu distribution for all docker-tests and integration-tests Github Workflows This closes #7858 Signed-off-by: David Handermann > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > Time Spent: 4h 10m > Remaining Estimate: 0h > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773145#comment-17773145 ] Chris Sampson commented on NIFI-12178: -- I've set the new tests to run daily, like system tests, and on update to specific files. But the frequency can certainly be adjusted. I've been trying to run tests during development in my personal account (hoping not to reach any limits), so as to reduce the overall apache load there too. We could set the frequency to weekly, then increase later if that would be a more sensible cadence? > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773068#comment-17773068 ] Otto Fowler commented on NIFI-12178: I think this stuff is great. The only thing I will add for consideration is that Apache's GitHub resources are not infinite, ( join the build list for the repetitive discussions on these limits as some projects exhaust them for all the others ) and we should be cognizant of that. > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772792#comment-17772792 ] Chris Sampson commented on NIFI-12178: -- Thanks [~markap14], useful to know as the AWS bundle ITs are definitely seeing a lot of failures. I've discussed with [~jgresock] before about a possible move to using Localstack via Testcontainers or similar, maybe how they've been integrated into processors such as Elasticsearch and Mongo. > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772642#comment-17772642 ] Mark Payne commented on NIFI-12178: --- I've been working on some of this as well, [~Chris S] , as I'm currently working on eliminating all of the deprecation warnings in AWS Processors. As part of that, I need to verify that all of the components still work, so I'm replacing all of the Integration Tests with the Localstack Test Container. It's been extremely helpful. So just wanted to mention this, to avoid duplicating any effort. > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772591#comment-17772591 ] David Handermann commented on NIFI-12178: - Thanks for the summary [~Chris S], sounds like you have made a lot of progress thus far! Filtering out the failing test classes sounds like a good place to start. As you noted, it provides a useful list for follow-on efforts. I agree this will be very helpful in maintaining meaningful integration tests. > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772587#comment-17772587 ] Chris Sampson commented on NIFI-12178: -- [~exceptionfactory] I started off by trying to run all of the tests/builds together, but have separated them into their own workflows ({{integration-tests}} and {{docker-tests}}) now due to disk space issues more than anything else. My approach currently is: * build and test Docker Images (which is working for an Ubuntu runner, but not MacOS as that doesn't come with Docker installed - we can install Colima but the {{nifi-docker}} tests fail, which I propose to be handled by NIFI-12177 in future) * build and run all {{integration-tests}} that aren't already covered by the existing {{system-tests}} workflow ** I've identified a lot of failures/errors already, but rather than trying to fix them (other than general issues like not having Docker running), I'm filtering out the {{IT}} classes from the [failsafe plugin|https://maven.apache.org/surefire/maven-failsafe-plugin/examples/single-test.html#multiple-formats-in-one] execution ** this will give us a list of failing test classes and/or methods (some classes have tests that pass but others are failing), which we can look to diagnose and fix as part of a follow-on ticket in the future This feels like a sensible/iterative approach to getting something running in GitHub to prevent deterioration of existing ITs, capture any new ITs added in future, and allow us to find and remove/fix existing failing ITs already in the repository. Open to suggestions on a different approach if you/anyone has other ideas (ongoing trials can be seen in my fork at https://github.com/ChrisSamo632/nifi/actions) > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12178) Add integration-tests and docker-tests GithHub Action Workflows
[ https://issues.apache.org/jira/browse/NIFI-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772579#comment-17772579 ] David Handermann commented on NIFI-12178: - Thanks for taking on this effort [~Chris S], I think this would be a very helpful improvement. Handling integration tests may be somewhat challenging, as some work on their own with or without Testcontainers, but others require manual adjust for credentials. This could be a good opportunity to remove some stale integration tests. It may be worth separating the effort into Docker workflow builds and integration test builds depending on the types of changes required. JUnit 5 also provides some new annotations for conditional enabling based on environment variables or system properties. This could be used to filter and select integration tests that do not need any additional configuration or external properties. This could also support adjusting some integration tests to use system properties instead of relying on manual changes to set properties like credentials and URLs. > Add integration-tests and docker-tests GithHub Action Workflows > --- > > Key: NIFI-12178 > URL: https://issues.apache.org/jira/browse/NIFI-12178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.latest > > > There are a lot of {{integration-tests}} and {{docker}} builds in the NiFi > repository that aren't being regularly executed and verified. > GitHub Action Workflows should be added (similar to the existing > {{system-tests}} Workflow) that regularly exercise these tests, allowing the > community to better monitor the health of NiFi components. -- This message was sent by Atlassian Jira (v8.20.10#820010)