[
https://issues.apache.org/jira/browse/NIFI-12190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-12190:
---------------------------------
Description:
NIFI-12177 adds an {{integration-tests}} GitHub Actions Workflow to regularly
run the {{integration-tests}} defined throughout the NiFi repo. However, many
tests have had to be skipped from the {{maven-failsafe plugin}} execution due
to failures within the GitHub runner.
Typically these are caused by:
* the need for external resources to be setup
* incorrect assertions that might have worked during the original IT creation
but have not been updated since
* or missing/incompatible OS resources within the GitHub runner
* missing/misconfigured dependencies needed to execute the integration tests
This Epic is to review the failing ITs within NiFi codebase and either:
* fix them, e.g. correct assertions, and/or by using
[Testcontainers|https://testcontainers.com/] for hosting external resources,
such as for the Elasticsearch Processors
* remove them if no longer useful/cannot be reliably executed on
developer's/GitHub runners
* annotate them to be skipped in CI environments if the tests are useful and
work on a developer machine but won't be able to work in a CI runner, e.g. a
[DisabledIfEnvironmentVariable|https://junit.org/junit5/docs/5.7.0/api/org.junit.jupiter.api/org/junit/jupiter/api/condition/DisabledIfEnvironmentVariable.html]
to check whether the {{CI}} env var is present and set to {{true}}
Once fixed, the appropriate entry from the top-level {{pom.xml}}'s
{{integration-tests-ci}} profile configuration for the
{{maven-failsafe-plugin}} **must** be removed.
At the time of writing, the ITs known to be failing (either in part or as a
whole) are:
{quote}
!AwsSecretsManagerSensitivePropertyProviderIT,
!AwsKmsSensitivePropertyProviderIT,
!AzureKeyVaultKeySensitivePropertyProviderIT,
!GcpKmsSensitivePropertyProviderIT,
!StandardHashiCorpVaultCommunicationServiceIT,
!ConversionWithSchemaInferenceIT#testXmlToAvro,
!ConversionWithSchemaInferenceIT#testXmlToJson,
!ConversionWithSchemaInferenceIT#testJsonToCsv,
!ConversionWithSchemaInferenceIT#testCsvToXml,
!ConversionWithSchemaInferenceIT#testXmlToAvroToXml,
!ConversionWithSchemaInferenceIT#testCsvToAvro,
!ConversionWithSchemaInferenceIT#testCsvToJson,
!ConversionWithSchemaInferenceIT#testJsonToAvroToJson,
!ConversionWithSchemaInferenceIT#testCsvToAvroToCsv,
!ConversionWithExplicitSchemaIT#testJsonToCsv,
!ConversionWithExplicitSchemaIT#testJsonToAvro,
!ConversionWithExplicitSchemaIT#testJsonToAvroToJson,
!ITConsumeKinesisStreamEndpointOverride,
!ITConsumeKinesisStreamConnectAWS,
!ITPutKinesisStream,
!ITPutKinesisStreamWithEndpointOverride#testIntegrationSuccess,
!ITPutKinesisFirehoseWithEndpointOverride#testIntegrationSuccess,
!ITPutKinesisFirehose,
!ITTagS3Object,
!ITFetchS3Object,
!ITDeleteS3Object,
!ITPutS3Object,
!ITListS3,
!ITPutSNS#testPublish,
!ITPutSNS#testPublishWithCredentialsProviderService,
!ITDeleteSQS,
!ITGetSQS,
!ITPutSQS,
!ITPutLambda,
!GetMongoIT#testDatabaseEL,
!PutGridFSIT#testFileNameAndHashUniqueness,
!PutGridFSIT#testHashUniqueness,
!ITPutAzureCosmosDBRecord,
!ITListAzureBlobStorage_v12,
!ITPutAzureDataLakeStorage,
!ITFetchAzureDataLakeStorage,
!ITListAzureDataLakeStorage,
!ITFetchAzureBlobStorage_v12,
!ITPutAzureBlobStorage_v12,
!ITDeleteAzureDataLakeStorage,
!ITDeleteAzureBlobStorage_v12,
!ITMoveAzureDataLakeStorage,
!AzureGraphUserGroupProviderIT,
!JMSPublisherConsumerIT#validateMessageRedeliveryWhenNotAcked,
!ITPutInfluxDB,
!ITExecuteInfluxDBQuery,
!GremlinClientServiceYamlSettingsAndBytecodeIT,
!GremlinClientServiceControllerSettingsIT,
!ITestConsumeEmail#validateUrl,
!PrometheusReportingTaskIT#testNullLabel,
!SnowflakeConnectionPoolIT,
!SnowflakePipeIT,
!SalesforceRestServiceIT,
!QuerySalesforceObjectIT,
!PutSalesforceObjectIT,
!FetchDropboxIT,
!PutDropboxIT,
!ListDropboxIT,
!FetchBoxFileIT,
!PutBoxFileIT,
!ListBoxFileIT,
!JsonConfigBasedBoxClientServiceIT,
!QueryIoTDBIT,
!PutIoTDBRecordIT,
!StandardProcessSessionIT#testCloneThenWriteCountsClaimReferencesProperly,
!StandardProcessSessionIT#testUpdateFlowFileRepoFailsOnSessionCommit,
!JMSPublisherConsumerIT#testMultipleThreads,
!ITRedisDistributedMapCacheClientService#testSentinelRedis,
!ITRedisDistributedMapCacheClientService#testSentinelRedisWithAuthentication,
!ITRedisDistributedMapCacheClientService#testSentinelRedisWithDefaultUserAuthentication,
!ITRedisDistributedMapCacheClientService#testSentinelRedisWithUsernameAndPasswordAuthentication,
!ITRedisDistributedMapCacheClientService#testStandaloneRedis,
!ITRedisDistributedMapCacheClientService#testStandaloneRedisWithAuthentication,
!SecureDatabaseIT,
!SecureFileIT,
!SecureProxyIT#testAccessStatus,
!SecureProxyIT#testAccessStatusAsProxiedAdmin,
!SecureProxyIT#testAccessStatusUsingRegistryClient,
!SecureNiFiRegistryClientIT#testGetAccessStatus,
!SecureNiFiRegistryClientIT#testTenantsClientGroups,
!SecureNiFiRegistryClientIT#testTenantsClientUsers,
!SecureNiFiRegistryClientIT#testCrudOperations,
!UnsecuredNiFiRegistryClientIT#testFlowSnapshotsWithParameterContextAndEncodingVersion,
!UnsecuredNiFiRegistryClientIT#testNiFiRegistryClient,
!ITConnectionAccessControl,
!ITLabelAccessControl,
!ITOutputPortAccessControl,
!ITProcessorAccessControl,
!ITFlowAccessControl,
!ITFunnelAccessControl,
!ITInputPortAccessControl,
!ITCountersAccessControl,
!ITAllowProxiedAnonymousAccess,
!ITPreventProxiedAnonymousAccess,
!ITPreventDirectAnonymousAccess,
!ITAllowDirectAnonymousAccess,
!ITProcessGroupAccessControl,
!StatelessNiFiSinkTaskIT#testSimpleFlow,
!StatelessNiFiSinkTaskIT#testParameters,
!StatelessNiFiSinkTaskIT#testWrongOutputPort
{quote}
This list was generated in the order that the tests were found to be failing,
and is written in the format to indicate that [failsafe should skip these test
classes/methods|https://maven.apache.org/surefire/maven-failsafe-plugin/examples/single-test.html#multiple-formats-in-one]
was:
NIFI-12177 adds an {{integraiton-tests}} GitHub Actions Workflow to regularly
run the {{integration-tests}} defined throughout the NiFi repo. However, many
tests have had to be skipped from the {{failsafe plugin}} execution due to
failures within the GitHub runner.
Typically these are caused by:
* the need for external resources to be setup
* incorrect assertions that might have worked during the original IT creation
but have not been updated since
* or missing/incompatible OS resources within the GitHub runner
This Epic is to review the failing ITs within NiFi codebase and either:
* fix them, e.g. correct assertions, and/or by using Testcontainers for hosting
external resources, such as for the Elasticsearch Processors
* remove them if no longer useful/cannot be reliably executed on
developer's/GitHub runners
* annotate them to be skipped in CI environments if the tests are useful and
work on a developer machine but won't be able to work in a CI runner, e.g. a
[DisabledIfEnvironmentVariable|https://junit.org/junit5/docs/5.7.0/api/org.junit.jupiter.api/org/junit/jupiter/api/condition/DisabledIfEnvironmentVariable.html]
to check whether the `CI` env var is present and set to `true`
Once fixed, the appropriate entry from the {{integration-tests}} Workflow's
{{MAVEN_FAILSAFE_SKIP_ITS}} env var **must** be removed
At the time of writing, the ITs known to be failing (either in part or as a
whole) are:
{quote} !AwsSecretsManagerSensitivePropertyProviderIT,
!AwsKmsSensitivePropertyProviderIT,
!AzureKeyVaultKeySensitivePropertyProviderIT,
!GcpKmsSensitivePropertyProviderIT,
!StandardHashiCorpVaultCommunicationServiceIT,
!ConversionWithSchemaInferenceIT#testXmlToAvro,
!ConversionWithSchemaInferenceIT#testXmlToJson,
!ConversionWithSchemaInferenceIT#testJsonToCsv,
!ConversionWithSchemaInferenceIT#testCsvToXml,
!ConversionWithSchemaInferenceIT#testXmlToAvroToXml,
!ConversionWithSchemaInferenceIT#testCsvToAvro,
!ConversionWithSchemaInferenceIT#testCsvToJson,
!ConversionWithSchemaInferenceIT#testJsonToAvroToJson,
!ConversionWithSchemaInferenceIT#testCsvToAvroToCsv,
!ConversionWithExplicitSchemaIT#testJsonToCsv,
!ConversionWithExplicitSchemaIT#testJsonToAvro,
!ConversionWithExplicitSchemaIT#testJsonToAvroToJson,
!ITConsumeKinesisStreamEndpointOverride,
!ITConsumeKinesisStreamConnectAWS,
!ITPutKinesisStream,
!ITPutKinesisStreamWithEndpointOverride#testIntegrationSuccess,
!ITPutKinesisFirehoseWithEndpointOverride#testIntegrationSuccess,
!ITPutKinesisFirehose,
!ITTagS3Object,
!ITFetchS3Object,
!ITDeleteS3Object,
!ITPutS3Object,
!ITListS3,
!ITPutSNS#testPublish,
!ITPutSNS#testPublishWithCredentialsProviderService,
!ITDeleteSQS,
!ITGetSQS,
!ITPutSQS,
!ITPutLambda,
!GetMongoIT#testDatabaseEL,
!PutGridFSIT#testFileNameAndHashUniqueness,
!PutGridFSIT#testHashUniqueness,
!ITPutAzureCosmosDBRecord,
!ITListAzureBlobStorage_v12,
!ITPutAzureDataLakeStorage,
!ITFetchAzureDataLakeStorage,
!ITListAzureDataLakeStorage,
!ITFetchAzureBlobStorage_v12,
!ITPutAzureBlobStorage_v12,
!ITDeleteAzureDataLakeStorage,
!ITDeleteAzureBlobStorage_v12,
!ITMoveAzureDataLakeStorage,
!AzureGraphUserGroupProviderIT,
!JMSPublisherConsumerIT#validateMessageRedeliveryWhenNotAcked,
!ITPutInfluxDB,
!ITExecuteInfluxDBQuery,
!GremlinClientServiceYamlSettingsAndBytecodeIT,
!GremlinClientServiceControllerSettingsIT,
!ITestConsumeEmail#validateUrl,
!PrometheusReportingTaskIT#testNullLabel,
!SnowflakeConnectionPoolIT,
!SalesforceRestServiceIT,
!QuerySalesforceObjectIT,
!PutSalesforceObjectIT,
!FetchDropboxIT,
!PutDropboxIT,
!ListDropboxIT,
!FetchBoxFileIT,
!PutBoxFileIT,
!ListBoxFileIT,
!JsonConfigBasedBoxClientServiceIT,
!QueryIoTDBIT,
!PutIoTDBRecordIT,
!StandardProcessSessionIT#testCloneThenWriteCountsClaimReferencesProperly,
!StandardProcessSessionIT#testUpdateFlowFileRepoFailsOnSessionCommit,
!JMSPublisherConsumerIT#testMultipleThreads,
!ITRedisDistributedMapCacheClientService#testSentinelRedisWithAuthentication,
!ITRedisDistributedMapCacheClientService#testSentinelRedis,
!SnowflakePipeIT,
!SecureFileIT,
!SecureProxyIT#testAccessStatus,
!SecureProxyIT#testAccessStatusAsProxiedAdmin,
!SecureProxyIT#testAccessStatusUsingRegistryClient,
!SecureNiFiRegistryClientIT#testGetAccessStatus,
!SecureNiFiRegistryClientIT#testTenantsClientGroups,
!SecureNiFiRegistryClientIT#testTenantsClientUsers,
!SecureNiFiRegistryClientIT#testCrudOperations,
!SecureDatabaseIT
!ITConnectionAccessControl,
!ITLabelAccessControl,
!ITOutputPortAccessControl,
!ITProcessorAccessControl,
!ITFlowAccessControl,
!ITFunnelAccessControl,
!ITInputPortAccessControl
!ITCountersAccessControl,
!ITAllowProxiedAnonymousAccess,
!ITPreventProxiedAnonymousAccess,
!ITPreventDirectAnonymousAccess,
!ITAllowDirectAnonymousAccess,
!ITProcessGroupAccessControl{quote}
This list was generated in the order that the tests were found to be failing,
and is written in the format to indicate that {{failsafe should skip these test
classes/methods|https://maven.apache.org/surefire/maven-failsafe-plugin/examples/single-test.html#multiple-formats-in-one]
> Fix failing Integration Tests
> -----------------------------
>
> Key: NIFI-12190
> URL: https://issues.apache.org/jira/browse/NIFI-12190
> Project: Apache NiFi
> Issue Type: Epic
> Reporter: Chris Sampson
> Priority: Major
> Fix For: 2.latest
>
>
> NIFI-12177 adds an {{integration-tests}} GitHub Actions Workflow to regularly
> run the {{integration-tests}} defined throughout the NiFi repo. However, many
> tests have had to be skipped from the {{maven-failsafe plugin}} execution due
> to failures within the GitHub runner.
> Typically these are caused by:
> * the need for external resources to be setup
> * incorrect assertions that might have worked during the original IT creation
> but have not been updated since
> * or missing/incompatible OS resources within the GitHub runner
> * missing/misconfigured dependencies needed to execute the integration tests
> This Epic is to review the failing ITs within NiFi codebase and either:
> * fix them, e.g. correct assertions, and/or by using
> [Testcontainers|https://testcontainers.com/] for hosting external resources,
> such as for the Elasticsearch Processors
> * remove them if no longer useful/cannot be reliably executed on
> developer's/GitHub runners
> * annotate them to be skipped in CI environments if the tests are useful and
> work on a developer machine but won't be able to work in a CI runner, e.g. a
> [DisabledIfEnvironmentVariable|https://junit.org/junit5/docs/5.7.0/api/org.junit.jupiter.api/org/junit/jupiter/api/condition/DisabledIfEnvironmentVariable.html]
> to check whether the {{CI}} env var is present and set to {{true}}
> Once fixed, the appropriate entry from the top-level {{pom.xml}}'s
> {{integration-tests-ci}} profile configuration for the
> {{maven-failsafe-plugin}} **must** be removed.
> At the time of writing, the ITs known to be failing (either in part or as a
> whole) are:
> {quote}
> !AwsSecretsManagerSensitivePropertyProviderIT,
> !AwsKmsSensitivePropertyProviderIT,
> !AzureKeyVaultKeySensitivePropertyProviderIT,
> !GcpKmsSensitivePropertyProviderIT,
> !StandardHashiCorpVaultCommunicationServiceIT,
>
> !ConversionWithSchemaInferenceIT#testXmlToAvro,
>
> !ConversionWithSchemaInferenceIT#testXmlToJson,
>
> !ConversionWithSchemaInferenceIT#testJsonToCsv,
> !ConversionWithSchemaInferenceIT#testCsvToXml,
>
> !ConversionWithSchemaInferenceIT#testXmlToAvroToXml,
>
> !ConversionWithSchemaInferenceIT#testCsvToAvro,
>
> !ConversionWithSchemaInferenceIT#testCsvToJson,
>
> !ConversionWithSchemaInferenceIT#testJsonToAvroToJson,
>
> !ConversionWithSchemaInferenceIT#testCsvToAvroToCsv,
> !ConversionWithExplicitSchemaIT#testJsonToCsv,
>
> !ConversionWithExplicitSchemaIT#testJsonToAvro,
>
> !ConversionWithExplicitSchemaIT#testJsonToAvroToJson,
> !ITConsumeKinesisStreamEndpointOverride,
> !ITConsumeKinesisStreamConnectAWS,
> !ITPutKinesisStream,
>
> !ITPutKinesisStreamWithEndpointOverride#testIntegrationSuccess,
>
> !ITPutKinesisFirehoseWithEndpointOverride#testIntegrationSuccess,
> !ITPutKinesisFirehose,
> !ITTagS3Object,
> !ITFetchS3Object,
> !ITDeleteS3Object,
> !ITPutS3Object,
> !ITListS3,
> !ITPutSNS#testPublish,
>
> !ITPutSNS#testPublishWithCredentialsProviderService,
> !ITDeleteSQS,
> !ITGetSQS,
> !ITPutSQS,
> !ITPutLambda,
> !GetMongoIT#testDatabaseEL,
> !PutGridFSIT#testFileNameAndHashUniqueness,
> !PutGridFSIT#testHashUniqueness,
> !ITPutAzureCosmosDBRecord,
> !ITListAzureBlobStorage_v12,
> !ITPutAzureDataLakeStorage,
> !ITFetchAzureDataLakeStorage,
> !ITListAzureDataLakeStorage,
> !ITFetchAzureBlobStorage_v12,
> !ITPutAzureBlobStorage_v12,
> !ITDeleteAzureDataLakeStorage,
> !ITDeleteAzureBlobStorage_v12,
> !ITMoveAzureDataLakeStorage,
> !AzureGraphUserGroupProviderIT,
>
> !JMSPublisherConsumerIT#validateMessageRedeliveryWhenNotAcked,
> !ITPutInfluxDB,
> !ITExecuteInfluxDBQuery,
>
> !GremlinClientServiceYamlSettingsAndBytecodeIT,
> !GremlinClientServiceControllerSettingsIT,
> !ITestConsumeEmail#validateUrl,
> !PrometheusReportingTaskIT#testNullLabel,
> !SnowflakeConnectionPoolIT,
> !SnowflakePipeIT,
> !SalesforceRestServiceIT,
> !QuerySalesforceObjectIT,
> !PutSalesforceObjectIT,
> !FetchDropboxIT,
> !PutDropboxIT,
> !ListDropboxIT,
> !FetchBoxFileIT,
> !PutBoxFileIT,
> !ListBoxFileIT,
> !JsonConfigBasedBoxClientServiceIT,
> !QueryIoTDBIT,
> !PutIoTDBRecordIT,
>
> !StandardProcessSessionIT#testCloneThenWriteCountsClaimReferencesProperly,
>
> !StandardProcessSessionIT#testUpdateFlowFileRepoFailsOnSessionCommit,
> !JMSPublisherConsumerIT#testMultipleThreads,
>
> !ITRedisDistributedMapCacheClientService#testSentinelRedis,
>
> !ITRedisDistributedMapCacheClientService#testSentinelRedisWithAuthentication,
>
> !ITRedisDistributedMapCacheClientService#testSentinelRedisWithDefaultUserAuthentication,
>
> !ITRedisDistributedMapCacheClientService#testSentinelRedisWithUsernameAndPasswordAuthentication,
>
> !ITRedisDistributedMapCacheClientService#testStandaloneRedis,
>
> !ITRedisDistributedMapCacheClientService#testStandaloneRedisWithAuthentication,
> !SecureDatabaseIT,
> !SecureFileIT,
> !SecureProxyIT#testAccessStatus,
> !SecureProxyIT#testAccessStatusAsProxiedAdmin,
>
> !SecureProxyIT#testAccessStatusUsingRegistryClient,
>
> !SecureNiFiRegistryClientIT#testGetAccessStatus,
>
> !SecureNiFiRegistryClientIT#testTenantsClientGroups,
>
> !SecureNiFiRegistryClientIT#testTenantsClientUsers,
>
> !SecureNiFiRegistryClientIT#testCrudOperations,
>
> !UnsecuredNiFiRegistryClientIT#testFlowSnapshotsWithParameterContextAndEncodingVersion,
>
> !UnsecuredNiFiRegistryClientIT#testNiFiRegistryClient,
> !ITConnectionAccessControl,
> !ITLabelAccessControl,
> !ITOutputPortAccessControl,
> !ITProcessorAccessControl,
> !ITFlowAccessControl,
> !ITFunnelAccessControl,
> !ITInputPortAccessControl,
> !ITCountersAccessControl,
> !ITAllowProxiedAnonymousAccess,
> !ITPreventProxiedAnonymousAccess,
> !ITPreventDirectAnonymousAccess,
> !ITAllowDirectAnonymousAccess,
> !ITProcessGroupAccessControl,
> !StatelessNiFiSinkTaskIT#testSimpleFlow,
> !StatelessNiFiSinkTaskIT#testParameters,
> !StatelessNiFiSinkTaskIT#testWrongOutputPort
> {quote}
> This list was generated in the order that the tests were found to be failing,
> and is written in the format to indicate that [failsafe should skip these
> test
> classes/methods|https://maven.apache.org/surefire/maven-failsafe-plugin/examples/single-test.html#multiple-formats-in-one]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
