[
https://issues.apache.org/jira/browse/NIFI-15775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18070621#comment-18070621
]
ASF subversion and git services commented on NIFI-15775:
--------------------------------------------------------
Commit a658c3335ed3ed9ab4ff7ecb25ae2f266f5c1cb3 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a658c3335ed ]
NIFI-15775 Refactor Snowflake Snowpipe Ingest without SDK (#11090)
* NIFI-15775 Refactored Snowflake Snowpipe Ingest without SDK
- Removed snowflake-ingest-sdk dependency
- Refactored SnowflakeIngestManagerProviderService interface with ingest methods
- Adjusted Snowflake Processors to use new methods
- Added SnowpipeIngestClient implementing Snowpipe REST API methods
> Refactor Snowflake Ingest Integration without Ingest SDK
> --------------------------------------------------------
>
> Key: NIFI-15775
> URL: https://issues.apache.org/jira/browse/NIFI-15775
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: David Handermann
> Assignee: David Handermann
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The {{SnowflakeIngestManagerProviderService}} interface abstracts access to
> the Snowflake {{SimpleIngestManager}} class, which is the primary integration
> point for ingesting files to Snowflake Snowpipe services, and polling
> ingestion history. The
> [snowflake-ingest-sdk|https://central.sonatype.com/artifact/net.snowflake/snowflake-ingest-sdk]
> library includes the {{SimpleIngestManager}} along with other classes that
> support Snowpipe Streaming. The library implements the [Snowpipe Streaming
> Classic
> Architecture|https://docs.snowflake.com/en/user-guide/snowpipe-streaming/snowpipe-streaming-classic-overview],
> which is planned for future deprecation.
> With current Snowflake file ingestion not using Snowpipe Streaming features,
> the Service interface and implementation should be refactored to work
> directly with the [Snowpipe REST
> API|https://docs.snowflake.com/en/user-guide/data-load-snowpipe-rest-apis].
> This should remove the dependency on the {{snowflake-ingest-sdk}} library
> while maintaining current Snowpipe file ingest capabilities.
> Removing the {{snowflake-ingest-sdk}} requires changing the interface
> contract of {{{}SnowflakeIngestManagerProviderService{}}}, removing the
> reference to the {{SimpleIngestManager}} class. With usage limited to the
> {{GetSnowflakeIngestStatus}} and {{StartSnowflakeIngest}} Processors, the
> impact is localized to the {{{}nifi-snowflake-bundle{}}}, keeping all changes
> within the bundle boundary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
