Sivaprasanna Ok makes sense. Are you in a position of interest/expertise/time to make those changes as well?
Thanks On Thu, May 3, 2018 at 12:11 PM, Sivaprasanna <sivaprasanna...@gmail.com> wrote: > Hi. As I had mentioned, upgrading to the latest version of the library is > not as simple as I thought. Google Cloud team introduced many breaking > changes. Many of the APIs (classes & methods) have been > scrapped/replaced/modified/refactored/renamed. > > In short, a simple change of version may demand changes on the processor's > code, especially on the AbstractProcessors (AbstractGCS, AbstractGCP) which > may pose backward compatibility issues, I'm afraid. > > Thanks, > Sivaprasanna > > On Thu, May 3, 2018 at 9:26 PM, Joe Witt <joe.w...@gmail.com> wrote: > >> Sivaprasanna >> >> I might not completely follow but is there a 3rd option to upgrade to >> a more recent library and solve the use of the proper jars >> problem(smaller nar)? >> >> Thanks >> >> On Thu, May 3, 2018 at 11:51 AM, Sivaprasanna <sivaprasanna...@gmail.com> >> wrote: >> > Hi >> > >> > I've started the initial works on implementing Google Cloud Pub/Sub >> > processors. The associated Jira ID is NIFI-5133 >> > <https://issues.apache.org/jira/browse/NIFI-5133>. This will go to the >> > existing GCP bundle which currently has only the storage processors. Upon >> > some inspection, I noticed the following: >> > >> > - As of now, the bundle uses google-cloud >> > <http://mvnrepository.com/artifact/com.google.cloud/google-cloud> as >> its >> > dependency which is like uber/fat jar that contains most of the Google >> > Cloud's client library SDKs including storage, bigquery, pubsub, etc. >> The >> > main point is it is using a very older version (0.8.0) >> > - I thought of using google-cloud-bom >> > <http://mvnrepository.com/artifact/com.google.cloud/google-cloud-bom> >> in >> > the bundle's POM >> > <https://github.com/apache/nifi/blob/master/nifi-nar- >> bundles/nifi-gcp-bundle/pom.xml> >> > and then use the required artifacts in the processor's POM >> > <https://github.com/apache/nifi/blob/master/nifi-nar- >> bundles/nifi-gcp-bundle/nifi-gcp-processors/pom.xml>. >> > The benefit is, it will help us reduce the overall size of the NAR. >> > >> > When I tried to do #2, I realized this is not a simple version change >> but a >> > change that brings backward compatibility issues. Ex: Some APIs used in >> the >> > older version i.e. 0.8.0 have now been entirely scrapped and moved to >> > different library. We can do either two things: >> > >> > 1. User the Pub/Sub APIs from the older version but the problem is >> it's >> > very old and the problem of upgrading would soon catchup with us. >> > 2. Or we can continue to use the older version of google-cloud-storage >> > only for the storage processors and introduce the #2 mentioned above >> but I >> > don't think then the new processors can't properly extend the existing >> > AbstractGCPProcessor >> > <https://github.com/apache/nifi/blob/master/nifi-nar- >> bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/ >> org/apache/nifi/processors/gcp/AbstractGCPProcessor.java>. >> > >> > >> > A quick glance on the processor code and the POM would help you >> understand >> > my concern. >> > >> > I'm stuck up here so any help & guidance in this regard is very much >> > appreciated. :) >> > >> > Thanks, >> > >> > Sivaprasanna >>
