Hi I've started the initial works on implementing Google Cloud Pub/Sub processors. The associated Jira ID is NIFI-5133 <https://issues.apache.org/jira/browse/NIFI-5133>. This will go to the existing GCP bundle which currently has only the storage processors. Upon some inspection, I noticed the following:
- As of now, the bundle uses google-cloud <http://mvnrepository.com/artifact/com.google.cloud/google-cloud> as its dependency which is like uber/fat jar that contains most of the Google Cloud's client library SDKs including storage, bigquery, pubsub, etc. The main point is it is using a very older version (0.8.0) - I thought of using google-cloud-bom <http://mvnrepository.com/artifact/com.google.cloud/google-cloud-bom> in the bundle's POM <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-gcp-bundle/pom.xml> and then use the required artifacts in the processor's POM <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/pom.xml>. The benefit is, it will help us reduce the overall size of the NAR. When I tried to do #2, I realized this is not a simple version change but a change that brings backward compatibility issues. Ex: Some APIs used in the older version i.e. 0.8.0 have now been entirely scrapped and moved to different library. We can do either two things: 1. User the Pub/Sub APIs from the older version but the problem is it's very old and the problem of upgrading would soon catchup with us. 2. Or we can continue to use the older version of google-cloud-storage only for the storage processors and introduce the #2 mentioned above but I don't think then the new processors can't properly extend the existing AbstractGCPProcessor <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/AbstractGCPProcessor.java>. A quick glance on the processor code and the POM would help you understand my concern. I'm stuck up here so any help & guidance in this regard is very much appreciated. :) Thanks, Sivaprasanna
