[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2896 ---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218059033
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java
---
@@ -270,20 +291,10 @@ public final void closeClient() {
}
}
-protected MongoDatabase getDatabase(final ProcessContext context) {
-return getDatabase(context, null);
-}
-
protected MongoDatabase getDatabase(final ProcessContext context,
final FlowFile flowFile) {
final String databaseName =
context.getProperty(DATABASE_NAME).evaluateAttributeExpressions(flowFile).getValue();
--- End diff --
@zenfenan This is how we would be doing the database/collection
configuration.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218058211
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
private MongoDatabase db;
private MongoCollection col;
+private String uri;
@OnEnabled
-public void onEnabled(final ConfigurationContext context) throws
InitializationException, IOException, InterruptedException {
+public void onEnabled(final ConfigurationContext context) {
+this.uri =
context.getProperty(URI).evaluateAttributeExpressions().getValue();
this.createClient(context);
-this.db =
this.mongoClient.getDatabase(context.getProperty(MongoDBControllerService.DATABASE_NAME).getValue());
-this.col =
this.db.getCollection(context.getProperty(MongoDBControllerService.COLLECTION_NAME).getValue());
-}
-
-@OnDisabled
-public void onDisable() {
-this.mongoClient.close();
-}
-
-@Override
-public long count(Document query) {
-return this.col.count(query);
-}
-
-@Override
-public void delete(Document query) {
-this.col.deleteMany(query);
}
-@Override
-public boolean exists(Document query) {
-return this.col.count(query) > 0;
-}
-
-@Override
-public Document findOne(Document query) {
-MongoCursor cursor =
this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
+static List descriptors = new ArrayList<>();
-@Override
-public Document findOne(Document query, Document projection) {
-MongoCursor cursor = projection != null
-?
this.col.find(query).projection(projection).limit(1).iterator()
-: this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
-
-@Override
-public List findMany(Document query) {
-return findMany(query, null, -1);
+static {
+descriptors.add(URI);
+descriptors.add(SSL_CONTEXT_SERVICE);
--- End diff --
We need to be able to provide flowfile attribute-configured EL support for
`Database Name` and `Collection Name`. So what this PR does is it defers that
specific configuration to the processors (and lookup service) so that they can
be more flexibly configured.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218057634
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
--- End diff --
Done.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218051121
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
--- End diff --
In principle, I agree. I'm just not sure we have much options because that
would be a "breaking change for the sake of change" that would legitimately get
a lot of o_O reactions from the rest of the PMC unless the story is pretty
compelling.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r217945557
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
--- End diff --
If possible, I would be +1 for a controller service with a better name. I'm
not saying `MongoDBControllerService` is a bad name but a name that rhymes with
functionality that this controller service offers would sound even better.
something like `MongoDBClientProvider` or something.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r217945280
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
private MongoDatabase db;
private MongoCollection col;
+private String uri;
@OnEnabled
-public void onEnabled(final ConfigurationContext context) throws
InitializationException, IOException, InterruptedException {
+public void onEnabled(final ConfigurationContext context) {
+this.uri =
context.getProperty(URI).evaluateAttributeExpressions().getValue();
this.createClient(context);
-this.db =
this.mongoClient.getDatabase(context.getProperty(MongoDBControllerService.DATABASE_NAME).getValue());
-this.col =
this.db.getCollection(context.getProperty(MongoDBControllerService.COLLECTION_NAME).getValue());
-}
-
-@OnDisabled
-public void onDisable() {
-this.mongoClient.close();
-}
-
-@Override
-public long count(Document query) {
-return this.col.count(query);
-}
-
-@Override
-public void delete(Document query) {
-this.col.deleteMany(query);
}
-@Override
-public boolean exists(Document query) {
-return this.col.count(query) > 0;
-}
-
-@Override
-public Document findOne(Document query) {
-MongoCursor cursor =
this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
+static List descriptors = new ArrayList<>();
-@Override
-public Document findOne(Document query, Document projection) {
-MongoCursor cursor = projection != null
-?
this.col.find(query).projection(projection).limit(1).iterator()
-: this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
-
-@Override
-public List findMany(Document query) {
-return findMany(query, null, -1);
+static {
+descriptors.add(URI);
+descriptors.add(SSL_CONTEXT_SERVICE);
--- End diff --
Why are we only exposing `URI` alone? I thought this PR intends to offer a
controller service where we can configure a connection to a MongoDB database &
collection and use that controller service optionally instead of the processor
level `URI`, `Database Name`, `Collection Name`.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r217945056
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
--- End diff --
`CapabilityDescription` has to be updated.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r214684778
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java
---
@@ -244,22 +266,20 @@ public final void closeClient() {
}
}
-protected MongoDatabase getDatabase(final ProcessContext context) {
-return getDatabase(context, null);
-}
+//protected MongoDatabase getDatabase(final ProcessContext context) {
+//return getDatabase(context, null);
+//}
protected MongoDatabase getDatabase(final ProcessContext context,
final FlowFile flowFile) {
final String databaseName =
context.getProperty(DATABASE_NAME).evaluateAttributeExpressions(flowFile).getValue();
-if (StringUtils.isEmpty(databaseName)) {
-throw new ProcessException("Database name was empty after
expression language evaluation.");
-}
-return mongoClient.getDatabase(databaseName);
-}
-protected MongoCollection getCollection(final ProcessContext
context) {
-return getCollection(context, null);
+return clientService!= null ?
clientService.getDatabase(databaseName) : mongoClient.getDatabase(databaseName);
}
+//protected MongoCollection getCollection(final
ProcessContext context) {
+//return getCollection(context, null);
+//}
+
--- End diff --
Same here.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r214684734
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java
---
@@ -244,22 +266,20 @@ public final void closeClient() {
}
}
-protected MongoDatabase getDatabase(final ProcessContext context) {
-return getDatabase(context, null);
-}
+//protected MongoDatabase getDatabase(final ProcessContext context) {
+//return getDatabase(context, null);
+//}
--- End diff --
Comments to be removed.
---
[GitHub] nifi pull request #2896: NIFI-5239 Made a client service an optional source ...
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2896 NIFI-5239 Made a client service an optional source of connection pool⦠â¦ing in Mongo processors. NIFI-5239 Updated two driver versions. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5239 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2896.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2896 commit c9085c8c552ba13acded3e0e79fb645e865df6bd Author: Mike Thomsen Date: 2018-07-15T12:07:21Z NIFI-5239 Made a client service an optional source of connection pooling in Mongo processors. NIFI-5239 Updated two driver versions. ---
