[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620479#comment-16620479 ] ASF GitHub Bot commented on NIFI-5239: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2896 > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620478#comment-16620478 ] ASF GitHub Bot commented on NIFI-5239: -- Github user zenfenan commented on the issue: https://github.com/apache/nifi/pull/2896 +1 from me. Thanks for the contribution. Merged to the master. > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620477#comment-16620477 ] ASF subversion and git services commented on NIFI-5239: --- Commit 1dea8faa06fc954f6ee7e27082d25b63b65c231e in nifi's branch refs/heads/master from [~mike.thomsen] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=1dea8fa ] NIFI-5239 Made a client service an optional source of connection pooling in Mongo processors. This closes #2896 Signed-off-by: zenfenan > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617502#comment-16617502 ] ASF GitHub Bot commented on NIFI-5239: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2896 @zenfenan updated the capabilitydescription. That was the only change I felt I could make without running afoul of our policies on changes. > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617494#comment-16617494
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218059033
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java
---
@@ -270,20 +291,10 @@ public final void closeClient() {
}
}
-protected MongoDatabase getDatabase(final ProcessContext context) {
-return getDatabase(context, null);
-}
-
protected MongoDatabase getDatabase(final ProcessContext context,
final FlowFile flowFile) {
final String databaseName =
context.getProperty(DATABASE_NAME).evaluateAttributeExpressions(flowFile).getValue();
--- End diff --
@zenfenan This is how we would be doing the database/collection
configuration.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
> --
>
> Key: NIFI-5239
> URL: https://issues.apache.org/jira/browse/NIFI-5239
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The MongoDBControllerService should be able to provide the getDatabase and
> getCollection functionality that are built into the MongoDB processors
> through AbstractMongoDBProcessor. Using the controller service with the
> processors should be optional in the first release it's added and then
> mandatory going forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617488#comment-16617488
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218058211
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
private MongoDatabase db;
private MongoCollection col;
+private String uri;
@OnEnabled
-public void onEnabled(final ConfigurationContext context) throws
InitializationException, IOException, InterruptedException {
+public void onEnabled(final ConfigurationContext context) {
+this.uri =
context.getProperty(URI).evaluateAttributeExpressions().getValue();
this.createClient(context);
-this.db =
this.mongoClient.getDatabase(context.getProperty(MongoDBControllerService.DATABASE_NAME).getValue());
-this.col =
this.db.getCollection(context.getProperty(MongoDBControllerService.COLLECTION_NAME).getValue());
-}
-
-@OnDisabled
-public void onDisable() {
-this.mongoClient.close();
-}
-
-@Override
-public long count(Document query) {
-return this.col.count(query);
-}
-
-@Override
-public void delete(Document query) {
-this.col.deleteMany(query);
}
-@Override
-public boolean exists(Document query) {
-return this.col.count(query) > 0;
-}
-
-@Override
-public Document findOne(Document query) {
-MongoCursor cursor =
this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
+static List descriptors = new ArrayList<>();
-@Override
-public Document findOne(Document query, Document projection) {
-MongoCursor cursor = projection != null
-?
this.col.find(query).projection(projection).limit(1).iterator()
-: this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
-
-@Override
-public List findMany(Document query) {
-return findMany(query, null, -1);
+static {
+descriptors.add(URI);
+descriptors.add(SSL_CONTEXT_SERVICE);
--- End diff --
We need to be able to provide flowfile attribute-configured EL support for
`Database Name` and `Collection Name`. So what this PR does is it defers that
specific configuration to the processors (and lookup service) so that they can
be more flexibly configured.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
>
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617479#comment-16617479
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218057634
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
--- End diff --
Done.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
> --
>
> Key: NIFI-5239
> URL: https://issues.apache.org/jira/browse/NIFI-5239
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The MongoDBControllerService should be able to provide the getDatabase and
> getCollection functionality that are built into the MongoDB processors
> through AbstractMongoDBProcessor. Using the controller service with the
> processors should be optional in the first release it's added and then
> mandatory going forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617448#comment-16617448
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r218051121
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
--- End diff --
In principle, I agree. I'm just not sure we have much options because that
would be a "breaking change for the sake of change" that would legitimately get
a lot of o_O reactions from the rest of the PMC unless the story is pretty
compelling.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
> --
>
> Key: NIFI-5239
> URL: https://issues.apache.org/jira/browse/NIFI-5239
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The MongoDBControllerService should be able to provide the getDatabase and
> getCollection functionality that are built into the MongoDB processors
> through AbstractMongoDBProcessor. Using the controller service with the
> processors should be optional in the first release it's added and then
> mandatory going forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617055#comment-16617055
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r217945557
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
--- End diff --
If possible, I would be +1 for a controller service with a better name. I'm
not saying `MongoDBControllerService` is a bad name but a name that rhymes with
functionality that this controller service offers would sound even better.
something like `MongoDBClientProvider` or something.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
> --
>
> Key: NIFI-5239
> URL: https://issues.apache.org/jira/browse/NIFI-5239
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The MongoDBControllerService should be able to provide the getDatabase and
> getCollection functionality that are built into the MongoDB processors
> through AbstractMongoDBProcessor. Using the controller service with the
> processors should be optional in the first release it's added and then
> mandatory going forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617057#comment-16617057
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r217945056
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
--- End diff --
`CapabilityDescription` has to be updated.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
> --
>
> Key: NIFI-5239
> URL: https://issues.apache.org/jira/browse/NIFI-5239
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The MongoDBControllerService should be able to provide the getDatabase and
> getCollection functionality that are built into the MongoDB processors
> through AbstractMongoDBProcessor. Using the controller service with the
> processors should be optional in the first release it's added and then
> mandatory going forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617056#comment-16617056
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r217945280
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-services/src/main/java/org/apache/nifi/mongodb/MongoDBControllerService.java
---
@@ -17,156 +17,162 @@
package org.apache.nifi.mongodb;
-import com.mongodb.client.FindIterable;
+import com.mongodb.MongoClient;
+import com.mongodb.MongoClientOptions;
+import com.mongodb.MongoClientURI;
+import com.mongodb.WriteConcern;
import com.mongodb.client.MongoCollection;
-import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
-import com.mongodb.client.model.UpdateOptions;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnDisabled;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.authentication.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
-import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
import org.bson.Document;
-import java.io.IOException;
+import javax.net.ssl.SSLContext;
import java.util.ArrayList;
import java.util.List;
@Tags({"mongo", "mongodb", "service"})
@CapabilityDescription(
"Provides a controller service that wraps most of the functionality of
the MongoDB driver."
)
-public class MongoDBControllerService extends
AbstractMongoDBControllerService implements MongoDBClientService {
+public class MongoDBControllerService extends AbstractControllerService
implements MongoDBClientService {
private MongoDatabase db;
private MongoCollection col;
+private String uri;
@OnEnabled
-public void onEnabled(final ConfigurationContext context) throws
InitializationException, IOException, InterruptedException {
+public void onEnabled(final ConfigurationContext context) {
+this.uri =
context.getProperty(URI).evaluateAttributeExpressions().getValue();
this.createClient(context);
-this.db =
this.mongoClient.getDatabase(context.getProperty(MongoDBControllerService.DATABASE_NAME).getValue());
-this.col =
this.db.getCollection(context.getProperty(MongoDBControllerService.COLLECTION_NAME).getValue());
-}
-
-@OnDisabled
-public void onDisable() {
-this.mongoClient.close();
-}
-
-@Override
-public long count(Document query) {
-return this.col.count(query);
-}
-
-@Override
-public void delete(Document query) {
-this.col.deleteMany(query);
}
-@Override
-public boolean exists(Document query) {
-return this.col.count(query) > 0;
-}
-
-@Override
-public Document findOne(Document query) {
-MongoCursor cursor =
this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
+static List descriptors = new ArrayList<>();
-@Override
-public Document findOne(Document query, Document projection) {
-MongoCursor cursor = projection != null
-?
this.col.find(query).projection(projection).limit(1).iterator()
-: this.col.find(query).limit(1).iterator();
-Document retVal = cursor.tryNext();
-cursor.close();
-
-return retVal;
-}
-
-@Override
-public List findMany(Document query) {
-return findMany(query, null, -1);
+static {
+descriptors.add(URI);
+descriptors.add(SSL_CONTEXT_SERVICE);
--- End diff --
Why are we only exposing `URI` alone? I thought this PR intends to offer a
controller service where we can configure a connection to a MongoDB database &
collection and use that controller service optionally instead of the processor
level `URI`, `Database Name`, `Collection Name`.
> Make MongoDBControllerService able to act as a configuration source for
> Mo
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614790#comment-16614790 ] ASF GitHub Bot commented on NIFI-5239: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2896 @zenfenan the reason I asked you to do this one first before I reopen 5333 is that I realized it contains updates to the controller service and configuration properties that are required to smooth out the behavior in 5333. The GetMongoRecord processor is going to be controller service-only for configurations like the GridFS processors waiting in review. > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614012#comment-16614012 ] ASF GitHub Bot commented on NIFI-5239: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2896 @mattyb149 can you take a look? I attached a flow that exercises all of the processors with this change. It just needs mongo running like `docker run -d -p 27017:27017 --name mongo_test mongo:latest` > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614011#comment-16614011 ] ASF GitHub Bot commented on NIFI-5239: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2896 [Mongo_Client_Service_Test.xml.txt](https://github.com/apache/nifi/files/2380858/Mongo_Client_Service_Test.xml.txt) Test flow for all of the processors. > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613464#comment-16613464 ] ASF GitHub Bot commented on NIFI-5239: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2896 @zenfenan any updates? > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602268#comment-16602268 ] ASF GitHub Bot commented on NIFI-5239: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2896 Thanks, will get to this today unless something comes up! > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602158#comment-16602158
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r214684734
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java
---
@@ -244,22 +266,20 @@ public final void closeClient() {
}
}
-protected MongoDatabase getDatabase(final ProcessContext context) {
-return getDatabase(context, null);
-}
+//protected MongoDatabase getDatabase(final ProcessContext context) {
+//return getDatabase(context, null);
+//}
--- End diff --
Comments to be removed.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
> --
>
> Key: NIFI-5239
> URL: https://issues.apache.org/jira/browse/NIFI-5239
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The MongoDBControllerService should be able to provide the getDatabase and
> getCollection functionality that are built into the MongoDB processors
> through AbstractMongoDBProcessor. Using the controller service with the
> processors should be optional in the first release it's added and then
> mandatory going forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[
https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602157#comment-16602157
]
ASF GitHub Bot commented on NIFI-5239:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2896#discussion_r214684778
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java
---
@@ -244,22 +266,20 @@ public final void closeClient() {
}
}
-protected MongoDatabase getDatabase(final ProcessContext context) {
-return getDatabase(context, null);
-}
+//protected MongoDatabase getDatabase(final ProcessContext context) {
+//return getDatabase(context, null);
+//}
protected MongoDatabase getDatabase(final ProcessContext context,
final FlowFile flowFile) {
final String databaseName =
context.getProperty(DATABASE_NAME).evaluateAttributeExpressions(flowFile).getValue();
-if (StringUtils.isEmpty(databaseName)) {
-throw new ProcessException("Database name was empty after
expression language evaluation.");
-}
-return mongoClient.getDatabase(databaseName);
-}
-protected MongoCollection getCollection(final ProcessContext
context) {
-return getCollection(context, null);
+return clientService!= null ?
clientService.getDatabase(databaseName) : mongoClient.getDatabase(databaseName);
}
+//protected MongoCollection getCollection(final
ProcessContext context) {
+//return getCollection(context, null);
+//}
+
--- End diff --
Same here.
> Make MongoDBControllerService able to act as a configuration source for
> MongoDB processors
> --
>
> Key: NIFI-5239
> URL: https://issues.apache.org/jira/browse/NIFI-5239
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The MongoDBControllerService should be able to provide the getDatabase and
> getCollection functionality that are built into the MongoDB processors
> through AbstractMongoDBProcessor. Using the controller service with the
> processors should be optional in the first release it's added and then
> mandatory going forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599219#comment-16599219 ] ASF GitHub Bot commented on NIFI-5239: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2896 @zenfenan @mattyb149 can one of you review? The goal here, btw, is to make it so that admins are able to easily share a `MongoClient` between multiple processors. Per the [docs](http://mongodb.github.io/mongo-java-driver/3.5/driver/getting-started/quick-start/#make-a-connection) `MongoClient` is thread-safe and you should limit how many of them your application creates: > The MongoClient instance represents a pool of connections to the database; you will only need one instance of class MongoClient even with multiple threads. > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545387#comment-16545387 ] ASF GitHub Bot commented on NIFI-5239: -- GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2896 NIFI-5239 Made a client service an optional source of connection pool… …ing in Mongo processors. NIFI-5239 Updated two driver versions. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5239 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2896.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2896 commit c9085c8c552ba13acded3e0e79fb645e865df6bd Author: Mike Thomsen Date: 2018-07-15T12:07:21Z NIFI-5239 Made a client service an optional source of connection pooling in Mongo processors. NIFI-5239 Updated two driver versions. > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5239) Make MongoDBControllerService able to act as a configuration source for MongoDB processors
[ https://issues.apache.org/jira/browse/NIFI-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503566#comment-16503566 ] Mike Thomsen commented on NIFI-5239: For reference purposes, [according to the docs|http://mongodb.github.io/mongo-java-driver/3.5/driver/getting-started/quick-start/#make-a-connection], MongoClient is verified thread-safe. > Make MongoDBControllerService able to act as a configuration source for > MongoDB processors > -- > > Key: NIFI-5239 > URL: https://issues.apache.org/jira/browse/NIFI-5239 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > The MongoDBControllerService should be able to provide the getDatabase and > getCollection functionality that are built into the MongoDB processors > through AbstractMongoDBProcessor. Using the controller service with the > processors should be optional in the first release it's added and then > mandatory going forward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
