[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329446#comment-16329446
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2401


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
> Fix For: 1.6.0
>
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329444#comment-16329444
 ] 

ASF subversion and git services commented on NIFI-4759:
---

Commit ca54186b608682d028719ae836bd8d8f83fd24d7 in nifi's branch 
refs/heads/master from [~mike.thomsen]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ca54186 ]

NIFI-4759 - Fixed a bug that left a hard-coded reference to _id in as the 
update key for MongoDB upserts.

Signed-off-by: Pierre Villard 

This closes #2401.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329443#comment-16329443
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/2401
  
+1, LGTM, thanks for the PR @MikeThomsen and thanks for the review @mgaido91


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325583#comment-16325583
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2401
  
@joewitt @pvillard31 maybe you can help reviewing this too, thanks.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325568#comment-16325568
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2401
  
LGTM, thanks.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325552#comment-16325552
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2401
  
@mgaido91 I think it's all done now.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324175#comment-16324175
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2401#discussion_r161261787
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +257,74 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
--- End diff --

What about writing the JSON document on one line?


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324176#comment-16324176
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2401#discussion_r161261619
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +257,74 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
+runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "true");
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_QUERY_KEY, 
updateKeyProps[index]);
+for (int x = 0; x < 5; x++) {
--- End diff --

What about using  2 instead of 5?


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324174#comment-16324174
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2401#discussion_r161262183
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +257,74 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
+runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "true");
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_QUERY_KEY, 
updateKeyProps[index]);
+for (int x = 0; x < 5; x++) {
+runner.enqueue(upsert);
+}
+runner.run(5, true, true);
+runner.assertTransferCount(PutMongo.REL_FAILURE, 0);
+runner.assertTransferCount(PutMongo.REL_SUCCESS, 5);
+
+Document query = new Document(updateKeyProps[index], 
updateKeys[index]);
+Document result = collection.find(query).first();
+Assert.assertNotNull("Result was null", result);
+Assert.assertEquals("Count was wrong", 1, 
collection.count(query));
+runner.clearTransferState();
+index++;
+}
+}
+
+/*
--- End diff --

Nit: I'd remove this and the empty line above...


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323933#comment-16323933
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

GitHub user MikeThomsen opened a pull request:

https://github.com/apache/nifi/pull/2401

NIFI-4759 Fixed a bug that left a hard-coded reference to _id in as t…

…he update key for MongoDB upserts.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/MikeThomsen/nifi NIFI-4759

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2401.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2401


commit 6b269dce690d0d98494a646ffb4226b33bdc4412
Author: Mike Thomsen 
Date:   2018-01-10T13:35:09Z

NIFI-4759 Fixed a bug that left a hard-coded reference to _id in as the 
update key for MongoDB upserts.




> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323936#comment-16323936
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2392
  
@mgaido91 

I screwed up the pull request because of a mistake with a rebase. New PR is 
here:

https://github.com/apache/nifi/pull/2401


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323917#comment-16323917
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen closed the pull request at:

https://github.com/apache/nifi/pull/2392


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323906#comment-16323906
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161213967
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
--- End diff --

I just meant that is hard to read and understand the content of this JSON. 
I think that something like:
```
String[] upserts = new String[]{
"{...}",
"{...}",
"{...}"};
```
would improve readability.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323904#comment-16323904
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161213577
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
+runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "true");
+runner.setProperty(PutMongo.UPDATE_QUERY_KEY, 
updateKeyProps[index]);
+for (int x = 0; x < 5; x++) {
--- End diff --

then 2 is enough, isn't it?


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323902#comment-16323902
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161213509
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -166,6 +167,7 @@ public void testUpdateDoesNotInsert() throws Exception {
 byte[] bytes = documentToByteArray(doc);
 
 runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "false");
--- End diff --

can we remove it then, please?


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323898#comment-16323898
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161213119
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -166,6 +167,7 @@ public void testUpdateDoesNotInsert() throws Exception {
 byte[] bytes = documentToByteArray(doc);
 
 runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "false");
--- End diff --

We don't.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323899#comment-16323899
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161213199
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
+runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "true");
+runner.setProperty(PutMongo.UPDATE_QUERY_KEY, 
updateKeyProps[index]);
+for (int x = 0; x < 5; x++) {
+runner.enqueue(upsert);
+}
+runner.run(5, true, true);
+runner.assertTransferCount(PutMongo.REL_FAILURE, 0);
+runner.assertTransferCount(PutMongo.REL_SUCCESS, 5);
+
+Document query = new Document(updateKeyProps[index], 
updateKeys[index]);
+Document result = collection.find(query).first();
+Assert.assertNotNull("Result was null", result);
+Assert.assertEquals("Count was wrong", 1, 
collection.count(query));
+runner.clearTransferState();
+index++;
+}
+}
+
+/*
--- End diff --

I prefer to leave it in to explicitly mark the ending of the regression 
testing.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323897#comment-16323897
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161212684
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
--- End diff --

Not following you here.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323894#comment-16323894
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161212208
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
+runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "true");
+runner.setProperty(PutMongo.UPDATE_QUERY_KEY, 
updateKeyProps[index]);
+for (int x = 0; x < 5; x++) {
--- End diff --

Just a validation that if we run things multiple times the same result 
happens.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323893#comment-16323893
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r161212008
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java
 ---
@@ -173,13 +166,25 @@ public void process(final InputStream in) throws 
IOException {
 // update
 final boolean upsert = 
context.getProperty(UPSERT).asBoolean();
 final String updateKey = 
context.getProperty(UPDATE_QUERY_KEY).getValue();
-final Document query = new Document(updateKey, 
((Map)doc).get(updateKey));
+final Document query;
+
+Object keyVal = ((Map)doc).get(updateKey);
+if (updateKey.equals("_id")) {
+try {
+keyVal = new ObjectId((String) keyVal);
+} catch (Exception ex) {
+getLogger().error("{} is not a valid ObjectID, 
using raw value.", new Object[]{keyVal});
--- End diff --

That's the general pattern that is used in other places for building log 
statements.


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322392#comment-16322392
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160982784
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -166,6 +167,7 @@ public void testUpdateDoesNotInsert() throws Exception {
 byte[] bytes = documentToByteArray(doc);
 
 runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "false");
--- End diff --

why do we need to add this?


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322335#comment-16322335
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160975782
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
+runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "true");
+runner.setProperty(PutMongo.UPDATE_QUERY_KEY, 
updateKeyProps[index]);
+for (int x = 0; x < 5; x++) {
+runner.enqueue(upsert);
+}
+runner.run(5, true, true);
+runner.assertTransferCount(PutMongo.REL_FAILURE, 0);
+runner.assertTransferCount(PutMongo.REL_SUCCESS, 5);
+
+Document query = new Document(updateKeyProps[index], 
updateKeys[index]);
+Document result = collection.find(query).first();
+Assert.assertNotNull("Result was null", result);
+Assert.assertEquals("Count was wrong", 1, 
collection.count(query));
+runner.clearTransferState();
+index++;
+}
+}
+
+/*
--- End diff --

this comment can be removed, it seems useless to me


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322333#comment-16322333
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160975388
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
+runner.setProperty(PutMongo.MODE, "update");
+runner.setProperty(PutMongo.UPSERT, "true");
+runner.setProperty(PutMongo.UPDATE_QUERY_KEY, 
updateKeyProps[index]);
+for (int x = 0; x < 5; x++) {
--- End diff --

why doing this 5 times?


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322331#comment-16322331
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160975264
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
--- End diff --

nit: useless blank line


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322330#comment-16322330
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160975171
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java
 ---
@@ -173,13 +166,25 @@ public void process(final InputStream in) throws 
IOException {
 // update
 final boolean upsert = 
context.getProperty(UPSERT).asBoolean();
 final String updateKey = 
context.getProperty(UPDATE_QUERY_KEY).getValue();
-final Document query = new Document(updateKey, 
((Map)doc).get(updateKey));
+final Document query;
--- End diff --

this can be moved later where it is created


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322323#comment-16322323
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160974686
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java
 ---
@@ -173,13 +166,25 @@ public void process(final InputStream in) throws 
IOException {
 // update
 final boolean upsert = 
context.getProperty(UPSERT).asBoolean();
 final String updateKey = 
context.getProperty(UPDATE_QUERY_KEY).getValue();
-final Document query = new Document(updateKey, 
((Map)doc).get(updateKey));
+final Document query;
+
+Object keyVal = ((Map)doc).get(updateKey);
+if (updateKey.equals("_id")) {
--- End diff --

add ` && ObjectId.isValid(keyVal)`


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322315#comment-16322315
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160973857
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java
 ---
@@ -173,13 +166,25 @@ public void process(final InputStream in) throws 
IOException {
 // update
 final boolean upsert = 
context.getProperty(UPSERT).asBoolean();
 final String updateKey = 
context.getProperty(UPDATE_QUERY_KEY).getValue();
-final Document query = new Document(updateKey, 
((Map)doc).get(updateKey));
+final Document query;
+
+Object keyVal = ((Map)doc).get(updateKey);
+if (updateKey.equals("_id")) {
+try {
+keyVal = new ObjectId((String) keyVal);
+} catch (Exception ex) {
+getLogger().error("{} is not a valid ObjectID, 
using raw value.", new Object[]{keyVal});
+}
+}
+
+query = new Document(updateKey, keyVal);
 
 if (updateMode.equals(UPDATE_WITH_DOC.getValue())) {
+
--- End diff --

nit: useless blank line


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322314#comment-16322314
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160973717
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java
 ---
@@ -173,13 +166,25 @@ public void process(final InputStream in) throws 
IOException {
 // update
 final boolean upsert = 
context.getProperty(UPSERT).asBoolean();
 final String updateKey = 
context.getProperty(UPDATE_QUERY_KEY).getValue();
-final Document query = new Document(updateKey, 
((Map)doc).get(updateKey));
+final Document query;
+
+Object keyVal = ((Map)doc).get(updateKey);
+if (updateKey.equals("_id")) {
+try {
+keyVal = new ObjectId((String) keyVal);
+} catch (Exception ex) {
+getLogger().error("{} is not a valid ObjectID, 
using raw value.", new Object[]{keyVal});
--- End diff --

what about `keyVal + "is not a valid ObjectID, using raw value."`


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322284#comment-16322284
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160972295
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
--- End diff --

I think having the 3 JSONs one for each row would improve readability


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322280#comment-16322280
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2392#discussion_r160971847
  
--- Diff: 
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/PutMongoTest.java
 ---
@@ -256,4 +258,72 @@ public void testUpsertWithOperators() throws Exception 
{
 Assert.assertEquals("Msg had wrong value", msg, "Hi");
 }
 }
+
+/*
+ * Start NIFI-4759 Regression Tests
+ *
+ * 2 issues with ID field:
+ *
+ * * Assumed _id is the update key, causing failures when the user 
configured a different one in the UI.
+ * * Treated _id as a string even when it is an ObjectID sent from 
another processor as a string value.
+ *
+ * Expected behavior:
+ *
+ * * update key field should work no matter what (legal) value it is 
set to be.
+ * * _ids that are ObjectID should become real ObjectIDs when added to 
Mongo.
+ * * _ids that are arbitrary strings should be still go in as strings.
+ *
+ */
+
+@Test
+public void testNiFi_4759_Regressions() {
+String[] upserts = new String[]{
+"{\n" +
+"\t\"_id\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"_id\": \"5a5617b9c1f5de6d8276e87d\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+"{\n" +
+"\t\"updateKey\": \"12345\",\n" +
+"\t\"$set\": {\n" +
+"\t\t\"msg\": \"Hello, world\"\n" +
+"\t}\n" +
+"}",
+};
+
+String[] updateKeyProps = new String[] { "_id", "_id", "updateKey" 
};
+Object[] updateKeys = new Object[] { "12345", new 
ObjectId("5a5617b9c1f5de6d8276e87d"), "12345" };
+int index = 0;
+
+for (String upsert : upserts) {
+runner.setProperty(PutMongo.UPDATE_MODE, 
PutMongo.UPDATE_WITH_OPERATORS);
--- End diff --

this and the following two can be set outside the for loop


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4759) PutMongo does not handle updateKey field correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16320740#comment-16320740
 ] 

ASF GitHub Bot commented on NIFI-4759:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2392
  
@mgaido91 
@mattyb149 

Ok, added a UT that really kicks the tires on both problems. I updated the 
ticket with a new title and an explanation of both issues. Most of the code 
here is focused on handling ObjectIDs correctly.

https://issues.apache.org/jira/browse/NIFI-4759


> PutMongo does not handle updateKey field correctly
> --
>
> Key: NIFI-4759
> URL: https://issues.apache.org/jira/browse/NIFI-4759
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>
> Two issues:
> * The updateKey field is ignored in favor of _id in the update code block of 
> PutMongo.
> * _id fields are always treated as strings, even if they're valid ObjectIds 
> represented as a string. PutMongo should be able to handle these as ObjectIds.
> Regarding the first point, this works:
> {code:java}
> {
> "_id": "1234",
> "$set": { "msg": "Hello, world" }
> }
> {code}
> This does not:
> {code:java}
> {
> "uniqueKey": "12345",
> "$set": { "msg": "Hello, World" }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)