[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396687#comment-16396687
 ] 

ASF GitHub Bot commented on NIFI-4800:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2530


> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
> Fix For: 1.6.0
>
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396686#comment-16396686
 ] 

ASF subversion and git services commented on NIFI-4800:
---

Commit 7aabb99bccc946d460088dc2ba24d45f5c5d1937 in nifi's branch 
refs/heads/master from [~Deon]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=7aabb99 ]

NIFI-4800 Expose the flattenMode as property in FlattenJSON processor

Signed-off-by: Pierre Villard 

This closes #2530.


> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
> Fix For: 1.6.0
>
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396328#comment-16396328
 ] 

ASF GitHub Bot commented on NIFI-4800:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2530
  
+1 LGTM.


> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395247#comment-16395247
 ] 

ASF GitHub Bot commented on NIFI-4800:
--

Github user yjhyjhyjh0 commented on the issue:

https://github.com/apache/nifi/pull/2530
  
Thanks for the review.
I've added display name as most processor does and removed unnecessary line 
at unit tests.



> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394942#comment-16394942
 ] 

ASF GitHub Bot commented on NIFI-4800:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2530#discussion_r173719906
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/groovy/org/apache/nifi/processors/standard/TestFlattenJson.groovy
 ---
@@ -134,4 +134,44 @@ class TestFlattenJson {
 ])
 }
 }
+
+@Test
+void testFlattenModeNormal() {
+def testRunner = TestRunners.newTestRunner(FlattenJson.class)
+def json = prettyPrint(toJson([
+first: [
+second: [
+third: [
+"one", "two", "three", "four", 
"five"
+]
+]
+]
+]))
+
+testRunner.setValidateExpressionUsage(true);
+testRunner.setProperty(FlattenJson.FLATTEN_MODE, 
FlattenJson.FLATTEN_MODE_NORMAL)
+baseTest(testRunner, json,5) { parsed ->
+Assert.assertEquals("Separator not applied.", "one", 
parsed["first.second.third[0]"])
+}
+}
+
+@Test
+void testFlattenModeDotNotation() {
+def testRunner = TestRunners.newTestRunner(FlattenJson.class)
+def json = prettyPrint(toJson([
+first: [
+second: [
+third: [
+"one", "two", "three", "four", 
"five"
+]
+]
+]
+]))
+
+testRunner.setValidateExpressionUsage(true);
--- End diff --

not sure this line is required


> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394943#comment-16394943
 ] 

ASF GitHub Bot commented on NIFI-4800:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2530#discussion_r173719297
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/FlattenJson.java
 ---
@@ -89,13 +92,32 @@
 .expressionLanguageSupported(true)
 .build();
 
+public static final AllowableValue FLATTEN_MODE_NORMAL = new 
AllowableValue("normal", "normal",
+"Flattens every objects into a single level json");
+
+public static final AllowableValue FLATTEN_MODE_KEEP_ARRAYS = new 
AllowableValue("keep arrays", "keep arrays",
+"Flattens every objects and keep arrays format");
+
+public static final AllowableValue FLATTEN_MODE_DOT_NOTATION = new 
AllowableValue("dot notation", "dot notation",
+"Conforms to MongoDB dot notation to update also nested 
documents");
+
+public static final PropertyDescriptor FLATTEN_MODE = new 
PropertyDescriptor.Builder()
+.name("Flatten Mode")
--- End diff --

Could you use both displayName() and name()?


> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394944#comment-16394944
 ] 

ASF GitHub Bot commented on NIFI-4800:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2530#discussion_r173719874
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/groovy/org/apache/nifi/processors/standard/TestFlattenJson.groovy
 ---
@@ -134,4 +134,44 @@ class TestFlattenJson {
 ])
 }
 }
+
+@Test
+void testFlattenModeNormal() {
+def testRunner = TestRunners.newTestRunner(FlattenJson.class)
+def json = prettyPrint(toJson([
+first: [
+second: [
+third: [
+"one", "two", "three", "four", 
"five"
+]
+]
+]
+]))
+
+testRunner.setValidateExpressionUsage(true);
--- End diff --

not sure this line is required


> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394530#comment-16394530
 ] 

ASF GitHub Bot commented on NIFI-4800:
--

GitHub user yjhyjhyjh0 opened a pull request:

https://github.com/apache/nifi/pull/2530

NIFI-4800 Expose the flattenMode as property in FlattenJSON processor

Expose all 3 kinds of flatten mode.
Documentation along with unit tests.
Finish local nifi instance integration test.


### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yjhyjhyjh0/nifi NIFI-4800

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2530.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2530


commit d78cf4e088b075feef17c515cb3577b3198b1eaf
Author: Deon Huang 
Date:   2018-03-11T15:01:53Z

NIFI-4800 Expose the flattenMode as property in FlattenJSON processor




> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Assignee: Deon Huang
>Priority: Trivial
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor

[Deon Huang (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393219#comment-16393219
 ] 

Deon Huang commented on NIFI-4800:
--

Hi Alois,

I've encountered some problem with json array flatten requirement.
Just take a look at source code and flatten library wnameless json-flattener.
It's great to have all 3 modes expose to users as configurable property.

I would like to work on this improvement and will submit a pull request in few 
days.
Please let me know if there is any problem.
Thanks.

> Expose the flattenMode as property in FlattenJSON processor
> ---
>
> Key: NIFI-4800
> URL: https://issues.apache.org/jira/browse/NIFI-4800
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Alois Gruber
>Priority: Trivial
>
> the flattening class supports 3 different modes, which cannot be selected in 
> the processor. Especially the flattening of arrays would be helpful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)