[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
GitHub user YolandaMDavis opened a pull request: https://github.com/apache/nifi/pull/353 NIFI-361 - Create Processors to mutate JSON data This is an initial implementation of the TransformJSON processor using the Jolt library. TransformJSON supports Jolt specifications for the following transformations: Chain, Shift, Remove, and Default. Users will be able to add the TransformJSON processor, select the transformation they wish to apply and enter the specification for the given transformation. Details for creating Jolt specifications can be found [here](https://github.com/bazaarvoice/jolt) You can merge this pull request into a Git repository by running: $ git pull https://github.com/YolandaMDavis/nifi NIFI-361 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/353.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #353 commit 3d2c2acae11823dc98cdc06c5066720216185984 Author: Yolanda M. Davis Date: 2016-04-14T12:19:41Z NIFI-361 Initial implementation of TransformJSON using Jolt commit 3dbe30c7d5799e17c4cb8727f379de4ac36fca65 Author: Yolanda M. Davis Date: 2016-04-14T12:20:02Z NIFI-361 Documentation entry and license update --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user YolandaMDavis closed the pull request at: https://github.com/apache/nifi/pull/353 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
GitHub user YolandaMDavis opened a pull request: https://github.com/apache/nifi/pull/354 NIFI-361 - Create Processors to mutate JSON data This is an initial implementation of the TransformJSON processor using the Jolt library. TransformJSON supports Jolt specifications for the following transformations: Chain, Shift, Remove, and Default. Users will be able to add the TransformJSON processor, select the transformation they wish to apply and enter the specification for the given transformation. Details for creating Jolt specifications can be found [here](https://github.com/bazaarvoice/jolt) You can merge this pull request into a Git repository by running: $ git pull https://github.com/YolandaMDavis/nifi NIFI-361 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/354.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #354 commit 68b5d65de0e0b1787a45bea7bcc50fdb2b625655 Author: Yolanda M. Davis Date: 2016-04-14T12:19:41Z NIFI-361 Updates to test for processor including latest master merge commit 236471961bb577768167d3f16fd99bda3f2f1a54 Author: Yolanda M. Davis Date: 2016-04-14T20:18:54Z NIFI-361 add missing asterisk to documentation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r59997274
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r59997330
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r59997381
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60080948
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.buil
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60084343
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60084831
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60115891
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.buil
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60117078
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60118046
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60118214
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60164832
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.buil
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on the pull request: https://github.com/apache/nifi/pull/354#issuecomment-211703487 @YolandaMDavis this looks very good. One general comment; Just to finish the life-cycle loop I'd suggest adding a stop operation (operation annotated with @OnStopped) where based on what I see you would simply set _transformer_ to null, unless there is some stop/cleanup procedure required (e.g., close something etc.). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60167411
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
--- End diff --
Something @alopresto pointed out on another unrelated PR, yet a valid point
and we need to start making it as a rule and that is using _name_ and
_displayName_. In fact I'll quote Andy's comment:
".name should be a machine-safe string (i.e. ssl-context-service) which
will remain constant over the life of the processor because it is use
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60167981
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
--- End diff --
Looking at the tests I am now wondering if w
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user joewitt commented on the pull request: https://github.com/apache/nifi/pull/354#issuecomment-211708711 @YolandaMDavis @olegz some feedback. 1. The com.bazaar.jolt... dependency does not appear to be accounted for in the LICENSE/NOTICE updates. Their LICENSE does indicate they would like reference made to 'Copyright 2013-2014 Bazaarvoice, Inc.' This is found in https://github.com/bazaarvoice/jolt/blob/1271e1919693e63ce1efdf534dfee0e0579d0b2f/LICENSE So we should propogate that into the NOTICE file as well. Also we need to reference JOLT in our top level nifi-assembly/NOTICE as well if it is new and I'd assume at least for Bazaar it would be. 2. I don't believe I follow the recommendation to change the member variable 'transform' to volatile or to set it to null on stopped. The variable appears to be set at the proper lifecycle point in NiFi and thus would not be reassigned in some non-thread safe way. The only question then is could its use itself be non Thread safe and that does appear questionable to me. In that case marking the variable as volatile won't help at all anyway. If indeed it is not thread safe you will want to consider marking the class as executing serially or you could have a pool of transforms or you could transform batches at a time. Lots of ways to tackle it. Of course if that is thread safe already then you are fine and in any event the volatile should be unnecessary. Finally, I am not aware of any cause for setting it to null during onStopped or on unScheduled as if the processor was being removed from the flow then it would get cleaned up anyway. If there was a need to call a close meth od or something then yes that is good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user olegz commented on the pull request: https://github.com/apache/nifi/pull/354#issuecomment-211718256 @joewitt As I am thinking more about it, you have a point for @OnStopped, so I am retracting my comment. As for volatile, what's at question is the visibility of the _transformer_ by another thread. Basically if T1 executed @OnSchedule where it is set to a value, T2 may come in and it will still be null, not because of some race condition, but simply because the value set by T1 may not be visible by T2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user joewitt commented on the pull request: https://github.com/apache/nifi/pull/354#issuecomment-211724253 Oleg off-list pointed out the finer concerns with the member variable. While it seems pretty far out it also seems like it is accurate. In reading https://docs.oracle.com/javase/tutorial/essential/concurrency/atomic.html and https://docs.oracle.com/javase/tutorial/essential/concurrency/memconsist.html And indeed I cannot think of anything the framework is doing with thread management or how those lifecycle calls work that would enforce/guarantee a happens-before relationship for these processor member variables. @markap14 can you? That said the licensing and thread safety issue of the transform itself are still needing to be addressed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user joewitt commented on the pull request: https://github.com/apache/nifi/pull/354#issuecomment-211726075 I am less concerned about thread safety of the transforms themselves now. Jolt's docs do refer to this at the readme level https://github.com/bazaarvoice/jolt/tree/1271e1919693e63ce1efdf534dfee0e0579d0b2f#-performance. So that leaves the update to NOTICE(s) concern :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user davetorok commented on the pull request: https://github.com/apache/nifi/pull/354#issuecomment-211909411 We've been working with JOLT for awhile within NiFi and appreciate having it supported as a standard processor within NiFi! I have some suggestions. 1. I would change the PR to support all the built-in transformations including 'sort' and 'cardinality' which are not in the pull request. 2. 90% of what we've seen uses ChainR as that is a composition of 1 or more individual transforms, almost always a "ShiftR + DefaultR". In fact it is a little confusing to support options other than just "Chainr" since ChainR takes an array of individual transform Specs whereas the others are individual specs. At minimum I would make CHAINR the default. 3. Jolt supports additional custom transforms via fully-qualified Java classnames. We use a couple. I would recommend adding documentation on how to support it via a drop-in jar similar to how ExecuteScript supports additional resources. Example custom transform is here https://github.com/zacheusz/jolt-date --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user davetorok commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60224390
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
+
+public static final PropertyDescriptor JOLT_TRANSFORM
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60297981
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
--- End diff --
Concerning pointing to a file I thin
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60302366
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
+
+public static final PropertyDescriptor JOLT_TRANSF
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/354#discussion_r60631450
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformJSON.java
---
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.BufferedInputStream;
+import org.apache.nifi.util.StopWatch;
+
+import com.bazaarvoice.jolt.Shiftr;
+import com.bazaarvoice.jolt.Removr;
+import com.bazaarvoice.jolt.Chainr;
+import com.bazaarvoice.jolt.Defaultr;
+import com.bazaarvoice.jolt.Transform;
+import com.bazaarvoice.jolt.JsonUtils;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"json", "jolt", "transform", "shiftr", "chainr", "defaultr",
"removr"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Applies a list of JOLT specifications to the
flowfile JSON payload. A new FlowFile is created "
++ "with transformed content and is routed to the 'success'
relationship. If the JSON transform "
++ "fails, the original FlowFile is routed to the 'failure'
relationship")
+public class TransformJSON extends AbstractProcessor {
+
+public static final AllowableValue SHIFTR = new
AllowableValue("Shift", "Shift Transform DSL", "This JOLT transformation will
shift input JSON/data to create the output JSON/data.");
+public static final AllowableValue CHAINR = new
AllowableValue("Chain", "Chain Transform DSL", "Execute list of JOLT
transformations.");
+public static final AllowableValue DEFAULTR = new
AllowableValue("Default", "Default Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+public static final AllowableValue REMOVR = new
AllowableValue("Remove", "Remove Transform DSL", " This JOLT transformation
will apply default values to the output JSON/data.");
+
+public static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
+.name("Jolt Specification")
+.description("Jolt Specification for transform of JSON data.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.addValidator(new JOLTSpecValidator())
+.required(true)
+.build();
+
+public static final PropertyDescriptor JOLT_TRANSF
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user mattyb149 commented on the pull request: https://github.com/apache/nifi/pull/354#issuecomment-216614373 Reviewed, LGTM and works well, thanks very much! +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-361 - Create Processors to mutate JSON dat...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/354 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
