[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2101 ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r169969726 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/pom.xml --- @@ -0,0 +1,88 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-influxdb-bundle +1.6.0-SNAPSHOT + + +nifi-influxdb-processors +jar + + + + org.influxdb + influxdb-java + + +org.apache.commons +commons-lang3 + + +org.apache.nifi +nifi-api + + +org.apache.nifi +nifi-utils + + +org.apache.nifi +nifi-mock +test + + +org.slf4j +slf4j-simple +test + + +junit +junit +test + + +com.google.guava +guava +test + + + + +default + +true + + + + +org.apache.maven.plugins +maven-surefire-plugin + + +**/IT*.java --- End diff -- Oh my bad, I didn't pay close attention... Usually the IT classes do not include the word Test. That's why I thought the root profile would be enough. I'd suggest to rename the IT classes by removing the Test word to be consistent with the other bundles. ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mans2singh commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r169968256 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/pom.xml --- @@ -0,0 +1,88 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-influxdb-bundle +1.6.0-SNAPSHOT + + +nifi-influxdb-processors +jar + + + + org.influxdb + influxdb-java + + +org.apache.commons +commons-lang3 + + +org.apache.nifi +nifi-api + + +org.apache.nifi +nifi-utils + + +org.apache.nifi +nifi-mock +test + + +org.slf4j +slf4j-simple +test + + +junit +junit +test + + +com.google.guava +guava +test + + + + +default + +true + + + + +org.apache.maven.plugins +maven-surefire-plugin + + +**/IT*.java --- End diff -- @pvillard31 If I remove this profile/configuration from the pom.xml and run `mvn clean install` in that directory, the integration tests get executed and fail if InfluxDb is not running on the local server. ``` Tests in error: ITPutInfluxDBTest.setUp:58 » InfluxDBIO java.net.ConnectException: Failed to c... ITPutInfluxDBTest.setUp:58 » InfluxDBIO java.net.ConnectException: Failed to c... ITPutInfluxDBTest.setUp:58 » InfluxDBIO java.net.ConnectException: Failed to c... ITPutInfluxDBTest.setUp:58 » InfluxDBIO java.net.ConnectException: Failed to c... ITPutInfluxDBTest.setUp:58 » InfluxDBIO java.net.ConnectException: Failed to c... ITPutInfluxDBTest.setUp:58 » InfluxDBIO java.net.ConnectException: Failed to c... ``` Can you please let me know how I can skip running the integration tests without using this profile ? Thanks for @MikeThomsen and your feedback. ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r169635658 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/pom.xml --- @@ -0,0 +1,88 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-influxdb-bundle +1.6.0-SNAPSHOT + + +nifi-influxdb-processors +jar + + + + org.influxdb + influxdb-java + + +org.apache.commons +commons-lang3 + + +org.apache.nifi +nifi-api + + +org.apache.nifi +nifi-utils + + +org.apache.nifi +nifi-mock +test + + +org.slf4j +slf4j-simple +test + + +junit +junit +test + + +com.google.guava +guava +test + + + + +default + +true + + + + +org.apache.maven.plugins +maven-surefire-plugin + + +**/IT*.java --- End diff -- I'm not sure this is really needed as we're already defining what tests have to be included in the root pom XML file. xml **/*Test.class **/Test*.class **/*Spec.class ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r165818221
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/PutInfluxDB.java
---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@EventDriven
+@SupportsBatching
+@Tags({"influxdb", "measurement","insert", "write", "put", "timeseries"})
+@CapabilityDescription("Processor to write the content of a FlowFile (in
line protocol
https://docs.influxdata.com/influxdb/v1.3/write_protocols/line_protocol_tutorial/)
to InfluxDB (https://www.influxdb.com/). "
++ " The flow file can contain single measurement point or
multiple measurement points separated by line seperator. The timestamp (last
field) should be in nano-seconds resolution.")
+@WritesAttributes({
+@WritesAttribute(attribute =
AbstractInfluxDBProcessor.INFLUX_DB_ERROR_MESSAGE, description = "InfluxDB
error message"),
+})
+public class PutInfluxDB extends AbstractInfluxDBProcessor {
+
+public static AllowableValue CONSISTENCY_LEVEL_ALL = new
AllowableValue("ALL", "All", "Return success when all nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ANY = new
AllowableValue("ANY", "Any", "Return success when any nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ONE = new
AllowableValue("ONE", "One", "Return success when one node has responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_QUORUM = new
AllowableValue("QUORUM", "Quorum", "Return success when a majority of nodes
have responded with write success");
+
+public static final PropertyDescriptor CONSISTENCY_LEVEL = new
PropertyDescriptor.Builder()
+.name("influxdb-consistency-level")
+.displayName("Consistency Level")
+.description("InfluxDB consistency level")
+.required(true)
+.defaultValue(CONSISTENCY_LEVEL_ONE.getValue())
+.expressionLanguageSupported(true)
+.allowableValues(CONSISTENCY_LEVEL_ONE, CONSISTENCY_LEVEL_ANY,
CONSISTENCY_LEVEL_ALL, CONSISTENCY_LEVEL_QUORUM)
+.build();
+
+public static final PropertyDescriptor RETENTION_POLICY = new
PropertyDescriptor.Builder()
+.name("influxdb-retention-policy")
+.displayName("Retention Policy")
+.description("Retention policy for the saving the records")
+.defaultValue("autogen")
+
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r165812834
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -45,15 +50,26 @@
public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
.name("influxdb-url")
-.displayName("InfluxDB connection url")
-.description("InfluxDB url to connect to")
+.displayName("InfluxDB connection URL")
+.description("InfluxDB URL to connect to. Eg:
http://influxdb:8086";)
+.defaultValue("http://localhost:8086";)
--- End diff --
Solid improvement here.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r165812983
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/PutInfluxDB.java
---
@@ -78,18 +81,33 @@
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.build();
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Successful FlowFiles that are saved to InfluxDB
are routed to this relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("FlowFiles were not saved to InfluxDB are routed
to this relationship").build();
+
+static final Relationship REL_RETRY = new
Relationship.Builder().name("retry")
+.description("FlowFiles were not saved to InfluxDB due to
retryable exception are routed to this relationship").build();
+
+static final Relationship REL_MAX_SIZE_EXCEEDED = new
Relationship.Builder().name("failure-max-size")
--- End diff --
Something to consider here...
With PutHBaseRecord and (still pending merge) DeleteHBaseRow, I had a
similar situation. Users could easily chuck several hundred thousand HBase
operations at the processor all at once. So what was suggested to me, and I did
with both of them, was to break up the incoming flowfile into chunks and then
add a "retry.index" attribute to the flowfile if it failed. That way, users
could loop REL_RETRY to the processor and get everything ingested.
Though that might not apply in this case because InfluxDB doesn't have an
ID field that I know of. If you replay an event with the same timestamp, does
it overwrite or does it just add a new one?
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r165812819
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/PutInfluxDB.java
---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@EventDriven
+@SupportsBatching
+@Tags({"influxdb", "measurement","insert", "write", "put", "timeseries"})
+@CapabilityDescription("Processor to write the content of a FlowFile (in
line protocol
https://docs.influxdata.com/influxdb/v1.3/write_protocols/line_protocol_tutorial/)
to InfluxDB (https://www.influxdb.com/). "
++ " The flow file can contain single measurement point or
multiple measurement points separated by line seperator. The timestamp (last
field) should be in nano-seconds resolution.")
+@WritesAttributes({
+@WritesAttribute(attribute =
AbstractInfluxDBProcessor.INFLUX_DB_ERROR_MESSAGE, description = "InfluxDB
error message"),
+})
+public class PutInfluxDB extends AbstractInfluxDBProcessor {
+
+public static AllowableValue CONSISTENCY_LEVEL_ALL = new
AllowableValue("ALL", "All", "Return success when all nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ANY = new
AllowableValue("ANY", "Any", "Return success when any nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ONE = new
AllowableValue("ONE", "One", "Return success when one node has responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_QUORUM = new
AllowableValue("QUORUM", "Quorum", "Return success when a majority of nodes
have responded with write success");
+
+public static final PropertyDescriptor CONSISTENCY_LEVEL = new
PropertyDescriptor.Builder()
+.name("influxdb-consistency-level")
+.displayName("Consistency Level")
+.description("InfluxDB consistency level")
+.required(true)
+.defaultValue(CONSISTENCY_LEVEL_ONE.getValue())
+.expressionLanguageSupported(true)
+.allowableValues(CONSISTENCY_LEVEL_ONE, CONSISTENCY_LEVEL_ANY,
CONSISTENCY_LEVEL_ALL, CONSISTENCY_LEVEL_QUORUM)
+.build();
+
+public static final PropertyDescriptor RETENTION_POLICY = new
PropertyDescriptor.Builder()
+.name("influxdb-retention-policy")
+.displayName("Retention Policy")
+.description("Retention policy for the saving the records")
+.defaultValue("autogen")
+
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164461734
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
--- End diff --
Typo in "Successful", but you may want to specify what a "successful
FlowFile is", is it a flow file whose contents have been successfully
processed? I can see wanting to reuse the success relationship acros
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164547117
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/PutInfluxDB.java
---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@EventDriven
+@SupportsBatching
+@Tags({"influxdb", "measurement","insert", "write", "put", "timeseries"})
+@CapabilityDescription("Processor to write the content of a FlowFile (in
line protocol
https://docs.influxdata.com/influxdb/v1.3/write_protocols/line_protocol_tutorial/)
to InfluxDB (https://www.influxdb.com/). "
++ " The flow file can contain single measurement point or
multiple measurement points separated by line seperator. The timestamp (last
field) should be in nano-seconds resolution.")
+@WritesAttributes({
+@WritesAttribute(attribute =
AbstractInfluxDBProcessor.INFLUX_DB_ERROR_MESSAGE, description = "InfluxDB
error message"),
+})
+public class PutInfluxDB extends AbstractInfluxDBProcessor {
+
+public static AllowableValue CONSISTENCY_LEVEL_ALL = new
AllowableValue("ALL", "All", "Return success when all nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ANY = new
AllowableValue("ANY", "Any", "Return success when any nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ONE = new
AllowableValue("ONE", "One", "Return success when one node has responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_QUORUM = new
AllowableValue("QUORUM", "Quorum", "Return success when a majority of nodes
have responded with write success");
+
+public static final PropertyDescriptor CONSISTENCY_LEVEL = new
PropertyDescriptor.Builder()
+.name("influxdb-consistency-level")
+.displayName("Consistency Level")
+.description("InfluxDB consistency level")
+.required(true)
+.defaultValue(CONSISTENCY_LEVEL_ONE.getValue())
+.expressionLanguageSupported(true)
+.allowableValues(CONSISTENCY_LEVEL_ONE, CONSISTENCY_LEVEL_ANY,
CONSISTENCY_LEVEL_ALL, CONSISTENCY_LEVEL_QUORUM)
+.build();
+
+public static final PropertyDescriptor RETENTION_POLICY = new
PropertyDescriptor.Builder()
+.name("influxdb-retention-policy")
+.displayName("Retention Policy")
+.description("Retention policy for the saving the records")
+.defaultValue("autogen")
+
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r164456208 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-nar/src/main/resources/META-INF/LICENSE --- @@ -0,0 +1,209 @@ + + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those paten
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164462826
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
--- End diff --
Same comment here
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164463448
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
--- End diff --
Consider add
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164546479
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLU
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164544898
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLU
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164460715
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
--- End diff --
Also is data size the appropriate thing to use here, or should it be number
of records, or both? Any reason why Expression Language would not be supported?
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164460497
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
--- End diff --
Is this a "soft limit", meaning if I had 99 records that were .9 MB in size
and the 100th record was .2 in size, then all 100 records would be batched
together (even though the size is 1.1 MB)? In either case it may be helpful to
add more details to the description.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164469623
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
--- End diff --
Please add a property for Connection Timeout (check the standard-nar for
examples, I think ExecuteSQL has one). Unfortunately there is not a client-side
Query Timeout that I could find, or else that should've been added too. The
Connection Timeout should be set upon the underlying OkHttp client.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164459404
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
--- End diff --
Would prefer Database Name here (I know DB is fairly obvious but Database
is more descriptive, consistent, and the same as in the description)
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164459116
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
--- End diff --
Perhaps an example of an InfluxDB connection URL here? Also URL should be
capitalized.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164544734
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
--- End diff --
Consider supporting Expression Language here. Although it might not make
sense to use FlowFile attributes (since it would be less efficient to have a
connection per execution), supporting EL would allow the use of the Variable
Registry, Process Group variables, etc. to set the URL, which helps when
promoting flows from dev -> test -> production for example.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164545775
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.required(false)
+.description("Password for user")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLU
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164546743
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/PutInfluxDB.java
---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@EventDriven
+@SupportsBatching
+@Tags({"influxdb", "measurement","insert", "write", "put", "timeseries"})
+@CapabilityDescription("Processor to write the content of a FlowFile (in
line protocol
https://docs.influxdata.com/influxdb/v1.3/write_protocols/line_protocol_tutorial/)
to InfluxDB (https://www.influxdb.com/). "
--- End diff --
The URL can/will become a dead link in time, perhaps refer to the Line
Protocol and say "consult InfluxDB documentation for more details" or something
like that.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164218451
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLUX_DB_ERROR_MESSAGE =
"influ
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164209603
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLUX_DB_ERROR_MESSAGE =
"influ
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164209671
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLUX_DB_ERROR_MESSAGE =
"influ
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164210261
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/PutInfluxDB.java
---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@EventDriven
+@SupportsBatching
+@Tags({"influxdb", "measurement","insert", "write", "put", "timeseries"})
+@CapabilityDescription("Processor to write the content of a FlowFile (in
line protocol
https://docs.influxdata.com/influxdb/v1.3/write_protocols/line_protocol_tutorial/)
to InfluxDB (https://www.influxdb.com/). "
++ " The flow file can contain single measurement point or
multiple measurement points separated by line seperator")
+@WritesAttributes({
+@WritesAttribute(attribute =
AbstractInfluxDBProcessor.INFLUX_DB_ERROR_MESSAGE, description = "InfluxDB
error message"),
+})
+public class PutInfluxDB extends AbstractInfluxDBProcessor {
+
+public static AllowableValue CONSISTENCY_LEVEL_ALL = new
AllowableValue("ALL", "All", "Return success when all nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ANY = new
AllowableValue("ANY", "Any", "Return success when any nodes have responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_ONE = new
AllowableValue("ONE", "One", "Return success when one node has responded with
write success");
+public static AllowableValue CONSISTENCY_LEVEL_QUORUM = new
AllowableValue("QUORUM", "Quorum", "Return success when a majority of nodes
have responded with write success");
+
+public static final PropertyDescriptor CONSISTENCY_LEVEL = new
PropertyDescriptor.Builder()
+.name("influxdb-consistency-level")
+.displayName("Consistency Level")
+.description("InfluxDB consistency level")
+.required(true)
+.defaultValue(CONSISTENCY_LEVEL_ONE.getValue())
+.expressionLanguageSupported(true)
+
.allowableValues(CONSISTENCY_LEVEL_ONE,CONSISTENCY_LEVEL_ANY,CONSISTENCY_LEVEL_ALL,CONSISTENCY_LEVEL_QUORUM)
--- End diff --
Nit: unneeded white space.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164212464
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/test/java/org/apache/nifi/processors/influxdb/ITPutInfluxDBTest.java
---
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import static org.junit.Assert.assertEquals;
+import java.util.List;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Ignore;
+import org.junit.Test;
+
+/**
+ * Integration test for InfluxDB. Please ensure that the InfluxDB is
running
+ * on local host with default port and has database test with table test.
Please set user
+ * and password if applicable before running the integration tests.
+ */
+@Ignore("Comment this out for running tests against a real instance of
InfluxDB")
--- End diff --
Maven has the ability to do integration tests. You might want to think
about setting up a new profile to do that instead of using Ignore. I have [an
example here on this
PR](https://github.com/MikeThomsen/nifi/commit/cbc2d61f2d98fdf29ff6f1a46fbb524691fd11c0#diff-9fbcd1bfda73f61e6c31fb2fcb3371f3R111).
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164209408
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLUX_DB_ERROR_MESSAGE =
"influ
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164209928
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLUX_DB_ERROR_MESSAGE =
"influ
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164208704
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
--- End diff --
Might to try StandardValidators.NON_BLANK_VALIDATOR here otherwise the user
could try to send a blank username. Since it's not a required field, leaving it
blank should mean that it's not being specified.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164208755
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
--- End diff --
Try StandardValidators.NON_BLANK_VALIDATOR here too.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164209539
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLUX_DB_ERROR_MESSAGE =
"influ
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164212590
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/test/java/org/apache/nifi/processors/influxdb/ITPutInfluxDBTest.java
---
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import static org.junit.Assert.assertEquals;
+import java.util.List;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Ignore;
+import org.junit.Test;
+
+/**
+ * Integration test for InfluxDB. Please ensure that the InfluxDB is
running
+ * on local host with default port and has database test with table test.
Please set user
+ * and password if applicable before running the integration tests.
+ */
+@Ignore("Comment this out for running tests against a real instance of
InfluxDB")
+public class ITPutInfluxDBTest {
+private TestRunner runner;
+private InfluxDB influxDB;
+private String dbName = "test";
+private String dbUrl = "http://localhost:8086";;
+private String user = "admin";
+private String password = "admin";
+
+@Before
+public void setUp() throws Exception {
+runner = TestRunners.newTestRunner(PutInfluxDB.class);
+runner.setProperty(PutInfluxDB.DB_NAME, dbName);
+runner.setProperty(PutInfluxDB.USERNAME, user);
+runner.setProperty(PutInfluxDB.PASSWORD, password);
+runner.setProperty(PutInfluxDB.INFLUX_DB_URL, dbUrl);
+runner.setProperty(PutInfluxDB.CHARSET, "UTF-8");
+
runner.setProperty(PutInfluxDB.CONSISTENCY_LEVEL,PutInfluxDB.CONSISTENCY_LEVEL_ONE.getValue());
+runner.setProperty(PutInfluxDB.RETENTION_POLICY,"autogen");
+runner.setProperty(PutInfluxDB.MAX_RECORDS_SIZE, "1 KB");
+runner.assertValid();
+
+influxDB = InfluxDBFactory.connect(dbUrl,user,password);
+
+if ( influxDB.databaseExists(dbName) ) {
+QueryResult result = influxDB.query(new Query("DROP
measurement water", dbName));
+checkError(result);
+result = influxDB.query(new Query("DROP measurement testm",
dbName));
+checkError(result);
+} else {
+influxDB.createDatabase(dbName);
+int max = 10;
+while (!influxDB.databaseExists(dbName) && (max-- < 0)) {
+Thread.sleep(5);
+}
+if ( ! influxDB.databaseExists(dbName) ) {
+throw new Exception("unable to create database " + dbName);
+}
+}
+}
+
+protected void checkError(QueryResult result) {
+if ( result.hasError() )
+throw new IllegalStateException("Error while dropping
measurements " + result.getError());
+}
+
+@After
+public void tearDown() throws Exception {
+runner = null;
+if ( influxDB != null )
--- End diff --
Curly brackets...
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164210019
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/main/java/org/apache/nifi/processors/influxdb/AbstractInfluxDBProcessor.java
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+
+/**
+ * Abstract base class for InfluxDB processors
+ */
+abstract class AbstractInfluxDBProcessor extends AbstractProcessor {
+
+protected static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("influxdb-charset")
+.displayName("Character Set")
+.description("Specifies the character set of the document
data.")
+.required(true)
+.defaultValue("UTF-8")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor INFLUX_DB_URL = new
PropertyDescriptor.Builder()
+.name("influxdb-url")
+.displayName("InfluxDB connection url")
+.description("InfluxDB url to connect to")
+.required(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DB_NAME = new
PropertyDescriptor.Builder()
+.name("influxdb-dbname")
+.displayName("DB Name")
+.description("InfluxDB database to connect to")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("influxdb-username")
+.displayName("Username")
+.required(false)
+.description("Username for accessing InfluxDB")
+.addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("influxdb-password")
+.displayName("Password")
+.description("Password for user")
+.addValidator(Validator.VALID)
+.sensitive(true)
+.build();
+
+protected static final PropertyDescriptor MAX_RECORDS_SIZE = new
PropertyDescriptor.Builder()
+.name("influxdb-max-records-size")
+.displayName("Max size of records")
+.description("Maximum size of records allowed to be posted in
one batch")
+.defaultValue("1 MB")
+.required(true)
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
+static final Relationship REL_SUCCESS = new
Relationship.Builder().name("success")
+.description("Sucessful FlowFiles are routed to this
relationship").build();
+
+static final Relationship REL_FAILURE = new
Relationship.Builder().name("failure")
+.description("Failed FlowFiles are routed to this
relationship").build();
+
+public static final String INFLUX_DB_ERROR_MESSAGE =
"influ
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2101#discussion_r164212552
--- Diff:
nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/src/test/java/org/apache/nifi/processors/influxdb/ITPutInfluxDBTest.java
---
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.influxdb;
+
+import static org.junit.Assert.assertEquals;
+import java.util.List;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Ignore;
+import org.junit.Test;
+
+/**
+ * Integration test for InfluxDB. Please ensure that the InfluxDB is
running
+ * on local host with default port and has database test with table test.
Please set user
+ * and password if applicable before running the integration tests.
+ */
+@Ignore("Comment this out for running tests against a real instance of
InfluxDB")
+public class ITPutInfluxDBTest {
+private TestRunner runner;
+private InfluxDB influxDB;
+private String dbName = "test";
+private String dbUrl = "http://localhost:8086";;
+private String user = "admin";
+private String password = "admin";
+
+@Before
+public void setUp() throws Exception {
+runner = TestRunners.newTestRunner(PutInfluxDB.class);
+runner.setProperty(PutInfluxDB.DB_NAME, dbName);
+runner.setProperty(PutInfluxDB.USERNAME, user);
+runner.setProperty(PutInfluxDB.PASSWORD, password);
+runner.setProperty(PutInfluxDB.INFLUX_DB_URL, dbUrl);
+runner.setProperty(PutInfluxDB.CHARSET, "UTF-8");
+
runner.setProperty(PutInfluxDB.CONSISTENCY_LEVEL,PutInfluxDB.CONSISTENCY_LEVEL_ONE.getValue());
+runner.setProperty(PutInfluxDB.RETENTION_POLICY,"autogen");
+runner.setProperty(PutInfluxDB.MAX_RECORDS_SIZE, "1 KB");
+runner.assertValid();
+
+influxDB = InfluxDBFactory.connect(dbUrl,user,password);
+
+if ( influxDB.databaseExists(dbName) ) {
+QueryResult result = influxDB.query(new Query("DROP
measurement water", dbName));
+checkError(result);
+result = influxDB.query(new Query("DROP measurement testm",
dbName));
+checkError(result);
+} else {
+influxDB.createDatabase(dbName);
+int max = 10;
+while (!influxDB.databaseExists(dbName) && (max-- < 0)) {
+Thread.sleep(5);
+}
+if ( ! influxDB.databaseExists(dbName) ) {
+throw new Exception("unable to create database " + dbName);
+}
+}
+}
+
+protected void checkError(QueryResult result) {
+if ( result.hasError() )
--- End diff --
Please add curly brackets.
---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r162824952 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/pom.xml --- @@ -0,0 +1,43 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-nar-bundles +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r162824939 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-nar/pom.xml --- @@ -0,0 +1,44 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-influxdb-bundle +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r162824946 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-processors/pom.xml --- @@ -0,0 +1,66 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-influxdb-bundle +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r162824970 --- Diff: pom.xml --- @@ -1137,6 +1142,12 @@ 1.4.0-SNAPSHOT nar + +org.apache.nifi +nifi-influxdb-nar +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r162824957 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/pom.xml --- @@ -0,0 +1,43 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-nar-bundles +1.4.0-SNAPSHOT + + +nifi-influxdb-bundle +pom + + +nifi-influxdb-processors +nifi-influxdb-nar + + + + + +org.apache.nifi +nifi-influxdb-processors +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r160293228 --- Diff: nifi-nar-bundles/nifi-influxdb-bundle/nifi-influxdb-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,28 @@ +nifi-influxdb-nar +Copyright 2017 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +=== +MIT License +=== + +The following binary components are provided under the MIT License + + The MIT License (MIT) --- End diff -- same comment as before. No MIT license section belongs in NOTICE. Only LICENSE. thanks ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/2101#discussion_r160293115 --- Diff: nifi-assembly/NOTICE --- @@ -1543,6 +1543,7 @@ The following binary components are provided under the MIT License. See project (MIT License) EWS Java API (com.microsoft.ews-java-api:ews-java-api:2.0 - https://github.com/OfficeDev/ews-java-api) (MIT License) libffi (libffi-3.2.1 - http://sourceware.org/libffi/) + (MIT License) InfluxDB (org.influxdb:influxdb-java:2.7 https://github.com/influxdata/influxdb-java/) --- End diff -- there should not be any (MIT License) section in our NOTICE. These all belong in the LICENSE. I will move these other ones in the current master though ---
[GitHub] nifi pull request #2101: NIFI-4289 - InfluxDB put processor
GitHub user mans2singh opened a pull request: https://github.com/apache/nifi/pull/2101 NIFI-4289 - InfluxDB put processor Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mans2singh/nifi NIFI-4289 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2101.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2101 commit 770a939f7bd4623cb48ef6b64ed7caaeabe80f09 Author: mans2singh Date: 2017-08-19T23:06:36Z NIFI-4289 - InfluxDB put processor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
