[jira] [Updated] (NIFI-4786) Allow Expression Evaluation to Kinesis/Firehose Stream Name
[ https://issues.apache.org/jira/browse/NIFI-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Wing updated NIFI-4786: - Resolution: Fixed Fix Version/s: 1.6.0 Status: Resolved (was: Patch Available) > Allow Expression Evaluation to Kinesis/Firehose Stream Name > --- > > Key: NIFI-4786 > URL: https://issues.apache.org/jira/browse/NIFI-4786 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Dorian Bugeja >Priority: Minor > Labels: features, performance, pull-request-available > Fix For: 1.6.0 > > Attachments: > NIFI_4786___Allow_Expression_Evaluation_to_Kinesis_Firehose_Stream_Name_NIFI_4786___Allow_1.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Currenctly, the Stream name propoerty for both Firehose and Kinesis does not > support the expression language. Routing can be performed based on an > attribute of the flowfile and having a single component rather than multiple > for each one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2439: NIFI-4824 - Allow user to specific host ports on Docker Im...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2439 Alright!! Congratulations! Thank you for the effort and keep up the good work! ---
[jira] [Commented] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
[ https://issues.apache.org/jira/browse/NIFI-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347802#comment-16347802 ] ASF GitHub Bot commented on NIFI-4824: -- Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2439 Alright!! Congratulations! Thank you for the effort and keep up the good work! > Dockerhub Image - allow user to specific host ports on startup > -- > > Key: NIFI-4824 > URL: https://issues.apache.org/jira/browse/NIFI-4824 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0 >Reporter: Daniel Chaffelson >Priority: Critical > Fix For: 1.6.0 > > > As a user of the official apache/nifi Docker image, I would like to be able > to easily pass in the ports I want NiFi to start on. This is particularly > important since the 1.5.0 release, where remapping ports outside the > container cause NiFi to throw a security error. > This will be particularly useful in NiFI-1.5.0 and newer, where users are > often running multiple instances of the same version in order to test Flow > Versioning and other new features around the NiFi-Registry project. > This can be done by slightly modifying the startup scripts included in the > package to use an environment variable if available, or a default if it is > not provided. It is then very easy to pass an environment variable to the > Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4833) NIFI-4833 Add ScanHBase processor
[ https://issues.apache.org/jira/browse/NIFI-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347776#comment-16347776 ] ASF GitHub Bot commented on NIFI-4833: -- GitHub user bdesert opened a pull request: https://github.com/apache/nifi/pull/2446 NIFI-4833 Add ScanHBase processor ### Description: Add new processor ScanHBase and a test package. -- Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [v] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [v] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [v] Has your PR been rebased against the latest commit within the target branch (typically master)? - [v] Is your initial contribution a single, squashed commit? ### For code changes: - [v] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [v] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [v] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bdesert/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2446.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2446 commit 7a9dc565f43284a8535de052a812a252c6950613 Author: Ed Date: 2018-01-31T21:20:35Z NIFI-4833 Add ScanHBase processor > NIFI-4833 Add ScanHBase processor > - > > Key: NIFI-4833 > URL: https://issues.apache.org/jira/browse/NIFI-4833 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Ed Berezitsky >Assignee: Ed Berezitsky >Priority: Major > > Add ScanHBase (new) processor to retrieve records from HBase tables. > Today there are GetHBase and FetchHBaseRow. GetHBase can pull entire table or > only new rows after processor started; it also must be scheduled and doesn't > support incoming . FetchHBaseRow can pull rows with known rowkeys only. > This processor could provide functionality similar to what could be reached > by using hbase shell, defining following properties: > -scan based on range of row key IDs > -scan based on range of time stamps > -limit number of records pulled > -use filters > -reverse rows -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2446: NIFI-4833 Add ScanHBase processor
GitHub user bdesert opened a pull request: https://github.com/apache/nifi/pull/2446 NIFI-4833 Add ScanHBase processor ### Description: Add new processor ScanHBase and a test package. -- Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [v] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [v] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [v] Has your PR been rebased against the latest commit within the target branch (typically master)? - [v] Is your initial contribution a single, squashed commit? ### For code changes: - [v] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [v] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [v] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bdesert/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2446.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2446 commit 7a9dc565f43284a8535de052a812a252c6950613 Author: Ed Date: 2018-01-31T21:20:35Z NIFI-4833 Add ScanHBase processor ---
[jira] [Updated] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
[ https://issues.apache.org/jira/browse/NIFI-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4824: - Affects Version/s: (was: 1.6.0) (was: 2.0.0) > Dockerhub Image - allow user to specific host ports on startup > -- > > Key: NIFI-4824 > URL: https://issues.apache.org/jira/browse/NIFI-4824 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0 >Reporter: Daniel Chaffelson >Priority: Critical > Fix For: 1.6.0 > > > As a user of the official apache/nifi Docker image, I would like to be able > to easily pass in the ports I want NiFi to start on. This is particularly > important since the 1.5.0 release, where remapping ports outside the > container cause NiFi to throw a security error. > This will be particularly useful in NiFI-1.5.0 and newer, where users are > often running multiple instances of the same version in order to test Flow > Versioning and other new features around the NiFi-Registry project. > This can be done by slightly modifying the startup scripts included in the > package to use an environment variable if available, or a default if it is > not provided. It is then very easy to pass an environment variable to the > Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
[ https://issues.apache.org/jira/browse/NIFI-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-4824. -- Resolution: Fixed Fix Version/s: 1.6.0 > Dockerhub Image - allow user to specific host ports on startup > -- > > Key: NIFI-4824 > URL: https://issues.apache.org/jira/browse/NIFI-4824 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0 >Reporter: Daniel Chaffelson >Priority: Critical > Fix For: 1.6.0 > > > As a user of the official apache/nifi Docker image, I would like to be able > to easily pass in the ports I want NiFi to start on. This is particularly > important since the 1.5.0 release, where remapping ports outside the > container cause NiFi to throw a security error. > This will be particularly useful in NiFI-1.5.0 and newer, where users are > often running multiple instances of the same version in order to test Flow > Versioning and other new features around the NiFi-Registry project. > This can be done by slightly modifying the startup scripts included in the > package to use an environment variable if available, or a default if it is > not provided. It is then very easy to pass an environment variable to the > Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-4823) Make GetMongo JSON pretty printing optional
[ https://issues.apache.org/jira/browse/NIFI-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-4823. -- Resolution: Fixed Fix Version/s: 1.6.0 > Make GetMongo JSON pretty printing optional > --- > > Key: NIFI-4823 > URL: https://issues.apache.org/jira/browse/NIFI-4823 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > Fix For: 1.6.0 > > > Pretty printed JSON can explode the size of a MongoDB result set, > particularly on large queries. Right now, pretty printing is done all the > time and isn't configurable. It should be configurable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4747) GetHDFS Scheduling via CRON shows an error before it is scheduled
[ https://issues.apache.org/jira/browse/NIFI-4747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4747: - Resolution: Fixed Fix Version/s: 1.6.0 Status: Resolved (was: Patch Available) > GetHDFS Scheduling via CRON shows an error before it is scheduled > - > > Key: NIFI-4747 > URL: https://issues.apache.org/jira/browse/NIFI-4747 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0, 1.4.0 > Environment: Linux, Windows >Reporter: Julian Gimbel >Assignee: Pierre Villard >Priority: Major > Fix For: 1.6.0 > > Attachments: gethdfs_cron_scheduling.PNG, > gethdfs_cron_scheduling_error.png > > > We schedule a getHDFS Processor CRON driven at "0 0 15 * * ?" so that it > should run at 15:00:00 every day. If we start the processor it gives an error > message even before CRON should have triggered the Processor as shown below. > The question is why that is happening and how can it be fixed? > 2018-01-08 14:43:01,242 ERROR [StandardProcessScheduler Thread-5] > o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method > due to java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1483) > at > org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:103) > at > org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1302) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.ExecutionException: > java.lang.reflect.InvocationTargetException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:206) > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1466) > ... 9 common frames omitted > Caused by: java.lang.reflect.InvocationTargetException: null > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1306) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1302) > ... 6 common frames omitted > Caused by: java.io.IOException: PropertyDescriptor > PropertyDescriptor[Directory] has invalid value > /nifi/pele/pele2rtdp/20180108. The directory does not exist. > at > org.apache.nifi.processors.hadoop.GetHDFS.onScheduled(GetHDFS.java:242) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4761) Allow whitelisting expected Host values
[ https://issues.apache.org/jira/browse/NIFI-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347710#comment-16347710 ] Andy LoPresto commented on NIFI-4761: - Correct Robert, when running in HTTP mode, the header sanitization is disabled. > Allow whitelisting expected Host values > --- > > Key: NIFI-4761 > URL: https://issues.apache.org/jira/browse/NIFI-4761 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Matt Gilman >Assignee: Andy LoPresto >Priority: Major > Labels: configuration, header, host, security > Fix For: 1.6.0 > > > NiFi has been updated to only accept requests where the Host header contains > an expected value. Currently, the expected values are driven by the .host > properties in nifi.properties. When running behind a proxy, the value may be > the proxy host if the headers simply pass through. In this scenario, we > should offer the ability to whitelist values in case updating the proxy > configuration isn't possible. > Also, the proxy documentation in the admin guide should be updated to include > details regarding the Host name whitelisting. Also, should verify the context > path whitelisting is documented there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4835) Incorrect return type specified for registries/{registry-id}/buckets/{bucket-id}/flows
[
https://issues.apache.org/jira/browse/NIFI-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charlie Meyer updated NIFI-4835:
Description:
On
[https://github.com/apache/nifi/blob/b6117743d4c1c1a37a16ba746b9edbbdd276d69f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java#L1376]
{{response = BucketsEntity.class}}
should likely be
{{response = VersionedFlowsEntity.class}}
same copy/paste error on line 1412 also for versions, although that should be
{{VersionedFlowSnapshotMetadataSetEntity.class}}
was:
On
[https://github.com/apache/nifi/blob/b6117743d4c1c1a37a16ba746b9edbbdd276d69f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java#L1376]
{{response = BucketsEntity.class}}
should likely be
{{response = VersionedFlowsEntity.class}}
> Incorrect return type specified for
> registries/{registry-id}/buckets/{bucket-id}/flows
> --
>
> Key: NIFI-4835
> URL: https://issues.apache.org/jira/browse/NIFI-4835
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.5.0
>Reporter: Charlie Meyer
>Priority: Major
>
> On
> [https://github.com/apache/nifi/blob/b6117743d4c1c1a37a16ba746b9edbbdd276d69f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java#L1376]
> {{response = BucketsEntity.class}}
> should likely be
> {{response = VersionedFlowsEntity.class}}
>
> same copy/paste error on line 1412 also for versions, although that should be
> {{VersionedFlowSnapshotMetadataSetEntity.class}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-4835) Incorrect return type specified for registries/{registry-id}/buckets/{bucket-id}/flows
Charlie Meyer created NIFI-4835:
---
Summary: Incorrect return type specified for
registries/{registry-id}/buckets/{bucket-id}/flows
Key: NIFI-4835
URL: https://issues.apache.org/jira/browse/NIFI-4835
Project: Apache NiFi
Issue Type: Bug
Components: Flow Versioning
Affects Versions: 1.5.0
Reporter: Charlie Meyer
On
[https://github.com/apache/nifi/blob/b6117743d4c1c1a37a16ba746b9edbbdd276d69f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java#L1376]
{{response = BucketsEntity.class}}
should likely be
{{response = VersionedFlowsEntity.class}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
[ https://issues.apache.org/jira/browse/NIFI-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347667#comment-16347667 ] ASF GitHub Bot commented on NIFI-4824: -- Github user Chaffelson commented on the issue: https://github.com/apache/nifi/pull/2439 Thanks @jdye64 I think this is my first official contribution too > Dockerhub Image - allow user to specific host ports on startup > -- > > Key: NIFI-4824 > URL: https://issues.apache.org/jira/browse/NIFI-4824 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.2.0, 2.0.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Daniel Chaffelson >Priority: Critical > > As a user of the official apache/nifi Docker image, I would like to be able > to easily pass in the ports I want NiFi to start on. This is particularly > important since the 1.5.0 release, where remapping ports outside the > container cause NiFi to throw a security error. > This will be particularly useful in NiFI-1.5.0 and newer, where users are > often running multiple instances of the same version in order to test Flow > Versioning and other new features around the NiFi-Registry project. > This can be done by slightly modifying the startup scripts included in the > package to use an environment variable if available, or a default if it is > not provided. It is then very easy to pass an environment variable to the > Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2439: NIFI-4824 - Allow user to specific host ports on Docker Im...
Github user Chaffelson commented on the issue: https://github.com/apache/nifi/pull/2439 Thanks @jdye64 I think this is my first official contribution too ---
[jira] [Commented] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347664#comment-16347664 ] ASF GitHub Bot commented on NIFIREG-120: Github user Chaffelson commented on the issue: https://github.com/apache/nifi-registry/pull/89 I think this is now fixed, apologies @kevdoran > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > Fix For: 0.2.0 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing. > This could probably be ported, with some changes, from the same approach used > in the main NiFi project for continuity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #89: NIFIREG-120 Basic Docker Image Support
Github user Chaffelson commented on the issue: https://github.com/apache/nifi-registry/pull/89 I think this is now fixed, apologies @kevdoran ---
[jira] [Commented] (NIFI-4538) Add Process Group information to Search results
[ https://issues.apache.org/jira/browse/NIFI-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347642#comment-16347642 ] ASF GitHub Bot commented on NIFI-4538: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2364 @yuri1969 The introduction of the Parent group makes a lot of sense. However, I thought the idea of the Top level group was a little confusing. Technically, the Top level group is the Root group we just do not support navigating out of it. As an alternative, I wanted to suggest maybe changing that part to return the nearest versioned group (if applicable) that was just released in 1.5.0. This would be helpful in identifying which versioned flow the component exists in. Thoughts? > Add Process Group information to Search results > --- > > Key: NIFI-4538 > URL: https://issues.apache.org/jira/browse/NIFI-4538 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Burgess >Assignee: Yuri >Priority: Major > Attachments: Screenshot from 2017-12-23 21-08-45.png, Screenshot from > 2017-12-23 21-42-24.png > > > When querying for components in the Search bar, no Process Group (PG) > information is displayed. When copies of PGs are made on the canvas, the > search results can be hard to navigate, as you may jump into a different PG > than what you're looking for. > I propose adding (conditionally, based on user permissions) the immediate > parent PG name and/or ID, as well as the top-level PG. In this case I mean > top-level being the highest parent PG except root, unless the component's > immediate parent PG is root, in which case it wouldn't need to be displayed > (or could be displayed as the root PG, albeit a duplicate of the immediate). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2364: NIFI-4538 - Add Process Group information to...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2364 @yuri1969 The introduction of the Parent group makes a lot of sense. However, I thought the idea of the Top level group was a little confusing. Technically, the Top level group is the Root group we just do not support navigating out of it. As an alternative, I wanted to suggest maybe changing that part to return the nearest versioned group (if applicable) that was just released in 1.5.0. This would be helpful in identifying which versioned flow the component exists in. Thoughts? ---
[jira] [Commented] (NIFI-4246) OAuth 2 Authorization support - Client Credentials Grant
[
https://issues.apache.org/jira/browse/NIFI-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347588#comment-16347588
]
ASF GitHub Bot commented on NIFI-4246:
--
Github user jdye64 commented on the issue:
https://github.com/apache/nifi/pull/2085
@alopresto Is there anything I can do to help move this along?
> OAuth 2 Authorization support - Client Credentials Grant
>
>
> Key: NIFI-4246
> URL: https://issues.apache.org/jira/browse/NIFI-4246
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>Priority: Major
>
> If your interacting with REST endpoints on the web chances are you are going
> to run into an OAuth2 secured webservice. The IETF (Internet Engineering Task
> Force) defines 4 methods in which OAuth2 authorization can occur. This JIRA
> is focused solely on the Client Credentials Grant method defined at
> https://tools.ietf.org/html/rfc6749#section-4.4
> This implementation should provide a ControllerService in which the enduser
> can configure the credentials for obtaining the authorization grant (access
> token) from the resource owner. In turn a new property will be added to the
> InvokeHTTP processor (if it doesn't already exist from one of the other JIRA
> efforts similar to this one) where the processor can reference this
> controller service to obtain the access token and insert the appropriate HTTP
> header (Authorization: Bearer{access_token}) so that the InvokeHTTP processor
> can interact with the OAuth protected resources without having to worry about
> setting up the credentials for each InvokeHTTP processor saving time and
> complexity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2085: NIFI-4246 - Client Credentials Grant based OAuth2 Controll...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2085 @alopresto Is there anything I can do to help move this along? ---
[jira] [Commented] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
[ https://issues.apache.org/jira/browse/NIFI-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347585#comment-16347585 ] ASF GitHub Bot commented on NIFI-4824: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2439 > Dockerhub Image - allow user to specific host ports on startup > -- > > Key: NIFI-4824 > URL: https://issues.apache.org/jira/browse/NIFI-4824 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.2.0, 2.0.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Daniel Chaffelson >Priority: Critical > > As a user of the official apache/nifi Docker image, I would like to be able > to easily pass in the ports I want NiFi to start on. This is particularly > important since the 1.5.0 release, where remapping ports outside the > container cause NiFi to throw a security error. > This will be particularly useful in NiFI-1.5.0 and newer, where users are > often running multiple instances of the same version in order to test Flow > Versioning and other new features around the NiFi-Registry project. > This can be done by slightly modifying the startup scripts included in the > package to use an environment variable if available, or a default if it is > not provided. It is then very easy to pass an environment variable to the > Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2439: NIFI-4824 - Allow user to specific host ports on Do...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2439 ---
[jira] [Commented] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
[ https://issues.apache.org/jira/browse/NIFI-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347581#comment-16347581 ] ASF GitHub Bot commented on NIFI-4824: -- Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2439 +1 everything built and ran well for me. Merging now. > Dockerhub Image - allow user to specific host ports on startup > -- > > Key: NIFI-4824 > URL: https://issues.apache.org/jira/browse/NIFI-4824 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.2.0, 2.0.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Daniel Chaffelson >Priority: Critical > > As a user of the official apache/nifi Docker image, I would like to be able > to easily pass in the ports I want NiFi to start on. This is particularly > important since the 1.5.0 release, where remapping ports outside the > container cause NiFi to throw a security error. > This will be particularly useful in NiFI-1.5.0 and newer, where users are > often running multiple instances of the same version in order to test Flow > Versioning and other new features around the NiFi-Registry project. > This can be done by slightly modifying the startup scripts included in the > package to use an environment variable if available, or a default if it is > not provided. It is then very easy to pass an environment variable to the > Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2439: NIFI-4824 - Allow user to specific host ports on Docker Im...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2439 +1 everything built and ran well for me. Merging now. ---
[jira] [Commented] (NIFI-4538) Add Process Group information to Search results
[ https://issues.apache.org/jira/browse/NIFI-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347528#comment-16347528 ] ASF GitHub Bot commented on NIFI-4538: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2364 @yuri1969 Thanks! Will start reviewing... > Add Process Group information to Search results > --- > > Key: NIFI-4538 > URL: https://issues.apache.org/jira/browse/NIFI-4538 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Burgess >Assignee: Yuri >Priority: Major > Attachments: Screenshot from 2017-12-23 21-08-45.png, Screenshot from > 2017-12-23 21-42-24.png > > > When querying for components in the Search bar, no Process Group (PG) > information is displayed. When copies of PGs are made on the canvas, the > search results can be hard to navigate, as you may jump into a different PG > than what you're looking for. > I propose adding (conditionally, based on user permissions) the immediate > parent PG name and/or ID, as well as the top-level PG. In this case I mean > top-level being the highest parent PG except root, unless the component's > immediate parent PG is root, in which case it wouldn't need to be displayed > (or could be displayed as the root PG, albeit a duplicate of the immediate). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2364: NIFI-4538 - Add Process Group information to...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2364 @yuri1969 Thanks! Will start reviewing... ---
[jira] [Commented] (MINIFICPP-391) MiNiFi/NiFi versions are incorrect in DockerVerify & int test code
[ https://issues.apache.org/jira/browse/MINIFICPP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347524#comment-16347524 ] ASF GitHub Bot commented on MINIFICPP-391: -- GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/256 MINIFICPP-391 Corrected MiNiFi/NiFi versions for integration tests Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-391 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/256.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #256 commit fd2756d771321998627600b1a047b8b5fea38c7e Author: Andrew I. Christianson Date: 2018-01-31T20:02:27Z MINIFICPP-391 Corrected MiNiFi/NiFi versions for integration tests > MiNiFi/NiFi versions are incorrect in DockerVerify & int test code > -- > > Key: MINIFICPP-391 > URL: https://issues.apache.org/jira/browse/MINIFICPP-391 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > MiNiFi/NiFi versions are incorrect in DockerVerify & int test code causing > test failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #256: MINIFICPP-391 Corrected MiNiFi/NiFi versi...
GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/256 MINIFICPP-391 Corrected MiNiFi/NiFi versions for integration tests Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-391 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/256.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #256 commit fd2756d771321998627600b1a047b8b5fea38c7e Author: Andrew I. Christianson Date: 2018-01-31T20:02:27Z MINIFICPP-391 Corrected MiNiFi/NiFi versions for integration tests ---
[jira] [Created] (MINIFICPP-391) MiNiFi/NiFi versions are incorrect in DockerVerify & int test code
Andrew Christianson created MINIFICPP-391: - Summary: MiNiFi/NiFi versions are incorrect in DockerVerify & int test code Key: MINIFICPP-391 URL: https://issues.apache.org/jira/browse/MINIFICPP-391 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: Andrew Christianson Assignee: Andrew Christianson MiNiFi/NiFi versions are incorrect in DockerVerify & int test code causing test failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4834) ConsumeJMS does not scale when given more than 1 thread
[ https://issues.apache.org/jira/browse/NIFI-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347478#comment-16347478 ] ASF GitHub Bot commented on NIFI-4834: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2445 Reviewing... > ConsumeJMS does not scale when given more than 1 thread > --- > > Key: NIFI-4834 > URL: https://issues.apache.org/jira/browse/NIFI-4834 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When I run ConsumeJMS against a local broker, the performance is great. > However, if I run against a broker that is running remotely with a 75 ms > round trip time (i.e., somewhat high latency), then the performance is pretty > poor, allowing me to receive only about 30-40 msgs/sec (1-2 MB/sec). > Increasing the number of threads should result in multiple connections to the > JMS Broker, which would provide better throughput. However, when I increase > the number of Concurrent Tasks to 10, I see 10 consumers but only a single > connection being created, so the throughput is no better (in fact it's a bit > slower due to added lock contention). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2445: NIFI-4834: Updated AbstractJMSProcessor to use a separate ...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2445 Reviewing... ---
[jira] [Commented] (NIFI-4834) ConsumeJMS does not scale when given more than 1 thread
[ https://issues.apache.org/jira/browse/NIFI-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347460#comment-16347460 ] ASF GitHub Bot commented on NIFI-4834: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2445 NIFI-4834: Updated AbstractJMSProcessor to use a separate SingleConne… …ctionFactory per concurrent task instead of sharing one across the entire processor Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4834 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2445.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2445 commit cd091103e4a76e7b54e00257e5e18eaab3d389ec Author: Mark Payne Date: 2018-01-31T16:50:42Z NIFI-4834: Updated AbstractJMSProcessor to use a separate SingleConnectionFactory per concurrent task instead of sharing one across the entire processor > ConsumeJMS does not scale when given more than 1 thread > --- > > Key: NIFI-4834 > URL: https://issues.apache.org/jira/browse/NIFI-4834 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When I run ConsumeJMS against a local broker, the performance is great. > However, if I run against a broker that is running remotely with a 75 ms > round trip time (i.e., somewhat high latency), then the performance is pretty > poor, allowing me to receive only about 30-40 msgs/sec (1-2 MB/sec). > Increasing the number of threads should result in multiple connections to the > JMS Broker, which would provide better throughput. However, when I increase > the number of Concurrent Tasks to 10, I see 10 consumers but only a single > connection being created, so the throughput is no better (in fact it's a bit > slower due to added lock contention). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4834) ConsumeJMS does not scale when given more than 1 thread
[ https://issues.apache.org/jira/browse/NIFI-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-4834: - Status: Patch Available (was: Open) > ConsumeJMS does not scale when given more than 1 thread > --- > > Key: NIFI-4834 > URL: https://issues.apache.org/jira/browse/NIFI-4834 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When I run ConsumeJMS against a local broker, the performance is great. > However, if I run against a broker that is running remotely with a 75 ms > round trip time (i.e., somewhat high latency), then the performance is pretty > poor, allowing me to receive only about 30-40 msgs/sec (1-2 MB/sec). > Increasing the number of threads should result in multiple connections to the > JMS Broker, which would provide better throughput. However, when I increase > the number of Concurrent Tasks to 10, I see 10 consumers but only a single > connection being created, so the throughput is no better (in fact it's a bit > slower due to added lock contention). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2445: NIFI-4834: Updated AbstractJMSProcessor to use a se...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2445 NIFI-4834: Updated AbstractJMSProcessor to use a separate SingleConne⦠â¦ctionFactory per concurrent task instead of sharing one across the entire processor Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4834 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2445.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2445 commit cd091103e4a76e7b54e00257e5e18eaab3d389ec Author: Mark Payne Date: 2018-01-31T16:50:42Z NIFI-4834: Updated AbstractJMSProcessor to use a separate SingleConnectionFactory per concurrent task instead of sharing one across the entire processor ---
[jira] [Created] (NIFI-4834) ConsumeJMS does not scale when given more than 1 thread
Mark Payne created NIFI-4834: Summary: ConsumeJMS does not scale when given more than 1 thread Key: NIFI-4834 URL: https://issues.apache.org/jira/browse/NIFI-4834 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Assignee: Mark Payne Fix For: 1.6.0 When I run ConsumeJMS against a local broker, the performance is great. However, if I run against a broker that is running remotely with a 75 ms round trip time (i.e., somewhat high latency), then the performance is pretty poor, allowing me to receive only about 30-40 msgs/sec (1-2 MB/sec). Increasing the number of threads should result in multiple connections to the JMS Broker, which would provide better throughput. However, when I increase the number of Concurrent Tasks to 10, I see 10 consumers but only a single connection being created, so the throughput is no better (in fact it's a bit slower due to added lock contention). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
[ https://issues.apache.org/jira/browse/NIFI-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347427#comment-16347427 ] ASF GitHub Bot commented on NIFI-4824: -- Github user jdye64 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2439#discussion_r165158617 --- Diff: nifi-docker/dockerhub/README.md --- @@ -46,12 +46,22 @@ The configuration scripts are suitable for at least 1.4.0+. The minimum to run a NiFi instance is as follows: docker run --name nifi \ - -p 18080:8080 \ + -p 8080:8080 \ -d \ apache/nifi:latest -This will provide a running instance, exposing the instance UI to the host system on at port 18080, -viewable at `http://localhost:18080/nifi`. +This will provide a running instance, exposing the instance UI to the host system on at port 8080, +viewable at `http://localhost:8080/nifi`. + +You can also pass in environment variables to change the NiFi communication ports and hostname using the Docker '-e' switch as follows: + +docker run --name nifi \ + -p 9090:9090 \ + -d \ + -e NIFI_WEB_HTTP_PORT='9090' + apache/nifi:latest + +For a list of the environment variables recognised in this build, look into the .sh/secure.sh and .sh/start.sh scripts --- End diff -- Enjoy the English touch with "recognised" instead of "recognized" =) > Dockerhub Image - allow user to specific host ports on startup > -- > > Key: NIFI-4824 > URL: https://issues.apache.org/jira/browse/NIFI-4824 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.2.0, 2.0.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Daniel Chaffelson >Priority: Critical > > As a user of the official apache/nifi Docker image, I would like to be able > to easily pass in the ports I want NiFi to start on. This is particularly > important since the 1.5.0 release, where remapping ports outside the > container cause NiFi to throw a security error. > This will be particularly useful in NiFI-1.5.0 and newer, where users are > often running multiple instances of the same version in order to test Flow > Versioning and other new features around the NiFi-Registry project. > This can be done by slightly modifying the startup scripts included in the > package to use an environment variable if available, or a default if it is > not provided. It is then very easy to pass an environment variable to the > Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2439: NIFI-4824 - Allow user to specific host ports on Do...
Github user jdye64 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2439#discussion_r165158617 --- Diff: nifi-docker/dockerhub/README.md --- @@ -46,12 +46,22 @@ The configuration scripts are suitable for at least 1.4.0+. The minimum to run a NiFi instance is as follows: docker run --name nifi \ - -p 18080:8080 \ + -p 8080:8080 \ -d \ apache/nifi:latest -This will provide a running instance, exposing the instance UI to the host system on at port 18080, -viewable at `http://localhost:18080/nifi`. +This will provide a running instance, exposing the instance UI to the host system on at port 8080, +viewable at `http://localhost:8080/nifi`. + +You can also pass in environment variables to change the NiFi communication ports and hostname using the Docker '-e' switch as follows: + +docker run --name nifi \ + -p 9090:9090 \ + -d \ + -e NIFI_WEB_HTTP_PORT='9090' + apache/nifi:latest + +For a list of the environment variables recognised in this build, look into the .sh/secure.sh and .sh/start.sh scripts --- End diff -- Enjoy the English touch with "recognised" instead of "recognized" =) ---
[jira] [Updated] (NIFI-4833) NIFI-4833 Add ScanHBase processor
[ https://issues.apache.org/jira/browse/NIFI-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Berezitsky updated NIFI-4833: Description: Add ScanHBase (new) processor to retrieve records from HBase tables. Today there are GetHBase and FetchHBaseRow. GetHBase can pull entire table or only new rows after processor started; it also must be scheduled and doesn't support incoming . FetchHBaseRow can pull rows with known rowkeys only. This processor could provide functionality similar to what could be reached by using hbase shell, defining following properties: -scan based on range of row key IDs -scan based on range of time stamps -limit number of records pulled -use filters -reverse rows was: Add ScanHBase (new) processor to retrieve records from HBase tables. Today there are GetHBase and FetchHBaseRow. GetHBase can pull entire table or only new rows after processor started. FetchHBaseRow can pull records with known rowkeys. This processor could provide functionality similar to what could be reached by using hbase shell, defining following properties: -scan based on range of row key IDs -scan based on range of time stamps -limit number of records pulled -use filters -reverse rows > NIFI-4833 Add ScanHBase processor > - > > Key: NIFI-4833 > URL: https://issues.apache.org/jira/browse/NIFI-4833 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Ed Berezitsky >Assignee: Ed Berezitsky >Priority: Major > > Add ScanHBase (new) processor to retrieve records from HBase tables. > Today there are GetHBase and FetchHBaseRow. GetHBase can pull entire table or > only new rows after processor started; it also must be scheduled and doesn't > support incoming . FetchHBaseRow can pull rows with known rowkeys only. > This processor could provide functionality similar to what could be reached > by using hbase shell, defining following properties: > -scan based on range of row key IDs > -scan based on range of time stamps > -limit number of records pulled > -use filters > -reverse rows -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-388) DockerBuild.sh missing controller/ and LibExample/ dirs
[ https://issues.apache.org/jira/browse/MINIFICPP-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347378#comment-16347378 ] ASF GitHub Bot commented on MINIFICPP-388: -- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/253 It could be, but that would be a redesign of how it's currently working. I think that should be added as a separate redesign ticket rather than this bugfix. > DockerBuild.sh missing controller/ and LibExample/ dirs > --- > > Key: MINIFICPP-388 > URL: https://issues.apache.org/jira/browse/MINIFICPP-388 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > The controller/ and LibExample/ dirs are not copied to docker/minificppsource > during make docker, leading to a swift build failure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #253: MINIFICPP-388 Added missing controller/ and LibE...
Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/253 It could be, but that would be a redesign of how it's currently working. I think that should be added as a separate redesign ticket rather than this bugfix. ---
[jira] [Updated] (NIFI-4833) NIFI-4833 Add ScanHBase processor
[ https://issues.apache.org/jira/browse/NIFI-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Berezitsky updated NIFI-4833: Description: Add ScanHBase (new) processor to retrieve records from HBase tables. Today there are GetHBase and FetchHBaseRow. GetHBase can pull entire table or only new rows after processor started. FetchHBaseRow can pull records with known rowkeys. This processor could provide functionality similar to what could be reached by using hbase shell, defining following properties: -scan based on range of row key IDs -scan based on range of time stamps -limit number of records pulled -use filters -reverse rows was: Add ScanHBase (new) processor to retrieve records from HBase tables. This processor could have following features: -scan based on range of row key IDs -scan based on range of time stamps -limit number of records pulled -use filters > NIFI-4833 Add ScanHBase processor > - > > Key: NIFI-4833 > URL: https://issues.apache.org/jira/browse/NIFI-4833 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Ed Berezitsky >Assignee: Ed Berezitsky >Priority: Major > > Add ScanHBase (new) processor to retrieve records from HBase tables. > Today there are GetHBase and FetchHBaseRow. GetHBase can pull entire table or > only new rows after processor started. FetchHBaseRow can pull records with > known rowkeys. > This processor could provide functionality similar to what could be reached > by using hbase shell, defining following properties: > -scan based on range of row key IDs > -scan based on range of time stamps > -limit number of records pulled > -use filters > -reverse rows -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-390) EL generated sources from host are copied to docker build
[ https://issues.apache.org/jira/browse/MINIFICPP-390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347376#comment-16347376 ] ASF GitHub Bot commented on MINIFICPP-390: -- GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/255 MINIFICPP-390 Remove files generated from host so they will be genera… …ted in Docker environment Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-390 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/255.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #255 commit cd003c328b73e92ed18c7482b7d2d6b46b80bd52 Author: Andrew I. Christianson Date: 2018-01-31T18:48:27Z MINIFICPP-390 Remove files generated from host so they will be generated in Docker environment > EL generated sources from host are copied to docker build > - > > Key: MINIFICPP-390 > URL: https://issues.apache.org/jira/browse/MINIFICPP-390 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > EL source files generated for the host environment are copied to the docker > environment, causing build failure. These files should instead be generated > as part of the docker build process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #255: MINIFICPP-390 Remove files generated from...
GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/255 MINIFICPP-390 Remove files generated from host so they will be genera⦠â¦ted in Docker environment Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-390 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/255.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #255 commit cd003c328b73e92ed18c7482b7d2d6b46b80bd52 Author: Andrew I. Christianson Date: 2018-01-31T18:48:27Z MINIFICPP-390 Remove files generated from host so they will be generated in Docker environment ---
[jira] [Updated] (NIFI-4833) NIFI-4833 Add ScanHBase processor
[ https://issues.apache.org/jira/browse/NIFI-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Berezitsky updated NIFI-4833: Summary: NIFI-4833 Add ScanHBase processor (was: Add ScanHBase processor) > NIFI-4833 Add ScanHBase processor > - > > Key: NIFI-4833 > URL: https://issues.apache.org/jira/browse/NIFI-4833 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Ed Berezitsky >Assignee: Ed Berezitsky >Priority: Major > > Add ScanHBase (new) processor to retrieve records from HBase tables. > This processor could have following features: > -scan based on range of row key IDs > -scan based on range of time stamps > -limit number of records pulled > -use filters -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4833) Add ScanHBase processor
Ed Berezitsky created NIFI-4833: --- Summary: Add ScanHBase processor Key: NIFI-4833 URL: https://issues.apache.org/jira/browse/NIFI-4833 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Ed Berezitsky Assignee: Ed Berezitsky Add ScanHBase (new) processor to retrieve records from HBase tables. This processor could have following features: -scan based on range of row key IDs -scan based on range of time stamps -limit number of records pulled -use filters -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-390) EL generated sources from host are copied to docker build
Andrew Christianson created MINIFICPP-390: - Summary: EL generated sources from host are copied to docker build Key: MINIFICPP-390 URL: https://issues.apache.org/jira/browse/MINIFICPP-390 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: Andrew Christianson Assignee: Andrew Christianson EL source files generated for the host environment are copied to the docker environment, causing build failure. These files should instead be generated as part of the docker build process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-389) FlexLexer.h missing, causing broken Alpine docker build
[ https://issues.apache.org/jira/browse/MINIFICPP-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347278#comment-16347278 ] ASF GitHub Bot commented on MINIFICPP-389: -- GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/254 MINIFICPP-389 Added missing flex-dev dependency to prevent FlexLexer.… …h compilation error in docker build Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-389 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/254.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #254 commit 298adf85e3d37fd7dc4672276caf3ee7fd002cc4 Author: Andrew I. Christianson Date: 2018-01-31T17:56:16Z MINIFICPP-389 Added missing flex-dev dependency to prevent FlexLexer.h compilation error in docker build > FlexLexer.h missing, causing broken Alpine docker build > --- > > Key: MINIFICPP-389 > URL: https://issues.apache.org/jira/browse/MINIFICPP-389 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > Needs flex-dev in the docker build deps. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #254: MINIFICPP-389 Added missing flex-dev depe...
GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/254 MINIFICPP-389 Added missing flex-dev dependency to prevent FlexLexer.⦠â¦h compilation error in docker build Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-389 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/254.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #254 commit 298adf85e3d37fd7dc4672276caf3ee7fd002cc4 Author: Andrew I. Christianson Date: 2018-01-31T17:56:16Z MINIFICPP-389 Added missing flex-dev dependency to prevent FlexLexer.h compilation error in docker build ---
[jira] [Created] (MINIFICPP-389) FlexLexer.h missing, causing broken Alpine docker build
Andrew Christianson created MINIFICPP-389: - Summary: FlexLexer.h missing, causing broken Alpine docker build Key: MINIFICPP-389 URL: https://issues.apache.org/jira/browse/MINIFICPP-389 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: Andrew Christianson Assignee: Andrew Christianson Needs flex-dev in the docker build deps. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-388) DockerBuild.sh missing controller/ and LibExample/ dirs
[ https://issues.apache.org/jira/browse/MINIFICPP-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347262#comment-16347262 ] ASF GitHub Bot commented on MINIFICPP-388: -- GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/253 MINIFICPP-388 Added missing controller/ and LibExample/ dirs for dock… …er build Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-388 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/253.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #253 commit 414f5e8e1474b115b6e72055c83a9163c9bb74cc Author: Andrew I. Christianson Date: 2018-01-31T17:48:25Z MINIFICPP-388 Added missing controller/ and LibExample/ dirs for docker build > DockerBuild.sh missing controller/ and LibExample/ dirs > --- > > Key: MINIFICPP-388 > URL: https://issues.apache.org/jira/browse/MINIFICPP-388 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > The controller/ and LibExample/ dirs are not copied to docker/minificppsource > during make docker, leading to a swift build failure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #253: MINIFICPP-388 Added missing controller/ a...
GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/253 MINIFICPP-388 Added missing controller/ and LibExample/ dirs for dock⦠â¦er build Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-388 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/253.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #253 commit 414f5e8e1474b115b6e72055c83a9163c9bb74cc Author: Andrew I. Christianson Date: 2018-01-31T17:48:25Z MINIFICPP-388 Added missing controller/ and LibExample/ dirs for docker build ---
[jira] [Created] (MINIFICPP-388) DockerBuild.sh missing controller/ and LibExample/ dirs
Andrew Christianson created MINIFICPP-388: - Summary: DockerBuild.sh missing controller/ and LibExample/ dirs Key: MINIFICPP-388 URL: https://issues.apache.org/jira/browse/MINIFICPP-388 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: Andrew Christianson Assignee: Andrew Christianson The controller/ and LibExample/ dirs are not copied to docker/minificppsource during make docker, leading to a swift build failure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-2724) JMX Processor
[ https://issues.apache.org/jira/browse/NIFI-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347200#comment-16347200 ] ASF GitHub Bot commented on NIFI-2724: -- Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1016 @brianburnett once you make the recommended changes I can help you with the rebasing effort. > JMX Processor > - > > Key: NIFI-2724 > URL: https://issues.apache.org/jira/browse/NIFI-2724 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.0.0 > Environment: All platforms with Java RMI support for JMX >Reporter: Brian Burnett >Assignee: Andre F de Miranda >Priority: Minor > Labels: processor > Attachments: 0001-NIFI-2724-New-JMX-Processor.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > The JMX Processor feature addition includes only GetJMX without > SecurityManager capabilities at this time. The processor in its current > state is capable of pulling MBean Property and Attribute values from a remote > RMI Server. Each set of Mbean data is wrapped in a JSON formatted FlowFile > for downstream processing. It has the ability to control content with > whitelist and blacklist properties. > Possible use for this processor and the reason it was created is to help make > sense of Kafka server metrics. > Will followup with a SecurityManager Context Service and PutJMX Processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #1016: NIFI-2724 New JMX Processor
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1016 @brianburnett once you make the recommended changes I can help you with the rebasing effort. ---
[jira] [Commented] (NIFI-4747) GetHDFS Scheduling via CRON shows an error before it is scheduled
[ https://issues.apache.org/jira/browse/NIFI-4747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347185#comment-16347185 ] ASF GitHub Bot commented on NIFI-4747: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2391 > GetHDFS Scheduling via CRON shows an error before it is scheduled > - > > Key: NIFI-4747 > URL: https://issues.apache.org/jira/browse/NIFI-4747 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0, 1.4.0 > Environment: Linux, Windows >Reporter: Julian Gimbel >Assignee: Pierre Villard >Priority: Major > Attachments: gethdfs_cron_scheduling.PNG, > gethdfs_cron_scheduling_error.png > > > We schedule a getHDFS Processor CRON driven at "0 0 15 * * ?" so that it > should run at 15:00:00 every day. If we start the processor it gives an error > message even before CRON should have triggered the Processor as shown below. > The question is why that is happening and how can it be fixed? > 2018-01-08 14:43:01,242 ERROR [StandardProcessScheduler Thread-5] > o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method > due to java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1483) > at > org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:103) > at > org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1302) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.ExecutionException: > java.lang.reflect.InvocationTargetException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:206) > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1466) > ... 9 common frames omitted > Caused by: java.lang.reflect.InvocationTargetException: null > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1306) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1302) > ... 6 common frames omitted > Caused by: java.io.IOException: PropertyDescriptor > PropertyDescriptor[Directory] has invalid value > /nifi/pele/pele2rtdp/20180108. The directory does not exist. > at > org.apache.nifi.processors.hadoop.GetHDFS.onScheduled(GetHDFS.java:242) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4747) GetHDFS Scheduling via CRON shows an error before it is scheduled
[ https://issues.apache.org/jira/browse/NIFI-4747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347184#comment-16347184 ] ASF subversion and git services commented on NIFI-4747: --- Commit dc67bd2fdd762e48075dfa5edbe1a427025c6576 in nifi's branch refs/heads/master from [~pvillard] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=dc67bd2 ] NIFI-4747 - Removed directory existence check in GetHDFS This closes #2391 Signed-off-by: Jeremy Dyer > GetHDFS Scheduling via CRON shows an error before it is scheduled > - > > Key: NIFI-4747 > URL: https://issues.apache.org/jira/browse/NIFI-4747 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0, 1.4.0 > Environment: Linux, Windows >Reporter: Julian Gimbel >Assignee: Pierre Villard >Priority: Major > Attachments: gethdfs_cron_scheduling.PNG, > gethdfs_cron_scheduling_error.png > > > We schedule a getHDFS Processor CRON driven at "0 0 15 * * ?" so that it > should run at 15:00:00 every day. If we start the processor it gives an error > message even before CRON should have triggered the Processor as shown below. > The question is why that is happening and how can it be fixed? > 2018-01-08 14:43:01,242 ERROR [StandardProcessScheduler Thread-5] > o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method > due to java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1483) > at > org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:103) > at > org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1302) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.ExecutionException: > java.lang.reflect.InvocationTargetException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:206) > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1466) > ... 9 common frames omitted > Caused by: java.lang.reflect.InvocationTargetException: null > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1306) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1302) > ... 6 common frames omitted > Caused by: java.io.IOException: PropertyDescriptor > PropertyDescriptor[Directory] has invalid value > /nifi/pele/pele2rtdp/20180108. The directory does not exist. > at > org.apache.nifi.processors.hadoop.GetHDFS.onScheduled(GetHDFS.java:242) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2391: NIFI-4747 - Removed directory existence check in Ge...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2391 ---
[jira] [Commented] (MINIFICPP-387) Allow TF path to be manually specified via env var
[ https://issues.apache.org/jira/browse/MINIFICPP-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347173#comment-16347173 ] ASF GitHub Bot commented on MINIFICPP-387: -- GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/252 MINIFICPP-387 Allow TF path to be specified via environment variable Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-387 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/252.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #252 commit 65e5cb5f18e7ee8f1640a994974499a41a1be32e Author: Andrew I. Christianson Date: 2018-01-31T16:57:50Z MINIFICPP-387 Allow TF path to be specified via environment variable > Allow TF path to be manually specified via env var > -- > > Key: MINIFICPP-387 > URL: https://issues.apache.org/jira/browse/MINIFICPP-387 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Minor > > If TensorFlow is installed in a custom or unusual location, we should allow > the user to specify the path via environment variable as part of the CMake > build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #252: MINIFICPP-387 Allow TF path to be specifi...
GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/252 MINIFICPP-387 Allow TF path to be specified via environment variable Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-387 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/252.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #252 commit 65e5cb5f18e7ee8f1640a994974499a41a1be32e Author: Andrew I. Christianson Date: 2018-01-31T16:57:50Z MINIFICPP-387 Allow TF path to be specified via environment variable ---
[jira] [Created] (MINIFICPP-387) Allow TF path to be manually specified via env var
Andrew Christianson created MINIFICPP-387: - Summary: Allow TF path to be manually specified via env var Key: MINIFICPP-387 URL: https://issues.apache.org/jira/browse/MINIFICPP-387 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: Andrew Christianson Assignee: Andrew Christianson If TensorFlow is installed in a custom or unusual location, we should allow the user to specify the path via environment variable as part of the CMake build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4823) Make GetMongo JSON pretty printing optional
[ https://issues.apache.org/jira/browse/NIFI-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347143#comment-16347143 ] ASF GitHub Bot commented on NIFI-4823: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2441 > Make GetMongo JSON pretty printing optional > --- > > Key: NIFI-4823 > URL: https://issues.apache.org/jira/browse/NIFI-4823 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > Pretty printed JSON can explode the size of a MongoDB result set, > particularly on large queries. Right now, pretty printing is done all the > time and isn't configurable. It should be configurable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2441: NIFI-4823 Made pretty printing configurable in GetM...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2441 ---
[jira] [Commented] (NIFI-4823) Make GetMongo JSON pretty printing optional
[ https://issues.apache.org/jira/browse/NIFI-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347142#comment-16347142 ] ASF subversion and git services commented on NIFI-4823: --- Commit 6e7dfb9935622a5215115df93503ef4bfba75949 in nifi's branch refs/heads/master from [~mike.thomsen] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6e7dfb9 ] NIFI-4823 Made pretty printing configurable in GetMongo. This closes #2441 Signed-off-by: Jeremy Dyer > Make GetMongo JSON pretty printing optional > --- > > Key: NIFI-4823 > URL: https://issues.apache.org/jira/browse/NIFI-4823 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > Pretty printed JSON can explode the size of a MongoDB result set, > particularly on large queries. Right now, pretty printing is done all the > time and isn't configurable. It should be configurable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4823) Make GetMongo JSON pretty printing optional
[ https://issues.apache.org/jira/browse/NIFI-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347091#comment-16347091 ] ASF GitHub Bot commented on NIFI-4823: -- Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2441 +1 looks good to me. Will merge now > Make GetMongo JSON pretty printing optional > --- > > Key: NIFI-4823 > URL: https://issues.apache.org/jira/browse/NIFI-4823 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > Pretty printed JSON can explode the size of a MongoDB result set, > particularly on large queries. Right now, pretty printing is done all the > time and isn't configurable. It should be configurable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2441: NIFI-4823 Made pretty printing configurable in GetMongo.
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2441 +1 looks good to me. Will merge now ---
[jira] [Commented] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347078#comment-16347078 ] ASF GitHub Bot commented on NIFIREG-120: Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/89 @Chaffelson, In order to review the PR and merge it to master, we'll need to be able to rebase on ASF master. When I tried this, I got failures: ``` First, rewinding head to replay your work on top of it... Applying: NIFIREG-120 Ported simplified Dockerhub build from NiFi Applying: NIFIREG-120 Added a readme for usage instructions Applying: Added license header for contrib check Applying: Included the secure mode options from the main NiFi Docker project Applying: fix script Applying: add xmlstarlet Applying: expose ssl port Applying: fix secure script Applying: update start.sh Applying: NIFIREG-120 Ported simplified Dockerhub build from NiFi Using index info to reconstruct a base tree... M pom.xml Falling back to patching base and 3-way merge... Auto-merging nifi-registry-docker/pom.xml CONFLICT (add/add): Merge conflict in nifi-registry-docker/pom.xml Auto-merging nifi-registry-docker/dockerhub/sh/start.sh CONFLICT (add/add): Merge conflict in nifi-registry-docker/dockerhub/sh/start.sh Auto-merging nifi-registry-docker/dockerhub/sh/common.sh CONFLICT (add/add): Merge conflict in nifi-registry-docker/dockerhub/sh/common.sh Auto-merging nifi-registry-docker/dockerhub/Dockerfile CONFLICT (add/add): Merge conflict in nifi-registry-docker/dockerhub/Dockerfile error: Failed to merge in the changes. Patch failed at 0010 NIFIREG-120 Ported simplified Dockerhub build from NiFi The copy of the patch that failed is found in: .git/rebase-apply/patch When you have resolved this problem, run "git rebase --continue". If you prefer to skip this patch, run "git rebase --skip" instead. To check out the original branch and stop rebasing, run "git rebase --abort". ``` It looks like there are some commits that are duplicated due to merging a branch of this branch into itself, possibly? Unfortunately, I'm not able to work out on my own what the history of this branch should look like. It would be helpful if you could rebase and, ideally, squash this into a single commit. Let me know if you need assistance with this. Thanks! > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > Fix For: 0.2.0 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing. > This could probably be ported, with some changes, from the same approach used > in the main NiFi project for continuity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #89: NIFIREG-120 Basic Docker Image Support
Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/89 @Chaffelson, In order to review the PR and merge it to master, we'll need to be able to rebase on ASF master. When I tried this, I got failures: ``` First, rewinding head to replay your work on top of it... Applying: NIFIREG-120 Ported simplified Dockerhub build from NiFi Applying: NIFIREG-120 Added a readme for usage instructions Applying: Added license header for contrib check Applying: Included the secure mode options from the main NiFi Docker project Applying: fix script Applying: add xmlstarlet Applying: expose ssl port Applying: fix secure script Applying: update start.sh Applying: NIFIREG-120 Ported simplified Dockerhub build from NiFi Using index info to reconstruct a base tree... M pom.xml Falling back to patching base and 3-way merge... Auto-merging nifi-registry-docker/pom.xml CONFLICT (add/add): Merge conflict in nifi-registry-docker/pom.xml Auto-merging nifi-registry-docker/dockerhub/sh/start.sh CONFLICT (add/add): Merge conflict in nifi-registry-docker/dockerhub/sh/start.sh Auto-merging nifi-registry-docker/dockerhub/sh/common.sh CONFLICT (add/add): Merge conflict in nifi-registry-docker/dockerhub/sh/common.sh Auto-merging nifi-registry-docker/dockerhub/Dockerfile CONFLICT (add/add): Merge conflict in nifi-registry-docker/dockerhub/Dockerfile error: Failed to merge in the changes. Patch failed at 0010 NIFIREG-120 Ported simplified Dockerhub build from NiFi The copy of the patch that failed is found in: .git/rebase-apply/patch When you have resolved this problem, run "git rebase --continue". If you prefer to skip this patch, run "git rebase --skip" instead. To check out the original branch and stop rebasing, run "git rebase --abort". ``` It looks like there are some commits that are duplicated due to merging a branch of this branch into itself, possibly? Unfortunately, I'm not able to work out on my own what the history of this branch should look like. It would be helpful if you could rebase and, ideally, squash this into a single commit. Let me know if you need assistance with this. Thanks! ---
[jira] [Commented] (NIFI-4831) InvokeHTTP Processor overwrites filename attribute when doing a POST
[ https://issues.apache.org/jira/browse/NIFI-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347053#comment-16347053 ] Tyler Bryant commented on NIFI-4831: I am on NiFi 0.7.3 (I know it is an old version, and maybe i just need to update). > InvokeHTTP Processor overwrites filename attribute when doing a POST > > > Key: NIFI-4831 > URL: https://issues.apache.org/jira/browse/NIFI-4831 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tyler Bryant >Priority: Minor > > The 'filename' attribute gets overwritten when doing a POST in an InvokeHTTP > Processor. I can seem to get around it by putting a 'fileName' attribute in > before the InvokeHTTP Processor, and then it correctly keeps the 'filename'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4831) InvokeHTTP Processor overwrites filename attribute when doing a POST
[ https://issues.apache.org/jira/browse/NIFI-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347014#comment-16347014 ] zenfenan edited comment on NIFI-4831 at 1/31/18 3:31 PM: - Which NiFi version are you using? Can you be a bit more specific on how it happens? I can't seem to reproduce the issue on my end. I tried with GenerateFlowFile as well as GetFile and made a POST request and when the request is success, it goes to "Response" relationship and the filename is intact. It is not overwritten. was (Author: sivaprasanna): Can you be a bit more specific on how it happens? I can't seem to reproduce the issue on my end. I tried with GenerateFlowFile as well as GetFile and made a POST request and when the request is success, it goes to "Response" relationship and the filename is intact. It is not overwritten. > InvokeHTTP Processor overwrites filename attribute when doing a POST > > > Key: NIFI-4831 > URL: https://issues.apache.org/jira/browse/NIFI-4831 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tyler Bryant >Priority: Minor > > The 'filename' attribute gets overwritten when doing a POST in an InvokeHTTP > Processor. I can seem to get around it by putting a 'fileName' attribute in > before the InvokeHTTP Processor, and then it correctly keeps the 'filename'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4831) InvokeHTTP Processor overwrites filename attribute when doing a POST
[ https://issues.apache.org/jira/browse/NIFI-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347014#comment-16347014 ] zenfenan commented on NIFI-4831: Can you be a bit more specific on how it happens? I can't seem to reproduce the issue on my end. I tried with GenerateFlowFile as well as GetFile and made a POST request and when the request is success, it goes to "Response" relationship and the filename is intact. It is not overwritten. > InvokeHTTP Processor overwrites filename attribute when doing a POST > > > Key: NIFI-4831 > URL: https://issues.apache.org/jira/browse/NIFI-4831 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tyler Bryant >Priority: Minor > > The 'filename' attribute gets overwritten when doing a POST in an InvokeHTTP > Processor. I can seem to get around it by putting a 'fileName' attribute in > before the InvokeHTTP Processor, and then it correctly keeps the 'filename'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFIREG-133) [Documentation] Explain how variables are captured when versioning a PG
[ https://issues.apache.org/jira/browse/NIFIREG-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Lim reassigned NIFIREG-133: -- Assignee: Andrew Lim > [Documentation] Explain how variables are captured when versioning a PG > --- > > Key: NIFIREG-133 > URL: https://issues.apache.org/jira/browse/NIFIREG-133 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Pierre Villard >Assignee: Andrew Lim >Priority: Minor > > Context: > I have a process group PG that contains an embedded process group PG1. I have > a variable defined at PG level that is referenced in a PG1's processor. When > versioning both PG and PG1 in NiFi Registry, a copy of the variable defined > at PG level will be created at PG1 level. > Consequently, when importing PG in another environment, the variable needs to > be modified at PG1 level because the one created at PG1 level overwrites the > one initially defined at PG level. > > Would be nice to add this behavior in the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-2724) JMX Processor
[ https://issues.apache.org/jira/browse/NIFI-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346915#comment-16346915 ] ASF GitHub Bot commented on NIFI-2724: -- Github user brianburnett commented on the issue: https://github.com/apache/nifi/pull/1016 It was my intent to finish pushing this through. I can make the recommended changes but may need some assistance rebasing to the current version. > JMX Processor > - > > Key: NIFI-2724 > URL: https://issues.apache.org/jira/browse/NIFI-2724 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.0.0 > Environment: All platforms with Java RMI support for JMX >Reporter: Brian Burnett >Assignee: Andre F de Miranda >Priority: Minor > Labels: processor > Attachments: 0001-NIFI-2724-New-JMX-Processor.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > The JMX Processor feature addition includes only GetJMX without > SecurityManager capabilities at this time. The processor in its current > state is capable of pulling MBean Property and Attribute values from a remote > RMI Server. Each set of Mbean data is wrapped in a JSON formatted FlowFile > for downstream processing. It has the ability to control content with > whitelist and blacklist properties. > Possible use for this processor and the reason it was created is to help make > sense of Kafka server metrics. > Will followup with a SecurityManager Context Service and PutJMX Processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4832) How to pass basic authentication header while connecting to secured websocket
Anil created NIFI-4832: -- Summary: How to pass basic authentication header while connecting to secured websocket Key: NIFI-4832 URL: https://issues.apache.org/jira/browse/NIFI-4832 Project: Apache NiFi Issue Type: Bug Components: Configuration, Extensions Affects Versions: 1.3.0 Reporter: Anil Fix For: 1.4.0, 1.3.0, 1.2.0 I have a requirement to connect to secured websocket (wss) using basic authentication providing username and password. As per my understanding current "ConnectWebSocket" unable to basic authentication configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #1016: NIFI-2724 New JMX Processor
Github user brianburnett commented on the issue: https://github.com/apache/nifi/pull/1016 It was my intent to finish pushing this through. I can make the recommended changes but may need some assistance rebasing to the current version. ---
[jira] [Commented] (NIFI-4367) InvokedScriptedProcessor does not support scripted processor that extends AbstractProcessor
[ https://issues.apache.org/jira/browse/NIFI-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346780#comment-16346780 ] Marco Gaido commented on NIFI-4367: --- [~frett27] I sent you a PR, thanks. > InvokedScriptedProcessor does not support scripted processor that extends > AbstractProcessor > --- > > Key: NIFI-4367 > URL: https://issues.apache.org/jira/browse/NIFI-4367 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Patrice Freydiere >Priority: Major > Labels: InvokeScriptedProcessor, validation > > InvokeScriptedProcessor pass his ValidationContext to the inner script, > validate call > InvokeScriptedProcessor Line 465 :final > Collection instanceResults = instance.validate(context); > > The problem is that the invokedscript pass the ScriptFile PropertyDescriptor > that is validated, if the script derived from the > AbstractConfigurableComponent, (because the AbstractConfigurableComponent > validate all the context properties). > The context should be refined to remove the InvokeScriptedProcessor > Properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346734#comment-16346734
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036667
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
--- End diff --
Good Point, I think I will update the description as well as allow for the
inclusion of the JSON directly in a property. This goes well with allowing
someone to pass in a configuration as part of a flow.
> Realistic Time Series Processor Simulator
> -
>
> Key: NIFI-4164
> URL: https://issues.apache.org/jira/browse/NIFI-4164
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Chris Herrera
>Assignee: Chris Herrera
>Priority: Minor
> Labels: features
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to validate several flows that deal with sensor data, it would be
> good to have a built in time series simulator processor that generates data
> and can send it out via a flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346728#comment-16346728 ] ASF GitHub Bot commented on NIFIREG-120: Github user Chaffelson commented on the issue: https://github.com/apache/nifi-registry/pull/89 Argh I think I've broken this PR by rebaselineing my fork to test something else. Let me know if I need to recreate this. > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > Fix For: 0.2.0 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing. > This could probably be ported, with some changes, from the same approach used > in the main NiFi project for continuity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346737#comment-16346737
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036864
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
+.build();
+
+private List descriptors;
+
+private Set relationships;
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+
+final List descriptors = new ArrayList<>();
+
+descriptors.add(SIMULATOR_CONFIG);
+descriptors.add(PRINT_HEADER);
+
+this.descriptors = Collections.unmodifiableList(descriptors);
+
+final Set relationships = new HashSet<>();
+relationships.add(SUCCESS);
+this.relationships = Collections.unmodifiableSet(relationships);
+}
+
+@Override
+public Set getRel
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346736#comment-16346736
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036821
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
+.build();
+
+private List descriptors;
+
+private Set relationships;
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+
+final List descriptors = new ArrayList<>();
+
+descriptors.add(SIMULATOR_CONFIG);
+descriptors.add(PRINT_HEADER);
+
+this.descriptors = Collections.unmodifiableList(descriptors);
+
+final Set relationships = new HashSet<>();
+relationships.add(SUCCESS);
+this.relationships = Collections.unmodifiableSet(relationships);
+}
+
+@Override
+public Set getRel
[GitHub] nifi pull request #1997: NIFI-4164 Adding a realistic time simulator process...
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036821
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
+.build();
+
+private List descriptors;
+
+private Set relationships;
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+
+final List descriptors = new ArrayList<>();
+
+descriptors.add(SIMULATOR_CONFIG);
+descriptors.add(PRINT_HEADER);
+
+this.descriptors = Collections.unmodifiableList(descriptors);
+
+final Set relationships = new HashSet<>();
+relationships.add(SUCCESS);
+this.relationships = Collections.unmodifiableSet(relationships);
+}
+
+@Override
+public Set getRelationships() {
+return this.relationships;
+}
+
+@Override
+public final List
getSupportedPropertyDescriptors() {
+return descriptors;
+}
+
+@Override
+public void onProper
[GitHub] nifi pull request #1997: NIFI-4164 Adding a realistic time simulator process...
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036864
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
+.build();
+
+private List descriptors;
+
+private Set relationships;
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+
+final List descriptors = new ArrayList<>();
+
+descriptors.add(SIMULATOR_CONFIG);
+descriptors.add(PRINT_HEADER);
+
+this.descriptors = Collections.unmodifiableList(descriptors);
+
+final Set relationships = new HashSet<>();
+relationships.add(SUCCESS);
+this.relationships = Collections.unmodifiableSet(relationships);
+}
+
+@Override
+public Set getRelationships() {
+return this.relationships;
+}
+
+@Override
+public final List
getSupportedPropertyDescriptors() {
+return descriptors;
+}
+
+@Override
+public void onProper
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346735#comment-16346735
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036757
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
--- End diff --
Good Point.
> Realistic Time Series Processor Simulator
> -
>
> Key: NIFI-4164
> URL: https://issues.apache.org/jira/browse/NIFI-4164
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Chris Herrera
>Assignee: Chris Herrera
>Priority: Minor
> Labels: features
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to validate several flows that deal with sensor data, it would be
> good to have a built in time series simulator processor that generates data
> and
[GitHub] nifi pull request #1997: NIFI-4164 Adding a realistic time simulator process...
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036757
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
--- End diff --
Good Point.
---
[GitHub] nifi pull request #1997: NIFI-4164 Adding a realistic time simulator process...
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036667
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
--- End diff --
Good Point, I think I will update the description as well as allow for the
inclusion of the JSON directly in a property. This goes well with allowing
someone to pass in a configuration as part of a flow.
---
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346731#comment-16346731
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036464
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
--- End diff --
Its a good point I think its a valuable addition. I will modify this.
> Realistic Time Series Processor Simulator
> -
>
> Key: NIFI-4164
> URL: https://issues.apache.org/jira/browse/NIFI-4164
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Chris Herrera
>Assignee: Chris Herrera
>Priority: Minor
> Labels: features
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to validate several flows that deal with sensor data, it would be
> good to have a built in time series simulator processor that generates data
> and can send it out via a flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #1997: NIFI-4164 Adding a realistic time simulator process...
Github user cherrera2001 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165036464
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
--- End diff --
Its a good point I think its a valuable addition. I will modify this.
---
[GitHub] nifi-registry issue #89: NIFIREG-120 Basic Docker Image Support
Github user Chaffelson commented on the issue: https://github.com/apache/nifi-registry/pull/89 Argh I think I've broken this PR by rebaselineing my fork to test something else. Let me know if I need to recreate this. ---
[jira] [Commented] (NIFI-2724) JMX Processor
[ https://issues.apache.org/jira/browse/NIFI-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346721#comment-16346721 ] ASF GitHub Bot commented on NIFI-2724: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/1016 @pvillard31 If he doesn't respond, I can take a stab at addressing the feedback so this can be closed out. > JMX Processor > - > > Key: NIFI-2724 > URL: https://issues.apache.org/jira/browse/NIFI-2724 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.0.0 > Environment: All platforms with Java RMI support for JMX >Reporter: Brian Burnett >Assignee: Andre F de Miranda >Priority: Minor > Labels: processor > Attachments: 0001-NIFI-2724-New-JMX-Processor.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > The JMX Processor feature addition includes only GetJMX without > SecurityManager capabilities at this time. The processor in its current > state is capable of pulling MBean Property and Attribute values from a remote > RMI Server. Each set of Mbean data is wrapped in a JSON formatted FlowFile > for downstream processing. It has the ability to control content with > whitelist and blacklist properties. > Possible use for this processor and the reason it was created is to help make > sense of Kafka server metrics. > Will followup with a SecurityManager Context Service and PutJMX Processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346707#comment-16346707
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165031960
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
+.build();
+
+private List descriptors;
+
+private Set relationships;
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+
+final List descriptors = new ArrayList<>();
+
+descriptors.add(SIMULATOR_CONFIG);
+descriptors.add(PRINT_HEADER);
+
+this.descriptors = Collections.unmodifiableList(descriptors);
+
+final Set relationships = new HashSet<>();
+relationships.add(SUCCESS);
+this.relationships = Collections.unmodifiableSet(relationships);
+}
+
+@Override
+public Set getRela
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346700#comment-16346700
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165031639
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
+.build();
+
+private List descriptors;
+
+private Set relationships;
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+
+final List descriptors = new ArrayList<>();
+
+descriptors.add(SIMULATOR_CONFIG);
+descriptors.add(PRINT_HEADER);
+
+this.descriptors = Collections.unmodifiableList(descriptors);
+
+final Set relationships = new HashSet<>();
+relationships.add(SUCCESS);
+this.relationships = Collections.unmodifiableSet(relationships);
+}
+
+@Override
+public Set getRela
[GitHub] nifi issue #1016: NIFI-2724 New JMX Processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/1016 @pvillard31 If he doesn't respond, I can take a stab at addressing the feedback so this can be closed out. ---
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346709#comment-16346709
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165030668
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
--- End diff --
AllowableValue objects are a better way to go here because they provide a
value and a display label.
> Realistic Time Series Processor Simulator
> -
>
> Key: NIFI-4164
> URL: https://issues.apache.org/jira/browse/NIFI-4164
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Chris Herrera
>Assignee: Chris Herrera
>Priority: Minor
> Labels: features
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to validate several flows that deal with sensor data, it would be
> good to have a built in time series simulator processor that generates data
> and can send it out via a flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346715#comment-16346715 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165032832 --- Diff: pom.xml --- @@ -1300,6 +1300,12 @@ 1.4.0-SNAPSHOT nar + +org.apache.nifi +nifi-simulator-nar +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346697#comment-16346697 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165029072 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-nar/pom.xml --- @@ -0,0 +1,41 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-simulator-bundle +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346716#comment-16346716 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165033444 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/scala/com/apache/nifi/processors/simulator/SimController.scala --- @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.apache.nifi.processors.simulator + +import java.io.File +import be.cetic.tsimulus.config.Configuration +import be.cetic.tsimulus.timeseries._ +import com.github.nscala_time.time.Imports._ +import spray.json._ +import scala.io.Source + +object SimController --- End diff -- Since the Scala APIs can be called from Java, is there any reason to have this written in Scala? > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346704#comment-16346704
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165030487
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
--- End diff --
Some additional details describing how this would impact the output, such
as a minimalist sample might be really helpful to users.
> Realistic Time Series Processor Simulator
> -
>
> Key: NIFI-4164
> URL: https://issues.apache.org/jira/browse/NIFI-4164
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Chris Herrera
>Assignee: Chris Herrera
>Priority: Minor
> Labels: features
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to validate several flows that deal with sensor data, it would be
> good to have a built in time series simulator processor that generates data
> and can send it out via a flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346714#comment-16346714
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165033904
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/test/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowfileTest.java
---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.apache.nifi.processors.simulator;
+
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.Before;
+import org.junit.Test;
+
+
+public class GenerateTimeSeriesFlowfileTest {
+
+private TestRunner testRunner;
+
+@Before
+public void init() {
+testRunner =
TestRunners.newTestRunner(GenerateTimeSeriesFlowFile.class);
+}
+
+@Test
+public void testGenerateTimeSeries() {
+TestRunner runner = TestRunners.newTestRunner(new
GenerateTimeSeriesFlowFile());
+runner.setProperty(GenerateTimeSeriesFlowFile.SIMULATOR_CONFIG,
getClass().getResource("/configs/basicConfig.json").getPath());
+runner.assertValid();
+
+runner.run();
+
+runner.assertTransferCount(GenerateTimeSeriesFlowFile.SUCCESS, 1);
+}
+
+@Test
+public void testInvalidConfig() {
+TestRunner runner = TestRunners.newTestRunner(new
GenerateTimeSeriesFlowFile());
+runner.setProperty(GenerateTimeSeriesFlowFile.SIMULATOR_CONFIG,
"/my/invalid/path");
+runner.assertNotValid();
+}
+
+@Test
+public void testFalseHeaderCreation() {
+TestRunner runner = TestRunners.newTestRunner(new
GenerateTimeSeriesFlowFile());
+runner.setProperty(GenerateTimeSeriesFlowFile.PRINT_HEADER,
"false");
+runner.setProperty(GenerateTimeSeriesFlowFile.SIMULATOR_CONFIG,
getClass().getResource("/configs/unitTestConfig.json").getPath());
+runner.run();
+runner.assertTransferCount(GenerateTimeSeriesFlowFile.SUCCESS, 1);
+
runner.getFlowFilesForRelationship(GenerateTimeSeriesFlowFile.SUCCESS).get(0).assertContentEquals("test,2016-01-01T00:00:00.000,17.5");
--- End diff --
You should subclass the processor if you need to inject that timestamp in
there somehow.
> Realistic Time Series Processor Simulator
> -
>
> Key: NIFI-4164
> URL: https://issues.apache.org/jira/browse/NIFI-4164
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Chris Herrera
>Assignee: Chris Herrera
>Priority: Minor
> Labels: features
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to validate several flows that deal with sensor data, it would be
> good to have a built in time series simulator processor that generates data
> and can send it out via a flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346706#comment-16346706
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165030357
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
--- End diff --
I think you're going to need to get a little more detailed here. Depending
on how big the config file is, you might be better off trying to make it
constructable from the processor's properties.
> Realistic Time Series Processor Simulator
> -
>
> Key: NIFI-4164
> URL: https://issues.apache.org/jira/browse/NIFI-4164
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Chris Herrera
>Assignee: Chris Herrera
>Priority: Minor
> Labels: features
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to validate several flows that deal with sensor data, it would be
> good to have a built in time series simulator processor that generates data
> and can send it out via a flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[
https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346701#comment-16346701
]
ASF GitHub Bot commented on NIFI-4164:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1997#discussion_r165031839
--- Diff:
nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.apache.nifi.processors.simulator;
+
+import be.cetic.tsimulus.config.Configuration;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.LocalDateTime;
+import scala.Some;
+import scala.Tuple3;
+import scala.collection.JavaConverters;
+
+import java.util.List;
+import java.util.Set;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.ArrayList;
+
+@Tags({"Simulator, Timeseries, IOT, Testing"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@CapabilityDescription("Generates realistic time series data using the
TSimulus time series generator, and places the values into the flowfile in a
CSV format.")
+public class GenerateTimeSeriesFlowFile extends AbstractProcessor {
+
+private Configuration simConfig = null;
+private boolean isTest = false;
+
+public static final PropertyDescriptor SIMULATOR_CONFIG = new
PropertyDescriptor
+.Builder().name("SIMULATOR_CONFIG")
+.displayName("Simulator Configuration File")
+.description("The JSON configuration file to use to configure
TSimulus")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PRINT_HEADER = new
PropertyDescriptor
+.Builder().name("PRINT_HEADER")
+.displayName("Print Header")
+.description("Directs the processor whether to print a header
line or not.")
+.required(true)
+.allowableValues("true", "false")
+.defaultValue("false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final Relationship SUCCESS = new Relationship.Builder()
+.name("Success")
+.description("When the flowfile is successfully generated")
+.build();
+
+private List descriptors;
+
+private Set relationships;
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+
+final List descriptors = new ArrayList<>();
+
+descriptors.add(SIMULATOR_CONFIG);
+descriptors.add(PRINT_HEADER);
+
+this.descriptors = Collections.unmodifiableList(descriptors);
+
+final Set relationships = new HashSet<>();
+relationships.add(SUCCESS);
+this.relationships = Collections.unmodifiableSet(relationships);
+}
+
+@Override
+public Set getRela
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346696#comment-16346696 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165029109 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-nar/pom.xml --- @@ -0,0 +1,41 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-simulator-bundle +1.4.0-SNAPSHOT + + +nifi-simulator-nar +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
