[GitHub] nifi pull request: NIFI-627 incorporates mwmoser patch and some mi...

[mosermw]

Github user mosermw commented on the pull request:

https://github.com/apache/nifi/pull/274#issuecomment-195870964
  
Reviewed and +1 from me.  Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[trkurc]

Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195862994
  
Also, @jvwing I fixed that misspelling of unknown when I was hacking on 
RunNiFi


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[PuspenduBanerjee]

Github user PuspenduBanerjee commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195862566
  
@trkurc  Oh Man! Thanks for your guidance. Any chance of using power-shell 
in future?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[trkurc]

Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195862019
  
(See the section labeled "Processing quotation marks ". ). So bizarre.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[trkurc]

Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195861949
  
@PuspenduBanerjee sadly, no, no it is not. 
https://technet.microsoft.com/en-us/library/bb490880.aspx


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[trkurc]

Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195861254
  
push my changes up to a fork, in case anyone was interested 
(https://github.com/trkurc/nifi/commit/5ebf228a9ba5c5ce9f12189988920f94923e6210 
from https://github.com/trkurc/nifi/commits/NIFI-1481)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[trkurc]

Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195861175
  
The pid is not being recorded in the nifi.pid file after I run RunNiFi with 
start or run, haven't been able to diagnose why yet. @markap14 may have an idea.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[trkurc]

Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195861132
  
I discovered the same issue with the pid on windows 10. My fix was to use 
quotes a bit differently on the cmd.exe line (cmd.exe /C ""%JAVA_EXE%" 
%JAVA_PARAMS% %BOOTSTRAP_ACTION%" ) and (set JAVA_PARAMS=-cp 
"%TOOLS_JAR%";%CONF_DIR%;%LIB_DIR%\* -Xms12m -Xmx24m %JAVA_ARGS% 
org.apache.nifi.bootstrap.RunNiFi)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]

[jvwing]

Github user jvwing commented on the pull request:

https://github.com/apache/nifi/pull/218#issuecomment-195858721
  
I have a [suggested fix/hack for dealing with spaces in the Java path in 
env-nifi.bat](https://github.com/jvwing/nifi/commit/2094080675d0d976fd4cd883d9f97cd10994ba69)).
  The fix converts the TOOLS_JAR path into a short 8.3 path without spaces, so 
the eventual call to NiFi has only one quoted parameter.  The expanded 
TOOLS_JAR path was not working due to spaces, and two quoted paths make the 
whole thing blow up for reasons I do not comprehend.

However, this was not enough to get the env feature running on Windows 10.  
Running it outputs 

```
17:58:32.959 [main] DEBUG o.a.n.b.NotificationServiceManager - Found 0 
service elements
17:58:32.963 [main] INFO  o.a.n.b.NotificationServiceManager - Successfully 
loaded the following 0 services: []
17:58:32.965 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - Registered no 
Notification Services for Notification Type NIFI_STARTED
17:58:32.987 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - Registered no 
Notification Services for Notification Type NIFI_STOPPED
17:58:32.989 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - Registered no 
Notification Services for Notification Type NIFI_DIED
17:58:33.012 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File: 
bin\nifi.pid
17:58:33.013 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File: 
bin\nifi.pid
17:58:33.040 [main] DEBUG org.apache.nifi.bootstrap.Command - Properties: 
{port=40930}
17:58:33.041 [main] DEBUG org.apache.nifi.bootstrap.Command - Pinging 40930
17:58:33.078 [main] DEBUG org.apache.nifi.bootstrap.Command - Sent PING 
command
17:58:33.080 [main] DEBUG org.apache.nifi.bootstrap.Command - PING 
response: PING
17:58:33.081 [main] INFO  org.apache.nifi.bootstrap.Command - Apache NiFi 
is not running
```

Where the last line "Apache NiFi is not running" matches RunNiFi.java line 
559, where the env() method quits after not finding a PID.  Yes, NiFi was 
running when I tested this.  In comparison, the `status-nifi.bat` outputs

```
17:49:49.660 [main] DEBUG o.a.n.b.NotificationServiceManager - Found 0 
service elements
17:49:49.664 [main] INFO  o.a.n.b.NotificationServiceManager - Successfully 
loaded the following 0 services: []
17:49:49.666 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - Registered no 
Notification Services for Notification Type NIFI_STARTED
17:49:49.690 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - Registered no 
Notification Services for Notification Type NIFI_STOPPED
17:49:49.691 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - Registered no 
Notification Services for Notification Type NIFI_DIED
17:49:49.719 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File: 
bin\nifi.pid
17:49:49.723 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File: 
bin\nifi.pid
17:49:49.749 [main] DEBUG org.apache.nifi.bootstrap.Command - Properties: 
{port=40930}
17:49:49.752 [main] DEBUG org.apache.nifi.bootstrap.Command - Pinging 40930
17:49:49.798 [main] DEBUG org.apache.nifi.bootstrap.Command - Sent PING 
command
17:49:49.801 [main] DEBUG org.apache.nifi.bootstrap.Command - PING 
response: PING
17:49:49.804 [main] INFO  org.apache.nifi.bootstrap.Command - Apache NiFi 
is currently running, listening to Bootstrap on port 40930, PID=unknkown
```

Which recognizes that NiFi is running based on `isRespondingToPing()` (line 
532), and describes the PID as "unknkown".  Since it looks like you use the PID 
in the env method, I'm not immediately sure what to suggest next.  It might be 
possible to work out an "if Windows, do this to get the PID" flow.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1516 - AWS DynamoDB Get/Put/Delete Process...

[mans2singh]

Github user mans2singh commented on the pull request:

https://github.com/apache/nifi/pull/224#issuecomment-195833198
  
@apiri - I've rebased the branch to the current master and have resolve the 
conflicts.  However, on rebasing there were many updates from the master so I 
could not squash my changes into one commit.  If you see any conflicts, please 
let me know.  Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Reg: starting and Stopping processor

[Matt Gilman]

Please check out the last section in the link I referenced. It explains what 
your seeing with regards to NiFi's optimistic locking strategy with the flow.

Matt

Sent from my iPhone

> On Mar 12, 2016, at 1:28 PM, Sourav Gulati  
> wrote:
> 
> Matt,
> 
> I tried it:
> curl -i -X PUT -H 'Content-Type: application/json' -d 
> '{"revision":{"clientId":"561b4443-8be5-43ee-b1f7-b07740e87a0f"},"processor":{"id":"fad172d5-5d29-4a8b-923f-e888af039c52","state":"STOPPED"}}'
>  
> http://localhost:8080/nifi-api/controller/process-groups/42cb10e8-1856-4155-9aa2-bef736bf330e/processors/fad172d5-5d29-4a8b-923f-e888af039c52
> 
> This is the response i am getting:
> 
> HTTP/1.1 409 Conflict
> Date: Sat, 12 Mar 2016 18:22:34 GMT
> Content-Type: text/plain
> Transfer-Encoding: chunked
> Server: Jetty(9.2.11.v20150529)
> 
> This NiFi instance has been updated by 'anonymous'. Please refresh to 
> synchronize the view
> 
> But it did not stop the processor.
> 
> Please provide some suggesstions
> 
> Regards,
> Sourav Gulati
> Big Data
> 
> 
> 
> From: Matt Gilman 
> Sent: Friday, March 11, 2016 8:00 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: starting and Stopping processor
> 
> Sourav,
> 
> The command would like the following with substituting the relevant values.
> 
> curl -i -X PUT -H 'Content-Type: application/json' -d
> '{"revision":{"version":,"clientId":""},"processor":{"id":"d6d28ec3-75c8-4194-b171-79c3e813f7e3","state":"STOPPED"}}'
> http://localhost:8080/nifi-api/controller/process-groups/92e70e95-31c4-4eb3-beab-e6fd411378bd/processors/d6d28ec3-75c8-4194-b171-79c3e813f7e3
> 
> Also, check out this blog post that describes how NiFi uses optimistic
> locking to know when another user has modified the data flow [1]. The
> bottom contains a great sequence diagram describing the necessary values.
> 
> Matt
> 
> [1]
> https://community.hortonworks.com/content/kbentry/3160/update-nifi-flow-on-the-fly-via-api.html
> 
> On Thu, Mar 10, 2016 at 11:57 PM, Sourav Gulati > wrote:
> 
>> Hi Team,
>> 
>> Is there a way to start and stop processor using some rest api or command
>> line ? If yes, please provide the steps.
>> 
>> Regards,
>> Sourav Gulati
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message is
>> intended solely for the named addressee. If received in error, please
>> destroy and notify the sender. Any use of this email is prohibited when
>> received in error. Impetus does not represent, warrant and/or guarantee,
>> that the integrity of this communication has been maintained nor that the
>> communication is free of errors, virus, interception or interference.
> 
> 
> 
> 
> 
> 
> 
> 
> NOTE: This message may contain information that is confidential, proprietary, 
> privileged or otherwise protected by law. The message is intended solely for 
> the named addressee. If received in error, please destroy and notify the 
> sender. Any use of this email is prohibited when received in error. Impetus 
> does not represent, warrant and/or guarantee, that the integrity of this 
> communication has been maintained nor that the communication is free of 
> errors, virus, interception or interference.

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

[pvillard31]

Github user pvillard31 commented on the pull request:

https://github.com/apache/nifi/pull/272#issuecomment-195793923
  
Due to JIRA unavailability, I think the PR has not been linked to issue in 
JIRA. Just adding a comment to mention the PR in JIRA.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Added source filename metadata to IdentifyMimeT...

[jfrazee]

GitHub user jfrazee opened a pull request:

https://github.com/apache/nifi/pull/273

Added source filename metadata to IdentifyMimeType, fixes NIFI-1617

Tika mime type Detector determines  mime type on the basis of filename 
extensions in some cases (e.g., CSV). This adds the FlowFile filename attribute 
to the Detector Metadata if it's available.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jfrazee/nifi NIFI-1617

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/273.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #273


commit 2ece5d7943f63258bd3dd9abc62aa734392ecce0
Author: Joey Frazee 
Date:   2016-03-10T22:49:59Z

Added source filename metadata to IdentifyMimeType, fixes NIFI-1617




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Reg: starting and Stopping processor

[Sourav Gulati]

Matt,

I tried it:
curl -i -X PUT -H 'Content-Type: application/json' -d 
'{"revision":{"clientId":"561b4443-8be5-43ee-b1f7-b07740e87a0f"},"processor":{"id":"fad172d5-5d29-4a8b-923f-e888af039c52","state":"STOPPED"}}'
 
http://localhost:8080/nifi-api/controller/process-groups/42cb10e8-1856-4155-9aa2-bef736bf330e/processors/fad172d5-5d29-4a8b-923f-e888af039c52

This is the response i am getting:

HTTP/1.1 409 Conflict
Date: Sat, 12 Mar 2016 18:22:34 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
Server: Jetty(9.2.11.v20150529)

This NiFi instance has been updated by 'anonymous'. Please refresh to 
synchronize the view

But it did not stop the processor.

Please provide some suggesstions

Regards,
Sourav Gulati
Big Data



From: Matt Gilman 
Sent: Friday, March 11, 2016 8:00 PM
To: dev@nifi.apache.org
Subject: Re: Reg: starting and Stopping processor

Sourav,

The command would like the following with substituting the relevant values.

curl -i -X PUT -H 'Content-Type: application/json' -d
'{"revision":{"version":,"clientId":""},"processor":{"id":"d6d28ec3-75c8-4194-b171-79c3e813f7e3","state":"STOPPED"}}'
http://localhost:8080/nifi-api/controller/process-groups/92e70e95-31c4-4eb3-beab-e6fd411378bd/processors/d6d28ec3-75c8-4194-b171-79c3e813f7e3

Also, check out this blog post that describes how NiFi uses optimistic
locking to know when another user has modified the data flow [1]. The
bottom contains a great sequence diagram describing the necessary values.

Matt

[1]
https://community.hortonworks.com/content/kbentry/3160/update-nifi-flow-on-the-fly-via-api.html

On Thu, Mar 10, 2016 at 11:57 PM, Sourav Gulati  wrote:

> Hi Team,
>
> Is there a way to start and stop processor using some rest api or command
> line ? If yes, please provide the steps.
>
> Regards,
> Sourav Gulati
>
>
> 
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>








NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.

Re:Re: Multiple dataflow jobs management(lots of jobs)

[刘岩]

Hi Aldrinsome additional information.it39s a typical ETL offloading user case 
each extraction job should foucs on 1 table and 1 table only.  data will be 
written on HDFS , this is similar to Database Staging. The reason why we need 
to foucs on 1 table for each job is because there might be database error or 
disconnection occur during the extraction , if it39s running as  a script like 
extraction job with expression langurage, then it39s hard to do the re-running 
or excape thing on that table or tables.once the extraction is done, a triger 
like action will do the data cleansing.  this is similar to ODS layer of 
Datawarehousingif the data quality has passed the quality check , then it will 
be marked as cleaned. otherwise , it will return to previous step and redo the 
data extraction, or send alert/email to the  system administrator.if certain 
numbers of tables were all cleaned and checked , then it will  call some 
Transforming  processor to do the transforming ， then push the data into a 
datawarehouse (Hive in our case)Thank you very much Yan Liu 

Hortonworks Service Division 

Richinfo, Shenzhen, China (PR)

13/03/2016邮件原文发件人："刘岩" 收件人：users  
抄　送: dev  发送时间：2016-03-13 
00:12:27主题：Re:Re: Multiple dataflow jobs management(lots of jobs)Hi 
AldrinCurrently  we need to extract 60K tables per day , and the time window is 
limited to 8 Hours.  Which means that we need to run jobs concurrently , and we 
need a general description of what39s going on with all those 60K job flows and 
take further actions.  We have tried Kettle and Talend ,  Talend is a IDE-Based 
so not what we are looking for,  and Kettle was crashed due to the Mysql cannot 
handle the Kettle39s metadata with 10K jobs.So we want to use Nifi ,  this is 
really the product that we are looking for , but  the missing piece here is a 
DataFlow jobs Admin Page.  so we can have multiple Nifi instances running on 
different nodes, but monitoring the jobs in one page.  If it can intergrate 
with Ambari metrics API,  then we can develop an Ambari View for Nifi Jobs 
Monitoring just like HDFS View and Hive View. Thank you very much Yan Liu 

Hortonworks Service Division 

Richinfo, Shenzhen, China (PR)

06/03/2016邮件原文发件人：Aldrin Piri  收件人：users 
抄　送: dev  发送时间：2016-03-11 
02:27:11主题：Re: Mutiple dataflow jobs management(lots of jobs)Hi Yan,
We can get more into details and particulars if needed, but have you 
experimented with expression language?  I could see a Cron driven approach 
which covers your periodic efforts that feeds some number of ExecuteSQL 
processors (perhaps one for each database you are communicating with) each 
having a table.  This would certainly cut down on the need for the 30k 
processors on a one-to-one basis with a given processor.

In terms of monitoring the dataflows, could you describe what else you are 
searching for beyond the graph view?  NiFi tries to provide context for the 
flow of data but is not trying to be a sole monitoring, we can give information 
on a processor basis, but do not delve into specifics.  There is a summary view 
for the overall flow where you can monitor stats about the components and 
connections in the system. We support interoperation with monitoring systems 
via push (ReportingTask) and pull (REST API [2]) semantics. 

Any other details beyond your list of how this all interoperates might shed 
some more light on what you are trying to accomplish.  It seems like NiFi 
should be able to help with this.  With some additional information we may be 
able to provide further guidance or at least get some insights on use cases we 
could look to improve upon and extend NiFi to support.

Thanks!


[1] http://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html
[2] 
http://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#reporting-tasks
[3] http://nifi.apache.org/docs/nifi-docs/rest-api/index.html



On Sat, Mar 5, 2016 at 9:25 PM, 刘岩  wrote:Hi All 



i39m trying to adapt Nifi to production but can not find an admin console 
which monitoring the dataflows



   The scenarios is simple,  



   1.  we gather data from oracle database to hdfs and then to hive.

   2.  residules/incrementals are updated daily or monthly via Nifi.

   3.  full dump on some table are excuted daily or monthly via Nifi.



is it really simple ,  however , we have  7 oracle databases with over 30K  
tables needs to implement the above scenario.



which means that i will drag that ExcuteSQL  elements for like 30K time or so 
and also need to place them with a nice looking way on my little 21 inch screen 
. 



Just wondering if there is a table list like  ,groupable and searchable task 
control and monitoring feature for Nifi 





Thank you very much  in advance





Yan Liu 

Hortonworks Service Division 

Richinfo, Shenzhen, 

Re:Re: Multiple dataflow jobs management(lots of jobs)

[刘岩]

Hi AldrinCurrently  we need to extract 60K tables per day , and the time window 
is limited to 8 Hours.  Which means that we need to run jobs concurrently , and 
we need a general description of what39s going on with all those 60K job flows 
and take further actions.  We have tried Kettle and Talend ,  Talend is a 
IDE-Based so not what we are looking for,  and Kettle was crashed due to the 
Mysql cannot handle the Kettle39s metadata with 10K jobs.So we want to use Nifi 
,  this is really the product that we are looking for , but  the missing piece 
here is a DataFlow jobs Admin Page.  so we can have multiple Nifi instances 
running on different nodes, but monitoring the jobs in one page.  If it can 
intergrate with Ambari metrics API,  then we can develop an Ambari View for 
Nifi Jobs Monitoring just like HDFS View and Hive View. Thank you very much Yan 
Liu 

Hortonworks Service Division 

Richinfo, Shenzhen, China (PR)

06/03/2016邮件原文发件人：Aldrin Piri  收件人：users 
抄　送: dev  发送时间：2016-03-11 
02:27:11主题：Re: Mutiple dataflow jobs management(lots of jobs)Hi Yan,
We can get more into details and particulars if needed, but have you 
experimented with expression language?  I could see a Cron driven approach 
which covers your periodic efforts that feeds some number of ExecuteSQL 
processors (perhaps one for each database you are communicating with) each 
having a table.  This would certainly cut down on the need for the 30k 
processors on a one-to-one basis with a given processor.

In terms of monitoring the dataflows, could you describe what else you are 
searching for beyond the graph view?  NiFi tries to provide context for the 
flow of data but is not trying to be a sole monitoring, we can give information 
on a processor basis, but do not delve into specifics.  There is a summary view 
for the overall flow where you can monitor stats about the components and 
connections in the system. We support interoperation with monitoring systems 
via push (ReportingTask) and pull (REST API [2]) semantics. 

Any other details beyond your list of how this all interoperates might shed 
some more light on what you are trying to accomplish.  It seems like NiFi 
should be able to help with this.  With some additional information we may be 
able to provide further guidance or at least get some insights on use cases we 
could look to improve upon and extend NiFi to support.

Thanks!


[1] http://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html
[2] 
http://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#reporting-tasks
[3] http://nifi.apache.org/docs/nifi-docs/rest-api/index.html



On Sat, Mar 5, 2016 at 9:25 PM, 刘岩  wrote:Hi All 



i39m trying to adapt Nifi to production but can not find an admin console 
which monitoring the dataflows



   The scenarios is simple,  



   1.  we gather data from oracle database to hdfs and then to hive.

   2.  residules/incrementals are updated daily or monthly via Nifi.

   3.  full dump on some table are excuted daily or monthly via Nifi.



is it really simple ,  however , we have  7 oracle databases with over 30K  
tables needs to implement the above scenario.



which means that i will drag that ExcuteSQL  elements for like 30K time or so 
and also need to place them with a nice looking way on my little 21 inch screen 
. 



Just wondering if there is a table list like  ,groupable and searchable task 
control and monitoring feature for Nifi 





Thank you very much  in advance





Yan Liu 

Hortonworks Service Division 

Richinfo, Shenzhen, China (PR)

06/03/2016