[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

2016-01-19 Thread mattyb149 
Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50171832
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,19 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
--- End diff --

On second thought, a couple of those createConnection() / createStatement() 
things might throw exceptions, so I will keep it in and add the yield :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

2016-01-19 Thread JPercivall 
Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50174170
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,22 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
-if (incoming == null) {
+if (fileToProcess == null) {
+// This can happen if any exceptions occur while setting 
up the connection, statement, etc.
 logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure", new Object[] {selectQuery, e});
+context.yield();
 } else {
-logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, incoming, e});
-session.transfer(incoming, REL_FAILURE);
+logger.error("Unable to execute SQL select query {} due to 
{}; routing to failure", new Object[] {selectQuery, e});
--- End diff --

Almost there, need to check if there is an incoming connection 
(context.hasIncomingConnection) to see if the flowfile was used to created 
query. If so it should be logged (the way the incoming flowfile was).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Auto-Organize Layout

2016-01-19 Thread dan bress 
Maybe not exactly "auto-layout" but I would back a notion of having the
components snap to a coarser grain grid than what we currently have.
Sometimes I care a lot about having everything line up in the graph
horizontally and vertically, and it always takes a long time to achieve
this.

I could see this being achieved by snapping the component to the same spot
horizontally as the component above it when you move it underneath another
component.  Or some magical "auto snap" button that does its best to align
everything with its nearest neighbors.

On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:

> I like your idea Rob, that would help with lining up relationships too
> (straight lines).
>
> On Matt's note, I don't think there should be a "standard" either, although
> best practices are always out there.
>
> On Matt's note of putting failures up above processes, we do that too.
> Totally depends on who made the flow first.  Sometimes, people don't even
> follow a convention in the same flow.xml file.
>
> For these reasons, I'd recommend alternate views to the flow.
>
> We have a couple projects that just allow you to rearrange a node-based
> graph, based on your preference, hierarchy, circular, pyramid, etc.
>
> Applying this to NiFi, having a couple different default auto-layout
> options that you can swap your current view to, but NOT change the original
> flow, would be nice.
>
> It would let you walk into someone else's, potentially large, dataflow and
> have a familiar way to view the flow.
>
> Ryan
>
>
> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
>
> > I agree with Matt's points. I was just replying with something similar
> > basically saying I think trying to set a standard would not be
> > well-received.
> >
> > I believe what could be more useful are layout tools that would help
> users
> > place components to help achieve their preferred layouts. For example,
> the
> > ability to align (left, right, center) components
> > or horizontally/vertically distribute components evenly. Other features
> > such as snap-to and/or smart-guides could make it easier for users to
> > follow their organization's best practices when designing a flow.
> >
> > Rob
> >
> > On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
> matt.clarke@gmail.com
> > >
> > wrote:
> >
> > > Ryan,
> > >
> > >   Setting a standard is a difficult thing to do.  The
> complexity
> > > that can exist in many flows would make enforcing a standard difficult.
> > The
> > > first example you provide of success to points right while failures
> point
> > > up is not recommended. It would be better to have failures point down
> > since
> > > it is common to put labels over processor(s). Any relationships
> pointing
> > up
> > > would pass through these labels making both the relationship box and
> the
> > > label hard to read.  It is often coomon to see flows designed with a
> > > combination of left to right and top to bottom design.
> > >
> > > Matt
> > >
> > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> > > wrote:
> > >
> > > > Hi Rob,
> > > > Yea we did, it was at the end of the meeting.
> > > >
> > > > I think it would be useful to have a couple default type views
> that
> > > > help standardize flow layout across the community.
> > > >
> > > > For example, when we organize processors left-to-right, failure
> > > > relationships always point up, and success always point right.
> > > > Alternatively, when we organize processors up-and-down, failure
> > > > relationships always point left, and successes always point down.
> > > >
> > > > Of course, in some of these scenarios there are processors that
> > have
> > > > more than 1 success relationship, but that's when a good layout
> library
> > > > would come into play.
> > > >
> > > > What do you think?
> > > >
> > > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran 
> wrote:
> > > >
> > > > > Ryan - I think we spoke briefly (at a very high level) about this
> at
> > a
> > > > > prior meetup. What alternate views did you have in mind, and in
> what
> > > ways
> > > > > do you think they'd be useful?
> > > > >
> > > > > Rob
> > > > >
> > > > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H <
> > rhendrickson.w...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > It'd be pretty awesome if NiFi provided the ability to
> > auto-organize
> > > a
> > > > > > layout.
> > > > > >
> > > > > > Maybe even just a auto-organized layout that doesn't change the
> > > > flow.xml,
> > > > > > just an alternate view.
> > > > > >
> > > > > > Looking at these demos here: http://js.cytoscape.org/#demos
> > > > > >
> > > > > > Ryan
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Auto-Organize Layout

2016-01-19 Thread Matt Gilman 
+1 to adding capabilities that will allow users to build the desired layout
more efficiently. Allowing them to more easily align components,
snap-to-grid, lock, and bend connections would all fall into this category.

I'm a little indifferent to supporting an auto-layout feature. I've tried a
number of different options for creating an optimum layout of directed
graphs and the results are typically sub-par for this purpose. Some of the
layouts being described are very useful when analyzing a dataset and
learning about the relationships between the elements. However, I'm not
sure how applicable they would be for visualizing a data flow. Building a
simple top-down left-right auto layout for a cyclic graph would also yield
poor results in my opinion. Additionally, supporting a view that is not
representative of the underlying flow.xml is a departure from our current
design model.

In short, my preference would be to provide the tools for the users to
build the layout they want rather than provide auto layouts that may not be
helpful.

Matt

On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess  wrote:

> +1 for "snap to grid" feature
>
> Sent from my iPhone
>
> > On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
> >
> > Maybe not exactly "auto-layout" but I would back a notion of having the
> > components snap to a coarser grain grid than what we currently have.
> > Sometimes I care a lot about having everything line up in the graph
> > horizontally and vertically, and it always takes a long time to achieve
> > this.
> >
> > I could see this being achieved by snapping the component to the same
> spot
> > horizontally as the component above it when you move it underneath
> another
> > component.  Or some magical "auto snap" button that does its best to
> align
> > everything with its nearest neighbors.
> >
> >> On Tue, Jan 19, 2016 at 12:37 PM Ryan H 
> wrote:
> >>
> >> I like your idea Rob, that would help with lining up relationships too
> >> (straight lines).
> >>
> >> On Matt's note, I don't think there should be a "standard" either,
> although
> >> best practices are always out there.
> >>
> >> On Matt's note of putting failures up above processes, we do that too.
> >> Totally depends on who made the flow first.  Sometimes, people don't
> even
> >> follow a convention in the same flow.xml file.
> >>
> >> For these reasons, I'd recommend alternate views to the flow.
> >>
> >> We have a couple projects that just allow you to rearrange a node-based
> >> graph, based on your preference, hierarchy, circular, pyramid, etc.
> >>
> >> Applying this to NiFi, having a couple different default auto-layout
> >> options that you can swap your current view to, but NOT change the
> original
> >> flow, would be nice.
> >>
> >> It would let you walk into someone else's, potentially large, dataflow
> and
> >> have a familiar way to view the flow.
> >>
> >> Ryan
> >>
> >>
> >>> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
> >>>
> >>> I agree with Matt's points. I was just replying with something similar
> >>> basically saying I think trying to set a standard would not be
> >>> well-received.
> >>>
> >>> I believe what could be more useful are layout tools that would help
> >> users
> >>> place components to help achieve their preferred layouts. For example,
> >> the
> >>> ability to align (left, right, center) components
> >>> or horizontally/vertically distribute components evenly. Other features
> >>> such as snap-to and/or smart-guides could make it easier for users to
> >>> follow their organization's best practices when designing a flow.
> >>>
> >>> Rob
> >>>
> >>> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
> >> matt.clarke@gmail.com
> >>> wrote:
> >>>
>  Ryan,
> 
>   Setting a standard is a difficult thing to do.  The
> >> complexity
>  that can exist in many flows would make enforcing a standard
> difficult.
> >>> The
>  first example you provide of success to points right while failures
> >> point
>  up is not recommended. It would be better to have failures point down
> >>> since
>  it is common to put labels over processor(s). Any relationships
> >> pointing
> >>> up
>  would pass through these labels making both the relationship box and
> >> the
>  label hard to read.  It is often coomon to see flows designed with a
>  combination of left to right and top to bottom design.
> 
>  Matt
> 
>  On Tue, Jan 19, 2016 at 12:07 PM, Ryan H  >
>  wrote:
> 
> > Hi Rob,
> >Yea we did, it was at the end of the meeting.
> >
> >I think it would be useful to have a couple default type views
> >> that
> > help standardize flow layout across the community.
> >
> >For example, when we organize processors left-to-right, failure
> > relationships always point up, and success always point right.
>

Stop a Processor with a spam notification to start it again

2016-01-19 Thread Ryan H 
How about this.. Add another "stop" state that includes a notification to a
distro every 5 minutes a processor is stopped.  That way if you stop a
production processor, leave for the day, and forget to turn it back on,
it's at least spamming a distro to remind you or your group to click
"start" again.

Ryan

[GitHub] nifi pull request: NIFI-1273 Adding ListenRELP Processor

2016-01-19 Thread bbende 
GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/179

NIFI-1273 Adding ListenRELP Processor

This pull request adds a ListenRELP processor which includes refactoring 
code that was previously part of ListenSyslog into a reusable framework for 
implementing listener processors, as well as back-end code for handling the 
RELP protocol.

## Overivew
* The listen package in nifi-processor-utils contains the generic 
dispatcher/handler code along with an AbstractListenEventProcessor which is the 
base class for future processors that want to act as a server reading data off 
a channel.
* The relp package in nifi-standard-processors contains the RELP specific 
implementations for the "listen" classes, along with other back-end code for 
the RELP protocol.
* ListenRELP extends AbstractListenEventProcessor

## Testing

I tested this against rsyslog 8.x putting the following in 
/etc/rsyslog.conf:

module(load="omrelp")
action(type="omrelp" target="centralserv" port="2514")

With TLS:

action(type="omrelp" target="192.168.233.153″ port="20514″ tls="on"
tls.caCert="/home/test/cert/ca.pem"
tls.myCert="/home/test/cert/ubuntu1-cert.pem"
tls.myPrivKey="/home/test/cert/ubuntu1-key.pem"
tls.authmode="name"
tls.permittedpeer=["ubuntu-server"]
)

RELP Reference:

http://www.rsyslog.com/doc/relp.html
http://www.rsyslog.com/using-tls-with-relp/

RELPFrameProducer in nifi-standard-processors 
src/test/java/org.apache.nifi.processors.standard.relp can also be used to act 
as an rsyslog server sending messages to ListenRELP.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-1273

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/179.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #179


commit ed23c57d3a1f9c07f91f67d0ed42c10f91558f40
Author: Bryan Bende 
Date:   2016-01-05T22:41:33Z

NIFI-1273 Adding ListenRELP processor which includes refactoring code that 
was previously part of ListenSyslog into a reusable framework for implementing 
listener processors, as well as back-end code for handling the RELP protocol.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Auto-Organize Layout

2016-01-19 Thread Ryan H 
I like your idea Rob, that would help with lining up relationships too
(straight lines).

On Matt's note, I don't think there should be a "standard" either, although
best practices are always out there.

On Matt's note of putting failures up above processes, we do that too.
Totally depends on who made the flow first.  Sometimes, people don't even
follow a convention in the same flow.xml file.

For these reasons, I'd recommend alternate views to the flow.

We have a couple projects that just allow you to rearrange a node-based
graph, based on your preference, hierarchy, circular, pyramid, etc.

Applying this to NiFi, having a couple different default auto-layout
options that you can swap your current view to, but NOT change the original
flow, would be nice.

It would let you walk into someone else's, potentially large, dataflow and
have a familiar way to view the flow.

Ryan


On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:

> I agree with Matt's points. I was just replying with something similar
> basically saying I think trying to set a standard would not be
> well-received.
>
> I believe what could be more useful are layout tools that would help users
> place components to help achieve their preferred layouts. For example, the
> ability to align (left, right, center) components
> or horizontally/vertically distribute components evenly. Other features
> such as snap-to and/or smart-guides could make it easier for users to
> follow their organization's best practices when designing a flow.
>
> Rob
>
> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke  >
> wrote:
>
> > Ryan,
> >
> >   Setting a standard is a difficult thing to do.  The complexity
> > that can exist in many flows would make enforcing a standard difficult.
> The
> > first example you provide of success to points right while failures point
> > up is not recommended. It would be better to have failures point down
> since
> > it is common to put labels over processor(s). Any relationships pointing
> up
> > would pass through these labels making both the relationship box and the
> > label hard to read.  It is often coomon to see flows designed with a
> > combination of left to right and top to bottom design.
> >
> > Matt
> >
> > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> > wrote:
> >
> > > Hi Rob,
> > > Yea we did, it was at the end of the meeting.
> > >
> > > I think it would be useful to have a couple default type views that
> > > help standardize flow layout across the community.
> > >
> > > For example, when we organize processors left-to-right, failure
> > > relationships always point up, and success always point right.
> > > Alternatively, when we organize processors up-and-down, failure
> > > relationships always point left, and successes always point down.
> > >
> > > Of course, in some of these scenarios there are processors that
> have
> > > more than 1 success relationship, but that's when a good layout library
> > > would come into play.
> > >
> > > What do you think?
> > >
> > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran  wrote:
> > >
> > > > Ryan - I think we spoke briefly (at a very high level) about this at
> a
> > > > prior meetup. What alternate views did you have in mind, and in what
> > ways
> > > > do you think they'd be useful?
> > > >
> > > > Rob
> > > >
> > > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H <
> rhendrickson.w...@gmail.com>
> > > > wrote:
> > > >
> > > > > It'd be pretty awesome if NiFi provided the ability to
> auto-organize
> > a
> > > > > layout.
> > > > >
> > > > > Maybe even just a auto-organized layout that doesn't change the
> > > flow.xml,
> > > > > just an alternate view.
> > > > >
> > > > > Looking at these demos here: http://js.cytoscape.org/#demos
> > > > >
> > > > > Ryan
> > > > >
> > > >
> > >
> >
>

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

2016-01-19 Thread mattyb149 
Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50170753
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,19 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
--- End diff --

I think the if(fileToProcess == null) block can now be removed, it was left 
over from when there were (up to) two references to the flowfile. At this point 
fileToProcess can't be null unless session.create() throws an exception, which 
would be bad.

Point taken for the log statements and the penalize, will update to reflect 
your comments, thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Auto-Organize Layout

2016-01-19 Thread Matt Burgess 
+1 for "snap to grid" feature

Sent from my iPhone

> On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
> 
> Maybe not exactly "auto-layout" but I would back a notion of having the
> components snap to a coarser grain grid than what we currently have.
> Sometimes I care a lot about having everything line up in the graph
> horizontally and vertically, and it always takes a long time to achieve
> this.
> 
> I could see this being achieved by snapping the component to the same spot
> horizontally as the component above it when you move it underneath another
> component.  Or some magical "auto snap" button that does its best to align
> everything with its nearest neighbors.
> 
>> On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:
>> 
>> I like your idea Rob, that would help with lining up relationships too
>> (straight lines).
>> 
>> On Matt's note, I don't think there should be a "standard" either, although
>> best practices are always out there.
>> 
>> On Matt's note of putting failures up above processes, we do that too.
>> Totally depends on who made the flow first.  Sometimes, people don't even
>> follow a convention in the same flow.xml file.
>> 
>> For these reasons, I'd recommend alternate views to the flow.
>> 
>> We have a couple projects that just allow you to rearrange a node-based
>> graph, based on your preference, hierarchy, circular, pyramid, etc.
>> 
>> Applying this to NiFi, having a couple different default auto-layout
>> options that you can swap your current view to, but NOT change the original
>> flow, would be nice.
>> 
>> It would let you walk into someone else's, potentially large, dataflow and
>> have a familiar way to view the flow.
>> 
>> Ryan
>> 
>> 
>>> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
>>> 
>>> I agree with Matt's points. I was just replying with something similar
>>> basically saying I think trying to set a standard would not be
>>> well-received.
>>> 
>>> I believe what could be more useful are layout tools that would help
>> users
>>> place components to help achieve their preferred layouts. For example,
>> the
>>> ability to align (left, right, center) components
>>> or horizontally/vertically distribute components evenly. Other features
>>> such as snap-to and/or smart-guides could make it easier for users to
>>> follow their organization's best practices when designing a flow.
>>> 
>>> Rob
>>> 
>>> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
>> matt.clarke@gmail.com
>>> wrote:
>>> 
 Ryan,
 
  Setting a standard is a difficult thing to do.  The
>> complexity
 that can exist in many flows would make enforcing a standard difficult.
>>> The
 first example you provide of success to points right while failures
>> point
 up is not recommended. It would be better to have failures point down
>>> since
 it is common to put labels over processor(s). Any relationships
>> pointing
>>> up
 would pass through these labels making both the relationship box and
>> the
 label hard to read.  It is often coomon to see flows designed with a
 combination of left to right and top to bottom design.
 
 Matt
 
 On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
 wrote:
 
> Hi Rob,
>Yea we did, it was at the end of the meeting.
> 
>I think it would be useful to have a couple default type views
>> that
> help standardize flow layout across the community.
> 
>For example, when we organize processors left-to-right, failure
> relationships always point up, and success always point right.
>Alternatively, when we organize processors up-and-down, failure
> relationships always point left, and successes always point down.
> 
>Of course, in some of these scenarios there are processors that
>>> have
> more than 1 success relationship, but that's when a good layout
>> library
> would come into play.
> 
>What do you think?
> 
> On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran 
>> wrote:
> 
>> Ryan - I think we spoke briefly (at a very high level) about this
>> at
>>> a
>> prior meetup. What alternate views did you have in mind, and in
>> what
 ways
>> do you think they'd be useful?
>> 
>> Rob
>> 
>> On Tue, Jan 19, 2016 at 10:56 AM, Ryan H <
>>> rhendrickson.w...@gmail.com>
>> wrote:
>> 
>>> It'd be pretty awesome if NiFi provided the ability to
>>> auto-organize
 a
>>> layout.
>>> 
>>> Maybe even just a auto-organized layout that doesn't change the
> flow.xml,
>>> just an alternate view.
>>> 
>>> Looking at these demos here: http://js.cytoscape.org/#demos
>>> 
>>> Ryan
>>

Re: Auto-Organize Layout

2016-01-19 Thread Joe Witt 
Elli

"it is sometimes too easy to mis-align processors by dragging them accidentally"
  Great point  I must admit I too do that.  Often in really important
demos.  I've gotten good at making jokes about it.  Probably should
have gotten good at submitting a JIRA :-)

I'd like to understand more about your other idea for emphasizing
processors which are more important.  I can understand the idea I
think but I'm worried about how we could make the user experience
worth the effort for the person signaling the emphasis to be of use
for the people consuming that detail.

Thanks
Joe

On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess  wrote:
> +1 for "snap to grid" feature
>
> Sent from my iPhone
>
>> On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
>>
>> Maybe not exactly "auto-layout" but I would back a notion of having the
>> components snap to a coarser grain grid than what we currently have.
>> Sometimes I care a lot about having everything line up in the graph
>> horizontally and vertically, and it always takes a long time to achieve
>> this.
>>
>> I could see this being achieved by snapping the component to the same spot
>> horizontally as the component above it when you move it underneath another
>> component.  Or some magical "auto snap" button that does its best to align
>> everything with its nearest neighbors.
>>
>>> On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:
>>>
>>> I like your idea Rob, that would help with lining up relationships too
>>> (straight lines).
>>>
>>> On Matt's note, I don't think there should be a "standard" either, although
>>> best practices are always out there.
>>>
>>> On Matt's note of putting failures up above processes, we do that too.
>>> Totally depends on who made the flow first.  Sometimes, people don't even
>>> follow a convention in the same flow.xml file.
>>>
>>> For these reasons, I'd recommend alternate views to the flow.
>>>
>>> We have a couple projects that just allow you to rearrange a node-based
>>> graph, based on your preference, hierarchy, circular, pyramid, etc.
>>>
>>> Applying this to NiFi, having a couple different default auto-layout
>>> options that you can swap your current view to, but NOT change the original
>>> flow, would be nice.
>>>
>>> It would let you walk into someone else's, potentially large, dataflow and
>>> have a familiar way to view the flow.
>>>
>>> Ryan
>>>
>>>
 On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:

 I agree with Matt's points. I was just replying with something similar
 basically saying I think trying to set a standard would not be
 well-received.

 I believe what could be more useful are layout tools that would help
>>> users
 place components to help achieve their preferred layouts. For example,
>>> the
 ability to align (left, right, center) components
 or horizontally/vertically distribute components evenly. Other features
 such as snap-to and/or smart-guides could make it easier for users to
 follow their organization's best practices when designing a flow.

 Rob

 On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
>>> matt.clarke@gmail.com
 wrote:

> Ryan,
>
>  Setting a standard is a difficult thing to do.  The
>>> complexity
> that can exist in many flows would make enforcing a standard difficult.
 The
> first example you provide of success to points right while failures
>>> point
> up is not recommended. It would be better to have failures point down
 since
> it is common to put labels over processor(s). Any relationships
>>> pointing
 up
> would pass through these labels making both the relationship box and
>>> the
> label hard to read.  It is often coomon to see flows designed with a
> combination of left to right and top to bottom design.
>
> Matt
>
> On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> wrote:
>
>> Hi Rob,
>>Yea we did, it was at the end of the meeting.
>>
>>I think it would be useful to have a couple default type views
>>> that
>> help standardize flow layout across the community.
>>
>>For example, when we organize processors left-to-right, failure
>> relationships always point up, and success always point right.
>>Alternatively, when we organize processors up-and-down, failure
>> relationships always point left, and successes always point down.
>>
>>Of course, in some of these scenarios there are processors that
 have
>> more than 1 success relationship, but that's when a good layout
>>> library
>> would come into play.
>>
>>What do you think?
>>
>> On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran 
>>> wrote:
>>
>>> Ryan - I think we spoke briefly (at a very high level) about this
>>> at
 a
>>> prior

Re: Auto-Organize Layout

2016-01-19 Thread Elli Schwarz 
I also think that some way to help layout the flow would be very useful. One of 
the hardest parts of creating an easy to read workflow is the designing a good 
layout, and in many cases I simply want to be able to select a few processors 
and line them up (horizontally or vertically). Also, it is sometimes too easy 
to mis-align processors by dragging them accidentally. It would be nice if 
there would be a way to "lock" the layout of a group of selected processors so 
they can't be accidentally moved. Also, the ability to "undo" a layout change 
or save a layout (ie, just save the position of a group of selected processors) 
so that we can revert to it would also be helpful.

Another thing that could be helpful is a way to emphasize the more important 
processors, ie, the ones integral to the flow, as opposed to the many "PutFile" 
processors I have that are simply for logging (although I realize that 
sometimes a PutFile can be an integral part of a flow). I know I can select a 
group of processors and change the color, and also create a label to highlight 
a certain group of processors, but it might be nice to be able to make selected 
processors appear larger on the screen than others, to emphasize their 
importance. It is sometimes hard to follow the trail of a flow simply because 
there are many connections, so a way to highlight the "main" path might be 
useful.

-Elli


On Tuesday, January 19, 2016 4:20 PM, dan bress  wrote:
 
 

 Maybe not exactly "auto-layout" but I would back a notion of having the
components snap to a coarser grain grid than what we currently have.
Sometimes I care a lot about having everything line up in the graph
horizontally and vertically, and it always takes a long time to achieve
this.

I could see this being achieved by snapping the component to the same spot
horizontally as the component above it when you move it underneath another
component.  Or some magical "auto snap" button that does its best to align
everything with its nearest neighbors.

On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:

> I like your idea Rob, that would help with lining up relationships too
> (straight lines).
>
> On Matt's note, I don't think there should be a "standard" either, although
> best practices are always out there.
>
> On Matt's note of putting failures up above processes, we do that too.
> Totally depends on who made the flow first.  Sometimes, people don't even
> follow a convention in the same flow.xml file.
>
> For these reasons, I'd recommend alternate views to the flow.
>
> We have a couple projects that just allow you to rearrange a node-based
> graph, based on your preference, hierarchy, circular, pyramid, etc.
>
> Applying this to NiFi, having a couple different default auto-layout
> options that you can swap your current view to, but NOT change the original
> flow, would be nice.
>
> It would let you walk into someone else's, potentially large, dataflow and
> have a familiar way to view the flow.
>
> Ryan
>
>
> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
>
> > I agree with Matt's points. I was just replying with something similar
> > basically saying I think trying to set a standard would not be
> > well-received.
> >
> > I believe what could be more useful are layout tools that would help
> users
> > place components to help achieve their preferred layouts. For example,
> the
> > ability to align (left, right, center) components
> > or horizontally/vertically distribute components evenly. Other features
> > such as snap-to and/or smart-guides could make it easier for users to
> > follow their organization's best practices when designing a flow.
> >
> > Rob
> >
> > On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
> matt.clarke@gmail.com
> > >
> > wrote:
> >
> > > Ryan,
> > >
> > >          Setting a standard is a difficult thing to do.  The
> complexity
> > > that can exist in many flows would make enforcing a standard difficult.
> > The
> > > first example you provide of success to points right while failures
> point
> > > up is not recommended. It would be better to have failures point down
> > since
> > > it is common to put labels over processor(s). Any relationships
> pointing
> > up
> > > would pass through these labels making both the relationship box and
> the
> > > label hard to read.  It is often coomon to see flows designed with a
> > > combination of left to right and top to bottom design.
> > >
> > > Matt
> > >
> > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> > > wrote:
> > >
> > > > Hi Rob,
> > > >    Yea we did, it was at the end of the meeting.
> > > >
> > > >    I think it would be useful to have a couple default type views
> that
> > > > help standardize flow layout across the community.
> > > >
> > > >    For example, when we organize processors left-to-right, failure
> > > > relationships always point up, and success always point right.
> > >

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

2016-01-19 Thread JPercivall 
Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50158533
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,19 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
--- End diff --

This error handling should be expanded a bit more. In the event there is no 
incoming flowfile then the processor should yield when an error is occurred. 
Also the log statements should be changed a bit to better reflect the state the 
processor is in. For example the else statement says that the error is caused 
by the flowfile but that isn't true when it's a source processor (the flowfile 
is created to hold the content, doesn't influence the query).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Auto-Organize Layout

2016-01-19 Thread Rob Moran 
I agree with Matt's points. I was just replying with something similar
basically saying I think trying to set a standard would not be
well-received.

I believe what could be more useful are layout tools that would help users
place components to help achieve their preferred layouts. For example, the
ability to align (left, right, center) components
or horizontally/vertically distribute components evenly. Other features
such as snap-to and/or smart-guides could make it easier for users to
follow their organization's best practices when designing a flow.

Rob

On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke 
wrote:

> Ryan,
>
>   Setting a standard is a difficult thing to do.  The complexity
> that can exist in many flows would make enforcing a standard difficult. The
> first example you provide of success to points right while failures point
> up is not recommended. It would be better to have failures point down since
> it is common to put labels over processor(s). Any relationships pointing up
> would pass through these labels making both the relationship box and the
> label hard to read.  It is often coomon to see flows designed with a
> combination of left to right and top to bottom design.
>
> Matt
>
> On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> wrote:
>
> > Hi Rob,
> > Yea we did, it was at the end of the meeting.
> >
> > I think it would be useful to have a couple default type views that
> > help standardize flow layout across the community.
> >
> > For example, when we organize processors left-to-right, failure
> > relationships always point up, and success always point right.
> > Alternatively, when we organize processors up-and-down, failure
> > relationships always point left, and successes always point down.
> >
> > Of course, in some of these scenarios there are processors that have
> > more than 1 success relationship, but that's when a good layout library
> > would come into play.
> >
> > What do you think?
> >
> > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran  wrote:
> >
> > > Ryan - I think we spoke briefly (at a very high level) about this at a
> > > prior meetup. What alternate views did you have in mind, and in what
> ways
> > > do you think they'd be useful?
> > >
> > > Rob
> > >
> > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H 
> > > wrote:
> > >
> > > > It'd be pretty awesome if NiFi provided the ability to auto-organize
> a
> > > > layout.
> > > >
> > > > Maybe even just a auto-organized layout that doesn't change the
> > flow.xml,
> > > > just an alternate view.
> > > >
> > > > Looking at these demos here: http://js.cytoscape.org/#demos
> > > >
> > > > Ryan
> > > >
> > >
> >
>

Re: [DISCUSS] roadmap for the next 6-12 months

2016-01-19 Thread Joe Witt 
Team,

I had a lengthy conversation today with Matt Gilman to talk through
specifics of the Authorizer API concept.  The ideas are captured here
https://cwiki.apache.org/confluence/display/NIFI/Support+Authorizer+API.
Definitely would like to hear ideas from people experienced in working
with centralized authority providers.

This will be an important step to making Multi-tenant authorization possible.

Thanks
Joe

On Fri, Jan 8, 2016 at 11:25 AM, Tony Kurc  wrote:
> Joe! Looks amazing! Thanks for pulling this together
> Hello NiFi Community
>
> Please review the proposed roadmap and suggest things to add, remove,
> update, prioritize differently.
>
> Over the past year we have had great contributions that improved the
> quality of NiFi and the community.  As we know there are many JIRAs
> related to improvements or new features.  However, it can be difficult
> for new and existing community members to know when to jump in, how to
> help, or when things they are interested in might be able to happen.
> So among other things we should provide and maintain a documented
> roadmap to help this.
>
> The following suggested timeline follows pretty strictly against our
> previously stated objective of having releases roughly every 6 weeks.
> We do not need to stay strict to this but a time bound (as opposed to
> feature bound) release cycle does help provide a regular cadence to
> follow and helps give contributors a sense of when their efforts might
> end up in a release.
>
> The items below are largely just focused on some of the major muscle
> movements needed at a framework level.  There will of course be many
> many more JIRAs with each releas.
>
> NiFi 0.5.0 (Feb 5)
>
>   State Management
> https://cwiki.apache.org/confluence/display/NIFI/State+Management
>   Kerberos
>
> https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication
>   Interactive Queue
>
> https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management
>   Scripting Support
> https://issues.apache.org/jira/browse/NIFI-210
>
> NiFi 0.6.0 (Mar 18)
>
>   Deterministic Template Export
> https://issues.apache.org/jira/browse/NIFI-826
>   Schema/Format Editor (JSON to JSON,…?)
> Numerous JIRAs related to format/schema conversion.  An example
> implementation:
>
> https://github.com/fsauer65/NiFi-Extensions/tree/master/nifi-jsontransform-bundle
>
> NiFi 0.7.0 (Apr 29)
>
> NiFi 1.0.0 (Jun 17)
>
>   HA Cluster Management
> https://cwiki.apache.org/confluence/display/NIFI/Clustering+Redesign
>   HA Data
>
> https://cwiki.apache.org/confluence/display/NIFI/High+Availability+Processing
>   Multi-Tenant Authorization
> https://cwiki.apache.org/confluence/display/NIFI/Multi-Tentant+Dataflow
>   Improved Authorization API
>
> https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication
> NiFi's current authority provider gathers data then makes a decision.
> We should instead support PDP/PEP model [4].
>   Modernized UX
> https://cwiki.apache.org/confluence/display/NIFI/Redesign+User+Interface
>   Dependent Properties
> https://issues.apache.org/jira/browse/NIFI-1121
>
> NiFi 1.1.0 (Sep?)
>
>   Template/Extension Registry
> https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry
>   Variable Registry
> https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry
>
> The timing of these things is pretty important as far as
> interdependencies and how long they'll take to develop and validate.
> State management in 050 sets up an important way to have cluster-wide
> shared state and introduces zookeeper formally into NiFi which
> obviously will be important for the distributed systems behaviors
> we'll need as we restructure how clustering works and introduce data
> replication for HA data processing.  The 060 and 070 releases are
> intentionally quite light to allow for very focused work on a parallel
> branch of 1.x which probably should start right on the heels of 050.
> The 1.0 release will introduce some very key features, offer a chance
> to remove deprecated classes, ensure all APIs are extremely clearly
> documented for stability and audience, allow us to introduce UI/UX
> changes, and so on.
>
> Now, of course this is simply a proposal and these timings may not
> work out.  But this should serve as a good basis for discussion and
> consensus forming.
>
> As we progress I plan to update the roadmap [1], product requirements
> [2], and feature proposals [3] to reflect these things a bit more
> clearly than they are now.
>
> We should also have a discussion on how long we should be committed to
> supporting the 0.x line and what that means.  We need to document a
> commitment for the community.
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=58851850
> [2] https://cwiki.apache.org/confluence/display/NIFI/Product+requirements
> [3]

[GitHub] nifi pull request: NIFI-1273 Adding ListenRELP Processor

2016-01-19 Thread trkurc 
Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/179#issuecomment-173044611
  
I like the docs to help with reviewing! I good amount here to review and 
test!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1378 fixed JMS URI validation

2016-01-19 Thread trkurc 
Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/167#issuecomment-173043643
  
reviewing now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Stop a Processor with a spam notification to start it again

2016-01-19 Thread Mark Payne 
Ryan,

I've considered implementing something like this before but just never had the 
cycles.
I do think it's a good idea, though. The approach that I would take is to write 
a Reporting Task,
though. It would not be any change to the framework. Rather, the reporting task 
would just run,
say every 5 minutes or so and if it finds any processors that are stopped, keep 
track of when it
was stopped. After some amount of time, if it's still stopped, send an e-mail.

I think this is very doable. I don't know if you're interested in getting more 
involved with NiFi or not,
but I think it would be a great place to dig in, if you are interested.

Thanks
-Mark

> On Jan 19, 2016, at 6:02 PM, Ryan H  wrote:
> 
> How about this.. Add another "stop" state that includes a notification to a
> distro every 5 minutes a processor is stopped.  That way if you stop a
> production processor, leave for the day, and forget to turn it back on,
> it's at least spamming a distro to remind you or your group to click
> "start" again.
> 
> Ryan

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

2016-01-19 Thread JPercivall 
Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50213360
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,31 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
-if (incoming == null) {
-logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure", new Object[] {selectQuery, e});
+if (fileToProcess == null) {
+// This can happen if any exceptions occur while setting 
up the connection, statement, etc.
+logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure",
+new Object[] {selectQuery, e});
+context.yield();
 } else {
-logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, incoming, e});
-session.transfer(incoming, REL_FAILURE);
+if(context.hasIncomingConnection()) {
+logger.error("Unable to execute SQL select query {} 
for {} due to {}; routing to failure",
+new Object[] {selectQuery, fileToProcess, e});
+  session.transfer(fileToProcess, 
REL_FAILURE);
--- End diff --

I think this snuck in there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Javadoc

2016-01-19 Thread Joe Witt 
What is the best way to do this?  mvn site?

On Tue, Jan 19, 2016 at 8:41 AM, Sean Busbey  wrote:
> Hi Devin!
>
> Sorry for the gap. We're tracking the need for a web accessible copy
> of our API javadocs in NIFI-943. I don't believe anyone has taken up
> the task yet, so if you're looking to get more involved in the project
> we'd love the help!
>
> On Sun, Jan 17, 2016 at 1:38 PM, Devin Fisher
>  wrote:
>> Might be a silly question but I can't find a link the Javadocs anywhere on
>> your website. I've googled for it and only find the reference to the
>> Javadocs in the NiFi Developer’s Guide
>> . I'm hopeful that I'm just
>> not seeing it. If not, what is the recommended way to get a hold of it?
>>
>> Also, thanks in advanced.
>>
>> Devin

Re: Javadoc

2016-01-19 Thread Sean Busbey 
Hi Devin!

Sorry for the gap. We're tracking the need for a web accessible copy
of our API javadocs in NIFI-943. I don't believe anyone has taken up
the task yet, so if you're looking to get more involved in the project
we'd love the help!

On Sun, Jan 17, 2016 at 1:38 PM, Devin Fisher
 wrote:
> Might be a silly question but I can't find a link the Javadocs anywhere on
> your website. I've googled for it and only find the reference to the
> Javadocs in the NiFi Developer’s Guide
> . I'm hopeful that I'm just
> not seeing it. If not, what is the recommended way to get a hold of it?
>
> Also, thanks in advanced.
>
> Devin

[GitHub] nifi pull request: NIFI-569 Creating basis for integration-tests i...

2016-01-19 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/173


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-569 Creating basis for integration-tests i...

2016-01-19 Thread mcgilman 
Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/173#issuecomment-172864860
  
+1 Went ahead and merged the proposed changes for the build infrastructure. 
Specifics for this integration test can be updated as needed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: [NIFI-1394] - Unit test creates resources in HO...

2016-01-19 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/174


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: [NIFI-1394] - Unit test creates resources in HO...

2016-01-19 Thread mcgilman 
Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/174#issuecomment-172874349
  
Build looks good. No more artifacts showing up in user home directory. 
Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: [NIFI-1394] - Unit test creates resources in HO...

2016-01-19 Thread smarthi 
Github user smarthi commented on the pull request:

https://github.com/apache/nifi/pull/174#issuecomment-172875249
  
Thanks for merging this PR. Do we similarly modify all of the other 
TestJdbc* to take a TemporaryFolder ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: [NIFI-1394] - Unit test creates resources in HO...

2016-01-19 Thread mcgilman 
Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/174#issuecomment-172880667
  
I wasn't aware of any issues with the other JDBC tests. This JIRA was 
created to specifically address the tests resources being created in the user's 
HOME directory. If there are any issues with the other tests we should address 
those as well with a separate ticket. Thanks again for the contribution!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: [NIFI-1411] - TestJdbcTypesDerby: java.sql.SQLS...

2016-01-19 Thread smarthi 
GitHub user smarthi opened a pull request:

https://github.com/apache/nifi/pull/177

[NIFI-1411] - TestJdbcTypesDerby: java.sql.SQLSyntaxErrorException: TYPE 
'DATETIME' does not exist

... also fixed the other TestJdbcXXX.java classes to use @Rule 
TemporaryFolder

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/smarthi/incubator-nifi NIFI-1411

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/177.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #177


commit c3c6b129b1d75a0e8c5a76c94b691519fbcdce70
Author: smarthi 
Date:   2016-01-19T17:00:02Z

[NIFI-1411] - TestJdbcTypesDerby: java.sql.SQLSyntaxErrorException: TYPE 
'DATETIME' does not exist




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

2016-01-19 Thread mattyb149 
GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/178

NIFI-1409: Fix missing transfer on error in ExecuteSql

This change keeps track of the "working flowfile", either the incoming 
flowfile if one exists, or the one created if no incoming flowfile exists. Then 
if an exception is thrown, the correct flowfile reference is available for 
transfer to failure.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-1409

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/178.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #178


commit c20018f923f9feb4e50eb47586b36168e45e0931
Author: Matt Burgess 
Date:   2016-01-19T15:55:48Z

NIFI-1409: Fix missing transfer on error in ExecuteSql




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---