Re: Lots o branches in git

2015-08-17 Thread Dan Bress
My stale branches have been deleted.

Dan Bress
Software Engineer
ONYX Consulting Services


From: Mike Drob 
Sent: Monday, August 17, 2015 10:07 AM
To: dev@nifi.apache.org
Subject: Re: Lots o branches in git

Also useful is
git remote prune origin

which will clean up your local tracking branches for remote branches that
have been deleted. You can specify different remote names instead of
origin, as well.


On Mon, Aug 17, 2015 at 7:44 AM, Joe Witt  wrote:

> Thanks Dan.
>
> Please be sure you did the command needed to remove the remote/origin
> branch.  It is possible that the email just hasn't come through but i
> suspect not as the ASF git repo still shows them.  This is the command
> set you want:
>
> #remove remote branch
> git push origin —delete NIFI-633
> git push origin —delete NIFI-632
> git push origin —delete NIFI-680
> git push origin —delete NIFI-682
> git push origin —delete NIFI-704
> git push origin —delete NIFI-735
>
>
> Thanks
> Joe
>
> On Mon, Aug 17, 2015 at 8:32 AM, Dan Bress 
> wrote:
> > Joe, thanks for the heads up.  I'll delete all of mine, as they are all
> resolved in JIRA
> > NIFI-633: danbress
> > NIFI-632: danbress
> > NIFI-680: danbress
> > NIFI-682: danbress
> > NIFI-704: danbress
> > NIFI-735: danbress
> >
> > Dan Bress
> > Software Engineer
> > ONYX Consulting Services
> >
> > 
> > From: Joe Witt 
> > Sent: Sunday, August 16, 2015 9:34 PM
> > To: dev@nifi.apache.org
> > Subject: Lots o branches in git
> >
> > Team,
> >
> > Friendly reminder regarding branches that appear stale.  If you no
> > longer need them please consider removing them.
> >
> > Last activity more than 2 months ago:
> > --
> > ListHDFS : markup
> > NIFI-25: markap
> > NIFI-85: mcgilma
> > NIFI-180: mcgilma
> > NIFI-250: mcgilma
> > NIFI-271: apiri
> > NIFI-292: mcgilma
> > NIFI-353: mcgilma
> > NIFI-376: mcgilma
> > NIFI-433: devriesb
> > NIFI-475: mcgilma
> > NIFI-521: mcgilma
> > NIFI-633: danbress
> > improve-pro-performance: markap, mcgilma
> > journaling-prov-repo: markap
> > prov-query-language: markap
> >
> > Seems definitely eligible to remove (release is done and tagged):
> > --
> > release-nifi-0.1.0-rc13: markup
> > release-nifi-0.2.0-incubating: apiri
> > release-nifi-0.2.0-incubating-rc1: mcgilma
> >
> > Active within the last 2 months:
> > ---
> > NIFI-632: danbress
> > NIFI-640: mcgilma
> > NIFI-680: danbress
> > NIFI-682: danbress
> > NIFI-704: danbress
> > NIFI-724: mcgilma
> > NIFI-731: markap
> > NIFI-735: danbress
> > NIFI-744: markap
> > NIFI-793: markap
> > NIFI-818: markap
> >
> > Reminder on commands that can help:
> > ———
> > #remove local branch
> > git branch -D 
> > #remove remote branch
> > git push origin —delete 
> >
> > Thanks
> > Joe
>


Re: Process to create multiple files

2015-08-17 Thread Joe Percivall
For a very simple example you could check out the DuplicateFlowFile standard 
processor. 
Joe- - - - - - Joseph Percivalllinkedin.com/in/Percivall
c: (410) 693-8032e: joeperciv...@yahoo.com
 


 On Monday, August 17, 2015 2:36 PM, plj  wrote:
   

 Ok that makes sense. What processor would I extend/implement to allow me to 
send out multiple files from the process that unpacks the NITF bundle?  Any 
examples?

Thank,

Pat


From: Joe Witt [via Apache NiFi (incubating) Developer List] 
[mailto:ml-node+s39713n2513...@n7.nabble.com]
Sent: Monday, August 17, 2015 1:56 PM
To: Jones, Patrick L. 
Subject: Re: Process to create multiple files

Hello Pat,

Yeah makes sense.  You would need a custom processor to support the
NITF format [1]

There does appear to be a Java library to deal with it but its
licensing isn't ASL v2 compatible so we're not likely to be able to
play along anytime soon.  But that should be a good way to get started
building what you need.

The processor would unpack the items out of the NITF bundle and then
could send out the full size images if that is how it works.  Those
full size images can then be resized to thumbnails.  Tons of ways to
play this one and it is a very common style of use case.

Thanks
Joe

[1] https://en.wikipedia.org/wiki/National_Imagery_Transmission_Format
[2] https://github.com/codice/imaging-nitf

On Mon, Aug 17, 2015 at 1:45 PM, plj <[hidden 
email]> wrote:

> Howdy,
>
>                Thanks for the reply.  I think my situation is different than 
>you suggest.  I have an image file in NITF format.  That file may have 
>multiple images and multiple bands inside that one file.  The software that I 
>have that reads that file does create a thumbnail for each image and each band 
>inside that file.  I don't know how many thumbnails I need until I read the 
>files metadata.
>
>
> Thank
>
> Pat
>
> From: Dan Bress [via Apache NiFi (incubating) Developer List] [mailto:[hidden 
> email]]
> Sent: Monday, August 17, 2015 1:40 PM
> To: Jones, Patrick L. <[hidden 
> email]>
> Subject: Re: Process to create multiple files
>
> plj,
>    I would not recommend having this processor create multiple thumbnails.  
>What I would recommend is the following:
>
>    Create a new processor called "CreateThumbnail" or "RescaleImage"
>
>    Then have a configuration on the processor that says what size the output 
>image should be(e.g. 128x128, or 1/X of original size).
>
>    Your new processor will read in the incoming image, and rescale it down to 
>the user specified size and pass it forward.
>
>      Now if you want to create a 128x128 64x64 and 32x32 sized images you 
>would do the following.
>
>    (GetFile)->(RescaleImage configured to 128x128)->(PutFile)
>                      |->(RescaleImage configured to 64x64)->(PutFile)
>                      \->(RescaleImage configured to 32x32)->(PutFile)
>
> Where GetFile has 3 success relationships, each going to a different 
> RescaleImage processor.
>
> I think it makes more sense to have one processor create one file, then you 
> can use the flow to visually configure how many copies of the file you want 
> to make.  This should make this processor simpler and more reusable.
>
>
> Dan Bress
> Software Engineer
> ONYX Consulting Services
>
> 
> From: plj <[hidden email]>
> Sent: Monday, August 17, 2015 1:27 PM
> To: [hidden email]
> Subject: Process to create  multiple files
>
> Howdy,
>
>  I'm new to NiFi so please bear with me.  What I want to accomplish is:
>      read an image file
>      process the file to create one or more thumbnails from the image.
>      Send the resulting thumbnails along the flow
>
>    So I can use GetFile to read the file and then send it along.  I think I
> need to write a custom java processor that will process the image file and
> then send each of the thumbnail files (say .jpg for now) on to the next
> thing in the flow (say PutFile for example).
>
>    Are there suggestions on what I should implement or extend to create my
> custom processor?  It will take in one file and output multiple files.
> Would extending "PutFile" so that it processed and then puts each thumbnail
> on the flow be a good strategy?  Other ideas?
>
> Thank you,
>
>
>
>
>
> --
> View this message in context: 
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
> Nabble.com.
>
> 
> If you reply to this email, your message will be added to the discussion 
> below:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2511.html
> To unsubscribe from Process to create multiple files, click here<
> NAML

Re: Process to create multiple files

2015-08-17 Thread Bryan Bende
Pat,

You don't need to extend anything special to produce multiple files. You
can create as many FlowFiles as you want and transfer them to a
relationship, such as...

FlowFile childFlowFile = session.create(original);
// create more

List flowFiles = new ArrayList();
flowFiles.add(childFLowFile);

session.transfer(flowFIles, YOUR_RELATIONSHIP);

Good examples might be SplitJson or SplitXML.

-Bryan


On Mon, Aug 17, 2015 at 2:05 PM, plj  wrote:

> Ok that makes sense. What processor would I extend/implement to allow me
> to send out multiple files from the process that unpacks the NITF bundle?
> Any examples?
>
> Thank,
>
> Pat
>
>
> From: Joe Witt [via Apache NiFi (incubating) Developer List] [mailto:
> ml-node+s39713n2513...@n7.nabble.com]
> Sent: Monday, August 17, 2015 1:56 PM
> To: Jones, Patrick L. 
> Subject: Re: Process to create multiple files
>
> Hello Pat,
>
> Yeah makes sense.  You would need a custom processor to support the
> NITF format [1]
>
> There does appear to be a Java library to deal with it but its
> licensing isn't ASL v2 compatible so we're not likely to be able to
> play along anytime soon.  But that should be a good way to get started
> building what you need.
>
> The processor would unpack the items out of the NITF bundle and then
> could send out the full size images if that is how it works.  Those
> full size images can then be resized to thumbnails.  Tons of ways to
> play this one and it is a very common style of use case.
>
> Thanks
> Joe
>
> [1] https://en.wikipedia.org/wiki/National_Imagery_Transmission_Format
> [2] https://github.com/codice/imaging-nitf
>
> On Mon, Aug 17, 2015 at 1:45 PM, plj <[hidden
> email]> wrote:
>
> > Howdy,
> >
> >Thanks for the reply.  I think my situation is different
> than you suggest.  I have an image file in NITF format.  That file may have
> multiple images and multiple bands inside that one file.  The software that
> I have that reads that file does create a thumbnail for each image and each
> band inside that file.   I don't know how many thumbnails I need until I
> read the files metadata.
> >
> >
> > Thank
> >
> > Pat
> >
> > From: Dan Bress [via Apache NiFi (incubating) Developer List] 
> > [mailto:[hidden
> email]]
> > Sent: Monday, August 17, 2015 1:40 PM
> > To: Jones, Patrick L. <[hidden
> email]>
> > Subject: Re: Process to create multiple files
> >
> > plj,
> >I would not recommend having this processor create multiple
> thumbnails.  What I would recommend is the following:
> >
> >Create a new processor called "CreateThumbnail" or "RescaleImage"
> >
> > Then have a configuration on the processor that says what size the
> output image should be(e.g. 128x128, or 1/X of original size).
> >
> > Your new processor will read in the incoming image, and rescale it
> down to the user specified size and pass it forward.
> >
> >  Now if you want to create a 128x128 64x64 and 32x32 sized images
> you would do the following.
> >
> > (GetFile)->(RescaleImage configured to 128x128)->(PutFile)
> >   |->(RescaleImage configured to 64x64)->(PutFile)
> >   \->(RescaleImage configured to 32x32)->(PutFile)
> >
> > Where GetFile has 3 success relationships, each going to a different
> RescaleImage processor.
> >
> > I think it makes more sense to have one processor create one file, then
> you can use the flow to visually configure how many copies of the file you
> want to make.  This should make this processor simpler and more reusable.
> >
> >
> > Dan Bress
> > Software Engineer
> > ONYX Consulting Services
> >
> > 
> > From: plj <[hidden email]>
> > Sent: Monday, August 17, 2015 1:27 PM
> > To: [hidden email]
> > Subject: Process to create  multiple files
> >
> > Howdy,
> >
> >   I'm new to NiFi so please bear with me.  What I want to accomplish is:
> >   read an image file
> >  process the file to create one or more thumbnails from the image.
> >  Send the resulting thumbnails along the flow
> >
> > So I can use GetFile to read the file and then send it along.  I
> think I
> > need to write a custom java processor that will process the image file
> and
> > then send each of the thumbnail files (say .jpg for now) on to the next
> > thing in the flow (say PutFile for example).
> >
> > Are there suggestions on what I should implement or extend to create
> my
> > custom processor?  It will take in one file and output multiple files.
> > Would extending "PutFile" so that it processed and then puts each
> thumbnail
> > on the flow be a good strategy?  Other ideas?
> >
> > Thank you,
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
> > Sent from the Apache NiFi (incubating) Developer List mailing list
> archive at Nabble.com.
> >
> > 
> > If you reply to this em

RE: Process to create multiple files

2015-08-17 Thread plj
Ok that makes sense. What processor would I extend/implement to allow me to 
send out multiple files from the process that unpacks the NITF bundle?  Any 
examples?

Thank,

Pat


From: Joe Witt [via Apache NiFi (incubating) Developer List] 
[mailto:ml-node+s39713n2513...@n7.nabble.com]
Sent: Monday, August 17, 2015 1:56 PM
To: Jones, Patrick L. 
Subject: Re: Process to create multiple files

Hello Pat,

Yeah makes sense.  You would need a custom processor to support the
NITF format [1]

There does appear to be a Java library to deal with it but its
licensing isn't ASL v2 compatible so we're not likely to be able to
play along anytime soon.  But that should be a good way to get started
building what you need.

The processor would unpack the items out of the NITF bundle and then
could send out the full size images if that is how it works.  Those
full size images can then be resized to thumbnails.  Tons of ways to
play this one and it is a very common style of use case.

Thanks
Joe

[1] https://en.wikipedia.org/wiki/National_Imagery_Transmission_Format
[2] https://github.com/codice/imaging-nitf

On Mon, Aug 17, 2015 at 1:45 PM, plj <[hidden 
email]> wrote:

> Howdy,
>
>Thanks for the reply.  I think my situation is different than 
> you suggest.  I have an image file in NITF format.  That file may have 
> multiple images and multiple bands inside that one file.  The software that I 
> have that reads that file does create a thumbnail for each image and each 
> band inside that file.   I don't know how many thumbnails I need until I read 
> the files metadata.
>
>
> Thank
>
> Pat
>
> From: Dan Bress [via Apache NiFi (incubating) Developer List] [mailto:[hidden 
> email]]
> Sent: Monday, August 17, 2015 1:40 PM
> To: Jones, Patrick L. <[hidden 
> email]>
> Subject: Re: Process to create multiple files
>
> plj,
>I would not recommend having this processor create multiple thumbnails.  
> What I would recommend is the following:
>
>Create a new processor called "CreateThumbnail" or "RescaleImage"
>
> Then have a configuration on the processor that says what size the output 
> image should be(e.g. 128x128, or 1/X of original size).
>
> Your new processor will read in the incoming image, and rescale it down 
> to the user specified size and pass it forward.
>
>  Now if you want to create a 128x128 64x64 and 32x32 sized images you 
> would do the following.
>
> (GetFile)->(RescaleImage configured to 128x128)->(PutFile)
>   |->(RescaleImage configured to 64x64)->(PutFile)
>   \->(RescaleImage configured to 32x32)->(PutFile)
>
> Where GetFile has 3 success relationships, each going to a different 
> RescaleImage processor.
>
> I think it makes more sense to have one processor create one file, then you 
> can use the flow to visually configure how many copies of the file you want 
> to make.  This should make this processor simpler and more reusable.
>
>
> Dan Bress
> Software Engineer
> ONYX Consulting Services
>
> 
> From: plj <[hidden email]>
> Sent: Monday, August 17, 2015 1:27 PM
> To: [hidden email]
> Subject: Process to create  multiple files
>
> Howdy,
>
>   I'm new to NiFi so please bear with me.  What I want to accomplish is:
>   read an image file
>  process the file to create one or more thumbnails from the image.
>  Send the resulting thumbnails along the flow
>
> So I can use GetFile to read the file and then send it along.  I think I
> need to write a custom java processor that will process the image file and
> then send each of the thumbnail files (say .jpg for now) on to the next
> thing in the flow (say PutFile for example).
>
> Are there suggestions on what I should implement or extend to create my
> custom processor?  It will take in one file and output multiple files.
> Would extending "PutFile" so that it processed and then puts each thumbnail
> on the flow be a good strategy?  Other ideas?
>
> Thank you,
>
>
>
>
>
> --
> View this message in context: 
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
> Nabble.com.
>
> 
> If you reply to this email, your message will be added to the discussion 
> below:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2511.html
> To unsubscribe from Process to create multiple files, click here<
> NAML
>
>

Re: AMQP Processor Github link

2015-08-17 Thread Joe Witt
https://github.com/helicopterman22/nifi_amqp_processors

Thanks
Joe

On Mon, Aug 17, 2015 at 2:23 PM, DAVID SMITH  wrote:
> Please find below a link to my nifi-amqp-processor project, I would welcome 
> any feedback and also any help with creating Junit tests, particularly how to 
> create a mock broker/queue.
> Dave
>
>
>
> helicopterman22/nifi_amqp_processors
>
> |   |
> |   |  |   |   |   |   |   |
> | helicopterman22/nifi_amqp_processorsnifi_amqp_processors - A set of 
> processors for sending and retrieving messages from an AMQP broker |
> |  |
> | View on github.com | Preview by Yahoo |
> |  |
> |   |
>
> ×


AMQP Processor Github link

2015-08-17 Thread DAVID SMITH
Please find below a link to my nifi-amqp-processor project, I would welcome any 
feedback and also any help with creating Junit tests, particularly how to 
create a mock broker/queue.
Dave



helicopterman22/nifi_amqp_processors

|   |
|   |  |   |   |   |   |   |
| helicopterman22/nifi_amqp_processorsnifi_amqp_processors - A set of 
processors for sending and retrieving messages from an AMQP broker |
|  |
| View on github.com | Preview by Yahoo |
|  |
|   |

× 


Re: Process to create multiple files

2015-08-17 Thread Joe Witt
Hello Pat,

Yeah makes sense.  You would need a custom processor to support the
NITF format [1]

There does appear to be a Java library to deal with it but its
licensing isn't ASL v2 compatible so we're not likely to be able to
play along anytime soon.  But that should be a good way to get started
building what you need.

The processor would unpack the items out of the NITF bundle and then
could send out the full size images if that is how it works.  Those
full size images can then be resized to thumbnails.  Tons of ways to
play this one and it is a very common style of use case.

Thanks
Joe

[1] https://en.wikipedia.org/wiki/National_Imagery_Transmission_Format
[2] https://github.com/codice/imaging-nitf

On Mon, Aug 17, 2015 at 1:45 PM, plj  wrote:
> Howdy,
>
>Thanks for the reply.  I think my situation is different than 
> you suggest.  I have an image file in NITF format.  That file may have 
> multiple images and multiple bands inside that one file.  The software that I 
> have that reads that file does create a thumbnail for each image and each 
> band inside that file.   I don't know how many thumbnails I need until I read 
> the files metadata.
>
>
> Thank
>
> Pat
>
> From: Dan Bress [via Apache NiFi (incubating) Developer List] 
> [mailto:ml-node+s39713n251...@n7.nabble.com]
> Sent: Monday, August 17, 2015 1:40 PM
> To: Jones, Patrick L. 
> Subject: Re: Process to create multiple files
>
> plj,
>I would not recommend having this processor create multiple thumbnails.  
> What I would recommend is the following:
>
>Create a new processor called "CreateThumbnail" or "RescaleImage"
>
> Then have a configuration on the processor that says what size the output 
> image should be(e.g. 128x128, or 1/X of original size).
>
> Your new processor will read in the incoming image, and rescale it down 
> to the user specified size and pass it forward.
>
>  Now if you want to create a 128x128 64x64 and 32x32 sized images you 
> would do the following.
>
> (GetFile)->(RescaleImage configured to 128x128)->(PutFile)
>   |->(RescaleImage configured to 64x64)->(PutFile)
>   \->(RescaleImage configured to 32x32)->(PutFile)
>
> Where GetFile has 3 success relationships, each going to a different 
> RescaleImage processor.
>
> I think it makes more sense to have one processor create one file, then you 
> can use the flow to visually configure how many copies of the file you want 
> to make.  This should make this processor simpler and more reusable.
>
>
> Dan Bress
> Software Engineer
> ONYX Consulting Services
>
> 
> From: plj <[hidden email]>
> Sent: Monday, August 17, 2015 1:27 PM
> To: [hidden email]
> Subject: Process to create  multiple files
>
> Howdy,
>
>   I'm new to NiFi so please bear with me.  What I want to accomplish is:
>   read an image file
>  process the file to create one or more thumbnails from the image.
>  Send the resulting thumbnails along the flow
>
> So I can use GetFile to read the file and then send it along.  I think I
> need to write a custom java processor that will process the image file and
> then send each of the thumbnail files (say .jpg for now) on to the next
> thing in the flow (say PutFile for example).
>
> Are there suggestions on what I should implement or extend to create my
> custom processor?  It will take in one file and output multiple files.
> Would extending "PutFile" so that it processed and then puts each thumbnail
> on the flow be a good strategy?  Other ideas?
>
> Thank you,
>
>
>
>
>
> --
> View this message in context: 
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
> Nabble.com.
>
> 
> If you reply to this email, your message will be added to the discussion 
> below:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2511.html
> To unsubscribe from Process to create multiple files, click 
> here.
> NAML
>
>
>
>
> --
> View this message in context: 
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2512.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at

RE: Process to create multiple files

2015-08-17 Thread plj
Howdy,

   Thanks for the reply.  I think my situation is different than 
you suggest.  I have an image file in NITF format.  That file may have multiple 
images and multiple bands inside that one file.  The software that I have that 
reads that file does create a thumbnail for each image and each band inside 
that file.   I don't know how many thumbnails I need until I read the files 
metadata.


Thank

Pat

From: Dan Bress [via Apache NiFi (incubating) Developer List] 
[mailto:ml-node+s39713n251...@n7.nabble.com]
Sent: Monday, August 17, 2015 1:40 PM
To: Jones, Patrick L. 
Subject: Re: Process to create multiple files

plj,
   I would not recommend having this processor create multiple thumbnails.  
What I would recommend is the following:

   Create a new processor called "CreateThumbnail" or "RescaleImage"

Then have a configuration on the processor that says what size the output 
image should be(e.g. 128x128, or 1/X of original size).

Your new processor will read in the incoming image, and rescale it down to 
the user specified size and pass it forward.

 Now if you want to create a 128x128 64x64 and 32x32 sized images you would 
do the following.

(GetFile)->(RescaleImage configured to 128x128)->(PutFile)
  |->(RescaleImage configured to 64x64)->(PutFile)
  \->(RescaleImage configured to 32x32)->(PutFile)

Where GetFile has 3 success relationships, each going to a different 
RescaleImage processor.

I think it makes more sense to have one processor create one file, then you can 
use the flow to visually configure how many copies of the file you want to 
make.  This should make this processor simpler and more reusable.


Dan Bress
Software Engineer
ONYX Consulting Services


From: plj <[hidden email]>
Sent: Monday, August 17, 2015 1:27 PM
To: [hidden email]
Subject: Process to create  multiple files

Howdy,

  I'm new to NiFi so please bear with me.  What I want to accomplish is:
  read an image file
 process the file to create one or more thumbnails from the image.
 Send the resulting thumbnails along the flow

So I can use GetFile to read the file and then send it along.  I think I
need to write a custom java processor that will process the image file and
then send each of the thumbnail files (say .jpg for now) on to the next
thing in the flow (say PutFile for example).

Are there suggestions on what I should implement or extend to create my
custom processor?  It will take in one file and output multiple files.
Would extending "PutFile" so that it processed and then puts each thumbnail
on the flow be a good strategy?  Other ideas?

Thank you,





--
View this message in context: 
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
Nabble.com.


If you reply to this email, your message will be added to the discussion below:
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2511.html
To unsubscribe from Process to create multiple files, click 
here.
NAML




--
View this message in context: 
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2512.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
Nabble.com.

Re: Process to create multiple files

2015-08-17 Thread Dan Bress
plj,
   I would not recommend having this processor create multiple thumbnails.  
What I would recommend is the following:

   Create a new processor called "CreateThumbnail" or "RescaleImage"

Then have a configuration on the processor that says what size the output 
image should be(e.g. 128x128, or 1/X of original size).

Your new processor will read in the incoming image, and rescale it down to 
the user specified size and pass it forward.

 Now if you want to create a 128x128 64x64 and 32x32 sized images you would 
do the following.

(GetFile)->(RescaleImage configured to 128x128)->(PutFile)
  |->(RescaleImage configured to 64x64)->(PutFile) 
  \->(RescaleImage configured to 32x32)->(PutFile)

Where GetFile has 3 success relationships, each going to a different 
RescaleImage processor.

I think it makes more sense to have one processor create one file, then you can 
use the flow to visually configure how many copies of the file you want to 
make.  This should make this processor simpler and more reusable.
 

Dan Bress
Software Engineer
ONYX Consulting Services


From: plj 
Sent: Monday, August 17, 2015 1:27 PM
To: d...@nifi.incubator.apache.org
Subject: Process to create  multiple files

Howdy,

  I'm new to NiFi so please bear with me.  What I want to accomplish is:
  read an image file
 process the file to create one or more thumbnails from the image.
 Send the resulting thumbnails along the flow

So I can use GetFile to read the file and then send it along.  I think I
need to write a custom java processor that will process the image file and
then send each of the thumbnail files (say .jpg for now) on to the next
thing in the flow (say PutFile for example).

Are there suggestions on what I should implement or extend to create my
custom processor?  It will take in one file and output multiple files.
Would extending "PutFile" so that it processed and then puts each thumbnail
on the flow be a good strategy?  Other ideas?

Thank you,





--
View this message in context: 
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
Nabble.com.


Process to create multiple files

2015-08-17 Thread plj
Howdy,

  I'm new to NiFi so please bear with me.  What I want to accomplish is:
  read an image file
 process the file to create one or more thumbnails from the image.
 Send the resulting thumbnails along the flow

So I can use GetFile to read the file and then send it along.  I think I
need to write a custom java processor that will process the image file and
then send each of the thumbnail files (say .jpg for now) on to the next
thing in the flow (say PutFile for example).

Are there suggestions on what I should implement or extend to create my
custom processor?  It will take in one file and output multiple files. 
Would extending "PutFile" so that it processed and then puts each thumbnail
on the flow be a good strategy?  Other ideas?

Thank you,





--
View this message in context: 
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
Nabble.com.


Time to start closing in on 0.3.0

2015-08-17 Thread Joe Witt
Team,

The current open issues listed against 0.3.0 are here [1].

A handy way to find items listed in Patch Review state (not just on
0.3.0) is here [2].

There are  few big ticket items that seem to be good to push out (no
need to rush them and broader input could be good).  There are some
that are really critical to get right (needs thorough RTC and
testing).

Anything not listed on 0.3.0 that folks feel strongly should be there?

Thanks
Joe

[1] http://s.apache.org/ez
[2] https://issues.apache.org/jira/issues/?filter=12331963


Re: Problems w/ Unsubscribe

2015-08-17 Thread Joe Witt
Hello,

Sending an email to 'dev-unsubscr...@nifi.apache.org' should do the
trick provided you do so from the address that you've subscribed with.

I'll send the command to manually remove it.

Good luck

Thanks
Joe

On Mon, Aug 17, 2015 at 10:07 AM, Adams, Jeremiah
 wrote:
> I am having issues with unsubscribing from the email lists. I've tried
> several times via:
>
> dev-unsubscr...@nifi.incubator.apache.org,
> dev-unsubscr...@nifi-apache.org,
> dev-unsubscr...@nifi.apache.org
>
>
> Nothing seems to work.
>
> *Jeremiah Adams*
>
> Senior Software Developer
> Pearson
>
> 2154 East Commons Ave.
> Suite 400
> Centennial, CO 80122
>
>
> Always Learning
> Learn more at www.pearson.com


Problems w/ Unsubscribe

2015-08-17 Thread Adams, Jeremiah
I am having issues with unsubscribing from the email lists. I've tried
several times via:

dev-unsubscr...@nifi.incubator.apache.org,
dev-unsubscr...@nifi-apache.org,
dev-unsubscr...@nifi.apache.org


Nothing seems to work.

*Jeremiah Adams*

Senior Software Developer
Pearson

2154 East Commons Ave.
Suite 400
Centennial, CO 80122


Always Learning
Learn more at www.pearson.com


Re: Lots o branches in git

2015-08-17 Thread Mike Drob
Also useful is
git remote prune origin

which will clean up your local tracking branches for remote branches that
have been deleted. You can specify different remote names instead of
origin, as well.


On Mon, Aug 17, 2015 at 7:44 AM, Joe Witt  wrote:

> Thanks Dan.
>
> Please be sure you did the command needed to remove the remote/origin
> branch.  It is possible that the email just hasn't come through but i
> suspect not as the ASF git repo still shows them.  This is the command
> set you want:
>
> #remove remote branch
> git push origin —delete NIFI-633
> git push origin —delete NIFI-632
> git push origin —delete NIFI-680
> git push origin —delete NIFI-682
> git push origin —delete NIFI-704
> git push origin —delete NIFI-735
>
>
> Thanks
> Joe
>
> On Mon, Aug 17, 2015 at 8:32 AM, Dan Bress 
> wrote:
> > Joe, thanks for the heads up.  I'll delete all of mine, as they are all
> resolved in JIRA
> > NIFI-633: danbress
> > NIFI-632: danbress
> > NIFI-680: danbress
> > NIFI-682: danbress
> > NIFI-704: danbress
> > NIFI-735: danbress
> >
> > Dan Bress
> > Software Engineer
> > ONYX Consulting Services
> >
> > 
> > From: Joe Witt 
> > Sent: Sunday, August 16, 2015 9:34 PM
> > To: dev@nifi.apache.org
> > Subject: Lots o branches in git
> >
> > Team,
> >
> > Friendly reminder regarding branches that appear stale.  If you no
> > longer need them please consider removing them.
> >
> > Last activity more than 2 months ago:
> > --
> > ListHDFS : markup
> > NIFI-25: markap
> > NIFI-85: mcgilma
> > NIFI-180: mcgilma
> > NIFI-250: mcgilma
> > NIFI-271: apiri
> > NIFI-292: mcgilma
> > NIFI-353: mcgilma
> > NIFI-376: mcgilma
> > NIFI-433: devriesb
> > NIFI-475: mcgilma
> > NIFI-521: mcgilma
> > NIFI-633: danbress
> > improve-pro-performance: markap, mcgilma
> > journaling-prov-repo: markap
> > prov-query-language: markap
> >
> > Seems definitely eligible to remove (release is done and tagged):
> > --
> > release-nifi-0.1.0-rc13: markup
> > release-nifi-0.2.0-incubating: apiri
> > release-nifi-0.2.0-incubating-rc1: mcgilma
> >
> > Active within the last 2 months:
> > ---
> > NIFI-632: danbress
> > NIFI-640: mcgilma
> > NIFI-680: danbress
> > NIFI-682: danbress
> > NIFI-704: danbress
> > NIFI-724: mcgilma
> > NIFI-731: markap
> > NIFI-735: danbress
> > NIFI-744: markap
> > NIFI-793: markap
> > NIFI-818: markap
> >
> > Reminder on commands that can help:
> > ———
> > #remove local branch
> > git branch -D 
> > #remove remote branch
> > git push origin —delete 
> >
> > Thanks
> > Joe
>


Re: [DISCUSS] Feature proposal: Read-only mode as default

2015-08-17 Thread Joe Witt
Ok all the thread has died down and reading where there is a strong
consensus the main thing that comes out the other end here is the
desire for 'undo'.  I believe this is well captured in [1].

Other ideas where presented here that are also interesting and perhaps
once we get make progress on the other more clarity will occur on
those.  Great discussion all - thanks!

Joe

[1] 
https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows

On Fri, Aug 14, 2015 at 8:13 AM, Joe Skora  wrote:
> +1 for read-only as a toggle with a system property to configure the
> default.
>
> +1 (+2 +3) for Undo.  Undo is a common and expected application feature
> even in web UIs that eliminates a lot of pain and anxiety.  But Undo is
> complicated, especially in a web UI, so perhaps it could be phased-in?  I
> see a lot of value in even one level of undo, like GMAIL, so that might be
> a good target for a first phase.  That could eventually be expanded to
> multiple levels with less pressure and more time to get it right.
>
> On Tue, Aug 11, 2015 at 12:22 PM, Ryan Blue  wrote:
>
>> +1 for the consensus view as Joe summarized it.
>>
>> I also agree with only using confirmations sparingly.
>>
>> rb
>>
>>
>> On 08/11/2015 07:07 AM, Ricky Saltzer wrote:
>>
>>> +1 for read-only by default. It would be nice to have some easy way to
>>> tell
>>> if you're in edit/view mode, perhaps the canvas be black/white during view
>>> and color during edit? or, something along those lines.
>>>
>>> On Tue, Aug 11, 2015 at 9:57 AM, Michael Moser 
>>> wrote:
>>>
>>> "undo" seems to be the stretch goal that I think that would solve most
 concerns of unintended modifications to a graph.  +1

 Meanwhile, I'd like to caution against confirmation dialogs.  Extra
 clicks
 quickly annoy users while they work.  I suggest no dialog when deleting a
 single queue or processor, or when moving 'objects' around.  Perhaps
 bring
 a confirmation dialog into play only when deleting more than 1 'object'.

 Personally I really like the idea of a read-only mode toggle, even if it
 was not persisted as a user preference and was only remembered during the
 current browser 'session'.

 -- Mike

 On Tue, Aug 11, 2015 at 9:11 AM, Rob Moran  wrote:

 The consensus view looks good to me. I believe preserving the current
>
 model

> as Joe describes it is a smart approach.
>
> An undo action and restrained use of confirmation dialogs are minimal
> and
> should not significantly impede experienced operators. More often than
>
 not,

> I'd bet a user would expect similar functionality.
>
> As is evident by the views expressed around read-only/locking, it will
> be
> very difficult to please a majority of users with different user modes
>
 and

> UI states.
>
> On Tue, Aug 11, 2015 at 8:29 AM Joe Witt  wrote:
>
> To summarize where we're at ...
>>
>> Proposed approaches (summary):
>>
>> - Establish a default read-only view whereby an operator can enable
>> edit mode.  Use confirmation dialogs for deletes.
>>
>> - Keep the current model but add support for ‘undo’.
>>
>> - Let the user choose whether to lock their view or not as user
>>
> preference.
>
>>
>> - For delete add more protections to make accidents less likely and
>> for movement provide an explicit ‘move action’.
>>
>> The idea of locking seems to have some strong views on both sides and
>> both sides have even been argued by the same people (i now count
>> myself among that group).
>>
>> It looks like a consensus view there though is:
>>
>> - Try to make panning the view of the flow and moving components on
>> the flow two specific/discrete actions to avoid accidental movement.
>>
>> - Add support for undo
>>
>> - Provide sufficient dialog/protection for delete cases.
>>
>> This preserves the model whereby we generally trust that the user will
>> do the right thing and we’ll do more to help them and that when they
>> don’t they will learn and have help to promptly restore a good state.
>> How do folks feel about that?
>>
>> Thanks
>> Joe
>>
>> On Tue, Aug 11, 2015 at 5:11 AM, Alex Moundalexis 
>> wrote:
>>
>>> Counterpoint, accidents happen; I prefer to enable users to learn
>>>
>> from

> mistakes and exercise more care next time. You can't remove every
>>>
>> mildly
>
>> sharp edge without impacting experienced operators; resist the urge
>>>
>> at

> a
>
>> few
>>
>>> comments to dumb down the tool.
>>>
>>> If some protection is added to the UI, suggest an option for "expert
>>>
>> mode"
>>
>>> that retains original functionality... that way experienced operators
>>>
>> 

Re: Best way to replace a workflow

2015-08-17 Thread Joe Witt
Hello

I've had this thread flagged for a while now.  The sorts of concerns
raised in it have been fairly common patterns of requests from various
groups.  I believe they are fairly well accounted for in the present
list of feature proposals as seen here [1].

Please take a look and advise if you think something is missing.

Thanks
Joe

[1] https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals

On Wed, Apr 29, 2015 at 9:46 AM, Esteban Aliverti
 wrote:
> Thanks Adam,
>
> I guess I was using templates because I didn't know about the
> existence of flow.xml.gz
> :).
> Now, if I have 2 different instances of NiFi, and I want to override the
> workflow in one of them with the workflow of the other, do I need to do
> anything else than just overwrite flow.xml.gz?
>
> Regards,
>
>
>
> 
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
> On Wed, Apr 29, 2015 at 2:04 PM, Adam Taft  wrote:
>
>> You can get a lot of mileage using the following suggestions:
>>
>> -  There's nothing preventing you from storing the flow.xml.gz file into
>> version control.  While this file is binary and you won't see any diffs on
>> it, your VCS (git, svn, etc.) won't have a problem storing it.  If storing
>> the flow.xml.gz as binary is a problem, then you could simply uncompress it
>> to flow.xml first and version control that.
>>
>> -  You might likewise want to version control your entire conf directory.
>>
>> -  For any processor properties which can be configured using NIFI's
>> expression language, you can reference System properties and/or environment
>> variables.  This allows you to have variant configurations based on the
>> deployment context. Things like hostnames or database connection properties
>> could in theory be configured externally.
>>
>> -  Your custom controller services can likewise be configured to utilize
>> system properties or environment vars.  Most standard controller services
>> don't have this capability yet, but work is being done that might help in
>> this area as well.
>>
>> I personally don't like using templates to solve your problem.  Having flow
>> snippets that need to be stored and applied between dev/int/prod doesn't
>> make a lot of sense to me.  Better in my mind is to have a single
>> flow.xml.gz that you can carry between those tiers.  NIFI should allow you
>> to get most of the way there today, but if you run into problems, I'm sure
>> you could affect additional changes to help with this issue.
>>
>> Adam
>>
>>
>>
>> On Tue, Apr 28, 2015 at 9:58 AM, Esteban Aliverti <
>> esteban.alive...@gmail.com> wrote:
>>
>> > Joe,
>> > thanks for your prompt response.
>> >
>> > Let me give you more background about what I'm currently doing:
>> > We are currently a group of 3 developers working on a NiFi workflow. My
>> two
>> > colleagues are in charge of building the individual processors required
>> for
>> > the workflow we are trying to create. My role, even if I'm also a
>> Processor
>> > developer myself, is kind of the "integrator" of these processors. I
>> have a
>> > local NiFi running and I'm in charge of the "workflow" design. I use this
>> > local instance as a playground for develop and initial testing.
>> > Once I've updated my local workflow, I usually "export" it into another
>> > instance that is currently being used by QA. This QA instance is only
>> > modified after I validate that my local instance is stable enough. Given
>> > that there are a number of other developers using this QA instance as
>> > clients, the update process is not so easy. Before I introduce any change
>> > in QA, I need to coordinate with the QA department first. This is one of
>> > the biggest reasons I keep a local NiFi instance for development. I can
>> do
>> > whatever I want whenever I want to in my local instance but I need to be
>> > very careful when I modify the QA instance.
>> >
>> > I have asked similar questions in this forum in the past and one of your
>> > recommendations was to use the same workflow for both QA and DEV. You
>> > suggested me to use an attribute in the FlowFiles as some kind of "flag"
>> so
>> > it can be routed to the DEV branch of the workflow. I see 2 problems with
>> > this approach (maybe because of my own development workflow):
>> > 1.- Sometimes I don't have connectivity to the server where the QA NiFi
>> > instance is. I would like to be able to continue my development tasks
>> even
>> > if I'm offline.
>> > 2.- My workflow is not stateless. I have a bunch of Controller Services
>> > that are used through the workflow that intentionally keep state. If I
>> use
>> > the same NiFi instance for both QA and DEV, I will not only have to add a
>> > new "flag" attribute in my workflows, but I will also has to implement
>> some
>> > kind of mechanism in my Controller Services as well to separate DEV and
>> QA
>> > data. I don't think this is a really good idea.
>> >
>> > Regards,
>> >
>> >
>> > XX

Re: Update to nifi-nar-maven-plugin

2015-08-17 Thread Joe Witt
Matt,

I am supportive of this especially given the nice boost it provides
for people wanting to understand the structure/dependencies of a given
Nar.

Also 3.0.5 is from early 2013 and 3.1.0 is from later 2013 so we're
still going a long ways back [1].

Thanks
Joe

[1] https://maven.apache.org/docs/history.html

On Mon, Aug 17, 2015 at 8:12 AM, Matt Gilman  wrote:
> Awhile back I proposed a new maven plugin that would allow a processor
> developer to easily see the provided dependencies their NAR would be
> inheriting. This can become unclear if the processors depend on a
> Controller Service API NAR (either directly or transitively through another
> Controller Service API). After sending this email awhile ago, nobody seemed
> opposed to moving to a minimum maven version of >= 3.1.0. I would like to
> move forward with including NIFI-376 in our 0.3.0 release.
>
> See the original message below for all the details. Any objections?
>
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-376
>
> On Tue, Mar 17, 2015 at 3:47 PM, Matt Gilman 
> wrote:
>
>> I am in the process of adding some functionality to the
>> nifi-nar-maven-plugin. I'm hoping that someone has some guidance or more
>> knowledge of Maven than I do.
>>
>> Background -
>> NiFi provides classpath isolation using NARs [1]. A NAR can have a parent
>> NAR. This is typically used to reference controller service definitions.
>> For instance, if your processor needs to use the SSLContextService. The
>> parent NAR maps directly to the parent classloader of the NAR in question.
>>
>> Known Issue -
>> When developing a NAR that has a NAR dependency, you will need to know the
>> transitive dependencies that will be automatically added the NARs classpath
>> at runtime. By definition, NARs 'include dependencies' (like a WAR file).
>> This means that traditional maven dependency tree listings do not show the
>> transitive dependencies in question. Determining the provided dependencies
>> becomes very difficult even for those most familiar. The update to the
>> nifi-nar-maven-plugin prints this dependency hierarchy.
>>
>> Concession -
>> The proposed solution is not 100% accurate as the framework dynamically
>> injects some JEE/Jetty dependencies automatically to support loading custom
>> UIs from any NAR. Additionally items in the lib directory are also
>> automatically included in every classpath. I realize these are important
>> but they cannot be resolved using Maven at build time due to the dynamic
>> nature of the classpath at runtime (putting a jar in the lib directory).
>> However, the plugin still has merit in helping developers understand what
>> the implications are when depending on a NAR.
>>
>> Reason for this Email -
>> I've created the plugin [2]. However, the component that performs the
>> artifact resolution has moved from org.sonatype to org.eclipse as of Maven
>> 3.1. I'm not sure how I would go about supporting both at the same time.
>> Others have also had this issue [3] but I've not seen a solution that
>> addresses the concern directly.
>>
>> Ultimate Question -
>> 1) Should we be concerned with supporting Maven 3.0? The current minimum
>> version needed for NiFi is 3.0.5 according to the Quickstart guide [4].
>> 2) If we want to support Maven 3.0, is there are solution that can support
>> both Maven 3.0 and 3.1 concurrently? Is there something that I overlooked
>> or missed entirely?
>>
>> I'm hoping to avoid having to release multiple versions of the plugin.
>>
>> [1]
>> https://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#nars
>> [2]
>> https://github.com/apache/incubator-nifi/blob/NIFI-376/nifi-nar-maven-plugin/src/main/java/org/apache/nifi/NarProvidedDependenciesMojo.java
>> [3]
>> http://stackoverflow.com/questions/17685441/can-a-maven-mojo-relying-on-aether-be-compatible-with-maven-3-0-x-and-3-1-x
>> [4] https://nifi.incubator.apache.org/development/quickstart.html
>>


Re: Lots o branches in git

2015-08-17 Thread Joe Witt
Thanks Dan.

Please be sure you did the command needed to remove the remote/origin
branch.  It is possible that the email just hasn't come through but i
suspect not as the ASF git repo still shows them.  This is the command
set you want:

#remove remote branch
git push origin —delete NIFI-633
git push origin —delete NIFI-632
git push origin —delete NIFI-680
git push origin —delete NIFI-682
git push origin —delete NIFI-704
git push origin —delete NIFI-735


Thanks
Joe

On Mon, Aug 17, 2015 at 8:32 AM, Dan Bress  wrote:
> Joe, thanks for the heads up.  I'll delete all of mine, as they are all 
> resolved in JIRA
> NIFI-633: danbress
> NIFI-632: danbress
> NIFI-680: danbress
> NIFI-682: danbress
> NIFI-704: danbress
> NIFI-735: danbress
>
> Dan Bress
> Software Engineer
> ONYX Consulting Services
>
> 
> From: Joe Witt 
> Sent: Sunday, August 16, 2015 9:34 PM
> To: dev@nifi.apache.org
> Subject: Lots o branches in git
>
> Team,
>
> Friendly reminder regarding branches that appear stale.  If you no
> longer need them please consider removing them.
>
> Last activity more than 2 months ago:
> --
> ListHDFS : markup
> NIFI-25: markap
> NIFI-85: mcgilma
> NIFI-180: mcgilma
> NIFI-250: mcgilma
> NIFI-271: apiri
> NIFI-292: mcgilma
> NIFI-353: mcgilma
> NIFI-376: mcgilma
> NIFI-433: devriesb
> NIFI-475: mcgilma
> NIFI-521: mcgilma
> NIFI-633: danbress
> improve-pro-performance: markap, mcgilma
> journaling-prov-repo: markap
> prov-query-language: markap
>
> Seems definitely eligible to remove (release is done and tagged):
> --
> release-nifi-0.1.0-rc13: markup
> release-nifi-0.2.0-incubating: apiri
> release-nifi-0.2.0-incubating-rc1: mcgilma
>
> Active within the last 2 months:
> ---
> NIFI-632: danbress
> NIFI-640: mcgilma
> NIFI-680: danbress
> NIFI-682: danbress
> NIFI-704: danbress
> NIFI-724: mcgilma
> NIFI-731: markap
> NIFI-735: danbress
> NIFI-744: markap
> NIFI-793: markap
> NIFI-818: markap
>
> Reminder on commands that can help:
> ———
> #remove local branch
> git branch -D 
> #remove remote branch
> git push origin —delete 
>
> Thanks
> Joe


Re: Lots o branches in git

2015-08-17 Thread Dan Bress
Joe, thanks for the heads up.  I'll delete all of mine, as they are all 
resolved in JIRA
NIFI-633: danbress
NIFI-632: danbress
NIFI-680: danbress
NIFI-682: danbress
NIFI-704: danbress
NIFI-735: danbress

Dan Bress
Software Engineer
ONYX Consulting Services


From: Joe Witt 
Sent: Sunday, August 16, 2015 9:34 PM
To: dev@nifi.apache.org
Subject: Lots o branches in git

Team,

Friendly reminder regarding branches that appear stale.  If you no
longer need them please consider removing them.

Last activity more than 2 months ago:
--
ListHDFS : markup
NIFI-25: markap
NIFI-85: mcgilma
NIFI-180: mcgilma
NIFI-250: mcgilma
NIFI-271: apiri
NIFI-292: mcgilma
NIFI-353: mcgilma
NIFI-376: mcgilma
NIFI-433: devriesb
NIFI-475: mcgilma
NIFI-521: mcgilma
NIFI-633: danbress
improve-pro-performance: markap, mcgilma
journaling-prov-repo: markap
prov-query-language: markap

Seems definitely eligible to remove (release is done and tagged):
--
release-nifi-0.1.0-rc13: markup
release-nifi-0.2.0-incubating: apiri
release-nifi-0.2.0-incubating-rc1: mcgilma

Active within the last 2 months:
---
NIFI-632: danbress
NIFI-640: mcgilma
NIFI-680: danbress
NIFI-682: danbress
NIFI-704: danbress
NIFI-724: mcgilma
NIFI-731: markap
NIFI-735: danbress
NIFI-744: markap
NIFI-793: markap
NIFI-818: markap

Reminder on commands that can help:
———
#remove local branch
git branch -D 
#remove remote branch
git push origin —delete 

Thanks
Joe


Re: Update to nifi-nar-maven-plugin

2015-08-17 Thread Matt Gilman
Awhile back I proposed a new maven plugin that would allow a processor
developer to easily see the provided dependencies their NAR would be
inheriting. This can become unclear if the processors depend on a
Controller Service API NAR (either directly or transitively through another
Controller Service API). After sending this email awhile ago, nobody seemed
opposed to moving to a minimum maven version of >= 3.1.0. I would like to
move forward with including NIFI-376 in our 0.3.0 release.

See the original message below for all the details. Any objections?

Matt

[1] https://issues.apache.org/jira/browse/NIFI-376

On Tue, Mar 17, 2015 at 3:47 PM, Matt Gilman 
wrote:

> I am in the process of adding some functionality to the
> nifi-nar-maven-plugin. I'm hoping that someone has some guidance or more
> knowledge of Maven than I do.
>
> Background -
> NiFi provides classpath isolation using NARs [1]. A NAR can have a parent
> NAR. This is typically used to reference controller service definitions.
> For instance, if your processor needs to use the SSLContextService. The
> parent NAR maps directly to the parent classloader of the NAR in question.
>
> Known Issue -
> When developing a NAR that has a NAR dependency, you will need to know the
> transitive dependencies that will be automatically added the NARs classpath
> at runtime. By definition, NARs 'include dependencies' (like a WAR file).
> This means that traditional maven dependency tree listings do not show the
> transitive dependencies in question. Determining the provided dependencies
> becomes very difficult even for those most familiar. The update to the
> nifi-nar-maven-plugin prints this dependency hierarchy.
>
> Concession -
> The proposed solution is not 100% accurate as the framework dynamically
> injects some JEE/Jetty dependencies automatically to support loading custom
> UIs from any NAR. Additionally items in the lib directory are also
> automatically included in every classpath. I realize these are important
> but they cannot be resolved using Maven at build time due to the dynamic
> nature of the classpath at runtime (putting a jar in the lib directory).
> However, the plugin still has merit in helping developers understand what
> the implications are when depending on a NAR.
>
> Reason for this Email -
> I've created the plugin [2]. However, the component that performs the
> artifact resolution has moved from org.sonatype to org.eclipse as of Maven
> 3.1. I'm not sure how I would go about supporting both at the same time.
> Others have also had this issue [3] but I've not seen a solution that
> addresses the concern directly.
>
> Ultimate Question -
> 1) Should we be concerned with supporting Maven 3.0? The current minimum
> version needed for NiFi is 3.0.5 according to the Quickstart guide [4].
> 2) If we want to support Maven 3.0, is there are solution that can support
> both Maven 3.0 and 3.1 concurrently? Is there something that I overlooked
> or missed entirely?
>
> I'm hoping to avoid having to release multiple versions of the plugin.
>
> [1]
> https://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#nars
> [2]
> https://github.com/apache/incubator-nifi/blob/NIFI-376/nifi-nar-maven-plugin/src/main/java/org/apache/nifi/NarProvidedDependenciesMojo.java
> [3]
> http://stackoverflow.com/questions/17685441/can-a-maven-mojo-relying-on-aether-be-compatible-with-maven-3-0-x-and-3-1-x
> [4] https://nifi.incubator.apache.org/development/quickstart.html
>