Re: session recover behaviour in nifi-jms-processor
Just one quick addition: I understood that Nifi prefers message duplication over message loss (which we are fine with in principal, because we can de-duplicate later). However, in our current situation, we have to stop the connection after a certain time interval (mostly roughly 90minutes) due to the following behaviour: 1) session.recover within nifi-jms-processor leads to re-delivery of not-yet-acked messages (as described above) 2) over time, the number of re-delivered message grows, and is "piling up" 3) starting from a certain time, the "pile" of re-delivered messages has grown from 10 or 20 to hundreds 4) in this situation, the consumers "fall behind" too much and cannot cope with the newly arriving messages (in addition to the many not-yet-acked old ones) 5) the JMS topic starts to have a growing number of pending messages for the aforementioned reason 6) the JMS server side gets storage problems due to buffering the pending messages 7) we have to disconnect to avoid JMS server crashes This is just for clarification that the current implementation doesn't only lead to "some" duplicates (which would be fine), but rather to more severe problems on our side. -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-tp14940p14976.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: session recover behaviour in nifi-jms-processor
Just created this ticket: https://issues.apache.org/jira/browse/NIFI-3531 I'm happy to help in any way! -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-tp14940p14977.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Central processor repository
Hello, I remember a while ago that there was a short discussion if it would be good to have a central place somewhere where people could upload the processors they created and others could download them from this central point. It would make life easier compared to surfing the web to find them and I believe would also add to the popularity of Nifi. I don't know if this has been discussed more deeply. Are there any plans for such a central repository? Rgds, Uwe
Re: Central processor repository
Pretty good idea! Would appreciate a place where we all can upload sources (together with some information) and everybody can use or modify it. Best Regards, Uwe > Am 25.02.2017 um 13:10 schrieb Uwe Geercken : > > Hello, > > I remember a while ago that there was a short discussion if it would be good > to have a central place somewhere where people could upload the processors > they created and others could download them from this central point. It would > make life easier compared to surfing the web to find them and I believe would > also add to the popularity of Nifi. > > I don't know if this has been discussed more deeply. Are there any plans for > such a central repository? > > Rgds, > > Uwe
Help with back porting Email bundle from NiFi 1.1.1 to NiFi 0.7.2
Hi At work we are in need of a set of email/smtp processors, our system currently uses NiFi 0.7.2 on Centos 7 and Java 8. We can't upgrade the NiFi version at the present time.I found a set of processors which will do the job we require in NiFi 1.1.1, I have tried running the 1.1.1 nar bundle in 0.7.2 but I get errors stating that ComponentLog is not found.I have taken the source code and changed the parent nar in the poms to 0.7.1 and then changed componentLog to ProcessorLog in the classes. When I then try to rebuild the code I get the following error: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project nifi-email-processors: Compilation failure [ERROR] bootstrap class path not set in conjunction with -source 1.7 [ERROR] /home/dave/workspace-luna/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java:[241,86] error: lambda expressions are not supported in -source 1.7 I have Java 8 set as my default within eclipse, is there something in a higher level pom that is being acted on here. Can anyone help with this please? many thanksDave
Re: Help with back porting Email bundle from NiFi 1.1.1 to NiFi 0.7.2
Hi Dave, The top level pom.xml on the 0.x branch has the following 1.7 1.7 I think you'd need to change that to support Java 8 functionality. Cheers, Richard On Sat, 25 Feb 2017 at 20:19, DAVID SMITH wrote: > Hi > At work we are in need of a set of email/smtp processors, our system > currently uses NiFi 0.7.2 on Centos 7 and Java 8. > We can't upgrade the NiFi version at the present time.I found a set of > processors which will do the job we require in NiFi 1.1.1, I have tried > running the 1.1.1 nar bundle in 0.7.2 but I get errors stating that > ComponentLog is not found.I have taken the source code and changed the > parent nar in the poms to 0.7.1 and then changed componentLog to > ProcessorLog in the classes. When I then try to rebuild the code I get the > following error: > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-compiler-plugin:3.2:compile > (default-compile) on project nifi-email-processors: Compilation failure > [ERROR] bootstrap class path not set in conjunction with -source 1.7 > [ERROR] > /home/dave/workspace-luna/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java:[241,86] > error: lambda expressions are not supported in -source 1.7 > I have Java 8 set as my default within eclipse, is there something in a > higher level pom that is being acted on here. > Can anyone help with this please? > many thanksDave >
Re: Central processor repository
Uwe, Are you aware of this one? https://issues.apache.org/jira/browse/NIFI-2168 Andrew On Sat, Feb 25, 2017, 7:53 AM u...@moosheimer.com wrote: > Pretty good idea! > Would appreciate a place where we all can upload sources (together with > some information) and everybody can use or modify it. > > Best Regards, > Uwe > > > Am 25.02.2017 um 13:10 schrieb Uwe Geercken : > > > > Hello, > > > > I remember a while ago that there was a short discussion if it would be > good to have a central place somewhere where people could upload the > processors they created and others could download them from this central > point. It would make life easier compared to surfing the web to find them > and I believe would also add to the popularity of Nifi. > > > > I don't know if this has been discussed more deeply. Are there any plans > for such a central repository? > > > > Rgds, > > > > Uwe > >
Quick question on LICENSE/NOTICE when bundling controllers and processors
Hi there, Quick question on proper licensing: When bundling Processors, Services and APIs, where should the NOTICES and LICENSES be added to? The PR in question is https://github.com/apache/nifi/pull/1541 My current reading is that all NAR levels will have to include the proper references (although I may reduce a bit of the dependencies by excluding some of the deeper dependencies, specially at the nifi-irc-client-service-api-nar ). Would you agree? Cheers
Re: Quick question on LICENSE/NOTICE when bundling controllers and processors
I just start and I really don’t know much so let see what I can learn when time pass by and hope I can learn as much as you, thank’s > On Feb 25, 2017, at 5:12 PM, Andre wrote: > > Hi there, > > Quick question on proper licensing: > > When bundling Processors, Services and APIs, where should the NOTICES and > LICENSES be added to? > > The PR in question is https://github.com/apache/nifi/pull/1541 > > My current reading is that all NAR levels will have to include the proper > references (although I may reduce a bit of the dependencies by excluding > some of the deeper dependencies, specially at the > nifi-irc-client-service-api-nar ). > > Would you agree? > > Cheers
Re: Quick question on LICENSE/NOTICE when bundling controllers and processors
Hey Andre, I may be off, but to help you along, I will give you my take on things to the best of my understanding. If there are any wrong points, I hope someone can further clarify. For your case, it looks like simply you are simply using binary dependencies. For that case, you have to consider where these items are showing up and how they are released. Your inclusion of a dependency will affect the generated NAR (nifi-irc-processors-nar) and, while it seems to be missing in the current PR, the zip and tgz nifi-assembly artifacts. You shouldn't need to include it in levels lower than this assuming you are talking about JARs that compose the overall NAR. While you are linking these against dependencies, you are not explicitly bringing them into the project through the JARs incorporated in the NAR. Source inclusions are handled similarly but do go a level deeper as they are also bundled with the JARs and are present in the root LICENSE and NOTICE where applicable. Again, both are for similar reasons with the generated source package and JARs bundling this work including the source. Do keep in mind the transitive dependencies. Looking quickly through the pom for kitteh, I see usage of some netty libraries as well as mbassador. These would presumably also be collected upon building the NAR. Of course, the docs we have on the site are quite nice if you need some light reading material ;) https://nifi.apache.org/licensing-guide.html Both the guide and the links from it are good information and a nice reference to revisit when working through these things. Let us know if there are additional questions or if some additional clarification is needed. Hopefully my anecdotal thoughts are both somewhat helpful and mostly correct! On Sat, Feb 25, 2017 at 9:39 PM, Afonso Murakami wrote: > I just start and I really don’t know much so let see what I can learn when > time pass by and hope I can learn as much as you, thank’s > > On Feb 25, 2017, at 5:12 PM, Andre wrote: > > > > Hi there, > > > > Quick question on proper licensing: > > > > When bundling Processors, Services and APIs, where should the NOTICES and > > LICENSES be added to? > > > > The PR in question is https://github.com/apache/nifi/pull/1541 > > > > My current reading is that all NAR levels will have to include the proper > > references (although I may reduce a bit of the dependencies by excluding > > some of the deeper dependencies, specially at the > > nifi-irc-client-service-api-nar ). > > > > Would you agree? > > > > Cheers > >
Re: Quick question on LICENSE/NOTICE when bundling controllers and processors
Thank you for reply so quick I just start and really do not know exactly what I’m doing and I need a lot’s material to read and reference so I can understand better what I’m doing it again thank’s for the email. > On Feb 25, 2017, at 7:33 PM, Aldrin Piri wrote: > > Hey Andre, > > I may be off, but to help you along, I will give you my take on things to > the best of my understanding. If there are any wrong points, I hope > someone can further clarify. > > For your case, it looks like simply you are simply using binary > dependencies. For that case, you have to consider where these items are > showing up and how they are released. Your inclusion of a dependency will > affect the generated NAR (nifi-irc-processors-nar) and, while it seems to > be missing in the current PR, the zip and tgz nifi-assembly artifacts. You > shouldn't need to include it in levels lower than this assuming you are > talking about JARs that compose the overall NAR. While you are linking > these against dependencies, you are not explicitly bringing them into the > project through the JARs incorporated in the NAR. > > Source inclusions are handled similarly but do go a level deeper as they > are also bundled with the JARs and are present in the root LICENSE and > NOTICE where applicable. Again, both are for similar reasons with the > generated source package and JARs bundling this work including the source. > > Do keep in mind the transitive dependencies. Looking quickly through the > pom for kitteh, I see usage of some netty libraries as well as mbassador. > These would presumably also be collected upon building the NAR. > > Of course, the docs we have on the site are quite nice if you need some > light reading material ;) https://nifi.apache.org/licensing-guide.html > Both the guide and the links from it are good information and a nice > reference to revisit when working through these things. > > Let us know if there are additional questions or if some additional > clarification is needed. Hopefully my anecdotal thoughts are both somewhat > helpful and mostly correct! > > On Sat, Feb 25, 2017 at 9:39 PM, Afonso Murakami > wrote: > >> I just start and I really don’t know much so let see what I can learn when >> time pass by and hope I can learn as much as you, thank’s >>> On Feb 25, 2017, at 5:12 PM, Andre wrote: >>> >>> Hi there, >>> >>> Quick question on proper licensing: >>> >>> When bundling Processors, Services and APIs, where should the NOTICES and >>> LICENSES be added to? >>> >>> The PR in question is https://github.com/apache/nifi/pull/1541 >>> >>> My current reading is that all NAR levels will have to include the proper >>> references (although I may reduce a bit of the dependencies by excluding >>> some of the deeper dependencies, specially at the >>> nifi-irc-client-service-api-nar ). >>> >>> Would you agree? >>> >>> Cheers >> >>
