whether nifi can pull data from realtional database like oracle into hdfs
Dear sir, I wanna know whether i can pull data from realtional database like oracle into hdfs ,or pull data from realtional database like oracle into mysql by using nifi? Should i need to develop plugins or something? Looking forward to your early reply,Thank you very much . Sincerely yours.
Re: Recommendation on getting started as a contibutor
Mark Thanks for trying. Indeed I forgot to mention in the guide that one has to add ‘nifi.properties.file.path' as ‘VM Arguments’, updating the guide now. As for logging, everything should work. Even though NiFi uses logback adapter for slf4j, the IDE setup uses log4j and all required JARs should be on the class path. Could you pleas re-verify? Here is what I see in my current setup after running ./gradlew clean eclipse -> importing project into workspace and configuring ‘Run Configurations' Run Configurations: VM arguments: -Dnifi.properties.file.path=/dev/nifi/nifi-assembly/target/nifi-0.3.1-SNAPSHOT-bin/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties Working directory (“Other" is checked in): /dev/nifi/nifi-assembly/target/nifi-0.3.1-SNAPSHOT-bin/nifi-0.3.1-SNAPSHOT CLASSPATH: nifi-api-0.3.1-SNAPSHOT.jar nifi-runtime-0.3.1-SNAPSHOT.jar log4j-core-2.4.jar slf4j-api-1.7.12.jar slf4j-log4j12-1.7.12.jar jul-to-slf4j-1.7.12.jar nifi-nar-utils-0.3.1-SNAPSHOT.jar nifi-properties-0.3.1-SNAPSHOT.jar nifi-documentation-0.3.1-SNAPSHOT.jar log4j-api-2.4.jar log4j-1.2.17.jar Cheers Oleg On Oct 27, 2015, at 12:30 AM, Mark Petronic mailto:markpetro...@gmail.com>> wrote: On Mon, Oct 26, 2015 at 5:06 AM, Oleg Zhurakousky mailto:ozhurakou...@hortonworks.com>> wrote: nifi-ide-integration Oleg, thanks for the nifi-ide-integration github. I was able to get up and running in the debugger in Eclipse pretty easy with this. I did have to add this to my VM args in my run configuration and it was not documented in your github: -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties Wondering if I might have done something wrong although your github readme is pretty clear? I exactly followed those instructions after first cloning the asf repo, building Nifi, and then installing the target tar.gz. One final question, I see this on my Eclipse console: log4j:WARN No appenders could be found for logger (org.apache.nifi.NiFi). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. I see you you have a log4j.properties file in the nifi-ide-integration project's src/main/resources path and that path is in the class path for my Eclipse nifi-ide-integration project. Can you please help guide me through getting console logging working for Eclipse? I notice that nifi-ide-integration references a log4j property file whilst the installed Nifi conf dir has a logback.xml file.
Re: Recommendation on getting started as a contibutor
Also, validate that src/main/resources is one of the source directories, so log4j.properties is pulled into the startup class path. I was just able to reproduce your exact error by disassociating it from the class path. On Oct 27, 2015, at 6:51 AM, Oleg Zhurakousky mailto:ozhurakou...@hortonworks.com>> wrote: Mark Thanks for trying. Indeed I forgot to mention in the guide that one has to add ‘nifi.properties.file.path' as ‘VM Arguments’, updating the guide now. As for logging, everything should work. Even though NiFi uses logback adapter for slf4j, the IDE setup uses log4j and all required JARs should be on the class path. Could you pleas re-verify? Here is what I see in my current setup after running ./gradlew clean eclipse -> importing project into workspace and configuring ‘Run Configurations' Run Configurations: VM arguments: -Dnifi.properties.file.path=/dev/nifi/nifi-assembly/target/nifi-0.3.1-SNAPSHOT-bin/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties Working directory (“Other" is checked in): /dev/nifi/nifi-assembly/target/nifi-0.3.1-SNAPSHOT-bin/nifi-0.3.1-SNAPSHOT CLASSPATH: nifi-api-0.3.1-SNAPSHOT.jar nifi-runtime-0.3.1-SNAPSHOT.jar log4j-core-2.4.jar slf4j-api-1.7.12.jar slf4j-log4j12-1.7.12.jar jul-to-slf4j-1.7.12.jar nifi-nar-utils-0.3.1-SNAPSHOT.jar nifi-properties-0.3.1-SNAPSHOT.jar nifi-documentation-0.3.1-SNAPSHOT.jar log4j-api-2.4.jar log4j-1.2.17.jar Cheers Oleg On Oct 27, 2015, at 12:30 AM, Mark Petronic mailto:markpetro...@gmail.com>> wrote: On Mon, Oct 26, 2015 at 5:06 AM, Oleg Zhurakousky mailto:ozhurakou...@hortonworks.com>> wrote: nifi-ide-integration Oleg, thanks for the nifi-ide-integration github. I was able to get up and running in the debugger in Eclipse pretty easy with this. I did have to add this to my VM args in my run configuration and it was not documented in your github: -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties Wondering if I might have done something wrong although your github readme is pretty clear? I exactly followed those instructions after first cloning the asf repo, building Nifi, and then installing the target tar.gz. One final question, I see this on my Eclipse console: log4j:WARN No appenders could be found for logger (org.apache.nifi.NiFi). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. I see you you have a log4j.properties file in the nifi-ide-integration project's src/main/resources path and that path is in the class path for my Eclipse nifi-ide-integration project. Can you please help guide me through getting console logging working for Eclipse? I notice that nifi-ide-integration references a log4j property file whilst the installed Nifi conf dir has a logback.xml file.
[GitHub] nifi pull request: NIFI-1074 added initial support for IDE integra...
GitHub user olegz opened a pull request: https://github.com/apache/nifi/pull/108 NIFI-1074 added initial support for IDE integration Includes instructions for Eclipse You can merge this pull request into a Git repository by running: $ git pull https://github.com/olegz/nifi NIFI-1074 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/108.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #108 commit 73feb3b31c0df49d8aca941ef0fae13ef02ea378 Author: Oleg Zhurakousky Date: 2015-10-27T11:40:41Z NIFI-1074 added initial support for IDE integration Includes instructions for Eclipse --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Recommendation on getting started as a contibutor
On Tue, Oct 27, 2015 at 6:55 AM, Oleg Zhurakousky wrote: > I was just able to reproduce your exact error by disassociating it from the > class path Oleg, thanks for the response. 1. I verified that my working directory is correct and points to my running version of Nifi: /home/mpetronic/nifi-0.3.1-SNAPSHOT 2. I verified that src/main/resources is indeed the one and only entry listing in my build path settings under the "Source" tab. However, I don't see that reflected in the below command line. 3. Not sure how to dump the active classpath from Eclipse project configuration but, if I run Nifi under Eclipse and go to the properties of the running instance I see this as the command line used to run it. Question is why are all my classpaths pointing to files in the maven repository? Those are the values reflected in the project build path under the "Libraries" tab that I got by default after importing the nifi-ide-integration project. I did not edit anything there. /opt/java/jdk1.7.0_75/bin/java -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties -Dfile.encoding=ANSI_X3.4-1968 -classpath /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf org.apache.nifi.NiFi 4. I just got it to work by removing "src/main/resources" from the "Source" tab and added it to the "Libraries" tab using "Add Class Folder". Now when I run, I see console logging as expected and my new command line is below (where src/main/resources is now showing up as the last entry in the path): But why did I need this extra step? /opt/java/jdk1.7.0_75/bin/java -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties -Dfile.encoding=ANSI_X3.4-1968 -classpath /home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/repos/nifi-ide-integration/src/main/resources org.apache.nifi.NiFi May I ask you a related question about environment setup for contributing given I am new to contributing and want to get this right? I read the contributors/developers guides. They talk about three repos: the ASF repo, the github mirror, and a personal forked github repo. They talk about two way to commit: patch and pull request. 1. Is there
Re: Recommendation on getting started as a contibutor
It appears you’ve misconfigured your Run Configuration. It seems like your MainClass is org.apache.nifi.bootstrap.RunNifi. It should be org.apache.nifi.NiFi Can you verify? Oleg > On Oct 27, 2015, at 8:45 AM, Mark Petronic wrote: > > On Tue, Oct 27, 2015 at 6:55 AM, Oleg Zhurakousky > wrote: >> I was just able to reproduce your exact error by disassociating it from the >> class path > > Oleg, thanks for the response. > > 1. I verified that my working directory is correct and points to my > running version of Nifi: > > /home/mpetronic/nifi-0.3.1-SNAPSHOT > > 2. I verified that src/main/resources is indeed the one and only entry > listing in my build path settings under the "Source" tab. However, I > don't see that reflected in the below command line. > > 3. Not sure how to dump the active classpath from Eclipse project > configuration but, if I run Nifi under Eclipse and go to the > properties of the running instance I see this as the command line used > to run it. Question is why are all my classpaths pointing to files in > the maven repository? Those are the values reflected in the project > build path under the "Libraries" tab that I got by default after > importing the nifi-ide-integration project. I did not edit anything > there. > > /opt/java/jdk1.7.0_75/bin/java > -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties > -Dfile.encoding=ANSI_X3.4-1968 -classpath > /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf > org.apache.nifi.NiFi > > 4. I just got it to work by removing "src/main/resources" from the > "Source" tab and added it to the "Libraries" tab using "Add Class > Folder". Now when I run, I see console logging as expected and my new > command line is below (where src/main/resources is now showing up as > the last entry in the path): But why did I need this extra step? > > /opt/java/jdk1.7.0_75/bin/java > -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties > -Dfile.encoding=ANSI_X3.4-1968 -classpath > /home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/repos/nifi-ide-integration/src/main/resources > org.apache.nifi.NiFi > > May I ask you a relat
Re: Recommendation on getting started as a contibutor
Main Class is definitely configured to org.apache.nifi.NiFi per instructions. That's what I see in the sample command lines I sent you, too. Curious, what makes you think that was misconfigured? On Tue, Oct 27, 2015 at 8:52 AM, Oleg Zhurakousky wrote: > It appears you’ve misconfigured your Run Configuration. > It seems like your MainClass is org.apache.nifi.bootstrap.RunNifi. It should > be org.apache.nifi.NiFi > > Can you verify? > > Oleg > >> On Oct 27, 2015, at 8:45 AM, Mark Petronic wrote: >> >> On Tue, Oct 27, 2015 at 6:55 AM, Oleg Zhurakousky >> wrote: >>> I was just able to reproduce your exact error by disassociating it from the >>> class path >> >> Oleg, thanks for the response. >> >> 1. I verified that my working directory is correct and points to my >> running version of Nifi: >> >> /home/mpetronic/nifi-0.3.1-SNAPSHOT >> >> 2. I verified that src/main/resources is indeed the one and only entry >> listing in my build path settings under the "Source" tab. However, I >> don't see that reflected in the below command line. >> >> 3. Not sure how to dump the active classpath from Eclipse project >> configuration but, if I run Nifi under Eclipse and go to the >> properties of the running instance I see this as the command line used >> to run it. Question is why are all my classpaths pointing to files in >> the maven repository? Those are the values reflected in the project >> build path under the "Libraries" tab that I got by default after >> importing the nifi-ide-integration project. I did not edit anything >> there. >> >> /opt/java/jdk1.7.0_75/bin/java >> -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties >> -Dfile.encoding=ANSI_X3.4-1968 -classpath >> /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf >> org.apache.nifi.NiFi >> >> 4. I just got it to work by removing "src/main/resources" from the >> "Source" tab and added it to the "Libraries" tab using "Add Class >> Folder". Now when I run, I see console logging as expected and my new >> command line is below (where src/main/resources is now showing up as >> the last entry in the path): But why did I need this extra step? >> >> /opt/java/jdk1.7.0_75/bin/java >> -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties >> -Dfile.encoding=ANSI_X3.4-1968 -classpath >> /home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2
Re: Recommendation on getting started as a contibutor
Mark The following output comes from RunNifi which starts org.apache.nifi.NiFi as a separate JVM process which means you are not really in full DEBUG mode anyway: opt/java/jdk1.7.0_75/bin/java -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties -Dfile.encoding=ANSI_X3.4-1968 -classpath /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf org.apache.nifi.NiFi If you are in Eclipse, did you go through Run Configuration step where you configure your man class that should be org.apache.nifi.NiFi? Oleg > On Oct 27, 2015, at 8:55 AM, Mark Petronic wrote: > > Main Class is definitely configured to org.apache.nifi.NiFi per > instructions. That's what I see in the sample command lines I sent > you, too. Curious, what makes you think that was misconfigured? > > On Tue, Oct 27, 2015 at 8:52 AM, Oleg Zhurakousky > wrote: >> It appears you’ve misconfigured your Run Configuration. >> It seems like your MainClass is org.apache.nifi.bootstrap.RunNifi. It should >> be org.apache.nifi.NiFi >> >> Can you verify? >> >> Oleg >> >>> On Oct 27, 2015, at 8:45 AM, Mark Petronic wrote: >>> >>> On Tue, Oct 27, 2015 at 6:55 AM, Oleg Zhurakousky >>> wrote: I was just able to reproduce your exact error by disassociating it from the class path >>> >>> Oleg, thanks for the response. >>> >>> 1. I verified that my working directory is correct and points to my >>> running version of Nifi: >>> >>> /home/mpetronic/nifi-0.3.1-SNAPSHOT >>> >>> 2. I verified that src/main/resources is indeed the one and only entry >>> listing in my build path settings under the "Source" tab. However, I >>> don't see that reflected in the below command line. >>> >>> 3. Not sure how to dump the active classpath from Eclipse project >>> configuration but, if I run Nifi under Eclipse and go to the >>> properties of the running instance I see this as the command line used >>> to run it. Question is why are all my classpaths pointing to files in >>> the maven repository? Those are the values reflected in the project >>> build path under the "Libraries" tab that I got by default after >>> importing the nifi-ide-integration project. I did not edit anything >>> there. >>> >>> /opt/java/jdk1.7.0_75/bin/java >>> -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties >>> -Dfile.encoding=ANSI_X3.4-1968 -classpath >>> /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.
Re: Recommendation on getting started as a contibutor
Mark, I don't think there is a strong preference for patches vs. pull requests. A lot of contributors use pull requests, and I personally find it easier to review pull requests because you can give feedback in-line on the code. The more important thing is that whatever is being submitted should always tie back to a JIRA, and commit messages should start with the JIRA name. -Bryan On Tue, Oct 27, 2015 at 6:02 AM, Oleg Zhurakousky < ozhurakou...@hortonworks.com> wrote: > Mark > > The following output comes from RunNifi which starts org.apache.nifi.NiFi > as a separate JVM process which means you are not really in full DEBUG mode > anyway: > > opt/java/jdk1.7.0_75/bin/java > > -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties > -Dfile.encoding=ANSI_X3.4-1968 -classpath > > /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf > org.apache.nifi.NiFi > > If you are in Eclipse, did you go through Run Configuration step where you > configure your man class that should be org.apache.nifi.NiFi? > > Oleg > > > On Oct 27, 2015, at 8:55 AM, Mark Petronic > wrote: > > > > Main Class is definitely configured to org.apache.nifi.NiFi per > > instructions. That's what I see in the sample command lines I sent > > you, too. Curious, what makes you think that was misconfigured? > > > > On Tue, Oct 27, 2015 at 8:52 AM, Oleg Zhurakousky > > wrote: > >> It appears you’ve misconfigured your Run Configuration. > >> It seems like your MainClass is org.apache.nifi.bootstrap.RunNifi. It > should be org.apache.nifi.NiFi > >> > >> Can you verify? > >> > >> Oleg > >> > >>> On Oct 27, 2015, at 8:45 AM, Mark Petronic > wrote: > >>> > >>> On Tue, Oct 27, 2015 at 6:55 AM, Oleg Zhurakousky > >>> wrote: > I was just able to reproduce your exact error by disassociating it > from the class path > >>> > >>> Oleg, thanks for the response. > >>> > >>> 1. I verified that my working directory is correct and points to my > >>> running version of Nifi: > >>> > >>> /home/mpetronic/nifi-0.3.1-SNAPSHOT > >>> > >>> 2. I verified that src/main/resources is indeed the one and only entry > >>> listing in my build path settings under the "Source" tab. However, I > >>> don't see that reflected in the below command line. > >>> > >>> 3. Not sure how to dump the active classpath from Eclipse project > >>> configuration but, if I run Nifi under Eclipse and go to the > >>> properties of the running instance I see this as the command line used > >>> to run it. Question is why are all my classpaths pointing to files in > >>> the maven repository? Those are the values reflected in the project > >>> build path under the "Libraries" tab that I got by default after > >>> importing the nifi-ide-integration project. I did not edit anything > >>> there. > >>> > >>> /opt/java/jdk1.7.0_75/bin/java > >>> > -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties > >>> -Dfile.encoding=ANSI_X3.4-1968 -classpath > >>> > /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/mod
Re: Recommendation on getting started as a contibutor
Oleg, yes, absolutely configured as a Run/Debug Configuration. I can open my debug run configuration and click "Debug" and it starts up Nifi. I've set breakpoints in a the ExecuteProcess code then created a processor from the GUI and started it and hit the break point. So, that all seems to be working. On Tue, Oct 27, 2015 at 9:02 AM, Oleg Zhurakousky wrote: > Mark > > The following output comes from RunNifi which starts org.apache.nifi.NiFi as > a separate JVM process which means you are not really in full DEBUG mode > anyway: > > opt/java/jdk1.7.0_75/bin/java > -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties > -Dfile.encoding=ANSI_X3.4-1968 -classpath > /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf > org.apache.nifi.NiFi > > If you are in Eclipse, did you go through Run Configuration step where you > configure your man class that should be org.apache.nifi.NiFi? > > Oleg > >> On Oct 27, 2015, at 8:55 AM, Mark Petronic wrote: >> >> Main Class is definitely configured to org.apache.nifi.NiFi per >> instructions. That's what I see in the sample command lines I sent >> you, too. Curious, what makes you think that was misconfigured? >> >> On Tue, Oct 27, 2015 at 8:52 AM, Oleg Zhurakousky >> wrote: >>> It appears you’ve misconfigured your Run Configuration. >>> It seems like your MainClass is org.apache.nifi.bootstrap.RunNifi. It >>> should be org.apache.nifi.NiFi >>> >>> Can you verify? >>> >>> Oleg >>> On Oct 27, 2015, at 8:45 AM, Mark Petronic wrote: On Tue, Oct 27, 2015 at 6:55 AM, Oleg Zhurakousky wrote: > I was just able to reproduce your exact error by disassociating it from > the class path Oleg, thanks for the response. 1. I verified that my working directory is correct and points to my running version of Nifi: /home/mpetronic/nifi-0.3.1-SNAPSHOT 2. I verified that src/main/resources is indeed the one and only entry listing in my build path settings under the "Source" tab. However, I don't see that reflected in the below command line. 3. Not sure how to dump the active classpath from Eclipse project configuration but, if I run Nifi under Eclipse and go to the properties of the running instance I see this as the command line used to run it. Question is why are all my classpaths pointing to files in the maven repository? Those are the values reflected in the project build path under the "Libraries" tab that I got by default after importing the nifi-ide-integration project. I did not edit anything there. /opt/java/jdk1.7.0_75/bin/java -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties -Dfile.encoding=ANSI_X3.4-1968 -classpath /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files
Re: Recommendation on getting started as a contibutor
Now I am officially puzzled, since you are showing me the output from the class that is not supposed to run. Basically when you start NiFi via ./run-nifi.sh it executes org.apache.nifi.bootstrap.RunNiFi which sets the required configuration, class path etc., and starts org.apache.nifi.NiFi as a separate JVM process. That is what you see in your log which is being printed by RunNifi - 'opt/java/jdk1.7.0_75/bin/java -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties -Dfile.encoding=ANSI_X3.4-1968 -classpath . . . org.apache.nifi.NiFi’ With these instructions we are doing the complete opposite and bypassing RunNifi executing org.apache.nifi.NiFi directly, so I can’t see how is it even possible for you to see that log statement in your Eclipse console. Could you please post a screen shots with Main and Arguments tab of your Run Configuration? Oleg > On Oct 27, 2015, at 9:11 AM, Mark Petronic wrote: > > Oleg, yes, absolutely configured as a Run/Debug Configuration. I can > open my debug run configuration and click "Debug" and it starts up > Nifi. I've set breakpoints in a the ExecuteProcess code then created a > processor from the GUI and started it and hit the break point. So, > that all seems to be working. > > On Tue, Oct 27, 2015 at 9:02 AM, Oleg Zhurakousky > wrote: >> Mark >> >> The following output comes from RunNifi which starts org.apache.nifi.NiFi as >> a separate JVM process which means you are not really in full DEBUG mode >> anyway: >> >> opt/java/jdk1.7.0_75/bin/java >> -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties >> -Dfile.encoding=ANSI_X3.4-1968 -classpath >> /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf >> org.apache.nifi.NiFi >> >> If you are in Eclipse, did you go through Run Configuration step where you >> configure your man class that should be org.apache.nifi.NiFi? >> >> Oleg >> >>> On Oct 27, 2015, at 8:55 AM, Mark Petronic wrote: >>> >>> Main Class is definitely configured to org.apache.nifi.NiFi per >>> instructions. That's what I see in the sample command lines I sent >>> you, too. Curious, what makes you think that was misconfigured? >>> >>> On Tue, Oct 27, 2015 at 8:52 AM, Oleg Zhurakousky >>> wrote: It appears you’ve misconfigured your Run Configuration. It seems like your MainClass is org.apache.nifi.bootstrap.RunNifi. It should be org.apache.nifi.NiFi Can you verify? Oleg > On Oct 27, 2015, at 8:45 AM, Mark Petronic wrote: > > On Tue, Oct 27, 2015 at 6:55 AM, Oleg Zhurakousky > wrote: >> I was just able to reproduce your exact error by disassociating it from >> the class path > > Oleg, thanks for the response. > > 1. I verified that my working directory is correct and points to my > running version of Nifi: > > /home/mpetronic/nifi-0.3.1-SNAPSHOT > > 2. I verified that src/main/resources is indeed the one and only entry > listing in my build path settings under the "Source" tab. However, I > don't see that reflected in the below command line. > > 3. Not sure how to dump the active classpath from Eclipse project > configuration but, if I run Nifi under Eclipse and go to the > properties of the running instance I see this as the command line used > to run it. Question is why are all my classpaths pointing to files in > the maven repository? Those are the values reflected in the project > build path under
Re: whether nifi can pull data from realtional database like oracle into hdfs
Hey Cathy, There are many processors that cover pushing/pulling from databases as well as doing some transformations. You can check out the list here: https://nifi.apache.org/docs.html Let us know if you need more help, Joe - - - - - - Joseph Percivall linkedin.com/in/Percivall e: joeperciv...@yahoo.com On Tuesday, October 27, 2015 6:31 AM, cathy wrote: Dear sir, I wanna know whether i can pull data from realtional database like oracle into hdfs ,or pull data from realtional database like oracle into mysql by using nifi? Should i need to develop plugins or something? Looking forward to your early reply,Thank you very much . Sincerely yours.
Re: Recommendation on getting started as a contibutor
On Tue, Oct 27, 2015 at 9:27 AM, Oleg Zhurakousky wrote: > Now I am officially puzzled, since you are showing me the output from the > class that is not supposed to run. > > Basically when you start NiFi via ./run-nifi.sh it executes > org.apache.nifi.bootstrap.RunNiFi which sets the required configuration, > class path etc., and starts org.apache.nifi.NiFi as a separate JVM process. > That is what you see in your log which is being printed by RunNifi - > 'opt/java/jdk1.7.0_75/bin/java > -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties > -Dfile.encoding=ANSI_X3.4-1968 -classpath . . . org.apache.nifi.NiFi’ > > With these instructions we are doing the complete opposite and bypassing > RunNifi executing org.apache.nifi.NiFi directly, so I can’t see how is it > even possible for you to see that log statement in your Eclipse console. Ah, let me demystify you... That output is NOT from the logging console. I just copied that from the the process information by right-clicking on the running process in the "Debug" view in Eclipse. However, I was doing that from home. I'm at work now and I just repeated the whole install process in my dev setup here, first forked the Nifi github and did the clean install. Then cloned nifi-ide-integration and followed the steps and it just worked!! No idea what is happening at home. But, I think I am good to go. One more tweak to your README. When you select the external directory for your sources, there is a checkbox (at least in Eclipse Luna) to "Search subfolders". That needs to be selected before I can see the source files in the debugger. Maybe obvious but I used to be a tech writer so I cannot help myself. LOL > > Could you please post a screen shots with Main and Arguments tab of your Run > Configuration? > > Oleg > >> On Oct 27, 2015, at 9:11 AM, Mark Petronic wrote: >> >> Oleg, yes, absolutely configured as a Run/Debug Configuration. I can >> open my debug run configuration and click "Debug" and it starts up >> Nifi. I've set breakpoints in a the ExecuteProcess code then created a >> processor from the GUI and started it and hit the break point. So, >> that all seems to be working. >> >> On Tue, Oct 27, 2015 at 9:02 AM, Oleg Zhurakousky >> wrote: >>> Mark >>> >>> The following output comes from RunNifi which starts org.apache.nifi.NiFi >>> as a separate JVM process which means you are not really in full DEBUG mode >>> anyway: >>> >>> opt/java/jdk1.7.0_75/bin/java >>> -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties >>> -Dfile.encoding=ANSI_X3.4-1968 -classpath >>> /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf >>> org.apache.nifi.NiFi >>> >>> If you are in Eclipse, did you go through Run Configuration step where you >>> configure your man class that should be org.apache.nifi.NiFi? >>> >>> Oleg >>> On Oct 27, 2015, at 8:55 AM, Mark Petronic wrote: Main Class is definitely configured to org.apache.nifi.NiFi per instructions. That's what I see in the sample command lines I sent you, too. Curious, what makes you think that was misconfigured? On Tue, Oct 27, 2015 at 8:52 AM, Oleg Zhurakousky wrote: > It appears you’ve misconfigured your Run Configuration. > It seems like your MainClass is org.apache.nifi.bootstrap.RunNifi. It > should be org.apache.nifi.NiFi > > Can you verify? > > Oleg > >> On Oct 27, 2015, at 8:45 AM, Mark Petronic >> wrote: >> >> On Tue, Oct 27, 2015 a
Re: Recommendation on getting started as a contibutor
Thanks Mark, good point about check box, I'll update. Oleg Sent from my iPhone > On Oct 27, 2015, at 17:19, Mark Petronic wrote: > > On Tue, Oct 27, 2015 at 9:27 AM, Oleg Zhurakousky > wrote: >> Now I am officially puzzled, since you are showing me the output from the >> class that is not supposed to run. >> >> Basically when you start NiFi via ./run-nifi.sh it executes >> org.apache.nifi.bootstrap.RunNiFi which sets the required configuration, >> class path etc., and starts org.apache.nifi.NiFi as a separate JVM process. >> That is what you see in your log which is being printed by RunNifi - >> 'opt/java/jdk1.7.0_75/bin/java >> -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties >> -Dfile.encoding=ANSI_X3.4-1968 -classpath . . . org.apache.nifi.NiFi’ >> >> With these instructions we are doing the complete opposite and bypassing >> RunNifi executing org.apache.nifi.NiFi directly, so I can’t see how is it >> even possible for you to see that log statement in your Eclipse console. > > Ah, let me demystify you... That output is NOT from the logging > console. I just copied that from the the process information by > right-clicking on the running process in the "Debug" view in Eclipse. > However, I was doing that from home. I'm at work now and I just > repeated the whole install process in my dev setup here, first forked > the Nifi github and did the clean install. Then cloned > nifi-ide-integration and followed the steps and it just worked!! No > idea what is happening at home. But, I think I am good to go. > > One more tweak to your README. When you select the external directory > for your sources, there is a checkbox (at least in Eclipse Luna) to > "Search subfolders". That needs to be selected before I can see the > source files in the debugger. Maybe obvious but I used to be a tech > writer so I cannot help myself. LOL > >> >> Could you please post a screen shots with Main and Arguments tab of your Run >> Configuration? >> >> Oleg >> >>> On Oct 27, 2015, at 9:11 AM, Mark Petronic wrote: >>> >>> Oleg, yes, absolutely configured as a Run/Debug Configuration. I can >>> open my debug run configuration and click "Debug" and it starts up >>> Nifi. I've set breakpoints in a the ExecuteProcess code then created a >>> processor from the GUI and started it and hit the break point. So, >>> that all seems to be working. >>> >>> On Tue, Oct 27, 2015 at 9:02 AM, Oleg Zhurakousky >>> wrote: Mark The following output comes from RunNifi which starts org.apache.nifi.NiFi as a separate JVM process which means you are not really in full DEBUG mode anyway: opt/java/jdk1.7.0_75/bin/java -Dnifi.properties.file.path=/home/mpetronic/nifi-0.3.1-SNAPSHOT/./conf/nifi.properties -Dfile.encoding=ANSI_X3.4-1968 -classpath /home/mpetronic/repos/nifi-ide-integration/bin:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-api/0.3.1-SNAPSHOT/nifi-api-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-runtime/0.3.1-SNAPSHOT/nifi-runtime-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.4/d99532ba3603f27bebf4cdd3653feb0e0b84cf6/log4j-core-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.slf4j/jul-to-slf4j/1.7.12/8811e2e9ab9055e557598dc9aedc64fd43e0ab20/jul-to-slf4j-1.7.12.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-nar-utils/0.3.1-SNAPSHOT/nifi-nar-utils-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-properties/0.3.1-SNAPSHOT/nifi-properties-0.3.1-SNAPSHOT.jar:/home/mpetronic/.m2/repository/org/apache/nifi/nifi-documentation/0.3.1-SNAPSHOT/nifi-documentation-0.3.1-SNAPSHOT.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.4/cc68e72d6d14098ba044123e10e048d203d3fd47/log4j-api-2.4.jar:/home/mpetronic/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:/home/mpetronic/nifi-0.3.1-SNAPSHOT/conf org.apache.nifi.NiFi If you are in Eclipse, did you go through Run Configuration step where you configure your man class that should be org.apache.nifi.NiFi? Oleg > On Oct 27, 2015, at 8:55 AM, Mark Petronic wrote: > > Main Class is definitely configured to org.apache.nifi.NiFi per > instructions. That's what I see in the sample command lines I sent > you, too. Curious, what makes you think that was misconfigured? > > On Tue, Oct 27, 2015 at 8:52 AM, Oleg Zhurakousky > wrote: >> It appears you’ve misconfigured your Run Configuration. >> It seems like your
