Pat, I was able to recreate the first error you mentioned in a similar Ubuntu environment as the one you presented. I have created an issue [1] to work on it and will evaluate from there once that can be resolved.
[1] https://issues.apache.org/jira/browse/NIFI-705 On Fri, Jun 19, 2015 at 12:54 PM, plj <[email protected]> wrote: > Howdy, > > I am using a different VM than I started with because I > didn't have sudo permission on the other one. Below are the memory and > Ubuntu release info. > $ free -m > total used free shared buffers > cached > Mem: 7985 4467 3518 1 415 3351 > -/+ buffers/cache: 700 7285 > Swap: 4094 0 4094 > > > $ lsb_release -a > No LSB modules are available. > Distributor ID: Ubuntu > Description: Ubuntu 14.04.2 LTS > Release: 14.04 > Codename: trusty > > I started using this VM when you suggested I follow the sys admin guide's > settings. > > Thank you, > > Pat > From: Aldrin Piri [via Apache NiFi (incubating) Developer List] [mailto: > [email protected]] > Sent: Friday, June 19, 2015 11:02 AM > To: Jones, Patrick L. > Subject: Re: mvn fail > > Pat, > > How much memory does the system have? Certainly perplexing that there are > so many issues. > > I remember when you first wrote to the list you were using 12.04 LTS. I > saw in a subsequent message that your lsb_release shows 14.04. Is the same > "machine" (real or virtual) that just had updates/upgrades applied? Or is > this a different instance? I would like to try to recreate the environment > with what you have shown. > > Thanks! > > On Fri, Jun 19, 2015 at 8:29 AM, plj <[hidden > email]</user/SendEmail.jtp?type=node&node=1972&i=0>> wrote: > > > Howdy, > > > > As suggested I ran "mvn clean install" and in another > > session ran: > > ps -hax -o user | grep whateverusernameyoubuildas | uniq -c > > > > I ran it lots of times. It gave answers like > > $ ps -hax -o user | grep plj | uniq -c > > 14 plj > > > > Then I ran it after I started getting the "Caused by: > > java.lang.OutOfMemoryError: unable to create new native thread > > $ ps -hax -o user | grep plj | uniq -c > > -bash: fork: retry: Resource temporarily unavailable > > -bash: fork: retry: Resource temporarily unavailable > > -bash: fork: retry: Resource temporarily unavailable > > -bash: fork: retry: Resource temporarily unavailable > > -bash: fork: Resource temporarily unavailable > > > > I get the OutOfMemoryError many times and now it hangs (it used to finish > > with errors) when it gets to: > > > > Running org.apache.nifi.processors.standard.TestListenUDP > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec - > in > > org.apache.nifi.processors.standard.TestListenUDP > > > > My env is below. > > > > Thank you very much for the help > > > > Pat > > --------------- > > $ env > > TERM=xterm > > SHELL=/bin/bash > > XDG_SESSION_COOKIE=xxx > > SSH_CLIENT=xx.xxx.xx.xx xxxxx xx > > SSH_TTY=/dev/pts/6 > > http_proxy=http://xxx.xxx.org:80/ > > USER=plj > > > > > LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36: > > ftp_proxy=ftp://xxx.xxx.org:80/ > > MAVEN_OPTS=-Xms1024m -Xmx3076m -XX:MaxPermSize=256m > > MAIL=/var/mail/plj > > > > > PATH=/usr/lib/jvm/java-7-oracle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games > > QT_QPA_PLATFORMTHEME=appmenu-qt5 > > PWD=/home/plj/incubator-nifi/nifi > > JAVA_HOME=/usr/lib/jvm/java-7-oracle > > socks_proxy=socks://xxx.xxx.org:80/ > > LANG=en_US.UTF-8 > > https_proxy=https://xxx.xxx.org:80/ > > KRB5CCNAME=FILE:/tmp/xxx > > SHLVL=1 > > HOME=/home/plj > > LOGNAME=plj > > SSH_CONNECTION=xxx.xxx.xx.xx xxxx xx.xx.xx.xx xx > > LESSOPEN=| /usr/bin/lesspipe %s > > DISPLAY=localhost:10.0 > > LESSCLOSE=/usr/bin/lesspipe %s %s > > _=/usr/bin/env > > OLDPWD=/home/plj > > > > I also changed to a more recent java (doesn't seem to matter) > > $ java -version > > java version "1.7.0_80" > > Java(TM) SE Runtime Environment (build 1.7.0_80-b15) > > Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode) > > > > From: Joe Witt [via Apache NiFi (incubating) Developer List] [mailto: > > [hidden email]</user/SendEmail.jtp?type=node&node=1972&i=1>] > > Sent: Thursday, June 18, 2015 10:22 PM > > To: Jones, Patrick L. > > Subject: Re: mvn fail > > > > Pat > > > > You can ignore that error/warning when attempting to set the network > > parameter. It is likely less relevant with newer kernels and it is > > more of an optimization than a work/not work setting. > > > > As for the actual AssertionError it is likely a side effect of the > > other error condition previously included which was > > > > "Caused by: java.lang.OutOfMemoryError: unable to create new native > thread > > at java.lang.Thread.start0(Native Method) > > at java.lang.Thread.start(Thread.java:714)" > > > > This is the error that needs the attention. Your 'ulimit -a' output > > shows a max of 10,000 user processes. This should be *way more* than > > enough so there must be more to this story. While building nifi you > > can run in another session the following > > > > ps -hax -o user | grep whateverusernameyoubuildas | uniq -c > > > > It should never get close to the max of 10000 you have indicated. How > > much memory does this system have? Have you tried using a smaller > > amount of memory for the build info with Maven? Consider perhaps > > having > > > > MAVEN_OPTS=-Xms1024m -Xmx3076m -XX:MaxPermSize=256m > > > > Also can you please paste the results of typing 'env' here as well > > (remove anything sensitive of course please). > > > > Thanks > > Joe > > > > On Thu, Jun 18, 2015 at 1:39 PM, plj <[hidden > > email]</user/SendEmail.jtp?type=node&node=1969&i=0>> wrote: > > > > > I'm still getting the errors when I run mvn clean install from the nifi > > > directory. > > > Before I said that I had done the various sys admin required. When I > > went > > > over it again I discovered that when I issued: > > > $ sudo sysctl -w > > net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1" > > > I get > > > sysctl: cannot stat > > > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_time_wait: No > such > > > file or directory > > > > > > Note there is no directory /proc/sys/net/ipv4/netfilter and no file > > > tcp_timeout_time_wait > > > > > > any thoughts? > > > > > > > > > ------ error as reported previously -------- > > > Running org.apache.nifi.processors.standard.TestHandleHttpRequest > > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 6.675 > sec > > > <<< FAILURE! - in > > org.apache.nifi.processors.standard.TestHandleHttpRequest > > > > > > testRequestAddedToService(org.apache.nifi.processors.standard.TestHandleHttpRequest) > > > Time elapsed: 6.675 sec <<< FAILURE! > > > java.lang.AssertionError: Could not invoke methods annotated with > > > @OnScheduled annotation due to: > > java.lang.reflect.InvocationTargetException > > > at org.junit.Assert.fail(Assert.java:88) > > > at > > > > > > org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:199) > > > at > > > > > > org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:182) > > > at > > > > > > org.apache.nifi.processors.standard.TestHandleHttpRequest.testRequestAddedToService(TestHandleHttpRequest.java:100) > > > > > > Running org.apache.nifi.processors.standard.TestReplaceTextWithMapping > > > Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 > > sec - > > > in org.apache.nifi.processors.standard.TestReplaceTextWithMapping > > > Running org.apache.nifi.processors.standard.TestJmsConsumer > > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 > > sec - > > > in org.apache.nifi.processors.standard.TestJmsConsumer > > > Running org.apache.nifi.processors.standard.TestSplitText > > > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - > > in > > > org.apache.nifi.processors.standard.TestSplitText > > > Running org.apache.nifi.processors.standard.TestSplitXml > > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 > > sec - > > > in org.apache.nifi.processors.standard.TestSplitXml > > > Running org.apache.nifi.processors.standard.TestBase64EncodeContent > > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 > > sec - > > > in org.apache.nifi.processors.standard.TestBase64EncodeContent > > > Running org.apache.nifi.processors.standard.TestUnpackContent > > > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 > > sec - > > > in org.apache.nifi.processors.standard.TestUnpackContent > > > Running org.apache.nifi.processors.standard.TestExtractText > > > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 > > sec - > > > in org.apache.nifi.processors.standard.TestExtractText > > > Running org.apache.nifi.processors.standard.TestConvertCharacterSet > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - > > in > > > org.apache.nifi.processors.standard.TestConvertCharacterSet > > > Running org.apache.nifi.processors.standard.TestSegmentContent > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 > > sec - > > > in org.apache.nifi.processors.standard.TestSegmentContent > > > Running org.apache.nifi.processors.standard.TestCompressContent > > > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - > > in > > > org.apache.nifi.processors.standard.TestCompressContent > > > Running org.apache.nifi.processors.standard.TestMonitorActivity > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.063 > > sec - > > > in org.apache.nifi.processors.standard.TestMonitorActivity > > > Running org.apache.nifi.processors.standard.TestExecuteStreamCommand > > > Tests run: 7, Failures: 5, Errors: 1, Skipped: 1, Time elapsed: 2.723 > sec > > > <<< FAILURE! - in > > > org.apache.nifi.processors.standard.TestExecuteStreamCommand > > > > > > testExecuteIngestAndUpdateWithWorkingDir(org.apache.nifi.processors.standard.TestExecuteStreamCommand) > > > Time elapsed: 0.115 sec <<< FAILURE! > > > java.lang.AssertionError: java.lang.OutOfMemoryError: unable to create > > new > > > native thread > > > at > > > > > > org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:219) > > > at > > > > > > org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:182) > > > at > > > > > > org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:177) > > > at > > > > > > org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteIngestAndUpdateWithWorkingDir(TestExecuteStreamCommand.java:150) > > > Caused by: java.lang.OutOfMemoryError: unable to create new native > thread > > > at java.lang.Thread.start0(Native Method) > > > at java.lang.Thread.start(Thread.java:714) > > > at > > > > > > org.apache.nifi.processors.standard.ExecuteStreamCommand$StdInWriterCallback$1.process(ExecuteStreamCommand.java:376) > > > at > > > > > > org.apache.nifi.util.MockProcessSession.write(MockProcessSession.java:593) > > > at > > > > org.apache.nifi.util.MockProcessSession.write(MockProcessSession.java:58) > > > at > > > > > > org.apache.nifi.processors.standard.ExecuteStreamCommand$StdInWriterCallback.process(ExecuteStreamCommand.java:357) > > > at > > > > org.apache.nifi.util.MockProcessSession.read(MockProcessSession.java:411) > > > at > > > > > > org.apache.nifi.processors.standard.ExecuteStreamCommand.onTrigger(ExecuteStreamCommand.java:292) > > > at > > > > > > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > > > at > > > > > > org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:269) > > > at > > > > > > org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:263) > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > > > > > > > -- > > > View this message in context: > > > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/mvn-fail-tp1716p1962.html > > > Sent from the Apache NiFi (incubating) Developer List mailing list > > archive at Nabble.com. > > > > ________________________________ > > If you reply to this email, your message will be added to the discussion > > below: > > > > > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/mvn-fail-tp1716p1969.html > > To unsubscribe from mvn fail, click here< > > >. > > NAML< > > > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml > > > > > > > > > > > > > -- > > View this message in context: > > > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/mvn-fail-tp1716p1971.html > > Sent from the Apache NiFi (incubating) Developer List mailing list > archive > > at Nabble.com. > > > > ________________________________ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/mvn-fail-tp1716p1972.html > To unsubscribe from mvn fail, click here< > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1716&code=cGxqQG1pdHJlLm9yZ3wxNzE2fDU1MTMzODA1 > >. > NAML< > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml > > > > > > > -- > View this message in context: > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/mvn-fail-tp1716p1973.html > Sent from the Apache NiFi (incubating) Developer List mailing list archive > at Nabble.com. >
