Hi Joe, I am not using anything from Nifi 1.0 and its all 0.7. What indicates the use of 1.0?
Regards, Manish On Thu, Dec 1, 2016 at 12:07 PM, Joe Witt <[email protected]> wrote: > Manish > > It also appears from the log output you provided so far that this is a > combination of nifi 0.7 and 1.0 parts. We do not recommend doing this > and it is not likely to work. > > Please try the flow from a single release line. Apache NiFi 1.1.0 is > available for use now. > > Thanks > Joe > > On Thu, Dec 1, 2016 at 9:02 AM, Manish G <[email protected]> wrote: > > Got it. I was not aware of "bin/nifi.sh dump". I will try that once I see > > the issue again. > > > > Thanks, > > Manish > > > > On Thu, Dec 1, 2016 at 11:59 AM, Joe Witt <[email protected]> wrote: > >> > >> Manish > >> > >> When it is in the stuck state can you please run > >> bin/nifi.sh dump. If you can then share the nifi-bootstrap.log that > >> would aid us in narrowing in on a possible cause. > >> > >> Thanks > >> Joe > >> > >> On Thu, Dec 1, 2016 at 8:44 AM, Manish G <[email protected]> > wrote: > >> > Hi Joe, > >> > > >> > Here is what I can see in the App Log: > >> > > >> > 2016-12-01 09:28:52,004 ERROR [Timer-Driven Process Thread-4] > >> > testing.nifi.processor.hdfs.testingPutHDFS "" > >> > org.apache.hadoop.fs.azure.AzureException: > >> > java.util.NoSuchElementException: > >> > An error occurred while enumerating the result, check the original > >> > exception > >> > for details. > >> > at > >> > > >> > org.apache.hadoop.fs.azure.AzureNativeFileSystemStore. > retrieveMetadata(AzureNativeFileSystemStore.java:1930) > >> > ~[hadoop-azure-2.7.2.jar:na] > >> > at > >> > > >> > org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatus( > NativeAzureFileSystem.java:1592) > >> > ~[hadoop-azure-2.7.2.jar:na] > >> > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) > >> > ~[hadoop-common-2.7.2.jar:na] > >> > at > >> > > >> > testing.nifi.processor.hdfs.testingPutHDFS.onTrigger( > testingPutHDFS.java:260) > >> > ~[testing.nifi.processor-1.0.0.nar-unpacked/:na] > >> > at > >> > > >> > org.apache.nifi.processor.AbstractProcessor.onTrigger( > AbstractProcessor.java:27) > >> > [nifi-api-0.7.0.jar:0.7.0] > >> > at > >> > > >> > org.apache.nifi.controller.StandardProcessorNode.onTrigger( > StandardProcessorNode.java:1054) > >> > [nifi-framework-core-0.7.0.jar:0.7.0] > >> > at > >> > > >> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call( > ContinuallyRunProcessorTask.java:136) > >> > [nifi-framework-core-0.7.0.jar:0.7.0] > >> > at > >> > > >> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call( > ContinuallyRunProcessorTask.java:47) > >> > [nifi-framework-core-0.7.0.jar:0.7.0] > >> > at > >> > > >> > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1. > run(TimerDrivenSchedulingAgent.java:127) > >> > [nifi-framework-core-0.7.0.jar:0.7.0] > >> > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown > Source) > >> > [na:1.8.0_101] > >> > at java.util.concurrent.FutureTask.runAndReset(Unknown Source) > >> > [na:1.8.0_101] > >> > at > >> > > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.access$301(Unknown > >> > Source) [na:1.8.0_101] > >> > at > >> > > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.run(Unknown > >> > Source) [na:1.8.0_101] > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > >> > [na:1.8.0_101] > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > >> > [na:1.8.0_101] > >> > at java.lang.Thread.run(Unknown Source) [na:1.8.0_101] > >> > Caused by: java.util.NoSuchElementException: An error occurred while > >> > enumerating the result, check the original exception for details. > >> > at > >> > > >> > com.microsoft.azure.storage.core.LazySegmentedIterator. > hasNext(LazySegmentedIterator.java:113) > >> > ~[azure-storage-2.0.0.jar:na] > >> > at > >> > > >> > org.apache.hadoop.fs.azure.StorageInterfaceImpl$ > WrappingIterator.hasNext(StorageInterfaceImpl.java:128) > >> > ~[hadoop-azure-2.7.2.jar:na] > >> > at > >> > > >> > org.apache.hadoop.fs.azure.AzureNativeFileSystemStore. > retrieveMetadata(AzureNativeFileSystemStore.java:1909) > >> > ~[hadoop-azure-2.7.2.jar:na] > >> > ... 15 common frames omitted > >> > Caused by: com.microsoft.azure.storage.StorageException: Forbidden > >> > at > >> > > >> > com.microsoft.azure.storage.StorageException.translateFromHttpStatus( > StorageException.java:202) > >> > ~[azure-storage-2.0.0.jar:na] > >> > at > >> > > >> > com.microsoft.azure.storage.StorageException.translateException( > StorageException.java:172) > >> > ~[azure-storage-2.0.0.jar:na] > >> > at > >> > > >> > com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry( > ExecutionEngine.java:273) > >> > ~[azure-storage-2.0.0.jar:na] > >> > at > >> > > >> > com.microsoft.azure.storage.core.LazySegmentedIterator. > hasNext(LazySegmentedIterator.java:109) > >> > ~[azure-storage-2.0.0.jar:na] > >> > ... 17 common frames omitted > >> > Caused by: java.lang.NullPointerException: null > >> > at > >> > > >> > com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry( > ExecutionEngine.java:181) > >> > ~[azure-storage-2.0.0.jar:na] > >> > ... 18 common frames omitted > >> > > >> > Regards, > >> > Manish > >> > > >> > On Thu, Dec 1, 2016 at 10:36 AM, Joe Witt <[email protected]> wrote: > >> >> > >> >> Manish > >> >> > >> >> Please produce and share the thread dump I mentioned. > >> >> > >> >> Thanks > >> >> Joe > >> >> > >> >> On Dec 1, 2016 7:23 AM, "Manish G" <[email protected]> wrote: > >> >>> > >> >>> Hi, > >> >>> > >> >>> I don't know why, but this is happening now more frequently. Where > >> >>> should > >> >>> I look into to find the root cause? > >> >>> > >> >>> Thanks, > >> >>> Manish > >> >>> > >> >>> On Wed, Nov 30, 2016 at 9:20 PM, Manish G <[email protected]> > >> >>> wrote: > >> >>>> > >> >>>> Hi Joe, > >> >>>> > >> >>>> Thanks for the quick reply. Yes, the processor keeps running on a > >> >>>> single > >> >>>> thread (even after stopping). And the number remains there even > after > >> >>>> stopping. > >> >>>> Today, it happened on my customized putHDFS processor. Only thing > >> >>>> different in this processor is - I have added an additional > attribute > >> >>>> that > >> >>>> tells if the processor created the directory while loading the file > >> >>>> on HDFS. > >> >>>> I don't think this should be the issue though. > >> >>>> > >> >>>> Regards, > >> >>>> Manish > >> >>>> > >> >>>> > >> >>>> On Wed, Nov 30, 2016 at 7:05 PM, Joe Witt <[email protected]> > wrote: > >> >>>>> > >> >>>>> Manish > >> >>>>> > >> >>>>> When it is stuck do you see a number in the top right corner of > the > >> >>>>> processor? When you stop it does the number remain? That number > is > >> >>>>> telling you how many threads are still executing. Which processor > >> >>>>> are > >> >>>>> we talking about? When it is in the stuck state can you please > run > >> >>>>> bin/nifi.sh dump. If you can then share the nifi-bootstrap.log > that > >> >>>>> would aid us in narrowing in on a possible cause. > >> >>>>> > >> >>>>> Thanks > >> >>>>> Joe > >> >>>>> > >> >>>>> On Wed, Nov 30, 2016 at 7:02 PM, Manish G <[email protected] > > > >> >>>>> wrote: > >> >>>>> > > >> >>>>> > Hi, > >> >>>>> > > >> >>>>> > I have noticed that sometime a flow file gets stuck on a > processor > >> >>>>> > for a > >> >>>>> > very long time for no reason and then I can not even stop the > >> >>>>> > processor to > >> >>>>> > look at the flow flow file from queue. If I click on stop, then > >> >>>>> > processor > >> >>>>> > goes into a state where I cannot start/stop the processor. > >> >>>>> > > >> >>>>> > On restarting the NiFi, the file gets processed successfully and > >> >>>>> > routed to > >> >>>>> > success queue. I checked in App log, but everything seems to be > >> >>>>> > normal for > >> >>>>> > the flow file. I don't see anything mysterious in provenance too > >> >>>>> > (except > >> >>>>> > that queue time is in hours). > >> >>>>> > > >> >>>>> > Has anyone else faced a similar issue? What else should I check > to > >> >>>>> > identify > >> >>>>> > the root cause for this? > >> >>>>> > > >> >>>>> > Thanks, > >> >>>>> > Manish > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> > >> >>>> > >> >>>> With Warm Regards, > >> >>>> Manish > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> > >> >>> > >> >>> With Warm Regards, > >> >>> Manish > >> > > >> > > >> > > >> > > >> > -- > >> > > >> > > >> > With Warm Regards, > >> > Manish > > > > > > > > > > -- > > > > > > With Warm Regards, > > Manish > -- *With Warm Regards,* *Manish*
