[ https://issues.apache.org/jira/browse/APEXMALHAR-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vlad Rozov updated APEXMALHAR-2557: ----------------------------------- Fix Version/s: 4.0.0 > TimeBasedDedupOperator fails with NullPointer > ----------------------------------------------- > > Key: APEXMALHAR-2557 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2557 > Project: Apache Apex Malhar > Issue Type: Bug > Affects Versions: 3.8.0 > Reporter: Vivek Bhide > Priority: Major > Fix For: 4.0.0 > > > While using the TimeBasedDedupOperator for deduping, I see that operator > keeps failing with below NullPointer exception. > I also see that operator is always high on CPU usages. Almost reaching 100%. > No matter the setting any values for vcores or container memory for operator > {noformat} > 2018-03-22 15:10:10,037 INFO stram.FSRecoveryHandler > (FSRecoveryHandler.java:rotateLog(103)) - Creating > hdfs://littleredns/user/SVDATHDP/datatorrent/apps/application_1519410901484_187748/recovery/log > > 2018-03-22 15:10:10,056 INFO stram.StreamingContainerParent > (StreamingContainerParent.java:log(170)) - child msg: Stopped running due to > an exception. java.lang.NullPointerException > at > org.apache.hadoop.io.file.tfile.TFile$Writer.append(TFile.java:387) > at > com.datatorrent.lib.fileaccess.TFileWriter.append(TFileWriter.java:66) > at > org.apache.apex.malhar.lib.state.managed.BucketsFileSystem.writeBucketData(BucketsFileSystem.java:179) > > at > org.apache.apex.malhar.lib.state.managed.IncrementalCheckpointManager.transferWindowFiles(IncrementalCheckpointManager.java:139) > > at > org.apache.apex.malhar.lib.state.managed.IncrementalCheckpointManager$1.run(IncrementalCheckpointManager.java:110) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > context: > PTContainer[id=3(container_e3125_1519410901484_187748_01_000012),state=ACTIVE,operators=[PTOperator[id=3,name=dedupeOperator,state=ACTIVE]]] > > 2018-03-22 15:10:10,915 WARN stram.StreamingContainerManager > (StreamingContainerManager.java:processOperatorFailure(1439)) - Operator > failure: PTOperator[id=3,name=dedupeOperator,state=INACTIVE] count: 1 > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)