[ https://issues.apache.org/jira/browse/BEAM-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948096#comment-15948096 ]
ASF GitHub Bot commented on BEAM-1835: -------------------------------------- GitHub user ravwojdyla opened a pull request: https://github.com/apache/beam/pull/2365 [BEAM-1835] NPE pubsub Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-<Jira issue #>] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravwojdyla/incubator-beam npe_pubsub Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2365.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2365 ---- commit d3e3aeb117ec1fb8ed0c8b834d32da659c83d099 Author: Rafal Wojdyla <r...@spotify.com> Date: 2017-03-27T18:40:10Z Fix typo commit 1bab5cce57b56727495355c4d9a2ab1e1f122fbf Author: Rafal Wojdyla <r...@spotify.com> Date: 2017-03-29T23:11:19Z Never reuse reader for direct pipeline runner ---- > NPE in DirectRunner PubsubReader.ackBatch > ----------------------------------------- > > Key: BEAM-1835 > URL: https://issues.apache.org/jira/browse/BEAM-1835 > Project: Beam > Issue Type: Bug > Components: runner-direct, sdk-java-core > Reporter: Rafal Wojdyla > Assignee: Rafal Wojdyla > > DirectRunner streaming mode throws null pointer exception: > {noformat} > org.apache.beam.sdk.io.PubsubUnboundedSource$PubsubReader.ackBatch(PubsubUnboundedSource.java:639) > at > org.apache.beam.sdk.io.PubsubUnboundedSource$PubsubCheckpoint.finalizeCheckpoint(PubsubUnboundedSource.java:312) > at > org.apache.beam.runners.direct.UnboundedReadEvaluatorFactory$UnboundedReadEvaluator.finishRead(UnboundedReadEvaluatorFactory.java:223) > at > org.apache.beam.runners.direct.UnboundedReadEvaluatorFactory$UnboundedReadEvaluator.processElement(UnboundedReadEvaluatorFactory.java:144) > at > org.apache.beam.runners.direct.TransformExecutor.processElements(TransformExecutor.java:139) > at > org.apache.beam.runners.direct.TransformExecutor.run(TransformExecutor.java:107) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > This does not happen always, but for large enough number of events it's > pretty reproducible. The problems seems to be the concurrent reuse of a > reader among multiple threads, and a race condition, when one of the threads > "decided" to close the reader, based on: > {code} > private static final double DEFAULT_READER_REUSE_CHANCE = 0.95; > {code} > the close, nulls out pubsub client: > {code} > @Override > public void close() throws IOException { > if (pubsubClient != null) { > pubsubClient.close(); > pubsubClient = null; > } > } > {code} > which if still in use by other thread will result in NPE above. -- This message was sent by Atlassian JIRA (v6.3.15#6346)