[ https://issues.apache.org/jira/browse/BEAM-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126174#comment-17126174 ]
Harrison commented on BEAM-10192: --------------------------------- This may be related to BEAM-2988 > Duplicate PubSub subscriptions with python direct runner in Jupyter/Colab > environment > ------------------------------------------------------------------------------------- > > Key: BEAM-10192 > URL: https://issues.apache.org/jira/browse/BEAM-10192 > Project: Beam > Issue Type: Bug > Components: io-py-gcp > Affects Versions: 2.22.0 > Environment: Ubuntu 18 (Colab notebook), Python SDK > Reporter: Harrison > Priority: P2 > > When running a streaming pipeline on Colab with direct runner, ReadFromPubSub > can retain old subscriptions and cause message duplication. For example, > manually killing a cell that is running a streaming pubsub pipeline does not > delete the pubsub subscription. If the cell is rerun, the ReadFromPubSub > component will actually be subscribed twice which results in duplicate > messages. > Manually deleting old subscriptions (e.g. via the GCP dashboard) temporarily > fixes the problem. > This Colab notebook: > [https://gist.github.com/hgarrereyn/64ce87cbbcbe9c34ccdd13eafe49e3fb] > contains a runnable example of the bug. -- This message was sent by Atlassian Jira (v8.3.4#803005)