[
https://issues.apache.org/jira/browse/BEAM-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378717#comment-15378717
]
ASF GitHub Bot commented on BEAM-391:
-------------------------------------
Github user aaltay closed the pull request at:
https://github.com/apache/incubator-beam/pull/617
> Exceptions in gcsio upload thread causes pipeline to stall
> ----------------------------------------------------------
>
> Key: BEAM-391
> URL: https://issues.apache.org/jira/browse/BEAM-391
> Project: Beam
> Issue Type: Bug
> Components: sdk-py
> Reporter: Ahmet Altay
>
> gcsio got stuck with invalid bucket name
> GcsBufferedWriter._start_upload (gcsio.py) raises an exception if the bucket
> does not exist. This causes upload thread to silenty fail. It logs exception
> to the log but this does not stop the pipeline or closes the receiving end of
> the multiprocessing.Pipe(). Later a call in to write() blocks at
> self.conn.send_bytes(). Note that send may block if the buffer is full.
> Upload thread should have a finally clause to close the socket connection. Or
> better propagating the exception to its parent. This is true for other types
> of exceptions also.
> Another small issue in the GcsBufferedWriter.close(). It does not self
> self.close to True.
> reproduction: python -m apache_beam.examples.wordcount --output
> gs://no-such-thing/
> Prints the exception but goes on forever. Ctrl + C breaks the main thread
> shows where it got stuck.
> Similarly reproducible on the service.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
