phoerious opened a new issue, #24357: URL: https://github.com/apache/beam/issues/24357
### What happened? Disclaimer: This bug reports refers to the current master branch, which contains major changes to the container boot.go, but it's very likely that it applies to previous versions as well, since the parts which most likely trigger the bug were not changed. The Python SDK boot.go binary installs dependencies from a setup.py automatically here: https://github.com/apache/beam/blob/master/sdks/python/container/boot.go#L368 Most of the time, it works fine, but sometimes `pip` process seems to hang for no reason. The problem can be resolved by attaching to the container running the SDK and reading STDIN of the `boot` process: ```kubectl exec -ti pod/beam-flinker-cluster-taskmanager -c beam-worker-pool -- bash ps aux | grep boot cat /proc/<WORKER-PID>/fd/1 exit ``` I suppose this issue occurs due to a stdout buffer fillup. The boot binary uses the following utility for running `pip`, which redirects the processe's STDOUT to `os.stdout`. A solution would perhaps be to either not redirect stdout or to write it directly to `/dev/null`. https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/util/execx/exec.go#L26 ### Issue Priority Priority: 2 ### Issue Component Component: sdk-py-harness -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
