Do you want the job to fail if there is an error exit code? You could set checkCode to True spark.apache.org/docs/latest/api/python/pyspark.html?highlight=pipe#pyspark.RDD.pipe<http://spark.apache.org/docs/latest/api/python/pyspark.html?highlight=pipe#pyspark.RDD.pipe>
Otherwise maybe you want to output the status into stdout so you could process it individually. _____________________________ From: Xuchen Yao <[email protected]<mailto:[email protected]>> Sent: Friday, February 10, 2017 11:18 AM Subject: Getting exit code of pipe() To: <[email protected]<mailto:[email protected]>> Hello Community, I have the following Python code that calls an external command: rdd.pipe('run.sh', env=os.environ).collect() run.sh can either exit with status 1 or 0, how could I get the exit code from Python? Thanks! Xuchen
