Location of logs in local mode
I¹m submitting a script using spark-submit in local mode for testing, and I¹m having trouble figuring out where the logs are stored. The documentation indicates that they should be in the work folder in the directory in which Spark lives on my system, but I see no such folder there. I¹ve set the SPARK_LOCAL_DIRS and SPARK_LOG_DIR environment variables in spark-env.sh, but there doesn¹t seem to be any log output generated in the locations I¹ve specified there either. I¹m just using spark-submit with master local, I haven¹t run any of the standalone cluster scripts, so I¹m not sure if there¹s something I¹m missing here as far as a default output location for logging. Thanks, Brett smime.p7s Description: S/MIME cryptographic signature
Location of logs in local mode
I¹m submitting a script using spark-submit in local mode for testing, and I¹m having trouble figuring out where the logs are stored. The documentation indicates that they should be in the work folder in the directory in which Spark lives on my system, but I see no such folder there. I¹ve set the SPARK_LOCAL_DIRS and SPARK_LOG_DIR environment variables in spark-env.sh, but there doesn¹t seem to be any log output generated in the locations I¹ve specified there either. I¹m just using spark-submit with master local, I haven¹t run any of the standalone cluster scripts, so I¹m not sure if there¹s something I¹m missing here as far as a default output location for logging. Thanks, Brett smime.p7s Description: S/MIME cryptographic signature
