[jira] [Updated] (SPARK-47311) Suppress Python exceptions where PySpark is not in the Python path
[
https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-47311:
---
Labels: pull-request-available (was: )
> Suppress Python exceptions where PySpark is not in the Python path
> --
>
> Key: SPARK-47311
> URL: https://issues.apache.org/jira/browse/SPARK-47311
> Project: Spark
> Issue Type: Test
> Components: PySpark, SQL
>Affects Versions: 4.0.0
>Reporter: Hyukjin Kwon
>Priority: Minor
> Labels: pull-request-available
>
> {code}
> scala> sql("create table t(i int) using json")
> 24/03/06 16:09:44 WARN DataSourceManager: Skipping the lookup of Python Data
> Sources due to the failure.
> org.apache.spark.SparkException:
> Error from python worker:
> /opt/homebrew/Caskroom/miniconda/base/bin/python3: Error while finding
> module specification for 'pyspark.daemon' (ModuleNotFoundError: No module
> named 'pyspark')
> {code}
> When PySpark is not in the Python path at all, it always shows this warning
> message once for every Spark session initialization.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-47311) Suppress Python exceptions where PySpark is not in the Python path
[
https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-47311:
-
Description:
{code}
scala> sql("create table t(i int) using json")
24/03/06 16:09:44 WARN DataSourceManager: Skipping the lookup of Python Data
Sources due to the failure.
org.apache.spark.SparkException:
Error from python worker:
/opt/homebrew/Caskroom/miniconda/base/bin/python3: Error while finding module
specification for 'pyspark.daemon' (ModuleNotFoundError: No module named
'pyspark')
{code}
When PySpark is not in the Python path at all, it always shows this warning
message once for every Spark session initialization.
was:
{code}
scala> sql("create table t(i int) using json")
24/03/06 16:09:44 WARN DataSourceManager: Skipping the lookup of Python Data
Sources due to the failure.
org.apache.spark.SparkException:
Error from python worker:
/opt/homebrew/Caskroom/miniconda/base/bin/python3: Error while finding module
specification for 'pyspark.daemon' (ModuleNotFoundError: No module named
'pyspark')
{code}
This is too much.
> Suppress Python exceptions where PySpark is not in the Python path
> --
>
> Key: SPARK-47311
> URL: https://issues.apache.org/jira/browse/SPARK-47311
> Project: Spark
> Issue Type: Test
> Components: PySpark, SQL
>Affects Versions: 4.0.0
>Reporter: Hyukjin Kwon
>Priority: Minor
>
> {code}
> scala> sql("create table t(i int) using json")
> 24/03/06 16:09:44 WARN DataSourceManager: Skipping the lookup of Python Data
> Sources due to the failure.
> org.apache.spark.SparkException:
> Error from python worker:
> /opt/homebrew/Caskroom/miniconda/base/bin/python3: Error while finding
> module specification for 'pyspark.daemon' (ModuleNotFoundError: No module
> named 'pyspark')
> {code}
> When PySpark is not in the Python path at all, it always shows this warning
> message once for every Spark session initialization.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-47311) Suppress Python exceptions where PySpark is not in the Python path
[
https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-47311:
-
Summary: Suppress Python exceptions where PySpark is not in the Python path
(was: Surpress Python exceptions where PySpark is not in the Python path)
> Suppress Python exceptions where PySpark is not in the Python path
> --
>
> Key: SPARK-47311
> URL: https://issues.apache.org/jira/browse/SPARK-47311
> Project: Spark
> Issue Type: Test
> Components: PySpark, SQL
>Affects Versions: 4.0.0
>Reporter: Hyukjin Kwon
>Priority: Minor
>
> {code}
> scala> sql("create table t(i int) using json")
> 24/03/06 16:09:44 WARN DataSourceManager: Skipping the lookup of Python Data
> Sources due to the failure.
> org.apache.spark.SparkException:
> Error from python worker:
> /opt/homebrew/Caskroom/miniconda/base/bin/python3: Error while finding
> module specification for 'pyspark.daemon' (ModuleNotFoundError: No module
> named 'pyspark')
> {code}
> This is too much.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
