[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14518795#comment-14518795
]
eminent commented on SPARK-3808:
Yes. It's the cause.
After updating the %PATH%, spark was successfully launched. Thanks so much for
your help!
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.2.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Assignee: Masayoshi TSUZUKI
>Priority: Blocker
> Fix For: 1.2.0
>
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516705#comment-14516705
]
Masayoshi TSUZUKI commented on SPARK-3808:
--
It seems to fail to run jvm.
Perhaps you haven't got java.exe in environment variable %PATH%.
{code}
set PATH=%PATH%;C:\path\to\java.exe
{code}
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.2.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Assignee: Masayoshi TSUZUKI
>Priority: Blocker
> Fix For: 1.2.0
>
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516588#comment-14516588
]
eminent commented on SPARK-3808:
I still met the problem even if compute-classpath.cmd is up to day (there is
not '!=' in the cmd file)
Is there any other suggestion to solve the problem?
Environment:
Windows 7 64bit
Python 2.7.9
spark-1.2.0-bin-hadoop2.4
log:
Running python with PYTHONPATH=C:\spark\bin\..\python\lib\py4j-0.8.2.1-src.zip;C
:\spark\bin\..\python;
Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit (AMD64)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
else was unexpected at this time.
Traceback (most recent call last):
File "C:\spark\bin\..\python\pyspark\shell.py", line 45, in
sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
File "C:\spark\python\pyspark\context.py", line 102, in __init__
SparkContext._ensure_initialized(self, gateway=gateway)
File "C:\spark\python\pyspark\context.py", line 211, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway()
File "C:\spark\python\pyspark\java_gateway.py", line 73, in launch_gateway
raise Exception(error_msg)
Exception: Launching GatewayServer failed with exit code 255!
Warning: Expected GatewayServer to output a port, but found no output.
>>>
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.2.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Assignee: Masayoshi TSUZUKI
>Priority: Blocker
> Fix For: 1.2.0
>
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162935#comment-14162935
]
Masayoshi TSUZUKI commented on SPARK-3808:
--
I verified and it works well! Thank you [~andrewor14]
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.2.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Assignee: Masayoshi TSUZUKI
>Priority: Blocker
> Fix For: 1.2.0
>
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162310#comment-14162310
]
Andrew Or commented on SPARK-3808:
--
Hey [~tsudukim] can you verify that pyspark, spark-shell and spark-submit work
as expected in Windows now?
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.2.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Assignee: Masayoshi TSUZUKI
>Priority: Blocker
> Fix For: 1.2.0
>
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160096#comment-14160096
]
Masayoshi TSUZUKI commented on SPARK-3808:
--
The same problem occurs when we execute spark-shell.cmd and spark-submit.cmd in
Windows.
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.1.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Priority: Blocker
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160078#comment-14160078
]
Apache Spark commented on SPARK-3808:
-
User 'tsudukim' has created a pull request for this issue:
https://github.com/apache/spark/pull/2669
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.1.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Priority: Blocker
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows
[
https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160071#comment-14160071
]
Masayoshi TSUZUKI commented on SPARK-3808:
--
The root cause of this problem is an syntax error in bin\compute-classpath.cmd .
{code}
if "x%SPARK_CONF_DIR%"!="x" (
{code}
Batch script of Windows doesn't have the '!=' operator.
We should use 'not' and '==' instead.
> PySpark fails to start in Windows
> -
>
> Key: SPARK-3808
> URL: https://issues.apache.org/jira/browse/SPARK-3808
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Windows
>Affects Versions: 1.1.0
> Environment: Windows
>Reporter: Masayoshi TSUZUKI
>Priority: Blocker
>
> When we execute bin\pyspark.cmd in Windows, it fails to start.
> We get following messages.
> {noformat}
> C:\>bin\pyspark.cmd
> Running C:\\python.exe with
> PYTHONPATH=C:\\bin\..\python\lib\py4j-0.8.2.1-src.zip;C:\\bin\..\python;
> Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> ="x" was unexpected at this time.
> Traceback (most recent call last):
> File "C:\\bin\..\python\pyspark\shell.py", line 45, in
> sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
> File "C:\\python\pyspark\context.py", line 103, in __init__
> SparkContext._ensure_initialized(self, gateway=gateway)
> File "C:\\python\pyspark\context.py", line 212, in _ensure_initialized
> SparkContext._gateway = gateway or launch_gateway()
> File "C:\\python\pyspark\java_gateway.py", line 71, in launch_gateway
> raise Exception(error_msg)
> Exception: Launching GatewayServer failed with exit code 255!
> Warning: Expected GatewayServer to output a port, but found no output.
> >>>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
