[jira] [Updated] (SPARK-11023) Error initializing SparkContext. java.net.URISyntaxException
[ https://issues.apache.org/jira/browse/SPARK-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11023: -- Assignee: Marcelo Vanzin > Error initializing SparkContext. java.net.URISyntaxException > > > Key: SPARK-11023 > URL: https://issues.apache.org/jira/browse/SPARK-11023 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 1.5.0, 1.5.1 > Environment: pyspark + windows >Reporter: Jose Antonio >Assignee: Marcelo Vanzin > Labels: windows > Fix For: 1.5.2, 1.6.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Simliar to SPARK-10326. > [https://issues.apache.org/jira/browse/SPARK-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949470#comment-14949470] > C:\WINDOWS\system32>pyspark --master yarn-client > Python 2.7.10 |Anaconda 2.3.0 (64-bit)| (default, Sep 15 2015, 14:26:14) [MSC > v.1500 64 bit (AMD64)] > Type "copyright", "credits" or "license" for more information. > IPython 4.0.0 – An enhanced Interactive Python. > ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help -> Python's own help system. > object? -> Details about 'object', use 'object??' for extra details. > 15/10/08 09:28:05 WARN MetricsSystem: Using default name DAGScheduler for > source because spark.app.id is not set. > 15/10/08 09:28:06 WARN : Your hostname, PC-509512 resolves to a > loopback/non-reachable address: fe80:0:0:0:0:5efe:a5f:c318%net3, but we > couldn't find any external IP address! > 15/10/08 09:28:08 WARN BlockReaderLocal: The short-circuit local reads > feature cannot be used because UNIX Domain sockets are not available on > Windows. > 15/10/08 09:28:08 ERROR SparkContext: Error initializing SparkContext. > java.net.URISyntaxException: Illegal character in opaque part at index 2: > C:\spark\bin\..\python\lib\pyspark.zip > at java.net.URI$Parser.fail(Unknown Source) > at java.net.URI$Parser.checkChars(Unknown Source) > at java.net.URI$Parser.parse(Unknown Source) > at java.net.URI.(Unknown Source) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:558) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:557) > at scala.collection.immutable.List.foreach(List.scala:318) > at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:557) > at > org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:628) > at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:119) > at > org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56) > at > org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144) > at org.apache.spark.SparkContext.(SparkContext.scala:523) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:214) > at > py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79) > at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68) > at py4j.GatewayConnection.run(GatewayConnection.java:207) > at java.lang.Thread.run(Unknown Source) > 15/10/08 09:28:08 ERROR Utils: Uncaught exception in thread Thread-2 > java.lang.NullPointerException > at > org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152) > at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1228) > at org.apache.spark.SparkEnv.stop(SparkEnv.scala:100) > at > org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1749) > at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1185) > at org.apache.spark.SparkContext.stop(SparkContext.scala:1748) > at org.apache.spark.SparkContext.(SparkContext.scala:593) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at
[jira] [Updated] (SPARK-11023) Error initializing SparkContext. java.net.URISyntaxException
[ https://issues.apache.org/jira/browse/SPARK-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-11023: --- Target Version/s: 1.5.2, 1.6.0 Fix Version/s: 1.6.0 > Error initializing SparkContext. java.net.URISyntaxException > > > Key: SPARK-11023 > URL: https://issues.apache.org/jira/browse/SPARK-11023 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 1.5.0, 1.5.1 > Environment: pyspark + windows >Reporter: Jose Antonio > Labels: windows > Fix For: 1.6.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Simliar to SPARK-10326. > [https://issues.apache.org/jira/browse/SPARK-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949470#comment-14949470] > C:\WINDOWS\system32>pyspark --master yarn-client > Python 2.7.10 |Anaconda 2.3.0 (64-bit)| (default, Sep 15 2015, 14:26:14) [MSC > v.1500 64 bit (AMD64)] > Type "copyright", "credits" or "license" for more information. > IPython 4.0.0 – An enhanced Interactive Python. > ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help -> Python's own help system. > object? -> Details about 'object', use 'object??' for extra details. > 15/10/08 09:28:05 WARN MetricsSystem: Using default name DAGScheduler for > source because spark.app.id is not set. > 15/10/08 09:28:06 WARN : Your hostname, PC-509512 resolves to a > loopback/non-reachable address: fe80:0:0:0:0:5efe:a5f:c318%net3, but we > couldn't find any external IP address! > 15/10/08 09:28:08 WARN BlockReaderLocal: The short-circuit local reads > feature cannot be used because UNIX Domain sockets are not available on > Windows. > 15/10/08 09:28:08 ERROR SparkContext: Error initializing SparkContext. > java.net.URISyntaxException: Illegal character in opaque part at index 2: > C:\spark\bin\..\python\lib\pyspark.zip > at java.net.URI$Parser.fail(Unknown Source) > at java.net.URI$Parser.checkChars(Unknown Source) > at java.net.URI$Parser.parse(Unknown Source) > at java.net.URI.(Unknown Source) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:558) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:557) > at scala.collection.immutable.List.foreach(List.scala:318) > at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:557) > at > org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:628) > at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:119) > at > org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56) > at > org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144) > at org.apache.spark.SparkContext.(SparkContext.scala:523) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:214) > at > py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79) > at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68) > at py4j.GatewayConnection.run(GatewayConnection.java:207) > at java.lang.Thread.run(Unknown Source) > 15/10/08 09:28:08 ERROR Utils: Uncaught exception in thread Thread-2 > java.lang.NullPointerException > at > org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152) > at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1228) > at org.apache.spark.SparkEnv.stop(SparkEnv.scala:100) > at > org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1749) > at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1185) > at org.apache.spark.SparkContext.stop(SparkContext.scala:1748) > at org.apache.spark.SparkContext.(SparkContext.scala:593) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at
[jira] [Updated] (SPARK-11023) Error initializing SparkContext. java.net.URISyntaxException
[ https://issues.apache.org/jira/browse/SPARK-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Antonio updated SPARK-11023: - Affects Version/s: 1.6.0 > Error initializing SparkContext. java.net.URISyntaxException > > > Key: SPARK-11023 > URL: https://issues.apache.org/jira/browse/SPARK-11023 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 1.5.0, 1.5.1, 1.6.0 > Environment: pyspark + windows >Reporter: Jose Antonio > Labels: windows > Original Estimate: 24h > Remaining Estimate: 24h > > Simliar to SPARK-10326. > [https://issues.apache.org/jira/browse/SPARK-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949470#comment-14949470] > C:\WINDOWS\system32>pyspark --master yarn-client > Python 2.7.10 |Anaconda 2.3.0 (64-bit)| (default, Sep 15 2015, 14:26:14) [MSC > v.1500 64 bit (AMD64)] > Type "copyright", "credits" or "license" for more information. > IPython 4.0.0 – An enhanced Interactive Python. > ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help -> Python's own help system. > object? -> Details about 'object', use 'object??' for extra details. > 15/10/08 09:28:05 WARN MetricsSystem: Using default name DAGScheduler for > source because spark.app.id is not set. > 15/10/08 09:28:06 WARN : Your hostname, PC-509512 resolves to a > loopback/non-reachable address: fe80:0:0:0:0:5efe:a5f:c318%net3, but we > couldn't find any external IP address! > 15/10/08 09:28:08 WARN BlockReaderLocal: The short-circuit local reads > feature cannot be used because UNIX Domain sockets are not available on > Windows. > 15/10/08 09:28:08 ERROR SparkContext: Error initializing SparkContext. > java.net.URISyntaxException: Illegal character in opaque part at index 2: > C:\spark\bin\..\python\lib\pyspark.zip > at java.net.URI$Parser.fail(Unknown Source) > at java.net.URI$Parser.checkChars(Unknown Source) > at java.net.URI$Parser.parse(Unknown Source) > at java.net.URI.(Unknown Source) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:558) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:557) > at scala.collection.immutable.List.foreach(List.scala:318) > at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:557) > at > org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:628) > at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:119) > at > org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56) > at > org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144) > at org.apache.spark.SparkContext.(SparkContext.scala:523) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:214) > at > py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79) > at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68) > at py4j.GatewayConnection.run(GatewayConnection.java:207) > at java.lang.Thread.run(Unknown Source) > 15/10/08 09:28:08 ERROR Utils: Uncaught exception in thread Thread-2 > java.lang.NullPointerException > at > org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152) > at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1228) > at org.apache.spark.SparkEnv.stop(SparkEnv.scala:100) > at > org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1749) > at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1185) > at org.apache.spark.SparkContext.stop(SparkContext.scala:1748) > at org.apache.spark.SparkContext.(SparkContext.scala:593) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:214) > at >
[jira] [Updated] (SPARK-11023) Error initializing SparkContext. java.net.URISyntaxException
[ https://issues.apache.org/jira/browse/SPARK-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11023: -- Affects Version/s: (was: 1.6.0) [~jamartinh] tiny point of process: there is no 1.6.0 release that can be affected at this point. > Error initializing SparkContext. java.net.URISyntaxException > > > Key: SPARK-11023 > URL: https://issues.apache.org/jira/browse/SPARK-11023 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 1.5.0, 1.5.1 > Environment: pyspark + windows >Reporter: Jose Antonio > Labels: windows > Original Estimate: 24h > Remaining Estimate: 24h > > Simliar to SPARK-10326. > [https://issues.apache.org/jira/browse/SPARK-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949470#comment-14949470] > C:\WINDOWS\system32>pyspark --master yarn-client > Python 2.7.10 |Anaconda 2.3.0 (64-bit)| (default, Sep 15 2015, 14:26:14) [MSC > v.1500 64 bit (AMD64)] > Type "copyright", "credits" or "license" for more information. > IPython 4.0.0 – An enhanced Interactive Python. > ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help -> Python's own help system. > object? -> Details about 'object', use 'object??' for extra details. > 15/10/08 09:28:05 WARN MetricsSystem: Using default name DAGScheduler for > source because spark.app.id is not set. > 15/10/08 09:28:06 WARN : Your hostname, PC-509512 resolves to a > loopback/non-reachable address: fe80:0:0:0:0:5efe:a5f:c318%net3, but we > couldn't find any external IP address! > 15/10/08 09:28:08 WARN BlockReaderLocal: The short-circuit local reads > feature cannot be used because UNIX Domain sockets are not available on > Windows. > 15/10/08 09:28:08 ERROR SparkContext: Error initializing SparkContext. > java.net.URISyntaxException: Illegal character in opaque part at index 2: > C:\spark\bin\..\python\lib\pyspark.zip > at java.net.URI$Parser.fail(Unknown Source) > at java.net.URI$Parser.checkChars(Unknown Source) > at java.net.URI$Parser.parse(Unknown Source) > at java.net.URI.(Unknown Source) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:558) > at > org.apache.spark.deploy.yarn.Client$$anonfun$setupLaunchEnv$7.apply(Client.scala:557) > at scala.collection.immutable.List.foreach(List.scala:318) > at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:557) > at > org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:628) > at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:119) > at > org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56) > at > org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144) > at org.apache.spark.SparkContext.(SparkContext.scala:523) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:214) > at > py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79) > at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68) > at py4j.GatewayConnection.run(GatewayConnection.java:207) > at java.lang.Thread.run(Unknown Source) > 15/10/08 09:28:08 ERROR Utils: Uncaught exception in thread Thread-2 > java.lang.NullPointerException > at > org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152) > at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1228) > at org.apache.spark.SparkEnv.stop(SparkEnv.scala:100) > at > org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1749) > at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1185) > at org.apache.spark.SparkContext.stop(SparkContext.scala:1748) > at org.apache.spark.SparkContext.(SparkContext.scala:593) > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > at