Re: Building Spark behind a proxy

2015-01-30 Thread Arush Kharbanda 
Hi Somya,

I meant when you configure the JAVA_OPTS and when you don't configure the
JAVA_OPTS is there any difference in the error message?

Are you facing the same issue when you built using maven?

Thanks
Arush

On Thu, Jan 29, 2015 at 10:22 PM, Soumya Simanta 
wrote:

> I can do a
> wget
> http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom
> and get the file successfully on a shell.
>
>
>
> On Thu, Jan 29, 2015 at 11:51 AM, Boromir Widas 
> wrote:
>
>> At least a part of it is due to connection refused, can you check if curl
>> can reach the URL with proxies -
>> [FATAL] Non-resolvable parent POM: Could not transfer artifact
>> org.apache:apache:pom:14 from/to central (
>> http://repo.maven.apache.org/maven2): Error transferring file:
>> Connection refused from
>> http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom
>>
>> On Thu, Jan 29, 2015 at 11:35 AM, Soumya Simanta <
>> soumya.sima...@gmail.com> wrote:
>>
>>>
>>>
>>> On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda <
>>> ar...@sigmoidanalytics.com> wrote:
>>>
 Does  the error change on build with and without the built options?

>>> What do you mean by build options? I'm just doing ./sbt/sbt assembly
>>> from $SPARK_HOME
>>>
>>>
 Did you try using maven? and doing the proxy settings there.

>>>
>>>  No I've not tried maven yet. However, I did set proxy settings inside
>>> my .m2/setting.xml, but it didn't make any difference.
>>>
>>>
>>>
>>
>


-- 

[image: Sigmoid Analytics] 

*Arush Kharbanda* || Technical Teamlead

ar...@sigmoidanalytics.com || www.sigmoidanalytics.com

Re: Building Spark behind a proxy

2015-01-29 Thread Soumya Simanta 
I can do a
wget http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom

and get the file successfully on a shell.



On Thu, Jan 29, 2015 at 11:51 AM, Boromir Widas  wrote:

> At least a part of it is due to connection refused, can you check if curl
> can reach the URL with proxies -
> [FATAL] Non-resolvable parent POM: Could not transfer artifact
> org.apache:apache:pom:14 from/to central (
> http://repo.maven.apache.org/maven2): Error transferring file: Connection
> refused from
> http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom
>
> On Thu, Jan 29, 2015 at 11:35 AM, Soumya Simanta  > wrote:
>
>>
>>
>> On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda <
>> ar...@sigmoidanalytics.com> wrote:
>>
>>> Does  the error change on build with and without the built options?
>>>
>> What do you mean by build options? I'm just doing ./sbt/sbt assembly from
>> $SPARK_HOME
>>
>>
>>> Did you try using maven? and doing the proxy settings there.
>>>
>>
>>  No I've not tried maven yet. However, I did set proxy settings inside my
>> .m2/setting.xml, but it didn't make any difference.
>>
>>
>>
>

Re: Building Spark behind a proxy

2015-01-29 Thread Boromir Widas 
At least a part of it is due to connection refused, can you check if curl
can reach the URL with proxies -
[FATAL] Non-resolvable parent POM: Could not transfer artifact
org.apache:apache:pom:14 from/to central (
http://repo.maven.apache.org/maven2): Error transferring file: Connection
refused from
http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom

On Thu, Jan 29, 2015 at 11:35 AM, Soumya Simanta 
wrote:

>
>
> On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda <
> ar...@sigmoidanalytics.com> wrote:
>
>> Does  the error change on build with and without the built options?
>>
> What do you mean by build options? I'm just doing ./sbt/sbt assembly from
> $SPARK_HOME
>
>
>> Did you try using maven? and doing the proxy settings there.
>>
>
>  No I've not tried maven yet. However, I did set proxy settings inside my
> .m2/setting.xml, but it didn't make any difference.
>
>
>

Re: Building Spark behind a proxy

2015-01-29 Thread Soumya Simanta 
On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda <
ar...@sigmoidanalytics.com> wrote:

> Does  the error change on build with and without the built options?
>
What do you mean by build options? I'm just doing ./sbt/sbt assembly from
$SPARK_HOME


> Did you try using maven? and doing the proxy settings there.
>

 No I've not tried maven yet. However, I did set proxy settings inside my
.m2/setting.xml, but it didn't make any difference.

Re: Building Spark behind a proxy

2015-01-29 Thread Arush Kharbanda 
Does  the error change on build with and without the built options?
Did you try using maven? and doing the proxy settings there.

On Thu, Jan 29, 2015 at 9:17 PM, Soumya Simanta 
wrote:

> I'm trying to build Spark (v1.1.1 and v1.2.0) behind a proxy using
> ./sbt/sbt assembly and I get the following error. I've set the http and
> https proxy as well as the JAVA_OPTS. Any idea what am I missing ?
>
>
> [warn] one warning found
> org.apache.maven.model.building.ModelBuildingException: 1 problem was
> encountered while building the effective model for
> org.apache.spark:spark-parent:1.1.1
> [FATAL] Non-resolvable parent POM: Could not transfer artifact
> org.apache:apache:pom:14 from/to central (
> http://repo.maven.apache.org/maven2): Error transferring file: Connection
> refused from
> http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom
> and 'parent.relativePath' points at wrong local POM @ line 21, column 11
>
> at
> org.apache.maven.model.building.DefaultModelProblemCollector.newModelBuildingException(DefaultModelProblemCollector.java:195)
> at
> org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:841)
> at
> org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:664)
> at
> org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:310)
> at
> org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:232)
> at
> com.typesafe.sbt.pom.MvnPomResolver.loadEffectivePom(MavenPomResolver.scala:61)
> at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41)
> at
> com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:128)
> at
> com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:49)
> at
> com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:28)
> at SparkBuild$.projectDefinitions(SparkBuild.scala:165)
> at sbt.Load$.sbt$Load$$projectsFromBuild(Load.scala:458)
> at sbt.Load$$anonfun$24.apply(Load.scala:415)
> at sbt.Load$$anonfun$24.apply(Load.scala:415)
> at scala.collection.immutable.Stream.flatMap(Stream.scala:442)
> at sbt.Load$.loadUnit(Load.scala:415)
> at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256)
> at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256)
> at
> sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:93)
> at
> sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:92)
> at sbt.BuildLoader.apply(BuildLoader.scala:143)
> at sbt.Load$.loadAll(Load.scala:312)
> at sbt.Load$.loadURI(Load.scala:264)
> at sbt.Load$.load(Load.scala:260)
> at sbt.Load$.load(Load.scala:251)
> at sbt.Load$.apply(Load.scala:134)
> at sbt.Load$.defaultLoad(Load.scala:37)
> at sbt.BuiltinCommands$.doLoadProject(Main.scala:473)
> at
> sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:467)
> at
> sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:467)
> at
> sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:60)
> at
> sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:60)
> at
> sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:62)
> at
> sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:62)
> at sbt.Command$.process(Command.scala:95)
> at
> sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:100)
> at
> sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:100)
> at sbt.State$$anon$1.process(State.scala:179)
> at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:100)
> at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:100)
> at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
> at sbt.MainLoop$.next(MainLoop.scala:100)
> at sbt.MainLoop$.run(MainLoop.scala:93)
> at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:71)
> at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:66)
> at sbt.Using.apply(Using.scala:25)
> at sbt.MainLoop$.runWithNewLog(MainLoop.scala:66)
> at sbt.MainLoop$.runAndClearLast(MainLoop.scala:49)
> at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:33)
> at sbt.MainLoop$.runLogged(MainLoop.scala:25)
> at sbt.StandardMain$.runManaged(Main.scala:57)
> at sbt.xMain.run(Main.scala:29)
> at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
> at xsbt.boot.Launch$.withContextLoader(Launch.scala:129)
> at xsbt.boot.Launch$.run(Launch.scala:109)
>