A couple points:
You've got mismatched versions here -- 1.2.0 vs 1.2.1. You should fix
that but it's not your problem.
These are also supposed to be 'provided' scope dependencies in Maven.
You should get the Scala deps transitively and can import scala.*
classes. However, it would be a little
It works after sync, thanks for the pointers
On Tue, Mar 10, 2015 at 1:22 PM, Mohit Anchlia mohitanch...@gmail.com
wrote:
I navigated to maven dependency and found scala library. I also found
Tuple2.class and when I click on it in eclipse I get invalid LOC header
(bad signature)
If you are using tools like SBT/Maven/Gradle/etc, they figure out all the
recursive dependencies and includes them in the class path. I haven't
touched Eclipse in years so I am not sure off the top of my head what's
going on instead. Just in case you only downloaded the
spark-streaming_2.10.jar
You have to include Scala libraries in the Eclipse dependencies.
TD
On Tue, Mar 10, 2015 at 10:54 AM, Mohit Anchlia mohitanch...@gmail.com
wrote:
I am trying out streaming example as documented and I am using spark 1.2.1
streaming from maven for Java.
When I add this code I get compilation
How do I do that? I haven't used Scala before.
Also, linking page doesn't mention that:
http://spark.apache.org/docs/1.2.0/streaming-programming-guide.html#linking
On Tue, Mar 10, 2015 at 10:57 AM, Sean Owen so...@cloudera.com wrote:
It means you do not have Scala library classes in your
I navigated to maven dependency and found scala library. I also found
Tuple2.class and when I click on it in eclipse I get invalid LOC header
(bad signature)
java.util.zip.ZipException: invalid LOC header (bad signature)
at java.util.zip.ZipFile.read(Native Method)
I am wondering if I should
Ah, that's a typo in the example: use words.mapToPair
I can make a little PR to fix that.
On Tue, Mar 10, 2015 at 8:32 PM, Mohit Anchlia mohitanch...@gmail.com wrote:
I am getting following error. When I look at the sources it seems to be a
scala source, but not sure why it's complaining about
Or another option is to use Scala-IDE, which is built on top of Eclipse,
instead of pure Eclipse, so Scala comes with it.
Yong
From: so...@cloudera.com
Date: Tue, 10 Mar 2015 18:40:44 +
Subject: Re: Compilation error
To: mohitanch...@gmail.com
CC: t...@databricks.com; user
I am using maven and my dependency looks like this, but this doesn't seem
to be working
dependencies
dependency
groupIdorg.apache.spark/groupId
artifactIdspark-streaming_2.10/artifactId
version1.2.0/version
/dependency
dependency
groupIdorg.apache.spark/groupId
See if you can import scala libraries in your project.
On Tue, Mar 10, 2015 at 11:32 AM, Mohit Anchlia mohitanch...@gmail.com
wrote:
I am using maven and my dependency looks like this, but this doesn't seem
to be working
dependencies
dependency
groupIdorg.apache.spark/groupId
I ran the dependency command and see the following dependencies:
I only see org.scala-lang.
[INFO] org.spark.test:spak-test:jar:0.0.1-SNAPSHOT
[INFO] +- org.apache.spark:spark-streaming_2.10:jar:1.2.0:compile
[INFO] | +- org.eclipse.jetty:jetty-server:jar:8.1.14.v20131031:compile
[INFO] | |
works now. I should have checked :)
On Tue, Mar 10, 2015 at 1:44 PM, Sean Owen so...@cloudera.com wrote:
Ah, that's a typo in the example: use words.mapToPair
I can make a little PR to fix that.
On Tue, Mar 10, 2015 at 8:32 PM, Mohit Anchlia mohitanch...@gmail.com
wrote:
I am getting
Hi,triedmvn -Phbase-hadoop2,hadoop-2.4,yarn -Dhadoop.version=2.4.1 -DskipTests dependency:tree dep.txtAttached the dep. txt for your information.[WARNING]
[WARNING] Some problems were encountered while building the effective settings
[WARNING] Unrecognised tag: 'mirrors' (position: START_TAG
I see 0.98.5 in dep.txt
You should be good to go.
On Thu, Aug 28, 2014 at 3:16 AM, arthur.hk.c...@gmail.com
arthur.hk.c...@gmail.com wrote:
Hi,
tried
mvn -Phbase-hadoop2,hadoop-2.4,yarn -Dhadoop.version=2.4.1 -DskipTests
dependency:tree dep.txt
Attached the dep. txt for your
Hi,
I tried to start Spark but failed:
$ ./sbin/start-all.sh
starting org.apache.spark.deploy.master.Master, logging to
/mnt/hadoop/spark-1.0.2/sbin/../logs/spark-edhuser-org.apache.spark.deploy.master.Master-1-m133.out
failed to launch org.apache.spark.deploy.master.Master:
Failed to find
I didn't see that problem.
Did you run this command ?
mvn -Phbase-hadoop2,hadoop-2.4,yarn -Dhadoop.version=2.4.1 -DskipTests
clean package
Here is what I got:
TYus-MacBook-Pro:spark-1.0.2 tyu$ sbin/start-all.sh
starting org.apache.spark.deploy.master.Master, logging to
See SPARK-1297
The pull request is here:
https://github.com/apache/spark/pull/1893
On Wed, Aug 27, 2014 at 6:57 PM, arthur.hk.c...@gmail.com
arthur.hk.c...@gmail.com wrote:
(correction: Compilation Error: Spark 1.0.2 with HBase 0.98” , please
ignore if duplicated)
Hi,
I need to use
Hi Ted,
Thank you so much!!
As I am new to Spark, can you please advise the steps about how to apply this
patch to my spark-1.0.2 source folder?
Regards
Arthur
On 28 Aug, 2014, at 10:13 am, Ted Yu yuzhih...@gmail.com wrote:
See SPARK-1297
The pull request is here:
You can get the patch from this URL:
https://github.com/apache/spark/pull/1893.patch
BTW 0.98.5 has been released - you can specify 0.98.5-hadoop2 in the pom.xml
Cheers
On Wed, Aug 27, 2014 at 7:18 PM, arthur.hk.c...@gmail.com
arthur.hk.c...@gmail.com wrote:
Hi Ted,
Thank you so much!!
Hi Ted,
I tried the following steps to apply the patch 1893 but got Hunk FAILED, can
you please advise how to get thru this error? or is my spark-1.0.2 source not
the correct one?
Regards
Arthur
wget http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2.tgz
tar -vxf spark-1.0.2.tgz
cd spark-1.0.2
Can you use this command ?
patch -p1 -i 1893.patch
Cheers
On Wed, Aug 27, 2014 at 7:41 PM, arthur.hk.c...@gmail.com
arthur.hk.c...@gmail.com wrote:
Hi Ted,
I tried the following steps to apply the patch 1893 but got Hunk FAILED,
can you please advise how to get thru this error? or is my
Hi Ted,
Thanks.
Tried [patch -p1 -i 1893.patch](Hunk #1 FAILED at 45.)
Is this normal?
Regards
Arthur
patch -p1 -i 1893.patch
patching file examples/pom.xml
Hunk #1 FAILED at 45.
Hunk #2 succeeded at 94 (offset -16 lines).
1 out of 2 hunks FAILED -- saving rejects to file
Looks like the patch given by that URL only had the last commit.
I have attached pom.xml for spark-1.0.2 to SPARK-1297
You can download it and replace examples/pom.xml with the downloaded pom
I am running this command locally:
mvn -Phbase-hadoop2,hadoop-2.4,yarn -DskipTests clean package
I forgot to include '-Dhadoop.version=2.4.1' in the command below.
The modified command passed.
You can verify the dependence on hbase 0.98 through this command:
mvn -Phbase-hadoop2,hadoop-2.4,yarn -Dhadoop.version=2.4.1 -DskipTests
dependency:tree dep.txt
Cheers
On Wed, Aug 27, 2014 at
Right, the compile error is a casting issue telling me I cannot assign
a JavaPairRDDPartition,
Body to a JavaPairRDDObject, Object. It happens in the mapToPair()
method.
On 9 July 2014 19:52, Sean Owen so...@cloudera.com wrote:
You forgot the compile error!
On Wed, Jul 9, 2014 at 6:14 PM,
25 matches
Mail list logo
