Re: Issue with Spark shell and scalatest

hm still struggling with those two above scala> import org.apache.spark.internal.Logging :57: error: object internal is not a member of package org.apache.spark import org.apache.spark.internal.Logging ^ scala> import

Re: Issue with Spark shell and scalatest

For you, it should be spark-core_2.10-1.5.1.jar Please replace version of Spark in my example with the version you use. On Thu, Apr 21, 2016 at 1:23 PM, Mich Talebzadeh wrote: > Hi Ted > > I cannot see spark-core_2.11-2.0.0-SNAPSHOT.jar under > >

Re: Issue with Spark shell and scalatest

Hi Ted I cannot see spark-core_2.11-2.0.0-SNAPSHOT.jar under https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/ Sorry where are these artefacts please? Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Issue with Spark shell and scalatest

Plug in 1.5.1 for your jars: $ jar tvf ./core/target/spark-core_2.11-2.0.0-SNAPSHOT.jar | grep Logging ... 1781 Thu Apr 21 08:19:34 PDT 2016 org/apache/spark/internal/Logging$.class jar tvf external/kafka/target/spark-streaming-kafka_2.11-2.0.0-SNAPSHOT.jar | grep LeaderOffset ... 3310 Thu

Re: Issue with Spark shell and scalatest

These two are giving me griefs: scala> import org.apache.spark.internal.Logging :26: error: object internal is not a member of package org.apache.spark import org.apache.spark.internal.Logging scala> import org.apache.spark.streaming.kafka.KafkaCluster.LeaderOffset :29: error: object

Re: Issue with Spark shell and scalatest

Thanks jar tvf spark-core_2.10-1.5.1-tests.jar | grep SparkFunSuite 1787 Wed Sep 23 23:34:26 BST 2015 org/apache/spark/SparkFunSuite$$anonfun$withFixture$1.class 1780 Wed Sep 23 23:34:26 BST 2015 org/apache/spark/SparkFunSuite$$anonfun$withFixture$2.class 3982 Wed Sep 23 23:34:26 BST 2015

Re: Issue with Spark shell and scalatest

Please replace version number for the release you are using : spark-core_2.10-1.5.1-tests.jar On Thu, Apr 21, 2016 at 10:18 AM, Mich Talebzadeh wrote: > I don't seem to be able to locate spark-core_2.11-2.0.0-SNAPSHOT-tests.jar > file :( > > Dr Mich Talebzadeh > > >

Re: Issue with Spark shell and scalatest

I don't seem to be able to locate spark-core_2.11-2.0.0-SNAPSHOT-tests.jar file :( Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw *

Re: Issue with Spark shell and scalatest

It is in core-XX-tests jar: $ jar tvf ./core/target/spark-core_2.11-2.0.0-SNAPSHOT-tests.jar | grep SparkFunSuite 1830 Thu Apr 21 08:19:14 PDT 2016 org/apache/spark/SparkFunSuite$$anonfun$withFixture$1.class 1823 Thu Apr 21 08:19:14 PDT 2016

Re: Issue with Spark shell and scalatest

like war of attrition :) now I get with sbt object SparkFunSuite is not a member of package org.apache.spark Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Issue with Spark shell and scalatest

Have you tried the following ? libraryDependencies += "org.scalatest" %% "scalatest" % "2.2.6" On Thu, Apr 21, 2016 at 9:19 AM, Mich Talebzadeh wrote: > Unfortunately this sbt dependency is not working > > libraryDependencies += "org.apache.spark" %% "spark-core" %

Re: Issue with Spark shell and scalatest

Unfortunately this sbt dependency is not working libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.1" % "provided" libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.5.1" % "provided" libraryDependencies += "org.apache.spark" %% "spark-hive" % "1.5.1" % "provided"

Re: Issue with Spark shell and scalatest

Thanks Ted. It was a typo in my alias and it is sorted now slong='rlwrap spark-shell --master spark://50.140.197.217:7077 --jars

Re: Issue with Spark shell and scalatest

I tried on refreshed copy of master branch: $ bin/spark-shell --jars /home/hbase/.m2/repository/org/scalatest/scalatest_2.11/2.2.6/scalatest_2.11-2.2.6.jar ... scala> import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll} import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll} BTW I noticed

Re: Issue with Spark shell and scalatest

Mich: $ jar tvf /home/hbase/.m2/repository/org/scalatest/scalatest_2.11/2.2.6/scalatest_2.11-2.2.6.jar | grep BeforeAndAfter 4257 Sat Dec 26 14:35:48 PST 2015 org/scalatest/BeforeAndAfter$class.class 2602 Sat Dec 26 14:35:48 PST 2015 org/scalatest/BeforeAndAfter.class 1998 Sat Dec 26

Issue with Spark shell and scalatest

to Mario, Alonso, Luciano, user Hi, Following example in https://github.com/agsachin/spark/blob/CEP/external/kafka/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala#L532 Does anyone know which jar file this belongs to? I use *scalatest_2.11-2.2.6.jar *in my