Re: Anybody hit this issue in spark shell?

I searched code base and confirmed that there is no class from com.google.common.annotations being used. However, there're classes from com.google.common e.g. import com.google.common.io.{ByteStreams, Files} import com.google.common.net.InetAddresses FYI On Tue, Nov 10, 2015 at 11:22 AM,

Re: Anybody hit this issue in spark shell?

Scala compiler stores some metadata in the ScalaSig attribute. See the following link as an example: http://stackoverflow.com/questions/10130106/how-does-scala-know-the-difference-between-def-foo-and-def-foo/10130403#10130403 As maven-shade-plugin doesn't recognize ScalaSig, it cannot fix the

Re: Anybody hit this issue in spark shell?

In the PR, a new scala style rule is added banning use of @VisibleForTesting Similar rules can be added as seen fit. Cheers On Tue, Nov 10, 2015 at 11:22 AM, Shixiong Zhu wrote: > Scala compiler stores some metadata in the ScalaSig attribute. See the > following link as an

Re: Anybody hit this issue in spark shell?

We've had this in the past when using "@VisibleForTesting" in classes that for some reason the shell tries to process. QueryExecution.scala seems to use that annotation and that was added recently, so that's probably the issue. BTW, if anyone knows how Scala can find a reference to the original

Re: Anybody hit this issue in spark shell?

On Mon, Nov 9, 2015 at 5:54 PM, Ted Yu wrote: > If there is no option to let shell skip processing @VisibleForTesting , > should the annotation be dropped ? That's what we did last time this showed up. > On Mon, Nov 9, 2015 at 5:50 PM, Marcelo Vanzin

Re: Anybody hit this issue in spark shell?

Yeah, we should probably remove that. On Mon, Nov 9, 2015 at 5:54 PM, Ted Yu wrote: > If there is no option to let shell skip processing @VisibleForTesting , > should the annotation be dropped ? > > Cheers > > On Mon, Nov 9, 2015 at 5:50 PM, Marcelo Vanzin

Re: Anybody hit this issue in spark shell?

Hi Zhan: I hit the exact problem you hit. I rolled back to commit: de289bf279e14e47859b5fbcd70e97b9d0759f14 which does not have this problem. I suspect something delivered in the past 4 days caused this problem. On Mon, Nov 9, 2015 at 12:20 PM Ted Yu wrote: > I

Re: Anybody hit this issue in spark shell?

When we remove this, we should add a style-checker rule to ban the import so that it doesn't get added back by accident. On Mon, Nov 9, 2015 at 6:13 PM, Michael Armbrust wrote: > Yeah, we should probably remove that. > > On Mon, Nov 9, 2015 at 5:54 PM, Ted Yu

Re: Anybody hit this issue in spark shell?

Created https://github.com/apache/spark/pull/9585 Cheers On Mon, Nov 9, 2015 at 6:39 PM, Josh Rosen wrote: > When we remove this, we should add a style-checker rule to ban the import > so that it doesn't get added back by accident. > > On Mon, Nov 9, 2015 at 6:13 PM,

Re: Anybody hit this issue in spark shell?

If there is no option to let shell skip processing @VisibleForTesting , should the annotation be dropped ? Cheers On Mon, Nov 9, 2015 at 5:50 PM, Marcelo Vanzin wrote: > We've had this in the past when using "@VisibleForTesting" in classes > that for some reason the shell

Re: Anybody hit this issue in spark shell?

Thanks Ted. I am using latest master branch. I will try your build command and give it a try. Thank. Zhan Zhang On Nov 9, 2015, at 10:46 AM, Ted Yu > wrote: Which branch did you perform the build with ? I used the following command yesterday:

Re: Anybody hit this issue in spark shell?

Which branch did you perform the build with ? I used the following command yesterday: mvn -Phive -Phive-thriftserver -Pyarn -Phadoop-2.4 -Dhadoop.version=2.7.0 package -DskipTests Spark shell was working. Building with latest master branch. On Mon, Nov 9, 2015 at 10:37 AM, Zhan Zhang

Anybody hit this issue in spark shell?

Hi Folks, Does anybody meet the following issue? I use "mvn package -Phive -DskipTests” to build the package. Thanks. Zhan Zhang bin/spark-shell ... Spark context available as sc. error: error while loading QueryExecution, Missing dependency 'bad symbolic reference. A signature in

Re: Anybody hit this issue in spark shell?

I backtracked to: ef362846eb448769bcf774fc9090a5013d459464 The issue was still there. FYI On Mon, Nov 9, 2015 at 10:46 AM, Ted Yu wrote: > Which branch did you perform the build with ? > > I used the following command yesterday: > mvn -Phive -Phive-thriftserver -Pyarn