Re: Spark 2.1.1 (Scala 2.11.8) write to Phoenix 4.7 (HBase 1.1.2)

[Josh Mahonin]

Hi,

As per https://phoenix.apache.org/phoenix_spark.html, Apache Phoenix is
compiled against Spark2 only in versions 4.10 and above. If you must use
Phoenix 4.7 against Spark 2.x, you may need to apply PHOENIX- yourself:

https://github.com/apache/phoenix/commit/a0e5efcec5a1a732b2dce9794251242c3d66eea6#diff-600376dffeb79835ede4a0b285078036

Note that if you're using a vendor-provided Phoenix distribution, there may
already be support for the Spark version you're using. Please follow-up
with them if that's the case.

Best,

Josh




On Tue, Jan 30, 2018 at 10:06 AM, Margusja  wrote:

> Also I see that Logging is moved to internal/Logging. But is there package
> for my environment I can use?
>
> Margus
>
>
> On 30 Jan 2018, at 17:00, Margusja  wrote:
>
> Hi
>
> Followed page (https://phoenix.apache.org/phoenix_spark.html  phoenix.apache.org/phoenix_spark.html>) and trying to save to phoenix.
>
> Using spark-1.6.3 it is successful but using spark-2.1.1 it is not.
> First error I am getting using spark-2.1.1 is that:
>
> Error:scalac: missing or invalid dependency detected while loading class
> file 'ProductRDDFunctions.class'.
> Could not access type Logging in package org.apache.spark,
> because it (or its dependencies) are missing. Check your build definition
> for
> missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see
> the problematic classpath.)
> A full rebuild may help if 'ProductRDDFunctions.class' was compiled
> against an incompatible version of org.apache.spark.
>
> I can see that Logging is removed after 1.6.3 and does not exist in 2.1.1.
>
> What are my options?
>
> Br
> Margus
>
>
>

Re: Spark 2.1.1 (Scala 2.11.8) write to Phoenix 4.7 (HBase 1.1.2)

[Margusja]

Also I see that Logging is moved to internal/Logging. But is there package for 
my environment I can use?

Margus


> On 30 Jan 2018, at 17:00, Margusja  wrote:
> 
> Hi
> 
> Followed page (https://phoenix.apache.org/phoenix_spark.html 
>  
>  >) and trying to save to 
> phoenix.
> 
> Using spark-1.6.3 it is successful but using spark-2.1.1 it is not.
> First error I am getting using spark-2.1.1 is that:
> 
> Error:scalac: missing or invalid dependency detected while loading class file 
> 'ProductRDDFunctions.class'.
> Could not access type Logging in package org.apache.spark,
> because it (or its dependencies) are missing. Check your build definition for
> missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see 
> the problematic classpath.)
> A full rebuild may help if 'ProductRDDFunctions.class' was compiled against 
> an incompatible version of org.apache.spark.
> 
> I can see that Logging is removed after 1.6.3 and does not exist in 2.1.1.
> 
> What are my options?
> 
> Br
> Margus