, Kevin
Cc: Paul Brown; user@spark.apache.org
Subject: Re: Use pig load function in spark
Hello Kevin,
You can take a look at our generic load
function<https://spark.apache.org/docs/1.3.0/sql-programming-guide.html#generic-loadsave-functions>.
For example, you can use
val df = sqlContex
to use spark to read and process these data.
>
>
>
> So we want to figure out a way to reuse our existing load function in
> spark to read these data.
>
>
>
> Any idea?
>
>
>
> Best Regards,
>
> Kevin.
>
>
>
> *From:* Paul Brown [mailto:p.
.
From: Paul Brown [mailto:p...@mult.ifario.us]
Sent: 2015年3月24日 4:11
To: Dai, Kevin
Subject: Re: Use pig load function in spark
The answer is "Maybe, but you probably don't want to do that.".
A typical Pig load function is devoted to bridging external data into Pig's
type s
You may be able to utilize Spork (Pig on Apache Spark) as a mechanism to do
this: https://github.com/sigmoidanalytics/spork
On Mon, Mar 23, 2015 at 2:29 AM Dai, Kevin wrote:
> Hi, all
>
>
>
> Can spark use pig’s load function to load data?
>
>
>
> Best Regards,
>
> Kevin.
>
