Re: How to modify Hadoop APIs used by Spark?

[Dogtail L]

Oh, I want to modify existing Hadoop InputFormat.

On Mon, Sep 21, 2015 at 4:23 PM, Ted Yu  wrote:

> Can you clarify what you want to do:
> If you modify existing hadoop InputFormat, etc, it would be a matter of
> rebuilding hadoop and build Spark using the custom built hadoop as
> dependency.
>
> Do you introduce new InputFormat ?
>
> Cheers
>
> On Mon, Sep 21, 2015 at 1:20 PM, Dogtail Ray 
> wrote:
>
>> Hi all,
>>
>> I find that Spark uses some Hadoop APIs such as InputFormat, InputSplit,
>> etc., and I want to modify these Hadoop APIs. Do you know how can I
>> integrate my modified Hadoop code into Spark? Great thanks!
>>
>>
>

Re: How to modify Hadoop APIs used by Spark?

[Ted Yu]

Can you clarify what you want to do:
If you modify existing hadoop InputFormat, etc, it would be a matter of
rebuilding hadoop and build Spark using the custom built hadoop as
dependency.

Do you introduce new InputFormat ?

Cheers

On Mon, Sep 21, 2015 at 1:20 PM, Dogtail Ray  wrote:

> Hi all,
>
> I find that Spark uses some Hadoop APIs such as InputFormat, InputSplit,
> etc., and I want to modify these Hadoop APIs. Do you know how can I
> integrate my modified Hadoop code into Spark? Great thanks!
>
>

How to modify Hadoop APIs used by Spark?

[Dogtail Ray]

Hi all,

I find that Spark uses some Hadoop APIs such as InputFormat, InputSplit,
etc., and I want to modify these Hadoop APIs. Do you know how can I
integrate my modified Hadoop code into Spark? Great thanks!