subject:"How to modify Hadoop APIs used by Spark\?"

Re: How to modify Hadoop APIs used by Spark?

2015-09-21 Thread Dogtail L

Oh, I want to modify existing Hadoop InputFormat. On Mon, Sep 21, 2015 at 4:23 PM, Ted Yu wrote: > Can you clarify what you want to do: > If you modify existing hadoop InputFormat, etc, it would be a matter of > rebuilding hadoop and build Spark using the custom built

Re: How to modify Hadoop APIs used by Spark?

2015-09-21 Thread Ted Yu

Can you clarify what you want to do: If you modify existing hadoop InputFormat, etc, it would be a matter of rebuilding hadoop and build Spark using the custom built hadoop as dependency. Do you introduce new InputFormat ? Cheers On Mon, Sep 21, 2015 at 1:20 PM, Dogtail Ray

How to modify Hadoop APIs used by Spark?

2015-09-21 Thread Dogtail Ray

Hi all, I find that Spark uses some Hadoop APIs such as InputFormat, InputSplit, etc., and I want to modify these Hadoop APIs. Do you know how can I integrate my modified Hadoop code into Spark? Great thanks!