Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-05-29 Thread Robert Metzger
Thanks a lot! Let's continue the discussion in the ticket! (I might not be able to respond before Monday there) On Thu, May 28, 2020 at 5:08 PM Sivaprasanna wrote: > FYI. > > I created a Jira to track this improvement. > https://issues.apache.org/jira/browse/FLINK-18013 > > - > Sivaprasanna > >

Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-05-28 Thread Sivaprasanna
FYI. I created a Jira to track this improvement. https://issues.apache.org/jira/browse/FLINK-18013 - Sivaprasanna On Thu, May 28, 2020 at 12:22 PM Sivaprasanna wrote: > Awesome. : ) > Thanks, Robert for signing up to be the reviewer. I will create Jira and > share the link here. > > Stay

Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-05-28 Thread Sivaprasanna
Awesome. : ) Thanks, Robert for signing up to be the reviewer. I will create Jira and share the link here. Stay safe. - Sivaprasanna On Thu, May 28, 2020 at 12:13 PM Robert Metzger wrote: > Hi Sivaprasanna, > > thanks a lot for your proposal. Now that I ran into a HadoopUtils-related > issue

Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-05-28 Thread Robert Metzger
Hi Sivaprasanna, thanks a lot for your proposal. Now that I ran into a HadoopUtils-related issue myself [1] I see the benefit in this proposal. I'm happy to be the Flink committer that mentors this change. If we do this, I would like to have a small scope for the initial change: - create a

Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-04-30 Thread Sivaprasanna
Bump. Please let me know, if someone is interested in reviewing this one. I am willing to start working on this. BTW, a small and new addition to the list: With FLINK-10114 merged, OrcBulkWriterFactory can also reuse `SerializableHadoopConfiguration` along with SequenceFileWriterFactory and

Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-03-30 Thread Chesnay Schepler
I would recommend to wait until a committer has signed up for reviewing your changes before preparing any PR. Otherwise the chances are high that you invest a lot of time but the changes never get in. On 30/03/2020 11:42, Sivaprasanna wrote: Hello Till, I agree with having the scope limited

Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-03-30 Thread Sivaprasanna
Hello Till, I agree with having the scope limited and more concentrated. I can file a Jira and get started with the code changes, as and when someone has some bandwidth, the review can also be done. What do you think? Cheers, Sivaprasanna On Mon, Mar 30, 2020 at 3:00 PM Till Rohrmann wrote: >

Re: [DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-03-30 Thread Till Rohrmann
Hi Sivaprasanna, thanks for starting this discussion. In general I like the idea to remove duplications and move common code to a shared module. As a recommendation, I would exclude the whole part about Flink's Hadoop compatibility modules because they are legacy code and hardly used anymore.

[DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-03-19 Thread Sivaprasanna
Hi, Continuing on an earlier discussion[1] regarding having a separate module for Hadoop related utility components, I have gone through our project briefly and found the following components which I feel could be moved to a separate module for reusability, and better module structure. Module