hi @Vinoth Chandar,
Got it, thanks.
best,
lamber-ken
At 2020-01-09 23:52:52, "Vinoth Chandar" wrote:
>Hi lamber-ken,
>
>A ConfigOption class would be good indeed. +1 on starting incrementally
>with DataSource first and then iterating..
>
>Thanks
>Vinoth
>
>On Tue, Jan 7, 2020 at 6:58
Hi lamber-ken,
A ConfigOption class would be good indeed. +1 on starting incrementally
with DataSource first and then iterating..
Thanks
Vinoth
On Tue, Jan 7, 2020 at 6:58 PM lamberken wrote:
>
>
> Hi @Vinoth,
>
>
> It's time to pick up this topic. Based on the content we talked about,
> here
Hi @Vinoth,
It's time to pick up this topic. Based on the content we talked about, here are
my thoughts
1, Initial proposal aims to rework configuration framework includes(DataSource
and WriteClient level),
for compatibility, we can introduce a ConfigOption class and rework it on
DataSour
Sounds good.. This scoped down version per se, does not need a RFC.
On Wed, Dec 18, 2019 at 3:09 PM lamberken wrote:
>
>
> Hi @Vinoth
>
>
> I understand what you mean, I will continue to work on this when I finish
> reworking the new UI. :)
>
>
> best,
> lamber-ken
>
>
>
>
> At 2019-12-18 11:39:
Hi @Vinoth
I understand what you mean, I will continue to work on this when I finish
reworking the new UI. :)
best,
lamber-ken
At 2019-12-18 11:39:30, "Vinoth Chandar" wrote:
>Expect most users to use inputDF.write() approach... Uber uses the lower
>level RDD apis, like the DeltaStream
Expect most users to use inputDF.write() approach... Uber uses the lower
level RDD apis, like the DeltaStreamer tool does..
If we don't rename configs and still support a builder, it should be fine.
I think we can scope this down to introducing a ConfigOption class that
ties, the key,value, defau
Hi, @vinoth
Okay, I see. If we don't want existing users to do any upgrading or
reconfigurations, then this refactor work will not make much sense.
This issue can be closed, because ConfigOptions and these builders do the same
things.
From another side, if we finish this work before a stable
Hi,
Are you saying these classes needs to change? If so, understood. But are
you planning on renaming configs or relocating them? We dont want existing
users to do any upgrading or reconfigurations
On Fri, Dec 13, 2019 at 10:28 AM lamberken wrote:
>
>
> Hi,
>
>
> They need to change due to this
Hi,
We are trying to understand if existing jobs (datasource, deltastreamer,
anything else) needs to change due to this.
On Wed, Dec 11, 2019 at 7:18 PM lamberken wrote:
>
>
> Hi, @vinoth
>
>
> 1, Hoodie*Config classes are only used to set default value when call
> their build method currently.
