danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-840400049
Thanks @vinothchandar for the review, i have addressed your comments ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to G
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-839535634
> @danny0405 : Agree with @vinothchandar in what this change would
accomplish. If we have incremental syncing enabled and move timeline server
hosting to Job Manager, we will get
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-838285067
@vinothchandar Thanks, i have added the config options in the PR, i'm also
wondering the details how the filesystem view was synced with a new client
instant time request, could
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-838074201
> @danny0405 if your concern is memory usage alone and not the time taken to
spin up the timeline server for each write task, then worth exploring if you
want to use `EMBEDDED_KV
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-837889063
> each write task is sending a RPC call to itself? Could you confirm that.
Yes, i can confirm this. Each task initialized a write client when it starts
so a separate timeli
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-837659380
> > each write task write out these records buffer (grouping by file group
id) using a write client there.
>
> Understood. So each task manager is, sending RPC calls to the
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834955549
> > each writer task takes a AbstractWriteClient, and there may be multiple
write tasks in one process.
>
> @danny0405 please help me understand this better. By one process
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834118649
> Looking at the usage of `createEmbeddedTimelineService()`, there are
broadly two.
>
> * AbstractHoodieClient can start (during intiialization) and stop (during
close())
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834117293
> AbstractHoodieClient
They are not singleton, as i said, each writer task takes a
AbstractWriteClient, and there may be multiple write tasks in one process.
I agree
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834084750
Hi @vinothchandar Can you take a look, this PR is small and we need this in
emergency.
--
This is an automated message from the Apache Git Service.
To respond to the message, p
10 matches
Mail list logo
