HeartSaVioR commented on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-847523276
Let me close this PR as we have a revised PR. Thanks for the update,
@satishgopalani !
--
This is an automated message from the Apache Git Service.
To respond to the messa
HeartSaVioR commented on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-844550925
Thanks for confirming, @brkyvz !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
HeartSaVioR commented on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-843110304
Thanks for the input, @bozhang2820 ! Nice point on behavior with
Trigger.Once.
@satishgopalani Let's go with extending ReadLimit. ReadMinRows is OK, but we
may also n
HeartSaVioR commented on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-842955651
cc. @bozhang2820
Could you help taking a look, and provide some inputs on whether it makes
sense for extending ReadLimit? Thanks!
--
This is an automated message from
HeartSaVioR commented on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-842945316
I'd like to defer @brkyvz on the decision whether we'd need to address
ReadLimit here. I fully agree with the direction on the change, just want to
make sure we respect the
HeartSaVioR commented on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-834190258
Thanks for the contribution! The use case is interesting, especially the
sink is sensitive to the number of batches and is sub-optimal for lots of small
batches.
One
HeartSaVioR commented on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-834175330
ok to test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comme
