Re: [I] Iceberg Spark streaming skips rows of data [iceberg]

2024-08-19 Thread via GitHub
singhpk234 commented on issue #10156: URL: https://github.com/apache/iceberg/issues/10156#issuecomment-2297093657 @cccs-jc no i wasn't i tried this unit test : ``` @TestTemplate public void testResumingStreamReadFromCheckpointWithStreamFromTimestamp() throws Exception {

Re: [I] Iceberg Spark streaming skips rows of data [iceberg]

2024-08-18 Thread via GitHub
cccs-jc commented on issue #10156: URL: https://github.com/apache/iceberg/issues/10156#issuecomment-2295262592 @singhpk234 are you able to reproduce the issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] Iceberg Spark streaming skips rows of data [iceberg]

2024-07-02 Thread via GitHub
cccs-jc commented on issue #10156: URL: https://github.com/apache/iceberg/issues/10156#issuecomment-2189705490 The issue is that when a streamy query resumes (either it was killed, died, gracefully stopped) it does not resume where it left off but rather resumes based on the `stream-from-ti

Re: [I] Iceberg Spark streaming skips rows of data [iceberg]

2024-07-02 Thread via GitHub
singhpk234 commented on issue #10156: URL: https://github.com/apache/iceberg/issues/10156#issuecomment-2189687924 Haven't been looking into this actively. couple of questions : > That's because it applies the stream-from-timestamp when in fact it should not look at it at all

Re: [I] Iceberg Spark streaming skips rows of data [iceberg]

2024-06-24 Thread via GitHub
cccs-jc commented on issue #10156: URL: https://github.com/apache/iceberg/issues/10156#issuecomment-2186073673 @singhpk234 any progress on this issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[I] Iceberg Spark streaming skips rows of data [iceberg]

2024-04-16 Thread via GitHub
cccs-jc opened a new issue, #10156: URL: https://github.com/apache/iceberg/issues/10156 ### Apache Iceberg version 1.5.0 (latest release) ### Query engine Spark ### Please describe the bug 🐞 When using spark readStream the option `stream-from-timestamp` is u