[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r471910255 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLog.scala ## @@ -97,18 +97,15 @@ class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r471886440 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLog.scala ## @@ -97,18 +97,13 @@ class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r471885971 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala ## @@ -173,37 +177,64 @@ abstract

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r471885284 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala ## @@ -173,37 +177,64 @@ abstract

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r471883048 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala ## @@ -173,37 +177,64 @@ abstract

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r471878410 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala ## @@ -106,10 +106,8 @@ abstract class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r456729588 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala ## @@ -106,10 +106,8 @@ abstract class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r452217417 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLog.scala ## @@ -97,18 +97,13 @@ class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r452217417 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLog.scala ## @@ -97,18 +97,13 @@ class