[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252299#comment-14252299 ] Cody Koeninger commented on SPARK-3146: --- Yes, for the specific case of kafka,

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252397#comment-14252397 ] Cody Koeninger commented on SPARK-3146: --- ((K, V), (topicAndPartition, offset)) is

[jira] [Commented] (SPARK-4196) Streaming + checkpointing yields NotSerializableException for Hadoop Configuration from saveAsNewAPIHadoopFiles ?

[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196641#comment-14196641 ] Cody Koeninger commented on SPARK-4196: --- Have you tried replacing

[jira] [Created] (SPARK-4229) Create hadoop configuration in a consistent way

Cody Koeninger created SPARK-4229: - Summary: Create hadoop configuration in a consistent way Key: SPARK-4229 URL: https://issues.apache.org/jira/browse/SPARK-4229 Project: Spark Issue Type:

[jira] [Updated] (SPARK-4229) Create hadoop configuration in a consistent way

[ https://issues.apache.org/jira/browse/SPARK-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger updated SPARK-4229: -- Description: Some places use SparkHadoopUtil.get.conf, some create a new hadoop config.

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195146#comment-14195146 ] Cody Koeninger commented on SPARK-3146: --- I think this PR is an elegant way to solve

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128451#comment-14128451 ] Cody Koeninger commented on SPARK-3462: --- Created a PR for feedback.

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128682#comment-14128682 ] Cody Koeninger commented on SPARK-3462: --- Tested this on a cluster against unions of

[jira] [Created] (SPARK-3462) parquet pushdown for unionAll

Cody Koeninger created SPARK-3462: - Summary: parquet pushdown for unionAll Key: SPARK-3462 URL: https://issues.apache.org/jira/browse/SPARK-3462 Project: Spark Issue Type: Improvement

<    1   2   3   4   5