[jira] [Commented] (SPARK-25408) Move to idiomatic Java 8

[ https://issues.apache.org/jira/browse/SPARK-25408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639257#comment-16639257 ] Apache Spark commented on SPARK-25408: -- User 'Fokko' has created a pull request for this issue:

[jira] [Commented] (SPARK-25629) ParquetFilterSuite: filter pushdown - decimal 16 sec

[ https://issues.apache.org/jira/browse/SPARK-25629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639195#comment-16639195 ] Apache Spark commented on SPARK-25629: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25629) ParquetFilterSuite: filter pushdown - decimal 16 sec

[ https://issues.apache.org/jira/browse/SPARK-25629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25629: Assignee: Apache Spark > ParquetFilterSuite: filter pushdown - decimal 16 sec >

[jira] [Commented] (SPARK-25629) ParquetFilterSuite: filter pushdown - decimal 16 sec

[ https://issues.apache.org/jira/browse/SPARK-25629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639194#comment-16639194 ] Apache Spark commented on SPARK-25629: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25629) ParquetFilterSuite: filter pushdown - decimal 16 sec

[ https://issues.apache.org/jira/browse/SPARK-25629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25629: Assignee: (was: Apache Spark) > ParquetFilterSuite: filter pushdown - decimal 16 sec

[jira] [Assigned] (SPARK-25408) Move to idiomatic Java 8

[ https://issues.apache.org/jira/browse/SPARK-25408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25408: - Assignee: Fokko Driesprong > Move to idiomatic Java 8 > > >

[jira] [Resolved] (SPARK-17159) Improve FileInputDStream.findNewFiles list performance

[ https://issues.apache.org/jira/browse/SPARK-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17159. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22339

[jira] [Resolved] (SPARK-25408) Move to idiomatic Java 8

[ https://issues.apache.org/jira/browse/SPARK-25408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25408. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22399

[jira] [Resolved] (SPARK-25606) DateExpressionsSuite: Hour 1 min

[ https://issues.apache.org/jira/browse/SPARK-25606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25606. - Resolution: Fixed Fix Version/s: 3.0.0 > DateExpressionsSuite: Hour 1 min >

[jira] [Resolved] (SPARK-25609) DataFrameSuite: SPARK-22226: splitExpressions should not generate codes beyond 64KB 49 seconds

[ https://issues.apache.org/jira/browse/SPARK-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25609. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 3.0.0 > DataFrameSuite:

[jira] [Resolved] (SPARK-25605) CastSuite: cast string to timestamp 2 mins 31 sec

[ https://issues.apache.org/jira/browse/SPARK-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25605. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 3.0.0 > CastSuite: cast string

[jira] [Assigned] (SPARK-25606) DateExpressionsSuite: Hour 1 min

[ https://issues.apache.org/jira/browse/SPARK-25606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25606: --- Assignee: Yuming Wang > DateExpressionsSuite: Hour 1 min > > >

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639147#comment-16639147 ] Marcelo Vanzin commented on SPARK-25645: I think I might have suggested this in the other bug

[jira] [Assigned] (SPARK-17159) Improve FileInputDStream.findNewFiles list performance

[ https://issues.apache.org/jira/browse/SPARK-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-17159: - Assignee: Steve Loughran > Improve FileInputDStream.findNewFiles list performance >

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639079#comment-16639079 ] Devaraj K commented on SPARK-25645: --- {code:java|title=with hflush(no hsync)|borderStyle=solid}

[jira] [Commented] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639059#comment-16639059 ] Apache Spark commented on SPARK-25591: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639061#comment-16639061 ] Apache Spark commented on SPARK-25591: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25591: Assignee: (was: Apache Spark) > PySpark Accumulators with multiple PythonUDFs >

[jira] [Assigned] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25591: Assignee: Apache Spark > PySpark Accumulators with multiple PythonUDFs >

[jira] [Commented] (SPARK-25646) docker-image-tool.sh doesn't work on developer build

[ https://issues.apache.org/jira/browse/SPARK-25646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639042#comment-16639042 ] Apache Spark commented on SPARK-25646: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639046#comment-16639046 ] Devaraj K commented on SPARK-25645: --- Thanks [~vanzin] for the jira pointer, I haven't tried just with

[jira] [Commented] (SPARK-25646) docker-image-tool.sh doesn't work on developer build

[ https://issues.apache.org/jira/browse/SPARK-25646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639045#comment-16639045 ] Apache Spark commented on SPARK-25646: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25646) docker-image-tool.sh doesn't work on developer build

[ https://issues.apache.org/jira/browse/SPARK-25646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25646: Assignee: Apache Spark > docker-image-tool.sh doesn't work on developer build >

[jira] [Assigned] (SPARK-25646) docker-image-tool.sh doesn't work on developer build

[ https://issues.apache.org/jira/browse/SPARK-25646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25646: Assignee: (was: Apache Spark) > docker-image-tool.sh doesn't work on developer build

[jira] [Created] (SPARK-25646) docker-image-tool.sh doesn't work on developer build

Marcelo Vanzin created SPARK-25646: -- Summary: docker-image-tool.sh doesn't work on developer build Key: SPARK-25646 URL: https://issues.apache.org/jira/browse/SPARK-25646 Project: Spark

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639024#comment-16639024 ] Marcelo Vanzin commented on SPARK-25645: This seems similar to SPARK-24787. Have you tried with

[jira] [Created] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

Devaraj K created SPARK-25645: - Summary: Add provision to disable EventLoggingListener default flush/hsync/hflush for all events Key: SPARK-25645 URL: https://issues.apache.org/jira/browse/SPARK-25645

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639005#comment-16639005 ] Devaraj K commented on SPARK-25645: --- {code:java|title=Present Behavior(flushLogger=true for some

[jira] [Updated] (SPARK-25644) Fix java foreachBatch API

[ https://issues.apache.org/jira/browse/SPARK-25644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25644: - Target Version/s: 2.4.0 > Fix java foreachBatch API > - > >

[jira] [Assigned] (SPARK-25644) Fix java foreachBatch API

[ https://issues.apache.org/jira/browse/SPARK-25644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25644: Assignee: Shixiong Zhu (was: Apache Spark) > Fix java foreachBatch API >

[jira] [Created] (SPARK-25644) Fix java foreachBatch API

Shixiong Zhu created SPARK-25644: Summary: Fix java foreachBatch API Key: SPARK-25644 URL: https://issues.apache.org/jira/browse/SPARK-25644 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-25164) Parquet reader builds entire list of columns once for each column

[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638803#comment-16638803 ] Bruce Robbins commented on SPARK-25164: --- [~Tagar] I've opened SPARK-25643 to keep track of the

[jira] [Commented] (SPARK-25644) Fix java foreachBatch API

[ https://issues.apache.org/jira/browse/SPARK-25644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638806#comment-16638806 ] Apache Spark commented on SPARK-25644: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25644) Fix java foreachBatch API

[ https://issues.apache.org/jira/browse/SPARK-25644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25644: Assignee: Apache Spark (was: Shixiong Zhu) > Fix java foreachBatch API >

[jira] [Updated] (SPARK-25644) Fix java foreachBatch API

[ https://issues.apache.org/jira/browse/SPARK-25644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25644: - Description: The java foreachBatch API in DataStreamWriter should accept java.lang.Long rather

[jira] [Created] (SPARK-25643) Performance issues querying wide rows

Bruce Robbins created SPARK-25643: - Summary: Performance issues querying wide rows Key: SPARK-25643 URL: https://issues.apache.org/jira/browse/SPARK-25643 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25455) Spark bundles jackson library version, which is vulnerable

[ https://issues.apache.org/jira/browse/SPARK-25455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638781#comment-16638781 ] Madhusudan N commented on SPARK-25455: -- Linking to the original issue

[jira] [Updated] (SPARK-25455) Spark bundles jackson library version, which is vulnerable

[ https://issues.apache.org/jira/browse/SPARK-25455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhusudan N updated SPARK-25455: - Priority: Major (was: Minor) > Spark bundles jackson library version, which is vulnerable >

[jira] [Resolved] (SPARK-25479) Refactor DatasetBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25479. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via

[jira] [Assigned] (SPARK-25479) Refactor DatasetBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25479: - Assignee: Yuming Wang > Refactor DatasetBenchmark to use main method >

[jira] [Updated] (SPARK-25642) Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-25642: - Description: Recently, the ability to expose the metrics for YARN Shuffle Service was added as

[jira] [Commented] (SPARK-25642) Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638722#comment-16638722 ] Apache Spark commented on SPARK-25642: -- User 'pgandhi999' has created a pull request for this

[jira] [Assigned] (SPARK-25642) Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25642: Assignee: (was: Apache Spark) > Add new Metrics in External Shuffle Service to help

[jira] [Commented] (SPARK-25642) Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638723#comment-16638723 ] Apache Spark commented on SPARK-25642: -- User 'pgandhi999' has created a pull request for this

[jira] [Assigned] (SPARK-25642) Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25642: Assignee: Apache Spark > Add new Metrics in External Shuffle Service to help determine

[jira] [Created] (SPARK-25642) Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service

Parth Gandhi created SPARK-25642: Summary: Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service Key: SPARK-25642 URL:

[jira] [Commented] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638635#comment-16638635 ] Iqbal Singh commented on SPARK-24295: - hey [~XuanYuan],  DO we have any plans of pulling this one

[jira] [Assigned] (SPARK-25605) CastSuite: cast string to timestamp 2 mins 31 sec

[ https://issues.apache.org/jira/browse/SPARK-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25605: Assignee: Apache Spark > CastSuite: cast string to timestamp 2 mins 31 sec >

[jira] [Assigned] (SPARK-25606) DateExpressionsSuite: Hour 1 min

[ https://issues.apache.org/jira/browse/SPARK-25606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25606: Assignee: (was: Apache Spark) > DateExpressionsSuite: Hour 1 min >

[jira] [Commented] (SPARK-25606) DateExpressionsSuite: Hour 1 min

[ https://issues.apache.org/jira/browse/SPARK-25606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638442#comment-16638442 ] Apache Spark commented on SPARK-25606: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25606) DateExpressionsSuite: Hour 1 min

[ https://issues.apache.org/jira/browse/SPARK-25606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25606: Assignee: Apache Spark > DateExpressionsSuite: Hour 1 min >

[jira] [Commented] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638414#comment-16638414 ] Apache Spark commented on SPARK-25497: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-25605) CastSuite: cast string to timestamp 2 mins 31 sec

[ https://issues.apache.org/jira/browse/SPARK-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638429#comment-16638429 ] Apache Spark commented on SPARK-25605: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25605) CastSuite: cast string to timestamp 2 mins 31 sec

[ https://issues.apache.org/jira/browse/SPARK-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25605: Assignee: (was: Apache Spark) > CastSuite: cast string to timestamp 2 mins 31 sec >

[jira] [Commented] (SPARK-25605) CastSuite: cast string to timestamp 2 mins 31 sec

[ https://issues.apache.org/jira/browse/SPARK-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638428#comment-16638428 ] Apache Spark commented on SPARK-25605: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638412#comment-16638412 ] Apache Spark commented on SPARK-25497: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-25641) Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100

[ https://issues.apache.org/jira/browse/SPARK-25641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638334#comment-16638334 ] Apache Spark commented on SPARK-25641: -- User 'redsanket' has created a pull request for this issue:

[jira] [Commented] (SPARK-22226) splitExpression can create too many method calls (generating a Constant Pool limit error)

[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638341#comment-16638341 ] Apache Spark commented on SPARK-6: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25609) DataFrameSuite: SPARK-22226: splitExpressions should not generate codes beyond 64KB 49 seconds

[ https://issues.apache.org/jira/browse/SPARK-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25609: Assignee: (was: Apache Spark) > DataFrameSuite: SPARK-6: splitExpressions should

[jira] [Assigned] (SPARK-25609) DataFrameSuite: SPARK-22226: splitExpressions should not generate codes beyond 64KB 49 seconds

[ https://issues.apache.org/jira/browse/SPARK-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25609: Assignee: Apache Spark > DataFrameSuite: SPARK-6: splitExpressions should not

[jira] [Commented] (SPARK-25609) DataFrameSuite: SPARK-22226: splitExpressions should not generate codes beyond 64KB 49 seconds

[ https://issues.apache.org/jira/browse/SPARK-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638338#comment-16638338 ] Apache Spark commented on SPARK-25609: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-22226) splitExpression can create too many method calls (generating a Constant Pool limit error)

[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638340#comment-16638340 ] Apache Spark commented on SPARK-6: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-25609) DataFrameSuite: SPARK-22226: splitExpressions should not generate codes beyond 64KB 49 seconds

[ https://issues.apache.org/jira/browse/SPARK-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638335#comment-16638335 ] Apache Spark commented on SPARK-25609: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25641) Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100

[ https://issues.apache.org/jira/browse/SPARK-25641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25641: Assignee: Apache Spark > Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent

[jira] [Assigned] (SPARK-25641) Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100

[ https://issues.apache.org/jira/browse/SPARK-25641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25641: Assignee: (was: Apache Spark) > Change the

[jira] [Created] (SPARK-25641) Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100

Sanket Reddy created SPARK-25641: Summary: Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100 Key: SPARK-25641 URL: https://issues.apache.org/jira/browse/SPARK-25641

[jira] [Updated] (SPARK-25640) Clarify/Improve EvalType for grouped aggregate and window aggregate

[ https://issues.apache.org/jira/browse/SPARK-25640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-25640: --- Description: Currently, grouped aggregate and window aggregate uses different EvalType, however, they map

[jira] [Created] (SPARK-25640) Clarify/Improve EvalType for grouped aggregate and window aggregate

Li Jin created SPARK-25640: -- Summary: Clarify/Improve EvalType for grouped aggregate and window aggregate Key: SPARK-25640 URL: https://issues.apache.org/jira/browse/SPARK-25640 Project: Spark

[jira] [Commented] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638226#comment-16638226 ] Michael Heuer commented on SPARK-25588: --- > Looking at the stack trace, it seems like we are using

[jira] [Commented] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638151#comment-16638151 ] Wenchen Fan commented on SPARK-25588: - The code snippet is a little hard to understand without

[jira] [Commented] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638155#comment-16638155 ] Wenchen Fan commented on SPARK-25588: - BTW is it possible that ADAM has some problem with avro

[jira] [Resolved] (SPARK-25602) SparkPlan.getByteArrayRdd should not consume the input when not necessary

[ https://issues.apache.org/jira/browse/SPARK-25602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25602. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22621

[jira] [Commented] (SPARK-22371) dag-scheduler-event-loop thread stopped with error Attempted to access garbage collected accumulator 5605982

[ https://issues.apache.org/jira/browse/SPARK-22371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638119#comment-16638119 ] Wenchen Fan commented on SPARK-22371: - if it's fixed in 2.3.1, it goes without saying that it's

[jira] [Commented] (SPARK-22371) dag-scheduler-event-loop thread stopped with error Attempted to access garbage collected accumulator 5605982

[ https://issues.apache.org/jira/browse/SPARK-22371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638105#comment-16638105 ] Davide Mandrini commented on SPARK-22371: - Hello, is this fix included in version 2.3.2? >From

[jira] [Assigned] (SPARK-25638) Convert structs to CSV strings

[ https://issues.apache.org/jira/browse/SPARK-25638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25638: Assignee: Apache Spark > Convert structs to CSV strings > --

[jira] [Created] (SPARK-25639) Add documentation on foreachBatch, and multiple watermark policy

Tathagata Das created SPARK-25639: - Summary: Add documentation on foreachBatch, and multiple watermark policy Key: SPARK-25639 URL: https://issues.apache.org/jira/browse/SPARK-25639 Project: Spark

[jira] [Assigned] (SPARK-25639) Add documentation on foreachBatch, and multiple watermark policy

[ https://issues.apache.org/jira/browse/SPARK-25639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25639: Assignee: Apache Spark > Add documentation on foreachBatch, and multiple watermark

[jira] [Assigned] (SPARK-25639) Add documentation on foreachBatch, and multiple watermark policy

[ https://issues.apache.org/jira/browse/SPARK-25639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25639: Assignee: (was: Apache Spark) > Add documentation on foreachBatch, and multiple

[jira] [Commented] (SPARK-25639) Add documentation on foreachBatch, and multiple watermark policy

[ https://issues.apache.org/jira/browse/SPARK-25639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638052#comment-16638052 ] Apache Spark commented on SPARK-25639: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25638) Convert structs to CSV strings

[ https://issues.apache.org/jira/browse/SPARK-25638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25638: Assignee: (was: Apache Spark) > Convert structs to CSV strings >

[jira] [Commented] (SPARK-25638) Convert structs to CSV strings

[ https://issues.apache.org/jira/browse/SPARK-25638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637993#comment-16637993 ] Apache Spark commented on SPARK-25638: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-25638) Convert structs to CSV strings

[ https://issues.apache.org/jira/browse/SPARK-25638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637994#comment-16637994 ] Apache Spark commented on SPARK-25638: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637963#comment-16637963 ] Liang-Chi Hsieh commented on SPARK-25587: - Just ran few experiments. It seems caused by

[jira] [Created] (SPARK-25638) Convert structs to CSV strings

Maxim Gekk created SPARK-25638: -- Summary: Convert structs to CSV strings Key: SPARK-25638 URL: https://issues.apache.org/jira/browse/SPARK-25638 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637963#comment-16637963 ] Liang-Chi Hsieh edited comment on SPARK-25587 at 10/4/18 9:21 AM: -- Just

[jira] [Commented] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637951#comment-16637951 ] Sean Owen commented on SPARK-25587: --- Yes, possibly another instance of case classes and the shell

[jira] [Commented] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637900#comment-16637900 ] Wenchen Fan commented on SPARK-25587: - I believe this is an issue of Spark Shell. Looking at the