[jira] [Commented] (SPARK-31301) flatten the result dataframe of tests in stat

2020-03-31 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072428#comment-17072428 ] Huaxin Gao commented on SPARK-31301: It makes sense to flatten the output. Since the

[jira] [Commented] (SPARK-31301) flatten the result dataframe of tests in stat

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072408#comment-17072408 ] zhengruifeng commented on SPARK-31301: -- Another advantage that returning rows is th

[jira] [Reopened] (SPARK-31309) Migrate the ChiSquareTest from MLlib to ML

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reopened SPARK-31309: -- > Migrate the ChiSquareTest from MLlib to ML > -- > >

[jira] [Resolved] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31318. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28082 [https://gith

[jira] [Assigned] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31318: --- Assignee: Maxim Gekk > Split Parquet/Avro configs for rebasing dates/timestamps in read and

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072386#comment-17072386 ] Dongjoon Hyun commented on SPARK-31308: --- Ya. I fixed it. It was failed from my sid

[jira] [Resolved] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31308. --- Fix Version/s: 3.1.0 Assignee: L. C. Hsieh Resolution: Fixed This is resolve

[jira] [Resolved] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-31313. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28080 [https://github.com/a

[jira] [Created] (SPARK-31320) fix release script for 3.0.0

2020-03-31 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-31320: --- Summary: fix release script for 3.0.0 Key: SPARK-31320 URL: https://issues.apache.org/jira/browse/SPARK-31320 Project: Spark Issue Type: Bug Componen

[jira] [Assigned] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-31313: --- Assignee: Dongjoon Hyun > Add `m01` node name to support Minikube 1.8.x > -

[jira] [Updated] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31312: Fix Version/s: 2.4.6 > Transforming Hive simple UDF (using JAR) expression may incur CNFE in later

[jira] [Commented] (SPARK-31301) flatten the result dataframe of tests in stat

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072319#comment-17072319 ] zhengruifeng commented on SPARK-31301: -- {code:java} @Since("3.1.0") def testChiSqua

[jira] [Resolved] (SPARK-31300) Migrate the implementation of algorithms from MLlib to ML

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-31300. -- Resolution: Not A Problem > Migrate the implementation of algorithms from MLlib to ML > --

[jira] [Resolved] (SPARK-31309) Migrate the ChiSquareTest from MLlib to ML

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-31309. -- Resolution: Not A Problem > Migrate the ChiSquareTest from MLlib to ML > -

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072303#comment-17072303 ] L. C. Hsieh commented on SPARK-31308: - Not sure why this JIRA ticket is not closed a

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072300#comment-17072300 ] L. C. Hsieh commented on SPARK-29358: - > users might rely on the exception when the

[jira] [Comment Edited] (SPARK-31290) Add back the deprecated R APIs

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072299#comment-17072299 ] Hyukjin Kwon edited comment on SPARK-31290 at 4/1/20, 1:43 AM: ---

[jira] [Resolved] (SPARK-31290) Add back the deprecated R APIs

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31290. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed in https://github.com/apache/spark/commit

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072294#comment-17072294 ] Hyukjin Kwon commented on SPARK-29358: -- I am good with adding this behaviour if oth

[jira] [Reopened] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-29358: -- > Make unionByName optionally fill missing columns with nulls > --

[jira] [Updated] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29358: - Affects Version/s: (was: 3.0.0) 3.1.0 > Make unionByName optionally f

[jira] [Created] (SPARK-31319) Document UDF in SQL Reference

2020-03-31 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-31319: -- Summary: Document UDF in SQL Reference Key: SPARK-31319 URL: https://issues.apache.org/jira/browse/SPARK-31319 Project: Spark Issue Type: Sub-task Comp

[jira] [Assigned] (SPARK-31248) Flaky Test: ExecutorAllocationManagerSuite.interleaving add and remove

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31248: - Assignee: wuyi > Flaky Test: ExecutorAllocationManagerSuite.interleaving add and remove

[jira] [Resolved] (SPARK-31248) Flaky Test: ExecutorAllocationManagerSuite.interleaving add and remove

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31248. --- Fix Version/s: 3.1.0 Resolution: Fixed This is resolved via https://github.com/apache

[jira] [Resolved] (SPARK-31305) Add a page to list all commands in SQL Reference

2020-03-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31305. -- Fix Version/s: 3.0.0 Assignee: Huaxin Gao Resolution: Fixed Resolved b

[jira] [Resolved] (SPARK-17592) SQL: CAST string as INT inconsistent with Hive

2020-03-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-17592. - Resolution: Won't Fix > SQL: CAST string as INT inconsistent with Hive > ---

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072212#comment-17072212 ] fqaiser94 commented on SPARK-22231: --- Makes sense, thanks! > Support of map, filter, w

[jira] [Resolved] (SPARK-31304) Add ANOVATest example

2020-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31304. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28073 [https://gi

[jira] [Assigned] (SPARK-31304) Add ANOVATest example

2020-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31304: Assignee: kevin yu > Add ANOVATest example > - > > Ke

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072142#comment-17072142 ] DB Tsai commented on SPARK-17636: - [~MasterDDT] we are not able to merge a new feature t

[jira] [Resolved] (SPARK-31253) add metrics to shuffle reader

2020-03-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-31253. - Fix Version/s: 3.1.0 Resolution: Fixed > add metrics to shuffle reader >

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

2020-03-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072119#comment-17072119 ] Mitesh commented on SPARK-17636: [~cloud_fan] [~dbtsai] thanks for fixing! Is there any

[jira] [Resolved] (SPARK-31265) Add -XX:MaxDirectMemorySize jvm options in yarn mode

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31265. --- Resolution: Invalid > Add -XX:MaxDirectMemorySize jvm options in yarn mode > ---

[jira] [Updated] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31318: --- Parent: SPARK-30951 Issue Type: Sub-task (was: Improvement) > Split Parquet/Avro configs fo

[jira] [Created] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31318: -- Summary: Split Parquet/Avro configs for rebasing dates/timestamps in read and in write Key: SPARK-31318 URL: https://issues.apache.org/jira/browse/SPARK-31318 Project: Sp

[jira] [Updated] (SPARK-30775) Improve DOC for executor metrics

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30775: -- Fix Version/s: 3.0.1 > Improve DOC for executor metrics > > >

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071969#comment-17071969 ] DB Tsai commented on SPARK-22231: - [~fqaiser94] Thanks for continuing this work. We impl

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Michael Armbrust (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071968#comment-17071968 ] Michael Armbrust commented on SPARK-29358: -- I think we should reconsider closin

[jira] [Created] (SPARK-31317) Add withField method to Column class

2020-03-31 Thread DB Tsai (Jira)
DB Tsai created SPARK-31317: --- Summary: Add withField method to Column class Key: SPARK-31317 URL: https://issues.apache.org/jira/browse/SPARK-31317 Project: Spark Issue Type: Sub-task Com

[jira] [Assigned] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-22231: --- Assignee: (was: Jeremy Smith) > Support of map, filter, withColumn, dropColumn in nested list o

[jira] [Commented] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

2020-03-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071953#comment-17071953 ] Thomas Graves commented on SPARK-30873: --- so I think this is a dup of SPARK-30835.

[jira] [Updated] (SPARK-31230) use statement plans in DataFrameWriter(V2)

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31230: Fix Version/s: (was: 3.0.0) 3.0.1 > use statement plans in DataFrameWriter(

[jira] [Resolved] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31312. - Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution: Fixed > Transforming Hiv

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071945#comment-17071945 ] fqaiser94 commented on SPARK-22231: --- Excellent, thanks Reynold. If somebody could ass

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071936#comment-17071936 ] L. C. Hsieh commented on SPARK-31308: - Thanks [~dongjoon]. I changed the `Affected V

[jira] [Updated] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-31308: Affects Version/s: (was: 3.0.0) 3.1.0 > Make Python dependencies availa

[jira] [Resolved] (SPARK-31230) use statement plans in DataFrameWriter(V2)

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31230. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27992 [https://gith

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071928#comment-17071928 ] Dongjoon Hyun commented on SPARK-31308: --- Yes. For `New Feature` or `Improvement`,

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071926#comment-17071926 ] Dongjoon Hyun commented on SPARK-25102: --- Thanks, [~cloud_fan]. > Write Spark vers

[jira] [Commented] (SPARK-31072) Default to ParquetOutputCommitter even after configuring s3a committer as "partitioned"

2020-03-31 Thread Felix Kizhakkel Jose (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071819#comment-17071819 ] Felix Kizhakkel Jose commented on SPARK-31072: -- Hello, Any updates or helps

[jira] [Commented] (SPARK-31301) flatten the result dataframe of tests in stat

2020-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071807#comment-17071807 ] Sean R. Owen commented on SPARK-31301: -- Hm, I'm not sure I'd add a separate method

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071803#comment-17071803 ] Wenchen Fan commented on SPARK-25102: - ok nvm, I checked ORC and it doesn't have the

[jira] [Updated] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-31314: -- Issue Type: Bug (was: Improvement) > Revert SPARK-29285 to fix shuffle regression caused by c

[jira] [Commented] (SPARK-31316) SQLQueryTestSuite: Display the total generate time for generated java code.

2020-03-31 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071711#comment-17071711 ] jiaan.geng commented on SPARK-31316: I'm working on. > SQLQueryTestSuite: Display t

[jira] [Created] (SPARK-31316) SQLQueryTestSuite: Display the total generate time for generated java code.

2020-03-31 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-31316: -- Summary: SQLQueryTestSuite: Display the total generate time for generated java code. Key: SPARK-31316 URL: https://issues.apache.org/jira/browse/SPARK-31316 Project: Spar

[jira] [Updated] (SPARK-29285) Temporary shuffle and local block should be able to handle disk failures

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-29285: Fix Version/s: (was: 3.0.0) > Temporary shuffle and local block should be able to handle disk

[jira] [Commented] (SPARK-29285) Temporary shuffle and local block should be able to handle disk failures

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071699#comment-17071699 ] Wenchen Fan commented on SPARK-29285: - This is reverted in https://github.com/apache

[jira] [Reopened] (SPARK-29285) Temporary shuffle and local block should be able to handle disk failures

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-29285: - Assignee: (was: Kent Yao) > Temporary shuffle and local block should be able to handle dis

[jira] [Resolved] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31314. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28072 [https://gith

[jira] [Assigned] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31314: --- Assignee: Yuanjian Li > Revert SPARK-29285 to fix shuffle regression caused by creating tem

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071692#comment-17071692 ] Wenchen Fan commented on SPARK-25102: - It's not completely orthogonal as we can merg

[jira] [Created] (SPARK-31315) SQLQueryTestSuite: Display the total compile time for generated java code.

2020-03-31 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-31315: -- Summary: SQLQueryTestSuite: Display the total compile time for generated java code. Key: SPARK-31315 URL: https://issues.apache.org/jira/browse/SPARK-31315 Project: Spark

[jira] [Assigned] (SPARK-31297) Speed-up date-time rebasing

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31297: --- Assignee: Maxim Gekk > Speed-up date-time rebasing > --- > >

[jira] [Resolved] (SPARK-31297) Speed-up date-time rebasing

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31297. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28067 [https://gith

[jira] [Created] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Yuanjian Li (Jira)
Yuanjian Li created SPARK-31314: --- Summary: Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly Key: SPARK-31314 URL: https://issues.apache.org/jira/browse/SPARK-31314

[jira] [Created] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-03-31 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-31313: - Summary: Add `m01` node name to support Minikube 1.8.x Key: SPARK-31313 URL: https://issues.apache.org/jira/browse/SPARK-31313 Project: Spark Issue Type: I

[jira] [Created] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-03-31 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-31312: Summary: Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation Key: SPARK-31312 URL: https://issues.apache.org/jira/browse/SPARK-31312

[jira] [Reopened] (SPARK-30879) Refine doc-building workflow

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-30879: -- Assignee: (was: Nicholas Chammas) Reverted at https://github.com/apache/spark/commit/4d

[jira] [Updated] (SPARK-30879) Refine doc-building workflow

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30879: - Fix Version/s: (was: 3.1.0) > Refine doc-building workflow > >