[jira] [Resolved] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31318. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28082

[jira] [Assigned] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31318: --- Assignee: Maxim Gekk > Split Parquet/Avro configs for rebasing dates/timestamps in read

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072386#comment-17072386 ] Dongjoon Hyun commented on SPARK-31308: --- Ya. I fixed it. It was failed from my side at that time.

[jira] [Resolved] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31308. --- Fix Version/s: 3.1.0 Assignee: L. C. Hsieh Resolution: Fixed This is

[jira] [Resolved] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-31313. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28080

[jira] [Created] (SPARK-31320) fix release script for 3.0.0

2020-03-31 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-31320: --- Summary: fix release script for 3.0.0 Key: SPARK-31320 URL: https://issues.apache.org/jira/browse/SPARK-31320 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-31313: --- Assignee: Dongjoon Hyun > Add `m01` node name to support Minikube 1.8.x >

[jira] [Updated] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31312: Fix Version/s: 2.4.6 > Transforming Hive simple UDF (using JAR) expression may incur CNFE in

[jira] [Commented] (SPARK-31301) flatten the result dataframe of tests in stat

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072319#comment-17072319 ] zhengruifeng commented on SPARK-31301: -- {code:java} @Since("3.1.0") def testChiSquare( dataset:

[jira] [Resolved] (SPARK-31300) Migrate the implementation of algorithms from MLlib to ML

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-31300. -- Resolution: Not A Problem > Migrate the implementation of algorithms from MLlib to ML >

[jira] [Resolved] (SPARK-31309) Migrate the ChiSquareTest from MLlib to ML

2020-03-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-31309. -- Resolution: Not A Problem > Migrate the ChiSquareTest from MLlib to ML >

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072303#comment-17072303 ] L. C. Hsieh commented on SPARK-31308: - Not sure why this JIRA ticket is not closed automatically.

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072300#comment-17072300 ] L. C. Hsieh commented on SPARK-29358: - > users might rely on the exception when the schema is

[jira] [Resolved] (SPARK-31290) Add back the deprecated R APIs

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31290. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed in

[jira] [Comment Edited] (SPARK-31290) Add back the deprecated R APIs

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072299#comment-17072299 ] Hyukjin Kwon edited comment on SPARK-31290 at 4/1/20, 1:43 AM: --- Fixed in

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072294#comment-17072294 ] Hyukjin Kwon commented on SPARK-29358: -- I am good with adding this behaviour if other committers

[jira] [Reopened] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-29358: -- > Make unionByName optionally fill missing columns with nulls >

[jira] [Updated] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29358: - Affects Version/s: (was: 3.0.0) 3.1.0 > Make unionByName optionally

[jira] [Created] (SPARK-31319) Document UDF in SQL Reference

2020-03-31 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-31319: -- Summary: Document UDF in SQL Reference Key: SPARK-31319 URL: https://issues.apache.org/jira/browse/SPARK-31319 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-31248) Flaky Test: ExecutorAllocationManagerSuite.interleaving add and remove

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31248: - Assignee: wuyi > Flaky Test: ExecutorAllocationManagerSuite.interleaving add and

[jira] [Resolved] (SPARK-31248) Flaky Test: ExecutorAllocationManagerSuite.interleaving add and remove

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31248. --- Fix Version/s: 3.1.0 Resolution: Fixed This is resolved via

[jira] [Resolved] (SPARK-31305) Add a page to list all commands in SQL Reference

2020-03-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31305. -- Fix Version/s: 3.0.0 Assignee: Huaxin Gao Resolution: Fixed Resolved

[jira] [Resolved] (SPARK-17592) SQL: CAST string as INT inconsistent with Hive

2020-03-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-17592. - Resolution: Won't Fix > SQL: CAST string as INT inconsistent with Hive >

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072212#comment-17072212 ] fqaiser94 commented on SPARK-22231: --- Makes sense, thanks! > Support of map, filter, withColumn,

[jira] [Resolved] (SPARK-31304) Add ANOVATest example

2020-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31304. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28073

[jira] [Assigned] (SPARK-31304) Add ANOVATest example

2020-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31304: Assignee: kevin yu > Add ANOVATest example > - > >

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072142#comment-17072142 ] DB Tsai commented on SPARK-17636: - [~MasterDDT] we are not able to merge a new feature to an old

[jira] [Resolved] (SPARK-31253) add metrics to shuffle reader

2020-03-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-31253. - Fix Version/s: 3.1.0 Resolution: Fixed > add metrics to shuffle reader >

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

2020-03-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072119#comment-17072119 ] Mitesh commented on SPARK-17636: [~cloud_fan] [~dbtsai] thanks for fixing! Is there any plan to backport

[jira] [Resolved] (SPARK-31265) Add -XX:MaxDirectMemorySize jvm options in yarn mode

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31265. --- Resolution: Invalid > Add -XX:MaxDirectMemorySize jvm options in yarn mode >

[jira] [Updated] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31318: --- Parent: SPARK-30951 Issue Type: Sub-task (was: Improvement) > Split Parquet/Avro configs

[jira] [Created] (SPARK-31318) Split Parquet/Avro configs for rebasing dates/timestamps in read and in write

2020-03-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31318: -- Summary: Split Parquet/Avro configs for rebasing dates/timestamps in read and in write Key: SPARK-31318 URL: https://issues.apache.org/jira/browse/SPARK-31318 Project:

[jira] [Updated] (SPARK-30775) Improve DOC for executor metrics

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30775: -- Fix Version/s: 3.0.1 > Improve DOC for executor metrics > >

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071969#comment-17071969 ] DB Tsai commented on SPARK-22231: - [~fqaiser94] Thanks for continuing this work. We implemented this

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-03-31 Thread Michael Armbrust (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071968#comment-17071968 ] Michael Armbrust commented on SPARK-29358: -- I think we should reconsider closing this as won't

[jira] [Created] (SPARK-31317) Add withField method to Column class

2020-03-31 Thread DB Tsai (Jira)
DB Tsai created SPARK-31317: --- Summary: Add withField method to Column class Key: SPARK-31317 URL: https://issues.apache.org/jira/browse/SPARK-31317 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-22231: --- Assignee: (was: Jeremy Smith) > Support of map, filter, withColumn, dropColumn in nested list

[jira] [Commented] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

2020-03-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071953#comment-17071953 ] Thomas Graves commented on SPARK-30873: --- so I think this is a dup of SPARK-30835. > Handling Node

[jira] [Updated] (SPARK-31230) use statement plans in DataFrameWriter(V2)

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31230: Fix Version/s: (was: 3.0.0) 3.0.1 > use statement plans in

[jira] [Resolved] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31312. - Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution: Fixed > Transforming

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071945#comment-17071945 ] fqaiser94 commented on SPARK-22231: --- Excellent, thanks Reynold. If somebody could assign this ticket

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071936#comment-17071936 ] L. C. Hsieh commented on SPARK-31308: - Thanks [~dongjoon]. I changed the `Affected Version`. > Make

[jira] [Updated] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-31308: Affects Version/s: (was: 3.0.0) 3.1.0 > Make Python dependencies

[jira] [Resolved] (SPARK-31230) use statement plans in DataFrameWriter(V2)

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31230. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27992

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071928#comment-17071928 ] Dongjoon Hyun commented on SPARK-31308: --- Yes. For `New Feature` or `Improvement`, we should use

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071926#comment-17071926 ] Dongjoon Hyun commented on SPARK-25102: --- Thanks, [~cloud_fan]. > Write Spark version to

[jira] [Commented] (SPARK-31072) Default to ParquetOutputCommitter even after configuring s3a committer as "partitioned"

2020-03-31 Thread Felix Kizhakkel Jose (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071819#comment-17071819 ] Felix Kizhakkel Jose commented on SPARK-31072: -- Hello, Any updates or helps will be much

[jira] [Commented] (SPARK-31301) flatten the result dataframe of tests in stat

2020-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071807#comment-17071807 ] Sean R. Owen commented on SPARK-31301: -- Hm, I'm not sure I'd add a separate method for this either.

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071803#comment-17071803 ] Wenchen Fan commented on SPARK-25102: - ok nvm, I checked ORC and it doesn't have the "createdBy"

[jira] [Updated] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-31314: -- Issue Type: Bug (was: Improvement) > Revert SPARK-29285 to fix shuffle regression caused by

[jira] [Commented] (SPARK-31316) SQLQueryTestSuite: Display the total generate time for generated java code.

2020-03-31 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071711#comment-17071711 ] jiaan.geng commented on SPARK-31316: I'm working on. > SQLQueryTestSuite: Display the total

[jira] [Created] (SPARK-31316) SQLQueryTestSuite: Display the total generate time for generated java code.

2020-03-31 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-31316: -- Summary: SQLQueryTestSuite: Display the total generate time for generated java code. Key: SPARK-31316 URL: https://issues.apache.org/jira/browse/SPARK-31316 Project:

[jira] [Updated] (SPARK-29285) Temporary shuffle and local block should be able to handle disk failures

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-29285: Fix Version/s: (was: 3.0.0) > Temporary shuffle and local block should be able to handle disk

[jira] [Commented] (SPARK-29285) Temporary shuffle and local block should be able to handle disk failures

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071699#comment-17071699 ] Wenchen Fan commented on SPARK-29285: - This is reverted in

[jira] [Reopened] (SPARK-29285) Temporary shuffle and local block should be able to handle disk failures

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-29285: - Assignee: (was: Kent Yao) > Temporary shuffle and local block should be able to handle

[jira] [Resolved] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31314. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28072

[jira] [Assigned] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31314: --- Assignee: Yuanjian Li > Revert SPARK-29285 to fix shuffle regression caused by creating

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071692#comment-17071692 ] Wenchen Fan commented on SPARK-25102: - It's not completely orthogonal as we can merge these two.

[jira] [Created] (SPARK-31315) SQLQueryTestSuite: Display the total compile time for generated java code.

2020-03-31 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-31315: -- Summary: SQLQueryTestSuite: Display the total compile time for generated java code. Key: SPARK-31315 URL: https://issues.apache.org/jira/browse/SPARK-31315 Project:

[jira] [Assigned] (SPARK-31297) Speed-up date-time rebasing

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31297: --- Assignee: Maxim Gekk > Speed-up date-time rebasing > --- > >

[jira] [Resolved] (SPARK-31297) Speed-up date-time rebasing

2020-03-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31297. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28067

[jira] [Created] (SPARK-31314) Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

2020-03-31 Thread Yuanjian Li (Jira)
Yuanjian Li created SPARK-31314: --- Summary: Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly Key: SPARK-31314 URL: https://issues.apache.org/jira/browse/SPARK-31314

[jira] [Created] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-03-31 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-31313: - Summary: Add `m01` node name to support Minikube 1.8.x Key: SPARK-31313 URL: https://issues.apache.org/jira/browse/SPARK-31313 Project: Spark Issue Type:

[jira] [Created] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-03-31 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-31312: Summary: Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation Key: SPARK-31312 URL: https://issues.apache.org/jira/browse/SPARK-31312

[jira] [Reopened] (SPARK-30879) Refine doc-building workflow

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-30879: -- Assignee: (was: Nicholas Chammas) Reverted at

[jira] [Updated] (SPARK-30879) Refine doc-building workflow

2020-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30879: - Fix Version/s: (was: 3.1.0) > Refine doc-building workflow > >

[jira] [Updated] (SPARK-31311) Benchmark date-time rebasing in ORC datasource

2020-03-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31311: --- Description: * Benchmark saving dates/timestamps before and after 1582-10-15 * Benchmark loading

[jira] [Created] (SPARK-31311) Benchmark date-time rebasing in ORC datasource

2020-03-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31311: -- Summary: Benchmark date-time rebasing in ORC datasource Key: SPARK-31311 URL: https://issues.apache.org/jira/browse/SPARK-31311 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6761) Approximate quantile

2020-03-31 Thread Shyam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071490#comment-17071490 ] Shyam commented on SPARK-6761: -- [~mengxr] can you please advice on this bug... 

[jira] [Created] (SPARK-31310) percentile_approx function not working as expected

2020-03-31 Thread Shyam (Jira)
Shyam created SPARK-31310: - Summary: percentile_approx function not working as expected Key: SPARK-31310 URL: https://issues.apache.org/jira/browse/SPARK-31310 Project: Spark Issue Type: Bug