[jira] [Created] (SPARK-26461) Use ConfigEntry for hardcoded configs for dynamicAllocation category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26461: - Summary: Use ConfigEntry for hardcoded configs for dynamicAllocation category. Key: SPARK-26461 URL: https://issues.apache.org/jira/browse/SPARK-26461 Project: Spar

[jira] [Created] (SPARK-26468) Use ConfigEntry for hardcoded configs for task category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26468: - Summary: Use ConfigEntry for hardcoded configs for task category. Key: SPARK-26468 URL: https://issues.apache.org/jira/browse/SPARK-26468 Project: Spark Is

[jira] [Created] (SPARK-26463) Use ConfigEntry for hardcoded configs for scheduler category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26463: - Summary: Use ConfigEntry for hardcoded configs for scheduler category. Key: SPARK-26463 URL: https://issues.apache.org/jira/browse/SPARK-26463 Project: Spark

[jira] [Created] (SPARK-26478) Use ConfigEntry for hardcoded configs for rdd category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26478: - Summary: Use ConfigEntry for hardcoded configs for rdd category. Key: SPARK-26478 URL: https://issues.apache.org/jira/browse/SPARK-26478 Project: Spark Iss

[jira] [Created] (SPARK-26491) Use ConfigEntry for hardcoded configs for test category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26491: - Summary: Use ConfigEntry for hardcoded configs for test category. Key: SPARK-26491 URL: https://issues.apache.org/jira/browse/SPARK-26491 Project: Spark Is

[jira] [Created] (SPARK-26489) Use ConfigEntry for hardcoded configs for python category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26489: - Summary: Use ConfigEntry for hardcoded configs for python category. Key: SPARK-26489 URL: https://issues.apache.org/jira/browse/SPARK-26489 Project: Spark

[jira] [Created] (SPARK-26490) Use ConfigEntry for hardcoded configs for r category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26490: - Summary: Use ConfigEntry for hardcoded configs for r category. Key: SPARK-26490 URL: https://issues.apache.org/jira/browse/SPARK-26490 Project: Spark Issue

[jira] [Created] (SPARK-26488) Use ConfigEntry for hardcoded configs for modify.acl category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26488: - Summary: Use ConfigEntry for hardcoded configs for modify.acl category. Key: SPARK-26488 URL: https://issues.apache.org/jira/browse/SPARK-26488 Project: Spark

[jira] [Created] (SPARK-26483) Use ConfigEntry for hardcoded configs for ssl category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26483: - Summary: Use ConfigEntry for hardcoded configs for ssl category. Key: SPARK-26483 URL: https://issues.apache.org/jira/browse/SPARK-26483 Project: Spark Iss

[jira] [Created] (SPARK-26486) Use ConfigEntry for hardcoded configs for metrics category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26486: - Summary: Use ConfigEntry for hardcoded configs for metrics category. Key: SPARK-26486 URL: https://issues.apache.org/jira/browse/SPARK-26486 Project: Spark

[jira] [Created] (SPARK-26477) Use ConfigEntry for hardcoded configs for unsafe category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26477: - Summary: Use ConfigEntry for hardcoded configs for unsafe category. Key: SPARK-26477 URL: https://issues.apache.org/jira/browse/SPARK-26477 Project: Spark

[jira] [Created] (SPARK-26476) Use ConfigEntry for hardcoded configs for cleaner category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26476: - Summary: Use ConfigEntry for hardcoded configs for cleaner category. Key: SPARK-26476 URL: https://issues.apache.org/jira/browse/SPARK-26476 Project: Spark

[jira] [Created] (SPARK-26481) Use ConfigEntry for hardcoded configs for reducer category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26481: - Summary: Use ConfigEntry for hardcoded configs for reducer category. Key: SPARK-26481 URL: https://issues.apache.org/jira/browse/SPARK-26481 Project: Spark

[jira] [Created] (SPARK-26479) Use ConfigEntry for hardcoded configs for locality category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26479: - Summary: Use ConfigEntry for hardcoded configs for locality category. Key: SPARK-26479 URL: https://issues.apache.org/jira/browse/SPARK-26479 Project: Spark

[jira] [Created] (SPARK-26482) Use ConfigEntry for hardcoded configs for ui category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26482: - Summary: Use ConfigEntry for hardcoded configs for ui category. Key: SPARK-26482 URL: https://issues.apache.org/jira/browse/SPARK-26482 Project: Spark Issu

[jira] [Created] (SPARK-26485) Use ConfigEntry for hardcoded configs for master.rest/ui categories.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26485: - Summary: Use ConfigEntry for hardcoded configs for master.rest/ui categories. Key: SPARK-26485 URL: https://issues.apache.org/jira/browse/SPARK-26485 Project: Spark

[jira] [Created] (SPARK-26487) Use ConfigEntry for hardcoded configs for admin category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26487: - Summary: Use ConfigEntry for hardcoded configs for admin category. Key: SPARK-26487 URL: https://issues.apache.org/jira/browse/SPARK-26487 Project: Spark I

[jira] [Created] (SPARK-26480) Use ConfigEntry for hardcoded configs for broadcast category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26480: - Summary: Use ConfigEntry for hardcoded configs for broadcast category. Key: SPARK-26480 URL: https://issues.apache.org/jira/browse/SPARK-26480 Project: Spark

[jira] [Created] (SPARK-26484) Use ConfigEntry for hardcoded configs for authenticate category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26484: - Summary: Use ConfigEntry for hardcoded configs for authenticate category. Key: SPARK-26484 URL: https://issues.apache.org/jira/browse/SPARK-26484 Project: Spark

[jira] [Created] (SPARK-26469) Use ConfigEntry for hardcoded configs for io category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26469: - Summary: Use ConfigEntry for hardcoded configs for io category. Key: SPARK-26469 URL: https://issues.apache.org/jira/browse/SPARK-26469 Project: Spark Issu

[jira] [Created] (SPARK-26475) Use ConfigEntry for hardcoded configs for buffer category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26475: - Summary: Use ConfigEntry for hardcoded configs for buffer category. Key: SPARK-26475 URL: https://issues.apache.org/jira/browse/SPARK-26475 Project: Spark

[jira] [Created] (SPARK-26473) Use ConfigEntry for hardcoded configs for deploy category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26473: - Summary: Use ConfigEntry for hardcoded configs for deploy category. Key: SPARK-26473 URL: https://issues.apache.org/jira/browse/SPARK-26473 Project: Spark

[jira] [Created] (SPARK-26471) Use ConfigEntry for hardcoded configs for speculation category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26471: - Summary: Use ConfigEntry for hardcoded configs for speculation category. Key: SPARK-26471 URL: https://issues.apache.org/jira/browse/SPARK-26471 Project: Spark

[jira] [Created] (SPARK-26474) Use ConfigEntry for hardcoded configs for worker category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26474: - Summary: Use ConfigEntry for hardcoded configs for worker category. Key: SPARK-26474 URL: https://issues.apache.org/jira/browse/SPARK-26474 Project: Spark

[jira] [Created] (SPARK-26472) Use ConfigEntry for hardcoded configs for serializer category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26472: - Summary: Use ConfigEntry for hardcoded configs for serializer category. Key: SPARK-26472 URL: https://issues.apache.org/jira/browse/SPARK-26472 Project: Spark

[jira] [Created] (SPARK-26467) Use ConfigEntry for hardcoded configs for rpc category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26467: - Summary: Use ConfigEntry for hardcoded configs for rpc category. Key: SPARK-26467 URL: https://issues.apache.org/jira/browse/SPARK-26467 Project: Spark Iss

[jira] [Created] (SPARK-26462) Use ConfigEntry for hardcoded configs for memory category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26462: - Summary: Use ConfigEntry for hardcoded configs for memory category. Key: SPARK-26462 URL: https://issues.apache.org/jira/browse/SPARK-26462 Project: Spark

[jira] [Created] (SPARK-26466) Use ConfigEntry for hardcoded configs for submit category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26466: - Summary: Use ConfigEntry for hardcoded configs for submit category. Key: SPARK-26466 URL: https://issues.apache.org/jira/browse/SPARK-26466 Project: Spark

[jira] [Created] (SPARK-26465) Use ConfigEntry for hardcoded configs for jars category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26465: - Summary: Use ConfigEntry for hardcoded configs for jars category. Key: SPARK-26465 URL: https://issues.apache.org/jira/browse/SPARK-26465 Project: Spark Is

[jira] [Created] (SPARK-26464) Use ConfigEntry for hardcoded configs for storage category.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26464: - Summary: Use ConfigEntry for hardcoded configs for storage category. Key: SPARK-26464 URL: https://issues.apache.org/jira/browse/SPARK-26464 Project: Spark

[jira] [Created] (SPARK-26460) Use ConfigEntry for hardcoded kryo/kryoserializer configs.

2018-12-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26460: - Summary: Use ConfigEntry for hardcoded kryo/kryoserializer configs. Key: SPARK-26460 URL: https://issues.apache.org/jira/browse/SPARK-26460 Project: Spark

[jira] [Updated] (SPARK-26460) Use ConfigEntry for hardcoded configs for kryo/kryoserializer categories.

2018-12-27 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-26460: -- Summary: Use ConfigEntry for hardcoded configs for kryo/kryoserializer categories. (was: Use

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2018-12-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730084#comment-16730084 ] Saisai Shao commented on SPARK-20415: - Have you tried latest version of Spark, does

[jira] [Commented] (SPARK-26164) [SQL] Allow FileFormatWriter to write multiple partitions/buckets without sort

2018-12-27 Thread Cheng Su (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730029#comment-16730029 ] Cheng Su commented on SPARK-26164: -- [~cloud_fan], could you help take a look of [https

[jira] [Commented] (SPARK-26459) remove UpdateNullabilityInAttributeReferences

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730026#comment-16730026 ] Apache Spark commented on SPARK-26459: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-26459) remove UpdateNullabilityInAttributeReferences

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730027#comment-16730027 ] Apache Spark commented on SPARK-26459: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-26459) remove UpdateNullabilityInAttributeReferences

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26459: Assignee: Wenchen Fan (was: Apache Spark) > remove UpdateNullabilityInAttributeReference

[jira] [Assigned] (SPARK-26459) remove UpdateNullabilityInAttributeReferences

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26459: Assignee: Apache Spark (was: Wenchen Fan) > remove UpdateNullabilityInAttributeReference

[jira] [Created] (SPARK-26459) remove UpdateNullabilityInAttributeReferences

2018-12-27 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-26459: --- Summary: remove UpdateNullabilityInAttributeReferences Key: SPARK-26459 URL: https://issues.apache.org/jira/browse/SPARK-26459 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-26458) OneHotEncoderModel verifies the number of category values incorrectly when tries to transform a dataframe.

2018-12-27 Thread duruihuan (JIRA)
duruihuan created SPARK-26458: - Summary: OneHotEncoderModel verifies the number of category values incorrectly when tries to transform a dataframe. Key: SPARK-26458 URL: https://issues.apache.org/jira/browse/SPARK-264

[jira] [Updated] (SPARK-26446) Add cachedExecutorIdleTimeout docs at ExecutorAllocationManager

2018-12-27 Thread Qingxin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingxin Wu updated SPARK-26446: --- Description: Add docs to describe how remove policy act while considering the property  _*{{spark.dy

[jira] [Updated] (SPARK-26446) Add cachedExecutorIdleTimeout docs at ExecutorAllocationManager

2018-12-27 Thread Qingxin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingxin Wu updated SPARK-26446: --- Description:   Add docs to describe how remove policy act while considering the property {code:java

[jira] [Commented] (SPARK-22579) BlockManager.getRemoteValues and BlockManager.getRemoteBytes should be implemented using streaming

2018-12-27 Thread Andrey Siunov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730007#comment-16730007 ] Andrey Siunov commented on SPARK-22579: --- Is it true that the issue has been resolv

[jira] [Updated] (SPARK-26446) Add cachedExecutorIdleTimeout docs at ExecutorAllocationManager

2018-12-27 Thread Qingxin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingxin Wu updated SPARK-26446: --- Description: Add docs to describe how remove policy act while considering the property  {{spark.dyna

[jira] [Updated] (SPARK-26446) Add cachedExecutorIdleTimeout docs at ExecutorAllocationManager

2018-12-27 Thread Qingxin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingxin Wu updated SPARK-26446: --- Summary: Add cachedExecutorIdleTimeout docs at ExecutorAllocationManager (was: improve doc on Execu

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

2018-12-27 Thread Jackey Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730003#comment-16730003 ] Jackey Lee commented on SPARK-24630: [~jackylk] I have finished a detailed design d

[jira] [Updated] (SPARK-26437) Decimal data becomes bigint to query, unable to query

2018-12-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26437: -- Affects Version/s: 1.6.3 > Decimal data becomes bigint to query, unable to query > ---

[jira] [Resolved] (SPARK-26437) Decimal data becomes bigint to query, unable to query

2018-12-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26437. --- Resolution: Fixed Fix Version/s: 3.0.0 > Decimal data becomes bigint to query, unable

[jira] [Commented] (SPARK-26437) Decimal data becomes bigint to query, unable to query

2018-12-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729945#comment-16729945 ] Dongjoon Hyun commented on SPARK-26437: --- Hi, [~zengxl]. Thank you for reporting.

[jira] [Updated] (SPARK-26437) Decimal data becomes bigint to query, unable to query

2018-12-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26437: -- Affects Version/s: 2.0.2 2.1.3 2.2.2 > Decimal d

[jira] [Comment Edited] (SPARK-17572) Write.df is failing on spark cluster

2018-12-27 Thread Tarun Parmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729921#comment-16729921 ] Tarun Parmar edited comment on SPARK-17572 at 12/27/18 10:51 PM: -

[jira] [Commented] (SPARK-26437) Decimal data becomes bigint to query, unable to query

2018-12-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729925#comment-16729925 ] Dongjoon Hyun commented on SPARK-26437: --- Thanks, [~mgaido]. > Decimal data become

[jira] [Commented] (SPARK-17572) Write.df is failing on spark cluster

2018-12-27 Thread Tarun Parmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729921#comment-16729921 ] Tarun Parmar commented on SPARK-17572: -- I am facing similar issue, my Spark+Hadoop

[jira] [Assigned] (SPARK-26450) Map of schema is built too frequently in some wide queries

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26450: Assignee: Apache Spark > Map of schema is built too frequently in some wide queries > ---

[jira] [Assigned] (SPARK-26450) Map of schema is built too frequently in some wide queries

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26450: Assignee: (was: Apache Spark) > Map of schema is built too frequently in some wide qu

[jira] [Updated] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-12-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26021: -- Fix Version/s: (was: 2.4.1) > -0.0 and 0.0 not treated consistently, doesn't match Hive >

[jira] [Commented] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-12-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729804#comment-16729804 ] Dongjoon Hyun commented on SPARK-26021: --- This is reverted from `branch-2.4` via h

[jira] [Updated] (SPARK-26363) Avoid duplicated KV store lookups for task table

2018-12-27 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-26363: --- Description: In the method `taskList`(Since https://github.com/apache/spark/pull/21688), t

[jira] [Created] (SPARK-26457) Show hadoop configurations in HistoryServer environment tab

2018-12-27 Thread deshanxiao (JIRA)
deshanxiao created SPARK-26457: -- Summary: Show hadoop configurations in HistoryServer environment tab Key: SPARK-26457 URL: https://issues.apache.org/jira/browse/SPARK-26457 Project: Spark Issu

[jira] [Updated] (SPARK-26451) Change lead/lag argument name from count to offset

2018-12-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26451: -- Docs Text: The 'lag' function in Pyspark accepted an argument 'count' which should have been called '

[jira] [Resolved] (SPARK-26451) Change lead/lag argument name from count to offset

2018-12-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26451. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23357 [https://gi

[jira] [Assigned] (SPARK-26451) Change lead/lag argument name from count to offset

2018-12-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26451: Assignee: Deepyaman Datta > Change lead/lag argument name from count to offset >

[jira] [Commented] (SPARK-26450) Map of schema is built too frequently in some wide queries

2018-12-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729675#comment-16729675 ] Marco Gaido commented on SPARK-26450: - Great, thanks! > Map of schema is built too

[jira] [Assigned] (SPARK-26456) Cast date/timestamp by Date/TimestampFormatter

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26456: Assignee: (was: Apache Spark) > Cast date/timestamp by Date/TimestampFormatter >

[jira] [Assigned] (SPARK-26456) Cast date/timestamp by Date/TimestampFormatter

2018-12-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26456: Assignee: Apache Spark > Cast date/timestamp by Date/TimestampFormatter > ---

[jira] [Created] (SPARK-26456) Cast date/timestamp by Date/TimestampFormatter

2018-12-27 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26456: -- Summary: Cast date/timestamp by Date/TimestampFormatter Key: SPARK-26456 URL: https://issues.apache.org/jira/browse/SPARK-26456 Project: Spark Issue Type: Improv

[jira] [Resolved] (SPARK-26248) Infer date type from CSV

2018-12-27 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-26248. Resolution: Won't Fix > Infer date type from CSV > > > Ke

[jira] [Commented] (SPARK-26450) Map of schema is built too frequently in some wide queries

2018-12-27 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729647#comment-16729647 ] Bruce Robbins commented on SPARK-26450: --- I can attempt a patch later today. > Map

[jira] [Assigned] (SPARK-25892) AttributeReference.withMetadata method should have return type AttributeReference

2018-12-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25892: Assignee: kevin yu > AttributeReference.withMetadata method should have return type > At

[jira] [Resolved] (SPARK-25892) AttributeReference.withMetadata method should have return type AttributeReference

2018-12-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25892. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22918 [https://gi

[jira] [Created] (SPARK-26455) Spark Kinesis Integration with no SSL

2018-12-27 Thread Shashikant Bangera (JIRA)
Shashikant Bangera created SPARK-26455: -- Summary: Spark Kinesis Integration with no SSL Key: SPARK-26455 URL: https://issues.apache.org/jira/browse/SPARK-26455 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26454) IllegegalArgument Exception is Thrown while creating new UDF with JAR

2018-12-27 Thread Udbhav Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729628#comment-16729628 ] Udbhav Agrawal commented on SPARK-26454: [~dongjoon] [~cloud_fan] Creating the

[jira] [Created] (SPARK-26454) IllegegalArgument Exception is Thrown while creating new UDF with JAR

2018-12-27 Thread Udbhav Agrawal (JIRA)
Udbhav Agrawal created SPARK-26454: -- Summary: IllegegalArgument Exception is Thrown while creating new UDF with JAR Key: SPARK-26454 URL: https://issues.apache.org/jira/browse/SPARK-26454 Project: Sp

[jira] [Commented] (SPARK-26450) Map of schema is built too frequently in some wide queries

2018-12-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729621#comment-16729621 ] Marco Gaido commented on SPARK-26450: - Thanks for this JIRA [~bersprockets]. This ma

[jira] [Resolved] (SPARK-25910) accumulator updates from previous stage attempt should not fail

2018-12-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25910. -- Resolution: Duplicate > accumulator updates from previous stage attempt should not fail >

[jira] [Commented] (SPARK-26437) Decimal data becomes bigint to query, unable to query

2018-12-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729526#comment-16729526 ] Marco Gaido commented on SPARK-26437: - cc [~dongjoon] > Decimal data becomes bigint

[jira] [Resolved] (SPARK-26453) running spark sql cli is looking for wrong path of hive.metastore.warehouse.dir

2018-12-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26453. -- Resolution: Invalid Looks a question. Questions should go to mailing list before filing it as

[jira] [Updated] (SPARK-26453) running spark sql cli is looking for wrong path of hive.metastore.warehouse.dir

2018-12-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26453: - Target Version/s: (was: 2.4.0) > running spark sql cli is looking for wrong path of > hive.m

[jira] [Resolved] (SPARK-26191) Control number of truncated fields

2018-12-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-26191. --- Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 3.0.0 > Contro

[jira] [Comment Edited] (SPARK-25299) Use remote storage for persisting shuffle data

2018-12-27 Thread Peiyu Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729474#comment-16729474 ] Peiyu Zhuang edited comment on SPARK-25299 at 12/27/18 9:32 AM: --

[jira] [Created] (SPARK-26453) running spark sql cli is looking for wrong path of hive.metastore.warehouse.dir

2018-12-27 Thread anubhav tarar (JIRA)
anubhav tarar created SPARK-26453: - Summary: running spark sql cli is looking for wrong path of hive.metastore.warehouse.dir Key: SPARK-26453 URL: https://issues.apache.org/jira/browse/SPARK-26453 Pr

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-12-27 Thread Peiyu Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729474#comment-16729474 ] Peiyu Zhuang commented on SPARK-25299: -- Sure.  I just create a [SPIP in google doc

[jira] [Assigned] (SPARK-26435) Support creating partitioned table using Hive CTAS by specifying partition column names

2018-12-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26435: --- Assignee: Liang-Chi Hsieh > Support creating partitioned table using Hive CTAS by specifyin

[jira] [Resolved] (SPARK-26435) Support creating partitioned table using Hive CTAS by specifying partition column names

2018-12-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26435. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23376 [https://gith