[jira] [Resolved] (SPARK-28730) Configurable type coercion policy for table insertion

2019-08-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28730. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25453

[jira] [Assigned] (SPARK-28730) Configurable type coercion policy for table insertion

2019-08-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28730: --- Assignee: Gengliang Wang > Configurable type coercion policy for table insertion >

[jira] [Created] (SPARK-28857) Clean up the comments of PR template during merging

2019-08-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-28857: - Summary: Clean up the comments of PR template during merging Key: SPARK-28857 URL: https://issues.apache.org/jira/browse/SPARK-28857 Project: Spark Issue

[jira] [Assigned] (SPARK-28702) Display useful error message (instead of NPE) for invalid Dataset operations (e.g. calling actions inside of transformations)

2019-08-22 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-28702: -- Assignee: Shivu Sondur > Display useful error message (instead of NPE) for invalid Dataset

[jira] [Resolved] (SPARK-28702) Display useful error message (instead of NPE) for invalid Dataset operations (e.g. calling actions inside of transformations)

2019-08-22 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-28702. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25503

[jira] [Resolved] (SPARK-28832) Document SHOW SCHEMAS statement in SQL Reference.

2019-08-22 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jobit mathew resolved SPARK-28832. -- Resolution: Duplicate Closing the JIRA as it will cover as a part of SHOW DATABASES statement

[jira] [Commented] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

2019-08-22 Thread Liang-Chi Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913915#comment-16913915 ] Liang-Chi Hsieh commented on SPARK-23519: - Thanks for pinging me. I am going on a flight soon.

[jira] [Commented] (SPARK-28827) Document SELECT CURRENT_DATABASE in SQL Reference

2019-08-22 Thread Shivu Sondur (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913903#comment-16913903 ] Shivu Sondur commented on SPARK-28827: -- i will work on this > Document SELECT CURRENT_DATABASE in

[jira] [Commented] (SPARK-28823) Document CREATE ROLE Statement

2019-08-22 Thread Shivu Sondur (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913902#comment-16913902 ] Shivu Sondur commented on SPARK-28823: -- i will work on this > Document CREATE ROLE Statement >

[jira] [Commented] (SPARK-28482) Data incomplete when using pandas udf in Python 3

2019-08-22 Thread jiangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913879#comment-16913879 ] jiangyu commented on SPARK-28482: - hi, [~bryanc] , i have tested toPandas(), it is okay. Row numbers is

[jira] [Commented] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

2019-08-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913855#comment-16913855 ] Wenchen Fan commented on SPARK-23519: - I think this is a bug and should be fixed. cc [~viirya] do

[jira] [Commented] (SPARK-27594) spark.sql.orc.enableVectorizedReader causes milliseconds in Timestamp to be read incorrectly

2019-08-22 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913665#comment-16913665 ] Owen O'Malley commented on SPARK-27594: --- This is being caused by an ORC bug that was backported in

[jira] [Assigned] (SPARK-28769) Improve warning message in Barrier Execution Mode in case required slots > maximum slots

2019-08-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28769: - Assignee: Kousuke Saruta > Improve warning message in Barrier Execution Mode in case required

[jira] [Resolved] (SPARK-28769) Improve warning message in Barrier Execution Mode in case required slots > maximum slots

2019-08-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28769. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25487

[jira] [Commented] (SPARK-28482) Data incomplete when using pandas udf in Python 3

2019-08-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913611#comment-16913611 ] Bryan Cutler commented on SPARK-28482: -- I'm not really sure what you are doing above, are you

[jira] [Commented] (SPARK-28832) Document SHOW SCHEMAS statement in SQL Reference.

2019-08-22 Thread Dilip Biswal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913560#comment-16913560 ] Dilip Biswal commented on SPARK-28832: -- [~jobitmathew] Thanks .. Yeah it will be documented as part

[jira] [Closed] (SPARK-28846) Set OMP_NUM_THREADS to executor cores for python

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-28846. - > Set OMP_NUM_THREADS to executor cores for python >

[jira] [Resolved] (SPARK-28846) Set OMP_NUM_THREADS to executor cores for python

2019-08-22 Thread Ryan Blue (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Blue resolved SPARK-28846. --- Resolution: Duplicate > Set OMP_NUM_THREADS to executor cores for python >

[jira] [Commented] (SPARK-28856) DataSourceV2: Support SHOW DATABASES

2019-08-22 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913426#comment-16913426 ] Terry Kim commented on SPARK-28856: --- I will work on this. > DataSourceV2: Support SHOW DATABASES >

[jira] [Created] (SPARK-28856) DataSourceV2: Support SHOW DATABASES

2019-08-22 Thread Terry Kim (Jira)
Terry Kim created SPARK-28856: - Summary: DataSourceV2: Support SHOW DATABASES Key: SPARK-28856 URL: https://issues.apache.org/jira/browse/SPARK-28856 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-28577) Ensure executorMemoryHead requested value not less than MEMORY_OFFHEAP_SIZE when MEMORY_OFFHEAP_ENABLED is true

2019-08-22 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-28577: -- Docs Text: On YARN, The off heap memory size is now separately included for the user in the

[jira] [Updated] (SPARK-28577) Ensure executorMemoryHead requested value not less than MEMORY_OFFHEAP_SIZE when MEMORY_OFFHEAP_ENABLED is true

2019-08-22 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-28577: -- Labels: release-notes (was: ) > Ensure executorMemoryHead requested value not less than

[jira] [Updated] (SPARK-28854) Zipping iterators in mapPartitions will fail

2019-08-22 Thread Hao Yang Ang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Yang Ang updated SPARK-28854: - Description: scala> sc.parallelize(Seq(1, 2, 3)).mapPartitions(xs =>

[jira] [Resolved] (SPARK-13677) Support Tree-Based Feature Transformation for ML

2019-08-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13677. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25383

[jira] [Assigned] (SPARK-13677) Support Tree-Based Feature Transformation for ML

2019-08-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-13677: - Assignee: zhengruifeng > Support Tree-Based Feature Transformation for ML >

[jira] [Resolved] (SPARK-28776) SparkML MLWriter gets hadoop conf from spark context instead of session

2019-08-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28776. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25505

[jira] [Assigned] (SPARK-28776) SparkML MLWriter gets hadoop conf from spark context instead of session

2019-08-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28776: - Assignee: Helen Yu > SparkML MLWriter gets hadoop conf from spark context instead of session >

[jira] [Created] (SPARK-28855) Remove outdated Experimental, Evolving annotations

2019-08-22 Thread Sean Owen (Jira)
Sean Owen created SPARK-28855: - Summary: Remove outdated Experimental, Evolving annotations Key: SPARK-28855 URL: https://issues.apache.org/jira/browse/SPARK-28855 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-28854) Zipping iterators in mapPartitions will fail

2019-08-22 Thread Hao Yang Ang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Yang Ang updated SPARK-28854: - Description: scala> sc.parallelize(Seq(1, 2, 3)).mapPartitions(xs =>

[jira] [Created] (SPARK-28854) Zipping iterators in mapPartitions will fail

2019-08-22 Thread Hao Yang Ang (Jira)
Hao Yang Ang created SPARK-28854: Summary: Zipping iterators in mapPartitions will fail Key: SPARK-28854 URL: https://issues.apache.org/jira/browse/SPARK-28854 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-28512) New optional mode: throw runtime exceptions on casting failures

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28512: --- Description: In popular DBMS like MySQL/PostgreSQL/Oracle, runtime exceptions are thrown on

[jira] [Created] (SPARK-28853) Support conf to organize filePartitions by file path

2019-08-22 Thread ZhangYao (Jira)
ZhangYao created SPARK-28853: Summary: Support conf to organize filePartitions by file path Key: SPARK-28853 URL: https://issues.apache.org/jira/browse/SPARK-28853 Project: Spark Issue Type:

[jira] [Updated] (SPARK-28741) New optional mode: Throw exceptions when casting to integers causes overflow

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28741: --- Summary: New optional mode: Throw exceptions when casting to integers causes overflow

[jira] [Resolved] (SPARK-28503) Return null result on cast an out-of-range value to a integral type

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-28503. Resolution: Won't Fix After consideration, I decide to close this one and open

[jira] [Updated] (SPARK-28741) Throw exceptions when casting to integers causes overflow

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28741: --- Parent: SPARK-28589 Issue Type: Sub-task (was: New Feature) > Throw exceptions

[jira] [Updated] (SPARK-28741) Throw exceptions when casting to integers causes overflow

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28741: --- Parent: (was: SPARK-26217) Issue Type: New Feature (was: Sub-task) > Throw

[jira] [Created] (SPARK-28852) Implement GetCatalogsOperation

2019-08-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-28852: --- Summary: Implement GetCatalogsOperation Key: SPARK-28852 URL: https://issues.apache.org/jira/browse/SPARK-28852 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-28852) Implement GetCatalogsOperation for Thrift Server

2019-08-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28852: Summary: Implement GetCatalogsOperation for Thrift Server (was: Implement GetCatalogsOperation)

[jira] [Resolved] (SPARK-28848) insert overwrite local directory stored as parquet does not creates snappy.parquet data file at local directory path

2019-08-22 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S resolved SPARK-28848. - Resolution: Duplicate Will be fixed as part of SPARK-28659 > insert overwrite local directory stored

[jira] [Updated] (SPARK-28495) Introduce ANSI store assignment policy for table insertion

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28495: --- Description: In Spark version 2.4 and earlier, when inserting into a table, Spark will cast

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling

[jira] [Commented] (SPARK-28797) Document DROP FUNCTION statement in SQL Reference.

2019-08-22 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913219#comment-16913219 ] Sandeep Katta commented on SPARK-28797: --- PR is created https://github.com/apache/spark/pull/25553

[jira] [Comment Edited] (SPARK-28482) Data incomplete when using pandas udf in Python 3

2019-08-22 Thread jiangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913183#comment-16913183 ] jiangyu edited comment on SPARK-28482 at 8/22/19 9:37 AM: -- hi, [~bryanc] ,

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling

[jira] [Commented] (SPARK-28482) Data incomplete when using pandas udf in Python 3

2019-08-22 Thread jiangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913183#comment-16913183 ] jiangyu commented on SPARK-28482: - hi, [~bryanc] , maybe you should produce more data, like 100,000

[jira] [Updated] (SPARK-28495) Introduce ANSI store assignment policy for table insertion

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28495: --- Summary: Introduce ANSI store assignment policy for table insertion (was: Follow ANSI SQL

[jira] [Comment Edited] (SPARK-22876) spark.yarn.am.attemptFailuresValidityInterval does not work correctly

2019-08-22 Thread Nikita Gorbachevski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913181#comment-16913181 ] Nikita Gorbachevski edited comment on SPARK-22876 at 8/22/19 9:22 AM:

[jira] [Commented] (SPARK-22876) spark.yarn.am.attemptFailuresValidityInterval does not work correctly

2019-08-22 Thread Nikita Gorbachevski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913181#comment-16913181 ] Nikita Gorbachevski commented on SPARK-22876: - Hi [~praveentallapudi], these options still

[jira] [Updated] (SPARK-28730) Configurable type coercion policy for table insertion

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28730: --- Description: After all the discussions in the dev list:

[jira] [Updated] (SPARK-28851) Connect HBase using Spark SQL in Spark 2.x

2019-08-22 Thread ARUN KINDRA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ARUN KINDRA updated SPARK-28851: Description: Hi,   I am basically trying a sample Spark SQL Code which actually read data from

[jira] [Updated] (SPARK-28730) Configurable type coercion policy for table insertion

2019-08-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28730: --- Parent: SPARK-28589 Issue Type: Sub-task (was: Improvement) > Configurable type

[jira] [Created] (SPARK-28851) Connect HBase using Spark SQL in Spark 2.x

2019-08-22 Thread ARUN KINDRA (Jira)
ARUN KINDRA created SPARK-28851: --- Summary: Connect HBase using Spark SQL in Spark 2.x Key: SPARK-28851 URL: https://issues.apache.org/jira/browse/SPARK-28851 Project: Spark Issue Type:

[jira] [Updated] (SPARK-28850) Binary Files RDD allocates false number of threads

2019-08-22 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Lotz updated SPARK-28850: --- Description: When making a call to: {code:java} sc.binaryFiles(somePath){code}   It creates a

[jira] [Updated] (SPARK-28850) Binary Files RDD allocates false number of threads

2019-08-22 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Lotz updated SPARK-28850: --- Description: When making a call to: {code:java} sc.binaryFiles(somePath){code}   It creates a

[jira] [Updated] (SPARK-28850) Binary Files RDD allocates false number of threads

2019-08-22 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Lotz updated SPARK-28850: --- Description: When making a call to: {code:java} sc.binaryFiles(somePath){code}   It creates a

[jira] [Updated] (SPARK-28850) Binary Files RDD allocates false number of threads

2019-08-22 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Lotz updated SPARK-28850: --- Description: When making a call to: {code:java} sc.binaryFiles(somePath){code}   It creates a

[jira] [Created] (SPARK-28850) Binary Files RDD allocates false number of threads

2019-08-22 Thread Marco Lotz (Jira)
Marco Lotz created SPARK-28850: -- Summary: Binary Files RDD allocates false number of threads Key: SPARK-28850 URL: https://issues.apache.org/jira/browse/SPARK-28850 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28779) CSV writer doesn't handle older Mac line endings

2019-08-22 Thread nicolas paris (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913151#comment-16913151 ] nicolas paris commented on SPARK-28779: --- good to know thanks > CSV writer doesn't handle older

[jira] [Commented] (SPARK-28832) Document SHOW SCHEMAS statement in SQL Reference.

2019-08-22 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913149#comment-16913149 ] jobit mathew commented on SPARK-28832: -- [~dkbiswal],But commands are different right, eventhough

[jira] [Resolved] (SPARK-28836) Introduce TPCDSSchema

2019-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28836. -- Resolution: Won't Fix See {{TPCDSQueryBenchmark}}. We already have the ways in Spark code

[jira] [Resolved] (SPARK-28844) Fix typo in SQLConf FILE_COMRESSION_FACTOR

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28844. --- Fix Version/s: 2.4.4 2.3.4 3.0.0 Resolution:

[jira] [Updated] (SPARK-26895) When running spark 2.3 as a proxy user (--proxy-user), SparkSubmit fails to resolve globs owned by target user

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26895: -- Fix Version/s: 2.3.4 > When running spark 2.3 as a proxy user (--proxy-user), SparkSubmit

[jira] [Assigned] (SPARK-28847) Annotate HiveExternalCatalogVersionsSuite with ExtendedHiveTest

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28847: - Assignee: Dongjoon Hyun > Annotate HiveExternalCatalogVersionsSuite with

[jira] [Updated] (SPARK-28780) Delete the incorrect setWeightCol method in LinearSVCModel

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28780: -- Fix Version/s: 2.3.4 > Delete the incorrect setWeightCol method in LinearSVCModel >

[jira] [Updated] (SPARK-28780) Delete the incorrect setWeightCol method in LinearSVCModel

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28780: -- Fix Version/s: 2.4.4 > Delete the incorrect setWeightCol method in LinearSVCModel >

[jira] [Resolved] (SPARK-28847) Annotate HiveExternalCatalogVersionsSuite with ExtendedHiveTest

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28847. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25550

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling

[jira] [Resolved] (SPARK-28821) Document COMPUTE STAT in SQL Reference

2019-08-22 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA resolved SPARK-28821. -- Resolution: Duplicate Will be covered in ANALYZE TABLE JIRA for document >

[jira] [Updated] (SPARK-28848) insert overwrite local directory stored as parquet does not creates snappy.parquet data file at local directory path

2019-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28848: - Description: {code} 0: jdbc:hive2://10.18.18.214:23040/func> insert overwrite local directory

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Attachment: D18F4.png 95330.png 91ADA.png > Spark's

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling

[jira] [Created] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

2019-08-22 Thread Saisai Shao (Jira)
Saisai Shao created SPARK-28849: --- Summary: Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally Key: SPARK-28849 URL: https://issues.apache.org/jira/browse/SPARK-28849

[jira] [Updated] (SPARK-28844) Fix typo in SQLConf FILE_COMRESSION_FACTOR

2019-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28844: -- Affects Version/s: 3.0.0 2.3.3 > Fix typo in SQLConf

[jira] [Commented] (SPARK-28848) insert overwrite local directory stored as parquet does not creates snappy.parquet data file at local directory path

2019-08-22 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913027#comment-16913027 ] Ajith S commented on SPARK-28848: - Thanks for reporting. Will look into this. On initial thoughts looks

[jira] [Created] (SPARK-28848) insert overwrite local directory stored as parquet does not creates snappy.parquet data file at local directory path

2019-08-22 Thread ABHISHEK KUMAR GUPTA (Jira)
ABHISHEK KUMAR GUPTA created SPARK-28848: Summary: insert overwrite local directory stored as parquet does not creates snappy.parquet data file at local directory path Key: SPARK-28848 URL: