[jira] [Assigned] (SPARK-28089) File source v2: support reading output of file streaming Sink

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28089: Assignee: Apache Spark > File source v2: support reading output of file streaming Sink >

[jira] [Assigned] (SPARK-28089) File source v2: support reading output of file streaming Sink

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28089: Assignee: (was: Apache Spark) > File source v2: support reading output of file

[jira] [Created] (SPARK-28089) File source v2: support reading output of file streaming Sink

2019-06-17 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-28089: -- Summary: File source v2: support reading output of file streaming Sink Key: SPARK-28089 URL: https://issues.apache.org/jira/browse/SPARK-28089 Project: Spark

[jira] [Assigned] (SPARK-28088) String Functions: Enhance LPAD/RPAD function

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28088: Assignee: Apache Spark > String Functions: Enhance LPAD/RPAD function >

[jira] [Assigned] (SPARK-28088) String Functions: Enhance LPAD/RPAD function

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28088: Assignee: (was: Apache Spark) > String Functions: Enhance LPAD/RPAD function >

[jira] [Created] (SPARK-28088) String Functions: Enhance LPAD/RPAD function

2019-06-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28088: --- Summary: String Functions: Enhance LPAD/RPAD function Key: SPARK-28088 URL: https://issues.apache.org/jira/browse/SPARK-28088 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28058. -- Resolution: Fixed Fix Version/s: 2.4.4 3.0.0 Issue resolved by pull

[jira] [Assigned] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28058: Assignee: Liang-Chi Hsieh > Reading csv with DROPMALFORMED sometimes doesn't drop

[jira] [Updated] (SPARK-28072) Fix IncompatibleClassChangeError in `FromUnixTime` codegen on JDK9+

2019-06-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28072: -- Summary: Fix IncompatibleClassChangeError in `FromUnixTime` codegen on JDK9+ (was: Use

[jira] [Resolved] (SPARK-28056) Document SCALAR_ITER Pandas UDF

2019-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-28056. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24897

[jira] [Created] (SPARK-28087) String Functions: Add support split_part

2019-06-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28087: --- Summary: String Functions: Add support split_part Key: SPARK-28087 URL: https://issues.apache.org/jira/browse/SPARK-28087 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-27666) Do not release lock while TaskContext already completed

2019-06-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27666. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24699

[jira] [Assigned] (SPARK-27666) Do not release lock while TaskContext already completed

2019-06-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27666: --- Assignee: wuyi > Do not release lock while TaskContext already completed >

[jira] [Updated] (SPARK-27930) List all built-in UDFs have different names

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27930: Description: This ticket list all built-in UDFs have different names:  ||PostgreSQL||Spark

[jira] [Commented] (SPARK-27969) Non-deterministic expressions in filters or projects can unnecessarily prevent all scan-time column pruning, harming performance

2019-06-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866151#comment-16866151 ] Wenchen Fan commented on SPARK-27969: - We should think about what non-deterministic means in Spark,

[jira] [Resolved] (SPARK-24175) improve the Spark 2.4 migration guide

2019-06-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24175. - Resolution: Won't Fix > improve the Spark 2.4 migration guide >

[jira] [Updated] (SPARK-28067) Incorrect results in decimal aggregation with whole-stage code gen enabled

2019-06-17 Thread Mark Sirek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Sirek updated SPARK-28067: --- Description: The following test case involving a join followed by a sum aggregation returns the

[jira] [Resolved] (SPARK-28082) Add a note to DROPMALFORMED mode of CSV for column pruning

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28082. -- Resolution: Duplicate > Add a note to DROPMALFORMED mode of CSV for column pruning >

[jira] [Commented] (SPARK-28082) Add a note to DROPMALFORMED mode of CSV for column pruning

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866098#comment-16866098 ] Hyukjin Kwon commented on SPARK-28082: -- Sorry, [~viirya] for back and forth. I think your first

[jira] [Commented] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866097#comment-16866097 ] Hyukjin Kwon commented on SPARK-28058: -- Yes, that was what I saw and I thought it's a bug in

[jira] [Updated] (SPARK-27930) List all built-in UDFs have different names

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27930: Description: This ticket list all built-in UDFs have different names:  ||PostgreSQL||Spark

[jira] [Resolved] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28041. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24867

[jira] [Assigned] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28041: Assignee: Hyukjin Kwon > Increase the minimum pandas version to 0.23.2 >

[jira] [Assigned] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28041: Assignee: Bryan Cutler (was: Hyukjin Kwon) > Increase the minimum pandas version to

[jira] [Assigned] (SPARK-28056) Document SCALAR_ITER Pandas UDF

2019-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-28056: - Assignee: Xiangrui Meng > Document SCALAR_ITER Pandas UDF >

[jira] [Assigned] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28006: Assignee: Apache Spark > User-defined grouped transform pandas_udf for window operations

[jira] [Assigned] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28006: Assignee: (was: Apache Spark) > User-defined grouped transform pandas_udf for window

[jira] [Comment Edited] (SPARK-27767) Built-in function: generate_series

2019-06-17 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865912#comment-16865912 ] Dylan Guedes edited comment on SPARK-27767 at 6/17/19 8:02 PM: ---

[jira] [Commented] (SPARK-27767) Built-in function: generate_series

2019-06-17 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865912#comment-16865912 ] Dylan Guedes commented on SPARK-27767: -- [~smilegator] by the way, I just checked and there is a

[jira] [Created] (SPARK-28086) Adds `random()` sql function

2019-06-17 Thread Dylan Guedes (JIRA)
Dylan Guedes created SPARK-28086: Summary: Adds `random()` sql function Key: SPARK-28086 URL: https://issues.apache.org/jira/browse/SPARK-28086 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28085) Spark Scala API documentation URLs not working properly in Chrome

2019-06-17 Thread Andrew Leverentz (JIRA)
Andrew Leverentz created SPARK-28085: Summary: Spark Scala API documentation URLs not working properly in Chrome Key: SPARK-28085 URL: https://issues.apache.org/jira/browse/SPARK-28085 Project:

[jira] [Updated] (SPARK-27989) Add retries on the connection to the driver

2019-06-17 Thread Jose Luis Pedrosa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Luis Pedrosa updated SPARK-27989: -- Description:   Any failure in the executor when trying to connect to the driver,

[jira] [Commented] (SPARK-28084) LOAD DATA command resolving the partition column name considering case senstive manner

2019-06-17 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865758#comment-16865758 ] Sujith Chacko commented on SPARK-28084: --- insert command the partition column will be resolved

[jira] [Assigned] (SPARK-24898) Adding spark.checkpoint.compress to the docs

2019-06-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-24898: - Assignee: Sandeep > Adding spark.checkpoint.compress to the docs >

[jira] [Updated] (SPARK-24898) Adding spark.checkpoint.compress to the docs

2019-06-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24898: -- Issue Type: Task (was: Bug) > Adding spark.checkpoint.compress to the docs >

[jira] [Updated] (SPARK-28084) LOAD DATA command resolving the partition column name considering case senstive manner

2019-06-17 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-28084: -- Attachment: parition_casesensitive.PNG > LOAD DATA command resolving the partition column

[jira] [Created] (SPARK-28084) LOAD DATA command resolving the partition column name considering case senstive manner

2019-06-17 Thread Sujith Chacko (JIRA)
Sujith Chacko created SPARK-28084: - Summary: LOAD DATA command resolving the partition column name considering case senstive manner Key: SPARK-28084 URL: https://issues.apache.org/jira/browse/SPARK-28084

[jira] [Comment Edited] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865714#comment-16865714 ] Liang-Chi Hsieh edited comment on SPARK-28058 at 6/17/19 3:59 PM: --

[jira] [Updated] (SPARK-28082) Add a note to DROPMALFORMED mode of CSV for column pruning

2019-06-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28082: -- Component/s: Documentation > Add a note to DROPMALFORMED mode of CSV for column pruning >

[jira] [Updated] (SPARK-25341) Support rolling back a shuffle map stage and re-generate the shuffle files

2019-06-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25341: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support rolling back a

[jira] [Updated] (SPARK-28074) [SS] Document caveats on using multiple stateful operations in single query

2019-06-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28074: -- Component/s: Documentation > [SS] Document caveats on using multiple stateful operations in

[jira] [Resolved] (SPARK-28066) Optimize UTF8String.trim() for common case of no whitespace

2019-06-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28066. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via

[jira] [Commented] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865714#comment-16865714 ] Liang-Chi Hsieh commented on SPARK-28058: - [~hyukjin.kwon] Do you mean this is suspect to be a

[jira] [Created] (SPARK-28083) ANSI SQL: LIKE predicate: ESCAPE clause

2019-06-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28083: --- Summary: ANSI SQL: LIKE predicate: ESCAPE clause Key: SPARK-28083 URL: https://issues.apache.org/jira/browse/SPARK-28083 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-28082) Add a note to DROPMALFORMED mode of CSV for column pruning

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28082: Assignee: Apache Spark > Add a note to DROPMALFORMED mode of CSV for column pruning >

[jira] [Assigned] (SPARK-28082) Add a note to DROPMALFORMED mode of CSV for column pruning

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28082: Assignee: (was: Apache Spark) > Add a note to DROPMALFORMED mode of CSV for column

[jira] [Commented] (SPARK-28082) Add a note to DROPMALFORMED mode of CSV for column pruning

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865694#comment-16865694 ] Apache Spark commented on SPARK-28082: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865695#comment-16865695 ] Liang-Chi Hsieh commented on SPARK-28058: - [~stwhit] Thanks for letting us know that! Although

[jira] [Created] (SPARK-28082) Add a note to DROPMALFORMED mode of CSV for column pruning

2019-06-17 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-28082: --- Summary: Add a note to DROPMALFORMED mode of CSV for column pruning Key: SPARK-28082 URL: https://issues.apache.org/jira/browse/SPARK-28082 Project: Spark

[jira] [Updated] (SPARK-27989) Add retries on the connection to the driver

2019-06-17 Thread Jose Luis Pedrosa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Luis Pedrosa updated SPARK-27989: -- Component/s: (was: Kubernetes) > Add retries on the connection to the driver >

[jira] [Comment Edited] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Stuart White (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865679#comment-16865679 ] Stuart White edited comment on SPARK-28058 at 6/17/19 3:08 PM: --- Thank you

[jira] [Commented] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Stuart White (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865679#comment-16865679 ] Stuart White commented on SPARK-28058: -- Thank you both for your responses.   I now see that at the

[jira] [Assigned] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28058: Assignee: Apache Spark > Reading csv with DROPMALFORMED sometimes doesn't drop malformed

[jira] [Assigned] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28058: Assignee: (was: Apache Spark) > Reading csv with DROPMALFORMED sometimes doesn't

[jira] [Assigned] (SPARK-28081) word2vec 'large' count value too low for very large corpora

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28081: Assignee: Sean Owen (was: Apache Spark) > word2vec 'large' count value too low for very

[jira] [Assigned] (SPARK-28081) word2vec 'large' count value too low for very large corpora

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28081: Assignee: Apache Spark (was: Sean Owen) > word2vec 'large' count value too low for very

[jira] [Commented] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865664#comment-16865664 ] Liang-Chi Hsieh commented on SPARK-28058: - Although this isn't a bug, I think it might be worth

[jira] [Commented] (SPARK-28058) Reading csv with DROPMALFORMED sometimes doesn't drop malformed records

2019-06-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865650#comment-16865650 ] Liang-Chi Hsieh commented on SPARK-28058: - This is due to CSV parser column pruning. You can

[jira] [Created] (SPARK-28081) word2vec 'large' count value too low for very large corpora

2019-06-17 Thread Sean Owen (JIRA)
Sean Owen created SPARK-28081: - Summary: word2vec 'large' count value too low for very large corpora Key: SPARK-28081 URL: https://issues.apache.org/jira/browse/SPARK-28081 Project: Spark Issue

[jira] [Commented] (SPARK-28076) String Functions: SUBSTRING support regular expression

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865608#comment-16865608 ] Yuming Wang commented on SPARK-28076: - cc [~lipzhu] Would you like to pick up this? > String

[jira] [Updated] (SPARK-28012) Hive UDF supports struct type foldable expression

2019-06-17 Thread dzcxzl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-28012: --- Summary: Hive UDF supports struct type foldable expression (was: Hive UDF supports literal struct type) >

[jira] [Updated] (SPARK-28012) Hive UDF supports literal struct type

2019-06-17 Thread dzcxzl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-28012: --- Description: Currently using hive udf, the parameter is struct type, there will be an exception thrown.

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2019-06-17 Thread Dominic Ricard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865592#comment-16865592 ] Dominic Ricard commented on SPARK-21067: In 2.4, the problem also affect Parquet tables

[jira] [Created] (SPARK-28080) There is a problem to download and watch offline the history of an application with multiple attempts due to UI inconsistency

2019-06-17 Thread Gal Weiss (JIRA)
Gal Weiss created SPARK-28080: - Summary: There is a problem to download and watch offline the history of an application with multiple attempts due to UI inconsistency Key: SPARK-28080 URL:

[jira] [Updated] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2019-06-17 Thread Dominic Ricard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Ricard updated SPARK-21067: --- Affects Version/s: 2.4.3 > Thrift Server - CTAS fail with Unable to move source >

[jira] [Updated] (SPARK-28078) String Functions: Add support other 4 REGEXP functions

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28078: Summary: String Functions: Add support other 4 REGEXP functions (was: String Functions: Add

[jira] [Created] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-17 Thread F Jimenez (JIRA)
F Jimenez created SPARK-28079: - Summary: CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema Key: SPARK-28079 URL: https://issues.apache.org/jira/browse/SPARK-28079

[jira] [Created] (SPARK-28078) String Functions: Add support other 4 REGEXP_ functions

2019-06-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28078: --- Summary: String Functions: Add support other 4 REGEXP_ functions Key: SPARK-28078 URL: https://issues.apache.org/jira/browse/SPARK-28078 Project: Spark Issue

[jira] [Updated] (SPARK-28075) String Functions: Enhance TRIM function

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28075: Summary: String Functions: Enhance TRIM function (was: Enhance TRIM function) > String

[jira] [Created] (SPARK-28077) String Functions: Add support OVERLAY

2019-06-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28077: --- Summary: String Functions: Add support OVERLAY Key: SPARK-28077 URL: https://issues.apache.org/jira/browse/SPARK-28077 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-28076) String Functions: Support regular expression substring

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28076: Summary: String Functions: Support regular expression substring (was: Support regular expression

[jira] [Updated] (SPARK-28076) String Functions: SUBSTRING support regular expression

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28076: Summary: String Functions: SUBSTRING support regular expression (was: String Functions: Support

[jira] [Updated] (SPARK-28076) Support regular expression substring

2019-06-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28076: Description: ||Function||Return Type||Description||Example||Result|| |{{substring(_string_}} from 

[jira] [Created] (SPARK-28076) Support regular expression substring

2019-06-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28076: --- Summary: Support regular expression substring Key: SPARK-28076 URL: https://issues.apache.org/jira/browse/SPARK-28076 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

2019-06-17 Thread Chris Martin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865433#comment-16865433 ] Chris Martin commented on SPARK-27463: --  sounds good to me too. > Support Dataframe Cogroup via

[jira] [Assigned] (SPARK-28075) Enhance TRIM function

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28075: Assignee: Apache Spark > Enhance TRIM function > - > >

[jira] [Assigned] (SPARK-28075) Enhance TRIM function

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28075: Assignee: (was: Apache Spark) > Enhance TRIM function > - > >

[jira] [Created] (SPARK-28075) Enhance TRIM function

2019-06-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28075: --- Summary: Enhance TRIM function Key: SPARK-28075 URL: https://issues.apache.org/jira/browse/SPARK-28075 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2019-06-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23128: --- Assignee: Carson Wang (was: Maryann Xue) > A new approach to do adaptive execution in

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2019-06-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865396#comment-16865396 ] Wenchen Fan commented on SPARK-23128: - To split credits, I'm re-assigning this ticket to

[jira] [Assigned] (SPARK-28074) [SS] Document caveats on using multiple stateful operations in single query

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28074: Assignee: Apache Spark > [SS] Document caveats on using multiple stateful operations in

[jira] [Assigned] (SPARK-28074) [SS] Document caveats on using multiple stateful operations in single query

2019-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28074: Assignee: (was: Apache Spark) > [SS] Document caveats on using multiple stateful

[jira] [Created] (SPARK-28074) [SS] Document caveats on using multiple stateful operations in single query

2019-06-17 Thread Jungtaek Lim (JIRA)
Jungtaek Lim created SPARK-28074: Summary: [SS] Document caveats on using multiple stateful operations in single query Key: SPARK-28074 URL: https://issues.apache.org/jira/browse/SPARK-28074 Project: