[jira] [Commented] (SPARK-22830) Scala Coding style has been improved in Spark Examples

2017-12-18 Thread Chetan Khatri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296360#comment-16296360 ] Chetan Khatri commented on SPARK-22830: --- Appropriate Pull Request has been raised at

[jira] [Commented] (SPARK-22830) Scala Coding style has been improved in Spark Examples

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296356#comment-16296356 ] Apache Spark commented on SPARK-22830: -- User 'chetkhatri' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22830) Scala Coding style has been improved in Spark Examples

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22830: Assignee: (was: Apache Spark) > Scala Coding style has been improved in Spark

[jira] [Assigned] (SPARK-22830) Scala Coding style has been improved in Spark Examples

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22830: Assignee: Apache Spark > Scala Coding style has been improved in Spark Examples >

[jira] [Created] (SPARK-22830) Scala Coding style has been improved in Spark Examples

2017-12-18 Thread Chetan Khatri (JIRA)
Chetan Khatri created SPARK-22830: - Summary: Scala Coding style has been improved in Spark Examples Key: SPARK-22830 URL: https://issues.apache.org/jira/browse/SPARK-22830 Project: Spark

[jira] [Commented] (SPARK-22825) Incorrect results of Casting Array to String

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296308#comment-16296308 ] Takeshi Yamamuro commented on SPARK-22825: -- ok > Incorrect results of Casting Array to String >

[jira] [Commented] (SPARK-22825) Incorrect results of Casting Array to String

2017-12-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296305#comment-16296305 ] Reynold Xin commented on SPARK-22825: - [~maropu] you should :) > Incorrect results of Casting Array

[jira] [Commented] (SPARK-22825) Incorrect results of Casting Array to String

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296304#comment-16296304 ] Takeshi Yamamuro commented on SPARK-22825: -- [~smilegator] you're taking on this now? >

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296234#comment-16296234 ] Takeshi Yamamuro commented on SPARK-17647: -- Probably, is it okay to set

[jira] [Comment Edited] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296234#comment-16296234 ] Takeshi Yamamuro edited comment on SPARK-17647 at 12/19/17 5:29 AM:

[jira] [Commented] (SPARK-22828) Data corruption happens when same RDD being repeatedly used as parent RDD of a custom RDD which reads each parent RDD in concurrent threads

2017-12-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296154#comment-16296154 ] Sean Owen commented on SPARK-22828: --- Are you sure it's not that you're just violating some explicit or

[jira] [Updated] (SPARK-22126) Fix model-specific optimization support for ML tuning

2017-12-18 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-22126: --- Description: Fix model-specific optimization support for ML tuning. This is discussed in

[jira] [Assigned] (SPARK-22829) Add new built-in function date_trunc()

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22829: Assignee: Apache Spark > Add new built-in function date_trunc() >

[jira] [Assigned] (SPARK-22829) Add new built-in function date_trunc()

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22829: Assignee: (was: Apache Spark) > Add new built-in function date_trunc() >

[jira] [Commented] (SPARK-22829) Add new built-in function date_trunc()

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296056#comment-16296056 ] Apache Spark commented on SPARK-22829: -- User 'youngbink' has created a pull request for this issue:

[jira] [Created] (SPARK-22829) Add new built-in function date_trunc()

2017-12-18 Thread Youngbin Kim (JIRA)
Youngbin Kim created SPARK-22829: Summary: Add new built-in function date_trunc() Key: SPARK-22829 URL: https://issues.apache.org/jira/browse/SPARK-22829 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-22797) Add multiple column support to PySpark Bucketizer

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296034#comment-16296034 ] Apache Spark commented on SPARK-22797: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-22797) Add multiple column support to PySpark Bucketizer

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22797: Assignee: Apache Spark > Add multiple column support to PySpark Bucketizer >

[jira] [Assigned] (SPARK-22797) Add multiple column support to PySpark Bucketizer

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22797: Assignee: (was: Apache Spark) > Add multiple column support to PySpark Bucketizer >

[jira] [Updated] (SPARK-22828) Data corruption happens when same RDD being repeatedly used as parent RDD of a custom RDD which reads each parent RDD in concurrent threads

2017-12-18 Thread Yongqin Xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongqin Xiao updated SPARK-22828: - Shepherd: Jiang Xingbo > Data corruption happens when same RDD being repeatedly used as parent

[jira] [Updated] (SPARK-22828) Data corruption happens when same RDD being repeatedly used as parent RDD of a custom RDD which reads each parent RDD in concurrent threads

2017-12-18 Thread Yongqin Xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongqin Xiao updated SPARK-22828: - Description: I have defined a custom RDD that - computes the output based on input data using

[jira] [Updated] (SPARK-22828) Data corruption happens when same RDD being repeatedly used as parent RDD of a custom RDD which reads each parent RDD in concurrent threads

2017-12-18 Thread Yongqin Xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongqin Xiao updated SPARK-22828: - Description: I have defined a custom RDD that - computes the output based on input data using

[jira] [Created] (SPARK-22828) Data corruption happens when same RDD being repeatedly used as parent RDD of a custom RDD which reads each parent RDD in concurrent threads

2017-12-18 Thread Yongqin Xiao (JIRA)
Yongqin Xiao created SPARK-22828: Summary: Data corruption happens when same RDD being repeatedly used as parent RDD of a custom RDD which reads each parent RDD in concurrent threads Key: SPARK-22828 URL:

[jira] [Issue Comment Deleted] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-17647: - Comment: was deleted (was: It seems the master still handle ''%\\%' as `(?s).*\Q%\E`

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295992#comment-16295992 ] Takeshi Yamamuro commented on SPARK-17647: -- It seems the master still handle ''%\\%' as

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295976#comment-16295976 ] Takeshi Yamamuro commented on SPARK-17647: -- I'm looking into the code and I'll make a follow-up

[jira] [Comment Edited] (SPARK-22126) Fix model-specific optimization support for ML tuning

2017-12-18 Thread Bago Amirbekian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295964#comment-16295964 ] Bago Amirbekian edited comment on SPARK-22126 at 12/19/17 12:55 AM:

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

2017-12-18 Thread Bago Amirbekian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295964#comment-16295964 ] Bago Amirbekian commented on SPARK-22126: - Anyone who's following might want to scan the design

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

2017-12-18 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295958#comment-16295958 ] Nan Zhu commented on SPARK-22765: - [~xuefuz] Regarding this, "The symptom is that newly allocated

[jira] [Assigned] (SPARK-22827) Avoid throwing OutOfMemoryError in case of exception in spill

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22827: Assignee: Apache Spark > Avoid throwing OutOfMemoryError in case of exception in spill >

[jira] [Assigned] (SPARK-22827) Avoid throwing OutOfMemoryError in case of exception in spill

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22827: Assignee: (was: Apache Spark) > Avoid throwing OutOfMemoryError in case of exception

[jira] [Commented] (SPARK-22827) Avoid throwing OutOfMemoryError in case of exception in spill

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295926#comment-16295926 ] Apache Spark commented on SPARK-22827: -- User 'sitalkedia' has created a pull request for this issue:

[jira] [Created] (SPARK-22827) Avoid throwing OutOfMemoryError in case of exception in spill

2017-12-18 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-22827: --- Summary: Avoid throwing OutOfMemoryError in case of exception in spill Key: SPARK-22827 URL: https://issues.apache.org/jira/browse/SPARK-22827 Project: Spark

[jira] [Assigned] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

2017-12-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22777: -- Assignee: Anirudh Ramanathan > Docker container built for Kubernetes doesn't allow

[jira] [Resolved] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

2017-12-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22777. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20007

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

2017-12-18 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295814#comment-16295814 ] Xuefu Zhang commented on SPARK-22765: - As an update, I managed to backport SPARK-21656, among a few

[jira] [Resolved] (SPARK-22496) beeline display operation log

2017-12-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22496. -- Resolution: Fixed Assignee: StephenZou > beeline display operation log >

[jira] [Updated] (SPARK-22496) beeline display operation log

2017-12-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-22496: - Fix Version/s: 2.3.0 > beeline display operation log > - > >

[jira] [Resolved] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22813. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19998

[jira] [Assigned] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22813: Assignee: Kazuaki Ishizaki > run-tests.py fails when /usr/sbin/lsof does not exist >

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

2017-12-18 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295770#comment-16295770 ] Xuefu Zhang commented on SPARK-22683: - I did some tests with the proposed change here and saw general

[jira] [Assigned] (SPARK-22496) beeline display operation log

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22496: Assignee: Apache Spark > beeline display operation log > - >

[jira] [Assigned] (SPARK-22496) beeline display operation log

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22496: Assignee: (was: Apache Spark) > beeline display operation log >

[jira] [Commented] (SPARK-20657) Speed up Stage page

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295746#comment-16295746 ] Apache Spark commented on SPARK-20657: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20657) Speed up Stage page

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20657: Assignee: Apache Spark > Speed up Stage page > --- > >

[jira] [Assigned] (SPARK-20657) Speed up Stage page

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20657: Assignee: (was: Apache Spark) > Speed up Stage page > --- > >

[jira] [Assigned] (SPARK-22824) Spark Structured Streaming Source trait breaking change

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22824: Assignee: Apache Spark (was: Jose Torres) > Spark Structured Streaming Source trait

[jira] [Assigned] (SPARK-22824) Spark Structured Streaming Source trait breaking change

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22824: Assignee: Jose Torres (was: Apache Spark) > Spark Structured Streaming Source trait

[jira] [Commented] (SPARK-22824) Spark Structured Streaming Source trait breaking change

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295745#comment-16295745 ] Apache Spark commented on SPARK-22824: -- User 'joseph-torres' has created a pull request for this

[jira] [Commented] (SPARK-20654) Add controls for how much disk the SHS can use

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295691#comment-16295691 ] Apache Spark commented on SPARK-20654: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20654) Add controls for how much disk the SHS can use

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20654: Assignee: Apache Spark > Add controls for how much disk the SHS can use >

[jira] [Assigned] (SPARK-20654) Add controls for how much disk the SHS can use

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20654: Assignee: (was: Apache Spark) > Add controls for how much disk the SHS can use >

[jira] [Commented] (SPARK-22824) Spark Structured Streaming Source trait breaking change

2017-12-18 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295676#comment-16295676 ] Michael Armbrust commented on SPARK-22824: -- This is technically an internal API (as is all of

[jira] [Assigned] (SPARK-22824) Spark Structured Streaming Source trait breaking change

2017-12-18 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-22824: Assignee: Jose Torres > Spark Structured Streaming Source trait breaking change >

[jira] [Commented] (SPARK-22826) [SQL] findWiderTypeForTwo Fails over StructField of Array

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295672#comment-16295672 ] Apache Spark commented on SPARK-22826: -- User 'bdrillard' has created a pull request for this issue:

[jira] [Commented] (SPARK-22826) [SQL] findWiderTypeForTwo Fails over StructField of Array

2017-12-18 Thread Aleksander Eskilson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295673#comment-16295673 ] Aleksander Eskilson commented on SPARK-22826: - I created a PR for this issue,

[jira] [Assigned] (SPARK-22826) [SQL] findWiderTypeForTwo Fails over StructField of Array

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22826: Assignee: Apache Spark > [SQL] findWiderTypeForTwo Fails over StructField of Array >

[jira] [Assigned] (SPARK-22826) [SQL] findWiderTypeForTwo Fails over StructField of Array

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22826: Assignee: (was: Apache Spark) > [SQL] findWiderTypeForTwo Fails over StructField of

[jira] [Updated] (SPARK-22826) [SQL] findWiderTypeForTwo Fails over StructField of Array

2017-12-18 Thread Aleksander Eskilson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksander Eskilson updated SPARK-22826: Description: The {{findWiderTypeForTwo}} codepath in Catalyst {{TypeCoercion}}

[jira] [Updated] (SPARK-22826) [SQL] findWiderTypeForTwo Fails over StructField of Array

2017-12-18 Thread Aleksander Eskilson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksander Eskilson updated SPARK-22826: Description: The {{findWiderTypeForTwo}} codepath in Catalyst {{TypeCoercion}}

[jira] [Created] (SPARK-22826) [SQL] findWiderTypeForTwo Fails over StructField of Array

2017-12-18 Thread Aleksander Eskilson (JIRA)
Aleksander Eskilson created SPARK-22826: --- Summary: [SQL] findWiderTypeForTwo Fails over StructField of Array Key: SPARK-22826 URL: https://issues.apache.org/jira/browse/SPARK-22826 Project:

[jira] [Created] (SPARK-22825) Incorrect results of Casting Array to String

2017-12-18 Thread Xiao Li (JIRA)
Xiao Li created SPARK-22825: --- Summary: Incorrect results of Casting Array to String Key: SPARK-22825 URL: https://issues.apache.org/jira/browse/SPARK-22825 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-20653) Add auto-cleanup of old elements to the new app state store

2017-12-18 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-20653. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19751

[jira] [Assigned] (SPARK-20653) Add auto-cleanup of old elements to the new app state store

2017-12-18 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-20653: Assignee: Marcelo Vanzin > Add auto-cleanup of old elements to the new app state store >

[jira] [Issue Comment Deleted] (SPARK-22689) Could not resolve dependencies for project

2017-12-18 Thread Timur Khusainov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timur Khusainov updated SPARK-22689: Comment: was deleted (was: ?) > Could not resolve dependencies for project >

[jira] [Commented] (SPARK-22689) Could not resolve dependencies for project

2017-12-18 Thread Timur Khusainov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295603#comment-16295603 ] Timur Khusainov commented on SPARK-22689: - ? > Could not resolve dependencies for project >

[jira] [Assigned] (SPARK-22807) Change configuration options to use "container" instead of "docker"

2017-12-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22807: -- Assignee: Anirudh Ramanathan > Change configuration options to use "container"

[jira] [Commented] (SPARK-22812) Failing cran-check on master

2017-12-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295532#comment-16295532 ] Yin Huai commented on SPARK-22812: -- Thank you guys! > Failing cran-check on master >

[jira] [Resolved] (SPARK-22807) Change configuration options to use "container" instead of "docker"

2017-12-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22807. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19995

[jira] [Commented] (SPARK-22798) Add multiple column support to PySpark StringIndexer

2017-12-18 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295403#comment-16295403 ] Huaxin Gao commented on SPARK-22798: I am working on this. Will submit a PR once SPARK-11215 Add

[jira] [Commented] (SPARK-22796) Add multiple column support to PySpark QuantileDiscretizer

2017-12-18 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295401#comment-16295401 ] Huaxin Gao commented on SPARK-22796: I finished the code. Will submit a PR once SPARK-22397 Add

[jira] [Commented] (SPARK-22805) Use aliases for StorageLevel in event logs

2017-12-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295328#comment-16295328 ] Marcelo Vanzin commented on SPARK-22805: Is the question about compatibility? There's none, it's

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when there are enough tasks to run on them

2017-12-18 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295282#comment-16295282 ] Nan Zhu commented on SPARK-21656: - NOTE: the issue fixed by https://github.com/apache/spark/pull/18874 >

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Dong Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295261#comment-16295261 ] Dong Jiang commented on SPARK-17647: Are we sure this issue is resolved, I tested the following on

[jira] [Commented] (SPARK-22824) Spark Structured Streaming Source trait breaking change

2017-12-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295260#comment-16295260 ] Sean Owen commented on SPARK-22824: --- [~zsxwing] [~joseph.torres] was that meant to be an internal API?

[jira] [Comment Edited] (SPARK-8542) PMML export for Decision Trees

2017-12-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295145#comment-16295145 ] Björn-Elmar Macek edited comment on SPARK-8542 at 12/18/17 4:25 PM:

[jira] [Created] (SPARK-22824) Spark Structured Streaming Source trait breaking change

2017-12-18 Thread Matthieu Maitre (JIRA)
Matthieu Maitre created SPARK-22824: --- Summary: Spark Structured Streaming Source trait breaking change Key: SPARK-22824 URL: https://issues.apache.org/jira/browse/SPARK-22824 Project: Spark

[jira] [Comment Edited] (SPARK-8542) PMML export for Decision Trees

2017-12-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295145#comment-16295145 ] Björn-Elmar Macek edited comment on SPARK-8542 at 12/18/17 3:52 PM:

[jira] [Commented] (SPARK-8542) PMML export for Decision Trees

2017-12-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295145#comment-16295145 ] Björn-Elmar Macek commented on SPARK-8542: -- Are there any updates for this issue? You guys got

[jira] [Commented] (SPARK-22725) df.select on a Stream is broken, vs a List

2017-12-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295026#comment-16295026 ] Sean Owen commented on SPARK-22725: --- Sounds like an easy fix, why not do so in the PR? > df.select on

[jira] [Commented] (SPARK-22805) Use aliases for StorageLevel in event logs

2017-12-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295017#comment-16295017 ] Sean Owen commented on SPARK-22805: --- Yes, it's not forwards-compatible. Hm. I don't think we would

[jira] [Updated] (SPARK-22823) Race Condition when reading Broadcast shuffle input. Failed to get broadcast piece

2017-12-18 Thread Dmitrii Bundin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitrii Bundin updated SPARK-22823: --- Description: It seems we have a race condition when trying to read shuffle input which is a

[jira] [Updated] (SPARK-22823) Race Condition when reading Broadcast shuffle input. Failed to get broadcast piece

2017-12-18 Thread Dmitrii Bundin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitrii Bundin updated SPARK-22823: --- Priority: Major (was: Minor) > Race Condition when reading Broadcast shuffle input. Failed

[jira] [Updated] (SPARK-22823) Race Condition when reading Broadcast shuffle input. Failed to get broadcast piece

2017-12-18 Thread Dmitrii Bundin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitrii Bundin updated SPARK-22823: --- Summary: Race Condition when reading Broadcast shuffle input. Failed to get broadcast piece

[jira] [Updated] (SPARK-22823) Race Condition when reading Broadcast shuffle input

2017-12-18 Thread Dmitrii Bundin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitrii Bundin updated SPARK-22823: --- Component/s: Spark Core > Race Condition when reading Broadcast shuffle input >

[jira] [Updated] (SPARK-22823) Race Condition when reading Broadcast shuffle input

2017-12-18 Thread Dmitrii Bundin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitrii Bundin updated SPARK-22823: --- Description: It seems we have a race condition when trying to read shuffle input which is a

[jira] [Created] (SPARK-22823) Race Condition when reading Broadcast shuffle input

2017-12-18 Thread Dmitrii Bundin (JIRA)
Dmitrii Bundin created SPARK-22823: -- Summary: Race Condition when reading Broadcast shuffle input Key: SPARK-22823 URL: https://issues.apache.org/jira/browse/SPARK-22823 Project: Spark

[jira] [Commented] (SPARK-22806) Window Aggregate functions: unexpected result at ordered partition

2017-12-18 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294869#comment-16294869 ] Attila Zsolt Piros commented on SPARK-22806: Thanks! In this case within SPARK-22362 I will

[jira] [Assigned] (SPARK-22822) Basic tests for FunctionArgumentConversion and DecimalPrecision

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22822: Assignee: (was: Apache Spark) > Basic tests for FunctionArgumentConversion and

[jira] [Commented] (SPARK-22822) Basic tests for FunctionArgumentConversion and DecimalPrecision

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294813#comment-16294813 ] Apache Spark commented on SPARK-22822: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22822) Basic tests for FunctionArgumentConversion and DecimalPrecision

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22822: Assignee: Apache Spark > Basic tests for FunctionArgumentConversion and DecimalPrecision

[jira] [Created] (SPARK-22822) Basic tests for FunctionArgumentConversion and DecimalPrecision

2017-12-18 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-22822: --- Summary: Basic tests for FunctionArgumentConversion and DecimalPrecision Key: SPARK-22822 URL: https://issues.apache.org/jira/browse/SPARK-22822 Project: Spark

[jira] [Commented] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294768#comment-16294768 ] Apache Spark commented on SPARK-22777: -- User 'foxish' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22777: Assignee: (was: Apache Spark) > Docker container built for Kubernetes doesn't allow

[jira] [Assigned] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

2017-12-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22777: Assignee: Apache Spark > Docker container built for Kubernetes doesn't allow running

[jira] [Commented] (SPARK-22647) Docker files for image creation

2017-12-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294759#comment-16294759 ] Anirudh Ramanathan commented on SPARK-22647: [~wangyang] libc6-compat is in there now. Can

[jira] [Commented] (SPARK-22805) Use aliases for StorageLevel in event logs

2017-12-18 Thread Sergei Lebedev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294745#comment-16294745 ] Sergei Lebedev commented on SPARK-22805: It does, in a sense that 1.6 history server can no

[jira] [Commented] (SPARK-22805) Use aliases for StorageLevel in event logs

2017-12-18 Thread Anthony Truchet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294727#comment-16294727 ] Anthony Truchet commented on SPARK-22805: - As I understand it does not break any standard use and

[jira] [Resolved] (SPARK-22812) Failing cran-check on master

2017-12-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22812. -- Resolution: Fixed This is fixed via [~viirya]'s request to CRAN sysadmin team. Please see