[jira] [Commented] (SPARK-23912) High-order function: array_distinct(x) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434926#comment-16434926 ] Apache Spark commented on SPARK-23912: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23912) High-order function: array_distinct(x) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23912: Assignee: (was: Apache Spark) > High-order function: array_distinct(x) → array >

[jira] [Assigned] (SPARK-23912) High-order function: array_distinct(x) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23912: Assignee: Apache Spark > High-order function: array_distinct(x) → array >

[jira] [Assigned] (SPARK-23957) Sorts in subqueries are redundant and can be removed

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23957: Assignee: Apache Spark > Sorts in subqueries are redundant and can be removed >

[jira] [Commented] (SPARK-23957) Sorts in subqueries are redundant and can be removed

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434918#comment-16434918 ] Apache Spark commented on SPARK-23957: -- User 'henryr' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23957) Sorts in subqueries are redundant and can be removed

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23957: Assignee: (was: Apache Spark) > Sorts in subqueries are redundant and can be removed

[jira] [Resolved] (SPARK-23958) HadoopRdd filters empty files to avoid generating empty tasks that affect the performance of the Spark computing performance.

2018-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23958. -- Resolution: Duplicate > HadoopRdd filters empty files to avoid generating empty tasks that

[jira] [Resolved] (SPARK-23950) Coalescing an empty dataframe to 1 partition

2018-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23950. -- Resolution: Cannot Reproduce > Coalescing an empty dataframe to 1 partition >

[jira] [Resolved] (SPARK-23965) make python/py4j-src-0.x.y.zip file name Spark version-independent

2018-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23965. -- Resolution: Won't Fix > make python/py4j-src-0.x.y.zip file name Spark version-independent >

[jira] [Resolved] (SPARK-23955) typo in parameter name 'rawPredicition'

2018-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23955. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/21030 > typo in parameter

[jira] [Commented] (SPARK-23965) make python/py4j-src-0.x.y.zip file name Spark version-independent

2018-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434810#comment-16434810 ] Hyukjin Kwon commented on SPARK-23965: -- I would leave this resolved. I don't think it's a strong

[jira] [Commented] (SPARK-23965) make python/py4j-src-0.x.y.zip file name Spark version-independent

2018-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434809#comment-16434809 ] Hyukjin Kwon commented on SPARK-23965: -- I think that sounds we are going to more make the thridparty

[jira] [Updated] (SPARK-23956) Use effective RPC port in AM registration

2018-04-11 Thread Gera Shegalov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated SPARK-23956: -- Priority: Minor (was: Major) > Use effective RPC port in AM registration >

[jira] [Commented] (SPARK-23961) pyspark toLocalIterator throws an exception

2018-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434799#comment-16434799 ] Hyukjin Kwon commented on SPARK-23961: -- FWIW, I met this issue a while ago too (and I gave up with

[jira] [Commented] (SPARK-23920) High-order function: array_remove(x, element) → array

2018-04-11 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434786#comment-16434786 ] Huaxin Gao commented on SPARK-23920: I will work on this. Thanks! > High-order function:

[jira] [Commented] (SPARK-23966) Refactoring all checkpoint file writing logic in a common interface

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434682#comment-16434682 ] Apache Spark commented on SPARK-23966: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23966) Refactoring all checkpoint file writing logic in a common interface

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23966: Assignee: Tathagata Das (was: Apache Spark) > Refactoring all checkpoint file writing

[jira] [Assigned] (SPARK-23966) Refactoring all checkpoint file writing logic in a common interface

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23966: Assignee: Apache Spark (was: Tathagata Das) > Refactoring all checkpoint file writing

[jira] [Created] (SPARK-23966) Refactoring all checkpoint file writing logic in a common interface

2018-04-11 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-23966: - Summary: Refactoring all checkpoint file writing logic in a common interface Key: SPARK-23966 URL: https://issues.apache.org/jira/browse/SPARK-23966 Project: Spark

[jira] [Assigned] (SPARK-23956) Use effective RPC port in AM registration

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23956: Assignee: (was: Apache Spark) > Use effective RPC port in AM registration >

[jira] [Commented] (SPARK-23956) Use effective RPC port in AM registration

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434595#comment-16434595 ] Apache Spark commented on SPARK-23956: -- User 'gerashegalov' has created a pull request for this

[jira] [Assigned] (SPARK-23956) Use effective RPC port in AM registration

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23956: Assignee: Apache Spark > Use effective RPC port in AM registration >

[jira] [Updated] (SPARK-23965) make python/py4j-src-0.x.y.zip file name Spark version-independent

2018-04-11 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov updated SPARK-23965: -- Description: After each Spark release (that's normally packaged with slightly newer

[jira] [Commented] (SPARK-23955) typo in parameter name 'rawPredicition'

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434555#comment-16434555 ] Apache Spark commented on SPARK-23955: -- User 'codeforfun15' has created a pull request for this

[jira] [Assigned] (SPARK-23955) typo in parameter name 'rawPredicition'

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23955: Assignee: Apache Spark > typo in parameter name 'rawPredicition' >

[jira] [Assigned] (SPARK-23955) typo in parameter name 'rawPredicition'

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23955: Assignee: (was: Apache Spark) > typo in parameter name 'rawPredicition' >

[jira] [Created] (SPARK-23965) make python/py4j-src-0.x.y.zip file name Spark version-independent

2018-04-11 Thread Ruslan Dautkhanov (JIRA)
Ruslan Dautkhanov created SPARK-23965: - Summary: make python/py4j-src-0.x.y.zip file name Spark version-independent Key: SPARK-23965 URL: https://issues.apache.org/jira/browse/SPARK-23965

[jira] [Comment Edited] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-11 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431403#comment-16431403 ] Bruce Robbins edited comment on SPARK-23715 at 4/11/18 8:09 PM: I've been

[jira] [Commented] (SPARK-23914) High-order function: array_union(x, y) → array

2018-04-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434494#comment-16434494 ] Kazuaki Ishizaki commented on SPARK-23914: -- I will work for this, thank you. > High-order

[jira] [Commented] (SPARK-23913) High-order function: array_intersect(x, y) → array

2018-04-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434491#comment-16434491 ] Kazuaki Ishizaki commented on SPARK-23913: -- I will work for this, thank you. > High-order

[jira] [Commented] (SPARK-23964) why does Spillable wait for 32 elements?

2018-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434479#comment-16434479 ] Thomas Graves commented on SPARK-23964: --- I'm not sure, I'm trying to figure out if there is a

[jira] [Assigned] (SPARK-23931) High-order function: zip(array1, array2[, ...]) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23931: Assignee: Apache Spark > High-order function: zip(array1, array2[, ...]) → array >

[jira] [Commented] (SPARK-23931) High-order function: zip(array1, array2[, ...]) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434475#comment-16434475 ] Apache Spark commented on SPARK-23931: -- User 'DylanGuedes' has created a pull request for this

[jira] [Assigned] (SPARK-23931) High-order function: zip(array1, array2[, ...]) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23931: Assignee: (was: Apache Spark) > High-order function: zip(array1, array2[, ...]) →

[jira] [Commented] (SPARK-23964) why does Spillable wait for 32 elements?

2018-04-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434460#comment-16434460 ] Reynold Xin commented on SPARK-23964: - Was it trying to reduce overhead?   > why does Spillable

[jira] [Updated] (SPARK-23964) why does Spillable wait for 32 elements?

2018-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23964: -- Description: The spillable class has a check in maybeSpill as to when it tries to acquire

[jira] [Commented] (SPARK-23964) why does Spillable wait for 32 elements?

2018-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434454#comment-16434454 ] Thomas Graves commented on SPARK-23964: --- [~andrewor14]  [~matei] [~r...@databricks.com]   A few

[jira] [Commented] (SPARK-9312) The OneVsRest model does not provide confidence factor(not probability) along with the prediction

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434447#comment-16434447 ] Apache Spark commented on SPARK-9312: - User 'ludatabricks' has created a pull request for this issue:

[jira] [Updated] (SPARK-23964) why does Spillable wait for 32 elements?

2018-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23964: -- Description: The spillable class has a check: if (elementsRead % 32 == 0 && currentMemory >=

[jira] [Updated] (SPARK-23964) why does Spillable wait for 32 elements?

2018-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23964: -- Environment: (was: The spillable class has a check: if (elementsRead % 32 == 0 &&

[jira] [Created] (SPARK-23964) why does Spillable wait for 32 elements?

2018-04-11 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-23964: - Summary: why does Spillable wait for 32 elements? Key: SPARK-23964 URL: https://issues.apache.org/jira/browse/SPARK-23964 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-22883) ML test for StructuredStreaming: spark.ml.feature, A-M

2018-04-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-22883. --- Resolution: Fixed Fix Version/s: 2.3.1 Issue resolved by pull request 21042

[jira] [Updated] (SPARK-23961) pyspark toLocalIterator throws an exception

2018-04-11 Thread Michel Lemay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michel Lemay updated SPARK-23961: - Description: Given a dataframe and use toLocalIterator. If we do not consume all records, it

[jira] [Assigned] (SPARK-23963) Queries on text-based Hive tables grow disproportionately slower as the number of columns increase

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23963: Assignee: (was: Apache Spark) > Queries on text-based Hive tables grow

[jira] [Commented] (SPARK-23963) Queries on text-based Hive tables grow disproportionately slower as the number of columns increase

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434265#comment-16434265 ] Apache Spark commented on SPARK-23963: -- User 'bersprockets' has created a pull request for this

[jira] [Assigned] (SPARK-23963) Queries on text-based Hive tables grow disproportionately slower as the number of columns increase

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23963: Assignee: Apache Spark > Queries on text-based Hive tables grow disproportionately slower

[jira] [Updated] (SPARK-23948) Trigger mapstage's job listener in submitMissingTasks

2018-04-11 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23948: - Component/s: Scheduler > Trigger mapstage's job listener in submitMissingTasks >

[jira] [Updated] (SPARK-22883) ML test for StructuredStreaming: spark.ml.feature, A-M

2018-04-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22883: -- Fix Version/s: 2.4.0 > ML test for StructuredStreaming: spark.ml.feature, A-M >

[jira] [Updated] (SPARK-22883) ML test for StructuredStreaming: spark.ml.feature, A-M

2018-04-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22883: -- Target Version/s: 2.3.1, 2.4.0 > ML test for StructuredStreaming: spark.ml.feature,

[jira] [Commented] (SPARK-22883) ML test for StructuredStreaming: spark.ml.feature, A-M

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434238#comment-16434238 ] Apache Spark commented on SPARK-22883: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-23734) InvalidSchemaException While Saving ALSModel

2018-04-11 Thread Stanley Poon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434219#comment-16434219 ] Stanley Poon edited comment on SPARK-23734 at 4/11/18 4:58 PM: --- [~viirya]

[jira] [Updated] (SPARK-23734) InvalidSchemaException While Saving ALSModel

2018-04-11 Thread Stanley Poon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanley Poon updated SPARK-23734: - Environment: macOS 10.13.2 Scala 2.11.8 Spark 2.3.0  v2.3.0-rc5 (Feb 22 2018) was: macOS

[jira] [Commented] (SPARK-23734) InvalidSchemaException While Saving ALSModel

2018-04-11 Thread Stanley Poon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434219#comment-16434219 ] Stanley Poon commented on SPARK-23734: -- [~viirya] Thank you for checking into this. I added the

[jira] [Updated] (SPARK-23734) InvalidSchemaException While Saving ALSModel

2018-04-11 Thread Stanley Poon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanley Poon updated SPARK-23734: - Environment: macOS 10.13.2 Scala 2.11.8 Spark 2.3.0  v2.3.0-rc5 was: macOS 10.13.2 Scala

[jira] [Commented] (SPARK-23936) High-order function: map_concat(map1<K, V>, map2<K, V>, ..., mapN<K, V>) → map<K,V>

2018-04-11 Thread Marek Novotny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434185#comment-16434185 ] Marek Novotny commented on SPARK-23936: --- Shouldn't we overload _concat_ function for maps instead

[jira] [Updated] (SPARK-23963) Queries on text-based Hive tables grow disproportionately slower as the number of columns increase

2018-04-11 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-23963: -- Description: TableReader gets disproportionately slower as the number of columns in the query

[jira] [Updated] (SPARK-23963) Queries on text-based Hive tables grow disproportionately slower as the number of columns increase

2018-04-11 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-23963: -- Description: TableReader gets disproportionately slower as the number of columns in the query

[jira] [Created] (SPARK-23963) Queries on text-based Hive tables grow disproportionately slower as the number of columns increase

2018-04-11 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-23963: - Summary: Queries on text-based Hive tables grow disproportionately slower as the number of columns increase Key: SPARK-23963 URL:

[jira] [Commented] (SPARK-23962) Flaky tests from SQLMetricsTestUtils.currentExecutionIds

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434077#comment-16434077 ] Apache Spark commented on SPARK-23962: -- User 'squito' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23962) Flaky tests from SQLMetricsTestUtils.currentExecutionIds

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23962: Assignee: Apache Spark > Flaky tests from SQLMetricsTestUtils.currentExecutionIds >

[jira] [Assigned] (SPARK-23962) Flaky tests from SQLMetricsTestUtils.currentExecutionIds

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23962: Assignee: (was: Apache Spark) > Flaky tests from

[jira] [Assigned] (SPARK-22941) Allow SparkSubmit to throw exceptions instead of exiting / printing errors.

2018-04-11 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-22941: Assignee: Marcelo Vanzin > Allow SparkSubmit to throw exceptions instead of exiting /

[jira] [Resolved] (SPARK-22941) Allow SparkSubmit to throw exceptions instead of exiting / printing errors.

2018-04-11 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-22941. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20925

[jira] [Created] (SPARK-23962) Flaky tests from SQLMetricsTestUtils.currentExecutionIds

2018-04-11 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-23962: Summary: Flaky tests from SQLMetricsTestUtils.currentExecutionIds Key: SPARK-23962 URL: https://issues.apache.org/jira/browse/SPARK-23962 Project: Spark

[jira] [Updated] (SPARK-23962) Flaky tests from SQLMetricsTestUtils.currentExecutionIds

2018-04-11 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23962: - Attachment: unit-tests.log > Flaky tests from SQLMetricsTestUtils.currentExecutionIds >

[jira] [Updated] (SPARK-23959) UnresolvedException with DataSet created from Seq.empty since Spark 2.3.0

2018-04-11 Thread Sam De Backer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam De Backer updated SPARK-23959: -- Description: The following snippet works fine in Spark 2.2.1 but gives a rather cryptic

[jira] [Resolved] (SPARK-6951) History server slow startup if the event log directory is large

2018-04-11 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-6951. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20952

[jira] [Assigned] (SPARK-6951) History server slow startup if the event log directory is large

2018-04-11 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-6951: --- Assignee: Marcelo Vanzin > History server slow startup if the event log directory is large >

[jira] [Commented] (SPARK-12105) Add a DataFrame.show() with argument for output PrintStream

2018-04-11 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434015#comment-16434015 ] Tomasz Gawęda commented on SPARK-12105: --- +1, It's a quite common question on StackOverflow: "how to

[jira] [Assigned] (SPARK-12105) Add a DataFrame.show() with argument for output PrintStream

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12105: Assignee: Apache Spark > Add a DataFrame.show() with argument for output PrintStream >

[jira] [Assigned] (SPARK-12105) Add a DataFrame.show() with argument for output PrintStream

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12105: Assignee: (was: Apache Spark) > Add a DataFrame.show() with argument for output

[jira] [Assigned] (SPARK-23960) Mark HashAggregateExec.bufVars as transient

2018-04-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23960: --- Assignee: Kris Mok > Mark HashAggregateExec.bufVars as transient >

[jira] [Resolved] (SPARK-23960) Mark HashAggregateExec.bufVars as transient

2018-04-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23960. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21039

[jira] [Commented] (SPARK-23927) High-order function: sequence

2018-04-11 Thread Alex Wajda (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433926#comment-16433926 ] Alex Wajda commented on SPARK-23927: I will take this one. Thanks. > High-order function: sequence >

[jira] [Updated] (SPARK-23961) pyspark toLocalIterator throws an exception

2018-04-11 Thread Michel Lemay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michel Lemay updated SPARK-23961: - Issue Type: Bug (was: Improvement) > pyspark toLocalIterator throws an exception >

[jira] [Created] (SPARK-23961) pyspark toLocalIterator throws an exception

2018-04-11 Thread Michel Lemay (JIRA)
Michel Lemay created SPARK-23961: Summary: pyspark toLocalIterator throws an exception Key: SPARK-23961 URL: https://issues.apache.org/jira/browse/SPARK-23961 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-23930) High-order function: slice(x, start, length) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23930: Assignee: Apache Spark > High-order function: slice(x, start, length) → array >

[jira] [Commented] (SPARK-23930) High-order function: slice(x, start, length) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433837#comment-16433837 ] Apache Spark commented on SPARK-23930: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23930) High-order function: slice(x, start, length) → array

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23930: Assignee: (was: Apache Spark) > High-order function: slice(x, start, length) → array

[jira] [Resolved] (SPARK-23951) Use java classed in ExprValue and simplify a bunch of stuff

2018-04-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23951. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21026

[jira] [Assigned] (SPARK-23960) Mark HashAggregateExec.bufVars as transient

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23960: Assignee: Apache Spark > Mark HashAggregateExec.bufVars as transient >

[jira] [Assigned] (SPARK-23960) Mark HashAggregateExec.bufVars as transient

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23960: Assignee: (was: Apache Spark) > Mark HashAggregateExec.bufVars as transient >

[jira] [Commented] (SPARK-23960) Mark HashAggregateExec.bufVars as transient

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433672#comment-16433672 ] Apache Spark commented on SPARK-23960: -- User 'rednaxelafx' has created a pull request for this

[jira] [Created] (SPARK-23960) Mark HashAggregateExec.bufVars as transient

2018-04-11 Thread Kris Mok (JIRA)
Kris Mok created SPARK-23960: Summary: Mark HashAggregateExec.bufVars as transient Key: SPARK-23960 URL: https://issues.apache.org/jira/browse/SPARK-23960 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-23959) UnresolvedException with DataSet created from Seq.empty since Spark 2.3.0

2018-04-11 Thread Sam De Backer (JIRA)
Sam De Backer created SPARK-23959: - Summary: UnresolvedException with DataSet created from Seq.empty since Spark 2.3.0 Key: SPARK-23959 URL: https://issues.apache.org/jira/browse/SPARK-23959 Project:

[jira] [Commented] (SPARK-23930) High-order function: slice(x, start, length) → array

2018-04-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433550#comment-16433550 ] Marco Gaido commented on SPARK-23930: - I am working on this. > High-order function: slice(x, start,

[jira] [Commented] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2

2018-04-11 Thread Jepson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433536#comment-16433536 ] Jepson commented on SPARK-22968: [~apachespark] Thank you very much.  > java.lang.IllegalStateException:

[jira] [Assigned] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22968: Assignee: (was: Apache Spark) > java.lang.IllegalStateException: No current

[jira] [Assigned] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22968: Assignee: Apache Spark > java.lang.IllegalStateException: No current assignment for

[jira] [Commented] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433510#comment-16433510 ] Apache Spark commented on SPARK-22968: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23919) High-order function: array_position(x, element) → bigint

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23919: Assignee: Apache Spark > High-order function: array_position(x, element) → bigint >

[jira] [Commented] (SPARK-23919) High-order function: array_position(x, element) → bigint

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433460#comment-16433460 ] Apache Spark commented on SPARK-23919: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23919) High-order function: array_position(x, element) → bigint

2018-04-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23919: Assignee: (was: Apache Spark) > High-order function: array_position(x, element) →