[jira] [Commented] (SPARK-23928) High-order function: shuffle(x) → array

[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433406#comment-16433406 ] Liang-Chi Hsieh commented on SPARK-23928: - If no assignee and no one announces, it is no problem

[jira] [Assigned] (SPARK-23958) HadoopRdd filters empty files to avoid generating empty tasks that affect the performance of the Spark computing performance.

[ https://issues.apache.org/jira/browse/SPARK-23958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23958: Assignee: Apache Spark > HadoopRdd filters empty files to avoid generating empty tasks

[jira] [Assigned] (SPARK-23958) HadoopRdd filters empty files to avoid generating empty tasks that affect the performance of the Spark computing performance.

[ https://issues.apache.org/jira/browse/SPARK-23958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23958: Assignee: (was: Apache Spark) > HadoopRdd filters empty files to avoid generating

[jira] [Commented] (SPARK-23958) HadoopRdd filters empty files to avoid generating empty tasks that affect the performance of the Spark computing performance.

[ https://issues.apache.org/jira/browse/SPARK-23958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1640#comment-1640 ] Apache Spark commented on SPARK-23958: -- User 'guoxiaolongzte' has created a pull request for this

[jira] [Created] (SPARK-23958) HadoopRdd filters empty files to avoid generating empty tasks that affect the performance of the Spark computing performance.

guoxiaolongzte created SPARK-23958: -- Summary: HadoopRdd filters empty files to avoid generating empty tasks that affect the performance of the Spark computing performance. Key: SPARK-23958 URL:

[jira] [Updated] (SPARK-23955) typo in parameter name 'rawPredicition'

[ https://issues.apache.org/jira/browse/SPARK-23955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23955: - Priority: Trivial (was: Minor) > typo in parameter name 'rawPredicition' >

[jira] [Commented] (SPARK-23945) Column.isin() should accept a single-column DataFrame as input

[ https://issues.apache.org/jira/browse/SPARK-23945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433316#comment-16433316 ] Nicholas Chammas commented on SPARK-23945: -- I always looked at DataFrames and SQL as two

[jira] [Commented] (SPARK-23847) Add asc_nulls_first, asc_nulls_last to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433314#comment-16433314 ] Apache Spark commented on SPARK-23847: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-23955) typo in parameter name 'rawPredicition'

[ https://issues.apache.org/jira/browse/SPARK-23955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433302#comment-16433302 ] Hyukjin Kwon commented on SPARK-23955: -- Fixing a typo doesn't need a JIRA. Let's avoid this next

[jira] [Commented] (SPARK-23954) Converting spark dataframe containing int64 fields to R dataframes leads to impredictable errors.

[ https://issues.apache.org/jira/browse/SPARK-23954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433298#comment-16433298 ] Hyukjin Kwon commented on SPARK-23954: -- Can you check other JIRAs and see if there are duplicates? I

[jira] [Commented] (SPARK-23950) Coalescing an empty dataframe to 1 partition

[ https://issues.apache.org/jira/browse/SPARK-23950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433296#comment-16433296 ] Hyukjin Kwon commented on SPARK-23950: -- Seems fixed in the current master. Let me leave this

[jira] [Resolved] (SPARK-19947) RFormulaModel always throws Exception on transforming data with NULL or Unseen labels

[ https://issues.apache.org/jira/browse/SPARK-19947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-19947. --- Resolution: Fixed Fix Version/s: 2.4.0 I'll mark this as complete. Those

[jira] [Resolved] (SPARK-23562) RFormula handleInvalid should handle invalid values in non-string columns.

[ https://issues.apache.org/jira/browse/SPARK-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-23562. --- Resolution: Fixed Fix Version/s: 2.4.0 I think everything has been fixed, so

[jira] [Updated] (SPARK-23562) RFormula handleInvalid should handle invalid values in non-string columns.

[ https://issues.apache.org/jira/browse/SPARK-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23562: -- Shepherd: Joseph K. Bradley > RFormula handleInvalid should handle invalid values in

[jira] [Commented] (SPARK-23337) withWatermark raises an exception on struct objects

[ https://issues.apache.org/jira/browse/SPARK-23337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433222#comment-16433222 ] Michael Armbrust commented on SPARK-23337: -- The checkpoint will only grow if you are doing an

[jira] [Resolved] (SPARK-23944) Add Param set functions to LSHModel types

[ https://issues.apache.org/jira/browse/SPARK-23944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-23944. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21015

[jira] [Assigned] (SPARK-23944) Add Param set functions to LSHModel types

[ https://issues.apache.org/jira/browse/SPARK-23944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-23944: - Assignee: Lu Wang > Add Param set functions to LSHModel types >

[jira] [Created] (SPARK-23957) Sorts in subqueries are redundant and can be removed

Henry Robinson created SPARK-23957: -- Summary: Sorts in subqueries are redundant and can be removed Key: SPARK-23957 URL: https://issues.apache.org/jira/browse/SPARK-23957 Project: Spark

[jira] [Resolved] (SPARK-23871) add python api for VectorAssembler handleInvalid

[ https://issues.apache.org/jira/browse/SPARK-23871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-23871. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21003

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433100#comment-16433100 ] Nicholas Verbeck commented on SPARK-19680: -- KAFKA-3370 is a good solution to the bad preforming

[jira] [Created] (SPARK-23956) Use effective RPC port in AM registration

Gera Shegalov created SPARK-23956: - Summary: Use effective RPC port in AM registration Key: SPARK-23956 URL: https://issues.apache.org/jira/browse/SPARK-23956 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23871) add python api for VectorAssembler handleInvalid

[ https://issues.apache.org/jira/browse/SPARK-23871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23871: -- Shepherd: Joseph K. Bradley > add python api for VectorAssembler handleInvalid >

[jira] [Assigned] (SPARK-23871) add python api for VectorAssembler handleInvalid

[ https://issues.apache.org/jira/browse/SPARK-23871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-23871: - Assignee: Huaxin Gao > add python api for VectorAssembler handleInvalid >

[jira] [Closed] (SPARK-23869) Spark 2.3.0 left outer join not emitting null values instead waiting for the record in other stream

[ https://issues.apache.org/jira/browse/SPARK-23869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bharath kumar avusherla closed SPARK-23869. --- > Spark 2.3.0 left outer join not emitting null values instead waiting for the

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433039#comment-16433039 ] Cody Koeninger commented on SPARK-19680: [~nerdynick]  If you submit a PR to add documentation

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433001#comment-16433001 ] Nicholas Verbeck commented on SPARK-19680: -- I just spent way to long on this. Thought I was

[jira] [Commented] (SPARK-23926) High-order function: reverse(x) → array

[ https://issues.apache.org/jira/browse/SPARK-23926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432982#comment-16432982 ] Apache Spark commented on SPARK-23926: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23926) High-order function: reverse(x) → array

[ https://issues.apache.org/jira/browse/SPARK-23926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23926: Assignee: (was: Apache Spark) > High-order function: reverse(x) → array >

[jira] [Assigned] (SPARK-23926) High-order function: reverse(x) → array

[ https://issues.apache.org/jira/browse/SPARK-23926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23926: Assignee: Apache Spark > High-order function: reverse(x) → array >

[jira] [Commented] (SPARK-20865) caching dataset throws "Queries with streaming sources must be executed with writeStream.start()"

[ https://issues.apache.org/jira/browse/SPARK-20865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432969#comment-16432969 ] hamroune zahir commented on SPARK-20865: it is huge regression, on that sens we cannot get HEAD

[jira] [Commented] (SPARK-23931) High-order function: zip(array1, array2[, ...]) → array

[ https://issues.apache.org/jira/browse/SPARK-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432850#comment-16432850 ] Dylan Guedes commented on SPARK-23931: -- I would like to try this one. > High-order function:

[jira] [Created] (SPARK-23955) typo in parameter name 'rawPredicition'

John Bauer created SPARK-23955: -- Summary: typo in parameter name 'rawPredicition' Key: SPARK-23955 URL: https://issues.apache.org/jira/browse/SPARK-23955 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-23954) Converting spark dataframe containing int64 fields to R dataframes leads to impredictable errors.

nicolas paris created SPARK-23954: - Summary: Converting spark dataframe containing int64 fields to R dataframes leads to impredictable errors. Key: SPARK-23954 URL:

[jira] [Commented] (SPARK-19320) Allow guaranteed amount of GPU to be used when launching jobs

[ https://issues.apache.org/jira/browse/SPARK-19320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432796#comment-16432796 ] Apache Spark commented on SPARK-19320: -- User 'yanji84' has created a pull request for this issue:

[jira] [Commented] (SPARK-23912) High-order function: array_distinct(x) → array

[ https://issues.apache.org/jira/browse/SPARK-23912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432780#comment-16432780 ] Huaxin Gao commented on SPARK-23912: I will work on this. Thanks! > High-order function:

[jira] [Updated] (SPARK-21856) Update Python API for MultilayerPerceptronClassifierModel

[ https://issues.apache.org/jira/browse/SPARK-21856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21856: -- Fix Version/s: 2.3.0 > Update Python API for MultilayerPerceptronClassifierModel >

[jira] [Commented] (SPARK-23529) Specify hostpath volume and mount the volume in Spark driver and executor pods in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-23529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432746#comment-16432746 ] Apache Spark commented on SPARK-23529: -- User 'madanadit' has created a pull request for this issue:

[jira] [Commented] (SPARK-23928) High-order function: shuffle(x) → array

[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432739#comment-16432739 ] H Lu commented on SPARK-23928: -- Can I take this one? > High-order function: shuffle(x) → array >

[jira] [Resolved] (SPARK-8571) spark streaming hanging processes upon build exit

[ https://issues.apache.org/jira/browse/SPARK-8571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp resolved SPARK-8571. Resolution: Not A Problem > spark streaming hanging processes upon build exit >

[jira] [Assigned] (SPARK-23751) Kolmogorov-Smirnoff test Python API in pyspark.ml

[ https://issues.apache.org/jira/browse/SPARK-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-23751: - Assignee: Weichen Xu > Kolmogorov-Smirnoff test Python API in pyspark.ml >

[jira] [Resolved] (SPARK-23751) Kolmogorov-Smirnoff test Python API in pyspark.ml

[ https://issues.apache.org/jira/browse/SPARK-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-23751. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20904

[jira] [Commented] (SPARK-8571) spark streaming hanging processes upon build exit

[ https://issues.apache.org/jira/browse/SPARK-8571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432725#comment-16432725 ] shane knapp commented on SPARK-8571: just doing some email archaeology and found this. no, it's not

[jira] [Commented] (SPARK-8696) Streaming API for Online LDA

[ https://issues.apache.org/jira/browse/SPARK-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432686#comment-16432686 ] Joey Frazee commented on SPARK-8696: Is there still interest in this? The two use cases I've seen for

[jira] [Assigned] (SPARK-23923) High-order function: cardinality(x) → bigint

[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23923: Assignee: Apache Spark > High-order function: cardinality(x) → bigint >

[jira] [Assigned] (SPARK-23923) High-order function: cardinality(x) → bigint

[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23923: Assignee: (was: Apache Spark) > High-order function: cardinality(x) → bigint >

[jira] [Commented] (SPARK-23923) High-order function: cardinality(x) → bigint

[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432584#comment-16432584 ] Apache Spark commented on SPARK-23923: -- User 'kiszk' has created a pull request for this issue:

[jira] [Created] (SPARK-23953) Add get_json_scalar function

Timothy Chen created SPARK-23953: Summary: Add get_json_scalar function Key: SPARK-23953 URL: https://issues.apache.org/jira/browse/SPARK-23953 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432495#comment-16432495 ] Attila Zsolt Piros commented on SPARK-16630: Let me illustrate my problem with an example: -

[jira] [Assigned] (SPARK-23952) remove type parameter in DataReaderFactory

[ https://issues.apache.org/jira/browse/SPARK-23952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23952: Assignee: Wenchen Fan (was: Apache Spark) > remove type parameter in DataReaderFactory >

[jira] [Commented] (SPARK-23952) remove type parameter in DataReaderFactory

[ https://issues.apache.org/jira/browse/SPARK-23952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432472#comment-16432472 ] Apache Spark commented on SPARK-23952: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23864) Add Unsafe* copy methods to UnsafeWriter

[ https://issues.apache.org/jira/browse/SPARK-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23864. --- Resolution: Fixed Fix Version/s: 2.4.0 > Add Unsafe* copy methods to

[jira] [Assigned] (SPARK-23952) remove type parameter in DataReaderFactory

[ https://issues.apache.org/jira/browse/SPARK-23952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23952: Assignee: Apache Spark (was: Wenchen Fan) > remove type parameter in DataReaderFactory >

[jira] [Created] (SPARK-23952) remove type parameter in DataReaderFactory

Wenchen Fan created SPARK-23952: --- Summary: remove type parameter in DataReaderFactory Key: SPARK-23952 URL: https://issues.apache.org/jira/browse/SPARK-23952 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432467#comment-16432467 ] Thomas Graves commented on SPARK-16630: --- sorry I don't follow, the list we get from the blacklist

[jira] [Commented] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432463#comment-16432463 ] Ed Lee commented on SPARK-20617: Thank you for the clarification.  So conversely: {code:java}

[jira] [Assigned] (SPARK-23922) High-order function: arrays_overlap(x, y) → boolean

[ https://issues.apache.org/jira/browse/SPARK-23922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23922: Assignee: (was: Apache Spark) > High-order function: arrays_overlap(x, y) → boolean >

[jira] [Assigned] (SPARK-23922) High-order function: arrays_overlap(x, y) → boolean

[ https://issues.apache.org/jira/browse/SPARK-23922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23922: Assignee: Apache Spark > High-order function: arrays_overlap(x, y) → boolean >

[jira] [Commented] (SPARK-23922) High-order function: arrays_overlap(x, y) → boolean

[ https://issues.apache.org/jira/browse/SPARK-23922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432423#comment-16432423 ] Apache Spark commented on SPARK-23922: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432415#comment-16432415 ] Attila Zsolt Piros commented on SPARK-16630: I would need the expiry times to choose the most

[jira] [Assigned] (SPARK-23943) Improve observability of MesosRestServer/MesosClusterDispatcher

[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23943: Assignee: (was: Apache Spark) > Improve observability of

[jira] [Assigned] (SPARK-23943) Improve observability of MesosRestServer/MesosClusterDispatcher

[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23943: Assignee: Apache Spark > Improve observability of MesosRestServer/MesosClusterDispatcher

[jira] [Commented] (SPARK-23943) Improve observability of MesosRestServer/MesosClusterDispatcher

[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432379#comment-16432379 ] Apache Spark commented on SPARK-23943: -- User 'pmackles' has created a pull request for this issue:

[jira] [Commented] (SPARK-23951) Use java classed in ExprValue and simplify a bunch of stuff

[ https://issues.apache.org/jira/browse/SPARK-23951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432375#comment-16432375 ] Apache Spark commented on SPARK-23951: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23951) Use java classed in ExprValue and simplify a bunch of stuff

[ https://issues.apache.org/jira/browse/SPARK-23951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23951: Assignee: Herman van Hovell (was: Apache Spark) > Use java classed in ExprValue and

[jira] [Assigned] (SPARK-23951) Use java classed in ExprValue and simplify a bunch of stuff

[ https://issues.apache.org/jira/browse/SPARK-23951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23951: Assignee: Apache Spark (was: Herman van Hovell) > Use java classed in ExprValue and

[jira] [Updated] (SPARK-23888) speculative task should not run on a given host where another attempt is already running on

[ https://issues.apache.org/jira/browse/SPARK-23888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23888: - Labels: speculation (was: ) > speculative task should not run on a given host where another

[jira] [Updated] (SPARK-23888) speculative task should not run on a given host where another attempt is already running on

[ https://issues.apache.org/jira/browse/SPARK-23888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23888: - Component/s: Scheduler > speculative task should not run on a given host where another attempt

[jira] [Comment Edited] (SPARK-23929) pandas_udf schema mapped by position and not by name

[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432344#comment-16432344 ] Omri edited comment on SPARK-23929 at 4/10/18 2:22 PM: --- [~icexelloss], I couldn't

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432344#comment-16432344 ] Omri commented on SPARK-23929: -- [~icexelloss], I couldn't recreate the problem I had where the order was

[jira] [Commented] (SPARK-23884) hasLaunchedTask should be true when launchedAnyTask be true

[ https://issues.apache.org/jira/browse/SPARK-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432323#comment-16432323 ] Yu Wang commented on SPARK-23884: - [~Ngone51]I did not mention the patch and wanted to mention one >

[jira] [Resolved] (SPARK-23841) NodeIdCache should unpersist the last cached nodeIdsForInstances

[ https://issues.apache.org/jira/browse/SPARK-23841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23841. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20956

[jira] [Assigned] (SPARK-23841) NodeIdCache should unpersist the last cached nodeIdsForInstances

[ https://issues.apache.org/jira/browse/SPARK-23841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23841: - Assignee: zhengruifeng > NodeIdCache should unpersist the last cached nodeIdsForInstances >

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432244#comment-16432244 ] Thomas Graves commented on SPARK-16630: --- yes I think it would make sense as the union of all

[jira] [Commented] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432207#comment-16432207 ] Kingsley Jones commented on SPARK-12216: scala> val loader =

[jira] [Commented] (SPARK-23337) withWatermark raises an exception on struct objects

[ https://issues.apache.org/jira/browse/SPARK-23337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432200#comment-16432200 ] Aydin Kocas commented on SPARK-23337: - Hi Michael, in my case it's a blocking issue and unfortunately

[jira] [Updated] (SPARK-23943) Improve observability of MesosRestServer/MesosClusterDispatcher

[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-23943: - Description: Two changes in this PR: * A /health endpoint for a quick binary indication on the

[jira] [Comment Edited] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432165#comment-16432165 ] Attila Zsolt Piros edited comment on SPARK-16630 at 4/10/18 12:15 PM:

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432165#comment-16432165 ] Attila Zsolt Piros commented on SPARK-16630: I have question regarding limiting the number of

[jira] [Commented] (SPARK-23884) hasLaunchedTask should be true when launchedAnyTask be true

[ https://issues.apache.org/jira/browse/SPARK-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432149#comment-16432149 ] wuyi commented on SPARK-23884: -- [~gentlewang] why？ > hasLaunchedTask should be true when launchedAnyTask be

[jira] [Updated] (SPARK-23705) dataframe.groupBy() may inadvertently receive sequence of non-distinct strings

[ https://issues.apache.org/jira/browse/SPARK-23705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Wang updated SPARK-23705: Attachment: SPARK-23705.patch > dataframe.groupBy() may inadvertently receive sequence of non-distinct

[jira] [Commented] (SPARK-23705) dataframe.groupBy() may inadvertently receive sequence of non-distinct strings

[ https://issues.apache.org/jira/browse/SPARK-23705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432142#comment-16432142 ] Yu Wang commented on SPARK-23705: - [~khoatrantan2000] Could you assign this patch to me? >

[jira] [Comment Edited] (SPARK-23884) hasLaunchedTask should be true when launchedAnyTask be true

[ https://issues.apache.org/jira/browse/SPARK-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432118#comment-16432118 ] Yu Wang edited comment on SPARK-23884 at 4/10/18 11:47 AM: --- [~Ngone51]Could you

[jira] [Commented] (SPARK-23922) High-order function: arrays_overlap(x, y) → boolean

[ https://issues.apache.org/jira/browse/SPARK-23922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432124#comment-16432124 ] Marco Gaido commented on SPARK-23922: - I will work on this. > High-order function: arrays_overlap(x,

[jira] [Commented] (SPARK-23918) High-order function: array_min(x) → x

[ https://issues.apache.org/jira/browse/SPARK-23918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432119#comment-16432119 ] Apache Spark commented on SPARK-23918: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23918) High-order function: array_min(x) → x

[ https://issues.apache.org/jira/browse/SPARK-23918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23918: Assignee: (was: Apache Spark) > High-order function: array_min(x) → x >

[jira] [Assigned] (SPARK-23918) High-order function: array_min(x) → x

[ https://issues.apache.org/jira/browse/SPARK-23918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23918: Assignee: Apache Spark > High-order function: array_min(x) → x >

[jira] [Updated] (SPARK-23884) hasLaunchedTask should be true when launchedAnyTask be true

[ https://issues.apache.org/jira/browse/SPARK-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Wang updated SPARK-23884: Attachment: SPARK-23884.patch > hasLaunchedTask should be true when launchedAnyTask be true >

[jira] [Commented] (SPARK-23884) hasLaunchedTask should be true when launchedAnyTask be true

[ https://issues.apache.org/jira/browse/SPARK-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432118#comment-16432118 ] Yu Wang commented on SPARK-23884: - [~Ngone51]Can you assign this task to me? > hasLaunchedTask should be

[jira] [Updated] (SPARK-23948) Trigger mapstage's job listener in submitMissingTasks

[ https://issues.apache.org/jira/browse/SPARK-23948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated SPARK-23948: - Description: SparkContext submitted a map stage from "submitMapStage" to DAGScheduler, 

[jira] [Created] (SPARK-23951) Use java classed in ExprValue and simplify a bunch of stuff

Herman van Hovell created SPARK-23951: - Summary: Use java classed in ExprValue and simplify a bunch of stuff Key: SPARK-23951 URL: https://issues.apache.org/jira/browse/SPARK-23951 Project: Spark

[jira] [Commented] (SPARK-23917) High-order function: array_max(x) → x

[ https://issues.apache.org/jira/browse/SPARK-23917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432084#comment-16432084 ] Apache Spark commented on SPARK-23917: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23917) High-order function: array_max(x) → x

[ https://issues.apache.org/jira/browse/SPARK-23917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23917: Assignee: (was: Apache Spark) > High-order function: array_max(x) → x >

[jira] [Assigned] (SPARK-23917) High-order function: array_max(x) → x

[ https://issues.apache.org/jira/browse/SPARK-23917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23917: Assignee: Apache Spark > High-order function: array_max(x) → x >

[jira] [Assigned] (SPARK-23949) makes "&&" supports the function of predicate operator "and"

[ https://issues.apache.org/jira/browse/SPARK-23949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23949: Assignee: Apache Spark > makes "&&" supports the function of predicate operator "and" >

[jira] [Assigned] (SPARK-23949) makes "&&" supports the function of predicate operator "and"

[ https://issues.apache.org/jira/browse/SPARK-23949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23949: Assignee: (was: Apache Spark) > makes "&&" supports the function of predicate

[jira] [Commented] (SPARK-23949) makes "&&" supports the function of predicate operator "and"

[ https://issues.apache.org/jira/browse/SPARK-23949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432040#comment-16432040 ] Apache Spark commented on SPARK-23949: -- User 'httfighter' has created a pull request for this issue:

[jira] [Commented] (SPARK-23945) Column.isin() should accept a single-column DataFrame as input

[ https://issues.apache.org/jira/browse/SPARK-23945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432039#comment-16432039 ] Herman van Hovell commented on SPARK-23945: --- [~nchammas] we didn't add explicit dataset support

[jira] [Created] (SPARK-23950) Coalescing an empty dataframe to 1 partition

João Neves created SPARK-23950: -- Summary: Coalescing an empty dataframe to 1 partition Key: SPARK-23950 URL: https://issues.apache.org/jira/browse/SPARK-23950 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-23949) makes "&&" supports the function of predicate operator "and"

hantiantian created SPARK-23949: --- Summary: makes "&&" supports the function of predicate operator "and" Key: SPARK-23949 URL: https://issues.apache.org/jira/browse/SPARK-23949 Project: Spark

[jira] [Commented] (SPARK-23916) High-order function: array_join(x, delimiter, null_replacement) → varchar

[ https://issues.apache.org/jira/browse/SPARK-23916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431795#comment-16431795 ] Kazuaki Ishizaki commented on SPARK-23916: -- Sorry for my mistake regarding a PR with wrong JIRA

  1   2   >