[jira] [Commented] (SPARK-24612) Running into "Py4JJavaError" while converting list to Dataframe using Pyspark, Jupyter notebook

2018-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518994#comment-16518994 ] Hyukjin Kwon commented on SPARK-24612: -- Please refer this https://spark.apache.org/

[jira] [Assigned] (SPARK-24614) PySpark - Fix SyntaxWarning on tests.py

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24614: Assignee: Apache Spark > PySpark - Fix SyntaxWarning on tests.py > --

[jira] [Updated] (SPARK-24614) PySpark - Fix SyntaxWarning on tests.py

2018-06-20 Thread Rekha Joshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated SPARK-24614: Description: Pyspark - Fix SyntaxWarning on tests.py (was: Pyspark - Fix for SyntaxWarning on tes

[jira] [Assigned] (SPARK-24614) PySpark - Fix SyntaxWarning on tests.py

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24614: Assignee: (was: Apache Spark) > PySpark - Fix SyntaxWarning on tests.py > ---

[jira] [Updated] (SPARK-24614) PySpark - Fix SyntaxWarning on tests.py

2018-06-20 Thread Rekha Joshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated SPARK-24614: Description: Pyspark - Fix for SyntaxWarning on tests.py (was: Pyspark tests.py codestyle correct

[jira] [Commented] (SPARK-24614) PySpark - Fix SyntaxWarning on tests.py

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518991#comment-16518991 ] Apache Spark commented on SPARK-24614: -- User 'rekhajoshm' has created a pull reques

[jira] [Updated] (SPARK-24614) PySpark - Fix SyntaxWarning on tests.py

2018-06-20 Thread Rekha Joshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated SPARK-24614: Summary: PySpark - Fix SyntaxWarning on tests.py (was: Pyspark tests.py codestyle correction) >

[jira] [Resolved] (SPARK-24571) Support literals with values of the Char type

2018-06-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24571. - Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 2.4.0 > Support literals with valu

[jira] [Created] (SPARK-24614) Pyspark tests.py codestyle correction

2018-06-20 Thread Rekha Joshi (JIRA)
Rekha Joshi created SPARK-24614: --- Summary: Pyspark tests.py codestyle correction Key: SPARK-24614 URL: https://issues.apache.org/jira/browse/SPARK-24614 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24612) Running into "Py4JJavaError" while converting list to Dataframe using Pyspark, Jupyter notebook

2018-06-20 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518982#comment-16518982 ] A B commented on SPARK-24612: - [~hyukjin.kwon]  , What is the procedure to ask dev mailing l

[jira] [Commented] (SPARK-14410) SessionCatalog needs to check function existence

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518964#comment-16518964 ] Apache Spark commented on SPARK-14410: -- User 'rekhajoshm' has created a pull reques

[jira] [Resolved] (SPARK-24612) Running into "Py4JJavaError" while converting list to Dataframe using Pyspark, Jupyter notebook

2018-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24612. -- Resolution: Invalid Let's ask the question into dev mailing list and file a JIRA when we are c

[jira] [Commented] (SPARK-17091) Convert IN predicate to equivalent Parquet filter

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518907#comment-16518907 ] Apache Spark commented on SPARK-17091: -- User 'wangyum' has created a pull request f

[jira] [Resolved] (SPARK-23912) High-order function: array_distinct(x) → array

2018-06-20 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23912. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21050 https://g

[jira] [Assigned] (SPARK-23912) High-order function: array_distinct(x) → array

2018-06-20 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-23912: - Assignee: Huaxin Gao > High-order function: array_distinct(x) → array > ---

[jira] [Resolved] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-20 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan resolved SPARK-24547. Resolution: Fixed Fix Version/s: 2.4.0 > Spark on K8s docker-image-tool.sh

[jira] [Commented] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-20 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518738#comment-16518738 ] Anirudh Ramanathan commented on SPARK-24547: Resolved by https://github.com/

[jira] [Assigned] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-20 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan reassigned SPARK-24547: -- Assignee: (was: Anirudh Ramanathan) > Spark on K8s docker-image-tool.sh i

[jira] [Assigned] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-20 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan reassigned SPARK-24547: -- Assignee: Anirudh Ramanathan > Spark on K8s docker-image-tool.sh improvements

[jira] [Updated] (SPARK-24613) Cache with UDF could not be matched with subsequent dependent caches

2018-06-20 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-24613: Description: When caching a query, we generate its execution plan from the query's logical plan.

[jira] [Assigned] (SPARK-24613) Cache with UDF could not be matched with subsequent dependent caches

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24613: Assignee: (was: Apache Spark) > Cache with UDF could not be matched with subsequent d

[jira] [Commented] (SPARK-24613) Cache with UDF could not be matched with subsequent dependent caches

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518711#comment-16518711 ] Apache Spark commented on SPARK-24613: -- User 'maryannxue' has created a pull reques

[jira] [Assigned] (SPARK-24613) Cache with UDF could not be matched with subsequent dependent caches

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24613: Assignee: Apache Spark > Cache with UDF could not be matched with subsequent dependent ca

[jira] [Created] (SPARK-24613) Cache with UDF could not be matched with subsequent dependent caches

2018-06-20 Thread Maryann Xue (JIRA)
Maryann Xue created SPARK-24613: --- Summary: Cache with UDF could not be matched with subsequent dependent caches Key: SPARK-24613 URL: https://issues.apache.org/jira/browse/SPARK-24613 Project: Spark

[jira] [Updated] (SPARK-24613) Cache with UDF could not be matched with subsequent dependent caches

2018-06-20 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-24613: Priority: Minor (was: Major) > Cache with UDF could not be matched with subsequent dependent cach

[jira] [Updated] (SPARK-24613) Cache with UDF could not be matched with subsequent dependent caches

2018-06-20 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-24613: Issue Type: Bug (was: Improvement) > Cache with UDF could not be matched with subsequent dependen

[jira] [Updated] (SPARK-24612) Running into "Py4JJavaError" while converting list to Dataframe using Pyspark, Jupyter notebook

2018-06-20 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A B updated SPARK-24612: Environment: >python --version Python 3.6.5 :: Anaconda, Inc. >java -version java version "1.8.0_144" Java(TM) SE

[jira] [Updated] (SPARK-24612) Running into "Py4JJavaError" while converting list to Dataframe using Pyspark, Jupyter notebook

2018-06-20 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A B updated SPARK-24612: Summary: Running into "Py4JJavaError" while converting list to Dataframe using Pyspark, Jupyter notebook (was: Ru

[jira] [Created] (SPARK-24612) Running into "Py4JJavaError" while converting list to Dataframe using Python, Jupyter notebook

2018-06-20 Thread A B (JIRA)
A B created SPARK-24612: --- Summary: Running into "Py4JJavaError" while converting list to Dataframe using Python, Jupyter notebook Key: SPARK-24612 URL: https://issues.apache.org/jira/browse/SPARK-24612 Project:

[jira] [Assigned] (SPARK-24578) Reading remote cache block behavior changes and causes timeout issue

2018-06-20 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-24578: Assignee: Wenbo Zhao > Reading remote cache block behavior changes and causes timeout iss

[jira] [Resolved] (SPARK-24578) Reading remote cache block behavior changes and causes timeout issue

2018-06-20 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-24578. -- Resolution: Fixed Fix Version/s: 2.3.2 Issue resolved by pull request 21593 [https://gi

[jira] [Updated] (SPARK-24578) Reading remote cache block behavior changes and causes timeout issue

2018-06-20 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-24578: - Fix Version/s: 2.4.0 > Reading remote cache block behavior changes and causes timeout issue > --

[jira] [Assigned] (SPARK-24610) wholeTextFiles broken for small files

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24610: Assignee: (was: Apache Spark) > wholeTextFiles broken for small files > -

[jira] [Commented] (SPARK-24610) wholeTextFiles broken for small files

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518475#comment-16518475 ] Apache Spark commented on SPARK-24610: -- User 'dhruve' has created a pull request fo

[jira] [Assigned] (SPARK-24610) wholeTextFiles broken for small files

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24610: Assignee: Apache Spark > wholeTextFiles broken for small files >

[jira] [Created] (SPARK-24611) Clean up OutputCommitCoordinator

2018-06-20 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-24611: -- Summary: Clean up OutputCommitCoordinator Key: SPARK-24611 URL: https://issues.apache.org/jira/browse/SPARK-24611 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-19480) Higher order functions in SQL

2018-06-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19480. - Resolution: Duplicate Target Version/s: (was: 2.4.0, 3.0.0) > Higher order functions

[jira] [Resolved] (SPARK-24575) Prohibit window expressions inside WHERE and HAVING clauses

2018-06-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-24575. --- Resolution: Fixed Assignee: Anton Okolnychyi > Prohibit window expressions ins

[jira] [Created] (SPARK-24610) wholeTextFiles broken for small files

2018-06-20 Thread Dhruve Ashar (JIRA)
Dhruve Ashar created SPARK-24610: Summary: wholeTextFiles broken for small files Key: SPARK-24610 URL: https://issues.apache.org/jira/browse/SPARK-24610 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24578) Reading remote cache block behavior changes and causes timeout issue

2018-06-20 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518300#comment-16518300 ] Imran Rashid commented on SPARK-24578: -- Given the severity of the issue and that it

[jira] [Updated] (SPARK-24578) Reading remote cache block behavior changes and causes timeout issue

2018-06-20 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-24578: - Priority: Blocker (was: Major) > Reading remote cache block behavior changes and causes timeout

[jira] [Updated] (SPARK-24553) Job UI redirect causing http 302 error

2018-06-20 Thread Steven Kallman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Kallman updated SPARK-24553: --- Description: When on spark UI port 4040 jobs or stages tab, the href links for the individu

[jira] [Commented] (SPARK-24507) Description in "Level of Parallelism in Data Receiving" section of Spark Streaming Programming Guide in is not relevan for the recent Kafka direct apprach

2018-06-20 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518287#comment-16518287 ] Cody Koeninger commented on SPARK-24507: You're welcome to submit a doc PR that

[jira] [Assigned] (SPARK-24553) Job UI redirect causing http 302 error

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24553: Assignee: (was: Apache Spark) > Job UI redirect causing http 302 error >

[jira] [Commented] (SPARK-24553) Job UI redirect causing http 302 error

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518286#comment-16518286 ] Apache Spark commented on SPARK-24553: -- User 'SJKallman' has created a pull request

[jira] [Assigned] (SPARK-24553) Job UI redirect causing http 302 error

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24553: Assignee: Apache Spark > Job UI redirect causing http 302 error > ---

[jira] [Updated] (SPARK-24609) PySpark doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

2018-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24609: -- Description: In Scala doc ([https://spark.apache.org/docs/2.3.0/api/scala/index.html#org.apac

[jira] [Updated] (SPARK-24609) PySpark/SparkR doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

2018-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24609: -- Summary: PySpark/SparkR doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

[jira] [Created] (SPARK-24609) PySpark doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

2018-06-20 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-24609: - Summary: PySpark doc doesn't explain RandomForestClassifier.featureSubsetStrategy well Key: SPARK-24609 URL: https://issues.apache.org/jira/browse/SPARK-24609 Proje

[jira] [Commented] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread zenglinxi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518230#comment-16518230 ] zenglinxi commented on SPARK-24607: --- [~viirya]  I have some tests, it seems like rand

[jira] [Updated] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread zenglinxi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zenglinxi updated SPARK-24607: -- Description: Noticed the following queries can give different results: {code:java} select count(*) fro

[jira] [Updated] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread zenglinxi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zenglinxi updated SPARK-24607: -- Description: Noticed the following queries can give different results: {code:java} select count(*) fro

[jira] [Comment Edited] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518206#comment-16518206 ] Liang-Chi Hsieh edited comment on SPARK-24607 at 6/20/18 2:40 PM:

[jira] [Commented] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518206#comment-16518206 ] Liang-Chi Hsieh commented on SPARK-24607: - Thanks [~mgaido]! As I check {{Rand}

[jira] [Commented] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518196#comment-16518196 ] Marco Gaido commented on SPARK-24607: - [~viirya] please check the description in the

[jira] [Commented] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518184#comment-16518184 ] Liang-Chi Hsieh commented on SPARK-24607: - >From the following test, looks it is

[jira] [Updated] (SPARK-24578) Reading remote cache block behavior changes and causes timeout issue

2018-06-20 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-24578: - Target Version/s: 2.3.2, 2.4.0 > Reading remote cache block behavior changes and causes timeout

[jira] [Created] (SPARK-24608) report number of iteration/progress for ML training

2018-06-20 Thread R (JIRA)
R created SPARK-24608: - Summary: report number of iteration/progress for ML training Key: SPARK-24608 URL: https://issues.apache.org/jira/browse/SPARK-24608 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24606: Priority: Major (was: Blocker) > Decimals multiplication and division may be null due to the resu

[jira] [Commented] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518130#comment-16518130 ] Marco Gaido commented on SPARK-24606: - Critical and Blocker are reserved for committ

[jira] [Resolved] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-24606. - Resolution: Duplicate > Decimals multiplication and division may be null due to the result preci

[jira] [Commented] (SPARK-17333) Make pyspark interface friendly with static analysis

2018-06-20 Thread Alexander Gorokhov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518126#comment-16518126 ] Alexander Gorokhov commented on SPARK-17333: Hi everyone There was almost a

[jira] [Updated] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Yan Jian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Jian updated SPARK-24606: - Description: Spark performs mul / div on Decimals via Java's BigDecimal, whose scale may be greater tha

[jira] [Created] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread zenglinxi (JIRA)
zenglinxi created SPARK-24607: - Summary: Distribute by rand() can lead to data inconsistency Key: SPARK-24607 URL: https://issues.apache.org/jira/browse/SPARK-24607 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Yan Jian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Jian updated SPARK-24606: - Description: Spark performs mul / div on Decimals via Java's BigDecimal, whose scale may greater than i

[jira] [Created] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Yan Jian (JIRA)
Yan Jian created SPARK-24606: Summary: Decimals multiplication and division may be null due to the result precision overflow Key: SPARK-24606 URL: https://issues.apache.org/jira/browse/SPARK-24606 Project

[jira] [Assigned] (SPARK-24598) SPARK SQL:Datatype overflow conditions gives incorrect result

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24598: Assignee: (was: Apache Spark) > SPARK SQL:Datatype overflow conditions gives incorrec

[jira] [Commented] (SPARK-24598) SPARK SQL:Datatype overflow conditions gives incorrect result

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518092#comment-16518092 ] Apache Spark commented on SPARK-24598: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-24598) SPARK SQL:Datatype overflow conditions gives incorrect result

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24598: Assignee: Apache Spark > SPARK SQL:Datatype overflow conditions gives incorrect result >

[jira] [Assigned] (SPARK-24605) size(null) should return null

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24605: Assignee: (was: Apache Spark) > size(null) should return null > -

[jira] [Assigned] (SPARK-24605) size(null) should return null

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24605: Assignee: Apache Spark > size(null) should return null > - >

[jira] [Commented] (SPARK-24605) size(null) should return null

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518064#comment-16518064 ] Apache Spark commented on SPARK-24605: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-24605) size(null) should return null

2018-06-20 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518039#comment-16518039 ] Maxim Gekk commented on SPARK-24605: I am working on a PR which introduces new behav

[jira] [Created] (SPARK-24605) size(null) should return null

2018-06-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-24605: -- Summary: size(null) should return null Key: SPARK-24605 URL: https://issues.apache.org/jira/browse/SPARK-24605 Project: Spark Issue Type: Improvement C

[jira] [Updated] (SPARK-24603) Typo in comments

2018-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24603: - Priority: Trivial (was: Major) > Typo in comments > > > Key: S

[jira] [Commented] (SPARK-24601) Bump Jackson version to 2.9.6

2018-06-20 Thread Mate Juhasz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517985#comment-16517985 ] Mate Juhasz commented on SPARK-24601: - Hi, nice to see this ticket as I saw a lot of

[jira] [Assigned] (SPARK-24603) Typo in comments

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24603: Assignee: (was: Apache Spark) > Typo in comments > > >

[jira] [Assigned] (SPARK-24603) Typo in comments

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24603: Assignee: Apache Spark > Typo in comments > > > Key: SPA

[jira] [Created] (SPARK-24604) upgrade to spark 2.3.0 makes MPC model training slower

2018-06-20 Thread Enrique Molina (JIRA)
Enrique Molina created SPARK-24604: -- Summary: upgrade to spark 2.3.0 makes MPC model training slower Key: SPARK-24604 URL: https://issues.apache.org/jira/browse/SPARK-24604 Project: Spark Is

[jira] [Commented] (SPARK-24603) Typo in comments

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517959#comment-16517959 ] Apache Spark commented on SPARK-24603: -- User 'Fokko' has created a pull request for

[jira] [Created] (SPARK-24603) Typo in comments

2018-06-20 Thread Fokko Driesprong (JIRA)
Fokko Driesprong created SPARK-24603: Summary: Typo in comments Key: SPARK-24603 URL: https://issues.apache.org/jira/browse/SPARK-24603 Project: Spark Issue Type: Bug Components

[jira] [Assigned] (SPARK-24601) Bump Jackson version to 2.9.6

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24601: Assignee: (was: Apache Spark) > Bump Jackson version to 2.9.6 > -

[jira] [Commented] (SPARK-24601) Bump Jackson version to 2.9.6

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517932#comment-16517932 ] Apache Spark commented on SPARK-24601: -- User 'Fokko' has created a pull request for

[jira] [Assigned] (SPARK-24601) Bump Jackson version to 2.9.6

2018-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24601: Assignee: Apache Spark > Bump Jackson version to 2.9.6 > - >

[jira] [Created] (SPARK-24602) In Spark SQL, ALTER TABLE--CHANGE column1 column2 datatype is not supported in 2.3.1

2018-06-20 Thread Sushanta Sen (JIRA)
Sushanta Sen created SPARK-24602: Summary: In Spark SQL, ALTER TABLE--CHANGE column1 column2 datatype is not supported in 2.3.1 Key: SPARK-24602 URL: https://issues.apache.org/jira/browse/SPARK-24602

[jira] [Created] (SPARK-24601) Bump Jackson version to 2.9.6

2018-06-20 Thread Fokko Driesprong (JIRA)
Fokko Driesprong created SPARK-24601: Summary: Bump Jackson version to 2.9.6 Key: SPARK-24601 URL: https://issues.apache.org/jira/browse/SPARK-24601 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24600) Improve support for building different types of images in dockerfile

2018-06-20 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-24600: --- Description: Our docker images currently build and push docker images for pyspark an

[jira] [Created] (SPARK-24600) Improve support for building subset of images in dockerfile

2018-06-20 Thread Anirudh Ramanathan (JIRA)
Anirudh Ramanathan created SPARK-24600: -- Summary: Improve support for building subset of images in dockerfile Key: SPARK-24600 URL: https://issues.apache.org/jira/browse/SPARK-24600 Project: Spar

[jira] [Updated] (SPARK-24600) Improve support for building different types of images in dockerfile

2018-06-20 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-24600: --- Summary: Improve support for building different types of images in dockerfile (was: