[jira] [Assigned] (SPARK-25418) The metadata of DataSource table should not include Hive-generated storage properties.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25418: Assignee: (was: Apache Spark) > The metadata of DataSource table should not include H

[jira] [Assigned] (SPARK-25418) The metadata of DataSource table should not include Hive-generated storage properties.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25418: Assignee: Apache Spark > The metadata of DataSource table should not include Hive-generat

[jira] [Commented] (SPARK-25418) The metadata of DataSource table should not include Hive-generated storage properties.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613040#comment-16613040 ] Apache Spark commented on SPARK-25418: -- User 'ueshin' has created a pull request fo

[jira] [Created] (SPARK-25418) The metadata of DataSource table should not include Hive-generated storage properties.

2018-09-12 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-25418: - Summary: The metadata of DataSource table should not include Hive-generated storage properties. Key: SPARK-25418 URL: https://issues.apache.org/jira/browse/SPARK-25418

[jira] [Commented] (SPARK-25352) Perform ordered global limit when limit number is bigger than topKSortFallbackThreshold

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613033#comment-16613033 ] Apache Spark commented on SPARK-25352: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-25352) Perform ordered global limit when limit number is bigger than topKSortFallbackThreshold

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613032#comment-16613032 ] Apache Spark commented on SPARK-25352: -- User 'viirya' has created a pull request fo

[jira] [Resolved] (SPARK-25295) Pod names conflicts in client mode, if previous submission was not a clean shutdown.

2018-09-12 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li resolved SPARK-25295. -- Resolution: Fixed Fix Version/s: 2.4.0 > Pod names conflicts in client mode, if previous submis

[jira] [Updated] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25415: Priority: Major (was: Minor) > Make plan change log in RuleExecutor configurable by SQLConf > ---

[jira] [Resolved] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25415. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 3.0.0 > Make plan change log in R

[jira] [Commented] (SPARK-25397) SparkSession.conf fails when given default value with Python 3

2018-09-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612994#comment-16612994 ] Hyukjin Kwon commented on SPARK-25397: -- [~josephkb], do you want to backport this b

[jira] [Commented] (SPARK-25378) ArrayData.toArray(StringType) assume UTF8String in 2.4

2018-09-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612965#comment-16612965 ] Hyukjin Kwon commented on SPARK-25378: -- {quote} If it is not pubic, why didn't we h

[jira] [Commented] (SPARK-25378) ArrayData.toArray(StringType) assume UTF8String in 2.4

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612947#comment-16612947 ] Wenchen Fan commented on SPARK-25378: - [~viirya] Can you take a look and see how har

[jira] [Assigned] (SPARK-25357) Add metadata to SparkPlanInfo to dump more information like file path to event log

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25357: --- Assignee: Lantao Jin > Add metadata to SparkPlanInfo to dump more information like file pat

[jira] [Resolved] (SPARK-25402) Null handling in BooleanSimplification

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25402. - Resolution: Fixed Fix Version/s: 2.4.0 2.3.2 > Null handling in Boolea

[jira] [Resolved] (SPARK-25357) Add metadata to SparkPlanInfo to dump more information like file path to event log

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25357. - Resolution: Fixed Fix Version/s: 2.3.2 2.4.0 Issue resolved by pull re

[jira] [Resolved] (SPARK-25387) Malformed CSV causes NPE

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25387. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22374 [https://gith

[jira] [Assigned] (SPARK-25387) Malformed CSV causes NPE

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25387: --- Assignee: Maxim Gekk > Malformed CSV causes NPE > > >

[jira] [Resolved] (SPARK-23820) Allow the long form of call sites to be recorded in the log

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23820. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22398 [https://gith

[jira] [Commented] (SPARK-25291) Flakiness of tests in terms of executor memory (SecretsTestSuite)

2018-09-12 Thread Ilan Filonenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612911#comment-16612911 ] Ilan Filonenko commented on SPARK-25291: [~skonto] the problem stems from the ex

[jira] [Comment Edited] (SPARK-25291) Flakiness of tests in terms of executor memory (SecretsTestSuite)

2018-09-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612894#comment-16612894 ] Stavros Kontopoulos edited comment on SPARK-25291 at 9/13/18 12:41 AM: ---

[jira] [Comment Edited] (SPARK-25291) Flakiness of tests in terms of executor memory (SecretsTestSuite)

2018-09-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612894#comment-16612894 ] Stavros Kontopoulos edited comment on SPARK-25291 at 9/13/18 12:41 AM: ---

[jira] [Commented] (SPARK-25291) Flakiness of tests in terms of executor memory (SecretsTestSuite)

2018-09-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612894#comment-16612894 ] Stavros Kontopoulos commented on SPARK-25291: - [~ifilonenko] I can have a lo

[jira] [Commented] (SPARK-23012) Support for predicate pushdown and partition pruning when left joining large Hive tables

2018-09-12 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612881#comment-16612881 ] Yuming Wang commented on SPARK-23012: - It seems the following PR resolves your issue

[jira] [Commented] (SPARK-25416) ArrayPosition function may return incorrect result when right expression is implicitly downcasted.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612817#comment-16612817 ] Apache Spark commented on SPARK-25416: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-25416) ArrayPosition function may return incorrect result when right expression is implicitly downcasted.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25416: Assignee: Apache Spark > ArrayPosition function may return incorrect result when right ex

[jira] [Commented] (SPARK-25416) ArrayPosition function may return incorrect result when right expression is implicitly downcasted.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612818#comment-16612818 ] Apache Spark commented on SPARK-25416: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-25416) ArrayPosition function may return incorrect result when right expression is implicitly downcasted.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25416: Assignee: (was: Apache Spark) > ArrayPosition function may return incorrect result wh

[jira] [Assigned] (SPARK-25417) ArrayContains function may return incorrect result when right expression is implicitly down casted

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25417: Assignee: Apache Spark > ArrayContains function may return incorrect result when right ex

[jira] [Commented] (SPARK-25417) ArrayContains function may return incorrect result when right expression is implicitly down casted

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612816#comment-16612816 ] Apache Spark commented on SPARK-25417: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-25417) ArrayContains function may return incorrect result when right expression is implicitly down casted

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25417: Assignee: (was: Apache Spark) > ArrayContains function may return incorrect result wh

[jira] [Created] (SPARK-25417) ArrayContains function may return incorrect result when right expression is implicitly down casted

2018-09-12 Thread Dilip Biswal (JIRA)
Dilip Biswal created SPARK-25417: Summary: ArrayContains function may return incorrect result when right expression is implicitly down casted Key: SPARK-25417 URL: https://issues.apache.org/jira/browse/SPARK-25417

[jira] [Commented] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612804#comment-16612804 ] Apache Spark commented on SPARK-25415: -- User 'dilipbiswal' has created a pull reque

[jira] [Commented] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612803#comment-16612803 ] Apache Spark commented on SPARK-25415: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25415: Assignee: (was: Apache Spark) > Make plan change log in RuleExecutor configurable by

[jira] [Commented] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612766#comment-16612766 ] Apache Spark commented on SPARK-25415: -- User 'maryannxue' has created a pull reques

[jira] [Assigned] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25415: Assignee: Apache Spark > Make plan change log in RuleExecutor configurable by SQLConf > -

[jira] [Updated] (SPARK-25416) ArrayPosition function may return incorrect result when right expression is implicitly downcasted.

2018-09-12 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Biswal updated SPARK-25416: - Summary: ArrayPosition function may return incorrect result when right expression is implicitly

[jira] [Created] (SPARK-25416) ArrayPosition may return incorrect result when right expression is downcasted.

2018-09-12 Thread Dilip Biswal (JIRA)
Dilip Biswal created SPARK-25416: Summary: ArrayPosition may return incorrect result when right expression is downcasted. Key: SPARK-25416 URL: https://issues.apache.org/jira/browse/SPARK-25416 Projec

[jira] [Created] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf

2018-09-12 Thread Maryann Xue (JIRA)
Maryann Xue created SPARK-25415: --- Summary: Make plan change log in RuleExecutor configurable by SQLConf Key: SPARK-25415 URL: https://issues.apache.org/jira/browse/SPARK-25415 Project: Spark I

[jira] [Commented] (SPARK-25295) Pod names conflicts in client mode, if previous submission was not a clean shutdown.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612662#comment-16612662 ] Apache Spark commented on SPARK-25295: -- User 'skonto' has created a pull request fo

[jira] [Assigned] (SPARK-25295) Pod names conflicts in client mode, if previous submission was not a clean shutdown.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25295: Assignee: Apache Spark > Pod names conflicts in client mode, if previous submission was n

[jira] [Assigned] (SPARK-25295) Pod names conflicts in client mode, if previous submission was not a clean shutdown.

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25295: Assignee: (was: Apache Spark) > Pod names conflicts in client mode, if previous submi

[jira] [Updated] (SPARK-20350) Apply Complementation Laws during boolean expression simplification

2018-09-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20350: -- Component/s: (was: Optimizer) SQL > Apply Complementation Laws during boo

[jira] [Closed] (SPARK-20799) Unable to infer schema for ORC/Parquet on S3N when secrets are in the URL

2018-09-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-20799. - > Unable to infer schema for ORC/Parquet on S3N when secrets are in the URL > --

[jira] [Resolved] (SPARK-20799) Unable to infer schema for ORC/Parquet on S3N when secrets are in the URL

2018-09-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-20799. --- Resolution: Won't Fix > Unable to infer schema for ORC/Parquet on S3N when secrets are in th

[jira] [Commented] (SPARK-20799) Unable to infer schema for ORC/Parquet on S3N when secrets are in the URL

2018-09-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612642#comment-16612642 ] Dongjoon Hyun commented on SPARK-20799: --- +1 for closing this as a WONTFIX. > Unab

[jira] [Commented] (SPARK-25402) Null handling in BooleanSimplification

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612530#comment-16612530 ] Apache Spark commented on SPARK-25402: -- User 'gatorsmile' has created a pull reques

[jira] [Commented] (SPARK-25402) Null handling in BooleanSimplification

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612529#comment-16612529 ] Apache Spark commented on SPARK-25402: -- User 'gatorsmile' has created a pull reques

[jira] [Resolved] (SPARK-25363) Schema pruning doesn't work if nested column is used in where clause

2018-09-12 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25363. - Resolution: Fixed Fix Version/s: 2.4.0 3.0.0 Issue resolved by pull request 22

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612436#comment-16612436 ] Joseph K. Bradley commented on SPARK-25321: --- You're right; these are breaking

[jira] [Commented] (SPARK-25380) Generated plans occupy over 50% of Spark driver memory

2018-09-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612433#comment-16612433 ] Marcelo Vanzin commented on SPARK-25380: Yep. That's a 200MB plan description st

[jira] [Commented] (SPARK-25378) ArrayData.toArray(StringType) assume UTF8String in 2.4

2018-09-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612399#comment-16612399 ] Xiangrui Meng commented on SPARK-25378: --- Comments from [~vomjom] at https://github

[jira] [Commented] (SPARK-25295) Pod names conflicts in client mode, if previous submission was not a clean shutdown.

2018-09-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612337#comment-16612337 ] Stavros Kontopoulos commented on SPARK-25295: - guys I started working on a s

[jira] [Assigned] (SPARK-25352) Perform ordered global limit when limit number is bigger than topKSortFallbackThreshold

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25352: --- Assignee: Liang-Chi Hsieh > Perform ordered global limit when limit number is bigger than

[jira] [Resolved] (SPARK-25352) Perform ordered global limit when limit number is bigger than topKSortFallbackThreshold

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25352. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22344 [https://gith

[jira] [Comment Edited] (SPARK-24627) [Spark2.3.0] After HDFS Token expire kinit not able to submit job using beeline

2018-09-12 Thread Ayush Anubhava (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612085#comment-16612085 ] Ayush Anubhava edited comment on SPARK-24627 at 9/12/18 12:53 PM:

[jira] [Commented] (SPARK-24627) [Spark2.3.0] After HDFS Token expire kinit not able to submit job using beeline

2018-09-12 Thread Ayush Anubhava (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612085#comment-16612085 ] Ayush Anubhava commented on SPARK-24627: Check the the principal name given in s

[jira] [Resolved] (SPARK-25371) Vector Assembler with no input columns leads to opaque error

2018-09-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25371. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 2

[jira] [Commented] (SPARK-25150) Joining DataFrames derived from the same source yields confusing/incorrect results

2018-09-12 Thread Evelyn Bayes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612026#comment-16612026 ] Evelyn Bayes commented on SPARK-25150: -- Hey Peter, don't stress it. I'm new to the

[jira] [Commented] (SPARK-20153) Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application

2018-09-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612001#comment-16612001 ] Steve Loughran commented on SPARK-20153: bq. Amazon EMR does not currently suppo

[jira] [Commented] (SPARK-20799) Unable to infer schema for ORC/Parquet on S3N when secrets are in the URL

2018-09-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611995#comment-16611995 ] Steve Loughran commented on SPARK-20799: Update: Hadoop 3.3+ will remove all sup

[jira] [Assigned] (SPARK-25414) The numInputRows metrics can be incorrect for streaming self-join

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25414: Assignee: Wenchen Fan (was: Apache Spark) > The numInputRows metrics can be incorrect fo

[jira] [Assigned] (SPARK-25414) The numInputRows metrics can be incorrect for streaming self-join

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25414: Assignee: Apache Spark (was: Wenchen Fan) > The numInputRows metrics can be incorrect fo

[jira] [Commented] (SPARK-25414) The numInputRows metrics can be incorrect for streaming self-join

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611932#comment-16611932 ] Apache Spark commented on SPARK-25414: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-25414) The numInputRows metrics can be incorrect for streaming self-join

2018-09-12 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-25414: --- Summary: The numInputRows metrics can be incorrect for streaming self-join Key: SPARK-25414 URL: https://issues.apache.org/jira/browse/SPARK-25414 Project: Spark

[jira] [Updated] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread sandeep katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sandeep katta updated SPARK-25413: -- Attachment: decimalBoundaryDataHive.csv > [2.3.2.rc5 Blocker] Precision Value is going for tos

[jira] [Updated] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread sandeep katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sandeep katta updated SPARK-25413: -- Attachment: (was: decimalBoundaryDataHive.csv) > [2.3.2.rc5 Blocker] Precision Value is go

[jira] [Updated] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread sandeep katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sandeep katta updated SPARK-25413: -- Description: sql("create table if not exists hiveBigDecimal(ID Int, date Timestamp, country S

[jira] [Commented] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611759#comment-16611759 ] Ajith S commented on SPARK-25413: - Thank you for reporting the issue sandeep. I think th

[jira] [Assigned] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25413: Assignee: (was: Apache Spark) > [2.3.2.rc5 Blocker] Precision Value is going for toss

[jira] [Commented] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611758#comment-16611758 ] Apache Spark commented on SPARK-25413: -- User 'ajithme' has created a pull request f

[jira] [Assigned] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25413: Assignee: Apache Spark > [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg i

[jira] [Updated] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread sandeep katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sandeep katta updated SPARK-25413: -- Priority: Blocker (was: Minor) > [2.3.2.rc5 Blocker] Precision Value is going for toss when A

[jira] [Updated] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread sandeep katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sandeep katta updated SPARK-25413: -- Attachment: decimalBoundaryDataHive.csv > [2.3.2.rc5 Blocker] Precision Value is going for tos

[jira] [Created] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-09-12 Thread sandeep katta (JIRA)
sandeep katta created SPARK-25413: - Summary: [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done Key: SPARK-25413 URL: https://issues.apache.org/jira/browse/SPARK-25413 Project: Spa

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-09-12 Thread Peter Knight (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611733#comment-16611733 ] Peter Knight commented on SPARK-21542: -- Thanks for the replay [~JohnHBauer]. Yes I

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2018-09-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611717#comment-16611717 ] Kazuaki Ishizaki commented on SPARK-20184: -- In {{branch-2.4}}, we still see the