[jira] [Commented] (SPARK-21444) Fetch failure due to node reboot causes job failure

2019-05-19 Thread Igor Berman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843357#comment-16843357 ] Igor Berman commented on SPARK-21444: - this Jira affects 2.2.3 version as well  as

[jira] [Commented] (SPARK-27758) Features won't generate after 1M rows

2019-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843361#comment-16843361 ] Sean Owen commented on SPARK-27758: --- This should clarify what you mean when you say fe

[jira] [Commented] (SPARK-27613) Caching an RDD composed of Row Objects produces some kind of key recombination

2019-05-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843379#comment-16843379 ] Hyukjin Kwon commented on SPARK-27613: -- no feedback. > Caching an RDD composed of

[jira] [Resolved] (SPARK-27613) Caching an RDD composed of Row Objects produces some kind of key recombination

2019-05-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27613. -- Resolution: Incomplete > Caching an RDD composed of Row Objects produces some kind of key reco

[jira] [Created] (SPARK-27771) Add documentation for grouping functions (cube, rollup, grouping and grouping_id)

2019-05-19 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-27771: Summary: Add documentation for grouping functions (cube, rollup, grouping and grouping_id) Key: SPARK-27771 URL: https://issues.apache.org/jira/browse/SPARK-27771 Pro

[jira] [Assigned] (SPARK-27771) Add documentation for grouping functions (cube, rollup, grouping and grouping_id)

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27771: Assignee: Apache Spark > Add documentation for grouping functions (cube, rollup, grouping

[jira] [Assigned] (SPARK-27771) Add documentation for grouping functions (cube, rollup, grouping and grouping_id)

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27771: Assignee: (was: Apache Spark) > Add documentation for grouping functions (cube, rollu

[jira] [Commented] (SPARK-27723) Unable to pull the oracle table data using patitionColumn date/timeStamp

2019-05-19 Thread Shyama (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843422#comment-16843422 ] Shyama commented on SPARK-27723: [~smilegator] sure , please let me know how to do it i.

[jira] [Commented] (SPARK-27723) Unable to pull the oracle table data using patitionColumn date/timeStamp

2019-05-19 Thread Shyama (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843425#comment-16843425 ] Shyama commented on SPARK-27723: [~yumwang] When I run in the spark-cluster , same thing

[jira] [Commented] (SPARK-27723) Unable to pull the oracle table data using patitionColumn date/timeStamp

2019-05-19 Thread Shyama (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843432#comment-16843432 ] Shyama commented on SPARK-27723: [~yumwang]    any help plz  [https://stackoverflow.com/

[jira] [Updated] (SPARK-27749) hadoop-3.2 support hive-thriftserver

2019-05-19 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27749: Summary: hadoop-3.2 support hive-thriftserver (was: Fix hadoop-3.2 hive-thriftserver module test

[jira] [Assigned] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26412: Assignee: Weichen Xu (was: Apache Spark) > Allow Pandas UDF to take an iterator of pd.Da

[jira] [Assigned] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26412: Assignee: Apache Spark (was: Weichen Xu) > Allow Pandas UDF to take an iterator of pd.Da

[jira] [Created] (SPARK-27772) SQLTestUtils Refactoring

2019-05-19 Thread William Wong (JIRA)
William Wong created SPARK-27772: Summary: SQLTestUtils Refactoring Key: SPARK-27772 URL: https://issues.apache.org/jira/browse/SPARK-27772 Project: Spark Issue Type: Improvement Co

[jira] [Updated] (SPARK-27772) SQLTestUtils Refactoring

2019-05-19 Thread William Wong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Wong updated SPARK-27772: - Description: {code:java} // code placeholder {code} The current `SQLTestUtils` created many `wit

[jira] [Updated] (SPARK-27772) SQLTestUtils Refactoring

2019-05-19 Thread William Wong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Wong updated SPARK-27772: - Description: The current `SQLTestUtils` created many `withXXX` utility functions to clean up ta

[jira] [Updated] (SPARK-27772) SQLTestUtils Refactoring

2019-05-19 Thread William Wong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Wong updated SPARK-27772: - Description: The current `SQLTestUtils` created many `withXXX` utility functions to clean up ta

[jira] [Updated] (SPARK-27772) SQLTestUtils Refactoring

2019-05-19 Thread William Wong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Wong updated SPARK-27772: - Description: The current `SQLTestUtils` created many `withXXX` utility functions to clean up ta

[jira] [Updated] (SPARK-27772) SQLTestUtils Refactoring

2019-05-19 Thread William Wong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Wong updated SPARK-27772: - Description: The current `SQLTestUtils` created many `withXXX` utility functions to clean up ta

[jira] [Comment Edited] (SPARK-26841) Timestamp pushdown on Kafka table

2019-05-19 Thread Richard Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843509#comment-16843509 ] Richard Yu edited comment on SPARK-26841 at 5/19/19 5:53 PM: -

[jira] [Commented] (SPARK-26841) Timestamp pushdown on Kafka table

2019-05-19 Thread Richard Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843509#comment-16843509 ] Richard Yu commented on SPARK-26841: [~Bartalos] Could you expain what use cases the

[jira] [Updated] (SPARK-27768) Incorrect statistical aggregate results when the inputs have constants infinity

2019-05-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27768: Description: When the inputs contain the constant 'infinity', Spark SQL does not generate the expected re

[jira] [Updated] (SPARK-27768) Infinity, -Infinity, NaN should be recognized in a case insensitive manner

2019-05-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27768: Summary: Infinity, -Infinity, NaN should be recognized in a case insensitive manner (was: Incorrect stati

[jira] [Updated] (SPARK-27771) Add SQL description for grouping functions (cube, rollup, grouping and grouping_id)

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27771: -- Summary: Add SQL description for grouping functions (cube, rollup, grouping and grouping_id)

[jira] [Resolved] (SPARK-27771) Add SQL description for grouping functions (cube, rollup, grouping and grouping_id)

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27771. --- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 3.0.0

[jira] [Reopened] (SPARK-27402) Fix hadoop-3.2 test issue(except the hive-thriftserver module)

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-27402: --- Reopen this since [~yumwang] is discovered the issue. > Fix hadoop-3.2 test issue(except the hi

[jira] [Assigned] (SPARK-27402) Fix hadoop-3.2 test issue(except the hive-thriftserver module)

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27402: Assignee: Yuming Wang (was: Apache Spark) > Fix hadoop-3.2 test issue(except the hive-th

[jira] [Assigned] (SPARK-27402) Fix hadoop-3.2 test issue(except the hive-thriftserver module)

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27402: Assignee: Apache Spark (was: Yuming Wang) > Fix hadoop-3.2 test issue(except the hive-th

[jira] [Commented] (SPARK-27768) Infinity, -Infinity, NaN should be recognized in a case insensitive manner

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843595#comment-16843595 ] Dongjoon Hyun commented on SPARK-27768: --- FYI, Spark behavior is compatible with Hi

[jira] [Comment Edited] (SPARK-27768) Infinity, -Infinity, NaN should be recognized in a case insensitive manner

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843595#comment-16843595 ] Dongjoon Hyun edited comment on SPARK-27768 at 5/20/19 2:53 AM: --

[jira] [Commented] (SPARK-27772) SQLTestUtils Refactoring

2019-05-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843596#comment-16843596 ] Hyukjin Kwon commented on SPARK-27772: -- Can you describe one actual case that {{wit

[jira] [Commented] (SPARK-27768) Infinity, -Infinity, NaN should be recognized in a case insensitive manner

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843597#comment-16843597 ] Dongjoon Hyun commented on SPARK-27768: --- Spark 3.0.0 had better be consistent with

[jira] [Resolved] (SPARK-27758) Features won't generate after 1M rows

2019-05-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27758. -- Resolution: Incomplete > Features won't generate after 1M rows > -

[jira] [Commented] (SPARK-27716) Complete the transactions support for part of jdbc datasource operations.

2019-05-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843649#comment-16843649 ] Liang-Chi Hsieh commented on SPARK-27716: - If the added support doesn't cover al

[jira] [Updated] (SPARK-27694) Support auto-updating table statistics for data source CTAS command

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27694: -- Summary: Support auto-updating table statistics for data source CTAS command (was: CTAS creat

[jira] [Resolved] (SPARK-27694) Support auto-updating table statistics for data source CTAS command

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27694. --- Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 This is resolve

[jira] [Resolved] (SPARK-27693) DataSourceV2: Add default catalog property

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27693. --- Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 3.0.0 This is resolved

[jira] [Updated] (SPARK-27702) Allow using some alternatives for service accounts

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27702: -- Issue Type: Improvement (was: Bug) > Allow using some alternatives for service accounts > ---

[jira] [Commented] (SPARK-27702) Allow using some alternatives for service accounts

2019-05-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843670#comment-16843670 ] Dongjoon Hyun commented on SPARK-27702: --- Hi, [~Udbhav Agrawal] Thank you for filli

[jira] [Created] (SPARK-27773) Add shuffle service metric for number of exceptions caught in TransportChannelHandler

2019-05-19 Thread Steven Rand (JIRA)
Steven Rand created SPARK-27773: --- Summary: Add shuffle service metric for number of exceptions caught in TransportChannelHandler Key: SPARK-27773 URL: https://issues.apache.org/jira/browse/SPARK-27773 P

[jira] [Updated] (SPARK-27689) Error to execute hive views with spark

2019-05-19 Thread Juan Antonio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Antonio updated SPARK-27689: - Description: I have a python error when I execute the following code using hive views but it wo

[jira] [Assigned] (SPARK-27773) Add shuffle service metric for number of exceptions caught in TransportChannelHandler

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27773: Assignee: (was: Apache Spark) > Add shuffle service metric for number of exceptions c

[jira] [Assigned] (SPARK-27773) Add shuffle service metric for number of exceptions caught in TransportChannelHandler

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27773: Assignee: Apache Spark > Add shuffle service metric for number of exceptions caught in >

[jira] [Created] (SPARK-27774) Avoid hardcoded configs

2019-05-19 Thread wenxuanguan (JIRA)
wenxuanguan created SPARK-27774: --- Summary: Avoid hardcoded configs Key: SPARK-27774 URL: https://issues.apache.org/jira/browse/SPARK-27774 Project: Spark Issue Type: Improvement Compo

[jira] [Commented] (SPARK-27774) Avoid hardcoded configs

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843684#comment-16843684 ] Apache Spark commented on SPARK-27774: -- User 'wenxuanguan' has created a pull reque

[jira] [Assigned] (SPARK-27774) Avoid hardcoded configs

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27774: Assignee: (was: Apache Spark) > Avoid hardcoded configs > --- > >

[jira] [Assigned] (SPARK-27774) Avoid hardcoded configs

2019-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27774: Assignee: Apache Spark > Avoid hardcoded configs > --- > >

[jira] [Commented] (SPARK-27689) Error to execute hive views with spark

2019-05-19 Thread Juan Antonio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843686#comment-16843686 ] Juan Antonio commented on SPARK-27689: -- I have just updated the code to simplify it

[jira] [Created] (SPARK-27775) Support multiple return values for udf

2019-05-19 Thread Xianjin YE (JIRA)
Xianjin YE created SPARK-27775: -- Summary: Support multiple return values for udf Key: SPARK-27775 URL: https://issues.apache.org/jira/browse/SPARK-27775 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-27322) Select from multiple catalogs

2019-05-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27322: --- Summary: Select from multiple catalogs (was: SELECT from multiple catalogs) > Select from multiple