[jira] [Assigned] (SPARK-22422) Add Adjusted R2 to RegressionMetrics

[ https://issues.apache.org/jira/browse/SPARK-22422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22422: Assignee: Apache Spark > Add Adjusted R2 to RegressionMetrics > --

[jira] [Assigned] (SPARK-22422) Add Adjusted R2 to RegressionMetrics

[ https://issues.apache.org/jira/browse/SPARK-22422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22422: Assignee: (was: Apache Spark) > Add Adjusted R2 to RegressionMetrics > ---

[jira] [Commented] (SPARK-22422) Add Adjusted R2 to RegressionMetrics

[ https://issues.apache.org/jira/browse/SPARK-22422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235195#comment-16235195 ] Apache Spark commented on SPARK-22422: -- User 'tengpeng' has created a pull request f

[jira] [Created] (SPARK-22422) Add Adjusted R2 to RegressionMetrics

Joseph Peng created SPARK-22422: --- Summary: Add Adjusted R2 to RegressionMetrics Key: SPARK-22422 URL: https://issues.apache.org/jira/browse/SPARK-22422 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235127#comment-16235127 ] xinzhang commented on SPARK-21067: -- [~dricard] Please say issue here link and try . [htt

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235119#comment-16235119 ] xinzhang commented on SPARK-21725: -- [~mgaido] Finally.I found the pro where is . add the

[jira] [Created] (SPARK-22421) is there a plan for Structured streaming monitoring UI ?

zhaoshijie created SPARK-22421: -- Summary: is there a plan for Structured streaming monitoring UI ? Key: SPARK-22421 URL: https://issues.apache.org/jira/browse/SPARK-22421 Project: Spark Issue T

[jira] [Commented] (SPARK-22398) Partition directories with leading 0s cause wrong results

[ https://issues.apache.org/jira/browse/SPARK-22398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235083#comment-16235083 ] Liang-Chi Hsieh commented on SPARK-22398: - [~mgaido], I'd prefer to treat them as

[jira] [Commented] (SPARK-20761) Union uses column order rather than schema

[ https://issues.apache.org/jira/browse/SPARK-20761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235070#comment-16235070 ] Hyukjin Kwon commented on SPARK-20761: -- Ah, now there is an API. {{unionByName}}. Pr

[jira] [Commented] (SPARK-12359) Add showString() to DataSet API.

[ https://issues.apache.org/jira/browse/SPARK-12359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235067#comment-16235067 ] Hyukjin Kwon commented on SPARK-12359: -- Thanks for the information and link. > Add

[jira] [Commented] (SPARK-22398) Partition directories with leading 0s cause wrong results

[ https://issues.apache.org/jira/browse/SPARK-22398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235062#comment-16235062 ] Hyukjin Kwon commented on SPARK-22398: -- [~mgaido], isn't it actually actually a dupl

[jira] [Created] (SPARK-22420) Spark SQL return invalid json string for struct with date/datetime field

pin_zhang created SPARK-22420: - Summary: Spark SQL return invalid json string for struct with date/datetime field Key: SPARK-22420 URL: https://issues.apache.org/jira/browse/SPARK-22420 Project: Spark

[jira] [Resolved] (SPARK-22419) Hive and Hive Thriftserver jars missing from "without hadoop" build

[ https://issues.apache.org/jira/browse/SPARK-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Kramer resolved SPARK-22419. - Resolution: Fixed Fix Version/s: 2.1.1 > Hive and Hive Thriftserver jars missing from "wit

[jira] [Commented] (SPARK-22419) Hive and Hive Thriftserver jars missing from "without hadoop" build

[ https://issues.apache.org/jira/browse/SPARK-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235047#comment-16235047 ] Adam Kramer commented on SPARK-22419: - I'm going to resolve this issue for now since

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235042#comment-16235042 ] Saisai Shao commented on SPARK-22405: - Thanks [~hvanhovell] for your comment, let me

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235039#comment-16235039 ] xinzhang edited comment on SPARK-21725 at 11/2/17 1:09 AM: --- cou

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235039#comment-16235039 ] xinzhang commented on SPARK-21725: -- could u tell me which version hadoop in your env . c

[jira] [Commented] (SPARK-21866) SPIP: Image support in Spark

[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235038#comment-16235038 ] Joseph K. Bradley commented on SPARK-21866: --- [~WeichenXu123] I prefer a datasou

[jira] [Created] (SPARK-22419) Hive and Hive Thriftserver jars missing from "without hadoop" build

Adam Kramer created SPARK-22419: --- Summary: Hive and Hive Thriftserver jars missing from "without hadoop" build Key: SPARK-22419 URL: https://issues.apache.org/jira/browse/SPARK-22419 Project: Spark

[jira] [Comment Edited] (SPARK-14974) spark sql job create too many files in HDFS when doing insert overwrite hive table

[ https://issues.apache.org/jira/browse/SPARK-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235027#comment-16235027 ] Yiting Shan edited comment on SPARK-14974 at 11/2/17 12:56 AM:

[jira] [Commented] (SPARK-14974) spark sql job create too many files in HDFS when doing insert overwrite hive table

[ https://issues.apache.org/jira/browse/SPARK-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235027#comment-16235027 ] Yiting Shan commented on SPARK-14974: - I am seeing similar issue. Insert overwrite to

[jira] [Commented] (SPARK-22243) streaming job failed to restart from checkpoint

[ https://issues.apache.org/jira/browse/SPARK-22243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235019#comment-16235019 ] Apache Spark commented on SPARK-22243: -- User 'ChenjunZou' has created a pull request

[jira] [Assigned] (SPARK-22243) streaming job failed to restart from checkpoint

[ https://issues.apache.org/jira/browse/SPARK-22243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22243: Assignee: (was: Apache Spark) > streaming job failed to restart from checkpoint >

[jira] [Assigned] (SPARK-22243) streaming job failed to restart from checkpoint

[ https://issues.apache.org/jira/browse/SPARK-22243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22243: Assignee: Apache Spark > streaming job failed to restart from checkpoint > ---

[jira] [Created] (SPARK-22418) Add test cases for NULL Handling

Xiao Li created SPARK-22418: --- Summary: Add test cases for NULL Handling Key: SPARK-22418 URL: https://issues.apache.org/jira/browse/SPARK-22418 Project: Spark Issue Type: Test Components:

[jira] [Commented] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234899#comment-16234899 ] Russell Spitzer commented on SPARK-15689: - I think knowing whether or not the cou

[jira] [Comment Edited] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234899#comment-16234899 ] Russell Spitzer edited comment on SPARK-15689 at 11/1/17 10:52 PM:

[jira] [Resolved] (SPARK-22414) Can't set driver env variables on yarn

[ https://issues.apache.org/jira/browse/SPARK-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Brasil resolved SPARK-22414. --- Resolution: Not A Problem > Can't set driver env variables on yarn >

[jira] [Commented] (SPARK-22414) Can't set driver env variables on yarn

[ https://issues.apache.org/jira/browse/SPARK-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234893#comment-16234893 ] Flavio Brasil commented on SPARK-22414: --- Sorry, I didn't see this config. It'd be h

[jira] [Comment Edited] (SPARK-18838) High latency of event processing for large jobs

[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234891#comment-16234891 ] Anthony Truchet edited comment on SPARK-18838 at 11/1/17 10:42 PM:

[jira] [Commented] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234892#comment-16234892 ] Wenchen Fan commented on SPARK-15689: - Spark wants to get `unhandledFilters` first so

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234891#comment-16234891 ] Anthony Truchet commented on SPARK-18838: - I'm interested to work on a backport f

[jira] [Created] (SPARK-22417) createDataFrame from a pandas.DataFrame reads datetime64 values as longs

Bryan Cutler created SPARK-22417: Summary: createDataFrame from a pandas.DataFrame reads datetime64 values as longs Key: SPARK-22417 URL: https://issues.apache.org/jira/browse/SPARK-22417 Project: Spa

[jira] [Commented] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234820#comment-16234820 ] Russell Spitzer commented on SPARK-15689: - It does not, we can tell that a count

[jira] [Commented] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234811#comment-16234811 ] Wenchen Fan commented on SPARK-15689: - how would a count(agg function) exist in filte

[jira] [Updated] (SPARK-22416) Move OrcOptions from `sql/hive` to `sql/core`

[ https://issues.apache.org/jira/browse/SPARK-22416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22416: -- Priority: Minor (was: Major) Issue Type: Task (was: Bug) > Move OrcOptions from `sql/hive` to `

[jira] [Commented] (SPARK-22416) Move OrcOptions from `sql/hive` to `sql/core`

[ https://issues.apache.org/jira/browse/SPARK-22416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234784#comment-16234784 ] Apache Spark commented on SPARK-22416: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-22416) Move OrcOptions from `sql/hive` to `sql/core`

[ https://issues.apache.org/jira/browse/SPARK-22416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22416: Assignee: (was: Apache Spark) > Move OrcOptions from `sql/hive` to `sql/core` > --

[jira] [Assigned] (SPARK-22416) Move OrcOptions from `sql/hive` to `sql/core`

[ https://issues.apache.org/jira/browse/SPARK-22416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22416: Assignee: Apache Spark > Move OrcOptions from `sql/hive` to `sql/core` > -

[jira] [Created] (SPARK-22416) Move OrcOptions from `sql/hive` to `sql/core`

Dongjoon Hyun created SPARK-22416: - Summary: Move OrcOptions from `sql/hive` to `sql/core` Key: SPARK-22416 URL: https://issues.apache.org/jira/browse/SPARK-22416 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-22414) Can't set driver env variables on yarn

[ https://issues.apache.org/jira/browse/SPARK-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234775#comment-16234775 ] Marcelo Vanzin commented on SPARK-22414: Have you tried {{spark.yarn.appMasterEnv

[jira] [Created] (SPARK-22415) lint-r fails if lint-r.R installs any new packages

Joel Croteau created SPARK-22415: Summary: lint-r fails if lint-r.R installs any new packages Key: SPARK-22415 URL: https://issues.apache.org/jira/browse/SPARK-22415 Project: Spark Issue Type

[jira] [Created] (SPARK-22414) Can't set driver env variables on yarn

Flavio Brasil created SPARK-22414: - Summary: Can't set driver env variables on yarn Key: SPARK-22414 URL: https://issues.apache.org/jira/browse/SPARK-22414 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234743#comment-16234743 ] Vinitha Reddy Gankidi commented on SPARK-22412: --- Okay, thanks for letting m

[jira] [Assigned] (SPARK-22413) Type coercion for IN is not coherent between Literals and subquery

[ https://issues.apache.org/jira/browse/SPARK-22413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22413: Assignee: (was: Apache Spark) > Type coercion for IN is not coherent between Literals

[jira] [Commented] (SPARK-22413) Type coercion for IN is not coherent between Literals and subquery

[ https://issues.apache.org/jira/browse/SPARK-22413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234731#comment-16234731 ] Apache Spark commented on SPARK-22413: -- User 'mgaido91' has created a pull request f

[jira] [Assigned] (SPARK-22413) Type coercion for IN is not coherent between Literals and subquery

[ https://issues.apache.org/jira/browse/SPARK-22413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22413: Assignee: Apache Spark > Type coercion for IN is not coherent between Literals and subquer

[jira] [Created] (SPARK-22413) Type coercion for IN is not coherent between Literals and subquery

Marco Gaido created SPARK-22413: --- Summary: Type coercion for IN is not coherent between Literals and subquery Key: SPARK-22413 URL: https://issues.apache.org/jira/browse/SPARK-22413 Project: Spark

[jira] [Commented] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234715#comment-16234715 ] Sean Owen commented on SPARK-22412: --- We generally don't make a JIRA for a one line comm

[jira] [Updated] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22412: -- Priority: Trivial (was: Minor) > Fix incorrect comment in DataSourceScanExec > ---

[jira] [Assigned] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22412: Assignee: (was: Apache Spark) > Fix incorrect comment in DataSourceScanExec >

[jira] [Commented] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234702#comment-16234702 ] Apache Spark commented on SPARK-22412: -- User 'vgankidi' has created a pull request f

[jira] [Assigned] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22412: Assignee: Apache Spark > Fix incorrect comment in DataSourceScanExec > ---

[jira] [Updated] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinitha Reddy Gankidi updated SPARK-22412: -- Component/s: (was: Documentation) Spark Core > Fix incorre

[jira] [Updated] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinitha Reddy Gankidi updated SPARK-22412: -- Component/s: (was: Spark Core) SQL > Fix incorrect comment

[jira] [Created] (SPARK-22412) Fix incorrect comment in DataSourceScanExec

Vinitha Reddy Gankidi created SPARK-22412: - Summary: Fix incorrect comment in DataSourceScanExec Key: SPARK-22412 URL: https://issues.apache.org/jira/browse/SPARK-22412 Project: Spark

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234679#comment-16234679 ] Aihua Xu commented on SPARK-18673: -- Hive is working on the Hadoop3.x support (HIVE-15016

[jira] [Commented] (SPARK-22411) Heuristic to combine splits in DataSourceScanExec isn't accurate when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234676#comment-16234676 ] Apache Spark commented on SPARK-22411: -- User 'vgankidi' has created a pull request f

[jira] [Assigned] (SPARK-22411) Heuristic to combine splits in DataSourceScanExec isn't accurate when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22411: Assignee: (was: Apache Spark) > Heuristic to combine splits in DataSourceScanExec isn'

[jira] [Assigned] (SPARK-22411) Heuristic to combine splits in DataSourceScanExec isn't accurate when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22411: Assignee: Apache Spark > Heuristic to combine splits in DataSourceScanExec isn't accurate

[jira] [Updated] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19644: - Description: I am using streaming on the production for some aggregation and fetching data from

[jira] [Updated] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19644: - Description: I am using streaming on the production for some aggregation and fetching data from

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234655#comment-16234655 ] Shixiong Zhu commented on SPARK-19644: -- I added more components since it also affect

[jira] [Updated] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19644: - Description: I am using streaming on the production for some aggregation and fetching data from

[jira] [Updated] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19644: - Component/s: Structured Streaming > Memory leak in Spark Streaming (Encoder/Scala Reflection) > -

[jira] [Updated] (SPARK-19644) Memory leak in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19644: - Component/s: SQL > Memory leak in Spark Streaming > -- > >

[jira] [Updated] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19644: - Summary: Memory leak in Spark Streaming (Encoder/Scala Reflection) (was: Memory leak in Spark St

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234643#comment-16234643 ] Shixiong Zhu commented on SPARK-19644: -- By the way, you can confirm this issue by ch

[jira] [Comment Edited] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234640#comment-16234640 ] Russell Spitzer edited comment on SPARK-15689 at 11/1/17 7:58 PM: -

[jira] [Commented] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234640#comment-16234640 ] Russell Spitzer commented on SPARK-15689: - Something I just noticed, it may be he

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234638#comment-16234638 ] Shixiong Zhu commented on SPARK-19644: -- I happened to investigate a similar issue an

[jira] [Created] (SPARK-22411) Heuristic to combine splits in DataSourceScanExec isn't accurate when dynamic allocation is enabled

Vinitha Reddy Gankidi created SPARK-22411: - Summary: Heuristic to combine splits in DataSourceScanExec isn't accurate when dynamic allocation is enabled Key: SPARK-22411 URL: https://issues.apache.org/jira

[jira] [Commented] (SPARK-22372) Make YARN client extend SparkApplication

[ https://issues.apache.org/jira/browse/SPARK-22372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234476#comment-16234476 ] Apache Spark commented on SPARK-22372: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-22372) Make YARN client extend SparkApplication

[ https://issues.apache.org/jira/browse/SPARK-22372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22372: Assignee: (was: Apache Spark) > Make YARN client extend SparkApplication > ---

[jira] [Assigned] (SPARK-22372) Make YARN client extend SparkApplication

[ https://issues.apache.org/jira/browse/SPARK-22372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22372: Assignee: Apache Spark > Make YARN client extend SparkApplication > --

[jira] [Created] (SPARK-22410) Excessive spill for Pyspark UDF when a row has shrunk

Clément Stenac created SPARK-22410: -- Summary: Excessive spill for Pyspark UDF when a row has shrunk Key: SPARK-22410 URL: https://issues.apache.org/jira/browse/SPARK-22410 Project: Spark Iss

[jira] [Commented] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234428#comment-16234428 ] Reynold Xin commented on SPARK-20928: - Maybe we can add some information metadata (li

[jira] [Assigned] (SPARK-22409) Add function type argument to pandas_udf

[ https://issues.apache.org/jira/browse/SPARK-22409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22409: Assignee: (was: Apache Spark) > Add function type argument to pandas_udf > ---

[jira] [Commented] (SPARK-22409) Add function type argument to pandas_udf

[ https://issues.apache.org/jira/browse/SPARK-22409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234376#comment-16234376 ] Apache Spark commented on SPARK-22409: -- User 'icexelloss' has created a pull request

[jira] [Assigned] (SPARK-22409) Add function type argument to pandas_udf

[ https://issues.apache.org/jira/browse/SPARK-22409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22409: Assignee: Apache Spark > Add function type argument to pandas_udf > --

[jira] [Commented] (SPARK-22344) Prevent R CMD check from using /tmp

[ https://issues.apache.org/jira/browse/SPARK-22344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234372#comment-16234372 ] Shivaram Venkataraman commented on SPARK-22344: --- Right I was considering th

[jira] [Created] (SPARK-22409) Add function type argument to pandas_udf

Li Jin created SPARK-22409: -- Summary: Add function type argument to pandas_udf Key: SPARK-22409 URL: https://issues.apache.org/jira/browse/SPARK-22409 Project: Spark Issue Type: Sub-task C

[jira] [Updated] (SPARK-22409) Add function type argument to pandas_udf

[ https://issues.apache.org/jira/browse/SPARK-22409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-22409: --- Priority: Major (was: Trivial) > Add function type argument to pandas_udf >

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234332#comment-16234332 ] Marco Gaido commented on SPARK-21725: - I don't have any idea about which is the diffe

[jira] [Assigned] (SPARK-22408) RelationalGroupedDataset's distinct pivot value calculation launches unnecessary stages

[ https://issues.apache.org/jira/browse/SPARK-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22408: Assignee: (was: Apache Spark) > RelationalGroupedDataset's distinct pivot value calcul

[jira] [Commented] (SPARK-22408) RelationalGroupedDataset's distinct pivot value calculation launches unnecessary stages

[ https://issues.apache.org/jira/browse/SPARK-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234295#comment-16234295 ] Apache Spark commented on SPARK-22408: -- User 'pwoody' has created a pull request for

[jira] [Assigned] (SPARK-22408) RelationalGroupedDataset's distinct pivot value calculation launches unnecessary stages

[ https://issues.apache.org/jira/browse/SPARK-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22408: Assignee: Apache Spark > RelationalGroupedDataset's distinct pivot value calculation launc

[jira] [Updated] (SPARK-22408) RelationalGroupedDataset's distinct pivot value calculation launches unnecessary stages

[ https://issues.apache.org/jira/browse/SPARK-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Woody updated SPARK-22408: -- Summary: RelationalGroupedDataset's distinct pivot value calculation launches unnecessary stage

[jira] [Created] (SPARK-22408) RelationalGroupedDataset's distinct pivot value calculation can launch many stages

Patrick Woody created SPARK-22408: - Summary: RelationalGroupedDataset's distinct pivot value calculation can launch many stages Key: SPARK-22408 URL: https://issues.apache.org/jira/browse/SPARK-22408

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234225#comment-16234225 ] Ryan Blue commented on SPARK-2984: -- I don't have a good solution here. You could maybe is

[jira] [Commented] (SPARK-22398) Partition directories with leading 0s cause wrong results

[ https://issues.apache.org/jira/browse/SPARK-22398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234192#comment-16234192 ] Marco Gaido commented on SPARK-22398: - [~viirya] sorry for the unrequested ping, I sa

[jira] [Assigned] (SPARK-22190) Add Spark executor task metrics to Dropwizard metrics

[ https://issues.apache.org/jira/browse/SPARK-22190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22190: --- Assignee: Luca Canali > Add Spark executor task metrics to Dropwizard metrics >

[jira] [Resolved] (SPARK-22190) Add Spark executor task metrics to Dropwizard metrics

[ https://issues.apache.org/jira/browse/SPARK-22190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22190. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19426 [https://githu

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234149#comment-16234149 ] xinzhang commented on SPARK-21725: -- I can't believe it. I build hadoop 2.8 last night. I

[jira] [Commented] (SPARK-22371) dag-scheduler-event-loop thread stopped with error Attempted to access garbage collected accumulator 5605982

[ https://issues.apache.org/jira/browse/SPARK-22371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234112#comment-16234112 ] Marco Gaido commented on SPARK-22371: - Could you please provide an easy way to reprod

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234105#comment-16234105 ] Marco Gaido commented on SPARK-21725: - I tried using a mysql metastore and the target

[jira] [Resolved] (SPARK-19112) add codec for ZStandard

[ https://issues.apache.org/jira/browse/SPARK-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19112. --- Resolution: Fixed Assignee: Sital Kedia Fix Version/s: 2.3.0 > add co

[jira] [Assigned] (SPARK-21088) CrossValidator, TrainValidationSplit should collect all models when fitting: Python API

[ https://issues.apache.org/jira/browse/SPARK-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21088: Assignee: (was: Apache Spark) > CrossValidator, TrainValidationSplit should collect al

[jira] [Assigned] (SPARK-21088) CrossValidator, TrainValidationSplit should collect all models when fitting: Python API

[ https://issues.apache.org/jira/browse/SPARK-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21088: Assignee: Apache Spark > CrossValidator, TrainValidationSplit should collect all models wh

[jira] [Commented] (SPARK-21088) CrossValidator, TrainValidationSplit should collect all models when fitting: Python API

[ https://issues.apache.org/jira/browse/SPARK-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234087#comment-16234087 ] Apache Spark commented on SPARK-21088: -- User 'WeichenXu123' has created a pull reque

  1   2   >