[jira] [Commented] (SPARK-17388) Support for inferring type date/timestamp/decimal for partition column

[ https://issues.apache.org/jira/browse/SPARK-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460546#comment-15460546 ] Apache Spark commented on SPARK-17388: -- User 'HyukjinKwon' has created a pull reques

[jira] [Updated] (SPARK-17388) Support for inferring type date/timestamp/decimal for partition column

[ https://issues.apache.org/jira/browse/SPARK-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17388: - Priority: Major (was: Minor) > Support for inferring type date/timestamp/decimal for partition c

[jira] [Assigned] (SPARK-17388) Support for inferring type date/timestamp/decimal for partition column

[ https://issues.apache.org/jira/browse/SPARK-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17388: Assignee: (was: Apache Spark) > Support for inferring type date/timestamp/decimal for

[jira] [Assigned] (SPARK-17388) Support for inferring type date/timestamp/decimal for partition column

[ https://issues.apache.org/jira/browse/SPARK-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17388: Assignee: Apache Spark > Support for inferring type date/timestamp/decimal for partition c

[jira] [Updated] (SPARK-17388) Support for inferring type date/timestamp/decimal for partition column

[ https://issues.apache.org/jira/browse/SPARK-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17388: - Summary: Support for inferring type date/timestamp/decimal for partition column (was: Support fo

[jira] [Created] (SPARK-17388) Support for inferring type date/timestamp for partition column

Hyukjin Kwon created SPARK-17388: Summary: Support for inferring type date/timestamp for partition column Key: SPARK-17388 URL: https://issues.apache.org/jira/browse/SPARK-17388 Project: Spark

[jira] [Commented] (SPARK-16942) CREATE TABLE LIKE generates External table when source table is an External Hive Serde table

[ https://issues.apache.org/jira/browse/SPARK-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460530#comment-15460530 ] Apache Spark commented on SPARK-16942: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-16943) CREATE TABLE LIKE generates a non-empty table when source is a data source table

[ https://issues.apache.org/jira/browse/SPARK-16943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460529#comment-15460529 ] Apache Spark commented on SPARK-16943: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-17353) CREATE TABLE LIKE statements when Source is a VIEW

[ https://issues.apache.org/jira/browse/SPARK-17353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460528#comment-15460528 ] Apache Spark commented on SPARK-17353: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-16959) Table Comment in the CatalogTable returned from HiveMetastore is Always Empty

[ https://issues.apache.org/jira/browse/SPARK-16959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460531#comment-15460531 ] Apache Spark commented on SPARK-16959: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460062#comment-15460062 ] gurmukh singh commented on SPARK-17211: --- [~davies] After applying the patch, tested

[jira] [Updated] (SPARK-16948) Use metastore schema instead of inferring schema for ORC in HiveMetastoreCatalog

[ https://issues.apache.org/jira/browse/SPARK-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-16948: - Summary: Use metastore schema instead of inferring schema for ORC in HiveMetastoreCatalog

[jira] [Updated] (SPARK-16948) Use metastore schema instead of inferring schema in ORC in HiveMetastoreCatalog

[ https://issues.apache.org/jira/browse/SPARK-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-16948: - Summary: Use metastore schema instead of inferring schema in ORC in HiveMetastoreCatalog

[jira] [Assigned] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17386: Assignee: Apache Spark > Default trigger interval causes excessive RPC calls > ---

[jira] [Assigned] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17386: Assignee: (was: Apache Spark) > Default trigger interval causes excessive RPC calls >

[jira] [Commented] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459940#comment-15459940 ] Apache Spark commented on SPARK-17386: -- User 'frreiss' has created a pull request fo

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459925#comment-15459925 ] Miguel Tormo commented on SPARK-17211: -- [~gurmukhd], you say it works only for heap

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459905#comment-15459905 ] Marcelo Vanzin commented on SPARK-17387: A note about the workaround I posted: it

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459877#comment-15459877 ] gurmukh singh commented on SPARK-17211: --- Sure, will update soon with my findings.

[jira] [Created] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

Marcelo Vanzin created SPARK-17387: -- Summary: Creating SparkContext() from python without spark-submit ignores user conf Key: SPARK-17387 URL: https://issues.apache.org/jira/browse/SPARK-17387 Projec

[jira] [Updated] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frederick Reiss updated SPARK-17386: Description: The default trigger interval for a Structured Streaming query is {{Processing

[jira] [Created] (SPARK-17386) Default trigger interval causes excessive RPC calls

Frederick Reiss created SPARK-17386: --- Summary: Default trigger interval causes excessive RPC calls Key: SPARK-17386 URL: https://issues.apache.org/jira/browse/SPARK-17386 Project: Spark Iss

[jira] [Commented] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459859#comment-15459859 ] Apache Spark commented on SPARK-16334: -- User 'sameeragarwal' has created a pull requ

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459835#comment-15459835 ] Davies Liu commented on SPARK-17211: Could you try the patch ? https://github.com/apa

[jira] [Commented] (SPARK-17385) Update Data in mySql using spark

[ https://issues.apache.org/jira/browse/SPARK-17385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459827#comment-15459827 ] Suresh Thalamati commented on SPARK-17385: -- Update is not supported from spark.

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459770#comment-15459770 ] gurmukh singh commented on SPARK-17211: --- Thanks, I have tested by disabling UseCom

[jira] [Commented] (SPARK-15891) Make YARN logs less noisy

[ https://issues.apache.org/jira/browse/SPARK-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459765#comment-15459765 ] Apache Spark commented on SPARK-15891: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-15891) Make YARN logs less noisy

[ https://issues.apache.org/jira/browse/SPARK-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15891: Assignee: (was: Apache Spark) > Make YARN logs less noisy > -

[jira] [Assigned] (SPARK-15891) Make YARN logs less noisy

[ https://issues.apache.org/jira/browse/SPARK-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15891: Assignee: Apache Spark > Make YARN logs less noisy > - > >

[jira] [Resolved] (SPARK-17298) Require explicit CROSS join for cartesian products by default

[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17298. --- Resolution: Fixed Assignee: Srinath Fix Version/s: 2.1.0 > Require ex

[jira] [Resolved] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16334. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull reque

[jira] [Resolved] (SPARK-17230) Writing decimal to csv will result empty string if the decimal exceeds (20, 18)

[ https://issues.apache.org/jira/browse/SPARK-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17230. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Writing decimal to csv wil

[jira] [Commented] (SPARK-17368) Scala value classes create encoder problems and break at runtime

[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459707#comment-15459707 ] Jakob Odersky commented on SPARK-17368: --- Yeah macros would be awesome, something wi

[jira] [Commented] (SPARK-17368) Scala value classes create encoder problems and break at runtime

[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459649#comment-15459649 ] Aris Vlasakakis commented on SPARK-17368: - I actually had an identical first thou

[jira] [Created] (SPARK-17385) Update Data in mySql using spark

Farman Ali created SPARK-17385: -- Summary: Update Data in mySql using spark Key: SPARK-17385 URL: https://issues.apache.org/jira/browse/SPARK-17385 Project: Spark Issue Type: Bug Compon

[jira] [Commented] (SPARK-17368) Scala value classes create encoder problems and break at runtime

[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459587#comment-15459587 ] Jakob Odersky commented on SPARK-17368: --- I'm currently taking a look at this but my

[jira] [Comment Edited] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459539#comment-15459539 ] Herman van Hovell edited comment on SPARK-13721 at 9/2/16 8:40 PM:

[jira] [Commented] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459539#comment-15459539 ] Herman van Hovell commented on SPARK-13721: --- Could you explain what this would

[jira] [Commented] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459507#comment-15459507 ] Sameer Agarwal commented on SPARK-16334: [~tradersancho], [~keith.j.kraus] - Than

[jira] [Created] (SPARK-17384) SQL - Running query with outer join from 1.6 fails

Don Drake created SPARK-17384: - Summary: SQL - Running query with outer join from 1.6 fails Key: SPARK-17384 URL: https://issues.apache.org/jira/browse/SPARK-17384 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459490#comment-15459490 ] Don Drake commented on SPARK-13721: --- My nested structures aren't simple types, they are

[jira] [Assigned] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16334: Assignee: Sameer Agarwal (was: Apache Spark) > SQL query on parquet table java.lang.Array

[jira] [Commented] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459479#comment-15459479 ] Apache Spark commented on SPARK-16334: -- User 'sameeragarwal' has created a pull requ

[jira] [Assigned] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16334: Assignee: Apache Spark (was: Sameer Agarwal) > SQL query on parquet table java.lang.Array

[jira] [Updated] (SPARK-17316) Don't block StandaloneSchedulerBackend.executorRemoved

[ https://issues.apache.org/jira/browse/SPARK-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17316: - Fix Version/s: 1.6.3 > Don't block StandaloneSchedulerBackend.executorRemoved > -

[jira] [Resolved] (SPARK-17283) Cancel job in RDD.take() as soon as enough output is receieved

[ https://issues.apache.org/jira/browse/SPARK-17283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17283. Resolution: Later Closing as "Later" for now, since a simpler approach might yield similar gains.

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459422#comment-15459422 ] Sean Owen commented on SPARK-17381: --- The only thing i can think of that accumulates row

[jira] [Updated] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoSen Lee updated SPARK-17383: External issue URL: https://github.com/apache/spark/pull/14940 Description: In the labe

[jira] [Commented] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459364#comment-15459364 ] Apache Spark commented on SPARK-17383: -- User 'bookling' has created a pull request f

[jira] [Assigned] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17383: Assignee: Apache Spark > improvement LabelPropagation of graphx lib >

[jira] [Assigned] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17383: Assignee: (was: Apache Spark) > improvement LabelPropagation of graphx lib > -

[jira] [Created] (SPARK-17383) improvement LabelPropagation of graphx lib

XiaoSen Lee created SPARK-17383: --- Summary: improvement LabelPropagation of graphx lib Key: SPARK-17383 URL: https://issues.apache.org/jira/browse/SPARK-17383 Project: Spark Issue Type: Improvem

[jira] [Resolved] (SPARK-16711) YarnShuffleService doesn't re-init properly on YARN rolling upgrade

[ https://issues.apache.org/jira/browse/SPARK-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16711. Resolution: Fixed Assignee: Thomas Graves Fix Version/s: 2.1.0 > YarnShuffl

[jira] [Commented] (SPARK-17376) Spark version should be available in R

[ https://issues.apache.org/jira/browse/SPARK-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459113#comment-15459113 ] Apache Spark commented on SPARK-17376: -- User 'felixcheung' has created a pull reques

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459102#comment-15459102 ] Dongjoon Hyun commented on SPARK-17211: --- Oh, now I see the point of this issue. >

[jira] [Updated] (SPARK-17376) Spark version should be available in R

[ https://issues.apache.org/jira/browse/SPARK-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-17376: -- Assignee: Felix Cheung > Spark version should be available in R > -

[jira] [Resolved] (SPARK-17376) Spark version should be available in R

[ https://issues.apache.org/jira/browse/SPARK-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-17376. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue

[jira] [Updated] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17261: --- Assignee: Jeff Zhang > Using HiveContext after re-creating SparkContext in Spark 2.0 throws > "Java.

[jira] [Resolved] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17261. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull reque

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459040#comment-15459040 ] Davies Liu commented on SPARK-17211: [~migtor] Could you try this patch ? https://git

[jira] [Resolved] (SPARK-17351) Refactor JDBCRDD to expose JDBC -> SparkSQL conversion functionality

[ https://issues.apache.org/jira/browse/SPARK-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17351. --- Resolution: Fixed Fix Version/s: 2.1.0 > Refactor JDBCRDD to expose JDBC -> Sp

[jira] [Comment Edited] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459021#comment-15459021 ] Joao Duarte edited comment on SPARK-17381 at 9/2/16 4:49 PM: -

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459021#comment-15459021 ] Joao Duarte commented on SPARK-17381: - Hi Sean. Thanks for commenting. I set to Bloc

[jira] [Commented] (SPARK-10925) Exception when joining DataFrames

[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458996#comment-15458996 ] Chen Zhang commented on SPARK-10925: I have the same issue too. Very annoying. After

[jira] [Updated] (SPARK-16883) SQL decimal type is not properly cast to number when collecting SparkDataFrame

[ https://issues.apache.org/jira/browse/SPARK-16883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-16883: -- Assignee: Miao Wang > SQL decimal type is not properly cast to number when coll

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458947#comment-15458947 ] Miguel Tormo commented on SPARK-17211: -- I've just tried master from git, exactly the

[jira] [Updated] (SPARK-15509) R MLlib algorithms should support input columns "features" and "label"

[ https://issues.apache.org/jira/browse/SPARK-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-15509: -- Assignee: Xin Ren > R MLlib algorithms should support input columns "features"

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joao Duarte updated SPARK-17381: Description: I am running a Spark Streaming application from a Kinesis stream. After some hours ru

[jira] [Resolved] (SPARK-17382) Hello, I'd like to ask committers to help me get access to assign tasks to myself. I only plan to work on trivial bug fixes right now to get familiar with the project.

[ https://issues.apache.org/jira/browse/SPARK-17382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17382. --- Resolution: Invalid dev@ would be the better place to ask. We don't assign issues until they're compl

[jira] [Created] (SPARK-17382) Hello, I'd like to ask committers to help me get access to assign tasks to myself. I only plan to work on trivial bug fixes right now to get familiar with the project.

Dayne Sorvisto created SPARK-17382: -- Summary: Hello, I'd like to ask committers to help me get access to assign tasks to myself. I only plan to work on trivial bug fixes right now to get familiar with the project. Key: SPARK-173

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17381: -- Priority: Major (was: Blocker) [~joaomaiaduarte] don't set Blocker. You may be onto something here, bu

[jira] [Commented] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458862#comment-15458862 ] Sean Owen commented on SPARK-17380: --- This doesn't show evidence of a memory leak. You m

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458861#comment-15458861 ] Adam Roberts commented on SPARK-17379: -- Sounds good, testing it out now and then I'l

[jira] [Resolved] (SPARK-10637) DataFrames: saving with nested User Data Types

[ https://issues.apache.org/jira/browse/SPARK-10637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10637. --- Resolution: Duplicate > DataFrames: saving with nested User Data Types >

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joao updated SPARK-17381: - Description: I am running a Spark Streaming application from a Kinesis stream. After some hours running it gets

[jira] [Commented] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458827#comment-15458827 ] Michal Kielbowicz commented on SPARK-17335: --- Cool! I unfortunately was limited

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joao updated SPARK-17381: - Description: I am running a Spark Streaming application from a Kinesis stream. After some hours running it gets

[jira] [Created] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

Joao created SPARK-17381: Summary: Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics Key: SPARK-17381 URL: https://issues.apache.org/jira/browse/SPARK-17381 Project: Spark Issue Type: Bu

[jira] [Assigned] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17335: Assignee: Apache Spark > Creating Hive table from Spark data > ---

[jira] [Assigned] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17335: Assignee: (was: Apache Spark) > Creating Hive table from Spark data >

[jira] [Commented] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458811#comment-15458811 ] Apache Spark commented on SPARK-17335: -- User 'hvanhovell' has created a pull request

[jira] [Resolved] (SPARK-16984) executeTake tries all partitions if first parition is empty

[ https://issues.apache.org/jira/browse/SPARK-16984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16984. --- Resolution: Fixed Assignee: Robert Kruszewski Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-16614) DirectJoin with DataSource for SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-16614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458784#comment-15458784 ] Russell Spitzer commented on SPARK-16614: - Yes. This would be similar to how Pres

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458718#comment-15458718 ] Sean Owen commented on SPARK-17379: --- I see, how about just updating to 4.0.41 then? It

[jira] [Updated] (SPARK-16935) Verification of Function-related ExternalCatalog APIs

[ https://issues.apache.org/jira/browse/SPARK-16935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16935: Assignee: Xiao Li > Verification of Function-related ExternalCatalog APIs > ---

[jira] [Resolved] (SPARK-16935) Verification of Function-related ExternalCatalog APIs

[ https://issues.apache.org/jira/browse/SPARK-16935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16935. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull req

[jira] [Comment Edited] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458656#comment-15458656 ] Adam Roberts edited comment on SPARK-17379 at 9/2/16 2:18 PM: -

[jira] [Comment Edited] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458673#comment-15458673 ] Xeto edited comment on SPARK-17380 at 9/2/16 2:24 PM: -- Attaching Gan

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Attachment: memory-after-freeze.png Used memory after Spark has frozen - is finally reduced but doesn't help Spar

[jira] [Commented] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458669#comment-15458669 ] Xeto commented on SPARK-17380: -- Could you advise how to obtain such an evidence? Not storing

[jira] [Updated] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17211: -- Summary: Broadcast join produces incorrect results when compressed Oops differs between driver, executo

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458656#comment-15458656 ] Adam Roberts commented on SPARK-17379: -- Good point about the 1.6 stream, this change

[jira] [Commented] (SPARK-17377) Joining Datasets read and aggregated from a partitioned Parquet file gives wrong results

[ https://issues.apache.org/jira/browse/SPARK-17377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458642#comment-15458642 ] Sean Owen commented on SPARK-17377: --- Probably related: https://issues.apache.org/jira/b

[jira] [Commented] (SPARK-17378) Upgrade snappy-java to 1.1.2.6

[ https://issues.apache.org/jira/browse/SPARK-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458630#comment-15458630 ] Sean Owen commented on SPARK-17378: --- Also looks good for 2.0 and 1.6. > Upgrade snappy

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17380: -- Priority: Major (was: Critical) I don't think this is necessarily evidence of a memory leak. With a le

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458622#comment-15458622 ] Sean Owen commented on SPARK-17379: --- Looks OK, for 2.0 or even 1.6 if it's fixing reaso

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Summary: Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?) (was: Spa

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Priority: Critical (was: Major) Description: Running Spark Streaming 2.0.0 on AWS EMR 5.0.0 consuming fro

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Attachment: memory.png Attaching Ganglia cluster memory growth graph > Spark streaming with a multi shard Kinesi

[jira] [Commented] (SPARK-17307) Document what all access is needed on S3 bucket when trying to save a model

[ https://issues.apache.org/jira/browse/SPARK-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458485#comment-15458485 ] Steve Loughran commented on SPARK-17307: I don't think it should. I think maybe s

  1   2   >