[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15460062#comment-15460062 ] gurmukh singh commented on SPARK-17211: --- [~davies] After applying the patch, tested with various

[jira] [Updated] (SPARK-16948) Use metastore schema instead of inferring schema for ORC in HiveMetastoreCatalog

[ https://issues.apache.org/jira/browse/SPARK-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-16948: - Summary: Use metastore schema instead of inferring schema for ORC in

[jira] [Updated] (SPARK-16948) Use metastore schema instead of inferring schema in ORC in HiveMetastoreCatalog

[ https://issues.apache.org/jira/browse/SPARK-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-16948: - Summary: Use metastore schema instead of inferring schema in ORC in HiveMetastoreCatalog

[jira] [Assigned] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17386: Assignee: Apache Spark > Default trigger interval causes excessive RPC calls >

[jira] [Assigned] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17386: Assignee: (was: Apache Spark) > Default trigger interval causes excessive RPC calls >

[jira] [Commented] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459940#comment-15459940 ] Apache Spark commented on SPARK-17386: -- User 'frreiss' has created a pull request for this issue:

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459925#comment-15459925 ] Miguel Tormo commented on SPARK-17211: -- [~gurmukhd], you say it works only for heap < 32 GB, which

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459905#comment-15459905 ] Marcelo Vanzin commented on SPARK-17387: A note about the workaround I posted: it doesn't seem to

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459877#comment-15459877 ] gurmukh singh commented on SPARK-17211: --- Sure, will update soon with my findings. > Broadcast join

[jira] [Created] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

Marcelo Vanzin created SPARK-17387: -- Summary: Creating SparkContext() from python without spark-submit ignores user conf Key: SPARK-17387 URL: https://issues.apache.org/jira/browse/SPARK-17387

[jira] [Updated] (SPARK-17386) Default trigger interval causes excessive RPC calls

[ https://issues.apache.org/jira/browse/SPARK-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frederick Reiss updated SPARK-17386: Description: The default trigger interval for a Structured Streaming query is

[jira] [Created] (SPARK-17386) Default trigger interval causes excessive RPC calls

Frederick Reiss created SPARK-17386: --- Summary: Default trigger interval causes excessive RPC calls Key: SPARK-17386 URL: https://issues.apache.org/jira/browse/SPARK-17386 Project: Spark

[jira] [Commented] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459859#comment-15459859 ] Apache Spark commented on SPARK-16334: -- User 'sameeragarwal' has created a pull request for this

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459835#comment-15459835 ] Davies Liu commented on SPARK-17211: Could you try the patch ?

[jira] [Commented] (SPARK-17385) Update Data in mySql using spark

[ https://issues.apache.org/jira/browse/SPARK-17385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459827#comment-15459827 ] Suresh Thalamati commented on SPARK-17385: -- Update is not supported from spark. Only option

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459770#comment-15459770 ] gurmukh singh commented on SPARK-17211: --- Thanks, I have tested by disabling UseCompressedOops

[jira] [Commented] (SPARK-15891) Make YARN logs less noisy

[ https://issues.apache.org/jira/browse/SPARK-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459765#comment-15459765 ] Apache Spark commented on SPARK-15891: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15891) Make YARN logs less noisy

[ https://issues.apache.org/jira/browse/SPARK-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15891: Assignee: (was: Apache Spark) > Make YARN logs less noisy > -

[jira] [Assigned] (SPARK-15891) Make YARN logs less noisy

[ https://issues.apache.org/jira/browse/SPARK-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15891: Assignee: Apache Spark > Make YARN logs less noisy > - > >

[jira] [Resolved] (SPARK-17298) Require explicit CROSS join for cartesian products by default

[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17298. --- Resolution: Fixed Assignee: Srinath Fix Version/s: 2.1.0 > Require

[jira] [Resolved] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16334. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Resolved] (SPARK-17230) Writing decimal to csv will result empty string if the decimal exceeds (20, 18)

[ https://issues.apache.org/jira/browse/SPARK-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17230. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Writing decimal to csv

[jira] [Commented] (SPARK-17368) Scala value classes create encoder problems and break at runtime

[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459707#comment-15459707 ] Jakob Odersky commented on SPARK-17368: --- Yeah macros would be awesome, something with Scala.meta

[jira] [Commented] (SPARK-17368) Scala value classes create encoder problems and break at runtime

[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459649#comment-15459649 ] Aris Vlasakakis commented on SPARK-17368: - I actually had an identical first thought from my

[jira] [Created] (SPARK-17385) Update Data in mySql using spark

Farman Ali created SPARK-17385: -- Summary: Update Data in mySql using spark Key: SPARK-17385 URL: https://issues.apache.org/jira/browse/SPARK-17385 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17368) Scala value classes create encoder problems and break at runtime

[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459587#comment-15459587 ] Jakob Odersky commented on SPARK-17368: --- I'm currently taking a look at this but my first analysis

[jira] [Comment Edited] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459539#comment-15459539 ] Herman van Hovell edited comment on SPARK-13721 at 9/2/16 8:40 PM: ---

[jira] [Commented] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459539#comment-15459539 ] Herman van Hovell commented on SPARK-13721: --- Could you explain what this would looks like? I am

[jira] [Commented] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459507#comment-15459507 ] Sameer Agarwal commented on SPARK-16334: [~tradersancho], [~keith.j.kraus] - Thank you once again

[jira] [Created] (SPARK-17384) SQL - Running query with outer join from 1.6 fails

Don Drake created SPARK-17384: - Summary: SQL - Running query with outer join from 1.6 fails Key: SPARK-17384 URL: https://issues.apache.org/jira/browse/SPARK-17384 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459490#comment-15459490 ] Don Drake commented on SPARK-13721: --- My nested structures aren't simple types, they are structs (case

[jira] [Assigned] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16334: Assignee: Sameer Agarwal (was: Apache Spark) > SQL query on parquet table

[jira] [Commented] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459479#comment-15459479 ] Apache Spark commented on SPARK-16334: -- User 'sameeragarwal' has created a pull request for this

[jira] [Assigned] (SPARK-16334) SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16334: Assignee: Apache Spark (was: Sameer Agarwal) > SQL query on parquet table

[jira] [Updated] (SPARK-17316) Don't block StandaloneSchedulerBackend.executorRemoved

[ https://issues.apache.org/jira/browse/SPARK-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17316: - Fix Version/s: 1.6.3 > Don't block StandaloneSchedulerBackend.executorRemoved >

[jira] [Resolved] (SPARK-17283) Cancel job in RDD.take() as soon as enough output is receieved

[ https://issues.apache.org/jira/browse/SPARK-17283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17283. Resolution: Later Closing as "Later" for now, since a simpler approach might yield similar gains.

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459422#comment-15459422 ] Sean Owen commented on SPARK-17381: --- The only thing i can think of that accumulates row-like data are

[jira] [Updated] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoSen Lee updated SPARK-17383: External issue URL: https://github.com/apache/spark/pull/14940 Description: In the

[jira] [Commented] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459364#comment-15459364 ] Apache Spark commented on SPARK-17383: -- User 'bookling' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17383: Assignee: Apache Spark > improvement LabelPropagation of graphx lib >

[jira] [Assigned] (SPARK-17383) improvement LabelPropagation of graphx lib

[ https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17383: Assignee: (was: Apache Spark) > improvement LabelPropagation of graphx lib >

[jira] [Created] (SPARK-17383) improvement LabelPropagation of graphx lib

XiaoSen Lee created SPARK-17383: --- Summary: improvement LabelPropagation of graphx lib Key: SPARK-17383 URL: https://issues.apache.org/jira/browse/SPARK-17383 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-16711) YarnShuffleService doesn't re-init properly on YARN rolling upgrade

[ https://issues.apache.org/jira/browse/SPARK-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16711. Resolution: Fixed Assignee: Thomas Graves Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-17376) Spark version should be available in R

[ https://issues.apache.org/jira/browse/SPARK-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459113#comment-15459113 ] Apache Spark commented on SPARK-17376: -- User 'felixcheung' has created a pull request for this

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459102#comment-15459102 ] Dongjoon Hyun commented on SPARK-17211: --- Oh, now I see the point of this issue. > Broadcast join

[jira] [Updated] (SPARK-17376) Spark version should be available in R

[ https://issues.apache.org/jira/browse/SPARK-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-17376: -- Assignee: Felix Cheung > Spark version should be available in R >

[jira] [Resolved] (SPARK-17376) Spark version should be available in R

[ https://issues.apache.org/jira/browse/SPARK-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-17376. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue

[jira] [Updated] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17261: --- Assignee: Jeff Zhang > Using HiveContext after re-creating SparkContext in Spark 2.0 throws >

[jira] [Resolved] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17261. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459040#comment-15459040 ] Davies Liu commented on SPARK-17211: [~migtor] Could you try this patch ?

[jira] [Resolved] (SPARK-17351) Refactor JDBCRDD to expose JDBC -> SparkSQL conversion functionality

[ https://issues.apache.org/jira/browse/SPARK-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17351. --- Resolution: Fixed Fix Version/s: 2.1.0 > Refactor JDBCRDD to expose JDBC ->

[jira] [Comment Edited] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459021#comment-15459021 ] Joao Duarte edited comment on SPARK-17381 at 9/2/16 4:49 PM: - Hi Sean.

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459021#comment-15459021 ] Joao Duarte commented on SPARK-17381: - Hi Sean. Thanks for commenting. I set to Blocker because I

[jira] [Commented] (SPARK-10925) Exception when joining DataFrames

[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458996#comment-15458996 ] Chen Zhang commented on SPARK-10925: I have the same issue too. Very annoying. After I do several

[jira] [Updated] (SPARK-16883) SQL decimal type is not properly cast to number when collecting SparkDataFrame

[ https://issues.apache.org/jira/browse/SPARK-16883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-16883: -- Assignee: Miao Wang > SQL decimal type is not properly cast to number when

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458947#comment-15458947 ] Miguel Tormo commented on SPARK-17211: -- I've just tried master from git, exactly the same results.

[jira] [Updated] (SPARK-15509) R MLlib algorithms should support input columns "features" and "label"

[ https://issues.apache.org/jira/browse/SPARK-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-15509: -- Assignee: Xin Ren > R MLlib algorithms should support input columns "features"

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joao Duarte updated SPARK-17381: Description: I am running a Spark Streaming application from a Kinesis stream. After some hours

[jira] [Resolved] (SPARK-17382) Hello, I'd like to ask committers to help me get access to assign tasks to myself. I only plan to work on trivial bug fixes right now to get familiar with the project.

[ https://issues.apache.org/jira/browse/SPARK-17382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17382. --- Resolution: Invalid dev@ would be the better place to ask. We don't assign issues until they're

[jira] [Created] (SPARK-17382) Hello, I'd like to ask committers to help me get access to assign tasks to myself. I only plan to work on trivial bug fixes right now to get familiar with the project.

Dayne Sorvisto created SPARK-17382: -- Summary: Hello, I'd like to ask committers to help me get access to assign tasks to myself. I only plan to work on trivial bug fixes right now to get familiar with the project. Key:

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17381: -- Priority: Major (was: Blocker) [~joaomaiaduarte] don't set Blocker. You may be onto something here,

[jira] [Commented] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458862#comment-15458862 ] Sean Owen commented on SPARK-17380: --- This doesn't show evidence of a memory leak. You may be low on

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458861#comment-15458861 ] Adam Roberts commented on SPARK-17379: -- Sounds good, testing it out now and then I'll either get the

[jira] [Resolved] (SPARK-10637) DataFrames: saving with nested User Data Types

[ https://issues.apache.org/jira/browse/SPARK-10637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10637. --- Resolution: Duplicate > DataFrames: saving with nested User Data Types >

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joao updated SPARK-17381: - Description: I am running a Spark Streaming application from a Kinesis stream. After some hours running it gets

[jira] [Commented] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458827#comment-15458827 ] Michal Kielbowicz commented on SPARK-17335: --- Cool! I unfortunately was limited by my company

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joao updated SPARK-17381: - Description: I am running a Spark Streaming application from a Kinesis stream. After some hours running it gets

[jira] [Created] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

Joao created SPARK-17381: Summary: Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics Key: SPARK-17381 URL: https://issues.apache.org/jira/browse/SPARK-17381 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17335: Assignee: Apache Spark > Creating Hive table from Spark data >

[jira] [Assigned] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17335: Assignee: (was: Apache Spark) > Creating Hive table from Spark data >

[jira] [Commented] (SPARK-17335) Creating Hive table from Spark data

[ https://issues.apache.org/jira/browse/SPARK-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458811#comment-15458811 ] Apache Spark commented on SPARK-17335: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Resolved] (SPARK-16984) executeTake tries all partitions if first parition is empty

[ https://issues.apache.org/jira/browse/SPARK-16984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16984. --- Resolution: Fixed Assignee: Robert Kruszewski Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-16614) DirectJoin with DataSource for SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-16614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458784#comment-15458784 ] Russell Spitzer commented on SPARK-16614: - Yes. This would be similar to how Presto works by

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458718#comment-15458718 ] Sean Owen commented on SPARK-17379: --- I see, how about just updating to 4.0.41 then? It has the fix you

[jira] [Updated] (SPARK-16935) Verification of Function-related ExternalCatalog APIs

[ https://issues.apache.org/jira/browse/SPARK-16935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16935: Assignee: Xiao Li > Verification of Function-related ExternalCatalog APIs >

[jira] [Resolved] (SPARK-16935) Verification of Function-related ExternalCatalog APIs

[ https://issues.apache.org/jira/browse/SPARK-16935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16935. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Comment Edited] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458656#comment-15458656 ] Adam Roberts edited comment on SPARK-17379 at 9/2/16 2:18 PM: -- Good point

[jira] [Comment Edited] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458673#comment-15458673 ] Xeto edited comment on SPARK-17380 at 9/2/16 2:24 PM: -- Attaching Ganglia post

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Attachment: memory-after-freeze.png Used memory after Spark has frozen - is finally reduced but doesn't help

[jira] [Commented] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458669#comment-15458669 ] Xeto commented on SPARK-17380: -- Could you advise how to obtain such an evidence? Not storing anything in

[jira] [Updated] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17211: -- Summary: Broadcast join produces incorrect results when compressed Oops differs between driver,

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458656#comment-15458656 ] Adam Roberts commented on SPARK-17379: -- Good point about the 1.6 stream, this change isn't as

[jira] [Commented] (SPARK-17377) Joining Datasets read and aggregated from a partitioned Parquet file gives wrong results

[ https://issues.apache.org/jira/browse/SPARK-17377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458642#comment-15458642 ] Sean Owen commented on SPARK-17377: --- Probably related:

[jira] [Commented] (SPARK-17378) Upgrade snappy-java to 1.1.2.6

[ https://issues.apache.org/jira/browse/SPARK-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458630#comment-15458630 ] Sean Owen commented on SPARK-17378: --- Also looks good for 2.0 and 1.6. > Upgrade snappy-java to 1.1.2.6

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17380: -- Priority: Major (was: Critical) I don't think this is necessarily evidence of a memory leak. With a

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.1.5.Final

[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458622#comment-15458622 ] Sean Owen commented on SPARK-17379: --- Looks OK, for 2.0 or even 1.6 if it's fixing reasonably important

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Summary: Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?) (was:

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Priority: Critical (was: Major) Description: Running Spark Streaming 2.0.0 on AWS EMR 5.0.0 consuming

[jira] [Updated] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days

[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xeto updated SPARK-17380: - Attachment: memory.png Attaching Ganglia cluster memory growth graph > Spark streaming with a multi shard

[jira] [Commented] (SPARK-17307) Document what all access is needed on S3 bucket when trying to save a model

[ https://issues.apache.org/jira/browse/SPARK-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458485#comment-15458485 ] Steve Loughran commented on SPARK-17307: I don't think it should. I think maybe some

[jira] [Created] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days

Xeto created SPARK-17380: Summary: Spark streaming with a multi shard Kinesis freezes after several days Key: SPARK-17380 URL: https://issues.apache.org/jira/browse/SPARK-17380 Project: Spark Issue

[jira] [Commented] (SPARK-11560) Optimize KMeans implementation

[ https://issues.apache.org/jira/browse/SPARK-11560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458467#comment-15458467 ] Apache Spark commented on SPARK-11560: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Commented] (SPARK-8519) Blockify distance computation in k-means

[ https://issues.apache.org/jira/browse/SPARK-8519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458466#comment-15458466 ] Apache Spark commented on SPARK-8519: - User 'yanboliang' has created a pull request for this issue:

[jira] [Commented] (SPARK-17373) spark+hive+hbase+hbaseIntegration not working

[ https://issues.apache.org/jira/browse/SPARK-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458431#comment-15458431 ] prasannaP commented on SPARK-17373: --- I am new to spark.Now I am able to query Hive managed tables

[jira] [Comment Edited] (SPARK-17373) spark+hive+hbase+hbaseIntegration not working

[ https://issues.apache.org/jira/browse/SPARK-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458406#comment-15458406 ] prasannaP edited comment on SPARK-17373 at 9/2/16 12:35 PM: Thanks for your

[jira] [Comment Edited] (SPARK-17373) spark+hive+hbase+hbaseIntegration not working

[ https://issues.apache.org/jira/browse/SPARK-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458406#comment-15458406 ] prasannaP edited comment on SPARK-17373 at 9/2/16 12:34 PM: Thanks for reply

[jira] [Commented] (SPARK-17373) spark+hive+hbase+hbaseIntegration not working

[ https://issues.apache.org/jira/browse/SPARK-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458406#comment-15458406 ] prasannaP commented on SPARK-17373: --- How can i add HBase classes and in which classpath. Can you

[jira] [Updated] (SPARK-17373) spark+hive+hbase+hbaseIntegration not working

[ https://issues.apache.org/jira/browse/SPARK-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] prasannaP updated SPARK-17373: -- Description: SparkSQL+Hive+Hbase+HbaseIntegration doesn't work Hi, I am getting error when I am

[jira] [Commented] (SPARK-7877) Support non-persistent cluster mode

[ https://issues.apache.org/jira/browse/SPARK-7877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458326#comment-15458326 ] Philipp Hoffmann commented on SPARK-7877: - Submitted a pull request for this to make the timeout

[jira] [Assigned] (SPARK-7877) Support non-persistent cluster mode

[ https://issues.apache.org/jira/browse/SPARK-7877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7877: --- Assignee: (was: Apache Spark) > Support non-persistent cluster mode >

  1   2   >