[jira] [Updated] (SPARK-17602) PySpark - Performance Optimization Large Size of Broadcast Variable

2016-09-19 Thread Xiao Ming Bao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Ming Bao updated SPARK-17602: -- Attachment: PySpark – Performance Optimization for Large Size of Broadcast variable.pdf Design

[jira] [Commented] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

2016-09-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505810#comment-15505810 ] Reynold Xin commented on SPARK-15698: - This one is important for streaming running 24

[jira] [Updated] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

2016-09-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15698: Priority: Major (was: Minor) > Ability to remove old metadata for structure streaming MetadataLog

[jira] [Commented] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

2016-09-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505799#comment-15505799 ] Saisai Shao commented on SPARK-15698: - I think [~rxin] set this target version before

[jira] [Resolved] (SPARK-17513) StreamExecution should discard unneeded metadata

2016-09-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17513. - Resolution: Fixed Assignee: Frederick Reiss Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

2016-09-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17603: Issue Type: Improvement (was: Bug) > Utilize Hive-generated Statistics For Partitioned Tables > --

[jira] [Assigned] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17603: Assignee: Apache Spark > Utilize Hive-generated Statistics For Partitioned Tables > --

[jira] [Commented] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505580#comment-15505580 ] Apache Spark commented on SPARK-17603: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17603: Assignee: (was: Apache Spark) > Utilize Hive-generated Statistics For Partitioned Tabl

[jira] [Created] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

2016-09-19 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17603: --- Summary: Utilize Hive-generated Statistics For Partitioned Tables Key: SPARK-17603 URL: https://issues.apache.org/jira/browse/SPARK-17603 Project: Spark Issue Type: Bu

[jira] [Resolved] (SPARK-17163) Merge MLOR into a single LOR interface

2016-09-19 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-17163. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14834 [https://github.com/ap

[jira] [Updated] (SPARK-17163) Merge MLOR into a single LOR interface

2016-09-19 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-17163: Assignee: Seth Hendrickson > Merge MLOR into a single LOR interface > -

[jira] [Updated] (SPARK-17528) MutableProjection should not cache content from the input row

2016-09-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17528: Target Version/s: 2.1.0 (was: 2.0.1, 2.1.0) > MutableProjection should not cache content from the

[jira] [Resolved] (SPARK-17160) GetExternalRowField does not properly escape field names, causing generated code not to compile

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17160. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull reque

[jira] [Closed] (SPARK-17054) SparkR can not run in yarn-cluster mode on mac os

2016-09-19 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang closed SPARK-17054. -- Resolution: Won't Fix Close it as it is resolved somewhere else. > SparkR can not run in yarn-cluster

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

2016-09-19 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505411#comment-15505411 ] Yin Huai commented on SPARK-17549: -- Forgot to say. Thank you for the investigation! Shou

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505409#comment-15505409 ] Apache Spark commented on SPARK-17549: -- User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

2016-09-19 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505404#comment-15505404 ] Yin Huai commented on SPARK-17549: -- [~vanzin] Let's revert this patch for now. So, this

[jira] [Created] (SPARK-17602) PySpark - Performance Optimization Large Size of Broadcast Variable

2016-09-19 Thread Xiao Ming Bao (JIRA)
Xiao Ming Bao created SPARK-17602: - Summary: PySpark - Performance Optimization Large Size of Broadcast Variable Key: SPARK-17602 URL: https://issues.apache.org/jira/browse/SPARK-17602 Project: Spark

[jira] [Comment Edited] (SPARK-17597) HiveContext cannot create a table named sort

2016-09-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505316#comment-15505316 ] Hyukjin Kwon edited comment on SPARK-17597 at 9/20/16 2:15 AM:

[jira] [Commented] (SPARK-17597) HiveContext cannot create a table named sort

2016-09-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505340#comment-15505340 ] Hyukjin Kwon commented on SPARK-17597: -- [~saif.a.ellafi] Do you mind if I ask to con

[jira] [Comment Edited] (SPARK-17597) HiveContext cannot create a table named sort

2016-09-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505316#comment-15505316 ] Hyukjin Kwon edited comment on SPARK-17597 at 9/20/16 2:16 AM:

[jira] [Commented] (SPARK-17597) HiveContext cannot create a table named sort

2016-09-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505316#comment-15505316 ] Hyukjin Kwon commented on SPARK-17597: -- BTW, I can't reproduce this against master.

[jira] [Commented] (SPARK-17570) Avoid Hash and Exchange in Sort Merge join if bucketing factor is multiple for tables

2016-09-19 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505234#comment-15505234 ] Tejas Patil commented on SPARK-17570: - ping !!! > Avoid Hash and Exchange in Sort Me

[jira] [Commented] (SPARK-10815) API design: data sources and sinks

2016-09-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505186#comment-15505186 ] Reynold Xin commented on SPARK-10815: - Source depends on DataFrame, which can really

[jira] [Commented] (SPARK-10815) API design: data sources and sinks

2016-09-19 Thread Frederick Reiss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505102#comment-15505102 ] Frederick Reiss commented on SPARK-10815: - I'm confused by the current descriptio

[jira] [Resolved] (SPARK-16296) add null check for key when create map data in encoder

2016-09-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16296. - Resolution: Fixed Fix Version/s: 2.1.0 Target Version/s: 2.1.0 (was: 2.0.1) > a

[jira] [Commented] (SPARK-16296) add null check for key when create map data in encoder

2016-09-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505099#comment-15505099 ] Wenchen Fan commented on SPARK-16296: - This is a minor issue and it's hard to fix it

[jira] [Commented] (SPARK-17601) SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long

2016-09-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505066#comment-15505066 ] Hyukjin Kwon commented on SPARK-17601: -- We might have to avoid to open multiple rela

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

2016-09-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505060#comment-15505060 ] Marcelo Vanzin commented on SPARK-17549: Replying to myself: yes, this seems to b

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

2016-09-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504950#comment-15504950 ] Marcelo Vanzin commented on SPARK-17549: [~yhuai] There was something that was b

[jira] [Created] (SPARK-17601) SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long

2016-09-19 Thread Gang Wu (JIRA)
Gang Wu created SPARK-17601: --- Summary: SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long Key: SPARK-17601 URL: https://issues.apache.org/jira

[jira] [Commented] (SPARK-17160) GetExternalRowField does not properly escape field names, causing generated code not to compile

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504897#comment-15504897 ] Apache Spark commented on SPARK-17160: -- User 'JoshRosen' has created a pull request

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504892#comment-15504892 ] Apache Spark commented on SPARK-17477: -- User 'wgtmac' has created a pull request for

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504811#comment-15504811 ] Apache Spark commented on SPARK-17477: -- User 'wgtmac' has created a pull request for

[jira] [Commented] (SPARK-17494) Floor/ceil of decimal returns wrong result if it's in compact format

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504803#comment-15504803 ] Apache Spark commented on SPARK-17494: -- User 'davies' has created a pull request for

[jira] [Updated] (SPARK-17494) Floor/ceil of decimal returns wrong result if it's in compact format

2016-09-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17494: --- Summary: Floor/ceil of decimal returns wrong result if it's in compact format (was: Floor function r

[jira] [Updated] (SPARK-17592) SQL: CAST string as INT inconsistent with Hive

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17592: --- Labels: correctness (was: ) > SQL: CAST string as INT inconsistent with Hive > -

[jira] [Created] (SPARK-17600) Cannot set public address for Worker and Master Web UI

2016-09-19 Thread Jakub Liska (JIRA)
Jakub Liska created SPARK-17600: --- Summary: Cannot set public address for Worker and Master Web UI Key: SPARK-17600 URL: https://issues.apache.org/jira/browse/SPARK-17600 Project: Spark Issue Ty

[jira] [Commented] (SPARK-17051) we should use hadoopConf in InsertIntoHiveTable

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504649#comment-15504649 ] Josh Rosen commented on SPARK-17051: [~cloud_fan], what's the status of this issue? S

[jira] [Commented] (SPARK-17599) Folder deletion after globbing may fail StructuredStreaming jobs

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504619#comment-15504619 ] Apache Spark commented on SPARK-17599: -- User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-17599) Folder deletion after globbing may fail StructuredStreaming jobs

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17599: Assignee: (was: Apache Spark) > Folder deletion after globbing may fail StructuredStre

[jira] [Assigned] (SPARK-17599) Folder deletion after globbing may fail StructuredStreaming jobs

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17599: Assignee: Apache Spark > Folder deletion after globbing may fail StructuredStreaming jobs

[jira] [Updated] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

2016-09-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17100: --- Fix Version/s: (was: 2.2.0) 2.1.0 > pyspark filter on a udf column after join

[jira] [Resolved] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

2016-09-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17100. Resolution: Fixed Fix Version/s: 2.2.0 2.0.1 Issue resolved by pull reque

[jira] [Assigned] (SPARK-17160) GetExternalRowField does not properly escape field names, causing generated code not to compile

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-17160: -- Assignee: Josh Rosen > GetExternalRowField does not properly escape field names, causing gener

[jira] [Created] (SPARK-17599) Folder deletion after globbing may fail StructuredStreaming jobs

2016-09-19 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-17599: --- Summary: Folder deletion after globbing may fail StructuredStreaming jobs Key: SPARK-17599 URL: https://issues.apache.org/jira/browse/SPARK-17599 Project: Spark

[jira] [Updated] (SPARK-17598) User-friendly name for Spark Thrift Server in web UI

2016-09-19 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-17598: Attachment: spark-thriftserver-webui.png > User-friendly name for Spark Thrift Server in we

[jira] [Created] (SPARK-17598) User-friendly name for Spark Thrift Server in web UI

2016-09-19 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-17598: --- Summary: User-friendly name for Spark Thrift Server in web UI Key: SPARK-17598 URL: https://issues.apache.org/jira/browse/SPARK-17598 Project: Spark Is

[jira] [Commented] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504410#comment-15504410 ] Josh Rosen commented on SPARK-15698: [~jerryshao] [~zsxwing], should 2.0.1 really be

[jira] [Updated] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16295: --- Assignee: (was: Cheng Lian) > Extract SQL programming guide example snippets from source files in

[jira] [Resolved] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-16295. Resolution: Fixed Fix Version/s: 2.0.1 > Extract SQL programming guide example snippets from

[jira] [Updated] (SPARK-16323) Avoid unnecessary cast when doing integral divide

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16323: --- Target Version/s: 2.1.0 (was: 2.0.1, 2.1.0) > Avoid unnecessary cast when doing integral divide > --

[jira] [Commented] (SPARK-16323) Avoid unnecessary cast when doing integral divide

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504403#comment-15504403 ] Josh Rosen commented on SPARK-16323: FYI I'm going to untarget this from 2.0.1 becaus

[jira] [Commented] (SPARK-16296) add null check for key when create map data in encoder

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504395#comment-15504395 ] Josh Rosen commented on SPARK-16296: [~cloud_fan], I notice that this issue is target

[jira] [Commented] (SPARK-17596) Streaming job lacks Scala runtime methods

2016-09-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504337#comment-15504337 ] Sean Owen commented on SPARK-17596: --- This sounds like a Scala version mismatch problem,

[jira] [Commented] (SPARK-17588) java.lang.AssertionError: assertion failed: lapack.dppsv returned 105. when running glm using gaussian link function.

2016-09-19 Thread sai pavan kumar chitti (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504369#comment-15504369 ] sai pavan kumar chitti commented on SPARK-17588: input is a single csv fi

[jira] [Commented] (SPARK-17588) java.lang.AssertionError: assertion failed: lapack.dppsv returned 105. when running glm using gaussian link function.

2016-09-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504350#comment-15504350 ] Sean Owen commented on SPARK-17588: --- I meant more like, what is the size of the input?

[jira] [Commented] (SPARK-17365) Kill multiple executors together to reduce lock contention

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504347#comment-15504347 ] Apache Spark commented on SPARK-17365: -- User 'dhruve' has created a pull request for

[jira] [Updated] (SPARK-16439) Incorrect information in SQL Query details

2016-09-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16439: --- Fix Version/s: (was: 2.2.0) 2.1.0 > Incorrect information in SQL Query details

[jira] [Commented] (SPARK-17594) Bug in left-outer join

2016-09-19 Thread Virgil Palanciuc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504331#comment-15504331 ] Virgil Palanciuc commented on SPARK-17594: -- Hmmm.. no, I can't reproduce it with

[jira] [Assigned] (SPARK-17494) Floor function rounds up during join

2016-09-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-17494: -- Assignee: Davies Liu > Floor function rounds up during join >

[jira] [Commented] (SPARK-17597) HiveContext cannot create a table named sort

2016-09-19 Thread Saif Addin Ellafi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504340#comment-15504340 ] Saif Addin Ellafi commented on SPARK-17597: --- Regarded it as a problem since the

[jira] [Updated] (SPARK-16439) Incorrect information in SQL Query details

2016-09-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16439: --- Assignee: Davies Liu (was: Maciej Bryński) > Incorrect information in SQL Query details > --

[jira] [Updated] (SPARK-17597) HiveContext cannot create a table named sort

2016-09-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17597: -- Summary: HiveContext cannot create a table named sort (was: HiveContext cannot create a table named so

[jira] [Resolved] (SPARK-16439) Incorrect information in SQL Query details

2016-09-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16439. Resolution: Fixed Fix Version/s: (was: 2.0.0) 2.2.0

[jira] [Commented] (SPARK-16407) Allow users to supply custom StreamSinkProviders

2016-09-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504301#comment-15504301 ] Michael Armbrust commented on SPARK-16407: -- You are taking an experimental inter

[jira] [Commented] (SPARK-16407) Allow users to supply custom StreamSinkProviders

2016-09-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504303#comment-15504303 ] Michael Armbrust commented on SPARK-16407: -- You are taking an experimental inter

[jira] [Created] (SPARK-17597) HiveContext cannot create a table named sot

2016-09-19 Thread Saif Addin Ellafi (JIRA)
Saif Addin Ellafi created SPARK-17597: - Summary: HiveContext cannot create a table named sot Key: SPARK-17597 URL: https://issues.apache.org/jira/browse/SPARK-17597 Project: Spark Issue T

[jira] [Commented] (SPARK-17594) Bug in left-outer join

2016-09-19 Thread Virgil Palanciuc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504285#comment-15504285 ] Virgil Palanciuc commented on SPARK-17594: -- It's not - initially my example was

[jira] [Commented] (SPARK-17594) Bug in left-outer join

2016-09-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504266#comment-15504266 ] Sean Owen commented on SPARK-17594: --- It still looks like exactly the same issue. Why is

[jira] [Created] (SPARK-17596) Streaming job lacks Scala runtime methods

2016-09-19 Thread Evgeniy Tsvigun (JIRA)
Evgeniy Tsvigun created SPARK-17596: --- Summary: Streaming job lacks Scala runtime methods Key: SPARK-17596 URL: https://issues.apache.org/jira/browse/SPARK-17596 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-17594) Bug in left-outer join

2016-09-19 Thread Virgil Palanciuc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virgil Palanciuc updated SPARK-17594: - Description: I have a bug where I think a left-join returns wrong results, by mistakenly

[jira] [Issue Comment Deleted] (SPARK-12635) More efficient (column batch) serialization for Python/R

2016-09-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12635: Comment: was deleted (was: User 'nongli' has created a pull request for this issue: https://github.

[jira] [Commented] (SPARK-5377) Dynamically add jar into Spark Driver's classpath.

2016-09-19 Thread Jon Morra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504217#comment-15504217 ] Jon Morra commented on SPARK-5377: -- I would like to revisit this issue as well. Some of

[jira] [Reopened] (SPARK-17594) Bug in left-outer join

2016-09-19 Thread Virgil Palanciuc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virgil Palanciuc reopened SPARK-17594: -- Reopening with the proper example > Bug in left-outer join > -- > >

[jira] [Resolved] (SPARK-17438) Master UI should show the correct core limit when `ApplicationInfo.executorLimit` is set

2016-09-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17438. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Master UI should show the co

[jira] [Resolved] (SPARK-17473) jdbc docker tests are failing with java.lang.AbstractMethodError:

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17473. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Fixed by [~tsuresh]'s PR (th

[jira] [Updated] (SPARK-17473) jdbc docker tests are failing with java.lang.AbstractMethodError:

2016-09-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17473: --- Assignee: Suresh Thalamati > jdbc docker tests are failing with java.lang.AbstractMethodError: >

[jira] [Commented] (SPARK-17588) java.lang.AssertionError: assertion failed: lapack.dppsv returned 105. when running glm using gaussian link function.

2016-09-19 Thread sai pavan kumar chitti (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504083#comment-15504083 ] sai pavan kumar chitti commented on SPARK-17588: here is the output of sc

[jira] [Updated] (SPARK-17589) Fix test case `create external table`

2016-09-19 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17589: - Assignee: Xiao Li > Fix test case `create external table` > - > >

[jira] [Resolved] (SPARK-17589) Fix test case `create external table`

2016-09-19 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17589. -- Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 15145 [https://github.com/

[jira] [Assigned] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5992: --- Assignee: Apache Spark > Locality Sensitive Hashing (LSH) for MLlib > ---

[jira] [Assigned] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5992: --- Assignee: (was: Apache Spark) > Locality Sensitive Hashing (LSH) for MLlib >

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503971#comment-15503971 ] Apache Spark commented on SPARK-5992: - User 'Yunni' has created a pull request for thi

[jira] [Assigned] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17595: Assignee: (was: Apache Spark) > Inefficient selection in Word2VecModel.findSynonyms >

[jira] [Assigned] (SPARK-14082) Add support for GPU resource when running on Mesos

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14082: Assignee: Apache Spark > Add support for GPU resource when running on Mesos >

[jira] [Assigned] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17595: Assignee: Apache Spark > Inefficient selection in Word2VecModel.findSynonyms > ---

[jira] [Commented] (SPARK-14082) Add support for GPU resource when running on Mesos

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503928#comment-15503928 ] Apache Spark commented on SPARK-14082: -- User 'tnachen' has created a pull request fo

[jira] [Commented] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503929#comment-15503929 ] Apache Spark commented on SPARK-17595: -- User 'willb' has created a pull request for

[jira] [Assigned] (SPARK-14082) Add support for GPU resource when running on Mesos

2016-09-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14082: Assignee: (was: Apache Spark) > Add support for GPU resource when running on Mesos > -

[jira] [Resolved] (SPARK-17558) Bump Hadoop 2.7 version from 2.7.2 to 2.7.3

2016-09-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17558. --- Resolution: Fixed Assignee: Steve Loughran (was: Reynold Xin) Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-17558) Bump Hadoop 2.7 version from 2.7.2 to 2.7.3

2016-09-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17558: -- Priority: Trivial (was: Major) Issue Type: Improvement (was: New Feature) > Bump Hadoop 2.7 ver

[jira] [Resolved] (SPARK-17259) Hadoop 2.7 profile to depend on Hadoop 2.7.3

2016-09-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-17259. Resolution: Duplicate > Hadoop 2.7 profile to depend on Hadoop 2.7.3 >

[jira] [Commented] (SPARK-17594) Bug in left-outer join

2016-09-19 Thread Virgil Palanciuc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503666#comment-15503666 ] Virgil Palanciuc commented on SPARK-17594: -- Sorry. In my defence, I started to s

[jira] [Commented] (SPARK-17593) list files on s3 very slow

2016-09-19 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503675#comment-15503675 ] Gaurav Shah commented on SPARK-17593: - I definitely agree that flattening out will he

[jira] [Commented] (SPARK-17593) list files on s3 very slow

2016-09-19 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503668#comment-15503668 ] Gaurav Shah commented on SPARK-17593: - Thanks [~ste...@apache.org] S3 is definitely s

[jira] [Commented] (SPARK-17593) list files on s3 very slow

2016-09-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503663#comment-15503663 ] Steve Loughran commented on SPARK-17593: Looking at the dir tree, anything you co

[jira] [Created] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

2016-09-19 Thread William Benton (JIRA)
William Benton created SPARK-17595: -- Summary: Inefficient selection in Word2VecModel.findSynonyms Key: SPARK-17595 URL: https://issues.apache.org/jira/browse/SPARK-17595 Project: Spark Issue

[jira] [Updated] (SPARK-17593) list files on s3 very slow

2016-09-19 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Shah updated SPARK-17593: Description: lets say we have following partitioned data: {code} events_v3 -- event_date=2015-01-01

  1   2   >