[jira] [Commented] (SPARK-15205) Codegen can compile the same source code more than twice

2016-05-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288484#comment-15288484 ] Reynold Xin commented on SPARK-15205: - This is not a bug per se. I've changed it to improvement and

[jira] [Updated] (SPARK-15205) Codegen can compile the same source code more than twice

2016-05-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15205: Target Version/s: 2.1.0 Issue Type: Improvement (was: Bug) > Codegen can compile the

[jira] [Updated] (SPARK-15377) Enabling SASL Spark 1.6.1

2016-05-17 Thread Fabian Tan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Tan updated SPARK-15377: --- Description: Hi there, I wonder if anyone gotten SASL to work with Spark 1.6.1 on YARN? At this

[jira] [Created] (SPARK-15377) Enabling SASL Spark 1.6.1

2016-05-17 Thread Fabian Tan (JIRA)
Fabian Tan created SPARK-15377: -- Summary: Enabling SASL Spark 1.6.1 Key: SPARK-15377 URL: https://issues.apache.org/jira/browse/SPARK-15377 Project: Spark Issue Type: Question

[jira] [Updated] (SPARK-15372) TPC-DS Qury 84 returns wrong results against TPC official

2016-05-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15372: Affects Version/s: (was: 2.0.0) Target Version/s: 2.0.0 Fix Version/s: (was:

[jira] [Commented] (SPARK-15372) TPC-DS Qury 84 returns wrong results against TPC official

2016-05-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288457#comment-15288457 ] Reynold Xin commented on SPARK-15372: - [~freiss] you are saying this is not a problem right? >

[jira] [Updated] (SPARK-15376) DataFrame write.jdbc() inserts more rows than acutal

2016-05-17 Thread xiaoyu chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaoyu chen updated SPARK-15376: Description: It's a odd bug, occur under this situation: {code:title=Bar.scala} val

[jira] [Created] (SPARK-15376) DataFrame write.jdbc() inserts more rows than acutal

2016-05-17 Thread xiaoyu chen (JIRA)
xiaoyu chen created SPARK-15376: --- Summary: DataFrame write.jdbc() inserts more rows than acutal Key: SPARK-15376 URL: https://issues.apache.org/jira/browse/SPARK-15376 Project: Spark Issue

[jira] [Updated] (SPARK-15346) Reduce duplicate computation in picking initial points in LocalKMeans

2016-05-17 Thread Abraham Zhan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abraham Zhan updated SPARK-15346: - Description: h2.Main Issue I found that for KMans|| in mllib, when dataset is in large scale,

[jira] [Updated] (SPARK-15375) Add ConsoleSink for structure streaming to display the dataframe on the fly

2016-05-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-15375: Summary: Add ConsoleSink for structure streaming to display the dataframe on the fly (was: Add

[jira] [Assigned] (SPARK-15375) Add ConsoleSink for structure sink to display the dataframe on the fly

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15375: Assignee: (was: Apache Spark) > Add ConsoleSink for structure sink to display the

[jira] [Assigned] (SPARK-15375) Add ConsoleSink for structure sink to display the dataframe on the fly

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15375: Assignee: Apache Spark > Add ConsoleSink for structure sink to display the dataframe on

[jira] [Commented] (SPARK-15375) Add ConsoleSink for structure sink to display the dataframe on the fly

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288138#comment-15288138 ] Apache Spark commented on SPARK-15375: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2016-05-17 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-4105: -- Comment: was deleted (was: Can you suggest how I reproduce this error?) >

[jira] [Created] (SPARK-15375) Add ConsoleSink for structure sink to display the dataframe on the fly

2016-05-17 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-15375: --- Summary: Add ConsoleSink for structure sink to display the dataframe on the fly Key: SPARK-15375 URL: https://issues.apache.org/jira/browse/SPARK-15375 Project: Spark

[jira] [Assigned] (SPARK-14851) Support radix sort with nullable longs

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14851: Assignee: (was: Apache Spark) > Support radix sort with nullable longs >

[jira] [Assigned] (SPARK-14851) Support radix sort with nullable longs

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14851: Assignee: Apache Spark > Support radix sort with nullable longs >

[jira] [Commented] (SPARK-14851) Support radix sort with nullable longs

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288103#comment-15288103 ] Apache Spark commented on SPARK-14851: -- User 'ericl' has created a pull request for this issue:

[jira] [Updated] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-15345: --- Summary: SparkSession's conf doesn't take effect when there's already an existing SparkContext

[jira] [Assigned] (SPARK-15345) SparkSession's conf doesn't take effect when this already an existing SparkContext

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15345: Assignee: Apache Spark > SparkSession's conf doesn't take effect when this already an

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when this already an existing SparkContext

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288101#comment-15288101 ] Apache Spark commented on SPARK-15345: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15345) SparkSession's conf doesn't take effect when this already an existing SparkContext

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15345: Assignee: (was: Apache Spark) > SparkSession's conf doesn't take effect when this

[jira] [Updated] (SPARK-15345) SparkSession's conf doesn't take effect when this already an existing SparkContext

2016-05-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-15345: --- Summary: SparkSession's conf doesn't take effect when this already an existing SparkContext (was:

[jira] [Updated] (SPARK-15345) Cannot connect to Hive databases

2016-05-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-15345: --- Component/s: SQL > Cannot connect to Hive databases > > >

[jira] [Updated] (SPARK-15345) Cannot connect to Hive databases

2016-05-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-15345: --- Priority: Blocker (was: Critical) > Cannot connect to Hive databases >

[jira] [Commented] (SPARK-15345) Cannot connect to Hive databases

2016-05-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288089#comment-15288089 ] Jeff Zhang commented on SPARK-15345: The root cause is that in pyspark SparkContext is created first

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2016-05-17 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288084#comment-15288084 ] Zhongshuai Pei commented on SPARK-4105: --- Can you suggest how I reproduce this error? >

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2016-05-17 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288085#comment-15288085 ] Zhongshuai Pei commented on SPARK-4105: --- Can you suggest how I reproduce this error? >

[jira] [Commented] (SPARK-15374) Spark created Parquet files cause NPE when a column has only NULL values

2016-05-17 Thread Euan de Kock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288078#comment-15288078 ] Euan de Kock commented on SPARK-15374: -- Sample script to replicate this error (Last line will fail

[jira] [Updated] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15367: Assignee: Xiao Li > Add refreshTable back > -- > > Key:

[jira] [Updated] (SPARK-15374) Spark created Parquet files cause NPE when a column has only NULL values

2016-05-17 Thread Euan de Kock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Euan de Kock updated SPARK-15374: - Description: When an external table is built from Spark, and is subsequently accessed by Hive

[jira] [Created] (SPARK-15374) Spark created Parquet files cause NPE when a column has only NULL values

2016-05-17 Thread Euan de Kock (JIRA)
Euan de Kock created SPARK-15374: Summary: Spark created Parquet files cause NPE when a column has only NULL values Key: SPARK-15374 URL: https://issues.apache.org/jira/browse/SPARK-15374 Project:

[jira] [Updated] (SPARK-15370) Some correlated subqueries return incorrect answers

2016-05-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15370: Target Version/s: 2.0.0 > Some correlated subqueries return incorrect answers >

[jira] [Commented] (SPARK-15368) Spark History Server does not pick up extraClasspath

2016-05-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288049#comment-15288049 ] Saisai Shao commented on SPARK-15368: - Maybe you could try {{SPARK_CLASSPATH}}, though deprecated

[jira] [Commented] (SPARK-15371) YARNShuffleService doesn't get current local-dirs from NodeManager

2016-05-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288013#comment-15288013 ] Saisai Shao commented on SPARK-15371: - I think what you mentioned above is relate to this JIRA

[jira] [Updated] (SPARK-15340) Limit the size of the map used to cache JobConfs to void OOM

2016-05-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15340: Target Version/s: 2.0.0 > Limit the size of the map used to cache JobConfs to void OOM >

[jira] [Commented] (SPARK-15372) TPC-DS Qury 84 returns wrong results against TPC official

2016-05-17 Thread Frederick Reiss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288004#comment-15288004 ] Frederick Reiss commented on SPARK-15372: - There is no CONCAT function in the SQL standard. The

[jira] [Updated] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15373: -- Description: Currently, SparkUI shows two timezones in a single page when the timezone of

[jira] [Commented] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287957#comment-15287957 ] Apache Spark commented on SPARK-15373: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15373: Assignee: Apache Spark > SparkUI should show consistent timezones. >

[jira] [Assigned] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15373: Assignee: (was: Apache Spark) > SparkUI should show consistent timezones. >

[jira] [Updated] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15373: -- Description: Currently, SparkUI shows two timezones in a single page when the timezone of

[jira] [Updated] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15373: -- Description: Currently, SparkUI shows two timezones in a single page when the timezone of

[jira] [Updated] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15373: -- Description: Currently, SparkUI shows two timezones in a single page when the timezone of

[jira] [Updated] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15373: -- Description: Currently, SparkUI shows two timezones in a single page when the timezone of

[jira] [Updated] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15373: -- Attachment: timezone.png > SparkUI should show consistent timezones. >

[jira] [Created] (SPARK-15373) SparkUI should show consistent timezones.

2016-05-17 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-15373: - Summary: SparkUI should show consistent timezones. Key: SPARK-15373 URL: https://issues.apache.org/jira/browse/SPARK-15373 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15345) Cannot connect to Hive databases

2016-05-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287907#comment-15287907 ] Jeff Zhang commented on SPARK-15345: Try to work on it. > Cannot connect to Hive databases >

[jira] [Commented] (SPARK-14346) SHOW CREATE TABLE command (Native)

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287863#comment-15287863 ] Apache Spark commented on SPARK-14346: -- User 'yhuai' has created a pull request for this issue:

[jira] [Updated] (SPARK-15372) TPC-DS Qury 84 returns wrong results against TPC official

2016-05-17 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-15372: --- Description: The official TPC-DS query 84 returns wrong results when compared to its official

[jira] [Updated] (SPARK-15372) TPC-DS Qury 84 returns wrong results against TPC official

2016-05-17 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-15372: --- Description: The official TPC-DS query 84 returns wrong results when compared to its official

[jira] [Updated] (SPARK-15372) TPC-DS Qury 84 returns wrong results against TPC official

2016-05-17 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-15372: --- Labels: SPARK-15071 (was: ) > TPC-DS Qury 84 returns wrong results against TPC official >

[jira] [Created] (SPARK-15372) TPC-DS Qury 84 returns wrong results against TPC official

2016-05-17 Thread JESSE CHEN (JIRA)
JESSE CHEN created SPARK-15372: -- Summary: TPC-DS Qury 84 returns wrong results against TPC official Key: SPARK-15372 URL: https://issues.apache.org/jira/browse/SPARK-15372 Project: Spark Issue

[jira] [Assigned] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15367: Assignee: Apache Spark > Add refreshTable back > -- > >

[jira] [Commented] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287725#comment-15287725 ] Apache Spark commented on SPARK-15367: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15367: Assignee: (was: Apache Spark) > Add refreshTable back > -- > >

[jira] [Created] (SPARK-15371) YARNShuffleService doesn't get current local-dirs from NodeManager

2016-05-17 Thread Jeff Field (JIRA)
Jeff Field created SPARK-15371: -- Summary: YARNShuffleService doesn't get current local-dirs from NodeManager Key: SPARK-15371 URL: https://issues.apache.org/jira/browse/SPARK-15371 Project: Spark

[jira] [Updated] (SPARK-11735) Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped

2016-05-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-11735: - Assignee: Shixiong Zhu > Add a check in the constructor of SqlContext to make sure the SparkContext is

[jira] [Resolved] (SPARK-11735) Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped

2016-05-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-11735. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13154

[jira] [Comment Edited] (SPARK-15368) Spark History Server does not pick up extraClasspath

2016-05-17 Thread Dawson Choong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287658#comment-15287658 ] Dawson Choong edited comment on SPARK-15368 at 5/17/16 9:45 PM: I see,

[jira] [Comment Edited] (SPARK-15368) Spark History Server does not pick up extraClasspath

2016-05-17 Thread Dawson Choong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287658#comment-15287658 ] Dawson Choong edited comment on SPARK-15368 at 5/17/16 9:45 PM: I see,

[jira] [Commented] (SPARK-15368) Spark History Server does not pick up extraClasspath

2016-05-17 Thread Dawson Choong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287658#comment-15287658 ] Dawson Choong commented on SPARK-15368: --- I see, sorry for the confusion. May I ask how I would

[jira] [Commented] (SPARK-11735) Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287625#comment-15287625 ] Apache Spark commented on SPARK-11735: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-15361) ML 2.0 QA: Scala APIs audit for clustering

2016-05-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15361: -- Shepherd: Joseph K. Bradley Assignee: Yanbo Liang > ML 2.0 QA: Scala APIs audit

[jira] [Commented] (SPARK-15370) Some correlated subqueries return incorrect answers

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287567#comment-15287567 ] Apache Spark commented on SPARK-15370: -- User 'frreiss' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15370) Some correlated subqueries return incorrect answers

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15370: Assignee: Apache Spark > Some correlated subqueries return incorrect answers >

[jira] [Assigned] (SPARK-15370) Some correlated subqueries return incorrect answers

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15370: Assignee: (was: Apache Spark) > Some correlated subqueries return incorrect answers >

[jira] [Created] (SPARK-15370) Some correlated subqueries return incorrect answers

2016-05-17 Thread Frederick Reiss (JIRA)
Frederick Reiss created SPARK-15370: --- Summary: Some correlated subqueries return incorrect answers Key: SPARK-15370 URL: https://issues.apache.org/jira/browse/SPARK-15370 Project: Spark

[jira] [Updated] (SPARK-15365) Metastore relation should fallback to HDFS size if statistics are not available from table meta data.

2016-05-17 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated SPARK-15365: - Description: Currently if a table is used in join operation we rely on Metastore

[jira] [Commented] (SPARK-10520) Dates cannot be summarised

2016-05-17 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287538#comment-15287538 ] Barry Becker commented on SPARK-10520: -- We would also like to have avg date aggregate work out of

[jira] [Commented] (SPARK-15364) Implement Python picklers for ml.Vector and ml.Matrix under spark.ml.python

2016-05-17 Thread praveen dareddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287533#comment-15287533 ] praveen dareddy commented on SPARK-15364: - Hi, [~mengxr] I would like to work on this issue.

[jira] [Updated] (SPARK-15362) Make spark.ml KMeansModel load backwards compatible

2016-05-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15362: -- Shepherd: Joseph K. Bradley Assignee: Yanbo Liang > Make spark.ml KMeansModel load

[jira] [Commented] (SPARK-15368) Spark History Server does not pick up extraClasspath

2016-05-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287462#comment-15287462 ] Sean Owen commented on SPARK-15368: --- Hm, but that's not how you'd specify an arg to the history server.

[jira] [Resolved] (SPARK-15244) Type of column name created with sqlContext.createDataFrame() is not consistent.

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15244. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13097

[jira] [Updated] (SPARK-15244) Type of column name created with sqlContext.createDataFrame() is not consistent.

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15244: --- Assignee: Dongjoon Hyun > Type of column name created with sqlContext.createDataFrame() is not >

[jira] [Resolved] (SPARK-14615) Use the new ML Vector and Matrix in the ML pipeline based algorithms

2016-05-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14615. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12627

[jira] [Created] (SPARK-15369) Investigate selectively using Jython for parts of PySpark

2016-05-17 Thread holdenk (JIRA)
holdenk created SPARK-15369: --- Summary: Investigate selectively using Jython for parts of PySpark Key: SPARK-15369 URL: https://issues.apache.org/jira/browse/SPARK-15369 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14615) Use the new ML Vector and Matrix in the ML pipeline based algorithms

2016-05-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14615: -- Priority: Blocker (was: Major) > Use the new ML Vector and Matrix in the ML pipeline based

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2016-05-17 Thread Jason Reid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287352#comment-15287352 ] Jason Reid commented on SPARK-4105: --- FWIW - I am able to reproduce this error consistently for a job,

[jira] [Updated] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-15367: - Priority: Critical (was: Major) > Add refreshTable back > -- > >

[jira] [Created] (SPARK-15368) Spark History Server does not pick up extraClasspath

2016-05-17 Thread Dawson Choong (JIRA)
Dawson Choong created SPARK-15368: - Summary: Spark History Server does not pick up extraClasspath Key: SPARK-15368 URL: https://issues.apache.org/jira/browse/SPARK-15368 Project: Spark Issue

[jira] [Commented] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287312#comment-15287312 ] Xiao Li commented on SPARK-15367: - : ) No problem! > Add refreshTable back > -- > >

[jira] [Commented] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287295#comment-15287295 ] Yin Huai commented on SPARK-15367: -- Yea. Sure. Thanks! > Add refreshTable back >

[jira] [Commented] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287292#comment-15287292 ] Xiao Li commented on SPARK-15367: - Not sure if anybody starts it? If not, I can work on it. Thanks! >

[jira] [Updated] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-15367: - Description: refreshTable was a method in HiveContext. It was deleted accidentally while we were

[jira] [Created] (SPARK-15367) Add refreshTable back

2016-05-17 Thread Yin Huai (JIRA)
Yin Huai created SPARK-15367: Summary: Add refreshTable back Key: SPARK-15367 URL: https://issues.apache.org/jira/browse/SPARK-15367 Project: Spark Issue Type: Bug Components: SQL

[jira] [Created] (SPARK-15366) Add Application Detail UI uri to Spark Json API

2016-05-17 Thread Edgardo Vega (JIRA)
Edgardo Vega created SPARK-15366: Summary: Add Application Detail UI uri to Spark Json API Key: SPARK-15366 URL: https://issues.apache.org/jira/browse/SPARK-15366 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15044) spark-sql will throw "input path does not exist" exception if it handles a partition which exists in hive table, but the path is removed manually

2016-05-17 Thread saurabh paliwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287256#comment-15287256 ] saurabh paliwal commented on SPARK-15044: - Anyway, as a work-around, I have just caught the

[jira] [Resolved] (SPARK-15182) Copy MLlib doc to ML: ml.feature

2016-05-17 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-15182. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12957

[jira] [Updated] (SPARK-15182) Copy MLlib doc to ML: ml.feature

2016-05-17 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-15182: --- Assignee: yuhao yang > Copy MLlib doc to ML: ml.feature > >

[jira] [Assigned] (SPARK-15353) Making peer selection for block replication pluggable

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15353: Assignee: (was: Apache Spark) > Making peer selection for block replication pluggable

[jira] [Commented] (SPARK-15317) JobProgressListener takes a huge amount of memory with iterative DataFrame program in local, standalone

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287209#comment-15287209 ] Apache Spark commented on SPARK-15317: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15317) JobProgressListener takes a huge amount of memory with iterative DataFrame program in local, standalone

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15317: Assignee: (was: Apache Spark) > JobProgressListener takes a huge amount of memory

[jira] [Assigned] (SPARK-15317) JobProgressListener takes a huge amount of memory with iterative DataFrame program in local, standalone

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15317: Assignee: Apache Spark > JobProgressListener takes a huge amount of memory with iterative

[jira] [Commented] (SPARK-15353) Making peer selection for block replication pluggable

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287189#comment-15287189 ] Apache Spark commented on SPARK-15353: -- User 'shubhamchopra' has created a pull request for this

[jira] [Assigned] (SPARK-15353) Making peer selection for block replication pluggable

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15353: Assignee: Apache Spark > Making peer selection for block replication pluggable >

[jira] [Assigned] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15357: -- Assignee: Davies Liu > Cooperative spilling should check consumer memory mode >

[jira] [Assigned] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15357: Assignee: Apache Spark > Cooperative spilling should check consumer memory mode >

[jira] [Assigned] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15357: Assignee: (was: Apache Spark) > Cooperative spilling should check consumer memory

[jira] [Resolved] (SPARK-10216) Avoid creating empty files during overwrite into Hive table with group by query

2016-05-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10216. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12855

  1   2   >