[jira] [Assigned] (SPARK-18395) Evaluate common subexpression like lazy variable with a function approach

2016-11-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18395: Assignee: Apache Spark > Evaluate common subexpression like lazy variable with a function

[jira] [Assigned] (SPARK-18395) Evaluate common subexpression like lazy variable with a function approach

2016-11-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18395: Assignee: (was: Apache Spark) > Evaluate common subexpression like lazy variable with

[jira] [Commented] (SPARK-18395) Evaluate common subexpression like lazy variable with a function approach

2016-11-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653351#comment-15653351 ] Apache Spark commented on SPARK-18395: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-18395) Evaluate common subexpression like lazy variable with a function approach

2016-11-09 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-18395: --- Summary: Evaluate common subexpression like lazy variable with a function approach Key: SPARK-18395 URL: https://issues.apache.org/jira/browse/SPARK-18395

[jira] [Commented] (SPARK-17691) Add aggregate function to collect list with maximum number of elements

2016-11-09 Thread Assaf Mendelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653344#comment-15653344 ] Assaf Mendelson commented on SPARK-17691: - While you can use mutable buffers with the aggregator

[jira] [Closed] (SPARK-18048) If expression behaves differently if true and false expression are interchanged in case of different data types.

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-18048. --- Resolution: Invalid > If expression behaves differently if true and false expression are >

[jira] [Comment Edited] (SPARK-18048) If expression behaves differently if true and false expression are interchanged in case of different data types.

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653331#comment-15653331 ] Wenchen Fan edited comment on SPARK-18048 at 11/10/16 7:45 AM: --- according

[jira] [Commented] (SPARK-18048) If expression behaves differently if true and false expression are interchanged in case of different data types.

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653331#comment-15653331 ] Wenchen Fan commented on SPARK-18048: - according to the discussion in the PR, this ticket is invalid,

[jira] [Commented] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653322#comment-15653322 ] Wenchen Fan commented on SPARK-18050: - can you put the stacktrace here too? > spark 2.0.1 enable

[jira] [Commented] (SPARK-14450) Python OneVsRest should train multiple models at once

2016-11-09 Thread fanlu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653309#comment-15653309 ] fanlu commented on SPARK-14450: --- Why scala version does not need to use parallelization > Python OneVsRest

[jira] [Commented] (SPARK-18064) Spark SQL can't load default config file

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653305#comment-15653305 ] Wenchen Fan commented on SPARK-18064: - since this ticket has no description and the reporter has no

[jira] [Closed] (SPARK-18064) Spark SQL can't load default config file

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-18064. --- Resolution: Invalid > Spark SQL can't load default config file >

[jira] [Commented] (SPARK-18168) Revert the change of SPARK-18167

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653256#comment-15653256 ] Wenchen Fan commented on SPARK-18168: - it's already reverted right? > Revert the change of

[jira] [Commented] (SPARK-18220) ClassCastException occurs when using select query on ORC file

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653246#comment-15653246 ] Wenchen Fan commented on SPARK-18220: - Is it an external orc file or written by Spark SQL? >

[jira] [Commented] (SPARK-16628) OrcConversions should not convert an ORC table represented by MetastoreRelation to HadoopFsRelation if metastore schema does not match schema stored in ORC files

2016-11-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653228#comment-15653228 ] Dongjoon Hyun commented on SPARK-16628: --- Hi, is there any progress on this issue? > OrcConversions

[jira] [Closed] (SPARK-12998) Enable OrcRelation when connecting via spark thrift server

2016-11-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-12998. - Resolution: Duplicate Hi, [~rajesh.balamohan]. I'll close this issue since the PR is closed and

[jira] [Closed] (SPARK-18271) hash udf in HiveSessionCatalog.hiveFunctions seq is redundant

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-18271. --- Resolution: Won't Fix > hash udf in HiveSessionCatalog.hiveFunctions seq is redundant >

[jira] [Closed] (SPARK-18344) TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-18344. --- Resolution: Duplicate > TRUNCATE TABLE should fail if no partition is matched for the given >

[jira] [Updated] (SPARK-18075) UDF doesn't work on non-local spark

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18075: Component/s: SQL > UDF doesn't work on non-local spark > --- > >

[jira] [Updated] (SPARK-18075) UDF doesn't work on non-local spark

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18075: Labels: (was: sql) > UDF doesn't work on non-local spark > --- >

[jira] [Updated] (SPARK-18172) AnalysisException in first/last during aggregation

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18172: Component/s: SQL > AnalysisException in first/last during aggregation >

[jira] [Updated] (SPARK-18075) UDF doesn't work on non-local spark

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18075: Labels: sql (was: ) > UDF doesn't work on non-local spark > --- >

[jira] [Resolved] (SPARK-18147) Broken Spark SQL Codegen

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18147. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15807

[jira] [Updated] (SPARK-18147) Broken Spark SQL Codegen

2016-11-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18147: Assignee: Liang-Chi Hsieh > Broken Spark SQL Codegen > > >

[jira] [Comment Edited] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653009#comment-15653009 ] Xiao Li edited comment on SPARK-18389 at 11/10/16 4:36 AM: --- The above example

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653009#comment-15653009 ] Xiao Li commented on SPARK-18389: - This should be fixed now. Let me submit a PR for it. > Disallow

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652978#comment-15652978 ] Xiao Li commented on SPARK-18389: - If we do not allow cyclic view reference, we need to detect it in

[jira] [Commented] (SPARK-18367) limit() makes the lame walk again

2016-11-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652968#comment-15652968 ] Nicholas Chammas commented on SPARK-18367: -- Even if I cut the number of records I'm processing

[jira] [Comment Edited] (SPARK-18367) limit() makes the lame walk again

2016-11-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652406#comment-15652406 ] Nicholas Chammas edited comment on SPARK-18367 at 11/10/16 3:24 AM:

[jira] [Commented] (SPARK-18367) limit() makes the lame walk again

2016-11-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652881#comment-15652881 ] Nicholas Chammas commented on SPARK-18367: -- To provide some context, this code base I'm

[jira] [Assigned] (SPARK-18318) ML, Graph 2.1 QA: API: New Scala APIs, docs

2016-11-09 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang reassigned SPARK-18318: --- Assignee: Yanbo Liang > ML, Graph 2.1 QA: API: New Scala APIs, docs >

[jira] [Commented] (SPARK-17691) Add aggregate function to collect list with maximum number of elements

2016-11-09 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652723#comment-15652723 ] Michael Armbrust commented on SPARK-17691: -- I think that should be able to use mutable buffers

[jira] [Closed] (SPARK-18391) Openstack deployment scenarios

2016-11-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-18391. --- Resolution: Not A Problem > Openstack deployment scenarios > -- > >

[jira] [Created] (SPARK-18394) Executing the same query twice in a row results in CodeGenerator cache misses

2016-11-09 Thread Jonny Serencsa (JIRA)
Jonny Serencsa created SPARK-18394: -- Summary: Executing the same query twice in a row results in CodeGenerator cache misses Key: SPARK-18394 URL: https://issues.apache.org/jira/browse/SPARK-18394

[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

2016-11-09 Thread Jason Pan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652585#comment-15652585 ] Jason Pan commented on SPARK-18353: --- No matter what the default value is at last, I think we need a

[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

2016-11-09 Thread Jason Pan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652576#comment-15652576 ] Jason Pan commented on SPARK-18353: --- append the summit command: spark-submit --class

[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

2016-11-09 Thread Jason Pan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652543#comment-15652543 ] Jason Pan commented on SPARK-18353: --- Hi Sean. I was using "--conf" to set the parameter when summit.

[jira] [Issue Comment Deleted] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

2016-11-09 Thread Jason Pan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Pan updated SPARK-18353: -- Comment: was deleted (was: Hi Sean. --conf also didn't make it work. Thanks.) >

[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

2016-11-09 Thread Jason Pan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652537#comment-15652537 ] Jason Pan commented on SPARK-18353: --- Hi Sean. --conf also didn't make it work. Thanks. >

[jira] [Resolved] (SPARK-18343) FileSystem$Statistics$StatisticsDataReferenceCleaner hangs on s3 write

2016-11-09 Thread Luke Miner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Miner resolved SPARK-18343. Resolution: Not A Bug This was due to some clash in versions between the libraries I was using.

[jira] [Commented] (SPARK-18343) FileSystem$Statistics$StatisticsDataReferenceCleaner hangs on s3 write

2016-11-09 Thread Luke Miner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652487#comment-15652487 ] Luke Miner commented on SPARK-18343: Updating some of those libraries to their latest versions fixed

[jira] [Issue Comment Deleted] (SPARK-18343) FileSystem$Statistics$StatisticsDataReferenceCleaner hangs on s3 write

2016-11-09 Thread Luke Miner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Miner updated SPARK-18343: --- Comment: was deleted (was: Any suggestions on how one might hunt down that library? I've included my

[jira] [Created] (SPARK-18393) DataFrame pivot output column names should respect aliases

2016-11-09 Thread Eric Liang (JIRA)
Eric Liang created SPARK-18393: -- Summary: DataFrame pivot output column names should respect aliases Key: SPARK-18393 URL: https://issues.apache.org/jira/browse/SPARK-18393 Project: Spark Issue

[jira] [Commented] (SPARK-18392) LSH API, algorithm, and documentation follow-ups

2016-11-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652453#comment-15652453 ] Joseph K. Bradley commented on SPARK-18392: --- There are a few items still being discussed: *

[jira] [Created] (SPARK-18392) LSH API, algorithm, and documentation follow-ups

2016-11-09 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18392: - Summary: LSH API, algorithm, and documentation follow-ups Key: SPARK-18392 URL: https://issues.apache.org/jira/browse/SPARK-18392 Project: Spark

[jira] [Commented] (SPARK-17916) CSV data source treats empty string as null no matter what nullValue option is

2016-11-09 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652414#comment-15652414 ] Eric Liang commented on SPARK-17916: In our case, a user wants the empty string (whether actually

[jira] [Commented] (SPARK-18367) limit() makes the lame walk again

2016-11-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652406#comment-15652406 ] Nicholas Chammas commented on SPARK-18367: -- I've spent the day trying to narrow down what is

[jira] [Assigned] (SPARK-18391) Openstack deployment scenarios

2016-11-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18391: Assignee: Apache Spark > Openstack deployment scenarios > --

[jira] [Commented] (SPARK-18391) Openstack deployment scenarios

2016-11-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652387#comment-15652387 ] Apache Spark commented on SPARK-18391: -- User 'al-indigo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18391) Openstack deployment scenarios

2016-11-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18391: Assignee: (was: Apache Spark) > Openstack deployment scenarios >

[jira] [Commented] (SPARK-18391) Openstack deployment scenarios

2016-11-09 Thread Oleg Borisenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652386#comment-15652386 ] Oleg Borisenko commented on SPARK-18391: https://github.com/apache/spark/pull/15836 > Openstack

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652366#comment-15652366 ] Xiangrui Meng commented on SPARK-18390: --- This is a bug because the user didn't ask a cartesian

[jira] [Resolved] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-18390. --- Resolution: Duplicate > Optimized plan tried to use Cartesian join when it is not enabled >

[jira] [Created] (SPARK-18391) Openstack deployment scenarios

2016-11-09 Thread Oleg Borisenko (JIRA)
Oleg Borisenko created SPARK-18391: -- Summary: Openstack deployment scenarios Key: SPARK-18391 URL: https://issues.apache.org/jira/browse/SPARK-18391 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-17829) Stable format for offset log

2016-11-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17829. -- Resolution: Fixed > Stable format for offset log > > >

[jira] [Updated] (SPARK-17829) Stable format for offset log

2016-11-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17829: - Fix Version/s: 2.1.0 > Stable format for offset log > > >

[jira] [Commented] (SPARK-12333) Support shuffle spill encryption in Spark

2016-11-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652280#comment-15652280 ] Marcelo Vanzin commented on SPARK-12333: {{blockManager.getDiskWriter}} uses the block manager's

[jira] [Commented] (SPARK-12333) Support shuffle spill encryption in Spark

2016-11-09 Thread Krish Dey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652263#comment-15652263 ] Krish Dey commented on SPARK-12333: --- The constructor still seems to be the same as it is. Doesn't this

[jira] [Comment Edited] (SPARK-5682) Add encrypted shuffle in spark

2016-11-09 Thread Krish Dey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652217#comment-15652217 ] Krish Dey edited comment on SPARK-5682 at 11/9/16 10:29 PM: The constructor

[jira] [Commented] (SPARK-5682) Add encrypted shuffle in spark

2016-11-09 Thread Krish Dey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652217#comment-15652217 ] Krish Dey commented on SPARK-5682: -- The method still seems to be the same as it is. Doesn't this to be

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652202#comment-15652202 ] Cheng Lian commented on SPARK-18390: I think this issue has already been fixed by SPARK-17298 and

[jira] [Commented] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652104#comment-15652104 ] Herman van Hovell commented on SPARK-18388: --- Could you try this on master? We added an

[jira] [Updated] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-09 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18388: - Component/s: (was: Spark Core) SQL > Running aggregation on many

[jira] [Commented] (SPARK-18374) Incorrect words in StopWords/english.txt

2016-11-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652076#comment-15652076 ] Sean Owen commented on SPARK-18374: --- It's a fair point indeed, because it would be much better to omit

[jira] [Commented] (SPARK-17059) Allow FileFormat to specify partition pruning strategy

2016-11-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652068#comment-15652068 ] Apache Spark commented on SPARK-17059: -- User 'pwoody' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652039#comment-15652039 ] Nattavut Sutyanyong edited comment on SPARK-18389 at 11/9/16 9:10 PM:

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652039#comment-15652039 ] Nattavut Sutyanyong commented on SPARK-18389: - In CREATE VIEW, if we will check that the

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652022#comment-15652022 ] Herman van Hovell commented on SPARK-18390: --- I don't think this is a bug. It is doing exactly

[jira] [Commented] (SPARK-18363) Connected component for large graph result is wrong

2016-11-09 Thread Philip Adetiloye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652009#comment-15652009 ] Philip Adetiloye commented on SPARK-18363: -- I logged a similar Issue with the graphframe but the

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651980#comment-15651980 ] Srinath commented on SPARK-18390: - FYI, these are in branch 2.1 {noformat} commit

[jira] [Resolved] (SPARK-18370) InsertIntoHadoopFsRelationCommand should keep track of its table

2016-11-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18370. - Resolution: Fixed Fix Version/s: 2.1.0 > InsertIntoHadoopFsRelationCommand should keep

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651947#comment-15651947 ] Herman van Hovell commented on SPARK-18390: --- [~vssrinath] Can you take a look? > Optimized

[jira] [Updated] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18390: -- Assignee: Srinath > Optimized plan tried to use Cartesian join when it is not enabled

[jira] [Commented] (SPARK-18343) FileSystem$Statistics$StatisticsDataReferenceCleaner hangs on s3 write

2016-11-09 Thread Luke Miner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651942#comment-15651942 ] Luke Miner commented on SPARK-18343: Any suggestions on how one might hunt down that library? I've

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651927#comment-15651927 ] Yin Huai commented on SPARK-18390: -- Can you do a explain? > Optimized plan tried to use Cartesian join

[jira] [Updated] (SPARK-18387) Test that expressions can be serialized

2016-11-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18387: Target Version/s: 2.0.3, 2.1.0 (was: 2.1.0) > Test that expressions can be serialized >

[jira] [Updated] (SPARK-18387) Test that expressions can be serialized

2016-11-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18387: Target Version/s: 2.1.0 Priority: Blocker (was: Major) > Test that expressions can be

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651865#comment-15651865 ] Nattavut Sutyanyong commented on SPARK-18389: - Yes. CREATE/ALTER is not the place where we

[jira] [Comment Edited] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651873#comment-15651873 ] Reynold Xin edited comment on SPARK-18389 at 11/9/16 7:51 PM: -- It'd make

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651873#comment-15651873 ] Reynold Xin commented on SPARK-18389: - It'd make more sense to do this check during the command

[jira] [Updated] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-18390: --- Description: {code} val df2 = spark.range(1e9.toInt).withColumn("one", lit(1)) val df3 =

[jira] [Updated] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18390: -- Description: I hit this error when I tried to test skewed joins. {code} val df2 =

[jira] [Updated] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18390: -- Description: I hit this error when I tried to test skewed joins. {code} val df2 =

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651859#comment-15651859 ] Xiangrui Meng commented on SPARK-18390: --- cc: [~yhuai] [~lian cheng] > Optimized plan tried to use

[jira] [Created] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-18390: - Summary: Optimized plan tried to use Cartesian join when it is not enabled Key: SPARK-18390 URL: https://issues.apache.org/jira/browse/SPARK-18390 Project: Spark

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651832#comment-15651832 ] Xiao Li commented on SPARK-18389: - Are you saying we do not need to detect it at `CREATE VIEW` and `ALTER

[jira] [Closed] (SPARK-18336) SQL started to fail with OOM and etc. after move from 1.6.2 to 2.0.2

2016-11-09 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Egor Pahomov closed SPARK-18336. Resolution: Invalid Moving from 1.6 to 2.0 forced me use spark-submit instead of using spark

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651834#comment-15651834 ] Xiao Li commented on SPARK-18389: - Are you saying we do not need to detect it at `CREATE VIEW` and `ALTER

[jira] [Issue Comment Deleted] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18389: Comment: was deleted (was: Are you saying we do not need to detect it at `CREATE VIEW` and `ALTER VIEW`?)

[jira] [Assigned] (SPARK-18211) Spark SQL ignores split.size

2016-11-09 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-18211: Assignee: Michael Armbrust > Spark SQL ignores split.size >

[jira] [Resolved] (SPARK-18211) Spark SQL ignores split.size

2016-11-09 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-18211. -- Resolution: Not A Problem As of Spark 2.0 we do our own splitting/bin-packing of files

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

2016-11-09 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651816#comment-15651816 ] Nattavut Sutyanyong commented on SPARK-18389: - The ALTER VIEW should run successfully keeping

[jira] [Commented] (SPARK-18131) Support returning Vector/Dense Vector from backend

2016-11-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651811#comment-15651811 ] Felix Cheung commented on SPARK-18131: -- I think it's good to have a wrapper, but as you say we

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

2016-11-09 Thread Saikat Kanjilal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651806#comment-15651806 ] Saikat Kanjilal edited comment on SPARK-9487 at 11/9/16 7:22 PM: - Ok , for

[jira] [Updated] (SPARK-10816) EventTime based sessionization

2016-11-09 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10816: - Target Version/s: (was: 2.2.0) > EventTime based sessionization >

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

2016-11-09 Thread Saikat Kanjilal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651806#comment-15651806 ] Saikat Kanjilal commented on SPARK-9487: Ok , for some odd reason my local branch had the changes

[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

2016-11-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651807#comment-15651807 ] Sean Owen commented on SPARK-18353: --- BTW [~JasonPan] it looks like you're setting JVM props, and not

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

2016-11-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651798#comment-15651798 ] Reynold Xin commented on SPARK-18209: - Here is a ticket

[jira] [Updated] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-09 Thread Raviteja Lokineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raviteja Lokineni updated SPARK-18388: -- Description: Usecase: I am generating weekly aggregates of every column of data

[jira] [Commented] (SPARK-18332) SparkR 2.1 QA: Programming guide, migration guide, vignettes updates

2016-11-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651801#comment-15651801 ] Joseph K. Bradley commented on SPARK-18332: --- I like splitting apart programming guide updates

  1   2   3   >