[jira] [Commented] (SPARK-2218) rename Equals to EqualsTo in Spark SQL expressions

[ https://issues.apache.org/jira/browse/SPARK-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038500#comment-14038500 ] Reynold Xin commented on SPARK-2218: Michael has a PR here

[jira] [Updated] (SPARK-2218) rename Equals to EqualTo in Spark SQL expressions

[ https://issues.apache.org/jira/browse/SPARK-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2218: --- Summary: rename Equals to EqualTo in Spark SQL expressions (was: rename Equals to EqualsTo in Spark

[jira] [Commented] (SPARK-2216) Cost-based join reordering

[ https://issues.apache.org/jira/browse/SPARK-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038510#comment-14038510 ] Cheng Hao commented on SPARK-2216: -- Yes, this can be a big change, i think we need to add

[jira] [Updated] (SPARK-2215) Multi-way join

[ https://issues.apache.org/jira/browse/SPARK-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-2215: - Description: Support the multi-way join (multiple table joins) in a single reduce stage if they have the

[jira] [Updated] (SPARK-2215) Multi-way join

[ https://issues.apache.org/jira/browse/SPARK-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-2215: - Description: Support the multi-way join (multiple table joins) in a single reduce stage if they have the

[jira] [Updated] (SPARK-2201) Improve FlumeInputDStream's stability

[ https://issues.apache.org/jira/browse/SPARK-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sunshangchun updated SPARK-2201: Summary: Improve FlumeInputDStream's stability (was: Improve FlumeInputDStream) Improve

[jira] [Commented] (SPARK-768) Fail a task when the remote block it is fetching is not serializable

[ https://issues.apache.org/jira/browse/SPARK-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038523#comment-14038523 ] Reynold Xin commented on SPARK-768: --- I think it was the first case. It used to be the

[jira] [Commented] (SPARK-2201) Improve FlumeInputDStream's stability

[ https://issues.apache.org/jira/browse/SPARK-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038525#comment-14038525 ] chao.wu commented on SPARK-2201: good idea Improve FlumeInputDStream's stability

[jira] [Updated] (SPARK-1477) Add the lifecycle interface

[ https://issues.apache.org/jira/browse/SPARK-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1477: --- Target Version/s: 1.1.0 Affects Version/s: 1.0.1 Add the lifecycle interface

[jira] [Updated] (SPARK-1477) Add the lifecycle interface

[ https://issues.apache.org/jira/browse/SPARK-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1477: --- Assignee: Guoqiang Li Add the lifecycle interface ---

[jira] [Resolved] (SPARK-2177) describe table result contains only one column

[ https://issues.apache.org/jira/browse/SPARK-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2177. Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 describe table result

[jira] [Commented] (SPARK-768) Fail a task when the remote block it is fetching is not serializable

[ https://issues.apache.org/jira/browse/SPARK-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038534#comment-14038534 ] Raymond Liu commented on SPARK-768: --- Hi Reynold If this is the first case, then I think,

[jira] [Commented] (SPARK-768) Fail a task when the remote block it is fetching is not serializable

[ https://issues.apache.org/jira/browse/SPARK-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038536#comment-14038536 ] Reynold Xin commented on SPARK-768: --- Thanks for confirming. I'm going to close this issue

[jira] [Resolved] (SPARK-768) Fail a task when the remote block it is fetching is not serializable

[ https://issues.apache.org/jira/browse/SPARK-768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-768. --- Resolution: Cannot Reproduce Assignee: Raymond Liu (was: Reynold Xin) Fail a task when the

[jira] [Resolved] (SPARK-1293) Support for reading/writing complex types in Parquet

[ https://issues.apache.org/jira/browse/SPARK-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-1293. Resolution: Fixed Fix Version/s: 1.0.1 Support for reading/writing complex types in

[jira] [Resolved] (SPARK-2210) cast to boolean on boolean value gets turned into NOT((boolean_condition) = 0)

[ https://issues.apache.org/jira/browse/SPARK-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2210. Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 cast to boolean on boolean

[jira] [Resolved] (SPARK-2203) PySpark does not infer default numPartitions in same way as Spark

[ https://issues.apache.org/jira/browse/SPARK-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2203. Resolution: Fixed Fix Version/s: 1.1.0 PySpark does not infer default numPartitions in

[jira] [Resolved] (SPARK-2196) Fix nullability of CaseWhen.

[ https://issues.apache.org/jira/browse/SPARK-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2196. Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 Assignee: Takuya

[jira] [Resolved] (SPARK-2209) Cast shouldn't do null check twice

[ https://issues.apache.org/jira/browse/SPARK-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2209. Resolution: Fixed Cast shouldn't do null check twice --

[jira] [Commented] (SPARK-2114) groupByKey and joins on raw data

[ https://issues.apache.org/jira/browse/SPARK-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038558#comment-14038558 ] Patrick Wendell commented on SPARK-2114: Hey Sandy, One thing that would be

[jira] [Resolved] (SPARK-2218) rename Equals to EqualTo in Spark SQL expressions

[ https://issues.apache.org/jira/browse/SPARK-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2218. Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 rename Equals to EqualTo

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038569#comment-14038569 ] Patrick Wendell commented on SPARK-2089: Hey Sandy, I think I removed this from

[jira] [Created] (SPARK-2219) AddJar doesn't work

Michael Armbrust created SPARK-2219: --- Summary: AddJar doesn't work Key: SPARK-2219 URL: https://issues.apache.org/jira/browse/SPARK-2219 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2220) Fix remaining Hive Commands

Michael Armbrust created SPARK-2220: --- Summary: Fix remaining Hive Commands Key: SPARK-2220 URL: https://issues.apache.org/jira/browse/SPARK-2220 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-2163) Set ``setConvergenceTol'' with a parameter of type Double instead of Int

[ https://issues.apache.org/jira/browse/SPARK-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Bai resolved SPARK-2163. - Resolution: Implemented Set ``setConvergenceTol'' with a parameter of type Double instead of Int

[jira] [Commented] (SPARK-2195) Parquet extraMetadata can contain key information

[ https://issues.apache.org/jira/browse/SPARK-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038661#comment-14038661 ] Andre Schumacher commented on SPARK-2195: - Since commit

[jira] [Commented] (SPARK-2112) ParquetTypesConverter should not create its own conf

[ https://issues.apache.org/jira/browse/SPARK-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038666#comment-14038666 ] Andre Schumacher commented on SPARK-2112: - Since commit

[jira] [Created] (SPARK-2222) Add multiclass evaluation metrics

Alexander Ulanov created SPARK-: --- Summary: Add multiclass evaluation metrics Key: SPARK- URL: https://issues.apache.org/jira/browse/SPARK- Project: Spark Issue Type: New

[jira] [Commented] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038921#comment-14038921 ] Erik Erlandson commented on SPARK-1493: --- I submitted a proposal patch for RAT-161,

[jira] [Updated] (SPARK-2163) Set ``setConvergenceTol'' with a parameter of type Double instead of Int

[ https://issues.apache.org/jira/browse/SPARK-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2163: - Fix Version/s: 1.1.0 1.0.1 Set ``setConvergenceTol'' with a parameter of

[jira] [Commented] (SPARK-2223) Building and running tests with maven is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038944#comment-14038944 ] Sean Owen commented on SPARK-2223: -- Is it not just because there are a lot of tests, and

[jira] [Created] (SPARK-2224) allow running tests for one sub module

Thomas Graves created SPARK-2224: Summary: allow running tests for one sub module Key: SPARK-2224 URL: https://issues.apache.org/jira/browse/SPARK-2224 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038956#comment-14038956 ] Sean Owen commented on SPARK-2224: -- mvn test -pl [module], no? allow running tests for

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038964#comment-14038964 ] Guoqiang Li commented on SPARK-2224: {{mvn -Phadoop-2.3 -Pyarn

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038971#comment-14038971 ] Thomas Graves commented on SPARK-2224: -- Thanks, wasn't away of -pl.. doesn't seem to

[jira] [Commented] (SPARK-2223) Building and running tests with maven is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038976#comment-14038976 ] Thomas Graves commented on SPARK-2223: -- Some of our nightly builds are taking 50

[jira] [Commented] (SPARK-1882) Support dynamic memory sharing in Mesos

[ https://issues.apache.org/jira/browse/SPARK-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038987#comment-14038987 ] Andrew Ash commented on SPARK-1882: --- Yeah, for homogeneous environments, I think you can

[jira] [Updated] (SPARK-2202) saveAsTextFile hangs on final 2 tasks

[ https://issues.apache.org/jira/browse/SPARK-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suren Hiraman updated SPARK-2202: - Attachment: spark_trace.2.txt spark_trace.1.txt Here are two stack traces from

[jira] [Resolved] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2224. -- Resolution: Invalid allow running tests for one sub module

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039081#comment-14039081 ] Thomas Graves commented on SPARK-2224: -- well that worked as in it actually ran in the

[jira] [Assigned] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1528: Assignee: Thomas Graves Spark on Yarn: Add option for user to specify additional

[jira] [Updated] (SPARK-2204) Scheduler for Mesos in fine-grained mode launches tasks on wrong executors

[ https://issues.apache.org/jira/browse/SPARK-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastien Rainville updated SPARK-2204: --- Description: MesosSchedulerBackend.resourceOffers(SchedulerDriver, List[Offer]) is

[jira] [Commented] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039186#comment-14039186 ] Thomas Graves commented on SPARK-1528: -- https://github.com/apache/spark/pull/1159

[jira] [Commented] (SPARK-2223) Building and running tests with maven is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039217#comment-14039217 ] Mridul Muralidharan commented on SPARK-2223: [~tgraves] You could try running

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039236#comment-14039236 ] Mridul Muralidharan commented on SPARK-2089: [~pwendell] SplitInfo is not from

[jira] [Updated] (SPARK-2180) Basic HAVING clauses support for HiveQL

[ https://issues.apache.org/jira/browse/SPARK-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2180: --- Summary: Basic HAVING clauses support for HiveQL (was: HiveQL doesn't support GROUP BY with HAVING

[jira] [Created] (SPARK-2225) Turn HAVING without GROUP BY into WHERE

Reynold Xin created SPARK-2225: -- Summary: Turn HAVING without GROUP BY into WHERE Key: SPARK-2225 URL: https://issues.apache.org/jira/browse/SPARK-2225 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2225) Turn HAVING without GROUP BY into WHERE

[ https://issues.apache.org/jira/browse/SPARK-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039335#comment-14039335 ] William Benton commented on SPARK-2225: --- So the Hive test suite treats HAVING

[jira] [Created] (SPARK-2226) HAVING should be able to contain aggregate expressions that don't appear in the aggregation list.

Reynold Xin created SPARK-2226: -- Summary: HAVING should be able to contain aggregate expressions that don't appear in the aggregation list. Key: SPARK-2226 URL: https://issues.apache.org/jira/browse/SPARK-2226

[jira] [Commented] (SPARK-2180) Basic HAVING clauses support for HiveQL

[ https://issues.apache.org/jira/browse/SPARK-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039344#comment-14039344 ] Reynold Xin commented on SPARK-2180: [~willbenton] I added two related issues. Let me

[jira] [Updated] (SPARK-2226) HAVING should be able to contain aggregate expressions that don't appear in the aggregation list.

[ https://issues.apache.org/jira/browse/SPARK-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2226: --- Assignee: Reynold Xin HAVING should be able to contain aggregate expressions that don't appear in

[jira] [Commented] (SPARK-2198) Partition the scala build file so that it is easier to maintain

[ https://issues.apache.org/jira/browse/SPARK-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039355#comment-14039355 ] Anant Daksh Asthana commented on SPARK-2198: I am in agreement with Helena.

[jira] [Assigned] (SPARK-2226) HAVING should be able to contain aggregate expressions that don't appear in the aggregation list.

[ https://issues.apache.org/jira/browse/SPARK-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-2226: -- Assignee: Reynold Xin (was: William Benton) HAVING should be able to contain aggregate

[jira] [Updated] (SPARK-2226) HAVING should be able to contain aggregate expressions that don't appear in the aggregation list.

[ https://issues.apache.org/jira/browse/SPARK-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2226: --- Assignee: William Benton (was: Reynold Xin) HAVING should be able to contain aggregate expressions

[jira] [Updated] (SPARK-2225) Turn HAVING without GROUP BY into WHERE

[ https://issues.apache.org/jira/browse/SPARK-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2225: --- Assignee: William Benton Turn HAVING without GROUP BY into WHERE

[jira] [Commented] (SPARK-2225) Turn HAVING without GROUP BY into WHERE

[ https://issues.apache.org/jira/browse/SPARK-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039365#comment-14039365 ] Reynold Xin commented on SPARK-2225: Yea I don't think we need to follow the negative

[jira] [Commented] (SPARK-704) ConnectionManager sometimes cannot detect loss of sending connections

[ https://issues.apache.org/jira/browse/SPARK-704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039396#comment-14039396 ] Charles Reiss commented on SPARK-704: - It's been a while since I reported this issue,

[jira] [Commented] (SPARK-1856) Standardize MLlib interfaces

[ https://issues.apache.org/jira/browse/SPARK-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039495#comment-14039495 ] Erik Erlandson commented on SPARK-1856: --- How does this jira relate (if at all) to

[jira] [Commented] (SPARK-1652) Fixes and improvements for spark-submit/configs

[ https://issues.apache.org/jira/browse/SPARK-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039565#comment-14039565 ] Nicholas Chammas commented on SPARK-1652: - [~pwendell], is there currently a

[jira] [Resolved] (SPARK-1970) Update unit test in XORShiftRandomSuite to use ChiSquareTest from commons-math3

[ https://issues.apache.org/jira/browse/SPARK-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1970. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1073

[jira] [Updated] (SPARK-1777) Pass cached blocks directly to disk if memory is not large enough

[ https://issues.apache.org/jira/browse/SPARK-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1777: - Attachment: spark-1777-design-doc.pdf Pass cached blocks directly to disk if memory is not large enough

[jira] [Resolved] (SPARK-2061) Deprecate `splits` in JavaRDDLike and add `partitions`

[ https://issues.apache.org/jira/browse/SPARK-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2061. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1062

[jira] [Created] (SPARK-2227) Support dfs command

Reynold Xin created SPARK-2227: -- Summary: Support dfs command Key: SPARK-2227 URL: https://issues.apache.org/jira/browse/SPARK-2227 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-2179) Public API for DataTypes and Schema

[ https://issues.apache.org/jira/browse/SPARK-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039682#comment-14039682 ] Yin Huai commented on SPARK-2179: - Just a note. For every data type we have, I think it is