spark git commit: [SPARK-14471][SQL] Aliases in SELECT could be used in GROUP BY

2017-04-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 26a9e2948 -> af3a1411a [SPARK-14471][SQL] Aliases in SELECT could be used in GROUP BY ## What changes were proposed in this pull request? This pr added a new rule in `Analyzer` to resolve aliases in `GROUP BY`. The current master throws

spark git commit: [SPARK-14471][SQL] Aliases in SELECT could be used in GROUP BY

2017-04-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master e3c816043 -> 59e3a5644 [SPARK-14471][SQL] Aliases in SELECT could be used in GROUP BY ## What changes were proposed in this pull request? This pr added a new rule in `Analyzer` to resolve aliases in `GROUP BY`. The current master throws an

spark git commit: [SPARK-20476][SQL] Block users to create a table that use commas in the column names

2017-04-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 f60ed0c2c -> 26a9e2948 [SPARK-20476][SQL] Block users to create a table that use commas in the column names ### What changes were proposed in this pull request? ```SQL hive> create table t1(`a,` string); OK Time taken: 1.399 seconds h

spark git commit: [SPARK-20476][SQL] Block users to create a table that use commas in the column names

2017-04-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 7fe824979 -> e3c816043 [SPARK-20476][SQL] Block users to create a table that use commas in the column names ### What changes were proposed in this pull request? ```SQL hive> create table t1(`a,` string); OK Time taken: 1.399 seconds hive>

spark git commit: [SPARKR][DOC] Document LinearSVC in R programming guide

2017-04-27 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 e02b6ebfd -> f60ed0c2c [SPARKR][DOC] Document LinearSVC in R programming guide ## What changes were proposed in this pull request? add link to svmLinear in the SparkR programming document. ## How was this patch tested? Build doc manu

spark git commit: [SPARKR][DOC] Document LinearSVC in R programming guide

2017-04-27 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master b90bf520f -> 7fe824979 [SPARKR][DOC] Document LinearSVC in R programming guide ## What changes were proposed in this pull request? add link to svmLinear in the SparkR programming document. ## How was this patch tested? Build doc manually

spark git commit: [SPARK-12837][CORE] Do not send the name of internal accumulator to executor side

2017-04-27 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.2 3d53d825e -> e02b6ebfd [SPARK-12837][CORE] Do not send the name of internal accumulator to executor side ## What changes were proposed in this pull request? When sending accumulator updates back to driver, the network overhead is pret

spark git commit: [SPARK-12837][CORE] Do not send the name of internal accumulator to executor side

2017-04-27 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 823baca2c -> b90bf520f [SPARK-12837][CORE] Do not send the name of internal accumulator to executor side ## What changes were proposed in this pull request? When sending accumulator updates back to driver, the network overhead is pretty

spark git commit: [SPARK-20452][SS][KAFKA] Fix a potential ConcurrentModificationException for batch Kafka DataFrame

2017-04-27 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.2 753e129f3 -> 3d53d825e [SPARK-20452][SS][KAFKA] Fix a potential ConcurrentModificationException for batch Kafka DataFrame ## What changes were proposed in this pull request? Cancel a batch Kafka query but one of task cannot be cancell

spark git commit: [SPARK-20452][SS][KAFKA] Fix a potential ConcurrentModificationException for batch Kafka DataFrame

2017-04-27 Thread tdas
Repository: spark Updated Branches: refs/heads/master 01c999e7f -> 823baca2c [SPARK-20452][SS][KAFKA] Fix a potential ConcurrentModificationException for batch Kafka DataFrame ## What changes were proposed in this pull request? Cancel a batch Kafka query but one of task cannot be cancelled,

spark git commit: [SPARK-20461][CORE][SS] Use UninterruptibleThread for Executor and fix the potential hang in CachedKafkaConsumer

2017-04-27 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.2 4512e2ae6 -> 753e129f3 [SPARK-20461][CORE][SS] Use UninterruptibleThread for Executor and fix the potential hang in CachedKafkaConsumer ## What changes were proposed in this pull request? This PR changes Executor's threads to `Uninter

spark git commit: [SPARK-20461][CORE][SS] Use UninterruptibleThread for Executor and fix the potential hang in CachedKafkaConsumer

2017-04-27 Thread tdas
Repository: spark Updated Branches: refs/heads/master 606432a13 -> 01c999e7f [SPARK-20461][CORE][SS] Use UninterruptibleThread for Executor and fix the potential hang in CachedKafkaConsumer ## What changes were proposed in this pull request? This PR changes Executor's threads to `Uninterrupt

spark git commit: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-27 Thread dbtsai
Repository: spark Updated Branches: refs/heads/branch-2.2 c29c6dead -> 4512e2ae6 [SPARK-20047][ML] Constrained Logistic Regression ## What changes were proposed in this pull request? MLlib ```LogisticRegression``` should support bound constrained optimization (only for L2 regularization). Use

spark git commit: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-27 Thread dbtsai
Repository: spark Updated Branches: refs/heads/master 039e32ca1 -> 606432a13 [SPARK-20047][ML] Constrained Logistic Regression ## What changes were proposed in this pull request? MLlib ```LogisticRegression``` should support bound constrained optimization (only for L2 regularization). Users c

spark git commit: [SPARK-20483][MINOR] Test for Mesos Coarse mode may starve other Mesos frameworks

2017-04-27 Thread dbtsai
Repository: spark Updated Branches: refs/heads/branch-2.2 c69d862b2 -> c29c6dead [SPARK-20483][MINOR] Test for Mesos Coarse mode may starve other Mesos frameworks ## What changes were proposed in this pull request? Add test case for scenarios where executor.cores is set as a (non)divisor of

spark git commit: [SPARK-20483][MINOR] Test for Mesos Coarse mode may starve other Mesos frameworks

2017-04-27 Thread dbtsai
Repository: spark Updated Branches: refs/heads/master a4aa4665a -> 039e32ca1 [SPARK-20483][MINOR] Test for Mesos Coarse mode may starve other Mesos frameworks ## What changes were proposed in this pull request? Add test case for scenarios where executor.cores is set as a (non)divisor of spar

spark git commit: [SPARK-20426] Lazy initialization of FileSegmentManagedBuffer for shuffle service.

2017-04-27 Thread tgraves
Repository: spark Updated Branches: refs/heads/branch-2.2 92b61f02d -> c69d862b2 [SPARK-20426] Lazy initialization of FileSegmentManagedBuffer for shuffle service. ## What changes were proposed in this pull request? When application contains large amount of shuffle blocks. NodeManager require

spark git commit: [SPARK-20487][SQL] `HiveTableScan` node is quite verbose in explained plan

2017-04-27 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 090b3370e -> 92b61f02d [SPARK-20487][SQL] `HiveTableScan` node is quite verbose in explained plan ## What changes were proposed in this pull request? Changed `TreeNode.argString` to handle `CatalogTable` separately (otherwise it would

spark git commit: [SPARK-20487][SQL] `HiveTableScan` node is quite verbose in explained plan

2017-04-27 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 26ac2ce05 -> a4aa4665a [SPARK-20487][SQL] `HiveTableScan` node is quite verbose in explained plan ## What changes were proposed in this pull request? Changed `TreeNode.argString` to handle `CatalogTable` separately (otherwise it would cal

spark git commit: [SPARK-20482][SQL] Resolving Casts is too strict on having time zone set

2017-04-27 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 87d27e544 -> 090b3370e [SPARK-20482][SQL] Resolving Casts is too strict on having time zone set ## What changes were proposed in this pull request? Relax the requirement that a `TimeZoneAwareExpression` has to have its `timeZoneId` se

spark git commit: [SPARK-20482][SQL] Resolving Casts is too strict on having time zone set

2017-04-27 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 85c6ce619 -> 26ac2ce05 [SPARK-20482][SQL] Resolving Casts is too strict on having time zone set ## What changes were proposed in this pull request? Relax the requirement that a `TimeZoneAwareExpression` has to have its `timeZoneId` set to

spark git commit: [SPARK-20426] Lazy initialization of FileSegmentManagedBuffer for shuffle service.

2017-04-27 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 561e9cc39 -> 85c6ce619 [SPARK-20426] Lazy initialization of FileSegmentManagedBuffer for shuffle service. ## What changes were proposed in this pull request? When application contains large amount of shuffle blocks. NodeManager requires l

spark git commit: [SPARK-20421][CORE] Mark internal listeners as deprecated.

2017-04-27 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.2 c86c078c1 -> 87d27e544 [SPARK-20421][CORE] Mark internal listeners as deprecated. These listeners weren't really meant for external consumption, but they're public and marked with DeveloperApi. Adding the deprecated tag warns people tha

spark git commit: [SPARK-20421][CORE] Mark internal listeners as deprecated.

2017-04-27 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 7633933e5 -> 561e9cc39 [SPARK-20421][CORE] Mark internal listeners as deprecated. These listeners weren't really meant for external consumption, but they're public and marked with DeveloperApi. Adding the deprecated tag warns people that th

spark git commit: [SPARK-20483] Mesos Coarse mode may starve other Mesos frameworks

2017-04-27 Thread dbtsai
Repository: spark Updated Branches: refs/heads/branch-2.2 75544c019 -> c86c078c1 [SPARK-20483] Mesos Coarse mode may starve other Mesos frameworks ## What changes were proposed in this pull request? Set maxCores to be a multiple of the smallest executor that can be launched. This ensures tha

spark git commit: [SPARK-20483] Mesos Coarse mode may starve other Mesos frameworks

2017-04-27 Thread dbtsai
Repository: spark Updated Branches: refs/heads/master ba7666274 -> 7633933e5 [SPARK-20483] Mesos Coarse mode may starve other Mesos frameworks ## What changes were proposed in this pull request? Set maxCores to be a multiple of the smallest executor that can be launched. This ensures that we

spark git commit: [SPARK-20208][DOCS][FOLLOW-UP] Add FP-Growth to SparkR programming guide

2017-04-27 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master b58cf77c4 -> ba7666274 [SPARK-20208][DOCS][FOLLOW-UP] Add FP-Growth to SparkR programming guide ## What changes were proposed in this pull request? Add `spark.fpGrowth` to SparkR programming guide. ## How was this patch tested? Manual te

spark git commit: [DOCS][MINOR] Add missing since to SparkR repeat_string note.

2017-04-27 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master b4724db19 -> b58cf77c4 [DOCS][MINOR] Add missing since to SparkR repeat_string note. ## What changes were proposed in this pull request? Replace note repeat_string 2.3.0 with note repeat_string since 2.3.0 ## How was this patch