[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17928 **[Test build #76784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76784/testReport)** for PR 17928 at commit [`a7a732b`](https://github.com/apache/spark/commit/a7a732bdbef47b4b4695843ab8005a4a85b037da). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove depre...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17944 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17644: [SPARK-17729] [SQL] Enable creating hive bucketed...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17644#discussion_r115915287
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -902,9 +902,14 @@ case class ShowCreateTableCommand(table:
TableIdentifier) extends RunnableComman
builder ++= partCols.mkString("PARTITIONED BY (", ", ", ")\n")
}
-if (metadata.bucketSpec.isDefined) {
- throw new UnsupportedOperationException(
-"Creating Hive table with bucket spec is not supported yet.")
+if (metadata.bucketSpec.nonEmpty) {
--- End diff --
nit: `isDefined` is more ideal.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17644: [SPARK-17729] [SQL] Enable creating hive bucketed...
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17644#discussion_r115914960 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -335,6 +336,32 @@ abstract class ExternalCatalog postToAll(RenameFunctionEvent(db, oldName, newName)) } + final def checkSchemaContainsBucketingColumns( --- End diff -- we should move this check to `SessionCatalog.alterTableSchema`, which already contains some checks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17928: [SPARK-20311][SQL] Support aliases for table valu...
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17928#discussion_r115914124
--- Diff: sql/core/src/test/resources/sql-tests/inputs/inline-table.sql ---
@@ -49,3 +49,6 @@ select * from values ("one", count(1)), ("two", 2) as
data(a, b);
-- string to timestamp
select * from values (timestamp('1991-12-06 00:00:00.0'),
array(timestamp('1991-12-06 01:00:00.0'), timestamp('1991-12-06 12:00:00.0')))
as data(a, b);
+
+-- cross-join inline tables
+SELECT * FROM VALUES ('one', 1), ('three', null) CROSS JOIN VALUES ('one',
1), ('three', null);
--- End diff --
Aha, ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17928: [SPARK-20311][SQL] Support aliases for table valu...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17928#discussion_r115913870
--- Diff: sql/core/src/test/resources/sql-tests/inputs/inline-table.sql ---
@@ -49,3 +49,6 @@ select * from values ("one", count(1)), ("two", 2) as
data(a, b);
-- string to timestamp
select * from values (timestamp('1991-12-06 00:00:00.0'),
array(timestamp('1991-12-06 01:00:00.0'), timestamp('1991-12-06 12:00:00.0')))
as data(a, b);
+
+-- cross-join inline tables
+SELECT * FROM VALUES ('one', 1), ('three', null) CROSS JOIN VALUES ('one',
1), ('three', null);
--- End diff --
does this expose the bug? If we treat CROSS as an alias, we still the same
result. how about we run EXPLAIN?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17770#discussion_r115913789 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -188,6 +188,9 @@ class Dataset[T] private[sql]( } } + // Wrap analyzed logical plan with an analysis barrier so we won't traverse/resolve it again. + @transient private val planWithBarrier: LogicalPlan = AnalysisBarrier(logicalPlan) --- End diff -- `logicalPlan` is necessary for caching. Otherwise we need to strip the barrier when wanting to cache it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17770 LGTM except some minor comments, cc @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17770#discussion_r115913503 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -188,6 +188,9 @@ class Dataset[T] private[sql]( } } + // Wrap analyzed logical plan with an analysis barrier so we won't traverse/resolve it again. + @transient private val planWithBarrier: LogicalPlan = AnalysisBarrier(logicalPlan) --- End diff -- shall we just add `AnalysisBarrier` when constructing `logicalPlan` instead of adding a new field `planWithBarrier`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17770#discussion_r115913413
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -2456,6 +2467,13 @@ object CleanupAliases extends Rule[LogicalPlan] {
}
}
+/** Remove the barrier nodes of analysis */
+object CleanupBarriers extends Rule[LogicalPlan] {
+ override def apply(plan: LogicalPlan): LogicalPlan = plan transform {
--- End diff --
Ok.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17770#discussion_r115913359
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
---
@@ -441,4 +441,17 @@ class AnalysisSuite extends AnalysisTest with
ShouldMatchers {
checkAnalysis(SubqueryAlias("tbl", testRelation).as("tbl2"),
testRelation)
}
+
+ test("analysis barrier") {
+// [[AnalysisBarrier]] will be removed after analysis
+checkAnalysis(
+ Project(Seq(UnresolvedAttribute("tbl.a")),
+AnalysisBarrier(SubqueryAlias("tbl", testRelation))),
+ Project(testRelation.output, SubqueryAlias("tbl", testRelation)))
+
+// Make sure we won't resolve the plans wrapped in an
[[AnalysisBarrier]]
+val barrier =
AnalysisBarrier(Project(Seq(UnresolvedAttribute("tbl.b")),
+ SubqueryAlias("tbl", testRelation)))
+assertAnalysisError(barrier, Seq("cannot resolve '`tbl.b`'"))
--- End diff --
`CheckAnalysis`. We wrap an unresolved plan with a barrier. So the
analyzer won't go through it and it remains unresolved.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17941: [SPARK-20684][R] Expose createGlobalTempView and dropGlo...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17941 Oh, I assumed there exists a case about cross languages inside SparkR and thought this is another API parity issue at first. Maybe, @falaki can give us the goal described in JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17770#discussion_r115912958
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -47,36 +47,11 @@ abstract class LogicalPlan extends
QueryPlan[LogicalPlan] with Logging {
def isStreaming: Boolean = children.exists(_.isStreaming == true)
/**
- * Returns a copy of this node where `rule` has been recursively applied
first to all of its
- * children and then itself (post-order). When `rule` does not apply to
a given node, it is left
- * unchanged. This function is similar to `transformUp`, but skips
sub-trees that have already
- * been marked as analyzed.
- *
- * @param rule the function use to transform this nodes children
- */
- def resolveOperators(rule: PartialFunction[LogicalPlan, LogicalPlan]):
LogicalPlan = {
-if (!analyzed) {
- val afterRuleOnChildren = mapChildren(_.resolveOperators(rule))
- if (this fastEquals afterRuleOnChildren) {
-CurrentOrigin.withOrigin(origin) {
- rule.applyOrElse(this, identity[LogicalPlan])
-}
- } else {
-CurrentOrigin.withOrigin(origin) {
- rule.applyOrElse(afterRuleOnChildren, identity[LogicalPlan])
-}
- }
-} else {
- this
-}
- }
-
- /**
* Recursively transforms the expressions of a tree, skipping nodes that
have already
* been analyzed.
*/
def resolveExpressions(r: PartialFunction[Expression, Expression]):
LogicalPlan = {
--- End diff --
Maybe not. Since there is not specified direction requirement described in
`resolveExpressions`. It might be safe to use `transformAllExpressions`. And
the transformation in an operator shouldn't affect the transformation in other
operators.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17935 **[Test build #76783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76783/testReport)** for PR 17935 at commit [`a904214`](https://github.com/apache/spark/commit/a9042148baeeb53c217308ea93e8997ec23b397e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16960: [SPARK-19447] Make Range operator generate "recordsRead"...
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/16960 I'll have a look at this this week and send a PR unless you beat me to it :) Thanks @ala! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17770#discussion_r115912603
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
---
@@ -441,4 +441,17 @@ class AnalysisSuite extends AnalysisTest with
ShouldMatchers {
checkAnalysis(SubqueryAlias("tbl", testRelation).as("tbl2"),
testRelation)
}
+
+ test("analysis barrier") {
+// [[AnalysisBarrier]] will be removed after analysis
+checkAnalysis(
+ Project(Seq(UnresolvedAttribute("tbl.a")),
+AnalysisBarrier(SubqueryAlias("tbl", testRelation))),
+ Project(testRelation.output, SubqueryAlias("tbl", testRelation)))
+
+// Make sure we won't resolve the plans wrapped in an
[[AnalysisBarrier]]
+val barrier =
AnalysisBarrier(Project(Seq(UnresolvedAttribute("tbl.b")),
+ SubqueryAlias("tbl", testRelation)))
+assertAnalysisError(barrier, Seq("cannot resolve '`tbl.b`'"))
--- End diff --
where is this exception thrown?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17770#discussion_r115912383
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -47,36 +47,11 @@ abstract class LogicalPlan extends
QueryPlan[LogicalPlan] with Logging {
def isStreaming: Boolean = children.exists(_.isStreaming == true)
/**
- * Returns a copy of this node where `rule` has been recursively applied
first to all of its
- * children and then itself (post-order). When `rule` does not apply to
a given node, it is left
- * unchanged. This function is similar to `transformUp`, but skips
sub-trees that have already
- * been marked as analyzed.
- *
- * @param rule the function use to transform this nodes children
- */
- def resolveOperators(rule: PartialFunction[LogicalPlan, LogicalPlan]):
LogicalPlan = {
-if (!analyzed) {
- val afterRuleOnChildren = mapChildren(_.resolveOperators(rule))
- if (this fastEquals afterRuleOnChildren) {
-CurrentOrigin.withOrigin(origin) {
- rule.applyOrElse(this, identity[LogicalPlan])
-}
- } else {
-CurrentOrigin.withOrigin(origin) {
- rule.applyOrElse(afterRuleOnChildren, identity[LogicalPlan])
-}
- }
-} else {
- this
-}
- }
-
- /**
* Recursively transforms the expressions of a tree, skipping nodes that
have already
* been analyzed.
*/
def resolveExpressions(r: PartialFunction[Expression, Expression]):
LogicalPlan = {
--- End diff --
do we really need a specific transform order for `transformAllExpressions`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17770#discussion_r115912307
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -2456,6 +2467,13 @@ object CleanupAliases extends Rule[LogicalPlan] {
}
}
+/** Remove the barrier nodes of analysis */
+object CleanupBarriers extends Rule[LogicalPlan] {
+ override def apply(plan: LogicalPlan): LogicalPlan = plan transform {
--- End diff --
`transformDown` should be better here, as we are adding new sub-trees
during transformation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17770#discussion_r115912123
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1027,8 +1030,9 @@ class Analyzer(
* The HAVING clause could also used a grouping columns that is not
presented in the SELECT.
*/
object ResolveMissingReferences extends Rule[LogicalPlan] {
-def apply(plan: LogicalPlan): LogicalPlan = plan.resolveOperators {
+def apply(plan: LogicalPlan): LogicalPlan = plan.transformUp {
// Skip sort with aggregate. This will be handled in
ResolveAggregateFunctions
+ case sa @ Sort(_, _, AnalysisBarrier(child: Aggregate)) => sa
case sa @ Sort(_, _, child: Aggregate) => sa
case s @ Sort(order, _, child) if child.resolved =>
--- End diff --
we can follow the join resolution, to clean the barriers in `child` at the
beginning.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17910 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76779/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17910 **[Test build #76779 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76779/testReport)** for PR 17910 at commit [`efda91d`](https://github.com/apache/spark/commit/efda91deda097ba2b13dd9003290da5c81186546). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove deprecated m...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17944 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove deprecated m...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17944 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76768/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76774/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17719 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove deprecated m...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17944 **[Test build #76768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76768/testReport)** for PR 17944 at commit [`185478c`](https://github.com/apache/spark/commit/185478c75183cb16a3991605b740f3c26d2687ee). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17719 **[Test build #76774 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76774/testReport)** for PR 17719 at commit [`46994fb`](https://github.com/apache/spark/commit/46994fb2f22135f19ab615265cbe9da8f24aaa15). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76773/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17719 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17719 **[Test build #76773 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76773/testReport)** for PR 17719 at commit [`cc3afd7`](https://github.com/apache/spark/commit/cc3afd7fa45468404d11dbf265966a1d298cc486). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76770/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17928 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76780/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17945 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17945 **[Test build #76780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76780/testReport)** for PR 17945 at commit [`92e6bba`](https://github.com/apache/spark/commit/92e6bba0b0f8a4968c83ec409141bb89ead42c50). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17928 **[Test build #76770 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76770/testReport)** for PR 17928 at commit [`54a05da`](https://github.com/apache/spark/commit/54a05daaf26c9ba7228e3d9c35085c3bc6015d65). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17945 **[Test build #76778 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76778/testReport)** for PR 17945 at commit [`468cef1`](https://github.com/apache/spark/commit/468cef15e16fbc5e66cf8211c4ea9070155e14d5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17945 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76778/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17945: [SPARK-20704][SPARKR] change CRAN test to run sin...
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17945#discussion_r115909228 --- Diff: R/pkg/inst/tests/testthat/jarTest.R --- @@ -16,7 +16,7 @@ # library(SparkR) -sc <- sparkR.session() +sc <- sparkR.session(master = "local[1]") --- End diff -- it's not accessible here - this is a separate R app --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17945: [SPARK-20704][SPARKR] change CRAN test to run sin...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/17945#discussion_r115909188
--- Diff: R/pkg/DESCRIPTION ---
@@ -13,6 +13,7 @@ Authors@R: c(person("Shivaram", "Venkataraman", role =
c("aut", "cre"),
License: Apache License (== 2.0)
URL: http://www.apache.org/ http://spark.apache.org/
BugReports: http://spark.apache.org/contributing.html
+OS_Type: unix
--- End diff --
right, I can revert this before merging. I want to have a branch to test
devtools install
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17887 **[Test build #76782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76782/testReport)** for PR 17887 at commit [`d8cd670`](https://github.com/apache/spark/commit/d8cd670aaf0050c8ebe96302709eadef671c615b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17644 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17887#discussion_r115908703 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi } @ExpressionDescription( - usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.") + usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.", + extended = """ +Arguments: + str - a string expression + regexp - a string expression. The pattern string should be a Java regular expression. + +Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser. +For example, if the `str` parameter is "abc\td", the `regexp` can match it is: +"^abctd$". + +Examples: + > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*' + true --- End diff -- Ok. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76767/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17644 **[Test build #76767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76767/testReport)** for PR 17644 at commit [`fc9d94f`](https://github.com/apache/spark/commit/fc9d94ff1ed712418c2866c860cabc7827cf9b4e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17910 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17910 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76777/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17910 **[Test build #76777 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76777/testReport)** for PR 17910 at commit [`82109ca`](https://github.com/apache/spark/commit/82109ca618940d2d2261da048928a748285b794a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17711: [SPARK-19951][SQL] Add string concatenate operator || to...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17711 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17711: [SPARK-19951][SQL] Add string concatenate operator || to...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17711 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76769/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17945: [SPARK-20704][SPARKR] change CRAN test to run sin...
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/17945#discussion_r115907813 --- Diff: R/pkg/inst/tests/testthat/jarTest.R --- @@ -16,7 +16,7 @@ # library(SparkR) -sc <- sparkR.session() +sc <- sparkR.session(master = "local[1]") --- End diff -- any specific reason these dont use `sparkRMaster` variable ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17711: [SPARK-19951][SQL] Add string concatenate operator || to...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17711 **[Test build #76769 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76769/testReport)** for PR 17711 at commit [`c88652c`](https://github.com/apache/spark/commit/c88652c6d3fe1070cc24109081d72834f576f73f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17945: [SPARK-20704][SPARKR] change CRAN test to run sin...
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/17945#discussion_r115907978
--- Diff: R/pkg/inst/tests/testthat/test_Serde.R ---
@@ -17,7 +17,7 @@
context("SerDe functionality")
-sparkSession <- sparkR.session(enableHiveSupport = FALSE)
+sparkSession <- sparkR.session(master = sparkRMaster, enableHiveSupport =
FALSE)
--- End diff --
Minor nit - Is `sparkRTestMaster` is a better name ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17945: [SPARK-20704][SPARKR] change CRAN test to run sin...
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/17945#discussion_r115907770
--- Diff: R/pkg/DESCRIPTION ---
@@ -13,6 +13,7 @@ Authors@R: c(person("Shivaram", "Venkataraman", role =
c("aut", "cre"),
License: Apache License (== 2.0)
URL: http://www.apache.org/ http://spark.apache.org/
BugReports: http://spark.apache.org/contributing.html
+OS_Type: unix
--- End diff --
Can we do this in a separate PR ? It'll be good to track this as a separate
JIRA / PR just to lookup why we did this. Also I'll hopefully hear from the
winbuilder maintainer if this is something we can avoid doing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17739 **[Test build #76781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76781/testReport)** for PR 17739 at commit [`cbd2402`](https://github.com/apache/spark/commit/cbd24021f0974afdded00c2667dffbfd5a05a909). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add a Bucketizer that can bin mul...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17819 @barrybecker4 `withColumns` API is first introduced in this PR. So you won't see it in Spark 2.1.1 or current codebase. Thanks for letting me know SPARK-12225. Yes, it is related. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17945 **[Test build #76780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76780/testReport)** for PR 17945 at commit [`92e6bba`](https://github.com/apache/spark/commit/92e6bba0b0f8a4968c83ec409141bb89ead42c50). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17935 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76772/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17935 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17935 **[Test build #76772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76772/testReport)** for PR 17935 at commit [`c19cbbb`](https://github.com/apache/spark/commit/c19cbbbff4bc6680430362dc201ce11f6ff9276e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17945: [SPARK-20704][SPARKR] change CRAN test to run single thr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17945 **[Test build #76778 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76778/testReport)** for PR 17945 at commit [`468cef1`](https://github.com/apache/spark/commit/468cef15e16fbc5e66cf8211c4ea9070155e14d5). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17910 **[Test build #76779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76779/testReport)** for PR 17910 at commit [`efda91d`](https://github.com/apache/spark/commit/efda91deda097ba2b13dd9003290da5c81186546). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17945: [SPARK-20704][SPARKR] change CRAN test to run sin...
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/17945 [SPARK-20704][SPARKR] change CRAN test to run single thread ## What changes were proposed in this pull request? also target unix only - [ ] need to test devtools installing this package from source on windows - [ ] need to test by running R CMD check --as-cran ## How was this patch tested? Jenkins You can merge this pull request into a Git repository by running: $ git pull https://github.com/felixcheung/spark rchangesforpackage Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/17945.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #17945 commit 468cef15e16fbc5e66cf8211c4ea9070155e14d5 Author: Felix Cheung Date: 2017-05-11T05:14:45Z change CRAN test to run single thread also target unix only --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17943: [SPARK-20682][SQL] Implement new ORC data source based o...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17943 **[Test build #76776 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76776/testReport)** for PR 17943 at commit [`d1417aa`](https://github.com/apache/spark/commit/d1417aa50f41b9b69f4a816a264d5ce96f201130). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17910: [SPARK-20669][ML] LogisticRegression family should be ca...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17910 **[Test build #76777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76777/testReport)** for PR 17910 at commit [`82109ca`](https://github.com/apache/spark/commit/82109ca618940d2d2261da048928a748285b794a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14615: [SPARK-17029] make toJSON not go through rdd form but op...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14615 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14615: [SPARK-17029] make toJSON not go through rdd form but op...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14615 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76766/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14615: [SPARK-17029] make toJSON not go through rdd form but op...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14615 **[Test build #76766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76766/testReport)** for PR 14615 at commit [`723ef1e`](https://github.com/apache/spark/commit/723ef1eea299f00355481d3013afd5ab07e010bf). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17719 Sure, I'd love to do tough, I probably missed your point. What's the scenario of the conflict you described? [The current logic in `readwriter.py`](https://github.com/maropu/spark/blob/46994fb2f22135f19ab615265cbe9da8f24aaa15/python/pyspark/sql/readwriter.py#L104) just checks types, then decides a schema. Could you give me a concrete example? Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17739 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76775/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17739 **[Test build #76775 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76775/testReport)** for PR 17739 at commit [`85f6c1d`](https://github.com/apache/spark/commit/85f6c1d8b617854586bd03ac3fdb564acd540744). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17739 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17739 **[Test build #76775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76775/testReport)** for PR 17739 at commit [`85f6c1d`](https://github.com/apache/spark/commit/85f6c1d8b617854586bd03ac3fdb564acd540744). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17943: [SPARK-20682][SQL] Implement new ORC data source based o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17943 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76764/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be a chil...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17858 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76765/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17943: [SPARK-20682][SQL] Implement new ORC data source based o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17943 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be a chil...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17858 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be a chil...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17858 **[Test build #76765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76765/testReport)** for PR 17858 at commit [`bf1b4ec`](https://github.com/apache/spark/commit/bf1b4eca9b9692d2dd11a5c166446cc2d5258e86). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17943: [SPARK-20682][SQL] Implement new ORC data source based o...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17943 **[Test build #76764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76764/testReport)** for PR 17943 at commit [`70bc00e`](https://github.com/apache/spark/commit/70bc00e3695ddec164ce626602a5e7f4b425f780). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17719 **[Test build #76774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76774/testReport)** for PR 17719 at commit [`46994fb`](https://github.com/apache/spark/commit/46994fb2f22135f19ab615265cbe9da8f24aaa15). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17926: [MINOR][SQL][PYSPARK] Allow user to specify numSlices in...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17926 FYI we added `numPartitions` in R - but that's primarily because we don't have `sc.parallelize` https://github.com/apache/spark/blob/master/R/pkg/R/SQLContext.R#L190 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17719: [SPARK-20431][SQL] Specify a schema by using a DDL-forma...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17719 **[Test build #76773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76773/testReport)** for PR 17719 at commit [`cc3afd7`](https://github.com/apache/spark/commit/cc3afd7fa45468404d11dbf265966a1d298cc486). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17935 **[Test build #76772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76772/testReport)** for PR 17935 at commit [`c19cbbb`](https://github.com/apache/spark/commit/c19cbbbff4bc6680430362dc201ce11f6ff9276e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17934: [SPARK-20501] [ML] ML 2.2 QA: New Scala APIs, doc...
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17934#discussion_r115900532 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -51,6 +51,7 @@ private[classification] trait LinearSVCParams extends ClassifierParams with HasR * Linear SVM Classifier * * This binary classifier optimizes the Hinge Loss using the OWLQN optimizer. + * Only supports L2 regularization currently. --- End diff -- is this something should be mentioned in R too? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17935 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76771/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17935 **[Test build #76771 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76771/testReport)** for PR 17935 at commit [`51aafde`](https://github.com/apache/spark/commit/51aafde24f94c6805d4bf4f0611c1c0de643e4d2). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class ColumnStatsMap(originalMap: AttributeMap[ColumnStat]) ` * ` s\"($` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17935 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17935: [SPARK-20690][SQL][WIP] Analyzer shouldn't add missing a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17935 **[Test build #76771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76771/testReport)** for PR 17935 at commit [`51aafde`](https://github.com/apache/spark/commit/51aafde24f94c6805d4bf4f0611c1c0de643e4d2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17928 **[Test build #76770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76770/testReport)** for PR 17928 at commit [`54a05da`](https://github.com/apache/spark/commit/54a05daaf26c9ba7228e3d9c35085c3bc6015d65). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove deprecated m...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17944 ok I saw #17867 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove depre...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/17944#discussion_r115898938
--- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala ---
@@ -109,24 +109,80 @@ private[ml] trait DecisionTreeParams extends
PredictorParams
setDefault(maxDepth -> 5, maxBins -> 32, minInstancesPerNode -> 1,
minInfoGain -> 0.0,
maxMemoryInMB -> 256, cacheNodeIds -> false, checkpointInterval -> 10)
+ /**
+ * @deprecated This method is deprecated and will be removed in 2.2.0.
+ * @group setParam
+ */
+ @deprecated("This method is deprecated and will be removed in 2.2.0.",
"2.1.0")
--- End diff --
is this PR not for the 2.2.0 release?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17774: [SPARK-18371][Streaming] Spark Streaming backpressure ge...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17774 @brkyvz @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17867: [SPARK-20606][ML] ML 2.2 QA: Remove deprecated methods f...
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/17867 I'm OK to revert this change and change the deprecation comments to say the items will be removed in ```3.0.0``` instead of ```2.x```. Will do them in two follow-up PRs. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17711: [SPARK-19951][SQL] Add string concatenate operator || to...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17711 **[Test build #76769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76769/testReport)** for PR 17711 at commit [`c88652c`](https://github.com/apache/spark/commit/c88652c6d3fe1070cc24109081d72834f576f73f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove deprecated m...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17944 **[Test build #76768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76768/testReport)** for PR 17944 at commit [`185478c`](https://github.com/apache/spark/commit/185478c75183cb16a3991605b740f3c26d2687ee). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17944: Revert "[SPARK-20606][ML] ML 2.2 QA: Remove depre...
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/17944 Revert "[SPARK-20606][ML] ML 2.2 QA: Remove deprecated methods for ML" This reverts commit b8733e0ad9f5a700f385e210450fd2c10137293e. Revert SPARK-20606. ## How was this patch tested? Existing tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/yanboliang/spark spark-20606-revert Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/17944.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #17944 commit 185478c75183cb16a3991605b740f3c26d2687ee Author: Yanbo Liang Date: 2017-05-11T03:19:15Z Revert "[SPARK-20606][ML] ML 2.2 QA: Remove deprecated methods for ML" This reverts commit b8733e0ad9f5a700f385e210450fd2c10137293e. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17942: [SPARK-20702][Core]TaskContextImpl.markTaskCompleted sho...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17942 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76762/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17942: [SPARK-20702][Core]TaskContextImpl.markTaskCompleted sho...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17942 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
