spark git commit: [SPARK-17069] Expose spark.range() as table-valued function in SQL

2016-08-18 Thread rxin 
Repository: spark
Updated Branches:
  refs/heads/branch-2.0 176af17a7 -> ea684b69c


[SPARK-17069] Expose spark.range() as table-valued function in SQL

This adds analyzer rules for resolving table-valued functions, and adds one 
builtin implementation for range(). The arguments for range() are the same as 
those of `spark.range()`.

Unit tests.

cc hvanhovell

Author: Eric Liang 

Closes #14656 from ericl/sc-4309.

(cherry picked from commit 412dba63b511474a6db3c43c8618d803e604bc6b)
Signed-off-by: Reynold Xin 


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ea684b69
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ea684b69
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ea684b69

Branch: refs/heads/branch-2.0
Commit: ea684b69cd6934bc093f4a5a8b0d8470e92157cd
Parents: 176af17
Author: Eric Liang 
Authored: Thu Aug 18 13:33:55 2016 +0200
Committer: Reynold Xin 
Committed: Thu Aug 18 18:36:50 2016 -0700

--
 .../apache/spark/sql/catalyst/parser/SqlBase.g4 |   1 +
 .../spark/sql/catalyst/analysis/Analyzer.scala  |   1 +
 .../analysis/ResolveTableValuedFunctions.scala  | 132 +++
 .../sql/catalyst/analysis/unresolved.scala  |  11 ++
 .../spark/sql/catalyst/parser/AstBuilder.scala  |   8 ++
 .../sql/catalyst/parser/PlanParserSuite.scala   |   8 +-
 .../sql-tests/inputs/table-valued-functions.sql |  20 +++
 .../results/table-valued-functions.sql.out  |  87 
 8 files changed, 267 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/ea684b69/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
--
diff --git 
a/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 
b/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
index aca7282..51f3804 100644
--- 
a/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
+++ 
b/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
@@ -426,6 +426,7 @@ relationPrimary
 | '(' queryNoWith ')' sample? (AS? strictIdentifier)?   
#aliasedQuery
 | '(' relation ')' sample? (AS? strictIdentifier)?  
#aliasedRelation
 | inlineTable   
#inlineTableDefault2
+| identifier '(' (expression (',' expression)*)? ')'
#tableValuedFunction
 ;
 
 inlineTable

http://git-wip-us.apache.org/repos/asf/spark/blob/ea684b69/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
--
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
index 57c3d9a..e0b8166 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
@@ -86,6 +86,7 @@ class Analyzer(
   WindowsSubstitution,
   EliminateUnions),
 Batch("Resolution", fixedPoint,
+  ResolveTableValuedFunctions ::
   ResolveRelations ::
   ResolveReferences ::
   ResolveDeserializer ::

http://git-wip-us.apache.org/repos/asf/spark/blob/ea684b69/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
--
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
new file mode 100644
index 000..7fdf7fa
--- /dev/null
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.

spark git commit: [SPARK-17069] Expose spark.range() as table-valued function in SQL

2016-08-18 Thread hvanhovell 
Repository: spark
Updated Branches:
  refs/heads/master b81421afb -> 412dba63b


[SPARK-17069] Expose spark.range() as table-valued function in SQL

## What changes were proposed in this pull request?

This adds analyzer rules for resolving table-valued functions, and adds one 
builtin implementation for range(). The arguments for range() are the same as 
those of `spark.range()`.

## How was this patch tested?

Unit tests.

cc hvanhovell

Author: Eric Liang 

Closes #14656 from ericl/sc-4309.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/412dba63
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/412dba63
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/412dba63

Branch: refs/heads/master
Commit: 412dba63b511474a6db3c43c8618d803e604bc6b
Parents: b81421a
Author: Eric Liang 
Authored: Thu Aug 18 13:33:55 2016 +0200
Committer: Herman van Hovell 
Committed: Thu Aug 18 13:33:55 2016 +0200

--
 .../apache/spark/sql/catalyst/parser/SqlBase.g4 |   1 +
 .../spark/sql/catalyst/analysis/Analyzer.scala  |   1 +
 .../analysis/ResolveTableValuedFunctions.scala  | 132 +++
 .../sql/catalyst/analysis/unresolved.scala  |  11 ++
 .../spark/sql/catalyst/parser/AstBuilder.scala  |   8 ++
 .../sql/catalyst/parser/PlanParserSuite.scala   |   8 +-
 .../sql-tests/inputs/table-valued-functions.sql |  20 +++
 .../results/table-valued-functions.sql.out  |  87 
 8 files changed, 267 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/412dba63/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
--
diff --git 
a/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 
b/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
index 6122bcd..cab7c3f 100644
--- 
a/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
+++ 
b/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
@@ -433,6 +433,7 @@ relationPrimary
 | '(' queryNoWith ')' sample? (AS? strictIdentifier)?   
#aliasedQuery
 | '(' relation ')' sample? (AS? strictIdentifier)?  
#aliasedRelation
 | inlineTable   
#inlineTableDefault2
+| identifier '(' (expression (',' expression)*)? ')'
#tableValuedFunction
 ;
 
 inlineTable

http://git-wip-us.apache.org/repos/asf/spark/blob/412dba63/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
--
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
index cfab6ae..333dd4d 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
@@ -86,6 +86,7 @@ class Analyzer(
   EliminateUnions,
   new SubstituteUnresolvedOrdinals(conf)),
 Batch("Resolution", fixedPoint,
+  ResolveTableValuedFunctions ::
   ResolveRelations ::
   ResolveReferences ::
   ResolveDeserializer ::

http://git-wip-us.apache.org/repos/asf/spark/blob/412dba63/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
--
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
new file mode 100644
index 000..7fdf7fa
--- /dev/null
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apac