[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1169#issuecomment-210055592
  
Thanks for your patience @arunmahadevan . Merged into master and 1.x-branch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1169


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1169#issuecomment-210001777
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[arunmahadevan]

Github user arunmahadevan commented on the pull request:

https://github.com/apache/storm/pull/1169#issuecomment-209760668
  
@harshach @haohui this has been pending for a long time. Could you take a 
look and merge if it looks fine?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[arunmahadevan]

Github user arunmahadevan commented on the pull request:

https://github.com/apache/storm/pull/1169#issuecomment-199183363
  
@haohui addressed comments, can you take a look again?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[haohui]

Github user haohui commented on a diff in the pull request:

https://github.com/apache/storm/pull/1169#discussion_r56775610
  
--- Diff: external/sql/storm-sql-core/src/codegen/includes/parserImpls.ftl 
---
@@ -83,4 +83,23 @@ SqlNode SqlCreateTable() :
 input_format_class_name, output_format_class_name, location,
 tbl_properties, select);
 }
+}
+
+/**
+ * CREATE FUNCTION functionname AS 'classname'
+ */
+SqlNode SqlCreateFunction() :
+{
+SqlParserPos pos;
+SqlIdentifier functionName;
+SqlNode className;
+}
+{
+ { pos = getPos(); }
+
+functionName = CompoundIdentifier()
+
+className = StringLiteral() {
--- End diff --

That way the clauses might result into a parsing error. Instead of 
reporting a parsing error, it might be more user friendly to throw an 
UnimplementedException when compiling the SQL statements. It is your call.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[haohui]

Github user haohui commented on a diff in the pull request:

https://github.com/apache/storm/pull/1169#discussion_r56775558
  
--- Diff: 
external/sql/storm-sql-core/src/test/org/apache/storm/sql/TestStormSql.java ---
@@ -129,4 +134,20 @@ public void testExternalNestedInvalidAccess() throws 
Exception {
 sql.execute(stmt, h);
 Assert.assertEquals(0, values.size());
   }
+
+  @Test
+  public void testExternalUdf() throws Exception {
--- End diff --

It would be a good idea to have unit tests for UDF + type conversions.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[haohui]

Github user haohui commented on a diff in the pull request:

https://github.com/apache/storm/pull/1169#discussion_r56775507
  
--- Diff: 
external/sql/storm-sql-core/src/jvm/org/apache/storm/sql/parser/SqlCreateFunction.java
 ---
@@ -0,0 +1,68 @@
+package org.apache.storm.sql.parser;
--- End diff --

Need license header


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1169#issuecomment-195806190
  
@haohui can you take a look


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[arunmahadevan]

Github user arunmahadevan commented on a diff in the pull request:

https://github.com/apache/storm/pull/1169#discussion_r54837625
  
--- Diff: external/sql/storm-sql-core/src/codegen/includes/parserImpls.ftl 
---
@@ -83,4 +83,23 @@ SqlNode SqlCreateTable() :
 input_format_class_name, output_format_class_name, location,
 tbl_properties, select);
 }
+}
+
+/**
+ * CREATE FUNCTION functionname AS 'classname'
+ */
+SqlNode SqlCreateFunction() :
+{
+SqlParserPos pos;
+SqlIdentifier functionName;
+SqlNode className;
+}
+{
+ { pos = getPos(); }
+
+functionName = CompoundIdentifier()
+
+className = StringLiteral() {
--- End diff --

The syntax closely follows the hive syntax for defining UDFs. Whats 
required is each UDF should map to a unique class with an `evaluate` method. 
Multiple such classes can be in the same jar and added to the classpath. (for 
storm, it could be placed under extlib directory). Will update the docs with 
this info.

Hive has an additional syntax to specify jar file in its DDL syntax which 
is not supported as of now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[satishd]

Github user satishd commented on a diff in the pull request:

https://github.com/apache/storm/pull/1169#discussion_r54837137
  
--- Diff: external/sql/storm-sql-core/src/codegen/includes/parserImpls.ftl 
---
@@ -83,4 +83,23 @@ SqlNode SqlCreateTable() :
 input_format_class_name, output_format_class_name, location,
 tbl_properties, select);
 }
+}
+
+/**
+ * CREATE FUNCTION functionname AS 'classname'
+ */
+SqlNode SqlCreateFunction() :
+{
+SqlParserPos pos;
+SqlIdentifier functionName;
+SqlNode className;
+}
+{
+ { pos = getPos(); }
+
+functionName = CompoundIdentifier()
+
+className = StringLiteral() {
--- End diff --

CREATE FUNCTION functionname AS 'classname' does not take any jar location 
or name. Where should user put UDF libs? Is it mandatory to put each UDF class 
in a separate jar? 
Can you please describe how should UDF libraries packaged and placed in 
storm environment?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [Storm-1585] Add DDL support for UDFs in storm...

[arunmahadevan]

GitHub user arunmahadevan opened a pull request:

https://github.com/apache/storm/pull/1169

[Storm-1585]  Add DDL support for UDFs in storm-sql

This patch proposes to expose the user defined function support added
in STORM-1585 via DDL statements. 

Note: This includes changes from https://github.com/apache/storm/pull/1162

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arunmahadevan/storm STORM-1585

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1169.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1169


commit 521b367aeec0526b762c5a5a9b2bea2b373e5fd5
Author: Arun Mahadevan 
Date:   2016-02-26T07:41:32Z

[STORM-1586] Added UDF support in ExprComplier

commit 9826ec4cc09558300adb7c6db50a939edb83db62
Author: Arun Mahadevan 
Date:   2016-02-29T18:26:30Z

[STORM-1585] Add DDL support for UDFs in Storm-sql

This patch proposes to expose the user defined function support added
in STORM-1585 via DDL statements.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---