[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12903987#action_12903987
]
Namit Jain commented on HIVE-471:
-
+1
will commit if the tests pass
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Fix For: 0.7.0
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
HIVE-471.3.patch, HIVE-471.4.patch, HIVE-471.5.patch, HIVE-471.6.patch.txt,
hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902928#action_12902928
]
Namit Jain commented on HIVE-471:
-
@Edward, negative tests are good. We are still adding a lot of them
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Fix For: 0.7.0
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
HIVE-471.3.patch, HIVE-471.4.patch, HIVE-471.5.patch, hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902583#action_12902583
]
Namit Jain commented on HIVE-471:
-
@Edward, can you regenerate the patch - this has been hanging for a long time,
and we should try to get it in
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.5.1
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
HIVE-471.3.patch, HIVE-471.4.patch, hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12870764#action_12870764
]
Paul Yang commented on HIVE-471:
+1
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.5.1
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Fix For: 0.6.0
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
HIVE-471.3.patch, hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12870775#action_12870775
]
Namit Jain commented on HIVE-471:
-
@Edward, can you regenerate the patch - I am getting some conflicts ?
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.5.1
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Fix For: 0.6.0
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
HIVE-471.3.patch, hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847398#action_12847398
]
Edward Capriolo commented on HIVE-471:
--
Can we go +1 on this? I have three sets of UDFs I want to follow up with and
since they all share the FunctionRegistry.java. They will all conflict.
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
HIVE-471.3.patch, hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12832120#action_12832120
]
Zheng Shao commented on HIVE-471:
-
Thanks Paul and Edward.
Edward, please proceed with the annotations and commit when you get time.
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12831584#action_12831584
]
Zheng Shao commented on HIVE-471:
-
In most cases, the class name/method name will be passed in as a Text
(StringWritable) or a Java String (there can be other cases but relatively
rare).
Also, the class name/method name usually do not change.
With the current implementation, we avoid doing object creation for most of the
time (the only function called is ObjectInspectorUtils.compare()).
If we convert the name to String and compare with String.equals, then we have
to do an object conversion (which may involve object creation) for each of the
rows.
I will add these as the comments in the code. Thanks for asking. That's a good
question to answer in the code comments.
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Attachments: hive-471-gen.diff, HIVE-471.1.patch, hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12831590#action_12831590
]
Edward Capriolo commented on HIVE-471:
--
@Zheng,
Thank you for following up on this UDF. my version internally but this version
looks/ is superior in most ways. I will test it out and do my own commit. I
will also add the annotations I think all the UDF's need.
Ed
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12831627#action_12831627
]
Paul Yang commented on HIVE-471:
Got it, thanks for the clarification. Other than that, the patch looks good.
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12739965#action_12739965
]
Ashish Thusoo commented on HIVE-471:
Sorry in the delay in responding. Zheng would be able to give you better
answers here.
For the error that you are getting I think you have to end up using
ObjectInspectorConvertors I think but Zheng can confirm.
For different return type you could take a look at the GenericUDFCase.java and
look at how it defines ReturnObjectInspectorResolver. Similar to that you could
have a ReflectionObjectInspectorResolver which returns the proper
ObjectInspector corresponding to the return type.
ObjectInspectors essentially encode the type and the in memory formats of the
object. Let us know if you are able to make progress by following
GenericUDFCase.java.
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Fix For: 0.4.0
Attachments: hive-471-gen.diff, hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12707666#action_12707666
]
Zheng Shao commented on HIVE-471:
-
One thing for improvement is that the reflection should NOT be done on a
per-row basis. We should do it once and save the Method (on the first
invocation).
A UDF for simple reflection
---
Key: HIVE-471
URL: https://issues.apache.org/jira/browse/HIVE-471
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor
Fix For: 0.4.0
Attachments: hive-471.diff
There are many methods in java that are static and have no arguments or can
be invoked with one simple parameter. More complicated functions will require
a UDF but one generic one can work as a poor-mans UDF.
{noformat}
SELECT reflect(java.lang.String, valueOf, 1), reflect(java.lang.String,
isEmpty)
FROM src LIMIT 1;
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
