Re: how to write a SerDe

2009-07-08 Thread Zheng Shao
Hi Robert, Hive does support customized file input/output format other than SequenceFile/TextFile. Please take a look at Hive.g which contains the grammar for specifying InputFileFormat and OutputFileFormat. You probably only need a InputFileFormat if you only plan to read from this kind of file.

Re: Some wish after serious consideration

2009-07-08 Thread He Yongqiang
"But I think no conflicts can be guaranteed, since conflicts are not raised by one patch. If no conflict appears to this patch, then there will be conflicts for other patches." I mean But I think no conflicts can NOT be guaranteed, since conflicts are not raised by one patch. If no conflict appe

Re: Some wish after serious consideration

2009-07-08 Thread He Yongqiang
On 09-7-9 下午2:14, "Min Zhou" wrote: > Hi all, > Having focused on hive for several month, here is some wish of me after > serious consideration > >1. All auto-gen code for hive was under the facebook commercial version >of thrift, which is older than the open source one, would lead to lo

Some wish after serious consideration

2009-07-08 Thread Min Zhou
Hi all, Having focused on hive for several month, here is some wish of me after serious consideration 1. All auto-gen code for hive was under the facebook commercial version of thrift, which is older than the open source one, would lead to lots of compatible problems and stop from all hel

[jira] Updated: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-553: Attachment: HIVE-553.4.patch Removed the println. Also I reverted to include the serialization/deserializati

[jira] Updated: (HIVE-617) Script to start classes with hadoop and hive environment

2009-07-08 Thread Edward Capriolo (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-617: - Attachment: TestHive.java hive-617.patch Included the patch, and a sample applicati

[jira] Updated: (HIVE-617) Script to start classes with hadoop and hive environment

2009-07-08 Thread Edward Capriolo (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-617: - Status: Patch Available (was: Open) patch adds bin/hive --jar > Script to start classes with hado

[jira] Created: (HIVE-617) Script to start classes with hadoop and hive environment

2009-07-08 Thread Edward Capriolo (JIRA)
Script to start classes with hadoop and hive environment Key: HIVE-617 URL: https://issues.apache.org/jira/browse/HIVE-617 Project: Hadoop Hive Issue Type: New Feature Compon

[jira] Commented: (HIVE-329) start and stop hive thrift server in daemon mode

2009-07-08 Thread Min Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729012#action_12729012 ] Min Zhou commented on HIVE-329: --- start need a port number, but stop needn't. > start and stop

[jira] Commented: (HIVE-533) predicate clauses with rand() should not be pushed down

2009-07-08 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729006#action_12729006 ] Namit Jain commented on HIVE-533: - Can you add some constants ? like 1/2 for boolean false/tr

[jira] Commented: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.

2009-07-08 Thread Min Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729009#action_12729009 ] Min Zhou commented on HIVE-555: --- @Zheng It would involved some logic out of the FuctionTask. Ac

[jira] Commented: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729004#action_12729004 ] Zheng Shao commented on HIVE-555: - @HIVE-555-3.patch. Can we change the error messages to som

[jira] Commented: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729003#action_12729003 ] Namit Jain commented on HIVE-553: - remove the system.out.println in binarysortableserde > Ad

[jira] Updated: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.

2009-07-08 Thread Min Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated HIVE-555: -- Attachment: HIVE-555-3.patch patch followed namit's comments. > create temporary function support not only udf,

[jira] Updated: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-553: Attachment: HIVE-553.3.patch Removed the support of Map type. We didn't find a good way to define the order o

[jira] Updated: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-553: Affects Version/s: (was: 0.4.0) 0.3.1 0.3.0 S

[jira] Commented: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728997#action_12728997 ] Zheng Shao commented on HIVE-553: - The plan is to have 2 SerDes: A. LazyBinarySerDe: with pr

how to write a SerDe

2009-07-08 Thread Roberto Congiu
Hi,I am writing a SerDe class to be able to query some proprietary format we have from hive. The format is basically a sequence of records that are maps coded in binary for which we have access libraries. The file is also gzipped. For what I understand, I need to 1 - write a FileInputFormat class

[jira] Commented: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.

2009-07-08 Thread Min Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728989#action_12728989 ] Min Zhou commented on HIVE-555: --- 1. I thought it would be a common function for generic udf er

Bay Area Hive User Group Meeting

2009-07-08 Thread Ashish Thusoo
Hi Folks, We will be having a user/developer meeting about Hive at Facebook HQ on Aug 3rd, 2009. The following is the event information: http://www.facebook.com/event.php?eid=101680018743 Somethings that we would like to cover: 1. Talk about Hive features in detail 2. Talk about Extensibility f

[jira] Commented: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.

2009-07-08 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728921#action_12728921 ] Namit Jain commented on HIVE-555: - Mostly looks good - I had a couple of minor comments. 1.

[jira] Commented: (HIVE-417) Implement Indexing in Hive

2009-07-08 Thread Prasad Chakka (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728900#action_12728900 ] Prasad Chakka commented on HIVE-417: One thing that isn't mentioned is that in List> col

[jira] Assigned: (HIVE-607) Create statistical UDFs.

2009-07-08 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-607: --- Assignee: Emil Ibrishimov (was: Namit Jain) > Create statistical UDFs. > > >

[jira] Assigned: (HIVE-615) new user functions needed for HIVE

2009-07-08 Thread Emil Ibrishimov (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ibrishimov reassigned HIVE-615: Assignee: Emil Ibrishimov (was: Namit Jain) > new user functions needed for HIVE > -

[jira] Commented: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Raghotham Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728813#action_12728813 ] Raghotham Murthy commented on HIVE-553: --- So, when LazyBinarySortableSerDe/LazyBinaryCom

Build failed in Hudson: Hive-trunk-h0.19 #148

2009-07-08 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/148/changes Changes: [rmurthy] HIVE-616. Make hive work with symbolic linked sub directories (Zheng Shao via rmurthy) [namit] HIVE-608, No type conversion in GenericUDF (Zheng Shao via namit)

Build failed in Hudson: Hive-trunk-h0.18 #149

2009-07-08 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/149/changes Changes: [rmurthy] HIVE-616. Make hive work with symbolic linked sub directories (Zheng Shao via rmurthy) [namit] HIVE-608, No type conversion in GenericUDF (Zheng Shao via namit)

Build failed in Hudson: Hive-trunk-h0.17 #147

2009-07-08 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/147/changes Changes: [rmurthy] HIVE-616. Make hive work with symbolic linked sub directories (Zheng Shao via rmurthy) [namit] HIVE-608, No type conversion in GenericUDF (Zheng Shao via namit)

[jira] Updated: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.

2009-07-08 Thread Min Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated HIVE-555: -- Attachment: HIVE-555-2.patch with unit tests. > create temporary function support not only udf, but also udaf,

[jira] Commented: (HIVE-417) Implement Indexing in Hive

2009-07-08 Thread Yongqiang He (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728605#action_12728605 ] Yongqiang He commented on HIVE-417: --- Had a talk with Prasad and Zheng about the index desig

[jira] Assigned: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao reassigned HIVE-553: --- Assignee: Zheng Shao > Add BinarySortableSerDe to Hive > --- > >

[jira] Updated: (HIVE-553) Add LazyBinarySerDe to Hive

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-553: Attachment: HIVE-553.2.patch This patch adds the BinarySortableSerDe, which is used to replace DynamicSerDe

[jira] Updated: (HIVE-553) Add BinarySortableSerDe to Hive

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-553: Summary: Add BinarySortableSerDe to Hive (was: Add LazyBinarySerDe to Hive) > Add BinarySortableSerDe to Hiv

[jira] Resolved: (HIVE-616) Make hive work with symbolic linked sub directories

2009-07-08 Thread Raghotham Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghotham Murthy resolved HIVE-616. --- Resolution: Fixed Fix Version/s: 0.4.0 Assignee: Zheng Shao Release Note:

[jira] Commented: (HIVE-616) Make hive work with symbolic linked sub directories

2009-07-08 Thread Raghotham Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728555#action_12728555 ] Raghotham Murthy commented on HIVE-616: --- +1 Will commit it. > Make hive work with sym

[jira] Updated: (HIVE-616) Make hive work with symbolic linked sub directories

2009-07-08 Thread Zheng Shao (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-616: Attachment: HIVE-616.1.patch This patch fixes the problem. > Make hive work with symbolic linked sub directo

[jira] Created: (HIVE-616) Make hive work with symbolic linked sub directories

2009-07-08 Thread Zheng Shao (JIRA)
Make hive work with symbolic linked sub directories --- Key: HIVE-616 URL: https://issues.apache.org/jira/browse/HIVE-616 Project: Hadoop Hive Issue Type: Bug Reporter: Zheng Shao