RE: Anyway in hive to measure query performance.

2010-11-02 Thread Siying Dong
We are still building infrastructure to make performance optimizing easier, but for now, all the measurements are kind of manual. Especially to the component/operations level, we don't have a good tool to tell it yet. What we are doing now, is to select some typical benchmark queries that cover

RE: Regarding HIVE-1737

2011-03-03 Thread Siying Dong
[mohitsi...@huawei.com] Sent: Tuesday, March 01, 2011 7:08 AM To: Siying Dong Cc: Namit Jain; chinna...@huawei.com; hive-...@hadoop.apache.org Subject: FW: Regarding HIVE-1737 Hi Namit/Siying, Ok, even I agree with your analysis. Both the fixed and variable row size evaluated wrongly here

Review Request: for HIVE-2068

2011-04-07 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/540/ --- Review request for hive and namit jain. Summary --- For HIVE-2068 This

Review Request: review board for HIVE-2093

2011-04-07 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/566/ --- Review request for hive, Yongqiang He and namit jain. Summary --- Still

Re: Review Request: review board for HIVE-2093

2011-04-08 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/566/ --- (Updated 2011-04-08 07:19:12.088932) Review request for hive, Yongqiang He and

RE: [ANNOUNCE] New Hive Committer - Siying Dong

2011-04-14 Thread Siying Dong
Thanks all! -Original Message- From: John Sichi [mailto:jsi...@fb.com] Sent: Thursday, April 14, 2011 11:23 AM To: dev@hive.apache.org Subject: Re: [ANNOUNCE] New Hive Committer - Siying Dong Good job, Siying! JVS On Apr 14, 2011, at 10:42 AM, yongqiang he wrote: Congrats Siying

Re: Review Request: for HIVE-2068

2011-04-15 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/540/ --- (Updated 2011-04-15 18:37:21.441402) Review request for hive and namit jain.

Re: Review Request: review board for HIVE-2093

2011-04-19 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/566/ --- (Updated 2011-04-19 19:25:22.632716) Review request for hive, Yongqiang He and

Review Request: Input Sampling Splits

2011-04-20 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/633/ --- Review request for hive, Ning Zhang and namit jain. Summary --- We need a

Re: Review Request: review board for HIVE-2093

2011-04-21 Thread Siying Dong
--- On 2011-04-19 19:25:22, Siying Dong wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/566/ --- (Updated 2011-04-19 19:25

Re: Review Request: review board for HIVE-2093

2011-04-21 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/566/ --- (Updated 2011-04-21 22:21:03.518662) Review request for hive, Yongqiang He and

Review Request: CommandNeedRetryException needs release locks and some related code cleaning up

2011-04-21 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/646/ --- Review request for hive, Yongqiang He and namit jain. Summary --- now when

Re: Review Request: CommandNeedRetryException needs release locks and some related code cleaning up

2011-04-21 Thread Siying Dong
don't get two approaches twisted. - Siying On 2011-04-22 00:11:43, Siying Dong wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/646

Re: Review Request: CommandNeedRetryException needs release locks and some related code cleaning up

2011-04-21 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/646/ --- (Updated 2011-04-22 04:22:16.136899) Review request for hive, Yongqiang He and

Re: Review Request: Input Sampling Splits

2011-04-26 Thread Siying Dong
sample the input data and we won't get much benefit. - Siying On 2011-04-20 18:28:29, Siying Dong wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/633

Re: Review Request: Input Sampling Splits

2011-04-26 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/633/ --- (Updated 2011-04-26 21:19:18.557345) Review request for hive, Ning Zhang and

Re: Review Request: Input Sampling Splits

2011-04-26 Thread Siying Dong
On 2011-04-26 20:50:30, Siying Dong wrote: trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java, line 498 https://reviews.apache.org/r/633/diff/1/?file=16093#file16093line498 I feel like it is a little hard to explain what this sample guarantees

Review Request: Block Sampling should adjust number of reducers accordingly

2011-05-03 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/685/ --- Review request for hive, Ning Zhang and namit jain. Summary --- Now number

Re: Review Request: HIVE-2035 Use block level merge on rcfile if intermediate merge is needed

2011-06-17 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/935/#review864 ---

Re: Review Request: HIVE-2035 Use block level merge on rcfile if intermediate merge is needed

2011-06-21 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/935/#review875 --- Can you make sure that in the test cases, the query need the merge

Review Request: Cli: Print Hadoop's CPU milliseconds

2011-06-23 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/948/ --- Review request for hive, Yongqiang He, Ning Zhang, and namit jain. Summary

Review Request: reduce name node calls in hive by creating temporary directories

2011-06-24 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/952/ --- Review request for hive, Yongqiang He, Ning Zhang, and namit jain. Summary

Re: Review Request: HIVE-306 Support INSERT INTO

2011-06-27 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/926/#review925 --- trunk/ql/src/test/queries/clientnegative/insert_into3.q

Re: Review Request: Local mode needs to work well with block sampling

2011-07-15 Thread Siying Dong
wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1132/ --- (Updated 2011-07-15 02:16:34) Review request for hive and Siying

Re: Review Request: Local mode needs to work well with block sampling

2011-07-15 Thread Siying Dong
/1132/ --- (Updated 2011-07-15 02:16:34) Review request for hive and Siying Dong. Summary --- A query should run in local mode when block sampling is used and the sample is small enough. The size of the sample

Re: Review Request: Cli: Print Hadoop's CPU milliseconds

2011-07-20 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/948/ --- (Updated 2011-07-20 06:27:19.820431) Review request for hive, Yongqiang He, Ning

Re: Review Request: reduce name node calls in hive by creating temporary directories

2011-07-20 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/952/ --- (Updated 2011-07-20 23:31:54.007436) Review request for hive, Yongqiang He, Ning

Re: Review Request: Cli: Print Hadoop's CPU milliseconds

2011-07-21 Thread Siying Dong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/948/ --- (Updated 2011-07-21 17:30:55.228025) Review request for hive, Yongqiang He, Ning

Re: Review Request: HIVE-2272: add TIMESTAMP data type

2011-08-05 Thread Siying Dong
/ --- (Updated 2011-07-28 21:59:38) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Summary --- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond

RE: HIVE-2282

2011-08-11 Thread Siying Dong
Kevin, probably there are still some non-deterministic in your test case. Can you careful examine it? -Original Message- From: John Sichi Sent: Thursday, August 11, 2011 4:44 PM To: Siying Dong; Kevin Wilfong Cc: dev@hive.apache.org Subject: HIVE-2282 The unit test

Re: Review Request: Warn user that precision is lost when bigint is implicitly cast to double.

2011-08-17 Thread Siying Dong
wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1515/ --- (Updated 2011-08-15 23:37:16) Review request for hive and Siying

Re: Review Request: Warn user that precision is lost when bigint is implicitly cast to double.

2011-08-17 Thread Siying Dong
/ --- (Updated 2011-08-17 18:34:44) Review request for hive and Siying Dong. Summary --- I added a check in the code for equality expressions (includes inequalities) with operands of different types, that throws an error or logs a warning, depending

[jira] Updated: (HIVE-1638) convert commonly used udfs to generic udfs

2010-10-18 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1638: -- Attachment: HIVE-1638.4.patch One file is missing in HIVE-1638.3.patch convert commonly used udfs

[jira] Updated: (HIVE-1638) convert commonly used udfs to generic udfs

2010-10-18 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1638: -- Attachment: HIVE-1638.5.patch fix the bug that Namit pointed out. convert commonly used udfs

[jira] Updated: (HIVE-1638) convert commonly used udfs to generic udfs

2010-10-18 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1638: -- Attachment: HIVE-1638.6.patch Minor fix in comment and desc. convert commonly used udfs to generic

[jira] Updated: (HIVE-1737) Two Bugs for Estimating Row Sizes in GroupByOperator

2010-10-20 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1737: -- Status: Patch Available (was: Open) Two Bugs for Estimating Row Sizes in GroupByOperator

[jira] Updated: (HIVE-1738) Optimize Key Comparison in GroupByOperator

2010-10-21 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1738: -- Status: Patch Available (was: Open) Optimize Key Comparison in GroupByOperator

[jira] Commented: (HIVE-1738) Optimize Key Comparison in GroupByOperator

2010-10-21 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923397#action_12923397 ] Siying Dong commented on HIVE-1738: --- One note: for the query above, input format

[jira] Updated: (HIVE-1738) Optimize Key Comparison in GroupByOperator

2010-10-21 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1738: -- Attachment: HIVE.1738.3.patch Modify according to Namit's comments. Optimize Key Comparison

[jira] Updated: (HIVE-1749) ExecMapper and ExecReducer reduce function calls to l4j.isInfoEnabled()

2010-10-25 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1749: -- Summary: ExecMapper and ExecReducer reduce function calls to l4j.isInfoEnabled() (was: ExecMapper

[jira] Commented: (HIVE-1750) Remove Partition Filtering Conditions when Possible

2010-11-02 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12927533#action_12927533 ] Siying Dong commented on HIVE-1750: --- Amareshwari, it's a good catch. I'll make a put

[jira] Commented: (HIVE-1750) Remove Partition Filtering Conditions when Possible

2010-11-02 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12927535#action_12927535 ] Siying Dong commented on HIVE-1750: --- In the case that at least one partition is a table

[jira] Commented: (HIVE-1721) use bloom filters to improve the performance of joins

2010-11-02 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12927551#action_12927551 ] Siying Dong commented on HIVE-1721: --- It is a common use case? Small table is so big

[jira] Commented: (HIVE-1721) use bloom filters to improve the performance of joins

2010-11-02 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12927567#action_12927567 ] Siying Dong commented on HIVE-1721: --- So the idea is, the filtered rows in the big table

[jira] Commented: (HIVE-1750) Remove Partition Filtering Conditions when Possible

2010-11-02 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12927697#action_12927697 ] Siying Dong commented on HIVE-1750: --- Namit, sorry I misunderstood. Yes, maybe

[jira] Updated: (HIVE-1750) Remove Partition Filtering Conditions when Possible

2010-11-03 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1750: -- Status: Patch Available (was: Open) Remove Partition Filtering Conditions when Possible

[jira] Updated: (HIVE-1751) Optimize ColumnarStructObjectInspector.getStructFieldData()

2010-11-03 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1751: -- Attachment: HIVE-1751.1.patch ExprNodeColumnEvaluator.evaluate() is very heavily used function

[jira] Updated: (HIVE-1751) Optimize ColumnarStructObjectInspector.getStructFieldData()

2010-11-03 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1751: -- Status: Patch Available (was: Open) Optimize ColumnarStructObjectInspector.getStructFieldData

[jira] Updated: (HIVE-1743) Group-by to determine equals of Keys in reverse order

2010-11-08 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1743: -- Attachment: HIVE-1743.1.patch Use revers order to compare list of keys. Also a couple of minor bugs

[jira] Created: (HIVE-1779) Implement GenericUDF str_to_map

2010-11-09 Thread Siying Dong (JIRA)
Implement GenericUDF str_to_map --- Key: HIVE-1779 URL: https://issues.apache.org/jira/browse/HIVE-1779 Project: Hive Issue Type: New Feature Reporter: Siying Dong Assignee: Siying Dong

[jira] Updated: (HIVE-1779) Implement GenericUDF str_to_map

2010-11-09 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1779: -- Attachment: HIVE-1779.1.patch Implement GenericUDF str_to_map

[jira] Updated: (HIVE-1779) Implement GenericUDF str_to_map

2010-11-09 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1779: -- Status: Patch Available (was: Open) Implement GenericUDF str_to_map

[jira] Updated: (HIVE-1743) Group-by to determine equals of Keys in reverse order

2010-11-09 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1743: -- Status: Patch Available (was: Open) Group-by to determine equals of Keys in reverse order

[jira] Updated: (HIVE-1752) Avoid UnionStructObjectInspector for partition columns when necessary

2010-11-11 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1752: -- Description: Once HIVE-1750 and HIVE-1538 are finished, union struct for partition columns and normal

[jira] Created: (HIVE-1783) CommonJoinOperator optimize the case that 1:1 join

2010-11-11 Thread Siying Dong (JIRA)
Dong Assignee: Siying Dong Priority: Minor CommonJoinOperator.genObject() is expensive. It does a recursive and keeps lots of states because it has to: 1. handle null cases for outer joins 2. handle the case of duplicated keys from one join party We can do a minor

[jira] Updated: (HIVE-1783) CommonJoinOperator optimize the case of 1:1 join

2010-11-11 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1783: -- Summary: CommonJoinOperator optimize the case of 1:1 join (was: CommonJoinOperator optimize the case

[jira] Updated: (HIVE-1783) CommonJoinOperator optimize the case of 1:1 join

2010-11-11 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1783: -- Attachment: HIVE-1783.1.patch CommonJoinOperator optimize the case of 1:1 join

[jira] Updated: (HIVE-1783) CommonJoinOperator optimize the case of 1:1 join

2010-11-11 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1783: -- Status: Patch Available (was: Open) CommonJoinOperator optimize the case of 1:1 join

[jira] Updated: (HIVE-1786) better documentation for str_to_map

2010-11-12 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1786: -- Attachment: HIVE.1786.1.patch Added describe function extend to function str_to_map(), mentioning

[jira] Updated: (HIVE-1786) better documentation for str_to_map

2010-11-12 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1786: -- Status: Patch Available (was: Open) better documentation for str_to_map

[jira] Updated: (HIVE-1783) CommonJoinOperator optimize the case of 1:1 join

2010-11-15 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1783: -- Attachment: HIVE-1783.2.patch Added a unit test. I ran the test with and without the patch applied

[jira] Updated: (HIVE-1783) CommonJoinOperator optimize the case of 1:1 join

2010-11-15 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1783: -- Status: Patch Available (was: Open) CommonJoinOperator optimize the case of 1:1 join

[jira] Assigned: (HIVE-1794) GenericUDFOr and GenericUDFAnd cannot receive boolean typed object

2010-11-15 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong reassigned HIVE-1794: - Assignee: Siying Dong GenericUDFOr and GenericUDFAnd cannot receive boolean typed object

[jira] Updated: (HIVE-1783) CommonJoinOperator optimize the case of 1:1 join

2010-11-17 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1783: -- Attachment: HIVE-1783.3.patch after previous patches. CommonJoinOperator optimize the case of 1:1

[jira] Updated: (HIVE-1783) CommonJoinOperator optimize the case of 1:1 join

2010-11-17 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1783: -- Attachment: HIVE-1783.4.patch with hive.outerjoin.supports.filters=true and false; CommonJoinOperator

[jira] Updated: (HIVE-1801) HiveInputFormat or CombineHiveInputFormat always sync blocks of RCFile twice

2010-11-18 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1801: -- Attachment: HIVE-1801.1.patch Still running tests. HiveInputFormat or CombineHiveInputFormat always

[jira] Updated: (HIVE-1801) HiveInputFormat or CombineHiveInputFormat always sync blocks of RCFile twice

2010-11-18 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1801: -- Status: Patch Available (was: Open) HiveInputFormat or CombineHiveInputFormat always sync blocks

[jira] Created: (HIVE-1802) Encode MapReduce Shuffling Keys Differently for Single string/bigint Key

2010-11-19 Thread Siying Dong (JIRA)
: Improvement Reporter: Siying Dong Assignee: Siying Dong Delimiters are not needed if we only have one shuffling key, and in the same time escaping delimiters are not needed. We can save some CPU time on serializing and shuffle slightly less amount of data to save memory

[jira] Updated: (HIVE-1802) Encode MapReduce Shuffling Keys Differently for Single string/bigint Key

2010-11-19 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1802: -- Status: Patch Available (was: Open) Encode MapReduce Shuffling Keys Differently for Single string

[jira] Updated: (HIVE-1787) optimize the code path when there are no outer joins

2010-11-19 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1787: -- Attachment: HIVE-1787.1.patch 1. improve CommonJoinOperator.genUniqueJoinObject() to avoid to use

[jira] Updated: (HIVE-1787) optimize the code path when there are no outer joins

2010-11-19 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1787: -- Status: Patch Available (was: Open) optimize the code path when there are no outer joins

[jira] Updated: (HIVE-1801) HiveInputFormat or CombineHiveInputFormat always sync blocks of RCFile twice

2010-11-22 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1801: -- Attachment: HIVE-1802.1.patch address Yongqiang's comment. HiveInputFormat or CombineHiveInputFormat

[jira] Updated: (HIVE-1802) Encode MapReduce Shuffling Keys Differently for Single string/bigint Key

2010-11-23 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1802: -- Status: Patch Available (was: Open) Encode MapReduce Shuffling Keys Differently for Single string

[jira] Commented: (HIVE-1802) Encode MapReduce Shuffling Keys Differently for Single string/bigint Key

2010-11-23 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12934980#action_12934980 ] Siying Dong commented on HIVE-1802: --- For any Group by, we needed 2 mem-copies. One from

[jira] Commented: (HIVE-1802) Encode MapReduce Shuffling Keys Differently for Single string/bigint Key

2010-11-29 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964932#action_12964932 ] Siying Dong commented on HIVE-1802: --- Yongqiang, after some face-to-face discussion

[jira] Commented: (HIVE-1844) Hanging hive client caused by TaskRunner's OutOfMemoryError

2010-12-10 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12970369#action_12970369 ] Siying Dong commented on HIVE-1844: --- I have two comments here: 1. Instead of catch

[jira] Commented: (HIVE-1936) hive.semantic.analyzer.hook cannot have multiple values

2011-01-28 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12988346#action_12988346 ] Siying Dong commented on HIVE-1936: --- That's what I'm planning to do. Do you have any

[jira] Commented: (HIVE-1936) hive.semantic.analyzer.hook cannot have multiple values

2011-01-28 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12988355#action_12988355 ] Siying Dong commented on HIVE-1936: --- Ashutosh, your codes will be definitely helpful

[jira] Updated: (HIVE-1936) hive.semantic.analyzer.hook cannot have multiple values

2011-01-28 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1936: -- Attachment: HIVE-1936.1.patch 1. separate different semantic anlaysis hooks by commas 2. add unit tests

[jira] Updated: (HIVE-1936) hive.semantic.analyzer.hook cannot have multiple values

2011-01-28 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1936: -- Status: Patch Available (was: Open) Still running test suites. hive.semantic.analyzer.hook cannot

[jira] Commented: (HIVE-1517) ability to select across a database

2011-02-08 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12992139#comment-12992139 ] Siying Dong commented on HIVE-1517: --- I notice that Carl's patch added cross database

[jira] Commented: (HIVE-1517) ability to select across a database

2011-02-08 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12992165#comment-12992165 ] Siying Dong commented on HIVE-1517: --- Sorry, I mean DESCRIBE. DROP TABLE is fine. You can

[jira] Commented: (HIVE-1517) ability to select across a database

2011-02-08 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12992199#comment-12992199 ] Siying Dong commented on HIVE-1517: --- Looks like ANALYZE TABLE doesn't need the table.xxx

[jira] Commented: (HIVE-1517) ability to select across a database

2011-02-09 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12992823#comment-12992823 ] Siying Dong commented on HIVE-1517: --- https://reviews.apache.org/r/413/diff/ to better

[jira] Commented: (HIVE-1517) ability to select across a database

2011-02-09 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12992875#comment-12992875 ] Siying Dong commented on HIVE-1517: --- Looks like we have some trouble with printing token

[jira] Updated: (HIVE-1517) ability to select across a database

2011-02-10 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1517: -- Status: Patch Available (was: Open) ability to select across a database

[jira] Updated: (HIVE-1517) ability to select across a database

2011-02-10 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1517: -- Attachment: HIVE-1517.4.patch Update from HIVE.1517.3.patch: Separate DB name and table name

[jira] Created: (HIVE-1991) Hive Shell to output number of mappers and number of reducers

2011-02-14 Thread Siying Dong (JIRA)
Components: CLI Reporter: Siying Dong Assignee: Siying Dong Priority: Trivial Number of mappers and number of reducers are nice information to be outputted for users to know. -- This message is automatically generated by JIRA. - For more information on JIRA

[jira] Updated: (HIVE-1991) Hive Shell to output number of mappers and number of reducers

2011-02-14 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1991: -- Attachment: (was: HIVE-1991.1.patch) Hive Shell to output number of mappers and number of reducers

[jira] Updated: (HIVE-1991) Hive Shell to output number of mappers and number of reducers

2011-02-14 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1991: -- Status: Patch Available (was: Open) Hive Shell to output number of mappers and number of reducers

[jira] Updated: (HIVE-1991) Hive Shell to output number of mappers and number of reducers

2011-02-14 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1991: -- Attachment: HIVE-1991.2.patch Thanks Ning for the comment. Changed getMapTaskReports

[jira] Commented: (HIVE-1517) ability to select across a database

2011-02-15 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12995182#comment-12995182 ] Siying Dong commented on HIVE-1517: --- Namit, Driver.java just lock db of the table when

[jira] Commented: (HIVE-1517) ability to select across a database

2011-02-16 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12995440#comment-12995440 ] Siying Dong commented on HIVE-1517: --- I applied that patch to a clean directory and I am

[jira] Work stopped: (HIVE-1517) ability to select across a database

2011-02-16 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-1517 stopped by Siying Dong. ability to select across a database --- Key: HIVE-1517 URL

[jira] Work started: (HIVE-1517) ability to select across a database

2011-02-16 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-1517 started by Siying Dong. ability to select across a database --- Key: HIVE-1517 URL

[jira] Updated: (HIVE-1517) ability to select across a database

2011-02-16 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1517: -- Attachment: HIVE-1517.5.patch fix test outputs of two new added tests after rebasing. ability

[jira] Updated: (HIVE-1517) ability to select across a database

2011-02-16 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1517: -- Status: Patch Available (was: Open) not huge difference though. I fixed two test outputs

[jira] Updated: (HIVE-1517) ability to select across a database

2011-02-16 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1517: -- Attachment: HIVE-1517.5.patch ability to select across a database

[jira] Updated: (HIVE-1517) ability to select across a database

2011-02-16 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1517: -- Attachment: HIVE-1517.6.patch This patch fixed a couple of test outputs for TestContriCliTest

[jira] Updated: (HIVE-1517) ability to select across a database

2011-02-18 Thread Siying Dong (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1517: -- Attachment: HIVE-1517.7.patch I fixed multiple tests. Two tests always fail: TestHBaseCliDriver

  1   2   3   4   >