[jira] Commented: (HIVE-1695) MapJoin followed by ReduceSink should be done as single MapReduce Job

2010-10-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12918607#action_12918607 ] Joydeep Sen Sarma commented on HIVE-1695: - i think there's already a jira

[jira] Commented: (HIVE-1675) SAXParseException on plan.xml during local mode.

2010-10-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12918609#action_12918609 ] Joydeep Sen Sarma commented on HIVE-1675: - what version of hadoop is this happening

[jira] Commented: (HIVE-1620) Patch to write directly to S3 from Hive

2010-10-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12918611#action_12918611 ] Joydeep Sen Sarma commented on HIVE-1620: - if we are overwriting a table ('insert

[jira] Updated: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-10-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1570: Attachment: 1570.4.patch added fix for hive-1520 - don't reset HADOOP_HEAPSIZE unless

[jira] Updated: (HIVE-1473) plan file should have a high replication factor

2010-10-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1473: Attachment: 1473.1.patch plan file should have a high replication factor

[jira] Updated: (HIVE-1520) hive.mapred.local.mem should only be used in case of local mode job submissions

2010-10-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1520: Attachment: 1520.1.patch hive.mapred.local.mem should only be used in case of local mode

[jira] Updated: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-10-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1570: Attachment: 1570.5.patch also adding trivial patch for HIVE-1473. filed separate patches

[jira] Updated: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-10-05 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1570: Attachment: 1570.3.patch added a console output for local mapred jobs containing location

[jira] Resolved: (HIVE-1685) scriptfile1.1 in minimr faling intermittently

2010-10-05 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma resolved HIVE-1685. - Resolution: Duplicate the test and the output are not in sync. it should fail everytime

[jira] Updated: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-10-04 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1570: Attachment: 1570.1.patch before running a map-reduce job in local mode we: 1. set a new

[jira] Updated: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-10-04 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1570: Attachment: 1570.2.patch working patch. no need for new test. had to modify some other

[jira] Updated: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-10-04 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1570: Status: Patch Available (was: Open) referencing an added file by it's name in a transform

[jira] Commented: (HIVE-1651) ScriptOperator should not forward any output to downstream operators if an exception is happened

2010-09-17 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12910786#action_12910786 ] Joydeep Sen Sarma commented on HIVE-1651: - if a hadoop task is being failed - how

[jira] Commented: (HIVE-1651) ScriptOperator should not forward any output to downstream operators if an exception is happened

2010-09-17 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12910837#action_12910837 ] Joydeep Sen Sarma commented on HIVE-1651: - yeah - but then the directory itself

[jira] Commented: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-09-15 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12909950#action_12909950 ] Joydeep Sen Sarma commented on HIVE-1570: - sure. confused - because the tests were

[jira] Updated: (HIVE-1580) cleanup ExecDriver.progress

2010-09-02 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1580: Attachment: hive-1580.1.patch cleanup multiple calls to getCounters (which turns out

[jira] Updated: (HIVE-1601) Hadoop 0.17 ant test broken by HIVE-1523

2010-08-27 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1601: Status: Patch Available (was: Open) Hadoop 0.17 ant test broken by HIVE-1523

[jira] Updated: (HIVE-1601) Hadoop 0.17 ant test broken by HIVE-1523

2010-08-27 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1601: Attachment: 1601.1.patch ant-contrib-1.0b3.jar - fix the jsp include

[jira] Commented: (HIVE-1602) List Partitioning

2010-08-27 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12903606#action_12903606 ] Joydeep Sen Sarma commented on HIVE-1602: - hmmm - not sure i understand. how can we

[jira] Commented: (HIVE-1602) List Partitioning

2010-08-27 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12903630#action_12903630 ] Joydeep Sen Sarma commented on HIVE-1602: - yikes. how is this queried afterwards

[jira] Commented: (HIVE-1602) List Partitioning

2010-08-27 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12903655#action_12903655 ] Joydeep Sen Sarma commented on HIVE-1602: - how will this be made transparent from

[jira] Commented: (HIVE-1602) List Partitioning

2010-08-27 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12903672#action_12903672 ] Joydeep Sen Sarma commented on HIVE-1602: - combining small partitions into one

[jira] Commented: (HIVE-1487) parallelize test query runs

2010-08-26 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902958#action_12902958 ] Joydeep Sen Sarma commented on HIVE-1487: - can people with experience running java

[jira] Updated: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-25 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1523: Attachment: hive-1523.4.patch - added exclude tests - minimr tests are excluded from

[jira] Commented: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-25 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902560#action_12902560 ] Joydeep Sen Sarma commented on HIVE-1523: - there's already a jira on running tests

[jira] Commented: (HIVE-1583) Hive should not override Hadoop specific system properties

2010-08-24 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902149#action_12902149 ] Joydeep Sen Sarma commented on HIVE-1583: - +1 on HADOOP_CLASSPATH. i am not sure

[jira] Commented: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-23 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901687#action_12901687 ] Joydeep Sen Sarma commented on HIVE-1523: - can someone review/commit this? i don't

[jira] Created: (HIVE-1579) showJobFailDebugInfo fails job if tasktracker does not respond

2010-08-21 Thread Joydeep Sen Sarma (JIRA)
Components: Query Processor Reporter: Joydeep Sen Sarma Assignee: Paul Yang here's the stack trace: java.lang.RuntimeException: Error while reading from task log url at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130

[jira] Commented: (HIVE-1578) Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures

2010-08-21 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900991#action_12900991 ] Joydeep Sen Sarma commented on HIVE-1578: - looks like the CHANGES.txt message

[jira] Created: (HIVE-1564) bucketizedhiveinputformat.q fails in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
bucketizedhiveinputformat.q fails in minimr mode Key: HIVE-1564 URL: https://issues.apache.org/jira/browse/HIVE-1564 Project: Hadoop Hive Issue Type: Bug Reporter: Joydeep Sen

[jira] Created: (HIVE-1565) archive.q fails in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
archive.q fails in minimr mode -- Key: HIVE-1565 URL: https://issues.apache.org/jira/browse/HIVE-1565 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Joydeep Sen

[jira] Updated: (HIVE-1562) CombineHiveInputFormat issues in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1562: Summary: CombineHiveInputFormat issues in minimr mode (was: sample10.q fails in minimr

[jira] Created: (HIVE-1566) diff_part_input_formats.q failure in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
diff_part_input_formats.q failure in minimr mode Key: HIVE-1566 URL: https://issues.apache.org/jira/browse/HIVE-1566 Project: Hadoop Hive Issue Type: Bug Reporter: Joydeep Sen

[jira] Commented: (HIVE-1564) bucketizedhiveinputformat.q fails in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900209#action_12900209 ] Joydeep Sen Sarma commented on HIVE-1564: - also groupby_bigdata.q shows same stack

[jira] Commented: (HIVE-1562) CombineHiveInputFormat issues in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900398#action_12900398 ] Joydeep Sen Sarma commented on HIVE-1562: - cool - thanks for the tip. looks like

[jira] Updated: (HIVE-1562) default hadoop version should be 0.20.2 or newer

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1562: Summary: default hadoop version should be 0.20.2 or newer (was: CombineHiveInputFormat

[jira] Created: (HIVE-1568) null_column.q fails in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
null_column.q fails in minimr mode -- Key: HIVE-1568 URL: https://issues.apache.org/jira/browse/HIVE-1568 Project: Hadoop Hive Issue Type: Bug Reporter: Joydeep Sen Sarma followup from hive-1523

[jira] Commented: (HIVE-1568) null_column.q fails in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900490#action_12900490 ] Joydeep Sen Sarma commented on HIVE-1568: - also ppd_multi_insert.q fails with same

[jira] Created: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Joydeep Sen Sarma Yongqiang tried this and it fails in local mode: add file ../data/scripts/dumpdata_script.py; select count(distinct subq.key) from (FROM src MAP src.key USING

RE: [DISCUSSION] Move to become a TLP

2010-08-19 Thread Joydeep Sen Sarma
+1 -Original Message- From: Carl Steinbach [mailto:c...@cloudera.com] Sent: Thursday, August 19, 2010 3:18 PM To: hive-dev@hadoop.apache.org Subject: Re: [DISCUSSION] Move to become a TLP +1 On Thu, Aug 19, 2010 at 3:15 PM, Ning Zhang nzh...@facebook.com wrote: +1 as well. On Aug

[jira] Commented: (HIVE-1570) referencing an added file by it's name in a transform script does not work in hive local mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900501#action_12900501 ] Joydeep Sen Sarma commented on HIVE-1570: - hmmm - how come scriptfile1.q works

[jira] Commented: (HIVE-1569) groupby_bigdata.q fails in minimr mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900521#action_12900521 ] Joydeep Sen Sarma commented on HIVE-1569: - can u look at scriptfile1.q it works

[jira] Created: (HIVE-1572) skewjoin.q output in minimr differs from local mode

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
Processor Reporter: Joydeep Sen Sarma checked in results: POSTHOOK: query: SELECT sum(hash(src1.key)), sum(hash(src1.val)), sum(hash(src2.key)) FROM T1 src1 JOIN T2 src2 ON src1.key+1 = src2.key JOIN T2 src3 ON src2.key = src3.key 370 11003 377 in minimr mode: POSTHOOK: query: SELECT

[jira] Created: (HIVE-1574) symlink_text_input_format.q needs fixes for minimr

2010-08-19 Thread Joydeep Sen Sarma (JIRA)
symlink_text_input_format.q needs fixes for minimr -- Key: HIVE-1574 URL: https://issues.apache.org/jira/browse/HIVE-1574 Project: Hadoop Hive Issue Type: Bug Reporter: Joydeep Sen

[jira] Updated: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-18 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1523: Attachment: hive-1523.1.patch fixed minimr test mode. enabled a couple of queries

[jira] Updated: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-18 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1523: Status: Patch Available (was: Open) Assignee: Joydeep Sen Sarma ql tests no longer

[jira] Created: (HIVE-1560) binaryoutputformat.q failure in minimr mode

2010-08-18 Thread Joydeep Sen Sarma (JIRA)
Reporter: Joydeep Sen Sarma this is a followup to HIVE-1523. ant -Dclustermode=miniMR -Dtestcase=TestCliDriver -Dqfile=binary_output_format.q test fails in a significant manner. all the rows are flattened out into one row: ntimeException -I at org -I at sun -I at java -I at junit -I

[jira] Commented: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-18 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900048#action_12900048 ] Joydeep Sen Sarma commented on HIVE-1523: - i am running through the above qfiles

alarming hive test failures in minimr mode

2010-08-18 Thread Joydeep Sen Sarma
Hey Devs, Since fixing 1523 - I have been trying to run hive queries in minimr mode. I am alarmed by what I am seeing: - assertions firing deep inside hive in minimr mode - test results outright different from local mode results (and not because of limit or because of

[jira] Created: (HIVE-1561) smb_mapjoin_8.q returns different results in miniMr mode

2010-08-18 Thread Joydeep Sen Sarma (JIRA)
: Query Processor Reporter: Joydeep Sen Sarma follow on to HIVE-1523: ant -Dclustermode=miniMR -Dtestcase=TestCliDriver -Dqfile=smb_mapjoin_8.q test POSTHOOK: query: select /*+mapjoin(a)*/ * from smb_bucket4_1 a full outer join smb_bucket4_2 b on a.key = b.key official results: 4

[jira] Created: (HIVE-1562) sample10.q fails in minimr mode

2010-08-18 Thread Joydeep Sen Sarma (JIRA)
sample10.q fails in minimr mode --- Key: HIVE-1562 URL: https://issues.apache.org/jira/browse/HIVE-1562 Project: Hadoop Hive Issue Type: Bug Reporter: Joydeep Sen Sarma followup from HIVE-1523

[jira] Updated: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-18 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1523: Attachment: hive-1523.2.patch with modified list of minimr tests: + property name

[jira] Commented: (HIVE-1293) Concurreny Model for Hive

2010-08-17 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899320#action_12899320 ] Joydeep Sen Sarma commented on HIVE-1293: - a little bummed that locks need

[jira] Commented: (HIVE-1293) Concurreny Model for Hive

2010-08-17 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899347#action_12899347 ] Joydeep Sen Sarma commented on HIVE-1293: - also - i am missing something here

[jira] Commented: (HIVE-1293) Concurreny Model for Hive

2010-08-17 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899647#action_12899647 ] Joydeep Sen Sarma commented on HIVE-1293: - can u check the getLockObjects() routine

[jira] Commented: (HIVE-1540) Read-only, columnar data file for nested data structures

2010-08-16 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899199#action_12899199 ] Joydeep Sen Sarma commented on HIVE-1540: - are there a lot of use cases for nested

[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR

2010-08-13 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12898118#action_12898118 ] Joydeep Sen Sarma commented on HIVE-1530: - don't dis-allow hive.* options

RE: how jdbm is used in map join

2010-08-12 Thread Joydeep Sen Sarma
i believe each mapper makes a copy since it reads in the data to be loaded into the dbm. this needs to be optimized at some point (ideally we should be putting the dbm in distributed cache) From: Gang Luo [lgpub...@yahoo.com.cn] Sent: Tuesday, August 10,

RE: How HIVE manages a join

2010-08-12 Thread Joydeep Sen Sarma
i hate this message: 'THIS PAGE WAS MOVED TO HIVE XDOCS ! DO NOT EDIT!Join Syntax' why must edits to the wiki be banned if there are xdocs? hadoop has both. there will always be things that are not captured in xdocs. it's pretty sad to discourage free form edits by people who want to

RE: SAXParseException on local mode?

2010-08-12 Thread Joydeep Sen Sarma
Hi Bennie - - i changed the default for this option to be false in a recent commit. - what version of java/hadoop are u running? it seems that the job.xml can get parsed on ur hadoop nodes, but not the client machine - so there must be some difference in the xml parsing library (that's part of

[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR

2010-08-12 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897993#action_12897993 ] Joydeep Sen Sarma commented on HIVE-1530: - removing the .xml files makes sense

[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR

2010-08-12 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12898080#action_12898080 ] Joydeep Sen Sarma commented on HIVE-1530: - ok - that makes sense. leave a hive

[jira] Commented: (HIVE-1524) parallel execution failed if mapred.job.name is set

2010-08-10 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897037#action_12897037 ] Joydeep Sen Sarma commented on HIVE-1524: - will commit once tests pass. parallel

[jira] Updated: (HIVE-1524) parallel execution failed if mapred.job.name is set

2010-08-10 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1524: Affects Version/s: 0.5.0 (was: 0.7.0) parallel execution failed

[jira] Updated: (HIVE-1524) parallel execution failed if mapred.job.name is set

2010-08-10 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1524: Status: Resolved (was: Patch Available) Resolution: Fixed committed - thanks Ning

[jira] Created: (HIVE-1520) hive.mapred.local.mem should only be used in case of local mode job submissions

2010-08-09 Thread Joydeep Sen Sarma (JIRA)
Issue Type: Bug Reporter: Joydeep Sen Sarma Currently - whenever we submit a map-reduce job via a child jvm process, hive sets HADOOP_HEAPSIZE to hive.mapred.local.mem (thereby limiting the max heap memory of the child jvm). the assumption being that we are submitting a job

[jira] Created: (HIVE-1521) compiling/testing against custom hadoop tree is broken

2010-08-09 Thread Joydeep Sen Sarma (JIRA)
: Build Infrastructure Reporter: Joydeep Sen Sarma see: http://wiki.apache.org/hadoop/Hive/DeveloperGuide#Advanced_Mode compiling with specific value of hadoop.root no longer works because of the shims stuff. we should deprecate/fix this. it is still _very_ desirably to be able to test

[jira] Updated: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-09 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1523: Component/s: Query Processor (was: Build Infrastructure) ql tests

[jira] Created: (HIVE-1523) ql tests no longer work in miniMR mode

2010-08-09 Thread Joydeep Sen Sarma (JIRA)
Reporter: Joydeep Sen Sarma as per title. here's the first exception i see: 2010-08-09 18:05:11,259 ERROR hive.log (MetaStoreUtils.java:logAndThrowMetaException(743)) - Got exception: java.io.FileNotFoun\ dException File file:/build/ql/test/data/warehouse/dest_j1 does not exist. 2010-08-09 18:05

RE: Hive local mode on by default for 0.6.0

2010-08-09 Thread Joydeep Sen Sarma
We enabled a feature called 'auto-local mode' (hive-1408). The query processor looks at the size of the input and decides dynamically whether local mode execution can be done. The determination is done on a per job level for a multi-job query. We enabled it by default in trunk so it can get

RE: Hive local mode on by default for 0.6.0

2010-08-09 Thread Joydeep Sen Sarma
..) ___ From: Edward Capriolo [edlinuxg...@gmail.com] Sent: Monday, August 09, 2010 8:04 PM To: hive-dev@hadoop.apache.org Subject: Re: Hive local mode on by default for 0.6.0 On Mon, Aug 9, 2010 at 9:28 PM, Joydeep Sen Sarma jssa...@facebook.com wrote: We enabled a feature called

[jira] Commented: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability

2010-08-07 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12896231#action_12896231 ] Joydeep Sen Sarma commented on HIVE-1513: - HADOOP_HEAPSIZE: hive-config.sh only

[jira] Updated: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability

2010-08-07 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1513: Attachment: hive-1513.3.patch Ning - u are right - i didn't notice the HEAP setting

[jira] Updated: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability

2010-08-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1513: Attachment: 1513.1.patch simple change to let hive started script include conf/hive-env.sh

[jira] Updated: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability

2010-08-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1513: Status: Patch Available (was: Open) Assignee: Joydeep Sen Sarma hive starter

[jira] Created: (HIVE-1516) optimize split sizes automatically taking into account amount of nature of map tasks

2010-08-06 Thread Joydeep Sen Sarma (JIRA)
: Hadoop Hive Issue Type: Improvement Components: Query Processor Reporter: Joydeep Sen Sarma two immediate cases come to mind: - pure filter job (ie. no map-side sort required) - full aggregate computations only (like count(1)). in these cases - the amount of data

[jira] Updated: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability

2010-08-06 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1513: Attachment: 1513.2.patch forgot to add one file hive starter scripts should load admin

[jira] Updated: (HIVE-1509) Monitor the working set of the number of files

2010-08-05 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1509: Status: Resolved (was: Patch Available) Fix Version/s: 0.7.0 Resolution

[jira] Created: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability

2010-08-05 Thread Joydeep Sen Sarma (JIRA)
Issue Type: Improvement Components: CLI Reporter: Joydeep Sen Sarma it's difficult to add environment variables to Hive starter scripts except by modifying the scripts directly. this is undesirable (since they are source code). Hive starter scripts should load

[jira] Commented: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability

2010-08-05 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895959#action_12895959 ] Joydeep Sen Sarma commented on HIVE-1513: - yes - it's possible. however a lot

[jira] Commented: (HIVE-1509) Monitor the working set of the number of files

2010-08-04 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895384#action_12895384 ] Joydeep Sen Sarma commented on HIVE-1509: - let me know once the tests pass 0.20

[jira] Commented: (HIVE-1509) Monitor the working set of the number of files

2010-08-04 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895411#action_12895411 ] Joydeep Sen Sarma commented on HIVE-1509: - ok - i will run tests on 20 and commit

[jira] Commented: (HIVE-1509) Monitor the working set of the number of files

2010-08-04 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895472#action_12895472 ] Joydeep Sen Sarma commented on HIVE-1509: - the test result for dyn_part3.q

[jira] Commented: (HIVE-1509) Monitor the working set of the number of files

2010-08-04 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895534#action_12895534 ] Joydeep Sen Sarma commented on HIVE-1509: - strange - let me retry. can u check

[jira] Resolved: (HIVE-1493) incorrect explanation when local mode not chosen automatically

2010-08-03 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma resolved HIVE-1493. - Assignee: Joydeep Sen Sarma Resolution: Fixed fixed via HIVE-1422 incorrect

[jira] Updated: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null

2010-08-02 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1422: Status: Patch Available (was: Reopened) skip counter update when RunningJob.getCounters

[jira] Created: (HIVE-1507) Supply DriverContext to Hooks

2010-08-02 Thread Joydeep Sen Sarma (JIRA)
Supply DriverContext to Hooks - Key: HIVE-1507 URL: https://issues.apache.org/jira/browse/HIVE-1507 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Joydeep Sen

[jira] Updated: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null

2010-08-02 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1422: Attachment: 1422.2.patch damn - this uncovered a bug in the tests and fixed an unncessary

[jira] Commented: (HIVE-1430) serializing/deserializing the query plan is useless and expensive

2010-07-31 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894253#action_12894253 ] Joydeep Sen Sarma commented on HIVE-1430: - hey - so i am a little puzzled

[jira] Commented: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null

2010-07-30 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894126#action_12894126 ] Joydeep Sen Sarma commented on HIVE-1422: - more hadoop goriness - i think John your

[jira] Updated: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null

2010-07-30 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1422: Attachment: 1422.2.patch - cleaned up ExecDriver a bit - removed some dead code, some

[jira] Commented: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null

2010-07-29 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893852#action_12893852 ] Joydeep Sen Sarma commented on HIVE-1422: - i looked at the hadoop source for 20

[jira] Updated: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

2010-07-28 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1408: Attachment: 1408.7.patch add option to let hive automatically run in local mode based

[jira] Commented: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

2010-07-28 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893088#action_12893088 ] Joydeep Sen Sarma commented on HIVE-1408: - #1 - we decide that i would try to take

[jira] Commented: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

2010-07-28 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893314#action_12893314 ] Joydeep Sen Sarma commented on HIVE-1408: - yeah - so the solution

[jira] Commented: (HIVE-417) Implement Indexing in Hive

2010-07-28 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893455#action_12893455 ] Joydeep Sen Sarma commented on HIVE-417: i am waiting for a commit on hive-1408

[jira] Commented: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

2010-07-28 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893462#action_12893462 ] Joydeep Sen Sarma commented on HIVE-1408: - Ning - anything else u need from me? i

[jira] Updated: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

2010-07-26 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1408: Attachment: (was: 1408.4.patch) add option to let hive automatically run in local mode

[jira] Updated: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

2010-07-26 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1408: Attachment: (was: 1408.3.patch) add option to let hive automatically run in local mode

[jira] Updated: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

2010-07-26 Thread Joydeep Sen Sarma (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1408: Attachment: (was: 1408.5.patch) add option to let hive automatically run in local mode

  1   2   3   4   >