[jira] Updated: (HIVE-547) NullPointerException in ExecDriver
[
https://issues.apache.org/jira/browse/HIVE-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Shao updated HIVE-547:
Attachment: HIVE-547.2.patch
This patch makes sure we don't output one line per second. The behavior is that
we will output at least one line per minute if no status change is detected.
NullPointerException in ExecDriver
--
Key: HIVE-547
URL: https://issues.apache.org/jira/browse/HIVE-547
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.4.0
Reporter: Zheng Shao
Assignee: Zheng Shao
Fix For: 0.4.0
Attachments: HIVE-547.1.patch, HIVE-547.2.patch
We saw a job failed with the following message in hive.log.
{noformat}
2009-06-06 22:50:55,275 ERROR exec.ExecDriver
(SessionState.java:printError(279)) - Ended Job = job_200905211352_145363
with exception 'java.lang.NullPointerException(null)'
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExecDriver.jobProgress(ExecDriver.java:193)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:395)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:307)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:213)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:176)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:273)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:166)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
{noformat}
The corresponding code is:
{code:java}
public RunningJob jobProgress(JobClient jc, RunningJob rj) throws
IOException {
String lastReport = ;
while (!rj.isComplete()) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
rj = jc.getJob(rj.getJobID());
String report = null;
193： report = map = + Math.round(rj.mapProgress() * 100) + %, reduce =
+ Math.round(rj.reduceProgress() * 100) + %;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HIVE-547) NullPointerException in ExecDriver
[
https://issues.apache.org/jira/browse/HIVE-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Shao reopened HIVE-547:
-
The current behavior is showing one log line per second. This may not be good
in case the job runs a long time.
NullPointerException in ExecDriver
--
Key: HIVE-547
URL: https://issues.apache.org/jira/browse/HIVE-547
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.4.0
Reporter: Zheng Shao
Assignee: Zheng Shao
Fix For: 0.4.0
Attachments: HIVE-547.1.patch, HIVE-547.2.patch
We saw a job failed with the following message in hive.log.
{noformat}
2009-06-06 22:50:55,275 ERROR exec.ExecDriver
(SessionState.java:printError(279)) - Ended Job = job_200905211352_145363
with exception 'java.lang.NullPointerException(null)'
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExecDriver.jobProgress(ExecDriver.java:193)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:395)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:307)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:213)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:176)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:273)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:166)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
{noformat}
The corresponding code is:
{code:java}
public RunningJob jobProgress(JobClient jc, RunningJob rj) throws
IOException {
String lastReport = ;
while (!rj.isComplete()) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
rj = jc.getJob(rj.getJobID());
String report = null;
193： report = map = + Math.round(rj.mapProgress() * 100) + %, reduce =
+ Math.round(rj.reduceProgress() * 100) + %;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-547) NullPointerException in ExecDriver
[
https://issues.apache.org/jira/browse/HIVE-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Shao updated HIVE-547:
Status: Patch Available (was: Reopened)
NullPointerException in ExecDriver
--
Key: HIVE-547
URL: https://issues.apache.org/jira/browse/HIVE-547
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.4.0
Reporter: Zheng Shao
Assignee: Zheng Shao
Fix For: 0.4.0
Attachments: HIVE-547.1.patch, HIVE-547.2.patch
We saw a job failed with the following message in hive.log.
{noformat}
2009-06-06 22:50:55,275 ERROR exec.ExecDriver
(SessionState.java:printError(279)) - Ended Job = job_200905211352_145363
with exception 'java.lang.NullPointerException(null)'
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExecDriver.jobProgress(ExecDriver.java:193)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:395)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:307)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:213)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:176)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:273)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:166)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
{noformat}
The corresponding code is:
{code:java}
public RunningJob jobProgress(JobClient jc, RunningJob rj) throws
IOException {
String lastReport = ;
while (!rj.isComplete()) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
rj = jc.getJob(rj.getJobID());
String report = null;
193： report = map = + Math.round(rj.mapProgress() * 100) + %, reduce =
+ Math.round(rj.reduceProgress() * 100) + %;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
JIRA_HIVE-547.2.patch_UNIT_TEST_SUCCEEDED
SUCCESS: BUILD AND UNIT TEST using PATCH HIVE-547.2.patch PASSED!!
[jira] Created: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.
create temporary function support not only udf, but also udaf, genericudf, etc. Key: HIVE-555 URL: https://issues.apache.org/jira/browse/HIVE-555 Project: Hadoop Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.4.0 Reporter: Min Zhou Assignee: Min Zhou Fix For: 0.4.0 Right now, command 'create temporary function' only support udf. we can also let user write their udaf, generic udf, and write generic udaf in the future. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-460: -- Attachment: hive-460-2009-06-10-2.patch A new patch against trunk code(hive-460-2009-06-10-2.patch) and it passed all TestCliDriver and TestParse in my local. Improve ColumnPruner to prune more aggressively and keep column information for input tables Key: HIVE-460 URL: https://issues.apache.org/jira/browse/HIVE-460 Project: Hadoop Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.4.0 Reporter: Zheng Shao Assignee: He Yongqiang Attachments: hive-460-2009-05-29-noaliasallowed.patch, hive-460-2009-06-04-2-195mapjoin.patch, hive-460-2009-06-04-testcli-passed.patch, hive-460-2009-06-06-2.patch, hive-460-2009-06-10-2.patch, HIVE-460.1.patch This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-555) create temporary function support not only udf, but also udaf, genericudf, etc.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated HIVE-555: -- Attachment: HIVE-555-1.patch patch w/o testcase create temporary function support not only udf, but also udaf, genericudf, etc. Key: HIVE-555 URL: https://issues.apache.org/jira/browse/HIVE-555 Project: Hadoop Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.4.0 Reporter: Min Zhou Assignee: Min Zhou Fix For: 0.4.0 Attachments: HIVE-555-1.patch Right now, command 'create temporary function' only support udf. we can also let user write their udaf, generic udf, and write generic udaf in the future. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-521) Move size, if, isnull, isnotnull to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated HIVE-521: -- Attachment: HIVE-521-all-v2.patch fixed issues commented by Zheng, UDFArgumentException and UDFArgumentLengthException added. Move size, if, isnull, isnotnull to GenericUDF -- Key: HIVE-521 URL: https://issues.apache.org/jira/browse/HIVE-521 Project: Hadoop Hive Issue Type: Improvement Affects Versions: 0.4.0 Reporter: Zheng Shao Assignee: Min Zhou Fix For: 0.4.0 Attachments: HIVE-521-all-v1.patch, HIVE-521-all-v2.patch, HIVE-521-IF-2.patch, HIVE-521-IF-3.patch, HIVE-521-IF-4.patch, HIVE-521-IF-5.patch, HIVE-521-IF.patch See HIVE-511 for an example of the move. size, if, isnull, isnotnull are all implemented with UDF but they are actually working on variable types of objects. We should move them to GenericUDF for better type handling. This also helps to clean up the hack in doing type matching/type conversion in UDF. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-521) Move size, if, isnull, isnotnull to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated HIVE-521: -- Attachment: HIVE-521-all-v3.patch catch UDFArgumentLengthException. Move size, if, isnull, isnotnull to GenericUDF -- Key: HIVE-521 URL: https://issues.apache.org/jira/browse/HIVE-521 Project: Hadoop Hive Issue Type: Improvement Affects Versions: 0.4.0 Reporter: Zheng Shao Assignee: Min Zhou Fix For: 0.4.0 Attachments: HIVE-521-all-v1.patch, HIVE-521-all-v2.patch, HIVE-521-all-v3.patch, HIVE-521-IF-2.patch, HIVE-521-IF-3.patch, HIVE-521-IF-4.patch, HIVE-521-IF-5.patch, HIVE-521-IF.patch See HIVE-511 for an example of the move. size, if, isnull, isnotnull are all implemented with UDF but they are actually working on variable types of objects. We should move them to GenericUDF for better type handling. This also helps to clean up the hack in doing type matching/type conversion in UDF. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-556) let hive support theta join
let hive support theta join
---
Key: HIVE-556
URL: https://issues.apache.org/jira/browse/HIVE-556
Project: Hadoop Hive
Issue Type: New Feature
Affects Versions: 0.4.0
Reporter: Min Zhou
Fix For: 0.4.0
Right now , hive only support equal joins . Somethings it's not enough, we
must consider implementing theta joins like
{code:sql}
SELECT
a.subid, a.id, t.url
FROM
tbl t JOIN aux_tbl a ON t.url rlike a.url_pattern
WHERE
t.dt='20090609'
AND a.dt='20090609';
{code}
any condition expression following 'ON' is appropriate.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
