> On March 8, 2016, 3:36 a.m., Szehon Ho wrote:
> > Thanks for fixing those earlier items.
> >
> > I am still worried that getOperationStatus will be called by another thrift
> > thread at any time. Don't we need to synchronize/protect all the fields
> > that it accesses, like the OperationStatus, TaskStatus, etc, and all their
> > fields?
> >
> > But now one problem right is that driver will nullify the plan after it
> > runs, and thus there will be periods of time when you can see the task and
> > then afterwards you will get nothing. It seems like it will not be great
> > behavior.
> >
> > Again, combining with QueryDisplay (enhancing Task class to hook up a
> > TaskDisplay immediately on QueryDisplay, everytime Task.setXXX is called)
> > will solve these issues. It can be worth considering, as I still believe
> > the logic is similar and the problems (the two I mentioned above) are the
> > same. Thanks.
>
> Rajat Khandelwal wrote:
> I'm working on the TaskDisplay idea, but there will *not* be a period of
> time when you stop seeing the task after seeing it once. I'm caching the root
> tasks in the driver. So when plan is nulified, root tasks are picked from the
> plan and cached with the driver.
Hi
As per your suggestion, I've enhanced the TaskInfo class (it's TaskDisplay
now). And you were right, it does give extra info in the operation status :).
Please look at the changes and review.
Getting the following in test cases:
task statuses:
[{"taskId":"Stage-0","taskState":"INITIALIZED","taskType":"FETCH","name":"FETCH"},{"beginTime":1457533768220,"endTime":1457533771077,"taskId":"Stage-1","taskState":"FINISHED","taskType":"MAPRED","name":"MAPRED","elapsedTime":2857}]
Which seems to be good enough.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44453/#review122382
-----------------------------------------------------------
On March 9, 2016, 8:33 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44453/
> -----------------------------------------------------------
>
> (Updated March 9, 2016, 8:33 p.m.)
>
>
> Review request for hive, Amareshwari Sriramadasu and Puneet Gupta.
>
>
> Bugs: HIVE-4570
> https://issues.apache.org/jira/browse/HIVE-4570
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Currently in Hive Server2, when the query is still executing only the status
> is set as STILL_EXECUTING.
>
> This issue is to give more information to the user such as progress and
> running job handles, if possible.
>
>
> Diffs
> -----
>
>
> itests/hive-unit/src/test/java/org/apache/hive/service/cli/TestEmbeddedThriftBinaryCLIService.java
> de66d9efb1cace9d32174e3020920d5e4002dc85
>
> itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
> 9765b9dbe1ae097c108437c85b11b7bcee8f49da
> ql/src/java/org/apache/hadoop/hive/ql/Driver.java
> b50c5a2c4e396975f3323b16426445f869ff7b89
> ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java
> c87c825f95d2ff49bc62f892eb1cea5abf3c65c6
> ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java
> 4933b34710afebde923b548572ffd7f98b7eb728
> ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java
> e199e5ee65404d2e9a38d20fa9d2ff72754962e9
> ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java
> ce020a52721b439c4a9a81ee75e9829f2da00221
> ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java
> 5bc3d9e732330af7890d7379334ed42be116ccbe
> ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistory.java
> 45cd533270a550ae07510fe3974ddceaed30af51
> service-rpc/if/TCLIService.thrift 0aa9d13464bf284a3048092372299efb8e1d6bcc
>
> service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TGetOperationStatusResp.java
> a7a8ebc5c663961e6f4ebdb9df78fa3d53e31d98
> service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon
> c51368947ffe5873c727fc7e74aa25e748707370
> service/src/java/org/apache/hive/service/cli/OperationStatus.java
> e45b828193daf46e26c1587f7e0b9ea1a43f4930
> service/src/java/org/apache/hive/service/cli/operation/Operation.java
> 22f725c948494865e176a8d18543caf462a77c19
> service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
> 100dc6a5ceb10bff1d6591dd4b92062f4d787b98
> service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java
> 8dff26467260c365d5869d5b3e3bae15c3acd92d
>
> service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java
> 5f01165d0dfd131a2599b90a8e1c4d4970650b7a
> service/src/test/org/apache/hive/service/cli/CLIServiceTest.java
> e78181a15993d99f1cab5a061c08bb21823d2171
>
> Diff: https://reviews.apache.org/r/44453/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
