I'm glad you checked this with an MR run. That help clarify the issue, ruling out the browser compatibility issue. As Tez is an application framework, it is not responsible for returning results to the browser. Using Hive CLI, it is the Hive CLI that is responsible for displaying the results of the query. For HiveServer2, it is HiveServer2's responsibility of displaying the results. It is true that the yarn and tez configurations for Hive CLI and HiveServer2 can be different, but it is not the responsibility of those settings to change how Tez returns results, as it is Hive CLI and HiveServer2 that returns results.
Do you have access to the support logs or to a support team that can retrieve the relevant parts of those logs that will help to investigate this further? The relevant Tez logs you have provided are helpful, but only to show that Tez is working correctly. Without having access to the the HiveServer2 or thirdeye-services logs, I'm afraid it will be very difficult to debug this issue further. On Mon, Mar 19, 2018 at 11:22 AM, Jyotsna G <[email protected]> wrote: > Thank you for your response Jonathan. > > When I change the execution engine to MR, results are fetched perfectly > for the rest query, it's only when I change the execution engine to tez > that I see the issue occurring. > > To rephrase my question, are there any further sessions specific > configurations for tez that I might have missed? Any yarn related > configurations for tez that I might have to look for? > > > > On Mon, Mar 19, 2018, 21:37 Jonathan Eagles <[email protected]> wrote: > >> Thanks for reaching out to us, Jyotsna. >> >> Let me make sure I understand what your setup and the issue you are >> experiencing. It sounds like the setup looks like this. >> >> Browser -> Third party service "thirdeye-services" -> HiveServer2 -> Tez >> >> Since you are showing logs and screenshots of the successful Tez Job. My >> initial thought is that HiveServer2 was able to successfully and correctly >> get the results from Tez. That would make me believe that either the >> connection between the browser to the third party service >> "thirdeye-services" isn't working (perhaps browser compatibility issue, >> security issue, or service issue) or the connection between the third party >> service "thirdeye-services" to HiveServer2 issue (issue retrieving results >> from HiveServer2). >> >> Since our expertise is only with the Tez portion of the setup and that >> the Tez portion of the setup seems to be working, the Tez community won't >> be the best at addressing this issue. My recommendation would be try the >> query in a different browser as a simple test (Chrome, Firefox) checking >> with third party "thirdeye-services" browser compatibility. If you have >> access to the thirdparty-services logs or logs to the HiveServer2 logs >> those will be helpful in debugging. When reaching out to the >> thirdeye-services support or HiveServer2 please reference these logs to get >> the best support from the community. The hive user list is >> [email protected] and I have don't have a support contact for the >> thirdparty software thirdeye-services that seems to be part of this setup. >> >> Thanks again for reaching to the Tez Community. >> >> Regards, >> jeagles >> >> On Mon, Mar 19, 2018 at 6:47 AM, Jyotsna G <[email protected]> wrote: >> >>> Hi, >>> We are making a rest URI request which hits Hive server 2 and translates >>> in to a hive query on TEZ. >>> >>> Sample uri: >>> slc00yeb.us.oracle.com:8989/thirdeye-services/v1.0/OSCS/ >>> query/OSCSRecords.json?COMP_FAMILY=FMW&COMP_NAME=WLS& >>> COMPTYPE1_FIRST=RC1&COMPTYPE1_SECOND=RC2&COMPTYPE2_FIRST= >>> manifest.publish.label.WLS_GENERIC:carb-manifest:12.2.1. >>> 3.0-170722.1880245:pom&COMPTYPE2_SECOND=manifest. >>> publish.label.WLS_GENERIC:carb-manifest:12.2.1.3.0- >>> 170726.1880744:pom&RELEASE_FIRST=12.2.1.3.0&RELEASE_ >>> SECOND=12.2.1.3.0&COMPTYPE=MANIFEST >>> >>> Hadoop version : 2.4.0 >>> Hive:1.2.2 >>> Tez : 5.4.0 >>> >>> All my queries run fine in HIVE CLI , also via standalone program >>> connecting to hive via JDBC ODBC connection and returns hive query results, >>> but only when I hit the rest URL, results are not returned. >>> >>> I have enabled the below settings for tez >>> >>> tez.session.am.dag.submit.timeout.secs 90 >>> >>> tez.am.mode.session : true >>> tez.am.container.reuse.enable: true >>> tez.allow.disabled.timeline-domains : true >>> tez.yarn.ats.enabled : true >>> tez.use.cluster.hadoop-libs : true >>> >>> The url on the browser keeps spinning without the json response, even >>> after the tez job on yarn cluster has finished successfully. Attached Tez >>> job diagnosis screenshot. >>> >>> From the dag log : >>> >>> 2018-03-19 11:34:36,367 INFO [AsyncDispatcher event handler] >>> impl.DAGImpl: dag_1521448362533_0019_1 transitioned from RUNNING to >>> SUCCEEDED >>> 2018-03-19 11:34:36,368 INFO [AsyncDispatcher event handler] >>> app.DAGAppMaster: DAG completed, dagId=dag_1521448362533_0019_1, >>> dagState=SUCCEEDED >>> >>> {"entity":"dag_1521448362533_0019_1","entitytype":"TEZ_DAG_ >>> ID","events":[{"ts":1521459276335,"eventtype":" >>> DAG_FINISHED"}],"otherinfo":{"startTime":1521459269619," >>> endTime":1521459276335,"timeTaken":6716,"status":" >>> SUCCEEDED","diagnostics":"","counters":{"counterGroups":[{" >>> counterGroupName":"org.apache.tez.common.counters.DAGCounter"," >>> counterGroupDisplayName":"org.apache.tez.common.counters. >>> DAGCounter","counters":[{"counterName":"NUM_SUCCEEDED_ >>> TASKS","counterDisplayName":"NUM_SUCCEEDED_TASKS","counterValue":2},{" >>> counterName":"TOTAL_LAUNCHED_TASKS","counterDisplayName":" >>> TOTAL_LAUNCHED_TASKS","counterValue":2},{"counterName":"DATA_LOCAL_ >>> TASKS","counterDisplayName":"DATA_LOCAL_TASKS","counterValue":1}]},{" >>> counterGroupName":"org.apache.tez.common.counters.FileSystemCounter","counterGroupDisplayName":"File >>> System Counters","counters":[{"counterName":"FILE_BYTES_READ" >>> ,"counterDisplayName":"FILE_BYTES_READ","counterValue": >>> 41993},{"counterName":"FILE_BYTES_WRITTEN","counterDisplayName":"FILE_ >>> BYTES_WRITTEN","counterValue":83954},{"counterName":"FILE_ >>> READ_OPS","counterDisplayName":"FILE_READ_OPS","counterValue":0},{" >>> counterName":"FILE_LARGE_READ_OPS","counterDisplayName":" >>> FILE_LARGE_READ_OPS","counterValue":0},{"counterName":"FILE_WRITE_OPS", >>> "counterDisplayName":"FILE_WRITE_OPS","counterValue":0},{ >>> "counterName":"HDFS_BYTES_READ","counterDisplayName":"HDFS_BYTES_READ"," >>> counterValue":59231},{"counterName":"HDFS_BYTES_ >>> WRITTEN","counterDisplayName":"HDFS_BYTES_WRITTEN"," >>> counterValue":39631},{"counterName":"HDFS_READ_OPS"," >>> counterDisplayName":"HDFS_READ_OPS","counterValue":5},{" >>> counterName":"HDFS_LARGE_READ_OPS","counterDisplayName":" >>> HDFS_LARGE_READ_OPS","counterValue":0},{"counterName":"HDFS_WRITE_OPS", >>> "counterDisplayName":"HDFS_WRITE_OPS","counterValue":2}]} >>> ,{"counterGroupName":"org.apache.tez.common.counters.TaskCounter"," >>> counterGroupDisplayName":"org.apache.tez.common.counters. >>> TaskCounter","counters":[{"counterName":"REDUCE_INPUT_ >>> GROUPS","counterDisplayName":"REDUCE_INPUT_GROUPS"," >>> counterValue":190},{"counterName":"REDUCE_INPUT_ >>> RECORDS","counterDisplayName":"REDUCE_INPUT_RECORDS"," >>> counterValue":191},{"counterName":"COMBINE_INPUT_ >>> RECORDS","counterDisplayName":"COMBINE_INPUT_RECORDS"," >>> counterValue":0},{"counterName":"SPILLED_RECORDS","counterDisplayName":" >>> SPILLED_RECORDS","counterValue":382},{"counterName":"NUM_SHUFFLED_ >>> INPUTS","counterDisplayName":"NUM_SHUFFLED_INPUTS","counterValue":1},{" >>> counterName":"NUM_SKIPPED_INPUTS","counterDisplayName":" >>> NUM_SKIPPED_INPUTS","counterValue":0},{"counterName":"NUM_FAILED_ >>> SHUFFLE_INPUTS","counterDisplayName":"NUM_FAILED_SHUFFLE_INPUTS"," >>> counterValue":0},{"counterName":"MERGED_MAP_ >>> OUTPUTS","counterDisplayName":"MERGED_MAP_OUTPUTS","counterValue":1},{" >>> counterName":"GC_TIME_MILLIS","counterDisplayName":"GC_TIME_ >>> MILLIS","counterValue":84},{"counterName":"CPU_MILLISECONDS"," >>> counterDisplayName":"CPU_MILLISECONDS","counterValue": >>> 5520},{"counterName":"PHYSICAL_MEMORY_BYTES"," >>> counterDisplayName":"PHYSICAL_MEMORY_BYTES","counterValue": >>> 661254144},{"counterName":"VIRTUAL_MEMORY_BYTES"," >>> counterDisplayName":"VIRTUAL_MEMORY_BYTES","counterValue": >>> 4294270976},{"counterName":"COMMITTED_HEAP_BYTES","counterDisplayName":" >>> COMMITTED_HEAP_BYTES","counterValue":402653184},{" >>> counterName":"INPUT_RECORDS_PROCESSED","counterDisplayName":"INPUT_ >>> RECORDS_PROCESSED","counterValue":8445},{"counterName":"OUTPUT_RECORDS", >>> "counterDisplayName":"OUTPUT_RECORDS","counterValue":191},{ >>> "counterName":"OUTPUT_BYTES","counterDisplayName":"OUTPUT_ >>> BYTES","counterValue":41350},{"counterName":"OUTPUT_BYTES_ >>> WITH_OVERHEAD","counterDisplayName":"OUTPUT_BYTES_WITH_OVERHEAD"," >>> counterValue":41957},{"counterName":"OUTPUT_BYTES_ >>> PHYSICAL","counterDisplayName":"OUTPUT_BYTES_PHYSICAL"," >>> counterValue":41961},{"counterName":"ADDITIONAL_SPILLS_BYTES_WRITTEN"," >>> counterDisplayName":"ADDITIONAL_SPILLS_BYTES_ >>> WRITTEN","counterValue":41961},{"counterName":"ADDITIONAL_ >>> SPILLS_BYTES_READ","counterDisplayName":"ADDITIONAL_SPILLS_BYTES_READ", >>> "counterValue":41961},{"counterName":"ADDITIONAL_SPILL_COUNT"," >>> counterDisplayName":"ADDITIONAL_SPILL_COUNT","counterValue":0},{" >>> counterName":"SHUFFLE_BYTES","counterDisplayName":"SHUFFLE_ >>> BYTES","counterValue":41961},{"counterName":"SHUFFLE_BYTES_ >>> DECOMPRESSED","counterDisplayName":"SHUFFLE_BYTES_DECOMPRESSED"," >>> counterValue":41957},{"counterName":"SHUFFLE_BYTES_ >>> TO_MEM","counterDisplayName":"SHUFFLE_BYTES_TO_MEM"," >>> counterValue":41961},{"counterName":"SHUFFLE_BYTES_ >>> TO_DISK","counterDisplayName":"SHUFFLE_BYTES_TO_DISK"," >>> counterValue":0},{"counterName":"SHUFFLE_BYTES_DISK_DIRECT"," >>> counterDisplayName":"SHUFFLE_BYTES_DISK_DIRECT","counterValue":0},{" >>> counterName":"NUM_MEM_TO_DISK_MERGES","counterDisplayName":" >>> NUM_MEM_TO_DISK_MERGES","counterValue":0},{"counterName":"NUM_DISK_TO_ >>> DISK_MERGES","counterDisplayName":"NUM_DISK_ >>> TO_DISK_MERGES","counterValue":0}]},{"counterGroupName":"HIVE"," >>> counterGroupDisplayName":"HIVE","counters":[{" >>> counterName":"CREATED_FILES","counterDisplayName":"CREATED_ >>> FILES","counterValue":1},{"counterName":"DESERIALIZE_ >>> ERRORS","counterDisplayName":"DESERIALIZE_ERRORS","counterValue":0},{" >>> counterName":"RECORDS_IN_Map_1","counterDisplayName":"RECORDS_I >>> >>> >>> But the json response isnt coming. What am I missing here? >>> >>> Thanks, >>> Jyotsna >>> >>> >>> >>
