[jira] [Updated] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-2849: - Description: The CPU performance overhead of the sort operation is tied to the JIT handling of the aliased byte buffers without an explicit alignment (i.e a byte[] buffer referenced as an IntBuffer). https://bugs.openjdk.java.net/browse/JDK-8026049 + https://bugs.openjdk.java.net/browse/JDK-8073093 Workaround that JIT limitation while keeping in mind the modern CPU micro-code optimizations around alignment. http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ was: The CPU performance overhead of the sort operation is tied to the JIT handling of the aliased byte buffers without an explicit alignment (i.e a byte[] buffer referenced as an IntBuffer). Workaround that JIT limitation while keeping in mind the modern CPU micro-code optimizations around alignment. http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Gopal V >Assignee: Gopal V > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-2849: Assignee: Gopal V > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Gopal V >Assignee: Gopal V > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-2849: - Affects Version/s: 0.8.1 > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Gopal V >Assignee: Gopal V > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903222#comment-14903222 ] Siddharth Seth commented on TEZ-2716: - +1. Looks good. > DefaultSorter.isRleNeeded not thread safe > - > > Key: TEZ-2716 > URL: https://issues.apache.org/jira/browse/TEZ-2716 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Siddharth Seth > Attachments: TEZ-2716.1.patch, TEZ-2716.2.patch > > > TEZ-1997. > Should be targeted at the same set of versions that TEZ-1997 goes into. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2843) Tez UI: Show error if in progress fails due to AM not reachable
[
https://issues.apache.org/jira/browse/TEZ-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902061#comment-14902061
]
Sreenath Somarajapuram commented on TEZ-2843:
-
1. Wouldn't it be better to stop polling on failure?
2. Trailing space at the end of "Failed To get in progress status. Manually
refresh to update status",
3. Is this change required
{code}
- amInfoUpdateService = App.Helpers.Pollster.create({
+ amInfoUpdateService = App.Helpers.Pollster.createWithMixins({
onPoll: function() {
that.updateAMDagInfo();
that.updateAMVerticesInfo();
-}
+},
{code}
> Tez UI: Show error if in progress fails due to AM not reachable
> ---
>
> Key: TEZ-2843
> URL: https://issues.apache.org/jira/browse/TEZ-2843
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2843.1.patch
>
>
> currently the in progress update can fail when the AM finishes, but no
> feedback is given to the user regarding the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TEZ-2843) Tez UI: Show error if in progress fails due to AM not reachable
[ https://issues.apache.org/jira/browse/TEZ-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Ramachandran updated TEZ-2843: -- Attachment: TEZ-2843.2.patch bq. Wouldn't it be better to stop polling on failure? in the case of a AM restart it could pick up the polling and continue. we cannot distinguish the case between a AM restart (failed and restarted) and am exited. will check if AM if it does not run on the same machine will register the ui endpoint again. fixed rest [~Sreenath] please review. > Tez UI: Show error if in progress fails due to AM not reachable > --- > > Key: TEZ-2843 > URL: https://issues.apache.org/jira/browse/TEZ-2843 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Prakash Ramachandran >Assignee: Prakash Ramachandran > Attachments: TEZ-2843.1.patch, TEZ-2843.2.patch > > > currently the in progress update can fail when the AM finishes, but no > feedback is given to the user regarding the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2844) Backport TEZ-2775 to branch-0.7
[ https://issues.apache.org/jira/browse/TEZ-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902060#comment-14902060 ] Rajesh Balamohan commented on TEZ-2844: --- lgtm. +1. Plz remove unused import in MergeManager before commit. > Backport TEZ-2775 to branch-0.7 > --- > > Key: TEZ-2844 > URL: https://issues.apache.org/jira/browse/TEZ-2844 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2844.1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902024#comment-14902024 ] Siddharth Seth commented on TEZ-2716: - - Instead of calling sortAndSpill in a synchronzied block, it would be better to only read the value of totalKeys inside a sync block, and then call sortAndSpill outside the block. - Does totalKeysProcessed in SpillThread need to be volatile. (Is set by the main thread and read in the spill thread) - The same handling required for sameKey ? I don't think this is modified under any lock. > DefaultSorter.isRleNeeded not thread safe > - > > Key: TEZ-2716 > URL: https://issues.apache.org/jira/browse/TEZ-2716 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Siddharth Seth > Attachments: TEZ-2716.1.patch > > > TEZ-1997. > Should be targeted at the same set of versions that TEZ-1997 goes into. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2847) Tez UI: Task details doesn't gets updated on manual refresh after job complete
[ https://issues.apache.org/jira/browse/TEZ-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903127#comment-14903127 ] Prakash Ramachandran commented on TEZ-2847: --- +1 committing shortly > Tez UI: Task details doesn't gets updated on manual refresh after job complete > -- > > Key: TEZ-2847 > URL: https://issues.apache.org/jira/browse/TEZ-2847 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 0.7.1, 0.8.1 > > Attachments: TEZ-2847.1.patch > > > When the AM gets stopped in between an in-progress poll, the UI display > details from the last poll and doesn't gets updated. > The update doesn't happen even on manual refresh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TEZ-2844) Backport TEZ-2775 to branch-0.7
[ https://issues.apache.org/jira/browse/TEZ-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth resolved TEZ-2844. - Resolution: Fixed Fix Version/s: 0.7.1 > Backport TEZ-2775 to branch-0.7 > --- > > Key: TEZ-2844 > URL: https://issues.apache.org/jira/browse/TEZ-2844 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: 0.7.1 > > Attachments: TEZ-2844.1.txt, TEZ-2844.2.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2844) Backport TEZ-2775 to branch-0.7
[ https://issues.apache.org/jira/browse/TEZ-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2844: Attachment: TEZ-2844.2.txt Updated patch with the unused imports removed. Thanks for the review. Committing to branch-7 > Backport TEZ-2775 to branch-0.7 > --- > > Key: TEZ-2844 > URL: https://issues.apache.org/jira/browse/TEZ-2844 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2844.1.txt, TEZ-2844.2.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-1117) Option to make YARN application failed on dag failure
[ https://issues.apache.org/jira/browse/TEZ-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903513#comment-14903513 ] Siddharth Seth commented on TEZ-1117: - Given the perpetual confusion with the FinalApplicationStatus - we could try adding a neutral 'COMPLETED' status - which makes more sense for long running applications, and apps like Tez, which handle multiple jobs in the same app - and track failure / success of these jobs independent of YARN. > Option to make YARN application failed on dag failure > - > > Key: TEZ-1117 > URL: https://issues.apache.org/jira/browse/TEZ-1117 > Project: Apache Tez > Issue Type: Improvement >Reporter: Rohini Palaniswamy > > Can we have an configuration to make the Application status FAILED on > termination if one of the DAGs fail? It is very confusing for users to see > the application SUCCEEDED. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2851) Support a way for upstream applications to pass in a caller context to Tez
[ https://issues.apache.org/jira/browse/TEZ-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903755#comment-14903755 ] Hitesh Shah commented on TEZ-2851: -- [~sseth] [~bikassaha] review please > Support a way for upstream applications to pass in a caller context to Tez > -- > > Key: TEZ-2851 > URL: https://issues.apache.org/jira/browse/TEZ-2851 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-2851.1.patch, TEZ-2851.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903821#comment-14903821 ] Gopal V commented on TEZ-2850: -- Good catch [~saikatr], that's 4kb of space overhead for 100 bytes of data. The perf fix was to fix the total # of JNI calls to libhadoop.so CRC32. With this fix, the Writable deserialization is unbuffered - so an IntWritable will trigger 1 JNI call out to libhadoop.so per 4 byte Integer read (also see HADOOP-10778). > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903787#comment-14903787 ] Siddharth Seth commented on TEZ-2850: - Nice find! I believe this change was made to reduce the number of times the checksum is computed, and to try and compute it in chunks of 4096 for better performance. cc [~gopalv] Other than the 4K buffer, there's a bunch of other objects, references etc per Segment - I won't be surprised if this adds up to a KB. Along with the memory spill limit, adding a limit on the number of in-memory segments would help. The memory-to-memory merger would normally have helped in this case, but that's not tested and should not be enabled. > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903498#comment-14903498 ] Saikat commented on TEZ-2850: - adding [~jeagles] [~rohini] for watch > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903501#comment-14903501 ] Hitesh Shah commented on TEZ-2850: -- \cc [~rajesh.balamohan] [~sseth] [~gopalv] > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
Saikat created TEZ-2850: --- Summary: Tez MergeManager OOM for small Map Outputs Key: TEZ-2850 URL: https://issues.apache.org/jira/browse/TEZ-2850 Project: Apache Tez Issue Type: Bug Reporter: Saikat -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saikat updated TEZ-2850: Attachment: OOM_3.png OOM_2.png OOM_1.png > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2851) Support a way for upstream applications to pass in a caller context to Tez
[ https://issues.apache.org/jira/browse/TEZ-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2851: - Attachment: TEZ-2851.1.patch > Support a way for upstream applications to pass in a caller context to Tez > -- > > Key: TEZ-2851 > URL: https://issues.apache.org/jira/browse/TEZ-2851 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-2851.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saikat updated TEZ-2850: Attachment: TEZ-2850_test.patch > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903481#comment-14903481 ] Saikat edited comment on TEZ-2850 at 9/22/15 9:35 PM: -- This is a unique scenario, that we faced, while running a Tez Job. A reducer vertex task fetches around 20 map outputs, each of around ~100 odd bytes. So total mapoutput size is around 20 * 100 ~ 20Mb. The MergeManager has a merge threshold check, where if it crosses this threshold, InmemoryMerger will be triggered and it will merge & spill the inmemory fetched map outputs to disk to free up memory. In our scenario, mergethreshold(~500mb) >> commitMemory(~20mb), So inMemory merger never gets triggerd. Finally when the finalMerge() is called in close(), MergeManager calls createInMemorySegments() to do the final merge. In this, when Tez creates a IFileInputStream object for the InMemoryReader, the IFileInputStream allocates a buffer of size 4096(hard coded). Thus the total size of a single inmemory segment comes to around 5kb, even though data in this segment is only in order of 100 bytes. So, for 20 map outputs, the total size is 20 * 5000 ~ 1G, which causes OOM! Attached is a snapshot of the heap dump which shows this scenario. was (Author: saikatr): This is a unique scenario, that we faced, while running a Tez Job. A reducer vertex task fetches around 20 map outputs, each of around ~100 odd bytes. So total mapoutput size is around 20 * 100 ~ 20Mb. The MergeManager has a merge threshold check, where if it crosses this threshold, InmemoryMerger will be triggered and it will spill the inmemory fetched map outputs to disk to free up memory. In our scenario, mergethreshold(~500mb) >> commitMemory(~20mb), So inMemory merger never gets triggerd. Finally when the finalMerge() is called in close(), MergeManager calls createInMemorySegments() to do the final merge. In this, when Tez creates a IFileInputStream object for the InMemoryReader, the IFileInputStream allocates a buffer of size 4096(hard coded). Thus the total size of a single inmemory segment comes to around 5kb, even though data in this segment is only in order of 100 bytes. So, for 20 map outputs, the total size is 20 * 5000 ~ 1G, which causes OOM! Attached is a snapshot of the heap dump which shows this scenario. > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903481#comment-14903481 ] Saikat commented on TEZ-2850: - This is a unique scenario, that we faced, while running a Tez Job. A reducer vertex task fetches around 20 map outputs, each of around ~100 odd bytes. So total mapoutput size is around 20 * 100 ~ 20Mb. The MergeManager has a merge threshold check, where if it crosses this threshold, InmemoryMerger will be triggered and it will spill the inmemory fetched map outputs to disk to free up memory. In our scenario, mergethreshold(~500mb) >> commitMemory(~20mb), So inMemory merger never gets triggerd. Finally when the finalMerge() is called in close(), MergeManager calls createInMemorySegments() to do the final merge. In this, when Tez creates a IFileInputStream object for the InMemoryReader, the IFileInputStream allocates a buffer of size 4096(hard coded). Thus the total size of a single inmemory segment comes to around 5kb, even though data in this segment is only in order of 100 bytes. So, for 20 map outputs, the total size is 20 * 5000 ~ 1G, which causes OOM! Attached is a snapshot of the heap dump which shows this scenario. > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2851) Support a way for upstream applications to pass in a caller context to Tez
Hitesh Shah created TEZ-2851: Summary: Support a way for upstream applications to pass in a caller context to Tez Key: TEZ-2851 URL: https://issues.apache.org/jira/browse/TEZ-2851 Project: Apache Tez Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2850) Tez MergeManager OOM for small Map Outputs
[ https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903497#comment-14903497 ] Saikat commented on TEZ-2850: - [~hitesh] I was going through Hadoop's IFileInputStream implementation(hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFileInputStream.java ) and found that this implementation of buffer[4096] is not present in hadoop but in Tez. I submitted a tentative patch in which IFileInputStream of Tez behaves exactly as that of Hadoop. Can you please throw some light what does this added buffer do? > Tez MergeManager OOM for small Map Outputs > -- > > Key: TEZ-2850 > URL: https://issues.apache.org/jira/browse/TEZ-2850 > Project: Apache Tez > Issue Type: Bug >Reporter: Saikat > Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2851) Support a way for upstream applications to pass in a caller context to Tez
[ https://issues.apache.org/jira/browse/TEZ-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2851: - Attachment: TEZ-2851.2.patch > Support a way for upstream applications to pass in a caller context to Tez > -- > > Key: TEZ-2851 > URL: https://issues.apache.org/jira/browse/TEZ-2851 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-2851.1.patch, TEZ-2851.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2758) Remove append API in RecoveryService after TEZ-1909
[ https://issues.apache.org/jira/browse/TEZ-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903577#comment-14903577 ] Hitesh Shah commented on TEZ-2758: -- [~zjffdu] The option to close on a new dag submission seems interesting. Only consideration would be whether a dag finished event for an older dag can show up after a dag submitted? Also, my original question was whether we read the main dag file even if summary log says completed? Assuming we do not read dag data if there is another dag in the summary then we can first close stream when new dag is submitted and also drop all older dag events that show up after the stream has been closed? > Remove append API in RecoveryService after TEZ-1909 > --- > > Key: TEZ-2758 > URL: https://issues.apache.org/jira/browse/TEZ-2758 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.6.2 >Reporter: Jeff Zhang >Assignee: Jeff Zhang > > After TEZ-1909, there would be no case for append recovery file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2758) Remove append API in RecoveryService after TEZ-1909
[ https://issues.apache.org/jira/browse/TEZ-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903577#comment-14903577 ] Hitesh Shah edited comment on TEZ-2758 at 9/22/15 11:27 PM: [~zjffdu] The option to close on a new dag submission seems interesting. Only consideration would be whether a dag finished event for an older dag can show up after a new dag is submitted? Also, my original question was whether we read the main dag file even if summary log says completed? Assuming we do not read dag data if there is another dag in the summary then we can first close all existing open streams when the new dag is submitted and also drop all older dag events that show up after the stream has been closed? was (Author: hitesh): [~zjffdu] The option to close on a new dag submission seems interesting. Only consideration would be whether a dag finished event for an older dag can show up after a dag submitted? Also, my original question was whether we read the main dag file even if summary log says completed? Assuming we do not read dag data if there is another dag in the summary then we can first close stream when new dag is submitted and also drop all older dag events that show up after the stream has been closed? > Remove append API in RecoveryService after TEZ-1909 > --- > > Key: TEZ-2758 > URL: https://issues.apache.org/jira/browse/TEZ-2758 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.6.2 >Reporter: Jeff Zhang >Assignee: Jeff Zhang > > After TEZ-1909, there would be no case for append recovery file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2851) Support a way for upstream applications to pass in a caller context to Tez
[
https://issues.apache.org/jira/browse/TEZ-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903634#comment-14903634
]
TezQA commented on TEZ-2851:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761735/TEZ-2851.1.patch
against master revision 81a1221.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 142 javac
compiler warnings (more than the master's current 140 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.dag.history.logging.ats.TestHistoryEventTimelineConversion
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1167//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/1167//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1167//console
This message is automatically generated.
> Support a way for upstream applications to pass in a caller context to Tez
> --
>
> Key: TEZ-2851
> URL: https://issues.apache.org/jira/browse/TEZ-2851
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Hitesh Shah
> Attachments: TEZ-2851.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2843 PreCommit Build #1163
Jira: https://issues.apache.org/jira/browse/TEZ-2843
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1163/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3463 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761461/TEZ-2843.1.patch
against master revision 0aa31af.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1163//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1163//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
cce2ff6bb00ac9c9c3affe33052b38beaf67deeb logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1160
Archived 53 artifacts
Archive block size is 32768
Received 8 blocks and 2994835 bytes
Compression is 8.0%
Took 0.93 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
All tests passed
Failed: TEZ-2843 PreCommit Build #1164
Jira: https://issues.apache.org/jira/browse/TEZ-2843
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1164/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3463 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761587/TEZ-2843.2.patch
against master revision 0aa31af.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1164//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1164//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
7b75776cf304f4b691c95ae3a95323480b218373 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1160
Archived 53 artifacts
Archive block size is 32768
Received 29 blocks and 2300108 bytes
Compression is 29.2%
Took 0.56 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Updated] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2716: -- Attachment: TEZ-2716.2.patch Thanks [~sseth]. Addressing review comments in .2 patch. > DefaultSorter.isRleNeeded not thread safe > - > > Key: TEZ-2716 > URL: https://issues.apache.org/jira/browse/TEZ-2716 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Siddharth Seth > Attachments: TEZ-2716.1.patch, TEZ-2716.2.patch > > > TEZ-1997. > Should be targeted at the same set of versions that TEZ-1997 goes into. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2843) Tez UI: Show error if in progress fails due to AM not reachable
[
https://issues.apache.org/jira/browse/TEZ-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902179#comment-14902179
]
TezQA commented on TEZ-2843:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761461/TEZ-2843.1.patch
against master revision 0aa31af.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1163//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1163//console
This message is automatically generated.
> Tez UI: Show error if in progress fails due to AM not reachable
> ---
>
> Key: TEZ-2843
> URL: https://issues.apache.org/jira/browse/TEZ-2843
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2843.1.patch, TEZ-2843.2.patch
>
>
> currently the in progress update can fail when the AM finishes, but no
> feedback is given to the user regarding the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2843) Tez UI: Show error if in progress fails due to AM not reachable
[
https://issues.apache.org/jira/browse/TEZ-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902234#comment-14902234
]
TezQA commented on TEZ-2843:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761587/TEZ-2843.2.patch
against master revision 0aa31af.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1164//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1164//console
This message is automatically generated.
> Tez UI: Show error if in progress fails due to AM not reachable
> ---
>
> Key: TEZ-2843
> URL: https://issues.apache.org/jira/browse/TEZ-2843
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2843.1.patch, TEZ-2843.2.patch
>
>
> currently the in progress update can fail when the AM finishes, but no
> feedback is given to the user regarding the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2716 PreCommit Build #1165
Jira: https://issues.apache.org/jira/browse/TEZ-2716
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1165/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3453 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761594/TEZ-2716.2.patch
against master revision 0aa31af.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1165//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1165//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
966f87dacda2cc76f42c19baff05dd61b7217b08 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1160
Archived 53 artifacts
Archive block size is 32768
Received 6 blocks and 3081453 bytes
Compression is 6.0%
Took 2.5 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe
[
https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902278#comment-14902278
]
TezQA commented on TEZ-2716:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761594/TEZ-2716.2.patch
against master revision 0aa31af.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1165//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1165//console
This message is automatically generated.
> DefaultSorter.isRleNeeded not thread safe
> -
>
> Key: TEZ-2716
> URL: https://issues.apache.org/jira/browse/TEZ-2716
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Siddharth Seth
> Attachments: TEZ-2716.1.patch, TEZ-2716.2.patch
>
>
> TEZ-1997.
> Should be targeted at the same set of versions that TEZ-1997 goes into.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2843) Tez UI: Show error if in progress fails due to AM not reachable
[ https://issues.apache.org/jira/browse/TEZ-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902088#comment-14902088 ] Sreenath Somarajapuram commented on TEZ-2843: - +1 LGTM > Tez UI: Show error if in progress fails due to AM not reachable > --- > > Key: TEZ-2843 > URL: https://issues.apache.org/jira/browse/TEZ-2843 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Prakash Ramachandran >Assignee: Prakash Ramachandran > Attachments: TEZ-2843.1.patch, TEZ-2843.2.patch > > > currently the in progress update can fail when the AM finishes, but no > feedback is given to the user regarding the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2842) Tez UI: Update Tez App details page while in-progress
[ https://issues.apache.org/jira/browse/TEZ-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2842: Fix Version/s: 0.8.1 0.7.1 > Tez UI: Update Tez App details page while in-progress > - > > Key: TEZ-2842 > URL: https://issues.apache.org/jira/browse/TEZ-2842 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 0.7.1, 0.8.1 > > Attachments: TEZ-2842.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2842) Tez UI: Update Tez App details page while in-progress
[ https://issues.apache.org/jira/browse/TEZ-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902306#comment-14902306 ] Sreenath Somarajapuram commented on TEZ-2842: - Thank you [~pramachandran]. Committed to master and branch-0.7. > Tez UI: Update Tez App details page while in-progress > - > > Key: TEZ-2842 > URL: https://issues.apache.org/jira/browse/TEZ-2842 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 0.7.1, 0.8.1 > > Attachments: TEZ-2842.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2847) Tez UI: Task details doesn't gets updated on manual refresh after job complete
[ https://issues.apache.org/jira/browse/TEZ-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902581#comment-14902581 ] Sreenath Somarajapuram commented on TEZ-2847: - [~pramachandran] If things looks good, please commit the patch as I would be travelling next 14 hours. > Tez UI: Task details doesn't gets updated on manual refresh after job complete > -- > > Key: TEZ-2847 > URL: https://issues.apache.org/jira/browse/TEZ-2847 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 0.7.1, 0.8.1 > > Attachments: TEZ-2847.1.patch > > > When the AM gets stopped in between an in-progress poll, the UI display > details from the last poll and doesn't gets updated. > The update doesn't happen even on manual refresh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2847) Tez UI: Task details doesn't gets updated on manual refresh after job complete
[ https://issues.apache.org/jira/browse/TEZ-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2847: Description: When the AM gets stopped in between an in-progress poll, the UI display details from the last poll and doesn't gets updated. The update doesn't happen even on manual refresh. > Tez UI: Task details doesn't gets updated on manual refresh after job complete > -- > > Key: TEZ-2847 > URL: https://issues.apache.org/jira/browse/TEZ-2847 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > > When the AM gets stopped in between an in-progress poll, the UI display > details from the last poll and doesn't gets updated. > The update doesn't happen even on manual refresh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2848) Tez UI: Followup to TEZ-2847, ensure reload is working as expected in all cases for all entities.
Sreenath Somarajapuram created TEZ-2848: --- Summary: Tez UI: Followup to TEZ-2847, ensure reload is working as expected in all cases for all entities. Key: TEZ-2848 URL: https://issues.apache.org/jira/browse/TEZ-2848 Project: Apache Tez Issue Type: Bug Reporter: Sreenath Somarajapuram -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2847) Tez UI: Task details doesn't gets updated on manual refresh after job complete
[ https://issues.apache.org/jira/browse/TEZ-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2847: Fix Version/s: 0.8.1 0.7.1 > Tez UI: Task details doesn't gets updated on manual refresh after job complete > -- > > Key: TEZ-2847 > URL: https://issues.apache.org/jira/browse/TEZ-2847 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 0.7.1, 0.8.1 > > Attachments: TEZ-2847.1.patch > > > When the AM gets stopped in between an in-progress poll, the UI display > details from the last poll and doesn't gets updated. > The update doesn't happen even on manual refresh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-2847) Tez UI: Task details doesn't gets updated on manual refresh after job complete
[ https://issues.apache.org/jira/browse/TEZ-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram reassigned TEZ-2847: --- Assignee: Sreenath Somarajapuram > Tez UI: Task details doesn't gets updated on manual refresh after job complete > -- > > Key: TEZ-2847 > URL: https://issues.apache.org/jira/browse/TEZ-2847 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2847) Tez UI: Task details doesn't gets updated on manual refresh after job complete
[ https://issues.apache.org/jira/browse/TEZ-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2847: Attachment: TEZ-2847.1.patch [~pramachandran] Please review. This patch is just for task and attempts. Created a followup jira (TEZ-2848) to handle reload across the board. > Tez UI: Task details doesn't gets updated on manual refresh after job complete > -- > > Key: TEZ-2847 > URL: https://issues.apache.org/jira/browse/TEZ-2847 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2847.1.patch > > > When the AM gets stopped in between an in-progress poll, the UI display > details from the last poll and doesn't gets updated. > The update doesn't happen even on manual refresh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2847) Tez UI: Task details doesn't gets updated on manual refresh after job complete
[
https://issues.apache.org/jira/browse/TEZ-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902706#comment-14902706
]
TezQA commented on TEZ-2847:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761636/TEZ-2847.1.patch
against master revision f1ab3c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1166//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1166//console
This message is automatically generated.
> Tez UI: Task details doesn't gets updated on manual refresh after job complete
> --
>
> Key: TEZ-2847
> URL: https://issues.apache.org/jira/browse/TEZ-2847
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Fix For: 0.7.1, 0.8.1
>
> Attachments: TEZ-2847.1.patch
>
>
> When the AM gets stopped in between an in-progress poll, the UI display
> details from the last poll and doesn't gets updated.
> The update doesn't happen even on manual refresh.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2847 PreCommit Build #1166
Jira: https://issues.apache.org/jira/browse/TEZ-2847
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1166/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3456 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761636/TEZ-2847.1.patch
against master revision f1ab3c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1166//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1166//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
be9f75fb5b0ceac3db1c5f0e133d128bea3553c5 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1160
Archived 53 artifacts
Archive block size is 32768
Received 29 blocks and 2299301 bytes
Compression is 29.2%
Took 0.6 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
All tests passed
