[jira] [Updated] (YARN-6377) NMTimelinePublisher#serviceStop does not stop timeline clients

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6377: - Description: While discussing YARN-6342, Varun found another issue regarding TimelineV2Client. "In

[jira] [Updated] (YARN-6377) NMTimelinePublisher#serviceStop does not stop timeline clients

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6377: - Attachment: YARN-6377.01.patch Attaching a trivial patch > NMTimelinePublisher#serviceStop does not stop

[jira] [Commented] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937364#comment-15937364 ] Haibo Chen commented on YARN-6376: -- bq. We should synchronize these two operations. Agreed. We may need to

[jira] [Updated] (YARN-6377) NMTimelinePublisher#serviceStop does not stop timeline clients

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6377: - Description: While discussing YARN-6342, Varun found out another issue regarding TimelineV2Client. "In

[jira] [Commented] (YARN-5269) Bubble exceptions and errors all the way up the calls, including to clients.

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937308#comment-15937308 ] Haibo Chen commented on YARN-5269: -- Per offline discussion with [~vrushalic] and [~jrottinghuis] in the

[jira] [Updated] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6376: - Issue Type: Sub-task (was: Bug) Parent: YARN-5355 > Exceptions caused by synchronous putEntities

[jira] [Comment Edited] (YARN-5269) Bubble exceptions and errors all the way up the calls, including to clients.

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937308#comment-15937308 ] Haibo Chen edited comment on YARN-5269 at 3/22/17 10:48 PM: Per offline

[jira] [Created] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector

2017-03-22 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6376: Summary: Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector Key: YARN-6376 URL: https://issues.apache.org/jira/browse/YARN-6376

[jira] [Updated] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6376: - Labels: yarn-5355-merge-blocker (was: ) > Exceptions caused by synchronous putEntities requests can be

[jira] [Commented] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937221#comment-15937221 ] Haibo Chen commented on YARN-6376: -- [~varun_saxena] Just added more details to this jira. > Exceptions

[jira] [Updated] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector

2017-03-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6376: - Description: TimelineCollector.putEntitities() is currently implemented by calling TimelineWriter.write()

[jira] [Commented] (YARN-6377) NMTimelinePublisher#serviceStop does not stop timeline clients

2017-03-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938701#comment-15938701 ] Haibo Chen commented on YARN-6377: -- Thanks for pointing it out, [~varun_saxena]! Will upload a new patch

[jira] [Commented] (YARN-6357) Implement putEntitiesAsync API in TimelineCollector

2017-03-28 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946127#comment-15946127 ] Haibo Chen commented on YARN-6357: -- Thanks [~varun_saxena] for your reviews and commit! > Implement

[jira] [Commented] (YARN-6342) Make TimelineV2Client's drain period after stop configurable

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947719#comment-15947719 ] Haibo Chen commented on YARN-6342: -- Upload a simple patch to make the drain period configurable. We could

[jira] [Updated] (YARN-6342) Make TimelineV2Client's drain period after stop configurable

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6342: - Attachment: YARN-6342.00.patch > Make TimelineV2Client's drain period after stop configurable >

[jira] [Updated] (YARN-3760) FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-3760: - Summary: FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close() (was: Log aggregation failures

[jira] [Assigned] (YARN-3760) Log aggregation failures

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-3760: Assignee: Haibo Chen > Log aggregation failures > - > >

[jira] [Commented] (YARN-3760) Log aggregation failures

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947608#comment-15947608 ] Haibo Chen commented on YARN-3760: -- bq. the ctor creates the fs data stream then a TFile.Writer w/o a

[jira] [Updated] (YARN-6342) Make TimelineV2Client's drain period after stop configurable

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6342: - Summary: Make TimelineV2Client's drain period after stop configurable (was: Issues in async API of

[jira] [Updated] (YARN-3760) FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-3760: - Attachment: YARN-3760.00.patch > FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close() >

[jira] [Updated] (YARN-3760) FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-3760: - Target Version/s: 2.8.1, 3.0.0-alpha3 (was: 2.8.0) > FSDataOutputStream leak in

[jira] [Updated] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6376: - Attachment: YARN-6376.00.patch > Exceptions caused by synchronous putEntities requests can be swallowed >

[jira] [Updated] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6376: - Summary: Exceptions caused by synchronous putEntities requests can be swallowed (was: Exceptions caused

[jira] [Assigned] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-6382: Assignee: Haibo Chen > Address race condition on TimelineWriter.flush() caused by buffer-sized

[jira] [Commented] (YARN-6342) Make TimelineV2Client's drain period after stop configurable

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948024#comment-15948024 ] Haibo Chen commented on YARN-6342: -- Thanks for your review, [~varun_saxena]! bq. We need to add this

[jira] [Moved] (YARN-6409) RM does not blacklist node for AM launch failures

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen moved MAPREDUCE-6872 to YARN-6409: - Affects Version/s: (was: 3.0.0-alpha2)

[jira] [Updated] (YARN-6342) Make TimelineV2Client's drain timeout after stop configurable

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6342: - Summary: Make TimelineV2Client's drain timeout after stop configurable (was: Make TimelineV2Client's

[jira] [Assigned] (YARN-6356) Allow different values of yarn.log-aggregation.retain-seconds for succeeded and failed jobs

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-6356: Assignee: Haibo Chen > Allow different values of yarn.log-aggregation.retain-seconds for succeeded

[jira] [Updated] (YARN-6342) Make TimelineV2Client's drain timeout after stop configurable

2017-03-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6342: - Attachment: YARN-6342.01.patch > Make TimelineV2Client's drain timeout after stop configurable >

[jira] [Commented] (YARN-6329) Remove unnecessary TODO comment from AppLogAggregatorImpl.java

2017-03-28 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945941#comment-15945941 ] Haibo Chen commented on YARN-6329: -- Thanks [~vbertschinger] for the patch! +1 nonbinding. > Remove

[jira] [Updated] (YARN-6414) ATSv2 tests fail due to guava version upgrade

2017-03-30 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6414: - Attachment: YARN-6414.01.patch > ATSv2 tests fail due to guava version upgrade >

[jira] [Commented] (YARN-6414) ATSv2 tests fail due to guava version upgrade

2017-03-30 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950099#comment-15950099 ] Haibo Chen commented on YARN-6414: -- I accidentally uploaded the wrong version. Attaching one that worked

[jira] [Commented] (YARN-6377) NMTimelinePublisher#serviceStop does not stop timeline clients

2017-03-31 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951245#comment-15951245 ] Haibo Chen commented on YARN-6377: -- Thanks [~varun_saxena] and [~vrushalic] for the reviews >

[jira] [Commented] (YARN-6377) NMTimelinePublisher#serviceStop does not stop timeline clients

2017-03-31 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951320#comment-15951320 ] Haibo Chen commented on YARN-6377: -- bq. do we have any timeline service related stop on the RM side? Not

[jira] [Commented] (YARN-6376) Exceptions caused by synchronous putEntities requests can be swallowed in TimelineCollector

2017-03-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939093#comment-15939093 ] Haibo Chen commented on YARN-6376: -- Will upload a patch once YARN-6357 is committed > Exceptions caused

[jira] [Updated] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush

2017-03-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6382: - Labels: yarn-5355-merge-blocker (was: ) > Address race condition on TimelineWriter.flush() caused by

[jira] [Created] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush

2017-03-23 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6382: Summary: Address race condition on TimelineWriter.flush() caused by buffer-sized flush Key: YARN-6382 URL: https://issues.apache.org/jira/browse/YARN-6382 Project: Hadoop

[jira] [Updated] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush

2017-03-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6382: - Issue Type: Sub-task (was: Bug) Parent: YARN-5355 > Address race condition on

[jira] [Updated] (YARN-6357) Implement TimelineCollector#putEntitiesAsync

2017-03-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6357: - Attachment: YARN-6357.03.patch > Implement TimelineCollector#putEntitiesAsync >

[jira] [Updated] (YARN-6146) Add Builder methods for TimelineEntityFilters

2017-03-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6146: - Attachment: YARN-6146.03.patch > Add Builder methods for TimelineEntityFilters >

[jira] [Updated] (YARN-6146) Add Builder methods for TimelineEntityFilters

2017-03-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6146: - Attachment: YARN-6146-YARN-5355.03.patch Upload a new patch to address Varun's comments > Add Builder

[jira] [Commented] (YARN-6146) Add Builder methods for TimelineEntityFilters

2017-03-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930446#comment-15930446 ] Haibo Chen commented on YARN-6146: -- The findbug warning is known IIRC, the checkstyles are existing

[jira] [Commented] (YARN-6319) race condition between deleting app dir and deleting container dir

2017-03-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930366#comment-15930366 ] Haibo Chen commented on YARN-6319: -- By linearizing container cleanup and app cleanup, I mean that

[jira] [Assigned] (YARN-6342) Issues in async API of TimelineClient

2017-03-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-6342: Assignee: Haibo Chen > Issues in async API of TimelineClient >

[jira] [Updated] (YARN-6357) Implement TimelineCollector#putEntitiesAsync

2017-03-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6357: - Attachment: YARN-6357.01.patch Upload an initial patch for review > Implement

[jira] [Comment Edited] (YARN-6357) Implement TimelineCollector#putEntitiesAsync

2017-03-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930832#comment-15930832 ] Haibo Chen edited comment on YARN-6357 at 3/17/17 10:26 PM: Upload an initial

[jira] [Commented] (YARN-6368) Decommissioning an NM results in a -1 exit code

2017-03-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933734#comment-15933734 ] Haibo Chen commented on YARN-6368: -- Thanks [~miklos.szeg...@cloudera.com] for the patch! It looks like

[jira] [Commented] (YARN-6342) Issues in async API of TimelineClient

2017-03-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933724#comment-15933724 ] Haibo Chen commented on YARN-6342: -- publishWithoutBlockingOnQueue() will only throw InterruptedExceptions

[jira] [Updated] (YARN-6357) Implement TimelineCollector#putEntitiesAsync

2017-03-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6357: - Attachment: YARN-6357.02.patch > Implement TimelineCollector#putEntitiesAsync >

[jira] [Commented] (YARN-6357) Implement TimelineCollector#putEntitiesAsync

2017-03-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933517#comment-15933517 ] Haibo Chen commented on YARN-6357: -- Thanks [~varun_saxena] for the review! I updated the patch accordingly

[jira] [Commented] (YARN-6357) Implement TimelineCollector#putEntitiesAsync

2017-03-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933545#comment-15933545 ] Haibo Chen commented on YARN-6357: -- One thing I noticed, is that TimelineWriter.write() is effectively an

[jira] [Commented] (YARN-6345) Add container tags to resource requests

2017-03-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927271#comment-15927271 ] Haibo Chen commented on YARN-6345: -- This may have some overlap with YARN-6268 > Add container tags to

[jira] [Commented] (YARN-6334) TestRMFailover#testAutomaticFailover always passes even RM didn't transition to Standby.

2017-03-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925133#comment-15925133 ] Haibo Chen commented on YARN-6334: -- Thanks for the fix, [~yufeigu]! Can we extract a new method that does

[jira] [Commented] (YARN-6319) race condition between deleting app dir and deleting container dir

2017-03-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925161#comment-15925161 ] Haibo Chen commented on YARN-6319: -- Thanks [~zhiguohong] for the analysis! This seems an important issue

[jira] [Commented] (YARN-6146) Add Builder methods for TimelineEntityFilters

2017-03-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928811#comment-15928811 ] Haibo Chen commented on YARN-6146: -- Thanks for your comments [~varun_saxena]! bq. why not use the builder

[jira] [Commented] (YARN-6319) race condition between deleting app dir and deleting container dir

2017-03-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929009#comment-15929009 ] Haibo Chen commented on YARN-6319: -- Thanks [~zhiguohong] for more explanation on option 2. While I agree

[jira] [Commented] (YARN-6302) Fail the node, if Linux Container Executor is not configured properly

2017-03-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927044#comment-15927044 ] Haibo Chen commented on YARN-6302: -- [~miklos.szeg...@cloudera.com] Can you elaborate more on why types of

[jira] [Comment Edited] (YARN-6302) Fail the node, if Linux Container Executor is not configured properly

2017-03-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927044#comment-15927044 ] Haibo Chen edited comment on YARN-6302 at 3/15/17 9:44 PM: ---

[jira] [Commented] (YARN-6319) race condition between deleting app dir and deleting container dir

2017-03-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15926799#comment-15926799 ] Haibo Chen commented on YARN-6319: -- [~zhiguohong] Sorry for my misunderstanding of the issue. So this is a

[jira] [Commented] (YARN-6334) TestRMFailover#testAutomaticFailover always passes even RM didn't transition to Standby.

2017-03-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15926883#comment-15926883 ] Haibo Chen commented on YARN-6334: -- Thanks for the update, [~yufeigu]! One question that I was missing is,

[jira] [Commented] (YARN-6334) TestRMFailover#testAutomaticFailover always passes even RM didn't transition to Standby.

2017-03-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927004#comment-15927004 ] Haibo Chen commented on YARN-6334: -- +1 non-binding > TestRMFailover#testAutomaticFailover always passes

[jira] [Comment Edited] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush

2017-04-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954358#comment-15954358 ] Haibo Chen edited comment on YARN-6382 at 4/3/17 11:55 PM: --- Thanks for the nice

[jira] [Commented] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush

2017-04-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954358#comment-15954358 ] Haibo Chen commented on YARN-6382: -- Thanks for the nice summary [~jrottinghuis]! bq. This write causes

[jira] [Commented] (YARN-6202) Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded

2017-04-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954140#comment-15954140 ] Haibo Chen commented on YARN-6202: -- Per offline discussion with Yufei, post to YARN-2917, AsyncDispatcher

[jira] [Updated] (YARN-6316) Provide help information and documentation for TimelineSchemaCreator

2017-04-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6316: - Attachment: YARN-6316.prelim.patch Update a preliminary patch for suggestion on unit test and help message

[jira] [Updated] (YARN-6409) RM does not blacklist node for AM launch failures

2017-04-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6409: - Description: Currently, node blacklisting upon AM failures only handles failures that happen after AM

[jira] [Updated] (YARN-6409) RM does not blacklist node for AM launch failures

2017-04-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6409: - Description: Currently, node blacklisting upon AM failures only handles failures that happen after AM

[jira] [Updated] (YARN-6455) Enhance the timelinewriter.flush() race condition fix in YARN-6382

2017-04-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6455: - Attachment: YARN-6455.00.patch Upload a patch based on Joep's idea > Enhance the timelinewriter.flush()

[jira] [Updated] (YARN-6409) RM does not blacklist node for AM launch failures

2017-04-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6409: - Attachment: YARN-6409.00.patch > RM does not blacklist node for AM launch failures >

[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures

2017-04-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963580#comment-15963580 ] Haibo Chen commented on YARN-6409: -- Sorry for my delayed reply, [~rohithsharma]. I have added description

[jira] [Commented] (YARN-6202) Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded

2017-04-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15953830#comment-15953830 ] Haibo Chen commented on YARN-6202: -- AsyncDispatcher are also used in a lot of other tests besides MockRM,

[jira] [Commented] (YARN-6433) Only accessible cgroup mount directories should be selected for a controller

2017-04-04 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955530#comment-15955530 ] Haibo Chen commented on YARN-6433: -- Thanks [~miklos.szeg...@cloudera.com] for the patch! +1 non-binding >

[jira] [Commented] (YARN-6424) TimelineCollector is not stopped when an app finishes in RM

2017-04-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15953854#comment-15953854 ] Haibo Chen commented on YARN-6424: -- Agree with [~varun_saxena] on that it should be an

[jira] [Commented] (YARN-6316) Provide help information and documentation for TimelineSchemaCreator

2017-03-31 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951833#comment-15951833 ] Haibo Chen commented on YARN-6316: -- Looked into this a little bit. I was thinking of have two subcommands,

[jira] [Commented] (YARN-6316) Provide help information and documentation for TimelineSchemaCreator

2017-03-31 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951836#comment-15951836 ] Haibo Chen commented on YARN-6316: -- bq. I am also wondering about providing a set of table creation

[jira] [Commented] (YARN-6277) Nodemanager heap memory leak

2017-03-30 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950316#comment-15950316 ] Haibo Chen commented on YARN-6277: -- Thanks [~Feng Yuan] for reporting the issue and working on a patch! If

[jira] [Commented] (YARN-6202) Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded

2017-03-30 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950274#comment-15950274 ] Haibo Chen commented on YARN-6202: -- Reading comments above Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY, it

[jira] [Updated] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush

2017-04-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6382: - Labels: (was: yarn-5355-merge-blocker) > Address race condition on TimelineWriter.flush() caused by

[jira] [Updated] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.

2017-04-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6323: - Labels: yarn-5355-merge-blocker (was: ) > Rolling upgrade/config change is broken on timeline v2. >

[jira] [Assigned] (YARN-3760) FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

2017-04-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-3760: Assignee: Haibo Chen (was: Miklos Szegedi) > FSDataOutputStream leak in

[jira] [Commented] (YARN-3760) FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

2017-04-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959826#comment-15959826 ] Haibo Chen commented on YARN-3760: -- Thanks [~djp] for review and commit! > FSDataOutputStream leak in

[jira] [Commented] (YARN-3760) FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

2017-04-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957202#comment-15957202 ] Haibo Chen commented on YARN-3760: -- [~miklos.szeg...@cloudera.com] Both the writer and the

[jira] [Updated] (YARN-3760) FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

2017-04-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-3760: - Attachment: YARN-3760.01.patch Updated the patch to address Miklos' comment per offline discussion. >

[jira] [Commented] (YARN-4061) [Fault tolerance] Fault tolerant writer for timeline v2

2017-04-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967137#comment-15967137 ] Haibo Chen commented on YARN-4061: -- [~jrottinghuis] do you think we should at least target this at hadoop

[jira] [Commented] (YARN-5269) Bubble exceptions and errors all the way up the calls, including to clients.

2017-04-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967126#comment-15967126 ] Haibo Chen commented on YARN-5269: -- bq. let's restrict the focus for this jira to showing any exception

[jira] [Updated] (YARN-5269) Bubble exceptions and errors all the way up the calls, including to clients.

2017-04-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-5269: - Labels: YARN-5355 (was: YARN-5355 yarn-5355-merge-blocker) > Bubble exceptions and errors all the way up

[jira] [Comment Edited] (YARN-5269) Bubble exceptions and errors all the way up the calls, including to clients.

2017-04-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967126#comment-15967126 ] Haibo Chen edited comment on YARN-5269 at 4/13/17 4:46 AM: --- bq. let's restrict

[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures

2017-04-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971314#comment-15971314 ] Haibo Chen commented on YARN-6409: -- The added unit test in the patch is flaky. Will address it in a new

[jira] [Commented] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977477#comment-15977477 ] Haibo Chen commented on YARN-6500: -- Thanks [~miklos.szeg...@cloudera.com] for the patch! A couple of nits:

[jira] [Commented] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979084#comment-15979084 ] Haibo Chen commented on YARN-6500: -- The findbugs warnings seems unrelated. +1. Will wait until next Monday

[jira] [Updated] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6500: - Component/s: nodemanager > Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler >

[jira] [Updated] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6500: - Affects Version/s: 3.0.0-alpha2 > Do not mount inaccessible cgroups directories in

[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-04-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978998#comment-15978998 ] Haibo Chen commented on YARN-6457: -- The master branch is 'trunk', so you can just create a PR against it.

[jira] [Commented] (YARN-5269) Bubble exceptions and errors all the way up the calls, including to clients.

2017-04-13 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968338#comment-15968338 ] Haibo Chen commented on YARN-5269: -- Yeah, especially if we want to provide detailed error information to

[jira] [Commented] (YARN-6475) Fix some long function checkstyle issues

2017-04-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977070#comment-15977070 ] Haibo Chen commented on YARN-6475: -- [~soumabrata] I have added you as a contributor, you can assign to

[jira] [Assigned] (YARN-6475) Fix some long function checkstyle issues

2017-04-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-6475: Assignee: Soumabrata Chakraborty > Fix some long function checkstyle issues >

[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-04-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977115#comment-15977115 ] Haibo Chen commented on YARN-6457: -- You are right. Looks like final is only available for use by

[jira] [Commented] (YARN-6396) Call verifyAndCreateRemoteLogDir at service initialization instead of application initialization to decrease load for name node

2017-04-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969199#comment-15969199 ] Haibo Chen commented on YARN-6396: -- Robert and Jian's comments remind me of that we have seen customers

[jira] [Updated] (YARN-6455) Enhance the timelinewriter.flush() race condition fix in YARN-6382

2017-04-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6455: - Issue Type: Sub-task (was: Improvement) Parent: YARN-5355 > Enhance the timelinewriter.flush()

[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures

2017-04-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975680#comment-15975680 ] Haibo Chen commented on YARN-6409: -- Ah, I see why the test was failing. I was trying to fail the AM launch

<    1   2   3   4   5   6   7   8   9   10   >