[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534687=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534687
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 12/Jan/21 06:34
Start Date: 12/Jan/21 06:34
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-758439351
Merged. Thank you @amahussein and @eric-badger
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534687)
Time Spent: 3h (was: 2h 50m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534684=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534684
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 12/Jan/21 06:34
Start Date: 12/Jan/21 06:34
Worklog Time Spent: 10m
Work Description: aajisaka merged pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534684)
Time Spent: 2h 50m (was: 2h 40m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534682=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534682
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 12/Jan/21 06:31
Start Date: 12/Jan/21 06:31
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-758437944
> So I guess go for it. Though I still think this is addressing a symptom,
not the root cause
I have the same opinion.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534682)
Time Spent: 2h 40m (was: 2.5h)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534604=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534604
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 11/Jan/21 23:25
Start Date: 11/Jan/21 23:25
Worklog Time Spent: 10m
Work Description: eric-badger commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-758288185
It doesn't really hurt anything to ask for more memory unless we tell it to
touch all pages or unless we fill all of the memory. If we fill all of the
memory, then I guess that means that we needed it. If we don't, it's not like
the kernel can't use it elsewhere (unless we touched those pages). So I guess
go for it. Though I still think this is addressing a symptom, not the root cause
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534604)
Time Spent: 2.5h (was: 2h 20m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534246=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534246
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 11/Jan/21 14:02
Start Date: 11/Jan/21 14:02
Worklog Time Spent: 10m
Work Description: amahussein edited a comment on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-757970031
@aajisaka , @ericbadger
Any thoughts whether we should move fwd with this PR?
The error still exists as per January 10th build
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/382/#showFailuresLink.
Furthermore, I see that there is a new module added to the build
`hadoop-cloud-storage-project/hadoop-huaweicloud` which is likely going to
increase the resources usage.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534246)
Time Spent: 2h 20m (was: 2h 10m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534243=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534243
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 11/Jan/21 14:01
Start Date: 11/Jan/21 14:01
Worklog Time Spent: 10m
Work Description: amahussein edited a comment on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-757970031
@aajisaka , @ericbadger
Any thoughts whether we should move fwd with this PR?
The error still exists as per January 10th build
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/382/#showFailuresLink.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534243)
Time Spent: 2h 10m (was: 2h)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534242
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 11/Jan/21 14:01
Start Date: 11/Jan/21 14:01
Worklog Time Spent: 10m
Work Description: amahussein commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-757970031
@aajisaka , @ericbadger
Any thoughts whether we should move fwd with this PR?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534242)
Time Spent: 2h (was: 1h 50m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=529239=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529239
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 29/Dec/20 19:02
Start Date: 29/Dec/20 19:02
Worklog Time Spent: 10m
Work Description: amahussein commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-752206607
@aajisaka Do you think we should try merging that change (increasing of 2GB)
until we find out what is causing the error of OOM?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 529239)
Time Spent: 1h 50m (was: 1h 40m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=527811=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527811
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 23/Dec/20 20:03
Start Date: 23/Dec/20 20:03
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-750452358
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +1 :green_heart: | shadedclient | 23m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 16s | | There were no new
shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 13s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 41m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2560/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2560 |
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux b4c8a855c5a9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 16a20503cac |
| Max. process+thread count | 538 (vs. ulimit of 5500) |
| modules | C: . U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2560/2/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527811)
Time Spent: 1h 40m (was: 1.5h)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=527750=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527750
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 23/Dec/20 18:33
Start Date: 23/Dec/20 18:33
Worklog Time Spent: 10m
Work Description: amahussein commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-750422564
> The nodes have 98GB of memory and two executors. Therefore we can use
about 40GB of memory for each job.
Thanks @aajisaka !
I suggest we increase the memory to 22GB.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527750)
Time Spent: 1.5h (was: 1h 20m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=527490=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527490
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 23/Dec/20 05:00
Start Date: 23/Dec/20 05:00
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-749933007
> I asked the infrastructure team how much memory we can use:
https://issues.apache.org/jira/browse/INFRA-21207
The nodes have 98GB of memory and two executors. Therefore we can use about
40GB of memory for each job.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527490)
Time Spent: 1h 20m (was: 1h 10m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=526922=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-526922
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 21/Dec/20 19:14
Start Date: 21/Dec/20 19:14
Worklog Time Spent: 10m
Work Description: ericbadger commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-749148544
I'm not sure how to profile this because I don't really know what access we
have to the machines that run Jenkins, what else runs on there, or anything
else. Logging memory over time would be a good first step to help narrow it
down. Then once we know what module is suspect, we can either investigate on
our own local machines or we can enable further debugging on that specific
module.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 526922)
Time Spent: 1h 10m (was: 1h)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=526140=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-526140
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 18/Dec/20 20:52
Start Date: 18/Dec/20 20:52
Worklog Time Spent: 10m
Work Description: amahussein commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-748313031
> I asked the infrastructure team how much memory we can use:
https://issues.apache.org/jira/browse/INFRA-21207
Thanks @aajisaka !
> It would be nice to know what's tying up all of our memory. Because 20 GB
is a lot for us to be using for unit tests
@ericbadger . Definitely!
It would be easier to narrow the scope of investigation if we had this error
reported the earliest it started to happen.
Unfortunately, it does not seem that it is the case.
The other way is to profile the memory of the image during the execution. Do
you have any suggestion of how to approach this?
A straightforward way would be to dump the system men to the log at the
beginning of each module.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 526140)
Time Spent: 1h (was: 50m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=525884=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-525884
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 18/Dec/20 07:24
Start Date: 18/Dec/20 07:24
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-747919101
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 44s | | trunk passed |
| +1 :green_heart: | compile | 20m 0s | | trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| +1 :green_heart: | compile | 17m 29s | | trunk passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| +1 :green_heart: | mvnsite | 25m 5s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 39s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 7m 11s | | trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| +1 :green_heart: | javadoc | 7m 31s | | trunk passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 27m 15s | | the patch passed |
| +1 :green_heart: | compile | 20m 47s | | the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| +1 :green_heart: | javac | 20m 47s | | the patch passed |
| +1 :green_heart: | compile | 20m 24s | | the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| +1 :green_heart: | javac | 20m 24s | | the patch passed |
| +1 :green_heart: | mvnsite | 24m 45s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 17s | | There were no new
shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 3s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 19m 49s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 9m 10s | | the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| +1 :green_heart: | javadoc | 9m 12s | | the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
_ Other Tests _ |
| -1 :x: | unit | 617m 25s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2560/1/artifact/out/patch-unit-root.txt)
| root in the patch passed. |
| +1 :green_heart: | asflicense | 1m 51s | | The patch does not
generate ASF License warnings. |
| | | 884m 20s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestDecommissionWithStriped |
| | hadoop.hdfs.qjournal.server.TestJournalNodeSync |
| | hadoop.hdfs.TestErasureCodingPolicies |
| | hadoop.yarn.applications.distributedshell.TestDistributedShell |
| | hadoop.tools.dynamometer.TestDynamometerInfra |
| | hadoop.mapreduce.v2.app.rm.TestRMContainerAllocator |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2560/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2560 |
| Optional Tests | dupname asflicense shellcheck shelldocs compile javac
javadoc mvninstall mvnsite unit shadedclient xml |
| uname | Linux 28251d6781c4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4c033bafa02 |
| Default Java | Private
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=525834=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-525834
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 18/Dec/20 02:22
Start Date: 18/Dec/20 02:22
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-747825118
I asked the infrastructure team how much memory we can use:
https://issues.apache.org/jira/browse/INFRA-21207
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 525834)
Time Spent: 40m (was: 0.5h)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=525697=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-525697
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 17/Dec/20 19:52
Start Date: 17/Dec/20 19:52
Worklog Time Spent: 10m
Work Description: ericbadger commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-747663571
It would be nice to know what's tying up all of our memory. Because 20 GB is
a lot for us to be using for unit tests
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 525697)
Time Spent: 0.5h (was: 20m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=525695=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-525695
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 17/Dec/20 19:51
Start Date: 17/Dec/20 19:51
Worklog Time Spent: 10m
Work Description: ericbadger commented on pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560#issuecomment-747662919
Jeez, are we really running out of memory in a 20 GB container? That seems
insane to me
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 525695)
Time Spent: 20m (was: 10m)
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins
[
https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=525639=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-525639
]
ASF GitHub Bot logged work on HADOOP-17438:
---
Author: ASF GitHub Bot
Created on: 17/Dec/20 16:39
Start Date: 17/Dec/20 16:39
Worklog Time Spent: 10m
Work Description: amahussein opened a new pull request #2560:
URL: https://github.com/apache/hadoop/pull/2560
This change to increase the memory limit from 20g to 22g.
The last commit is needed to trigger the tests and increase the timeout. It
should be skipped before merging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 525639)
Remaining Estimate: 0h
Time Spent: 10m
> Increase docker memory limit in Jenkins
> ---
>
> Key: HADOOP-17438
> URL: https://issues.apache.org/jira/browse/HADOOP-17438
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, scripts, test, yetus
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Yetus keeps failing with OOM.
>
> {code:bash}
> unable to create new native thread
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:717)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
> at
> java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
> at
> org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
> at
> org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
>
> This jira to increase the memory limit from 20g to 22g.
> *Note: This is only a workaround to get things more productive. If this
> change reduces the frequency of the OOM failure, there must be a follow-up
> profile the runtime to figure out which components are causing the docker to
> run out of memory.*
> CC: [~aajisaka], [~elgoiri], [~weichiu], [~ebadger], [~tasanuma],
> [~iwasakims], [~ayushtkn], [~inigoiri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
