[jira] [Commented] (HDDS-839) Wait for other services in the started script of hadoop-runner base docker image
[ https://issues.apache.org/jira/browse/HDDS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701135#comment-16701135 ] Anu Engineer commented on HDDS-839: --- +1, Thanks for the patch. > Wait for other services in the started script of hadoop-runner base docker > image > > > Key: HDDS-839 > URL: https://issues.apache.org/jira/browse/HDDS-839 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDDS-839-docker-hadoop-runner.001.patch, > HDDS-839-docker-hadoop-runner.002.patch > > > As described in the parent issue, we need a simple method to handle service > dependencies in kubernetes clusters (usually as a workaround when some > clients can't re-try with renewed dns information). > But it also could be useful to minimize the wait time in the docker-compose > clusters. > The easiest implementation is modifying the started script of the > apache/hadoop-runner base image and add a bash loop which checks the > availability of the TCP port (with netcat). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-839) Wait for other services in the started script of hadoop-runner base docker image
[ https://issues.apache.org/jira/browse/HDDS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695786#comment-16695786 ] Elek, Marton commented on HDDS-839: --- The 15 second sleep is not required any more because the WAITFOR functionality could replace it. The 15 seconds wait was a safety line to be sure that scm is started up but now we have the more precious WAITFOR=scm:9876 approach. I think we can do it in the same patch as they are strongly connected (for example reverting this WAITFOR patch requires to put back the sleep 15). > Wait for other services in the started script of hadoop-runner base docker > image > > > Key: HDDS-839 > URL: https://issues.apache.org/jira/browse/HDDS-839 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDDS-839-docker-hadoop-runner.001.patch, > HDDS-839-docker-hadoop-runner.002.patch > > > As described in the parent issue, we need a simple method to handle service > dependencies in kubernetes clusters (usually as a workaround when some > clients can't re-try with renewed dns information). > But it also could be useful to minimize the wait time in the docker-compose > clusters. > The easiest implementation is modifying the started script of the > apache/hadoop-runner base image and add a bash loop which checks the > availability of the TCP port (with netcat). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-839) Wait for other services in the started script of hadoop-runner base docker image
[
https://issues.apache.org/jira/browse/HDDS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693477#comment-16693477
]
Arpit Agarwal commented on HDDS-839:
Thanks for this improvement [~elek]. It looks like we are removing the 15
second sleep for OM, but WAITFOR is not being set. Will that be in a separate
patch?
{code}
- # To make sure SCM is running in dockerized environment we will sleep
- # Could be removed after HDFS-13203
- echo "Waiting 15 seconds for SCM startup"
- sleep 15
{code}
> Wait for other services in the started script of hadoop-runner base docker
> image
>
>
> Key: HDDS-839
> URL: https://issues.apache.org/jira/browse/HDDS-839
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDDS-839-docker-hadoop-runner.001.patch,
> HDDS-839-docker-hadoop-runner.002.patch
>
>
> As described in the parent issue, we need a simple method to handle service
> dependencies in kubernetes clusters (usually as a workaround when some
> clients can't re-try with renewed dns information).
> But it also could be useful to minimize the wait time in the docker-compose
> clusters.
> The easiest implementation is modifying the started script of the
> apache/hadoop-runner base image and add a bash loop which checks the
> availability of the TCP port (with netcat).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-839) Wait for other services in the started script of hadoop-runner base docker image
[
https://issues.apache.org/jira/browse/HDDS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691595#comment-16691595
]
Hadoop QA commented on HDDS-839:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} docker {color} | {color:blue} 0m
5s{color} | {color:blue} Dockerfile
'/home/jenkins/jenkins-slave/workspace/PreCommit-HDDS-Build/sourcedir/dev-support/docker/Dockerfile'
not found, falling back to built-in. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 3m
13s{color} | {color:red} Docker failed to build yetus/hadoop:date2018-11-19.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDDS-839 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948701/HDDS-839-docker-hadoop-runner.002.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/1755/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wait for other services in the started script of hadoop-runner base docker
> image
>
>
> Key: HDDS-839
> URL: https://issues.apache.org/jira/browse/HDDS-839
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDDS-839-docker-hadoop-runner.001.patch,
> HDDS-839-docker-hadoop-runner.002.patch
>
>
> As described in the parent issue, we need a simple method to handle service
> dependencies in kubernetes clusters (usually as a workaround when some
> clients can't re-try with renewed dns information).
> But it also could be useful to minimize the wait time in the docker-compose
> clusters.
> The easiest implementation is modifying the started script of the
> apache/hadoop-runner base image and add a bash loop which checks the
> availability of the TCP port (with netcat).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-839) Wait for other services in the started script of hadoop-runner base docker image
[
https://issues.apache.org/jira/browse/HDDS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687868#comment-16687868
]
Hadoop QA commented on HDDS-839:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} docker {color} | {color:blue} 0m
4s{color} | {color:blue} Dockerfile
'/home/jenkins/jenkins-slave/workspace/PreCommit-HDDS-Build/sourcedir/dev-support/docker/Dockerfile'
not found, falling back to built-in. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 3m
12s{color} | {color:red} Docker failed to build yetus/hadoop:date2018-11-15.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDDS-839 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948302/HDDS-839-docker-hadoop-runner.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/1717/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wait for other services in the started script of hadoop-runner base docker
> image
>
>
> Key: HDDS-839
> URL: https://issues.apache.org/jira/browse/HDDS-839
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDDS-839-docker-hadoop-runner.001.patch
>
>
> As described in the parent issue, we need a simple method to handle service
> dependencies in kubernetes clusters (usually as a workaround when some
> clients can't re-try with renewed dns information).
> But it also could be useful to minimize the wait time in the docker-compose
> clusters.
> The easiest implementation is modifying the started script of the
> apache/hadoop-runner base image and add a bash loop which checks the
> availability of the TCP port (with netcat).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-839) Wait for other services in the started script of hadoop-runner base docker image
[
https://issues.apache.org/jira/browse/HDDS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687784#comment-16687784
]
Elek, Marton commented on HDDS-839:
---
Patch is uploaded. To test it:
1. Apply the patch to the docker-hadoop-runner branch.
2. cd scripts
3. Test the waiting
In one terminal:
{code}
WAITFOR=localhost:1234 ./starter.sh ls
{code}
In an other terminal:
{code}
nc -l -p 1234
{code}
Script should continue only after the nc server is started
4. the normal case (without waiting)
{code]
./starter.sh ls
{code}
You can see the output immediately
5. Timeout
{code}
WAITFOR_TIMEOUT=10 WAITFOR=localhost:1234 ./starter.sh ls
{code}
You can see an error message (and 1 exit code) after ~10 seconds (unless you
start a server on port 1234)
> Wait for other services in the started script of hadoop-runner base docker
> image
>
>
> Key: HDDS-839
> URL: https://issues.apache.org/jira/browse/HDDS-839
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
>
> As described in the parent issue, we need a simple method to handle service
> dependencies in kubernetes clusters (usually as a workaround when some
> clients can't re-try with renewed dns information).
> But it also could be useful to minimize the wait time in the docker-compose
> clusters.
> The easiest implementation is modifying the started script of the
> apache/hadoop-runner base image and add a bash loop which checks the
> availability of the TCP port (with netcat).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
