[
https://issues.apache.org/jira/browse/YARN-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695040#comment-16695040
]
Eric Yang commented on YARN-8986:
---------------------------------
[~Charo Zhang] Jenkins runs with docker 17.05.0-ce. Hadoop compilation runs
inside docker container, which does not have docker binary available to unit
test. This is the reason that it fails. If you change
add_ports_mapping_to_command, then it will work correctly:
{code}
+ if (network_name == NULL) {
+ network_name = make_string("%s", "bridge");
+ }
+ char **ports_mapping_values =
get_configuration_values_delimiter("ports-mapping",
DOCKER_COMMAND_FILE_SECTION, command_config, ",");
+ char *docker_network_command = make_string("%s network inspect %s
--format='{{.Driver}}'", docker_binary, network_name);
+ FILE* docker_network = popen(docker_network_command, "r");
+ ret = fscanf(docker_network, "%s", network_type);
+ if (pclose (docker_network) != 0 || ret <= 0) {
+ fprintf (ERRORFILE, "Could not inspect docker network to get type %s.\n",
docker_network_command);
+ goto cleanup;
+ }
{code}
to
{code}
+ char **ports_mapping_values =
get_configuration_values_delimiter("ports-mapping",
DOCKER_COMMAND_FILE_SECTION, command_config, ",");
+ if (network_name != NULL) {
+ char *docker_network_command = make_string("%s network inspect %s
--format='{{.Driver}}'", docker_binary, network_name);
+ FILE* docker_network = popen(docker_network_command, "r");
+ ret = fscanf(docker_network, "%s", network_type);
+ if (pclose (docker_network) != 0 || ret <= 0) {
+ fprintf (ERRORFILE, "Could not inspect docker network to get type
%s.\n", docker_network_command);
+ goto cleanup;
+ }
+ // other network type exit successfully without ports mapping
+ if (strcasecmp(network_type, "bridge") != 0) {
+ ret = 0;
+ goto cleanup;
+ }
+ }
{code}
If network name is not specified, it will be allowed to use -p and -P flags.
This side step the requirement to have docker binary available to unit test and
validate the default case is working. It would be possible the refactor the
first 3 lines into another function and test the docker_network_command for
null and also artificial name for generating docker_network_command for maximum
coverage, but it is okay if you don't do the refactoring because some existing
unit tests should not fail when net parameter is not passed. It would be okay
to skip the actual docker command execution in unit test, since we already unit
test that part manually in our discussions.
> publish all exposed ports to random ports when using bridge network
> -------------------------------------------------------------------
>
> Key: YARN-8986
> URL: https://issues.apache.org/jira/browse/YARN-8986
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Affects Versions: 3.1.1
> Reporter: Charo Zhang
> Assignee: Charo Zhang
> Priority: Minor
> Labels: Docker
> Attachments: YARN-8986.001.patch, YARN-8986.002.patch,
> YARN-8986.003.patch, YARN-8986.004.patch, YARN-8986.005.patch,
> YARN-8986.006.patch, YARN-8986.007.patch
>
>
> it's better to publish all exposed ports to random ports（-P） or support port
> mapping（-p） for bridge network when using bridge network for docker container.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
