Yup, and that was my point in setting the node-ip kubelet argument, which kinda-replaces that variable, as of node-config moving to ConfigMaps.
@jerome: I should have told you, once you'ld have fixed your inventory setting your own node groups configurations, you should re-deploy from scratch. Make sure do drop everything, especially certificates and configurations that might still mention your former Ip addresses. Openshift has an uninstall playbook. Once applied, make sure all containers are down, /var/lib/etcd sould be empty, ... If you can re-deployall nodes, that's even better (the uninstall tends to leave stuff, ...) Also: if using crio, make sure to check the default value for openshift_node_groups, as it includes a few additional edits you'ld need ... Let us know if you have any questions Apologies for the late reply Regards. On Fri, May 31, 2019, 8:57 AM <bjoern.baertsc...@swisscom.com> wrote: > Hi > > > > This is the first time I write to this mailing list, I'd like to say hello > to everyone. > > > > I once had a similar issue when installing openshift on my notebook using > VirtualBox, I had 2 network interfaces per host (on NATed with internet > access and an internal only) and openshift took the "wrong" one. Then I had > to set the host variable 'openshift_ip' to explicitly set my ip address to > the of the "correct" device. > > > > I cannot find it with 3.11 documentation, but within 3.9. > > > > > https://docs.openshift.com/container-platform/3.9/install_config/install/advanced_install.html#configuring-host-variables > > > > regards, Björn > > > > > > *Von:* users-boun...@lists.openshift.redhat.com < > users-boun...@lists.openshift.redhat.com> *Im Auftrag von *Jérôme Meyer > *Gesendet:* Mittwoch, 29. Mai 2019 17:19 > *An:* Samuel Martín Moro <faus...@gmail.com> > *Cc:* users@lists.openshift.redhat.com > *Betreff:* RE:Issue by installing OKD OpenShift 3.11 > > > > > > Thanks for your help and advise. > > Unfortunately it don't work yet but perhaps it is a network issue... > > > > So, I'll explain more deeply my network architecture... > > > > All my VMs are using the default network 192.168.122.0/24 with forwarding > NAT to go to the Internet. My Laptop are the 192.168.122.1 and it is the > default gateway for all systems too (only one default gateway). This > network works with DHCP. > > > > Then, I've defined a separate intern subnet to perform the container > network: 192.168.100.0/24 as isolated network and internal routing only. > This network used static ip address and address are in DNS defined. > > > > Here're details: > > > > node1 > ens10: 192.168.100.101/24 > > eth1: 192.168.122.193/24 > > docker0: 172.17.0.1/16 > > > > node2 > > ens10: 192.168.100.102/24 > > eth1: 192.168.122.240/24 > > docker0: 172.17.0.1/16 > > > > master > > ens10: 192.168.100.100/24 > > eth1: 192.168.122.54/24 > > docker0: 172.17.0.1/16 > > > > services > > ens10: 192.168.100.103/24 > > eth1: 192.168.122.234/24 > docker0: 172.17.0.1/16 > > > > I'm connecting and start the ansible's job from my workstation VM > 192.168.100.50. > > > > Now, if I've right understood, the openshift service will bind http port > to the same subnet as the default gateway? In my case, it will be the > subnet 192.168.122... ? right? > > Could it that be the problem? > > > > I've defined all ip address for my system in openshift with 192.168.100 > subnet. Is that correct? > > It's possible to use 2 networks has in my case? > > > > It's not yet very clear how the network should be configured for openshift > hosts. I thought about defining a network for external connection > (internet) and a network for internal connection specific to openshift but > I'm not sure is it ok... > > > > Regards, J > > > > > > > > > > > > > > > ------------------------------ > > *De :* Samuel Martín Moro [faus...@gmail.com] > *Envoyé :* vendredi 24 mai 2019 21:45 > *À :* Jérôme Meyer > *Cc :* users@lists.openshift.redhat.com > *Objet :* Re: Issue by installing OKD OpenShift 3.11 > > Oh, that makes perfect sense > > I would assume that your default gateway points to your workstation, in > 192.168.100.0/24? > > > > -- although lately, I've seen some inconsistencies: usually, OpenShift > services would bind on the address assigned to whichever interface routes > to your default gateway. > > > > Assuming that switching your default gateway is not an option, then you > may force OpenShift bind address from your openshift_node_groups definition. > > Dealing with that variable in ini format is quite painful, and usually > leads to syntax errors, ... First we'll create a "group_vars" sub-folder > alongside our inventory. > > > > > > mkdir -p <path-to-inventory-base-directory>/group_vars > > > > > > > > In that folder, we would create a file OSEv3.yml, with the following > content: > > > > > > openshift_node_groups: > > - name: node-config-master-infra > labels: > - 'node-role.kubernetes.io/master=true' > - 'node-role.kubernetes.io/infra=true' > edits: > - key: kubeletArguments.node-ip > value: [ 192.168.122.54 ] > > - name: node-config-node1 > > labels: > - 'node-role.kubernetes.io/compute=true' > edits: > - key: kubeletArguments.node-ip > value: [ <insert-node1-ip-address> ] > > - name: node-config-node2 > > labels: > - 'node-role.kubernetes.io/compute=true' > edits: > - key: kubeletArguments.node-ip > value: [ <insert-node2-ip-address> ] > > > > > > > > see ./roles/openshift_facts/defaults/main.yml for the default > openshift_node_groups definition, if you're curious. > > > > Also make sure that each node from your cluster would load its own > configuration: > > > > > > > > [masters] > > master.olab.oshift.edu openshift_node_groups_name=node-config-master-infra > > > > [etcd:children] > masters > > > > [compute] > > node1.olab.oshift.edu openshift_node_groups_name=node-config-node1 > > node2.olab.oshift.edu openshift_node_groups_name=node-config-node2 > > > > [nodes:children] > > masters > > compute > > > > [nfs] > > ... > > > > [OSEv3:children] > > nodes > > nfs > > > > ... > > > > > > Let us know how that goes. > > > > > > Regards > > > > > > > > On Fri, May 24, 2019 at 3:05 PM Jérôme Meyer <jerome.me...@lcsystems.ch> > wrote: > > Hi, > > > > Thanks for your help and tips. Yeah, I've forgot this time to remove the > htpasswd entries.. ;( > > > > After changing the master definition as 'node-config-master-infra' in > inventory I've restart the deploy-cluster playbook again. > > As you wrote, I've got the master api and etcd information from docker > and checked the logs. > > > > So, some questions arises: > > > > 1. Why this following address is used : *192.168.122.54*? This > corresponds to the master interface. It's a nat address using dhcp to > connected to my pc. > 2. Apparently there're a issue with the etcd access on master: * > connection refused on 2379*. > 3. In the last log, it appears that the request is made on the ip > address *0.0.0.0:8444 > > <https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>*, > something is wrong in my config? > > > > Here're the ip interfaces list of master; where the 192.168.100.100 is > the communication network for openshift as defined in hostname and DNS. > > > > *Interface list* > > > > [root@master ~]# ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group > default qlen 1000 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: ens10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast > state UP group default qlen 1000 > link/ether 52:54:00:ca:44:c8 brd ff:ff:ff:ff:ff:ff > inet 192.168.100.100/24 brd 192.168.100.255 scope global noprefixroute > ens10 > valid_lft forever preferred_lft forever > inet6 fe80::5054:ff:feca:44c8/64 scope link > valid_lft forever preferred_lft forever > 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast > state UP group default qlen 1000 > link/ether 52:54:00:a8:8b:00 brd ff:ff:ff:ff:ff:ff > inet 192.168.122.54/24 brd 192.168.122.255 scope global noprefixroute > dynamic eth1 > valid_lft 3090sec preferred_lft 3090sec > inet6 fe80::c138:7cb0:f8af:7cba/64 scope link noprefixroute > valid_lft forever preferred_lft forever > 4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue > state DOWN group default > link/ether 02:42:a9:c9:8d:d3 brd ff:ff:ff:ff:ff:ff > inet 172.17.0.1/16 scope global docker0 > valid_lft forever preferred_lft forever > > > > *Log from etcd-master* > > > > > > {"log":"2019-05-24 14:19:57.592591 D | etcdserver/api/v2http: [GET] > /health remote:192.168.122.54:44748 > \n","stream":"stderr","time":"2019-05-24T12:19:57.592680803Z"} > {"log":"2019-05-24 14:20:07.580420 D | etcdserver/api/v2http: [GET] > /v2/members remote:192.168.122.54:45038 > \n","stream":"stderr","time":"2019-05-24T12:20:07.580688397Z"} > {"log":"2019-05-24 14:20:07.590218 D | etcdserver/api/v2http: [GET] > /health remote:192.168.122.54:45040 > \n","stream":"stderr","time":"2019-05-24T12:20:07.590356315Z"} > {"log":"2019-05-24 14:20:17.582661 D | etcdserver/api/v2http: [GET] > /v2/members remote:192.168.122.54:45336 > \n","stream":"stderr","time":"2019-05-24T12:20:17.582774753Z"} > {"log":"2019-05-24 14:20:17.595674 D | etcdserver/api/v2http: [GET] > /health remote:192.168.122.54:45338 > \n","stream":"stderr","time":"2019-05-24T12:20:17.595844742Z"} > {"log":"2019-05-24 14:20:27.581915 D | etcdserver/api/v2http: [GET] > /v2/members remote:192.168.122.54:45638 > \n","stream":"stderr","time":"2019-05-24T12:20:27.582036442Z"} > {"log":"2019-05-24 14:20:27.592091 D | etcdserver/api/v2http: [GET] > /health remote:192.168.122.54:45640 > \n","stream":"stderr","time":"2019-05-24T12:20:27.59225275Z"} > {"log":"2019-05-24 14:20:37.584090 D | etcdserver/api/v2http: [GET] > /v2/members remote:192.168.122.54:45932 > \n","stream":"stderr","time":"2019-05-24T12:20:37.584291782Z"} > {"log":"2019-05-24 14:20:37.593862 D | etcdserver/api/v2http: [GET] > /health remote:192.168.122.54:45934 > \n","stream":"stderr","time":"2019-05-24T12:20:37.593980682Z"} > > > > *Log from api-master* > > > > {"log":"I0524 14:18:50.016547 1 plugins.go:84] Registered admission > plugin > \"ResourceQuota\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016617699Z"} > {"log":"I0524 14:18:50.016581 1 plugins.go:84] Registered admission > plugin > \"PodSecurityPolicy\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016622959Z"} > {"log":"I0524 14:18:50.016622 1 plugins.go:84] Registered admission > plugin > \"Priority\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016659601Z"} > {"log":"I0524 14:18:50.016662 1 plugins.go:84] Registered admission > plugin > \"SecurityContextDeny\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01670916Z"} > {"log":"I0524 14:18:50.016713 1 plugins.go:84] Registered admission > plugin > \"ServiceAccount\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01678609Z"} > {"log":"I0524 14:18:50.016753 1 plugins.go:84] Registered admission > plugin > \"DefaultStorageClass\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016856209Z"} > {"log":"I0524 14:18:50.016784 1 plugins.go:84] Registered admission > plugin > \"PersistentVolumeClaimResize\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01686304Z"} > {"log":"I0524 14:18:50.016801 1 plugins.go:84] Registered admission > plugin > \"StorageObjectInUseProtection\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016865753Z"} > {"log":"F0524 14:19:20.021832 1 start_api.go:68] dial tcp > 192.168.100.100:2379: connect: connection > refused\n","stream":"stderr","time":"2019-05-24T12:19:20.02217046Z"} > > > > *Container log* > > > > [root@master controllers]# tail -f 7.log > {"log":"I0524 14:19:13.744728 1 reflector.go:133] Starting reflector > *v1.PersistentVolumeClaim (0s) from > k8s.io/client-go/informers/factory.go:130\n > <http://k8s.io/client-go/informers/factory.go:130/n> > ","stream":"stderr","time":"2019-05-24T12:19:13.747727009Z"} > {"log":"I0524 14:19:13.744754 1 reflector.go:171] Listing and > watching *v1.PersistentVolumeClaim from > k8s.io/client-go/informers/factory.go:130\n > <http://k8s.io/client-go/informers/factory.go:130/n> > ","stream":"stderr","time":"2019-05-24T12:19:13.74773138Z"} > {"log":"I0524 14:19:13.745323 1 reflector.go:133] Starting reflector > *v1.ReplicationController (0s) from > k8s.io/client-go/informers/factory.go:130\n > <http://k8s.io/client-go/informers/factory.go:130/n> > ","stream":"stderr","time":"2019-05-24T12:19:13.747735832Z"} > {"log":"I0524 14:19:13.745340 1 reflector.go:171] Listing and > watching *v1.ReplicationController from > k8s.io/client-go/informers/factory.go:130\n > <http://k8s.io/client-go/informers/factory.go:130/n> > ","stream":"stderr","time":"2019-05-24T12:19:13.747740084Z"} > {"log":"I0524 14:19:13.745907 1 reflector.go:133] Starting reflector > *v1beta1.ReplicaSet (0s) from k8s.io/client-go/informers/factory.go:130\n > <http://k8s.io/client-go/informers/factory.go:130/n> > ","stream":"stderr","time":"2019-05-24T12:19:13.747744229Z"} > {"log":"I0524 14:19:13.745925 1 reflector.go:171] Listing and > watching *v1beta1.ReplicaSet from > k8s.io/client-go/informers/factory.go:130\n > <http://k8s.io/client-go/informers/factory.go:130/n> > ","stream":"stderr","time":"2019-05-24T12:19:13.747748717Z"} > {"log":"I0524 14:19:13.746647 1 controllermanager.go:128] Version: > v1.11.0+d4cacc0\n","stream":"stderr","time":"2019-05-24T12:19:13.747753221Z"} > {"log":"I0524 14:19:13.746697 1 leaderelection.go:185] attempting to > acquire leader lease kube > -system/kube-controller-manager...\n","stream":"stderr","time":"2019-05-24T12:19:13.747757701Z"} > {"log":"I0524 14:19:13.746889 1 standalone_apiserver.go:101] Started > health checks at 0.0.0.0:8444 > <https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535> > \n","stream":"stderr","time":"2019-05-24T12:19:13.747761834Z"} > {"log":"F0524 14:19:13.747339 1 standalone_apiserver.go:117] listen > tcp4 0.0.0.0:8444 > <https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>: > bind: address already in > use\n","stream":"stderr","time":"2019-05-24T12:19:13.747765655Z"} > > Best regards, J > > > > > > *From:* Samuel Martín Moro <faus...@gmail.com> > *Sent:* Donnerstag, 23. Mai 2019 23:53 > *To:* Jérôme Meyer <jerome.me...@lcsystems.ch> > *Cc:* users@lists.openshift.redhat.com > *Subject:* Re: Issue by installing OKD OpenShift 3.11 > > > > Hi, > > > > > > As a general rule, you may want to check for the corresponding container > health and logs. > > > > You won't find any apache or nginx listening. The process serving on > :8443 is openshift, it should be started in a container. > > Note that the master-api container, in charge of that service, closely > rely on another container: etcd. Which is what ansible's waiting for, in > your logs. > > > > On the master node, use "docker ps" (worst case scenario, "docker ps -a"). > > Locate your etcd and master-api containers ID (first column). > > Then use "docker logs [-f] <container-id>", search for errors. > > > > You may find file copies of these logs in /var/log/containers (and > /var/log/pods). > > > > Let us know how that goes. > > > > And try to avoid mailing your htpasswd entries ;) > > > > > > Regards. > > > > > > > > On Thu, May 23, 2019 at 10:42 AM Jérôme Meyer <jerome.me...@lcsystems.ch> > wrote: > > Dear Team, > > I've encountered some issue to installing openshift (okd 3.11) on 3 vms > (1 master and 2 nodes). > I followed the recommendations and procedure as described in docs. > Then I launched the ansible prerequiste playbook without issue, all was > fine. But unfortunately the deploy_cluster playbook didn't finished. > Some errors appears when he start the pod. > > 2019-05-17 16:58:52,157 p=6592 u=root | FAILED - RETRYING: Wait for > control plane pods to appear (2 retries left). > 2019-05-17 16:58:57,607 p=6592 u=root | FAILED - RETRYING: Wait for > control plane pods to appear (1 retries left). > 2019-05-17 16:59:02,998 p=6592 u=root | failed: [master.lab.oshift.edu] > (item=etcd) => {"attempts": 60, "changed": false, "item": "etcd", "msg": > {"cmd": "/usr/bin/oc get pod master-etcd-master.lab.oshift.edu -o json -n > kube-system", "results": [{}], "returncode": 1, "stderr": "The connection > to the server master:8443 was refused - did you specify the right host or > port?\n", "stdout": ""}} > 2019-05-17 16:59:03,531 p=6592 u=root | FAILED - RETRYING: Wait for > control plane pods to appear (60 retries left). > 2019-05-17 16:59:08,980 p=6592 u=root | FAILED - RETRYING: Wait for > control plane pods to appear (59 retries left). > > Regarding this issue, I've checked the master server and I didn't seen > the http port 8443 open or no http/nginx/or whatever service are running, > strange..... > > > DNS server was installed on a vm called services and the dig command was > ok. > > > > Please let me know if I failed to install something or is the inventory > config wrong? what should I do to troubleshoot this problem? > > Thanks and best regards, J. > > > > > > *Here's the inventory file:* > > > > # cat inventory/hosts > ##################################################################### > # > # HOSTS configuration for our labs > # > # 2019-05-17 > # > ##################################################################### > > [workstation] > workstation.lab.oshift.edu > > [masters] > master.lab.oshift.edu > > [etcd] > master.lab.oshift.edu > > [nodes] > master.lab.oshift.edu openshift_node_group_name="node-config-master" > node1.lab.oshift.edu openshift_node_group_name="node-config-compute" > node2.lab.oshift.edu openshift_node_group_name="node-config-compute" > > [nfs] > services.lab.oshift.edu > > # Create an OSEv3 group that contains the masters and nodes groups > [OSEv3:children] > masters > nodes > etcd > nfs > > [OSEv3:vars] > > ############################################################################### > # Common/ Required configuration variables > follow # > > ############################################################################### > # How ansible access hosts > ansible_user=root > ansible_become=true > > openshift_deployment_type=origin > > openshift_release="3.11" > > openshift_master_default_subdomain=apps.lab.oshift.edu > > > ############################################################################### > # Additional configuration variables > follow # > > ############################################################################### > > # DEBUG > debug_level=4 > > # DISABLE SOME CHECKS > > openshift_disable_check=disk_availability,memory_availability,docker_storage > > # Enable etcd debug logging, defaults to false > etcd_debug=true > # Set etcd log levels by package > etcd_log_package_levels="etcdserver=WARNING,security=INFO" > > # htpasswd auth > openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': > 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}] > # Defining htpasswd users > openshift_master_htpasswd_users={'admin': > '$apr1$Ky/ZY39n$Z8/t3xJsnxGANzypVTtmD0', 'developer': > '$apr1$MdVAOTmy$8nB.ANU4OeciLjDeU68w/1'} > > # Option B - External NFS Host > openshift_hosted_registry_storage_kind=nfs > openshift_hosted_registry_storage_access_modes=['ReadWriteMany'] > openshift_hosted_registry_storage_nfs_directory=/openshift_storage > openshift_hosted_registry_storage_nfs_options='*(rw,root_squash)' > openshift_hosted_registry_storage_volume_name=registry > openshift_hosted_registry_storage_volume_size=10Gi > > # ENABLE FIREWALLD > os_firewall_use_firewalld=true > [root@workstation openshift-ansible]# > > > > _______________________________________________ > users mailing list > users@lists.openshift.redhat.com > http://lists.openshift.redhat.com/openshiftmm/listinfo/users > > > > > -- > > Samuel Martín Moro > {EPITECH.} 2011 > > "Nobody wants to say how this works. > Maybe nobody knows ..." > Xorg.conf(5) > > > > > -- > > Samuel Martín Moro > {EPITECH.} 2011 > > "Nobody wants to say how this works. > Maybe nobody knows ..." > Xorg.conf(5) >
_______________________________________________ users mailing list users@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/users
