Hi This is the first time I write to this mailing list, I'd like to say hello to everyone.
I once had a similar issue when installing openshift on my notebook using VirtualBox, I had 2 network interfaces per host (on NATed with internet access and an internal only) and openshift took the "wrong" one. Then I had to set the host variable 'openshift_ip' to explicitly set my ip address to the of the "correct" device. I cannot find it with 3.11 documentation, but within 3.9. https://docs.openshift.com/container-platform/3.9/install_config/install/advanced_install.html#configuring-host-variables regards, Björn Von: users-boun...@lists.openshift.redhat.com <users-boun...@lists.openshift.redhat.com> Im Auftrag von Jérôme Meyer Gesendet: Mittwoch, 29. Mai 2019 17:19 An: Samuel Martín Moro <faus...@gmail.com> Cc: users@lists.openshift.redhat.com Betreff: RE:Issue by installing OKD OpenShift 3.11 Thanks for your help and advise. Unfortunately it don't work yet but perhaps it is a network issue... So, I'll explain more deeply my network architecture... All my VMs are using the default network 192.168.122.0/24 with forwarding NAT to go to the Internet. My Laptop are the 192.168.122.1 and it is the default gateway for all systems too (only one default gateway). This network works with DHCP. Then, I've defined a separate intern subnet to perform the container network: 192.168.100.0/24 as isolated network and internal routing only. This network used static ip address and address are in DNS defined. Here're details: node1 ens10: 192.168.100.101/24 eth1: 192.168.122.193/24 docker0: 172.17.0.1/16 node2 ens10: 192.168.100.102/24 eth1: 192.168.122.240/24 docker0: 172.17.0.1/16 master ens10: 192.168.100.100/24 eth1: 192.168.122.54/24 docker0: 172.17.0.1/16 services ens10: 192.168.100.103/24 eth1: 192.168.122.234/24 docker0: 172.17.0.1/16 I'm connecting and start the ansible's job from my workstation VM 192.168.100.50. Now, if I've right understood, the openshift service will bind http port to the same subnet as the default gateway? In my case, it will be the subnet 192.168.122... ? right? Could it that be the problem? I've defined all ip address for my system in openshift with 192.168.100 subnet. Is that correct? It's possible to use 2 networks has in my case? It's not yet very clear how the network should be configured for openshift hosts. I thought about defining a network for external connection (internet) and a network for internal connection specific to openshift but I'm not sure is it ok... Regards, J ________________________________ De : Samuel Martín Moro [faus...@gmail.com] Envoyé : vendredi 24 mai 2019 21:45 À : Jérôme Meyer Cc : users@lists.openshift.redhat.com<mailto:users@lists.openshift.redhat.com> Objet : Re: Issue by installing OKD OpenShift 3.11 Oh, that makes perfect sense I would assume that your default gateway points to your workstation, in 192.168.100.0/24<http://192.168.100.0/24>? -- although lately, I've seen some inconsistencies: usually, OpenShift services would bind on the address assigned to whichever interface routes to your default gateway. Assuming that switching your default gateway is not an option, then you may force OpenShift bind address from your openshift_node_groups definition. Dealing with that variable in ini format is quite painful, and usually leads to syntax errors, ... First we'll create a "group_vars" sub-folder alongside our inventory. mkdir -p <path-to-inventory-base-directory>/group_vars In that folder, we would create a file OSEv3.yml, with the following content: openshift_node_groups: - name: node-config-master-infra labels: - 'node-role.kubernetes.io/master=true<http://node-role.kubernetes.io/master=true>' - 'node-role.kubernetes.io/infra=true<http://node-role.kubernetes.io/infra=true>' edits: - key: kubeletArguments.node-ip value: [ 192.168.122.54 ] - name: node-config-node1 labels: - 'node-role.kubernetes.io/compute=true<http://node-role.kubernetes.io/compute=true>' edits: - key: kubeletArguments.node-ip value: [ <insert-node1-ip-address> ] - name: node-config-node2 labels: - 'node-role.kubernetes.io/compute=true<http://node-role.kubernetes.io/compute=true>' edits: - key: kubeletArguments.node-ip value: [ <insert-node2-ip-address> ] see ./roles/openshift_facts/defaults/main.yml for the default openshift_node_groups definition, if you're curious. Also make sure that each node from your cluster would load its own configuration: [masters] master.olab.oshift.edu<http://master.olab.oshift.edu> openshift_node_groups_name=node-config-master-infra [etcd:children] masters [compute] node1.olab.oshift.edu<http://node1.olab.oshift.edu> openshift_node_groups_name=node-config-node1 node2.olab.oshift.edu<http://node2.olab.oshift.edu> openshift_node_groups_name=node-config-node2 [nodes:children] masters compute [nfs] ... [OSEv3:children] nodes nfs ... Let us know how that goes. Regards On Fri, May 24, 2019 at 3:05 PM Jérôme Meyer <jerome.me...@lcsystems.ch<mailto:jerome.me...@lcsystems.ch>> wrote: Hi, Thanks for your help and tips. Yeah, I've forgot this time to remove the htpasswd entries.. ;( After changing the master definition as 'node-config-master-infra' in inventory I've restart the deploy-cluster playbook again. As you wrote, I've got the master api and etcd information from docker and checked the logs. So, some questions arises: 1. Why this following address is used : 192.168.122.54? This corresponds to the master interface. It's a nat address using dhcp to connected to my pc. 2. Apparently there're a issue with the etcd access on master: connection refused on 2379. 3. In the last log, it appears that the request is made on the ip address 0.0.0.0:8444<https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>, something is wrong in my config? Here're the ip interfaces list of master; where the 192.168.100.100 is the communication network for openshift as defined in hostname and DNS. Interface list [root@master ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8<http://127.0.0.1/8> scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:ca:44:c8 brd ff:ff:ff:ff:ff:ff inet 192.168.100.100/24<http://192.168.100.100/24> brd 192.168.100.255 scope global noprefixroute ens10 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:feca:44c8/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:a8:8b:00 brd ff:ff:ff:ff:ff:ff inet 192.168.122.54/24<http://192.168.122.54/24> brd 192.168.122.255 scope global noprefixroute dynamic eth1 valid_lft 3090sec preferred_lft 3090sec inet6 fe80::c138:7cb0:f8af:7cba/64 scope link noprefixroute valid_lft forever preferred_lft forever 4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:a9:c9:8d:d3 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16<http://172.17.0.1/16> scope global docker0 valid_lft forever preferred_lft forever Log from etcd-master {"log":"2019-05-24 14:19:57.592591 D | etcdserver/api/v2http: [GET] /health remote:192.168.122.54:44748<http://192.168.122.54:44748>\n","stream":"stderr","time":"2019-05-24T12:19:57.592680803Z"} {"log":"2019-05-24 14:20:07.580420 D | etcdserver/api/v2http: [GET] /v2/members remote:192.168.122.54:45038<http://192.168.122.54:45038>\n","stream":"stderr","time":"2019-05-24T12:20:07.580688397Z"} {"log":"2019-05-24 14:20:07.590218 D | etcdserver/api/v2http: [GET] /health remote:192.168.122.54:45040<http://192.168.122.54:45040>\n","stream":"stderr","time":"2019-05-24T12:20:07.590356315Z"} {"log":"2019-05-24 14:20:17.582661 D | etcdserver/api/v2http: [GET] /v2/members remote:192.168.122.54:45336<http://192.168.122.54:45336>\n","stream":"stderr","time":"2019-05-24T12:20:17.582774753Z"} {"log":"2019-05-24 14:20:17.595674 D | etcdserver/api/v2http: [GET] /health remote:192.168.122.54:45338<http://192.168.122.54:45338>\n","stream":"stderr","time":"2019-05-24T12:20:17.595844742Z"} {"log":"2019-05-24 14:20:27.581915 D | etcdserver/api/v2http: [GET] /v2/members remote:192.168.122.54:45638<http://192.168.122.54:45638>\n","stream":"stderr","time":"2019-05-24T12:20:27.582036442Z"} {"log":"2019-05-24 14:20:27.592091 D | etcdserver/api/v2http: [GET] /health remote:192.168.122.54:45640<http://192.168.122.54:45640>\n","stream":"stderr","time":"2019-05-24T12:20:27.59225275Z"} {"log":"2019-05-24 14:20:37.584090 D | etcdserver/api/v2http: [GET] /v2/members remote:192.168.122.54:45932<http://192.168.122.54:45932>\n","stream":"stderr","time":"2019-05-24T12:20:37.584291782Z"} {"log":"2019-05-24 14:20:37.593862 D | etcdserver/api/v2http: [GET] /health remote:192.168.122.54:45934<http://192.168.122.54:45934>\n","stream":"stderr","time":"2019-05-24T12:20:37.593980682Z"} Log from api-master {"log":"I0524 14:18:50.016547 1 plugins.go:84] Registered admission plugin \"ResourceQuota\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016617699Z"} {"log":"I0524 14:18:50.016581 1 plugins.go:84] Registered admission plugin \"PodSecurityPolicy\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016622959Z"} {"log":"I0524 14:18:50.016622 1 plugins.go:84] Registered admission plugin \"Priority\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016659601Z"} {"log":"I0524 14:18:50.016662 1 plugins.go:84] Registered admission plugin \"SecurityContextDeny\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01670916Z"} {"log":"I0524 14:18:50.016713 1 plugins.go:84] Registered admission plugin \"ServiceAccount\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01678609Z"} {"log":"I0524 14:18:50.016753 1 plugins.go:84] Registered admission plugin \"DefaultStorageClass\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016856209Z"} {"log":"I0524 14:18:50.016784 1 plugins.go:84] Registered admission plugin \"PersistentVolumeClaimResize\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01686304Z"} {"log":"I0524 14:18:50.016801 1 plugins.go:84] Registered admission plugin \"StorageObjectInUseProtection\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016865753Z"} {"log":"F0524 14:19:20.021832 1 start_api.go:68] dial tcp 192.168.100.100:2379<http://192.168.100.100:2379>: connect: connection refused\n","stream":"stderr","time":"2019-05-24T12:19:20.02217046Z"} Container log [root@master controllers]# tail -f 7.log {"log":"I0524 14:19:13.744728 1 reflector.go:133] Starting reflector *v1.PersistentVolumeClaim (0s) from k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747727009Z"} {"log":"I0524 14:19:13.744754 1 reflector.go:171] Listing and watching *v1.PersistentVolumeClaim from k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.74773138Z"} {"log":"I0524 14:19:13.745323 1 reflector.go:133] Starting reflector *v1.ReplicationController (0s) from k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747735832Z"} {"log":"I0524 14:19:13.745340 1 reflector.go:171] Listing and watching *v1.ReplicationController from k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747740084Z"} {"log":"I0524 14:19:13.745907 1 reflector.go:133] Starting reflector *v1beta1.ReplicaSet (0s) from k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747744229Z"} {"log":"I0524 14:19:13.745925 1 reflector.go:171] Listing and watching *v1beta1.ReplicaSet from k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747748717Z"} {"log":"I0524 14:19:13.746647 1 controllermanager.go:128] Version: v1.11.0+d4cacc0\n","stream":"stderr","time":"2019-05-24T12:19:13.747753221Z"} {"log":"I0524 14:19:13.746697 1 leaderelection.go:185] attempting to acquire leader lease kube-system/kube-controller-manager...\n","stream":"stderr","time":"2019-05-24T12:19:13.747757701Z"} {"log":"I0524 14:19:13.746889 1 standalone_apiserver.go:101] Started health checks at 0.0.0.0:8444<https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>\n","stream":"stderr","time":"2019-05-24T12:19:13.747761834Z"} {"log":"F0524 14:19:13.747339 1 standalone_apiserver.go:117] listen tcp4 0.0.0.0:8444<https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>: bind: address already in use\n","stream":"stderr","time":"2019-05-24T12:19:13.747765655Z"} Best regards, J From: Samuel Martín Moro <faus...@gmail.com<mailto:faus...@gmail.com>> Sent: Donnerstag, 23. Mai 2019 23:53 To: Jérôme Meyer <jerome.me...@lcsystems.ch<mailto:jerome.me...@lcsystems.ch>> Cc: users@lists.openshift.redhat.com<mailto:users@lists.openshift.redhat.com> Subject: Re: Issue by installing OKD OpenShift 3.11 Hi, As a general rule, you may want to check for the corresponding container health and logs. You won't find any apache or nginx listening. The process serving on :8443 is openshift, it should be started in a container. Note that the master-api container, in charge of that service, closely rely on another container: etcd. Which is what ansible's waiting for, in your logs. On the master node, use "docker ps" (worst case scenario, "docker ps -a"). Locate your etcd and master-api containers ID (first column). Then use "docker logs [-f] <container-id>", search for errors. You may find file copies of these logs in /var/log/containers (and /var/log/pods). Let us know how that goes. And try to avoid mailing your htpasswd entries ;) Regards. On Thu, May 23, 2019 at 10:42 AM Jérôme Meyer <jerome.me...@lcsystems.ch<mailto:jerome.me...@lcsystems.ch>> wrote: Dear Team, I've encountered some issue to installing openshift (okd 3.11) on 3 vms (1 master and 2 nodes). I followed the recommendations and procedure as described in docs. Then I launched the ansible prerequiste playbook without issue, all was fine. But unfortunately the deploy_cluster playbook didn't finished. Some errors appears when he start the pod. 2019-05-17 16:58:52,157 p=6592 u=root | FAILED - RETRYING: Wait for control plane pods to appear (2 retries left). 2019-05-17 16:58:57,607 p=6592 u=root | FAILED - RETRYING: Wait for control plane pods to appear (1 retries left). 2019-05-17 16:59:02,998 p=6592 u=root | failed: [master.lab.oshift.edu<http://master.lab.oshift.edu>] (item=etcd) => {"attempts": 60, "changed": false, "item": "etcd", "msg": {"cmd": "/usr/bin/oc get pod master-etcd-master.lab.oshift.edu<http://master-etcd-master.lab.oshift.edu> -o json -n kube-system", "results": [{}], "returncode": 1, "stderr": "The connection to the server master:8443 was refused - did you specify the right host or port?\n", "stdout": ""}} 2019-05-17 16:59:03,531 p=6592 u=root | FAILED - RETRYING: Wait for control plane pods to appear (60 retries left). 2019-05-17 16:59:08,980 p=6592 u=root | FAILED - RETRYING: Wait for control plane pods to appear (59 retries left). Regarding this issue, I've checked the master server and I didn't seen the http port 8443 open or no http/nginx/or whatever service are running, strange..... DNS server was installed on a vm called services and the dig command was ok. Please let me know if I failed to install something or is the inventory config wrong? what should I do to troubleshoot this problem? Thanks and best regards, J. Here's the inventory file: # cat inventory/hosts ##################################################################### # # HOSTS configuration for our labs # # 2019-05-17 # ##################################################################### [workstation] workstation.lab.oshift.edu<http://workstation.lab.oshift.edu> [masters] master.lab.oshift.edu<http://master.lab.oshift.edu> [etcd] master.lab.oshift.edu<http://master.lab.oshift.edu> [nodes] master.lab.oshift.edu<http://master.lab.oshift.edu> openshift_node_group_name="node-config-master" node1.lab.oshift.edu<http://node1.lab.oshift.edu> openshift_node_group_name="node-config-compute" node2.lab.oshift.edu<http://node2.lab.oshift.edu> openshift_node_group_name="node-config-compute" [nfs] services.lab.oshift.edu<http://services.lab.oshift.edu> # Create an OSEv3 group that contains the masters and nodes groups [OSEv3:children] masters nodes etcd nfs [OSEv3:vars] ############################################################################### # Common/ Required configuration variables follow # ############################################################################### # How ansible access hosts ansible_user=root ansible_become=true openshift_deployment_type=origin openshift_release="3.11" openshift_master_default_subdomain=apps.lab.oshift.edu<http://apps.lab.oshift.edu> ############################################################################### # Additional configuration variables follow # ############################################################################### # DEBUG debug_level=4 # DISABLE SOME CHECKS openshift_disable_check=disk_availability,memory_availability,docker_storage # Enable etcd debug logging, defaults to false etcd_debug=true # Set etcd log levels by package etcd_log_package_levels="etcdserver=WARNING,security=INFO" # htpasswd auth openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}] # Defining htpasswd users openshift_master_htpasswd_users={'admin': '$apr1$Ky/ZY39n$Z8/t3xJsnxGANzypVTtmD0', 'developer': '$apr1$MdVAOTmy$8nB.ANU4OeciLjDeU68w/1'} # Option B - External NFS Host openshift_hosted_registry_storage_kind=nfs openshift_hosted_registry_storage_access_modes=['ReadWriteMany'] openshift_hosted_registry_storage_nfs_directory=/openshift_storage openshift_hosted_registry_storage_nfs_options='*(rw,root_squash)' openshift_hosted_registry_storage_volume_name=registry openshift_hosted_registry_storage_volume_size=10Gi # ENABLE FIREWALLD os_firewall_use_firewalld=true [root@workstation openshift-ansible]# _______________________________________________ users mailing list users@lists.openshift.redhat.com<mailto:users@lists.openshift.redhat.com> http://lists.openshift.redhat.com/openshiftmm/listinfo/users -- Samuel Martín Moro {EPITECH.} 2011 "Nobody wants to say how this works. Maybe nobody knows ..." Xorg.conf(5) -- Samuel Martín Moro {EPITECH.} 2011 "Nobody wants to say how this works. Maybe nobody knows ..." Xorg.conf(5)
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ users mailing list users@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/users
