Hi

This is the first time I write to this mailing list, I'd like to say hello to 
everyone.

I once had a similar issue when installing openshift on my notebook using 
VirtualBox, I had 2 network interfaces per host (on NATed with internet access 
and an internal only) and openshift took the "wrong" one. Then I had to set the 
host variable 'openshift_ip' to explicitly set my ip address to the of the 
"correct" device.

I cannot find it with 3.11 documentation, but within 3.9.

https://docs.openshift.com/container-platform/3.9/install_config/install/advanced_install.html#configuring-host-variables

regards, Björn


Von: users-boun...@lists.openshift.redhat.com 
<users-boun...@lists.openshift.redhat.com> Im Auftrag von Jérôme Meyer
Gesendet: Mittwoch, 29. Mai 2019 17:19
An: Samuel Martín Moro <faus...@gmail.com>
Cc: users@lists.openshift.redhat.com
Betreff: RE:Issue by installing OKD OpenShift 3.11


Thanks for your help and advise.
Unfortunately it don't work yet but perhaps it is a network issue...

So, I'll explain more deeply my network architecture...

All my VMs are using the default network 192.168.122.0/24 with forwarding NAT 
to go to the Internet. My Laptop are the 192.168.122.1 and it is the default 
gateway for all systems too (only one default gateway). This network works with 
DHCP.

Then, I've defined a separate intern subnet to perform the container network: 
192.168.100.0/24 as isolated network and internal routing only. This network 
used static ip address and address are in DNS defined.

Here're details:

node1
ens10: 192.168.100.101/24
eth1: 192.168.122.193/24
docker0: 172.17.0.1/16

node2
ens10: 192.168.100.102/24
eth1: 192.168.122.240/24
docker0: 172.17.0.1/16

master
ens10: 192.168.100.100/24
eth1: 192.168.122.54/24
docker0: 172.17.0.1/16

services
ens10: 192.168.100.103/24
eth1: 192.168.122.234/24
docker0: 172.17.0.1/16

I'm connecting and start the ansible's job from my workstation VM 
192.168.100.50.

Now, if I've right understood, the openshift service will bind http port to the 
same subnet as the default gateway? In my case, it will be the subnet 
192.168.122... ? right?
Could it that be the problem?

I've defined all ip address for my system in openshift with 192.168.100 subnet. 
Is that correct?
It's possible to use 2 networks has in my case?

It's not yet very clear how the network should be configured for openshift 
hosts. I thought about defining a network for external connection (internet) 
and a network for internal connection specific to openshift but I'm not sure is 
it ok...

Regards, J







________________________________
De : Samuel Martín Moro [faus...@gmail.com]
Envoyé : vendredi 24 mai 2019 21:45
À : Jérôme Meyer
Cc : users@lists.openshift.redhat.com<mailto:users@lists.openshift.redhat.com>
Objet : Re: Issue by installing OKD OpenShift 3.11
Oh, that makes perfect sense
I would assume that your default gateway points to your workstation, in 
192.168.100.0/24<http://192.168.100.0/24>?

-- although lately, I've seen some inconsistencies: usually, OpenShift services 
would bind on the address assigned to whichever interface routes to your 
default gateway.

Assuming that switching your default gateway is not an option, then you may 
force OpenShift bind address from your openshift_node_groups definition.
Dealing with that variable in ini format is quite painful, and usually leads to 
syntax errors, ... First we'll create a "group_vars" sub-folder alongside our 
inventory.


mkdir -p <path-to-inventory-base-directory>/group_vars



In that folder, we would create a file OSEv3.yml, with the following content:


openshift_node_groups:
- name: node-config-master-infra
  labels:
    - 
'node-role.kubernetes.io/master=true<http://node-role.kubernetes.io/master=true>'
    - 
'node-role.kubernetes.io/infra=true<http://node-role.kubernetes.io/infra=true>'
  edits:
  - key: kubeletArguments.node-ip
    value: [ 192.168.122.54 ]
- name: node-config-node1
  labels:
    - 
'node-role.kubernetes.io/compute=true<http://node-role.kubernetes.io/compute=true>'
  edits:
  - key: kubeletArguments.node-ip
    value: [ <insert-node1-ip-address> ]
- name: node-config-node2
  labels:
    - 
'node-role.kubernetes.io/compute=true<http://node-role.kubernetes.io/compute=true>'
  edits:
  - key: kubeletArguments.node-ip
    value: [ <insert-node2-ip-address> ]



see ./roles/openshift_facts/defaults/main.yml for the default 
openshift_node_groups definition, if you're curious.

Also make sure that each node from your cluster would load its own 
configuration:



[masters]
master.olab.oshift.edu<http://master.olab.oshift.edu> 
openshift_node_groups_name=node-config-master-infra

[etcd:children]
masters

[compute]
node1.olab.oshift.edu<http://node1.olab.oshift.edu> 
openshift_node_groups_name=node-config-node1
node2.olab.oshift.edu<http://node2.olab.oshift.edu> 
openshift_node_groups_name=node-config-node2

[nodes:children]
masters
compute

[nfs]
...

[OSEv3:children]
nodes
nfs

...


Let us know how that goes.


Regards



On Fri, May 24, 2019 at 3:05 PM Jérôme Meyer 
<jerome.me...@lcsystems.ch<mailto:jerome.me...@lcsystems.ch>> wrote:
Hi,

Thanks for your help and tips. Yeah, I've forgot this time to remove the 
htpasswd entries.. ;(

After changing the master definition as 'node-config-master-infra' in inventory 
I've restart the deploy-cluster playbook again.
As you wrote, I've got the master api and etcd information from docker and 
checked the logs.

So, some questions arises:


  1.  Why this following address is used : 192.168.122.54? This corresponds to 
the master interface. It's a nat address using dhcp to connected to my pc.
  2.  Apparently there're a issue with the etcd access on master: connection 
refused on 2379.
  3.  In the last log, it appears that the request is made on the ip address 
0.0.0.0:8444<https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>,
 something is wrong in my config?

Here're the ip interfaces list of master; where the 192.168.100.100 is the 
communication network for openshift as defined in hostname and DNS.

Interface list

[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group 
default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8<http://127.0.0.1/8> scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP 
group default qlen 1000
    link/ether 52:54:00:ca:44:c8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.100/24<http://192.168.100.100/24> brd 192.168.100.255 
scope global noprefixroute ens10
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:feca:44c8/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP 
group default qlen 1000
    link/ether 52:54:00:a8:8b:00 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.54/24<http://192.168.122.54/24> brd 192.168.122.255 scope 
global noprefixroute dynamic eth1
       valid_lft 3090sec preferred_lft 3090sec
    inet6 fe80::c138:7cb0:f8af:7cba/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state 
DOWN group default
    link/ether 02:42:a9:c9:8d:d3 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16<http://172.17.0.1/16> scope global docker0
       valid_lft forever preferred_lft forever

Log from etcd-master


{"log":"2019-05-24 14:19:57.592591 D | etcdserver/api/v2http: [GET] /health 
remote:192.168.122.54:44748<http://192.168.122.54:44748>\n","stream":"stderr","time":"2019-05-24T12:19:57.592680803Z"}
{"log":"2019-05-24 14:20:07.580420 D | etcdserver/api/v2http: [GET] /v2/members 
remote:192.168.122.54:45038<http://192.168.122.54:45038>\n","stream":"stderr","time":"2019-05-24T12:20:07.580688397Z"}
{"log":"2019-05-24 14:20:07.590218 D | etcdserver/api/v2http: [GET] /health 
remote:192.168.122.54:45040<http://192.168.122.54:45040>\n","stream":"stderr","time":"2019-05-24T12:20:07.590356315Z"}
{"log":"2019-05-24 14:20:17.582661 D | etcdserver/api/v2http: [GET] /v2/members 
remote:192.168.122.54:45336<http://192.168.122.54:45336>\n","stream":"stderr","time":"2019-05-24T12:20:17.582774753Z"}
{"log":"2019-05-24 14:20:17.595674 D | etcdserver/api/v2http: [GET] /health 
remote:192.168.122.54:45338<http://192.168.122.54:45338>\n","stream":"stderr","time":"2019-05-24T12:20:17.595844742Z"}
{"log":"2019-05-24 14:20:27.581915 D | etcdserver/api/v2http: [GET] /v2/members 
remote:192.168.122.54:45638<http://192.168.122.54:45638>\n","stream":"stderr","time":"2019-05-24T12:20:27.582036442Z"}
{"log":"2019-05-24 14:20:27.592091 D | etcdserver/api/v2http: [GET] /health 
remote:192.168.122.54:45640<http://192.168.122.54:45640>\n","stream":"stderr","time":"2019-05-24T12:20:27.59225275Z"}
{"log":"2019-05-24 14:20:37.584090 D | etcdserver/api/v2http: [GET] /v2/members 
remote:192.168.122.54:45932<http://192.168.122.54:45932>\n","stream":"stderr","time":"2019-05-24T12:20:37.584291782Z"}
{"log":"2019-05-24 14:20:37.593862 D | etcdserver/api/v2http: [GET] /health 
remote:192.168.122.54:45934<http://192.168.122.54:45934>\n","stream":"stderr","time":"2019-05-24T12:20:37.593980682Z"}

Log from api-master

{"log":"I0524 14:18:50.016547       1 plugins.go:84] Registered admission 
plugin 
\"ResourceQuota\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016617699Z"}
{"log":"I0524 14:18:50.016581       1 plugins.go:84] Registered admission 
plugin 
\"PodSecurityPolicy\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016622959Z"}
{"log":"I0524 14:18:50.016622       1 plugins.go:84] Registered admission 
plugin 
\"Priority\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016659601Z"}
{"log":"I0524 14:18:50.016662       1 plugins.go:84] Registered admission 
plugin 
\"SecurityContextDeny\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01670916Z"}
{"log":"I0524 14:18:50.016713       1 plugins.go:84] Registered admission 
plugin 
\"ServiceAccount\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01678609Z"}
{"log":"I0524 14:18:50.016753       1 plugins.go:84] Registered admission 
plugin 
\"DefaultStorageClass\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016856209Z"}
{"log":"I0524 14:18:50.016784       1 plugins.go:84] Registered admission 
plugin 
\"PersistentVolumeClaimResize\"\n","stream":"stderr","time":"2019-05-24T12:18:50.01686304Z"}
{"log":"I0524 14:18:50.016801       1 plugins.go:84] Registered admission 
plugin 
\"StorageObjectInUseProtection\"\n","stream":"stderr","time":"2019-05-24T12:18:50.016865753Z"}
{"log":"F0524 14:19:20.021832       1 start_api.go:68] dial tcp 
192.168.100.100:2379<http://192.168.100.100:2379>: connect: connection 
refused\n","stream":"stderr","time":"2019-05-24T12:19:20.02217046Z"}

Container log

[root@master controllers]# tail -f 7.log
{"log":"I0524 14:19:13.744728       1 reflector.go:133] Starting reflector 
*v1.PersistentVolumeClaim (0s) from 
k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747727009Z"}
{"log":"I0524 14:19:13.744754       1 reflector.go:171] Listing and watching 
*v1.PersistentVolumeClaim from 
k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.74773138Z"}
{"log":"I0524 14:19:13.745323       1 reflector.go:133] Starting reflector 
*v1.ReplicationController (0s) from 
k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747735832Z"}
{"log":"I0524 14:19:13.745340       1 reflector.go:171] Listing and watching 
*v1.ReplicationController from 
k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747740084Z"}
{"log":"I0524 14:19:13.745907       1 reflector.go:133] Starting reflector 
*v1beta1.ReplicaSet (0s) from 
k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747744229Z"}
{"log":"I0524 14:19:13.745925       1 reflector.go:171] Listing and watching 
*v1beta1.ReplicaSet from 
k8s.io/client-go/informers/factory.go:130\n<http://k8s.io/client-go/informers/factory.go:130/n>","stream":"stderr","time":"2019-05-24T12:19:13.747748717Z"}
{"log":"I0524 14:19:13.746647       1 controllermanager.go:128] Version: 
v1.11.0+d4cacc0\n","stream":"stderr","time":"2019-05-24T12:19:13.747753221Z"}
{"log":"I0524 14:19:13.746697       1 leaderelection.go:185] attempting to 
acquire leader lease  
kube-system/kube-controller-manager...\n","stream":"stderr","time":"2019-05-24T12:19:13.747757701Z"}
{"log":"I0524 14:19:13.746889       1 standalone_apiserver.go:101] Started 
health checks at 
0.0.0.0:8444<https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>\n","stream":"stderr","time":"2019-05-24T12:19:13.747761834Z"}
{"log":"F0524 14:19:13.747339       1 standalone_apiserver.go:117] listen tcp4 
0.0.0.0:8444<https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2f0.0.0.0%3a8444&umid=0e9c0b30-9ea4-4924-ae7d-cd3ece23a0ba&auth=f2aeef1e705192504f558e668703ea9246add7c9-495450228bf01359b2a33e6cd11c0fca64a55535>:
 bind: address already in 
use\n","stream":"stderr","time":"2019-05-24T12:19:13.747765655Z"}
Best regards, J


From: Samuel Martín Moro <faus...@gmail.com<mailto:faus...@gmail.com>>
Sent: Donnerstag, 23. Mai 2019 23:53
To: Jérôme Meyer <jerome.me...@lcsystems.ch<mailto:jerome.me...@lcsystems.ch>>
Cc: users@lists.openshift.redhat.com<mailto:users@lists.openshift.redhat.com>
Subject: Re: Issue by installing OKD OpenShift 3.11

Hi,


As a general rule, you may want to check for the corresponding container health 
and logs.

You won't find any apache or nginx listening. The process serving on :8443 is 
openshift, it should be started in a container.
Note that the master-api container, in charge of that service, closely rely on 
another container: etcd. Which is what ansible's waiting for, in your logs.

On the master node, use "docker ps" (worst case scenario, "docker ps -a").
Locate your etcd and master-api containers ID (first column).
Then use "docker logs [-f] <container-id>", search for errors.

You may find file copies of these logs in /var/log/containers (and 
/var/log/pods).

Let us know how that goes.

And try to avoid mailing your htpasswd entries ;)


Regards.



On Thu, May 23, 2019 at 10:42 AM Jérôme Meyer 
<jerome.me...@lcsystems.ch<mailto:jerome.me...@lcsystems.ch>> wrote:
Dear Team,

I've encountered some issue to installing openshift (okd 3.11) on 3 vms (1 
master and 2 nodes).
I followed the recommendations and procedure as described in docs.
Then I launched the ansible prerequiste playbook without issue, all was fine. 
But unfortunately the deploy_cluster playbook didn't finished.
Some errors appears when he start the pod.

2019-05-17 16:58:52,157 p=6592 u=root |  FAILED - RETRYING: Wait for control 
plane pods to appear (2 retries left).
2019-05-17 16:58:57,607 p=6592 u=root |  FAILED - RETRYING: Wait for control 
plane pods to appear (1 retries left).
2019-05-17 16:59:02,998 p=6592 u=root |  failed: 
[master.lab.oshift.edu<http://master.lab.oshift.edu>] (item=etcd) => 
{"attempts": 60, "changed": false, "item": "etcd", "msg": {"cmd": "/usr/bin/oc 
get pod 
master-etcd-master.lab.oshift.edu<http://master-etcd-master.lab.oshift.edu> -o 
json -n kube-system", "results": [{}], "returncode": 1, "stderr": "The 
connection to the server master:8443 was refused - did you specify the right 
host or port?\n", "stdout": ""}}
2019-05-17 16:59:03,531 p=6592 u=root |  FAILED - RETRYING: Wait for control 
plane pods to appear (60 retries left).
2019-05-17 16:59:08,980 p=6592 u=root |  FAILED - RETRYING: Wait for control 
plane pods to appear (59 retries left).
Regarding this issue, I've checked the master server and I didn't seen the http 
port 8443 open or no http/nginx/or whatever service are running, strange.....

DNS server was installed on a vm called services and the dig command was ok.

Please let me know if I failed to install something or is the inventory config 
wrong? what should I do to troubleshoot this problem?
Thanks and best regards, J.


Here's the inventory file:


# cat inventory/hosts
#####################################################################
#
# HOSTS configuration for our labs
#
# 2019-05-17
#
#####################################################################

[workstation]
workstation.lab.oshift.edu<http://workstation.lab.oshift.edu>

[masters]
master.lab.oshift.edu<http://master.lab.oshift.edu>

[etcd]
master.lab.oshift.edu<http://master.lab.oshift.edu>

[nodes]
master.lab.oshift.edu<http://master.lab.oshift.edu> 
openshift_node_group_name="node-config-master"
node1.lab.oshift.edu<http://node1.lab.oshift.edu> 
openshift_node_group_name="node-config-compute"
node2.lab.oshift.edu<http://node2.lab.oshift.edu> 
openshift_node_group_name="node-config-compute"

[nfs]
services.lab.oshift.edu<http://services.lab.oshift.edu>

# Create an OSEv3 group that contains the masters and nodes groups
[OSEv3:children]
masters
nodes
etcd
nfs

[OSEv3:vars]
###############################################################################
# Common/ Required configuration variables follow                             #
###############################################################################
# How ansible access hosts
ansible_user=root
ansible_become=true

openshift_deployment_type=origin

openshift_release="3.11"

openshift_master_default_subdomain=apps.lab.oshift.edu<http://apps.lab.oshift.edu>

###############################################################################
# Additional configuration variables follow                                   #
###############################################################################

# DEBUG
debug_level=4

# DISABLE SOME CHECKS
openshift_disable_check=disk_availability,memory_availability,docker_storage

# Enable etcd debug logging, defaults to false
etcd_debug=true
# Set etcd log levels by package
etcd_log_package_levels="etcdserver=WARNING,security=INFO"

# htpasswd auth
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 
'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
# Defining htpasswd users
openshift_master_htpasswd_users={'admin': 
'$apr1$Ky/ZY39n$Z8/t3xJsnxGANzypVTtmD0', 'developer': 
'$apr1$MdVAOTmy$8nB.ANU4OeciLjDeU68w/1'}

# Option B - External NFS Host
openshift_hosted_registry_storage_kind=nfs
openshift_hosted_registry_storage_access_modes=['ReadWriteMany']
openshift_hosted_registry_storage_nfs_directory=/openshift_storage
openshift_hosted_registry_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_registry_storage_volume_name=registry
openshift_hosted_registry_storage_volume_size=10Gi

# ENABLE FIREWALLD
os_firewall_use_firewalld=true
[root@workstation openshift-ansible]#

_______________________________________________
users mailing list
users@lists.openshift.redhat.com<mailto:users@lists.openshift.redhat.com>
http://lists.openshift.redhat.com/openshiftmm/listinfo/users


--
Samuel Martín Moro
{EPITECH.} 2011

"Nobody wants to say how this works.
 Maybe nobody knows ..."
                      Xorg.conf(5)


--
Samuel Martín Moro
{EPITECH.} 2011

"Nobody wants to say how this works.
 Maybe nobody knows ..."
                      Xorg.conf(5)

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

_______________________________________________
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Reply via email to