One more test (sorry) - inside a debug container, can you
$ curl -k https://172.30.0.1
It should return a list of json responses. If it can't, it either means:
MASTER_IP is not being correctly registered into your endpoints ("oc get
endpoints -n default" should include master ip) or a firewall or other
network connection is blocking access from containers on regular tcp/udp,
but not icmp
If it can, then another service on your host is likely blocking dns on the
container bridge network (which I've never seen be anything other than some
form of firewall).
On Mar 23, 2017, at 1:09 PM, Rishi Misra <[email protected]>
wrote:
Here are the results:
/====================================/
# oc get pods
NAME READY STATUS RESTARTS AGE
vote-1-7acnx 1/1 Running 0 38s
# oc debug pod/vote-1-7acnx
Debugging with pod/vote-1-7acnx-debug, original command: gunicorn app:app
-b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0
Waiting for pod to start ...
Pod IP: 172.17.0.2
If you don't see a command prompt, try pressing enter.
root@vote-1-7acnx-debug:/app# dig @MASTER_IP -p 53
kubernetes.default.svc.cluster.local
; <<>> DiG 9.9.5-9+deb8u10-Debian <<>> @MASTER_IP -p 53
kubernetes.default.svc.cluster.local
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
root@vote-1-7acnx-debug:/app# dig @MASTER_IP -p 53 www.google.com
; <<>> DiG 9.9.5-9+deb8u10-Debian <<>> @MASTER_IP -p 53 www.google.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
root@vote-1-7acnx-debug:/app#
/====================================/
On Thu, Mar 23, 2017 at 12:49 PM, Clayton Coleman <[email protected]>
wrote:
> Ok, can you create a running container (oc debug pod/NAME_OF_POD) and
> inside of it run the same dig commands (you'll need a docker image with dig
> already installed)
>
> On Thu, Mar 23, 2017 at 12:46 PM, Rishi Misra <[email protected]
> > wrote:
>
>> It seems to:
>>
>> /=================================/
>> # dig @MASTER_IP -p 53 kubernetes.default.svc.cluster.local
>>
>> ; <<>> DiG 9.9.4-RedHat-9.9.4-38.el7_3.2 <<>> @MASTER_IP -p 53
>> kubernetes.default.svc.cluster.local
>> ; (1 server found)
>> ;; global options: +cmd
>> ;; Got answer:
>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34034
>> ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
>>
>> ;; QUESTION SECTION:
>> ;kubernetes.default.svc.cluster.local. IN A
>>
>> ;; ANSWER SECTION:
>> kubernetes.default.svc.cluster.local. 30 IN A 172.30.0.1
>>
>> ;; Query time: 0 msec
>> ;; SERVER: MASTER_IP#53(MASTER_IP)
>> ;; WHEN: Thu Mar 23 12:41:04 EDT 2017
>> ;; MSG SIZE rcvd: 70
>>
>> # dig @MASTER_IP -p 53 www.google.com
>>
>> ; <<>> DiG 9.9.4-RedHat-9.9.4-38.el7_3.2 <<>> @MASTER_IP -p 53
>> www.google.com
>> ; (1 server found)
>> ;; global options: +cmd
>> ;; Got answer:
>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28549
>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 4, ADDITIONAL: 1
>>
>> ;; OPT PSEUDOSECTION:
>> ; EDNS: version: 0, flags:; udp: 4096
>> ;; QUESTION SECTION:
>> ;www.google.com. IN A
>>
>> ;; ANSWER SECTION:
>> www.google.com. 6 IN A 74.125.21.147
>> www.google.com. 6 IN A 74.125.21.99
>> www.google.com. 6 IN A 74.125.21.105
>> www.google.com. 6 IN A 74.125.21.104
>> www.google.com. 6 IN A 74.125.21.103
>> www.google.com. 6 IN A 74.125.21.106
>>
>> ;; AUTHORITY SECTION:
>> google.com. 30110 IN NS ns4.google.com.
>> google.com. 30110 IN NS ns3.google.com.
>> google.com. 30110 IN NS ns1.google.com.
>> google.com. 30110 IN NS ns2.google.com.
>>
>> ;; Query time: 1 msec
>> ;; SERVER: MASTER_IP#53(MASTER_IP)
>> ;; WHEN: Thu Mar 23 12:41:55 EDT 2017
>> ;; MSG SIZE rcvd: 211
>>
>> /=================================/
>>
>>
>> On Thu, Mar 23, 2017 at 12:40 PM, Clayton Coleman <[email protected]>
>> wrote:
>>
>>> Can you run:
>>>
>>> $ dig @MASTER_IP -p 53 kubernetes.default.svc.cluster.local
>>>
>>> from the host and verify it works? And if so, then try
>>>
>>> $ dig @MASTER_IP -p 53 www.google.com
>>>
>>>
>>>
>>> On Thu, Mar 23, 2017 at 12:35 PM, Rishi Misra <
>>> [email protected]> wrote:
>>>
>>>> Hi there,
>>>>
>>>> I am having difficulty getting DNS to work on OpenShift 1.4.
>>>>
>>>> When I start all-in-one server now I get following two messages which I
>>>> presume are generated for master and node running on same server:
>>>>
>>>> /==========================================/
>>>> openshift start > /tmp/openshift.log 2>&1
>>>>
>>>> W0323 11:30:52.170724 4883 run_components.go:207] Binding DNS on
>>>> port 8053 instead of 53, which may not be resolvable from all clients
>>>> I0323 11:30:52.171051 4883 logs.go:41] skydns: ready for queries on
>>>> cluster.local. for tcp4://0.0.0.0:8053 [rcache 0]
>>>> I0323 11:30:52.171059 4883 logs.go:41] skydns: ready for queries on
>>>> cluster.local. for udp4://0.0.0.0:8053 [rcache 0]
>>>> I0323 11:30:52.271301 4883 run_components.go:224] DNS listening at
>>>> 0.0.0.0:8053
>>>> ......
>>>>
>>>> I0323 11:30:53.894747 4883 *node.go:358*] Starting DNS on
>>>> x.xx.xx.xx:53
>>>> I0323 11:30:53.897994 4883 logs.go:41] skydns: ready for queries on
>>>> cluster.local. for tcp://9.23.30.33:53 [rcache 0]
>>>> I0323 11:30:53.898003 4883 logs.go:41] skydns: ready for queries on
>>>> cluster.local. for udp://9.23.30.33:53 [rcache 0]
>>>> I0323 11:30:53.926202 4883 start_master.go:703] Started Kubernetes
>>>> Controllers
>>>>
>>>> /==========================================/
>>>>
>>>> How do these two DNS communicate? (8053 and 53)
>>>>
>>>> However for some reason my pods are unable to resolve services by name
>>>> (IP works just fine whether pod ip or external IP). Running these images
>>>> directly on Docker also work fine. I have disabled firewall, configured
>>>> iptables as per docs to no avail.
>>>>
>>>> Here is how my pod behaves:
>>>>
>>>> /=========================================/
>>>> # oc rsh vote-1-0hl7j
>>>> root@vote-1-0hl7j:/app# cat /etc/resolv.conf
>>>> search votingapp.svc.cluster.local svc.cluster.local cluster.local
>>>> nameserver x.xx.xx.xx <---- (This is my server IP where both master and
>>>> node are running)
>>>> options ndots:5
>>>>
>>>> root@vote-1-0hl7j:/app# apt-get update
>>>> 0% [Connecting to httpredir.debian.org] [Connecting to
>>>> security.debian.org]^C
>>>> root@vote-1-0hl7j:/app# apt-get update
>>>> Err http://security.debian.org jessie/updates InRelease
>>>>
>>>> Err http://security.debian.org jessie/updates Release.gpg
>>>> Could not resolve 'security.debian.org'
>>>> /=========================================/
>>>>
>>>> # netstat -tulpn | grep 53
>>>> tcp 0 0 9.23.30.33:53 0.0.0.0:*
>>>> LISTEN 4883/openshift
>>>> tcp 0 0 0.0.0.0:8053 0.0.0.0:*
>>>> LISTEN 4883/openshift
>>>> udp 0 0 0.0.0.0:8053 0.0.0.0:*
>>>> 4883/openshift
>>>> udp 0 0 9.23.30.33:53 0.0.0.0:*
>>>> 4883/openshift
>>>>
>>>>
>>>>
>>>> Please let me know what else to look for .. I am pretty sure it has
>>>> something to do with DNS but I can't figure out how to debug this further.
>>>> I must be missing some config settings.
>>>>
>>>> Thanks.
>>>>
>>>> _______________________________________________
>>>> dev mailing list
>>>> [email protected]
>>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>>>>
>>>>
>>>
>>
>
_______________________________________________
dev mailing list
[email protected]
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
