I've gotten this working again, but I'm not sure how/why it stopped working
in the first place but I think ti was somehow related to the secrets
created with the registry. Perhaps one of them got changed or corrupted
(maybe when I took the nodes down to patch last week). Anyway, I was
deleting the registry like this:
oc delete dc/docker-registry
oc delete svc/docker-registry
oc delete sa/registry
for i in $(oc get secrets | grep registry | awk '{print $1}'); do oc
delete secret/$i; done # didn't do this the first few times.
systemctl restart origin-master
And recreating like this (based on
https://docs.openshift.org/latest/install_config/install/docker_registry.html
):
echo
'{"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"registry"}}'
| oc create -n default -f -
oadm policy add-scc-to-user privileged registry
oadm registry --replicas=1 --config=/etc/origin/master/admin.kubeconfig
--credentials=/etc/origin/master/openshift-registry.kubeconfig
oc volume deploymentconfigs/docker-registry --add --overwrite
--name=registry-storage --mount-path=/registry --source='{"nfs": {
"server": "mynfsserver.example.com", "path": "/my/registry/mnt/path"}}'
oc get -o yaml svc docker-registry | sed
's/\(sessionAffinity:\s*\).*/\1ClientIP/' | oc replace -f -
The key thing I changed that seem to fix the problem was to add a step that
deleted all of the registry* secrets as part of the clean-up phase.
When I was troubleshooting it was a bit of a mystery to me where the
various "dockercfg" secrets were coming from and if it was safe to delete
them. Are those generated as-needed? I never deleted or recreated the
"builder-dockercfg-*" secret in my projects despite rebuilding the registry
multiple times. If they were deleted would they be recreated?
Thanks for your help, Clayton. I appreciate it.
--
Robert Wehner
Return Path
On Wed, Mar 16, 2016 at 12:51 PM, Clayton Coleman <ccole...@redhat.com>
wrote:
> How did you create the registry? Do you have a set of ENV vars
> defined on the registry deployment config?
>
> On Wed, Mar 16, 2016 at 2:44 PM, Robert Wehner
> <robert.weh...@returnpath.com> wrote:
> > It looks like it is "oadm v1.1.2-1-gbe558b1".
> >
> >
> > --
> > Robert Wehner
> > Return Path
> >
> > On Wed, Mar 16, 2016 at 12:43 PM, Clayton Coleman <ccole...@redhat.com>
> > wrote:
> >>
> >> What version of oadm are you running?
> >>
> >> On Wed, Mar 16, 2016 at 2:32 PM, Robert Wehner
> >> <robert.weh...@returnpath.com> wrote:
> >> > I'm having trouble getting my builds to push to an internal registry.
> >> > I'm
> >> > running Origin v1.1.2. When I run through the nodejs example
> >> >
> >> > (
> https://docs.openshift.org/latest/getting_started/developers/developers_console.html
> ),
> >> > the build always fails pushing to the internal registry with this
> error:
> >> >
> >> > I0316 18:22:06.442945 1 sti.go:314] Successfully built
> >> > 172.30.223.171:5000/-example/nodejs-ex:latest
> >> > I0316 18:22:06.472264 1 cleanup.go:23] Removing temporary
> >> > directory
> >> > /tmp/s2i-build081599111
> >> > I0316 18:22:06.472314 1 fs.go:156] Removing directory
> >> > '/tmp/s2i-build081599111'
> >> > I0316 18:22:06.475260 1 cfg.go:45] Locating docker auth for
> image
> >> > 172.30.223.171:5000/-example/nodejs-ex:latest and type
> >> > PUSH_DOCKERCFG_PATH
> >> > I0316 18:22:06.475743 1 cfg.go:83] Using serviceaccount user for
> >> > Docker authentication for image
> >> > 172.30.223.171:5000/-example/nodejs-ex:latest
> >> > I0316 18:22:06.475769 1 sti.go:229] Using provided push secret
> for
> >> > pushing 172.30.223.171:5000/-example/nodejs-ex:latest image
> >> > I0316 18:22:06.475782 1 sti.go:233] Pushing
> >> > 172.30.223.171:5000/-example/nodejs-ex:latest image ...
> >> > I0316 18:22:08.276420 1 sti.go:238] Registry server Address:
> >> > I0316 18:22:08.276451 1 sti.go:239] Registry server User Name:
> >> > serviceaccount
> >> > I0316 18:22:08.276464 1 sti.go:240] Registry server Email:
> >> > serviceacco...@example.org
> >> > I0316 18:22:08.276476 1 sti.go:245] Registry server Password:
> >> > <<non-empty>>
> >> > F0316 18:22:08.276507 1 builder.go:202] Error: build error:
> Failed
> >> > to
> >> > push image. Response from registry is: Received unexpected HTTP
> status:
> >> > 500
> >> > Internal Server Error
> >> >
> >> > At the same time the registry pod logs the following error about not
> >> > being
> >> > able to create an imagestreammap:
> >> >
> >> > time="2016-03-16T18:22:08.273960074Z" level=error msg="response
> >> > completed
> >> > with error" err.code=UNKNOWN err.detail="User \"system:anonymous\"
> >> > cannot
> >> > create imagestreammappings in project \"-example\""
> err.message="unknown
> >> > error" go.version=go1.4.2 http.request.host="172.30.223.171:5000"
> >> > http.request.id=3ae567a3-b2d0-48b0-9584-050e56cb5bb2
> >> > http.request.method=PUT
> >> > http.request.remoteaddr="172.50.0.1:59125"
> >> > http.request.uri="/v2/-example/nodejs-ex/manifests/latest"
> >> > http.request.useragent="docker/1.8.2-el7.centos go/go1.4.2
> >> > kernel/3.10.0-327.4.5.el7.x86_64 os/linux arch/amd64"
> >> > http.response.contenttype="application/json; charset=utf-8"
> >> > http.response.duration=30.909695ms http.response.status=500
> >> > http.response.written=292
> >> > instance.id=41bc78ef-3afb-4dbe-94b7-b28401931d85
> >> > vars.name="-example/nodejs-ex" vars.reference=latest
> >> >
> >> > I feel like my builder serviceaccount or registry authentication
> config
> >> > is
> >> > messed up, but I don't understand what needs to change to fix this.
> I've
> >> > redeployed the registry multiple times in trying to troubleshoot/fix
> the
> >> > issue, but I'm still not clear on where the builder serviceaccounts
> get
> >> > their dockercfg secret or what about that secret might be wrong.
> >> >
> >> > Any help on where to look next?
> >> >
> >> > --
> >> > Robert Wehner
> >> >
> >> > _______________________________________________
> >> > users mailing list
> >> > users@lists.openshift.redhat.com
> >> > http://lists.openshift.redhat.com/openshiftmm/listinfo/users
> >> >
> >
> >
>
_______________________________________________
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
