Re: Can't push images after 1.3.0 upgrade

[Diego Castro]

Have you secured your registry? I've faced same issue upgrading my cluster.
You can workaround it by putting the following config on inventory file:

openshift_docker_hosted_registry_insecure=True


---
Diego Castro / The CloudFather
GetupCloud.com - Eliminamos a Gravidade

2017-01-05 23:24 GMT-03:00 Philippe Lafoucrière <
philippe.lafoucri...@tech-angels.com>:

> Hmm, I got it working by removing the :443 in our repo url (while this
> port was needed before to be able to push...)
>
>
>
> ___
> users mailing list
> users@lists.openshift.redhat.com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/users
>
>
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Philippe Lafoucrière]

Hmm, I got it working by removing the :443 in our repo url (while this port
was needed before to be able to push...)
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Philippe Lafoucrière]

It's really the images layers patching which is blocking. I can pull an
image from the registry, and push it again with no error:

The push refers to a repository
[our-registry:443/projectname/theimagestream]
38731c91ef63: Layer already exists
7d7e09f222b3: Layer already exists
latest: digest:
sha256:fa1f5a94b89552b2c9d370c7ff779d658e09547eae00d931a3b1f2502f1f7260
size: 3339



Cordialement,
Philippe Lafoucrière

-- 
Philippe Lafoucrière - CEO
http://www.tech-angels.com
https://gemnasium.com
France : +33 (0) 3 65 96 02 92
Canada: +1 (418) 478-1175
USA: +1 (954) 607-7443



On Thu, Jan 5, 2017 at 8:49 PM, Philippe Lafoucrière <
philippe.lafoucri...@tech-angels.com> wrote:

> I'm digging this up.
> We just upgraded our production cluster to OS 1.3, and having this issue
> again.
> Builds are working as expected, but we can't push using our CI any more,
> with the same exact symptoms as above :(
>
> Any idea?
> I have tried to reconcile roles, with no success.
>
> Thanks
>
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Philippe Lafoucrière]

I'm digging this up.
We just upgraded our production cluster to OS 1.3, and having this issue
again.
Builds are working as expected, but we can't push using our CI any more,
with the same exact symptoms as above :(

Any idea?
I have tried to reconcile roles, with no success.

Thanks
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Philippe Lafoucrière]

We're using :443/namespace/imagestream
We had to rollback to previous snapshots, we never managed to get it
working :(
​
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Jordan Liggitt]

How are you referencing the registry server? "" or ":"?
Missing auth headers on a patch request during push sounds a lot like these
issues:

https://github.com/docker/docker/issues/18469
https://github.com/docker/distribution/pull/1868



On Tue, Sep 27, 2016 at 5:47 PM, Philippe Lafoucrière <
philippe.lafoucri...@tech-angels.com> wrote:

> Note that I can pull the image with this account.
> I have tried to readd the role to the user:
>
> $ oadm policy add-cluster-role-to-user system:image-builder our_ci_user
>
> with no success.
> According to https://docs.openshift.com/container-platform/3.3/admin_
> guide/manage_authorization_policy.html, I should be able to update the
> layers.
>
> $ oadm policy who-can update imagestreams/layers
> -> my ci user is listed here
> ​
> Thanks
>
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Philippe Lafoucrière]

Note that I can pull the image with this account.
I have tried to readd the role to the user:

$ oadm policy add-cluster-role-to-user system:image-builder our_ci_user

with no success.
According to
https://docs.openshift.com/container-platform/3.3/admin_guide/manage_authorization_policy.html,
I
should be able to update the layers.

$ oadm policy who-can update imagestreams/layers
-> my ci user is listed here
​
Thanks
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Philippe Lafoucrière]

On Tue, Sep 27, 2016 at 4:29 PM, Jordan Liggitt  wrote:
>
> Do you have the registry logs available from the timeframe during the
push?


10.1.0.1 - - [27/Sep/2016:20:59:57
+] time="2016-09-27T20:59:58.948672089Z" level=error msg="error
authorizing context: authorization header required" go.version=go1.6.3
http.request.host=redacted http.request.id=24db7eaf-f66f-462a-9d2e-434b77ca7a30
http.request.method=PATCH http.request.remoteaddr=172.29.13.4
http.request.uri="/v2/gemnasium-staging/registry-scanner/blobs/uploads/a9c303fc-e85b-428c-b799-9cba00a40f77?_state=FguTxOGl3FNUtqk1-RNJvR8E7fvACwiGW_MQetCuFRp7Ik5hbWUiOiJnZW1uYXNpdW0tc3RhZ2luZy9yZWdpc3RyeS1zY2FubmVyIiwiVVVJRCI6ImE5YzMwM2ZjLWU4NWItNDI4Yy1iNzk5LTljYmEwMGE0MGY3NyIsIk9mZnNldCI6MCwiU3RhcnRlZEF0IjoiMjAxNi0wOS0yN1QyMDo1OTo1OC45MTEyMDE4MjhaIn0%3D"
http.request.useragent="docker/1.12.1 go/go1.6.3 git-commit/23cf638
kernel/3.16.0-4-amd64 os/linux arch/amd64
UpstreamClient(Docker-Client/1.12.1 \\(linux\\))"
instance.id=093a0322-b4bf-4b2a-bed3-5f02d0b2b0d7
vars.name="gemnasium-staging/registry-scanner"
vars.uuid=a9c303fc-e85b-428c-b799-9cba00a40f77
10.1.0.1 - - [27/Sep/2016:20:59:58 +] "PATCH
/v2/gemnasium-staging/registry-scanner/blobs/uploads/a9c303fc-e85b-428c-b799-9cba00a40f77?_state=FguTxOGl3FNUtqk1-RNJvR8E7fvACwiGW_MQetCuFRp7Ik5hbWUiOiJnZW1uYXNpdW0tc3RhZ2luZy9yZWdpc3RyeS1zY2FubmVyIiwiVVVJRCI6ImE5YzMwM2ZjLWU4NWItNDI4Yy1iNzk5LTljYmEwMGE0MGY3NyIsIk9mZnNldCI6MCwiU3RhcnRlZEF0IjoiMjAxNi0wOS0yN1QyMDo1OTo1OC45MTEyMDE4MjhaIn0%3D
HTTP/1.1" 401 248 "" "docker/1.12.1 go/go1.6.3 git-commit/23cf638
kernel/3.16.0-4-amd64 os/linux arch/amd64
UpstreamClient(Docker-Client/1.12.1 \\(linux\\))"
10.1.0.1 - - [27/Sep/2016:20:59:58 +] "PATCH
/v2/gemnasium-staging/registry-scanner/blobs/uploads/66b43d12-ad91-4c48-9e74-4d7fcc2f6eb8?_state=XTu8Xy0JVxFNNHlaCwnssuOkev1Vc_xy_iyGsSwtI5t7Ik5hbWUiOiJnZW1uYXNpdW0tc3RhZ2luZy9yZWdpc3RyeS1zY2FubmVyIiwiVVVJRCI6IjY2YjQzZDEyLWFkOTEtNGM0OC05ZTc0LTRkN2ZjYzJmNmViOCIsIk9mZnNldCI6MCwiU3RhcnRlZEF0IjoiMjAxNi0wOS0yN1QyMDo1OTo1OC45MTAyNDgzNzlaIn0%3D
HTTP/1.1" 401 248 "" "docker/1.12.1 go/go1.6.3 git-commit/23cf638
kernel/3.16.0-4-amd64 os/linux arch/amd64
UpstreamClient(Docker-Client/1.12.1 \\(linux\\))"

Thanks
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Can't push images after 1.3.0 upgrade

[Jordan Liggitt]

Do you have the registry logs available from the timeframe during the push?

On Tue, Sep 27, 2016 at 4:26 PM, Philippe Lafoucrière <
philippe.lafoucri...@tech-angels.com> wrote:

> Hi,
>
> Another issue we're facing after the upgrade to 1.3.0:
> our CI service account can't push images to the registry anymore.
> I have tried to push the image by hand:
>
> 202bc3fd6fe4: Pushing [==>]
> 7.114 MB
> be16db112b16: Pushing [==>]
> 280.6 kB
> unauthorized: authentication required
>
> In the sa description, the tokens seem to be the same (at least they have
> the same names).
> I have triedto reconcile policies :
>
> oadm policy reconcile-cluster-roles \
> --additive-only=true \
> --confirm
>
> oadm policy reconcile-cluster-role-bindings \
> --exclude-groups=system:authenticated \
> --exclude-groups=system:authenticated:oauth \
> --exclude-groups=system:unauthenticated \
> --exclude-users=system:anonymous \
> --additive-only=true \
> --confirm
>
> oadm policy reconcile-sccs \
> --additive-only=true \
> --confirm
>
> (but it should done by the playbook I think), and yet, I can't push any
> more :(
>
> Did we miss something during the upgrade?
>
> Thanks,
> Philippe
>
> ___
> users mailing list
> users@lists.openshift.redhat.com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/users
>
>
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Can't push images after 1.3.0 upgrade

[Philippe Lafoucrière]

Hi,

Another issue we're facing after the upgrade to 1.3.0:
our CI service account can't push images to the registry anymore.
I have tried to push the image by hand:

202bc3fd6fe4: Pushing [==>]
7.114 MB
be16db112b16: Pushing [==>]
280.6 kB
unauthorized: authentication required

In the sa description, the tokens seem to be the same (at least they have
the same names).
I have triedto reconcile policies :

oadm policy reconcile-cluster-roles \
--additive-only=true \
--confirm

oadm policy reconcile-cluster-role-bindings \
--exclude-groups=system:authenticated \
--exclude-groups=system:authenticated:oauth \
--exclude-groups=system:unauthenticated \
--exclude-users=system:anonymous \
--additive-only=true \
--confirm

oadm policy reconcile-sccs \
--additive-only=true \
--confirm

(but it should done by the playbook I think), and yet, I can't push any
more :(

Did we miss something during the upgrade?

Thanks,
Philippe
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users