[ceph-users] Re: pacific 16.2.15 QE validation status

[Yuri Weinstein]

Thank you all!

We want to merge the PR with whitelisting added
https://github.com/ceph/ceph/pull/55717 and will start the 16.2.15
build/release afterward.

On Mon, Feb 26, 2024 at 8:25 AM Laura Flores  wrote:

> Thank you Junior for your thorough review of the RADOS suite. Aside from a
> few remaining warnings in the final run that could benefit from
> whitelisting, these are not blockers.
>
> Rados-approved.
>
> On Mon, Feb 26, 2024 at 9:29 AM Kamoltat Sirivadhna 
> wrote:
>
>> details of RADOS run analysis:
>>
>> yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi
>> <
>> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/#collapseOne
>> >
>>
>>
>>
>>1. https://tracker.ceph.com/issues/64455  task/test_orch_cli: Health
>>check failed: cephadm background work is paused (CEPHADM_PAUSED)" in
>>cluster log (White list)
>>2. https://tracker.ceph.com/issues/64454
>> rados/cephadm/mgr-nfs-upgrade:
>>Health check failed: 1 stray daemon(s) not managed by cephadm
>>(CEPHADM_STRAY_DAEMON)" in cluster log (whitelist)
>>3. https://tracker.ceph.com/issues/63887: Starting alertmanager fails
>>from missing container (happens in Pacific)
>>4. Failed to reconnect to smithi155 [7566763
>><
>> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566763
>> >
>>]
>>5. https://tracker.ceph.com/issues/64278 Unable to update caps for
>>client.iscsi.iscsi.a (known failures)
>>6. https://tracker.ceph.com/issues/64452 Teuthology runs into
>>"TypeError: expected string or bytes-like object" during log scraping
>>(teuthology failure)
>>7. https://tracker.ceph.com/issues/64343 Expected warnings that need
>> to
>>be whitelisted cause rados/cephadm tests to fail for 7566717
>><
>> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566717
>> >
>>we neeed to add (ERR|WRN|SEC)
>>8. https://tracker.ceph.com/issues/58145 orch/cephadm: nfs tests
>> failing
>>to mount exports (mount -t nfs 10.0.31.120:/fake /mnt/foo' fails)
>>7566724 (resolved issue re-opened)
>>9. https://tracker.ceph.com/issues/63577 cephadm:
>>docker.io/library/haproxy: toomanyrequests: You have reached your pull
>>rate limit.
>>10. https://tracker.ceph.com/issues/54071 rdos/cephadm/osds: Invalid
>>command: missing required parameter hostname() 756674
>><
>> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566747
>> >
>>
>>
>> Note:
>>
>>1. Although 7566762 seems like a different failure from what is
>>displayed in pulpito, in the teuth log it failed because of
>>https://tracker.ceph.com/issues/64278.
>>2. rados/cephadm/thrash/ … failed a lot because of
>>https://tracker.ceph.com/issues/64452
>>3. 7566717
>><
>> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566717
>> >.
>>failed because we didn’t whitelist (ERR|WRN|SEC)
>> :tasks.cephadm:Checking
>>cluster log for badness...
>>4. 7566724 https://tracker.ceph.com/issues/58145 ganesha seems
>> resolved
>>1 year ago, but popped up again so re-opened tracker and ping Adam King
>>(resolved)
>>
>> 7566777, 7566781, 7566796 are due to
>> https://tracker.ceph.com/issues/63577
>>
>>
>>
>> White List and re-ran:
>>
>> yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/
>> <
>> https://pulpito.ceph.com/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/
>> >
>>
>> rados/cephadm/mds_upgrade_sequence/ —> failed to shutdown mon (known
>> failure discussed with A.King)
>>
>> rados/cephadm/mgr-nfs-upgrade —> failed to shutdown mon (known failure
>> discussed with A.King)
>>
>> rados/cephadm/osds —> zap disk error (known failure)
>>
>> rados/cephadm/smoke-roleless —>  toomanyrequests: You have reached your
>> pull rate limit. https://www.docker.com/increase-rate-limit. (known
>> failures)
>>
>> rados/cephadm/thrash —> Just needs to whitelist (CACHE_POOL_NEAR_FULL)
>> (known failures)
>>
>> rados/cephadm/upgrade —> CEPHADM_FAILED_DAEMON (WRN)  node-exporter
>> (known
>> failure discussed with A.King)
>>
>> rados/cephadm/workunits —> known failure:
>> https://tracker.ceph.com/issues/63887
>>
>> On Mon, Feb 26, 2024 at 10:22 AM Kamoltat Sirivadhna > >
>> wrote:
>>
>> > RADOS approved
>> >
>> > On Wed, Feb 21, 2024 at 11:27 AM Yuri Weinstein 
>> > wrote:
>> >
>> >> Still seeking approvals:
>> >>
>> >> rados - Radek, Junior, Travis, Adam King
>> >>
>> >> All other product areas have been approved and are ready for the
>> release
>> >> step.
>> >>
>> >> Pls also review the Release Notes:
>> >> https://github.com/ceph/ceph/pull/55694
>> >>
>> >>
>> >> On Tue, Feb 20, 2024 at 7:58 AM Yuri Weinstein 
>> >> wrote:
>> >> >
>> >> > We have restarted QE validation after fixing issues and merging
>> several
>> >> 

[ceph-users] Re: pacific 16.2.15 QE validation status

[Laura Flores]

Thank you Junior for your thorough review of the RADOS suite. Aside from a
few remaining warnings in the final run that could benefit from
whitelisting, these are not blockers.

Rados-approved.

On Mon, Feb 26, 2024 at 9:29 AM Kamoltat Sirivadhna 
wrote:

> details of RADOS run analysis:
>
> yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi
> <
> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/#collapseOne
> >
>
>
>
>1. https://tracker.ceph.com/issues/64455  task/test_orch_cli: Health
>check failed: cephadm background work is paused (CEPHADM_PAUSED)" in
>cluster log (White list)
>2. https://tracker.ceph.com/issues/64454 rados/cephadm/mgr-nfs-upgrade:
>Health check failed: 1 stray daemon(s) not managed by cephadm
>(CEPHADM_STRAY_DAEMON)" in cluster log (whitelist)
>3. https://tracker.ceph.com/issues/63887: Starting alertmanager fails
>from missing container (happens in Pacific)
>4. Failed to reconnect to smithi155 [7566763
><
> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566763
> >
>]
>5. https://tracker.ceph.com/issues/64278 Unable to update caps for
>client.iscsi.iscsi.a (known failures)
>6. https://tracker.ceph.com/issues/64452 Teuthology runs into
>"TypeError: expected string or bytes-like object" during log scraping
>(teuthology failure)
>7. https://tracker.ceph.com/issues/64343 Expected warnings that need to
>be whitelisted cause rados/cephadm tests to fail for 7566717
><
> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566717
> >
>we neeed to add (ERR|WRN|SEC)
>8. https://tracker.ceph.com/issues/58145 orch/cephadm: nfs tests
> failing
>to mount exports (mount -t nfs 10.0.31.120:/fake /mnt/foo' fails)
>7566724 (resolved issue re-opened)
>9. https://tracker.ceph.com/issues/63577 cephadm:
>docker.io/library/haproxy: toomanyrequests: You have reached your pull
>rate limit.
>10. https://tracker.ceph.com/issues/54071 rdos/cephadm/osds: Invalid
>command: missing required parameter hostname() 756674
><
> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566747
> >
>
>
> Note:
>
>1. Although 7566762 seems like a different failure from what is
>displayed in pulpito, in the teuth log it failed because of
>https://tracker.ceph.com/issues/64278.
>2. rados/cephadm/thrash/ … failed a lot because of
>https://tracker.ceph.com/issues/64452
>3. 7566717
><
> https://pulpito.ceph.com/yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi/7566717
> >.
>failed because we didn’t whitelist (ERR|WRN|SEC) :tasks.cephadm:Checking
>cluster log for badness...
>4. 7566724 https://tracker.ceph.com/issues/58145 ganesha seems resolved
>1 year ago, but popped up again so re-opened tracker and ping Adam King
>(resolved)
>
> 7566777, 7566781, 7566796 are due to https://tracker.ceph.com/issues/63577
>
>
>
> White List and re-ran:
>
> yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/
> <
> https://pulpito.ceph.com/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/
> >
>
> rados/cephadm/mds_upgrade_sequence/ —> failed to shutdown mon (known
> failure discussed with A.King)
>
> rados/cephadm/mgr-nfs-upgrade —> failed to shutdown mon (known failure
> discussed with A.King)
>
> rados/cephadm/osds —> zap disk error (known failure)
>
> rados/cephadm/smoke-roleless —>  toomanyrequests: You have reached your
> pull rate limit. https://www.docker.com/increase-rate-limit. (known
> failures)
>
> rados/cephadm/thrash —> Just needs to whitelist (CACHE_POOL_NEAR_FULL)
> (known failures)
>
> rados/cephadm/upgrade —> CEPHADM_FAILED_DAEMON (WRN)  node-exporter  (known
> failure discussed with A.King)
>
> rados/cephadm/workunits —> known failure:
> https://tracker.ceph.com/issues/63887
>
> On Mon, Feb 26, 2024 at 10:22 AM Kamoltat Sirivadhna 
> wrote:
>
> > RADOS approved
> >
> > On Wed, Feb 21, 2024 at 11:27 AM Yuri Weinstein 
> > wrote:
> >
> >> Still seeking approvals:
> >>
> >> rados - Radek, Junior, Travis, Adam King
> >>
> >> All other product areas have been approved and are ready for the release
> >> step.
> >>
> >> Pls also review the Release Notes:
> >> https://github.com/ceph/ceph/pull/55694
> >>
> >>
> >> On Tue, Feb 20, 2024 at 7:58 AM Yuri Weinstein 
> >> wrote:
> >> >
> >> > We have restarted QE validation after fixing issues and merging
> several
> >> PRs.
> >> > The new Build 3 (rebase of pacific) tests are summarized in the same
> >> > note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
> >> >
> >> > Seeking approvals:
> >> >
> >> > rados - Radek, Junior, Travis, Ernesto, Adam King
> >> > rgw - Casey
> >> > fs - Venky
> >> > rbd - Ilya
> >> > krbd - Ilya
> >> >
> >> > upgrade/octopus-x (pacific) 

[ceph-users] Re: pacific 16.2.15 QE validation status

[Kamoltat Sirivadhna]

details of RADOS run analysis:

yuriw-2024-02-19_19:25:49-rados-pacific-release-distro-default-smithi




   1. https://tracker.ceph.com/issues/64455  task/test_orch_cli: Health
   check failed: cephadm background work is paused (CEPHADM_PAUSED)" in
   cluster log (White list)
   2. https://tracker.ceph.com/issues/64454 rados/cephadm/mgr-nfs-upgrade:
   Health check failed: 1 stray daemon(s) not managed by cephadm
   (CEPHADM_STRAY_DAEMON)" in cluster log (whitelist)
   3. https://tracker.ceph.com/issues/63887: Starting alertmanager fails
   from missing container (happens in Pacific)
   4. Failed to reconnect to smithi155 [7566763
   

   ]
   5. https://tracker.ceph.com/issues/64278 Unable to update caps for
   client.iscsi.iscsi.a (known failures)
   6. https://tracker.ceph.com/issues/64452 Teuthology runs into
   "TypeError: expected string or bytes-like object" during log scraping
   (teuthology failure)
   7. https://tracker.ceph.com/issues/64343 Expected warnings that need to
   be whitelisted cause rados/cephadm tests to fail for 7566717
   

   we neeed to add (ERR|WRN|SEC)
   8. https://tracker.ceph.com/issues/58145 orch/cephadm: nfs tests failing
   to mount exports (mount -t nfs 10.0.31.120:/fake /mnt/foo' fails)
   7566724 (resolved issue re-opened)
   9. https://tracker.ceph.com/issues/63577 cephadm:
   docker.io/library/haproxy: toomanyrequests: You have reached your pull
   rate limit.
   10. https://tracker.ceph.com/issues/54071 rdos/cephadm/osds: Invalid
   command: missing required parameter hostname() 756674
   



Note:

   1. Although 7566762 seems like a different failure from what is
   displayed in pulpito, in the teuth log it failed because of
   https://tracker.ceph.com/issues/64278.
   2. rados/cephadm/thrash/ … failed a lot because of
   https://tracker.ceph.com/issues/64452
   3. 7566717
   
.
   failed because we didn’t whitelist (ERR|WRN|SEC) :tasks.cephadm:Checking
   cluster log for badness...
   4. 7566724 https://tracker.ceph.com/issues/58145 ganesha seems resolved
   1 year ago, but popped up again so re-opened tracker and ping Adam King
   (resolved)

7566777, 7566781, 7566796 are due to https://tracker.ceph.com/issues/63577



White List and re-ran:

yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/


rados/cephadm/mds_upgrade_sequence/ —> failed to shutdown mon (known
failure discussed with A.King)

rados/cephadm/mgr-nfs-upgrade —> failed to shutdown mon (known failure
discussed with A.King)

rados/cephadm/osds —> zap disk error (known failure)

rados/cephadm/smoke-roleless —>  toomanyrequests: You have reached your
pull rate limit. https://www.docker.com/increase-rate-limit. (known
failures)

rados/cephadm/thrash —> Just needs to whitelist (CACHE_POOL_NEAR_FULL)
(known failures)

rados/cephadm/upgrade —> CEPHADM_FAILED_DAEMON (WRN)  node-exporter  (known
failure discussed with A.King)

rados/cephadm/workunits —> known failure:
https://tracker.ceph.com/issues/63887

On Mon, Feb 26, 2024 at 10:22 AM Kamoltat Sirivadhna 
wrote:

> RADOS approved
>
> On Wed, Feb 21, 2024 at 11:27 AM Yuri Weinstein 
> wrote:
>
>> Still seeking approvals:
>>
>> rados - Radek, Junior, Travis, Adam King
>>
>> All other product areas have been approved and are ready for the release
>> step.
>>
>> Pls also review the Release Notes:
>> https://github.com/ceph/ceph/pull/55694
>>
>>
>> On Tue, Feb 20, 2024 at 7:58 AM Yuri Weinstein 
>> wrote:
>> >
>> > We have restarted QE validation after fixing issues and merging several
>> PRs.
>> > The new Build 3 (rebase of pacific) tests are summarized in the same
>> > note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
>> >
>> > Seeking approvals:
>> >
>> > rados - Radek, Junior, Travis, Ernesto, Adam King
>> > rgw - Casey
>> > fs - Venky
>> > rbd - Ilya
>> > krbd - Ilya
>> >
>> > upgrade/octopus-x (pacific) - Adam King, Casey PTL
>> >
>> > upgrade/pacific-p2p - Casey PTL
>> >
>> > ceph-volume - Guillaume, fixed by
>> > https://github.com/ceph/ceph/pull/55658 retesting
>> >
>> > On Thu, Feb 8, 2024 at 8:43 AM Casey Bodley  wrote:
>> > >
>> > > thanks, i've created https://tracker.ceph.com/issues/64360 to track
>> > > these backports to pacific/quincy/reef
>> > >
>> > > On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
>> > > >
>> > > > Hi,
>> > > >
>> > > > Is this PR: 

[ceph-users] Re: pacific 16.2.15 QE validation status

[Kamoltat Sirivadhna]

RADOS approved

On Wed, Feb 21, 2024 at 11:27 AM Yuri Weinstein  wrote:

> Still seeking approvals:
>
> rados - Radek, Junior, Travis, Adam King
>
> All other product areas have been approved and are ready for the release
> step.
>
> Pls also review the Release Notes: https://github.com/ceph/ceph/pull/55694
>
>
> On Tue, Feb 20, 2024 at 7:58 AM Yuri Weinstein 
> wrote:
> >
> > We have restarted QE validation after fixing issues and merging several
> PRs.
> > The new Build 3 (rebase of pacific) tests are summarized in the same
> > note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
> >
> > Seeking approvals:
> >
> > rados - Radek, Junior, Travis, Ernesto, Adam King
> > rgw - Casey
> > fs - Venky
> > rbd - Ilya
> > krbd - Ilya
> >
> > upgrade/octopus-x (pacific) - Adam King, Casey PTL
> >
> > upgrade/pacific-p2p - Casey PTL
> >
> > ceph-volume - Guillaume, fixed by
> > https://github.com/ceph/ceph/pull/55658 retesting
> >
> > On Thu, Feb 8, 2024 at 8:43 AM Casey Bodley  wrote:
> > >
> > > thanks, i've created https://tracker.ceph.com/issues/64360 to track
> > > these backports to pacific/quincy/reef
> > >
> > > On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
> > > >
> > > > Hi,
> > > >
> > > > Is this PR: https://github.com/ceph/ceph/pull/54918 included as
> well?
> > > >
> > > > You definitely want to build the Ubuntu / debian packages with the
> > > > proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.
> > > >
> > > > Thanks,
> > > >
> > > > Gr. Stefan
> > > >
> > > > P.s. Kudos to Mark Nelson for figuring it out / testing.
> > > > ___
> > > > ceph-users mailing list -- ceph-users@ceph.io
> > > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > > >
> > >
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>


-- 

Kamoltat Sirivadhna (HE/HIM)

SoftWare Engineer - Ceph Storage

ksiri...@redhat.comT: (857) <(919)716-5348>253-8927
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Yuri Weinstein]

Still seeking approvals:

rados - Radek, Junior, Travis, Adam King

All other product areas have been approved and are ready for the release step.

Pls also review the Release Notes: https://github.com/ceph/ceph/pull/55694


On Tue, Feb 20, 2024 at 7:58 AM Yuri Weinstein  wrote:
>
> We have restarted QE validation after fixing issues and merging several PRs.
> The new Build 3 (rebase of pacific) tests are summarized in the same
> note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals:
>
> rados - Radek, Junior, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky
> rbd - Ilya
> krbd - Ilya
>
> upgrade/octopus-x (pacific) - Adam King, Casey PTL
>
> upgrade/pacific-p2p - Casey PTL
>
> ceph-volume - Guillaume, fixed by
> https://github.com/ceph/ceph/pull/55658 retesting
>
> On Thu, Feb 8, 2024 at 8:43 AM Casey Bodley  wrote:
> >
> > thanks, i've created https://tracker.ceph.com/issues/64360 to track
> > these backports to pacific/quincy/reef
> >
> > On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
> > >
> > > Hi,
> > >
> > > Is this PR: https://github.com/ceph/ceph/pull/54918 included as well?
> > >
> > > You definitely want to build the Ubuntu / debian packages with the
> > > proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.
> > >
> > > Thanks,
> > >
> > > Gr. Stefan
> > >
> > > P.s. Kudos to Mark Nelson for figuring it out / testing.
> > > ___
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > >
> >
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Casey Bodley]

On Tue, Feb 20, 2024 at 10:58 AM Yuri Weinstein  wrote:
>
> We have restarted QE validation after fixing issues and merging several PRs.
> The new Build 3 (rebase of pacific) tests are summarized in the same
> note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals:
>
> rados - Radek, Junior, Travis, Ernesto, Adam King
> rgw - Casey

rgw approved

> fs - Venky
> rbd - Ilya
> krbd - Ilya
>
> upgrade/octopus-x (pacific) - Adam King, Casey PTL
>
> upgrade/pacific-p2p - Casey PTL

Yuri and i managed to get a green run here, approved

>
> ceph-volume - Guillaume, fixed by
> https://github.com/ceph/ceph/pull/55658 retesting
>
> On Thu, Feb 8, 2024 at 8:43 AM Casey Bodley  wrote:
> >
> > thanks, i've created https://tracker.ceph.com/issues/64360 to track
> > these backports to pacific/quincy/reef
> >
> > On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
> > >
> > > Hi,
> > >
> > > Is this PR: https://github.com/ceph/ceph/pull/54918 included as well?
> > >
> > > You definitely want to build the Ubuntu / debian packages with the
> > > proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.
> > >
> > > Thanks,
> > >
> > > Gr. Stefan
> > >
> > > P.s. Kudos to Mark Nelson for figuring it out / testing.
> > > ___
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > >
> >
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Nizamudeen A]

dashboard approved. our e2e specs are passing but the suite failed because
of a different error.
cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm
(CEPHADM_STRAY_DAEMON)" in cluster log


On Tue, Feb 20, 2024 at 9:29 PM Yuri Weinstein  wrote:

> We have restarted QE validation after fixing issues and merging several
> PRs.
> The new Build 3 (rebase of pacific) tests are summarized in the same
> note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals:
>
> rados - Radek, Junior, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky
> rbd - Ilya
> krbd - Ilya
>
> upgrade/octopus-x (pacific) - Adam King, Casey PTL
>
> upgrade/pacific-p2p - Casey PTL
>
> ceph-volume - Guillaume, fixed by
> https://github.com/ceph/ceph/pull/55658 retesting
>
> On Thu, Feb 8, 2024 at 8:43 AM Casey Bodley  wrote:
> >
> > thanks, i've created https://tracker.ceph.com/issues/64360 to track
> > these backports to pacific/quincy/reef
> >
> > On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
> > >
> > > Hi,
> > >
> > > Is this PR: https://github.com/ceph/ceph/pull/54918 included as well?
> > >
> > > You definitely want to build the Ubuntu / debian packages with the
> > > proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.
> > >
> > > Thanks,
> > >
> > > Gr. Stefan
> > >
> > > P.s. Kudos to Mark Nelson for figuring it out / testing.
> > > ___
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > >
> >
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Venky Shankar]

Hi Yuri,

On Tue, Feb 20, 2024 at 9:29 PM Yuri Weinstein  wrote:
>
> We have restarted QE validation after fixing issues and merging several PRs.
> The new Build 3 (rebase of pacific) tests are summarized in the same
> note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals:
>
> rados - Radek, Junior, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky

fs approved. failures are -
https://tracker.ceph.com/projects/cephfs/wiki/Pacific#20-Feb-2024

> rbd - Ilya
> krbd - Ilya
>
> upgrade/octopus-x (pacific) - Adam King, Casey PTL
>
> upgrade/pacific-p2p - Casey PTL
>
> ceph-volume - Guillaume, fixed by
> https://github.com/ceph/ceph/pull/55658 retesting
>
> On Thu, Feb 8, 2024 at 8:43 AM Casey Bodley  wrote:
> >
> > thanks, i've created https://tracker.ceph.com/issues/64360 to track
> > these backports to pacific/quincy/reef
> >
> > On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
> > >
> > > Hi,
> > >
> > > Is this PR: https://github.com/ceph/ceph/pull/54918 included as well?
> > >
> > > You definitely want to build the Ubuntu / debian packages with the
> > > proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.
> > >
> > > Thanks,
> > >
> > > Gr. Stefan
> > >
> > > P.s. Kudos to Mark Nelson for figuring it out / testing.
> > > ___
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > >
> >
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io



-- 
Cheers,
Venky
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Ilya Dryomov]

On Tue, Feb 20, 2024 at 4:59 PM Yuri Weinstein  wrote:
>
> We have restarted QE validation after fixing issues and merging several PRs.
> The new Build 3 (rebase of pacific) tests are summarized in the same
> note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals:
>
> rados - Radek, Junior, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky
> rbd - Ilya
> krbd - Ilya

rbd and krbd approved.

Thanks,

Ilya
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Yuri Weinstein]

We have restarted QE validation after fixing issues and merging several PRs.
The new Build 3 (rebase of pacific) tests are summarized in the same
note (see Build 3 runs) https://tracker.ceph.com/issues/64151#note-1

Seeking approvals:

rados - Radek, Junior, Travis, Ernesto, Adam King
rgw - Casey
fs - Venky
rbd - Ilya
krbd - Ilya

upgrade/octopus-x (pacific) - Adam King, Casey PTL

upgrade/pacific-p2p - Casey PTL

ceph-volume - Guillaume, fixed by
https://github.com/ceph/ceph/pull/55658 retesting

On Thu, Feb 8, 2024 at 8:43 AM Casey Bodley  wrote:
>
> thanks, i've created https://tracker.ceph.com/issues/64360 to track
> these backports to pacific/quincy/reef
>
> On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
> >
> > Hi,
> >
> > Is this PR: https://github.com/ceph/ceph/pull/54918 included as well?
> >
> > You definitely want to build the Ubuntu / debian packages with the
> > proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.
> >
> > Thanks,
> >
> > Gr. Stefan
> >
> > P.s. Kudos to Mark Nelson for figuring it out / testing.
> > ___
> > ceph-users mailing list -- ceph-users@ceph.io
> > To unsubscribe send an email to ceph-users-le...@ceph.io
> >
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Casey Bodley]

thanks, i've created https://tracker.ceph.com/issues/64360 to track
these backports to pacific/quincy/reef

On Thu, Feb 8, 2024 at 7:50 AM Stefan Kooman  wrote:
>
> Hi,
>
> Is this PR: https://github.com/ceph/ceph/pull/54918 included as well?
>
> You definitely want to build the Ubuntu / debian packages with the
> proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.
>
> Thanks,
>
> Gr. Stefan
>
> P.s. Kudos to Mark Nelson for figuring it out / testing.
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Stefan Kooman]

Hi,

Is this PR: https://github.com/ceph/ceph/pull/54918 included as well?

You definitely want to build the Ubuntu / debian packages with the 
proper CMAKE_CXX_FLAGS. The performance impact on RocksDB is _HUGE_.


Thanks,

Gr. Stefan

P.s. Kudos to Mark Nelson for figuring it out / testing.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Yuri Weinstein]

We are still working through the remaining issues and will do a full
cycle of testing soon.

Adam, the issues mentioned by Ilya below require some response and
resolution, pls take a look

> rbd looks good overall but we are missing iSCSI coverage due to
> https://tracker.ceph.com/issues/64126


On Wed, Jan 31, 2024 at 6:40 AM Ilya Dryomov  wrote:
>
> On Tue, Jan 30, 2024 at 9:24 PM Yuri Weinstein  wrote:
> >
> > Update.
> > Seeking approvals/reviews for:
> >
> > rados - Radek, Laura, Travis, Ernesto, Adam King
> > rgw - Casey
> > fs - Venky
> > rbd - Ilya
>
> Hi Yuri,
>
> rbd looks good overall but we are missing iSCSI coverage due to
> https://tracker.ceph.com/issues/64126:
>
> https://pulpito.ceph.com/yuriw-2024-01-25_17:27:10-rbd-pacific-release-distro-default-smithi/7532167
> https://pulpito.ceph.com/yuriw-2024-01-26_17:05:20-rbd-pacific-release-distro-default-smithi/7535193
>
> Adam, did you get a chance to look into it?
>
> > krbd - Ilya
>
> Please do another rerun for krbd -- I want to see one of those
> remaining jobs pass.
>
> Thanks,
>
> Ilya
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Konstantin Shalygin]

> 
> On Feb 7, 2024, at 16:59, Zakhar Kirpichenko  wrote:
> 
> Indeed, it looks like it's been recently reopened. Thanks for this!

Hi,

It was merged yesterday

Thanks for the right noise,
k
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Zakhar Kirpichenko]

Indeed, it looks like it's been recently reopened. Thanks for this!

/Z

On Wed, 7 Feb 2024 at 15:43, David Orman  wrote:

> That tracker's last update indicates it's slated for inclusion.
>
> On Thu, Feb 1, 2024, at 10:47, Zakhar Kirpichenko wrote:
> > Hi,
> >
> > Please consider not leaving this behind:
> https://github.com/ceph/ceph/pull/55109
> >
> > It's a serious bug, which potentially affects a whole node stability if
> > the affected mgr is colocated with OSDs. The bug was known for quite a
> > while and really shouldn't be left unfixed.
> >
> > /Z
> >
> > On Thu, 1 Feb 2024 at 18:45, Nizamudeen A  wrote:
> >> Thanks Laura,
> >>
> >> Raised a PR for  https://tracker.ceph.com/issues/57386
> >> https://github.com/ceph/ceph/pull/55415
> >>
> >>
> >> On Thu, Feb 1, 2024 at 5:15 AM Laura Flores  wrote:
> >>
> >> > I reviewed the rados suite. @Adam King ,
> @Nizamudeen A
> >> >  would appreciate a look from you, as there are some
> >> > orchestrator and dashboard trackers that came up.
> >> >
> >> > pacific-release, 16.2.15
> >> >
> >> > Failures:
> >> > 1. https://tracker.ceph.com/issues/62225
> >> > 2. https://tracker.ceph.com/issues/64278
> >> > 3. https://tracker.ceph.com/issues/58659
> >> > 4. https://tracker.ceph.com/issues/58658
> >> > 5. https://tracker.ceph.com/issues/64280 -- new tracker, worth a
> look
> >> > from Orch
> >> > 6. https://tracker.ceph.com/issues/63577
> >> > 7. https://tracker.ceph.com/issues/63894
> >> > 8. https://tracker.ceph.com/issues/64126
> >> > 9. https://tracker.ceph.com/issues/63887
> >> > 10. https://tracker.ceph.com/issues/61602
> >> > 11. https://tracker.ceph.com/issues/54071
> >> > 12. https://tracker.ceph.com/issues/57386
> >> > 13. https://tracker.ceph.com/issues/64281
> >> > 14. https://tracker.ceph.com/issues/49287
> >> >
> >> > Details:
> >> > 1. pacific upgrade test fails on 'ceph versions | jq -e' command -
> >> > Ceph - RADOS
> >> > 2. Unable to update caps for client.iscsi.iscsi.a - Ceph -
> Orchestrator
> >> > 3. mds_upgrade_sequence: failure when deploying node-exporter -
> Ceph -
> >> > Orchestrator
> >> > 4. mds_upgrade_sequence: Error: initializing source
> >> > docker://prom/alertmanager:v0.20.0 - Ceph - Orchestrator
> >> > 5. mgr-nfs-upgrade test times out from failed cephadm daemons -
> Ceph -
> >> > Orchestrator
> >> > 6. cephadm: docker.io/library/haproxy: toomanyrequests: You have
> >> > reached your pull rate limit. You may increase the limit by
> authenticating
> >> > and upgrading: https://www.docker.com/increase-rate-limit - Ceph -
> >> > Orchestrator
> >> > 7. qa: cephadm failed with an error code 1, alertmanager
> container not
> >> > found. - Ceph - Orchestrator
> >> > 8. ceph-iscsi build was retriggered and now missing
> >> > package_manager_version attribute - Ceph
> >> > 9. Starting alertmanager fails from missing container - Ceph -
> >> > Orchestrator
> >> > 10. pacific: cls/test_cls_sdk.sh: Health check failed: 1 pool(s)
> do
> >> > not have an application enabled (POOL_APP_NOT_ENABLED) - Ceph - RADOS
> >> > 11. rados/cephadm/osds: Invalid command: missing required
> parameter
> >> > hostname() - Ceph - Orchestrator
> >> > 12. cephadm/test_dashboard_e2e.sh: Expected to find content:
> '/^foo$/'
> >> > within the selector: 'cd-modal .badge' but never did - Ceph - Mgr -
> >> > Dashboard
> >> > 13. Failed to download key at
> >> > http://download.ceph.com/keys/autobuild.asc: Request failed:  >> > error [Errno 101] Network is unreachable> - Infrastructure
> >> > 14. podman: setting cgroup config for procHooks process caused:
> Unit
> >> > libpod-$hash.scope not found - Ceph - Orchestrator
> >> >
> >> > On Wed, Jan 31, 2024 at 1:41 PM Casey Bodley 
> wrote:
> >> >
> >> >> On Mon, Jan 29, 2024 at 4:39 PM Yuri Weinstein 
> >> >> wrote:
> >> >> >
> >> >> > Details of this release are summarized here:
> >> >> >
> >> >> > https://tracker.ceph.com/issues/64151#note-1
> >> >> >
> >> >> > Seeking approvals/reviews for:
> >> >> >
> >> >> > rados - Radek, Laura, Travis, Ernesto, Adam King
> >> >> > rgw - Casey
> >> >>
> >> >> rgw approved, thanks
> >> >>
> >> >> > fs - Venky
> >> >> > rbd - Ilya
> >> >> > krbd - in progress
> >> >> >
> >> >> > upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
> >> >> > upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
> >> >> >
> >> >> > upgrade/pacific-x (quincy) - in progress
> >> >> > upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
> >> >> >
> >> >> > ceph-volume - Guillaume
> >> >> >
> >> >> > TIA
> >> >> > YuriW
> >> >> > ___
> >> >> > ceph-users mailing list -- ceph-users@ceph.io
> >> >> > To unsubscribe send an email to ceph-users-le...@ceph.io
> >> >> >
> >> >> ___
> >> >> Dev mailing list -- d...@ceph.io
> >> >> To unsubscribe send an email to dev-le...@ceph.io
> >> >>
> >> >
> 

[ceph-users] Re: pacific 16.2.15 QE validation status

[David Orman]

That tracker's last update indicates it's slated for inclusion.

On Thu, Feb 1, 2024, at 10:47, Zakhar Kirpichenko wrote:
> Hi, 
>
> Please consider not leaving this behind: 
> https://github.com/ceph/ceph/pull/55109
>
> It's a serious bug, which potentially affects a whole node stability if 
> the affected mgr is colocated with OSDs. The bug was known for quite a 
> while and really shouldn't be left unfixed. 
>
> /Z
>
> On Thu, 1 Feb 2024 at 18:45, Nizamudeen A  wrote:
>> Thanks Laura,
>> 
>> Raised a PR for  https://tracker.ceph.com/issues/57386
>> https://github.com/ceph/ceph/pull/55415
>> 
>> 
>> On Thu, Feb 1, 2024 at 5:15 AM Laura Flores  wrote:
>> 
>> > I reviewed the rados suite. @Adam King , @Nizamudeen A
>> >  would appreciate a look from you, as there are some
>> > orchestrator and dashboard trackers that came up.
>> >
>> > pacific-release, 16.2.15
>> >
>> > Failures:
>> > 1. https://tracker.ceph.com/issues/62225
>> > 2. https://tracker.ceph.com/issues/64278
>> > 3. https://tracker.ceph.com/issues/58659
>> > 4. https://tracker.ceph.com/issues/58658
>> > 5. https://tracker.ceph.com/issues/64280 -- new tracker, worth a look
>> > from Orch
>> > 6. https://tracker.ceph.com/issues/63577
>> > 7. https://tracker.ceph.com/issues/63894
>> > 8. https://tracker.ceph.com/issues/64126
>> > 9. https://tracker.ceph.com/issues/63887
>> > 10. https://tracker.ceph.com/issues/61602
>> > 11. https://tracker.ceph.com/issues/54071
>> > 12. https://tracker.ceph.com/issues/57386
>> > 13. https://tracker.ceph.com/issues/64281
>> > 14. https://tracker.ceph.com/issues/49287
>> >
>> > Details:
>> > 1. pacific upgrade test fails on 'ceph versions | jq -e' command -
>> > Ceph - RADOS
>> > 2. Unable to update caps for client.iscsi.iscsi.a - Ceph - Orchestrator
>> > 3. mds_upgrade_sequence: failure when deploying node-exporter - Ceph -
>> > Orchestrator
>> > 4. mds_upgrade_sequence: Error: initializing source
>> > docker://prom/alertmanager:v0.20.0 - Ceph - Orchestrator
>> > 5. mgr-nfs-upgrade test times out from failed cephadm daemons - Ceph -
>> > Orchestrator
>> > 6. cephadm: docker.io/library/haproxy: toomanyrequests: You have
>> > reached your pull rate limit. You may increase the limit by authenticating
>> > and upgrading: https://www.docker.com/increase-rate-limit - Ceph -
>> > Orchestrator
>> > 7. qa: cephadm failed with an error code 1, alertmanager container not
>> > found. - Ceph - Orchestrator
>> > 8. ceph-iscsi build was retriggered and now missing
>> > package_manager_version attribute - Ceph
>> > 9. Starting alertmanager fails from missing container - Ceph -
>> > Orchestrator
>> > 10. pacific: cls/test_cls_sdk.sh: Health check failed: 1 pool(s) do
>> > not have an application enabled (POOL_APP_NOT_ENABLED) - Ceph - RADOS
>> > 11. rados/cephadm/osds: Invalid command: missing required parameter
>> > hostname() - Ceph - Orchestrator
>> > 12. cephadm/test_dashboard_e2e.sh: Expected to find content: '/^foo$/'
>> > within the selector: 'cd-modal .badge' but never did - Ceph - Mgr -
>> > Dashboard
>> > 13. Failed to download key at
>> > http://download.ceph.com/keys/autobuild.asc: Request failed: > > error [Errno 101] Network is unreachable> - Infrastructure
>> > 14. podman: setting cgroup config for procHooks process caused: Unit
>> > libpod-$hash.scope not found - Ceph - Orchestrator
>> >
>> > On Wed, Jan 31, 2024 at 1:41 PM Casey Bodley  wrote:
>> >
>> >> On Mon, Jan 29, 2024 at 4:39 PM Yuri Weinstein 
>> >> wrote:
>> >> >
>> >> > Details of this release are summarized here:
>> >> >
>> >> > https://tracker.ceph.com/issues/64151#note-1
>> >> >
>> >> > Seeking approvals/reviews for:
>> >> >
>> >> > rados - Radek, Laura, Travis, Ernesto, Adam King
>> >> > rgw - Casey
>> >>
>> >> rgw approved, thanks
>> >>
>> >> > fs - Venky
>> >> > rbd - Ilya
>> >> > krbd - in progress
>> >> >
>> >> > upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
>> >> > upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
>> >> >
>> >> > upgrade/pacific-x (quincy) - in progress
>> >> > upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
>> >> >
>> >> > ceph-volume - Guillaume
>> >> >
>> >> > TIA
>> >> > YuriW
>> >> > ___
>> >> > ceph-users mailing list -- ceph-users@ceph.io
>> >> > To unsubscribe send an email to ceph-users-le...@ceph.io
>> >> >
>> >> ___
>> >> Dev mailing list -- d...@ceph.io
>> >> To unsubscribe send an email to dev-le...@ceph.io
>> >>
>> >
>> >
>> > --
>> >
>> > Laura Flores
>> >
>> > She/Her/Hers
>> >
>> > Software Engineer, Ceph Storage 
>> >
>> > Chicago, IL
>> >
>> > lflo...@ibm.com | lflo...@redhat.com 
>> > M: +17087388804
>> >
>> >
>> >
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to 

[ceph-users] Re: pacific 16.2.15 QE validation status

[Ilya Dryomov]

On Thu, Feb 1, 2024 at 5:23 PM Yuri Weinstein  wrote:
>
> Update.
> Seeking approvals/reviews for:
>
> rados - Radek, Laura, Travis, Adam King (see Laura's comments below)
> rgw - Casey approved
> fs - Venky approved
> rbd - Ilya

No issues in RBD, formal approval is pending on [1] which also spills
into RADOS (same job, I believe).

> krbd - Ilya

Approved.

>
> upgrade/nautilus-x (pacific) - fixed by Casey
> upgrade/octopus-x (pacific) - Adam King is looking
> https://tracker.ceph.com/issues/64279
>
> upgrade/pacific-x (quincy) - blocked by
> https://tracker.ceph.com/issues/64256 (Laura, Dan, Adam pls take a
> look)
>
> upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)

I can answer only for test_librbd_python.sh failures -- 3 out of 5 jobs
there.  They popped up because these jobs are running tests from 16.2.7
against librbd from pacific-release (i.e. almost 16.2.15).  One of the
tests happened to pass an incorrect argument and was adjusted together
with the fix for the bug in question [2].

A different variation of this came up in [3] earlier.

I don't think there is any way to fix this other than to disable
test_rbd.TestImage.test_diff_iterate test in upgrades.  We can't
account for such version mismatches when writing tests.

[1] https://tracker.ceph.com/issues/64126
[2] https://tracker.ceph.com/issues/63846
[3] https://tracker.ceph.com/issues/63941

Thanks,

Ilya
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Zakhar Kirpichenko]

Hi,

Please consider not leaving this behind:
https://github.com/ceph/ceph/pull/55109

It's a serious bug, which potentially affects a whole node stability if the
affected mgr is colocated with OSDs. The bug was known for quite a while
and really shouldn't be left unfixed.

/Z

On Thu, 1 Feb 2024 at 18:45, Nizamudeen A  wrote:

> Thanks Laura,
>
> Raised a PR for  https://tracker.ceph.com/issues/57386
> https://github.com/ceph/ceph/pull/55415
>
>
> On Thu, Feb 1, 2024 at 5:15 AM Laura Flores  wrote:
>
> > I reviewed the rados suite. @Adam King , @Nizamudeen
> A
> >  would appreciate a look from you, as there are some
> > orchestrator and dashboard trackers that came up.
> >
> > pacific-release, 16.2.15
> >
> > Failures:
> > 1. https://tracker.ceph.com/issues/62225
> > 2. https://tracker.ceph.com/issues/64278
> > 3. https://tracker.ceph.com/issues/58659
> > 4. https://tracker.ceph.com/issues/58658
> > 5. https://tracker.ceph.com/issues/64280 -- new tracker, worth a
> look
> > from Orch
> > 6. https://tracker.ceph.com/issues/63577
> > 7. https://tracker.ceph.com/issues/63894
> > 8. https://tracker.ceph.com/issues/64126
> > 9. https://tracker.ceph.com/issues/63887
> > 10. https://tracker.ceph.com/issues/61602
> > 11. https://tracker.ceph.com/issues/54071
> > 12. https://tracker.ceph.com/issues/57386
> > 13. https://tracker.ceph.com/issues/64281
> > 14. https://tracker.ceph.com/issues/49287
> >
> > Details:
> > 1. pacific upgrade test fails on 'ceph versions | jq -e' command -
> > Ceph - RADOS
> > 2. Unable to update caps for client.iscsi.iscsi.a - Ceph -
> Orchestrator
> > 3. mds_upgrade_sequence: failure when deploying node-exporter - Ceph
> -
> > Orchestrator
> > 4. mds_upgrade_sequence: Error: initializing source
> > docker://prom/alertmanager:v0.20.0 - Ceph - Orchestrator
> > 5. mgr-nfs-upgrade test times out from failed cephadm daemons - Ceph
> -
> > Orchestrator
> > 6. cephadm: docker.io/library/haproxy: toomanyrequests: You have
> > reached your pull rate limit. You may increase the limit by
> authenticating
> > and upgrading: https://www.docker.com/increase-rate-limit - Ceph -
> > Orchestrator
> > 7. qa: cephadm failed with an error code 1, alertmanager container
> not
> > found. - Ceph - Orchestrator
> > 8. ceph-iscsi build was retriggered and now missing
> > package_manager_version attribute - Ceph
> > 9. Starting alertmanager fails from missing container - Ceph -
> > Orchestrator
> > 10. pacific: cls/test_cls_sdk.sh: Health check failed: 1 pool(s) do
> > not have an application enabled (POOL_APP_NOT_ENABLED) - Ceph - RADOS
> > 11. rados/cephadm/osds: Invalid command: missing required parameter
> > hostname() - Ceph - Orchestrator
> > 12. cephadm/test_dashboard_e2e.sh: Expected to find content:
> '/^foo$/'
> > within the selector: 'cd-modal .badge' but never did - Ceph - Mgr -
> > Dashboard
> > 13. Failed to download key at
> > http://download.ceph.com/keys/autobuild.asc: Request failed:  > error [Errno 101] Network is unreachable> - Infrastructure
> > 14. podman: setting cgroup config for procHooks process caused: Unit
> > libpod-$hash.scope not found - Ceph - Orchestrator
> >
> > On Wed, Jan 31, 2024 at 1:41 PM Casey Bodley  wrote:
> >
> >> On Mon, Jan 29, 2024 at 4:39 PM Yuri Weinstein 
> >> wrote:
> >> >
> >> > Details of this release are summarized here:
> >> >
> >> > https://tracker.ceph.com/issues/64151#note-1
> >> >
> >> > Seeking approvals/reviews for:
> >> >
> >> > rados - Radek, Laura, Travis, Ernesto, Adam King
> >> > rgw - Casey
> >>
> >> rgw approved, thanks
> >>
> >> > fs - Venky
> >> > rbd - Ilya
> >> > krbd - in progress
> >> >
> >> > upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
> >> > upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
> >> >
> >> > upgrade/pacific-x (quincy) - in progress
> >> > upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
> >> >
> >> > ceph-volume - Guillaume
> >> >
> >> > TIA
> >> > YuriW
> >> > ___
> >> > ceph-users mailing list -- ceph-users@ceph.io
> >> > To unsubscribe send an email to ceph-users-le...@ceph.io
> >> >
> >> ___
> >> Dev mailing list -- d...@ceph.io
> >> To unsubscribe send an email to dev-le...@ceph.io
> >>
> >
> >
> > --
> >
> > Laura Flores
> >
> > She/Her/Hers
> >
> > Software Engineer, Ceph Storage 
> >
> > Chicago, IL
> >
> > lflo...@ibm.com | lflo...@redhat.com 
> > M: +17087388804
> >
> >
> >
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Nizamudeen A]

Thanks Laura,

Raised a PR for  https://tracker.ceph.com/issues/57386
https://github.com/ceph/ceph/pull/55415


On Thu, Feb 1, 2024 at 5:15 AM Laura Flores  wrote:

> I reviewed the rados suite. @Adam King , @Nizamudeen A
>  would appreciate a look from you, as there are some
> orchestrator and dashboard trackers that came up.
>
> pacific-release, 16.2.15
>
> Failures:
> 1. https://tracker.ceph.com/issues/62225
> 2. https://tracker.ceph.com/issues/64278
> 3. https://tracker.ceph.com/issues/58659
> 4. https://tracker.ceph.com/issues/58658
> 5. https://tracker.ceph.com/issues/64280 -- new tracker, worth a look
> from Orch
> 6. https://tracker.ceph.com/issues/63577
> 7. https://tracker.ceph.com/issues/63894
> 8. https://tracker.ceph.com/issues/64126
> 9. https://tracker.ceph.com/issues/63887
> 10. https://tracker.ceph.com/issues/61602
> 11. https://tracker.ceph.com/issues/54071
> 12. https://tracker.ceph.com/issues/57386
> 13. https://tracker.ceph.com/issues/64281
> 14. https://tracker.ceph.com/issues/49287
>
> Details:
> 1. pacific upgrade test fails on 'ceph versions | jq -e' command -
> Ceph - RADOS
> 2. Unable to update caps for client.iscsi.iscsi.a - Ceph - Orchestrator
> 3. mds_upgrade_sequence: failure when deploying node-exporter - Ceph -
> Orchestrator
> 4. mds_upgrade_sequence: Error: initializing source
> docker://prom/alertmanager:v0.20.0 - Ceph - Orchestrator
> 5. mgr-nfs-upgrade test times out from failed cephadm daemons - Ceph -
> Orchestrator
> 6. cephadm: docker.io/library/haproxy: toomanyrequests: You have
> reached your pull rate limit. You may increase the limit by authenticating
> and upgrading: https://www.docker.com/increase-rate-limit - Ceph -
> Orchestrator
> 7. qa: cephadm failed with an error code 1, alertmanager container not
> found. - Ceph - Orchestrator
> 8. ceph-iscsi build was retriggered and now missing
> package_manager_version attribute - Ceph
> 9. Starting alertmanager fails from missing container - Ceph -
> Orchestrator
> 10. pacific: cls/test_cls_sdk.sh: Health check failed: 1 pool(s) do
> not have an application enabled (POOL_APP_NOT_ENABLED) - Ceph - RADOS
> 11. rados/cephadm/osds: Invalid command: missing required parameter
> hostname() - Ceph - Orchestrator
> 12. cephadm/test_dashboard_e2e.sh: Expected to find content: '/^foo$/'
> within the selector: 'cd-modal .badge' but never did - Ceph - Mgr -
> Dashboard
> 13. Failed to download key at
> http://download.ceph.com/keys/autobuild.asc: Request failed:  error [Errno 101] Network is unreachable> - Infrastructure
> 14. podman: setting cgroup config for procHooks process caused: Unit
> libpod-$hash.scope not found - Ceph - Orchestrator
>
> On Wed, Jan 31, 2024 at 1:41 PM Casey Bodley  wrote:
>
>> On Mon, Jan 29, 2024 at 4:39 PM Yuri Weinstein 
>> wrote:
>> >
>> > Details of this release are summarized here:
>> >
>> > https://tracker.ceph.com/issues/64151#note-1
>> >
>> > Seeking approvals/reviews for:
>> >
>> > rados - Radek, Laura, Travis, Ernesto, Adam King
>> > rgw - Casey
>>
>> rgw approved, thanks
>>
>> > fs - Venky
>> > rbd - Ilya
>> > krbd - in progress
>> >
>> > upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
>> > upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
>> >
>> > upgrade/pacific-x (quincy) - in progress
>> > upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
>> >
>> > ceph-volume - Guillaume
>> >
>> > TIA
>> > YuriW
>> > ___
>> > ceph-users mailing list -- ceph-users@ceph.io
>> > To unsubscribe send an email to ceph-users-le...@ceph.io
>> >
>> ___
>> Dev mailing list -- d...@ceph.io
>> To unsubscribe send an email to dev-le...@ceph.io
>>
>
>
> --
>
> Laura Flores
>
> She/Her/Hers
>
> Software Engineer, Ceph Storage 
>
> Chicago, IL
>
> lflo...@ibm.com | lflo...@redhat.com 
> M: +17087388804
>
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Yuri Weinstein]

Update.
Seeking approvals/reviews for:

rados - Radek, Laura, Travis, Adam King (see Laura's comments below)
rgw - Casey approved
fs - Venky approved
rbd - Ilya
krbd - Ilya

upgrade/nautilus-x (pacific) - fixed by Casey
upgrade/octopus-x (pacific) - Adam King is looking
https://tracker.ceph.com/issues/64279

upgrade/pacific-x (quincy) - blocked by
https://tracker.ceph.com/issues/64256 (Laura, Dan, Adam pls take a
look)

upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)

ceph-volume - Guillaume is fixing
https://tracker.ceph.com/issues/64248
https://github.com/ceph/ceph/pull/55376

In addition to all these issues, we are considering adding a fix for
https://tracker.ceph.com/issues/63425
(https://github.com/ceph/ceph/pull/54312).  Adam King and Ilya are
looking into this.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Laura Flores]

I reviewed the rados suite. @Adam King , @Nizamudeen A
 would appreciate a look from you, as there are some
orchestrator and dashboard trackers that came up.

pacific-release, 16.2.15

Failures:
1. https://tracker.ceph.com/issues/62225
2. https://tracker.ceph.com/issues/64278
3. https://tracker.ceph.com/issues/58659
4. https://tracker.ceph.com/issues/58658
5. https://tracker.ceph.com/issues/64280 -- new tracker, worth a look
from Orch
6. https://tracker.ceph.com/issues/63577
7. https://tracker.ceph.com/issues/63894
8. https://tracker.ceph.com/issues/64126
9. https://tracker.ceph.com/issues/63887
10. https://tracker.ceph.com/issues/61602
11. https://tracker.ceph.com/issues/54071
12. https://tracker.ceph.com/issues/57386
13. https://tracker.ceph.com/issues/64281
14. https://tracker.ceph.com/issues/49287

Details:
1. pacific upgrade test fails on 'ceph versions | jq -e' command - Ceph
- RADOS
2. Unable to update caps for client.iscsi.iscsi.a - Ceph - Orchestrator
3. mds_upgrade_sequence: failure when deploying node-exporter - Ceph -
Orchestrator
4. mds_upgrade_sequence: Error: initializing source
docker://prom/alertmanager:v0.20.0 - Ceph - Orchestrator
5. mgr-nfs-upgrade test times out from failed cephadm daemons - Ceph -
Orchestrator
6. cephadm: docker.io/library/haproxy: toomanyrequests: You have
reached your pull rate limit. You may increase the limit by authenticating
and upgrading: https://www.docker.com/increase-rate-limit - Ceph -
Orchestrator
7. qa: cephadm failed with an error code 1, alertmanager container not
found. - Ceph - Orchestrator
8. ceph-iscsi build was retriggered and now missing
package_manager_version attribute - Ceph
9. Starting alertmanager fails from missing container - Ceph -
Orchestrator
10. pacific: cls/test_cls_sdk.sh: Health check failed: 1 pool(s) do not
have an application enabled (POOL_APP_NOT_ENABLED) - Ceph - RADOS
11. rados/cephadm/osds: Invalid command: missing required parameter
hostname() - Ceph - Orchestrator
12. cephadm/test_dashboard_e2e.sh: Expected to find content: '/^foo$/'
within the selector: 'cd-modal .badge' but never did - Ceph - Mgr -
Dashboard
13. Failed to download key at
http://download.ceph.com/keys/autobuild.asc: Request failed:  - Infrastructure
14. podman: setting cgroup config for procHooks process caused: Unit
libpod-$hash.scope not found - Ceph - Orchestrator

On Wed, Jan 31, 2024 at 1:41 PM Casey Bodley  wrote:

> On Mon, Jan 29, 2024 at 4:39 PM Yuri Weinstein 
> wrote:
> >
> > Details of this release are summarized here:
> >
> > https://tracker.ceph.com/issues/64151#note-1
> >
> > Seeking approvals/reviews for:
> >
> > rados - Radek, Laura, Travis, Ernesto, Adam King
> > rgw - Casey
>
> rgw approved, thanks
>
> > fs - Venky
> > rbd - Ilya
> > krbd - in progress
> >
> > upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
> > upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
> >
> > upgrade/pacific-x (quincy) - in progress
> > upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
> >
> > ceph-volume - Guillaume
> >
> > TIA
> > YuriW
> > ___
> > ceph-users mailing list -- ceph-users@ceph.io
> > To unsubscribe send an email to ceph-users-le...@ceph.io
> >
> ___
> Dev mailing list -- d...@ceph.io
> To unsubscribe send an email to dev-le...@ceph.io
>


-- 

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage 

Chicago, IL

lflo...@ibm.com | lflo...@redhat.com 
M: +17087388804
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Casey Bodley]

On Mon, Jan 29, 2024 at 4:39 PM Yuri Weinstein  wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals/reviews for:
>
> rados - Radek, Laura, Travis, Ernesto, Adam King
> rgw - Casey

rgw approved, thanks

> fs - Venky
> rbd - Ilya
> krbd - in progress
>
> upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
> upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
>
> upgrade/pacific-x (quincy) - in progress
> upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
>
> ceph-volume - Guillaume
>
> TIA
> YuriW
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Ilya Dryomov]

On Tue, Jan 30, 2024 at 9:24 PM Yuri Weinstein  wrote:
>
> Update.
> Seeking approvals/reviews for:
>
> rados - Radek, Laura, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky
> rbd - Ilya

Hi Yuri,

rbd looks good overall but we are missing iSCSI coverage due to
https://tracker.ceph.com/issues/64126:

https://pulpito.ceph.com/yuriw-2024-01-25_17:27:10-rbd-pacific-release-distro-default-smithi/7532167
https://pulpito.ceph.com/yuriw-2024-01-26_17:05:20-rbd-pacific-release-distro-default-smithi/7535193

Adam, did you get a chance to look into it?

> krbd - Ilya

Please do another rerun for krbd -- I want to see one of those
remaining jobs pass.

Thanks,

Ilya
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Venky Shankar]

On Tue, Jan 30, 2024 at 3:08 AM Yuri Weinstein  wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals/reviews for:
>
> rados - Radek, Laura, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky

fs approved. Failures are -
https://tracker.ceph.com/projects/cephfs/wiki/Pacific#31-Jan-2024

> rbd - Ilya
> krbd - in progress
>
> upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
> upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
>
> upgrade/pacific-x (quincy) - in progress
> upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
>
> ceph-volume - Guillaume
>
> TIA
> YuriW
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>


-- 
Cheers,
Venky
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Yuri Weinstein]

Update.
Seeking approvals/reviews for:

rados - Radek, Laura, Travis, Ernesto, Adam King
rgw - Casey
fs - Venky
rbd - Ilya
krbd - Ilya

upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)

upgrade/pacific-x (quincy) - blocked by
https://tracker.ceph.com/issues/64256 (Laura, Dan, Adam pls take a
look)
upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)

ceph-volume - Guillaume is fixing

On Mon, Jan 29, 2024 at 1:38 PM Yuri Weinstein  wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals/reviews for:
>
> rados - Radek, Laura, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky
> rbd - Ilya
> krbd - in progress
>
> upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
> upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
>
> upgrade/pacific-x (quincy) - in progress
> upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
>
> ceph-volume - Guillaume
>
> TIA
> YuriW
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Nizamudeen A]

dashboard looks good! approved.

Regards,
Nizam

On Tue, Jan 30, 2024 at 3:09 AM Yuri Weinstein  wrote:

> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/64151#note-1
>
> Seeking approvals/reviews for:
>
> rados - Radek, Laura, Travis, Ernesto, Adam King
> rgw - Casey
> fs - Venky
> rbd - Ilya
> krbd - in progress
>
> upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
> upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)
>
> upgrade/pacific-x (quincy) - in progress
> upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)
>
> ceph-volume - Guillaume
>
> TIA
> YuriW
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: pacific 16.2.15 QE validation status

[Guillaume Abrioux]

Hi Yuri,

The ceph-volume failure is a valid bug.
Investigating for the root cause of it and will submit a patch.

Thanks!

--
Guillaume Abrioux
Software Engineer

From: Yuri Weinstein 
Date: Monday, 29 January 2024 at 22:38
To: dev , ceph-users 
Subject: [EXTERNAL] [ceph-users] pacific 16.2.15 QE validation status
Details of this release are summarized here:

https://tracker.ceph.com/issues/64151#note-1

Seeking approvals/reviews for:

rados - Radek, Laura, Travis, Ernesto, Adam King
rgw - Casey
fs - Venky
rbd - Ilya
krbd - in progress

upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed)
upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed)

upgrade/pacific-x (quincy) - in progress
upgrade/pacific-p2p - Ilya PTL (maybe rbd related?)

ceph-volume - Guillaume

TIA
YuriW
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

Unless otherwise stated above:

Compagnie IBM France
Siège Social : 17, avenue de l'Europe, 92275 Bois-Colombes Cedex
RCS Nanterre 552 118 465
Forme Sociale : S.A.S.
Capital Social : 664 069 390,60 €
SIRET : 552 118 465 03644 - Code NAF 6203Z
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io