[ceph-users] Re: quincy v17.2.6 QE Validation status

[Yuri Weinstein]

Update:

The release is ready to be published, but we have problems with the signing
VM.
That VM is being actively fixed as we speak (thank you, Dan, Ken, and Zack!)

However, we need to delay the publishing till early next week.
Thx and sorry for the delay.

YuriW


On Wed, Apr 5, 2023 at 7:16 AM Josh Durgin  wrote:

> The LRC upgraded with no problems, so this release is good to go!
>
> Josh
>
> On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein  wrote:
>
>> Josh, the release is ready for your review and approval.
>>
>> Adam, can you please update the LRC upgrade to 17.2.6 RC?
>>
>> Thx
>>
>>
>> On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein 
>> wrote:
>>
>>> The release has been approved.
>>>
>>> And the gibba cluster upgraded.
>>>
>>> We are awaiting the LRC upgrade and then/or in parallel will publish an
>>> RC for testing.
>>>
>>> ETA for the release publishing is 04/07/23
>>>
>>> On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:
>>>
 upgrades approved!

 Thanks,
 Neha

 On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <
 rzarz...@redhat.com> wrote:

> rados: approved!
>
> On Mon, Mar 27, 2023 at 7:02 PM Laura Flores 
> wrote:
>
>> Rados review, second round:
>>
>> Failures:
>> 1. https://tracker.ceph.com/issues/58560
>> 2. https://tracker.ceph.com/issues/58476
>> 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
>> 4. https://tracker.ceph.com/issues/49287
>> 5. https://tracker.ceph.com/issues/58585
>>
>> Details:
>> 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
>> Infrastructure
>> 2. test_non_existent_cluster: cluster does not exist - Ceph -
>> Orchestrator
>> 3. test_dashboard_e2e.sh: Conflicting peer dependency:
>> postcss@8.4.21 - Ceph - Mgr - Dashboard
>> 4. podman: setting cgroup config for procHooks process caused:
>> Unit libpod-$hash.scope not found - Ceph - Orchestrator
>> 5. rook: failed to pull kubelet image - Ceph - Orchestrator
>>
>> @Radoslaw Zarzynski  will give final approval
>> for rados.
>>
>> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
>> wrote:
>>
>>> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
>>> wrote:
>>> >
>>> > Details of this release are updated here:
>>> >
>>> > https://tracker.ceph.com/issues/59070#note-1
>>> > Release Notes - TBD
>>> >
>>> > The slowness we experienced seemed to be self-cured.
>>> > Neha, Radek, and Laura please provide any findings if you have
>>> them.
>>> >
>>> > Seeking approvals/reviews for:
>>> >
>>> > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2
>>> with
>>> > PRs merged on top of quincy-release)
>>> > rgw - Casey (rerun on Build 2 with PRs merged on top of
>>> quincy-release)
>>>
>>> rgw approved
>>>
>>> > fs - Venky
>>> >
>>> > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
>>> updates?)
>>> > upgrade/pacific-x - Neha, Laura, Ilya see
>>> https://tracker.ceph.com/issues/58914
>>> > upgrade/quincy-p2p - Neha, Laura
>>> > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue
>>> Adam
>>> > Kraitman any updates?)
>>> > powercycle - Brad
>>> >
>>> > Please reply to this email with approval and/or trackers of known
>>> > issues/PRs to address them.
>>> >
>>> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>>> > RC release - pending major suites approvals.
>>> >
>>> > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <
>>> ywein...@redhat.com> wrote:
>>> > >
>>> > > Details of this release are summarized here:
>>> > >
>>> > > https://tracker.ceph.com/issues/59070#note-1
>>> > > Release Notes - TBD
>>> > >
>>> > > The reruns were in the queue for 4 days because of some slowness
>>> issues.
>>> > > The core team (Neha, Radek, Laura, and others) are trying to
>>> narrow
>>> > > down the root cause.
>>> > >
>>> > > Seeking approvals/reviews for:
>>> > >
>>> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have
>>> to test
>>> > > and merge at least one PR
>>> https://github.com/ceph/ceph/pull/50575 for
>>> > > the core)
>>> > > rgw - Casey
>>> > > fs - Venky (the fs suite has an unusually high amount of failed
>>> jobs,
>>> > > any reason to suspect it in the observed slowness?)
>>> > > orch - Adam King
>>> > > rbd - Ilya
>>> > > krbd - Ilya
>>> > > upgrade/octopus-x - Laura is looking into failures
>>> > > upgrade/pacific-x - Laura is looking into failures
>>> > > upgrade/quincy-p2p - Laura is looking into failures
>>> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
>>> Kraitman
>>> > > is looking into it
>>> > > powercycle - 

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Crown Upholstery]

ceph-users@ceph.io

stop

From: Thomas Widhalm 
Sent: Wednesday, April 5, 2023 7:26 AM
To: ceph-users@ceph.io 
Subject: [ceph-users] Re: quincy v17.2.6 QE Validation status

Sorry for interfereing, but: Wh!! Thank you so much for the
great work. Can't wait for the release with a good chance to get access
to my data again.

On 05.04.23 16:15, Josh Durgin wrote:
> The LRC upgraded with no problems, so this release is good to go!
>
> Josh
>
> On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein  wrote:
>
>> Josh, the release is ready for your review and approval.
>>
>> Adam, can you please update the LRC upgrade to 17.2.6 RC?
>>
>> Thx
>>
>>
>> On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein 
>> wrote:
>>
>>> The release has been approved.
>>>
>>> And the gibba cluster upgraded.
>>>
>>> We are awaiting the LRC upgrade and then/or in parallel will publish an
>>> RC for testing.
>>>
>>> ETA for the release publishing is 04/07/23
>>>
>>> On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:
>>>
>>>> upgrades approved!
>>>>
>>>> Thanks,
>>>> Neha
>>>>
>>>> On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski 
>>>> wrote:
>>>>
>>>>> rados: approved!
>>>>>
>>>>> On Mon, Mar 27, 2023 at 7:02 PM Laura Flores 
>>>>> wrote:
>>>>>
>>>>>> Rados review, second round:
>>>>>>
>>>>>> Failures:
>>>>>>  1. https://tracker.ceph.com/issues/58560
>>>>>>  2. https://tracker.ceph.com/issues/58476
>>>>>>  3. https://tracker.ceph.com/issues/58475 -- pending Q backport
>>>>>>  4. https://tracker.ceph.com/issues/49287
>>>>>>  5. https://tracker.ceph.com/issues/58585
>>>>>>
>>>>>> Details:
>>>>>>  1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
>>>>>> Infrastructure
>>>>>>  2. test_non_existent_cluster: cluster does not exist - Ceph -
>>>>>> Orchestrator
>>>>>>  3. test_dashboard_e2e.sh: Conflicting peer dependency:
>>>>>> postcss@8.4.21 - Ceph - Mgr - Dashboard
>>>>>>  4. podman: setting cgroup config for procHooks process caused:
>>>>>> Unit libpod-$hash.scope not found - Ceph - Orchestrator
>>>>>>  5. rook: failed to pull kubelet image - Ceph - Orchestrator
>>>>>>
>>>>>> @Radoslaw Zarzynski  will give final approval
>>>>>> for rados.
>>>>>>
>>>>>> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
>>>>>> wrote:
>>>>>>
>>>>>>> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Details of this release are updated here:
>>>>>>>>
>>>>>>>> https://tracker.ceph.com/issues/59070#note-1
>>>>>>>> Release Notes - TBD
>>>>>>>>
>>>>>>>> The slowness we experienced seemed to be self-cured.
>>>>>>>> Neha, Radek, and Laura please provide any findings if you have them.
>>>>>>>>
>>>>>>>> Seeking approvals/reviews for:
>>>>>>>>
>>>>>>>> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2
>>>>>>> with
>>>>>>>> PRs merged on top of quincy-release)
>>>>>>>> rgw - Casey (rerun on Build 2 with PRs merged on top of
>>>>>>> quincy-release)
>>>>>>>
>>>>>>> rgw approved
>>>>>>>
>>>>>>>> fs - Venky
>>>>>>>>
>>>>>>>> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
>>>>>>> updates?)
>>>>>>>> upgrade/pacific-x - Neha, Laura, Ilya see
>>>>>>> https://tracker.ceph.com/issues/58914
>>>>>>>> upgrade/quincy-p2p - Neha, Laura
>>>>>>>> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue
>>>>>>> Adam
>>>>>>>> Kraitman any updates?)
>>>>>>>> powercycle - Brad
>>>>>>>>

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Crown Upholstery]

d...@ceph.io

stop

From: Thomas Widhalm 
Sent: Wednesday, April 5, 2023 7:26 AM
To: ceph-users@ceph.io 
Subject: [ceph-users] Re: quincy v17.2.6 QE Validation status

Sorry for interfereing, but: Wh!! Thank you so much for the
great work. Can't wait for the release with a good chance to get access
to my data again.

On 05.04.23 16:15, Josh Durgin wrote:
> The LRC upgraded with no problems, so this release is good to go!
>
> Josh
>
> On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein  wrote:
>
>> Josh, the release is ready for your review and approval.
>>
>> Adam, can you please update the LRC upgrade to 17.2.6 RC?
>>
>> Thx
>>
>>
>> On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein 
>> wrote:
>>
>>> The release has been approved.
>>>
>>> And the gibba cluster upgraded.
>>>
>>> We are awaiting the LRC upgrade and then/or in parallel will publish an
>>> RC for testing.
>>>
>>> ETA for the release publishing is 04/07/23
>>>
>>> On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:
>>>
>>>> upgrades approved!
>>>>
>>>> Thanks,
>>>> Neha
>>>>
>>>> On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski 
>>>> wrote:
>>>>
>>>>> rados: approved!
>>>>>
>>>>> On Mon, Mar 27, 2023 at 7:02 PM Laura Flores 
>>>>> wrote:
>>>>>
>>>>>> Rados review, second round:
>>>>>>
>>>>>> Failures:
>>>>>>  1. https://tracker.ceph.com/issues/58560
>>>>>>  2. https://tracker.ceph.com/issues/58476
>>>>>>  3. https://tracker.ceph.com/issues/58475 -- pending Q backport
>>>>>>  4. https://tracker.ceph.com/issues/49287
>>>>>>  5. https://tracker.ceph.com/issues/58585
>>>>>>
>>>>>> Details:
>>>>>>  1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
>>>>>> Infrastructure
>>>>>>  2. test_non_existent_cluster: cluster does not exist - Ceph -
>>>>>> Orchestrator
>>>>>>  3. test_dashboard_e2e.sh: Conflicting peer dependency:
>>>>>> postcss@8.4.21 - Ceph - Mgr - Dashboard
>>>>>>  4. podman: setting cgroup config for procHooks process caused:
>>>>>> Unit libpod-$hash.scope not found - Ceph - Orchestrator
>>>>>>  5. rook: failed to pull kubelet image - Ceph - Orchestrator
>>>>>>
>>>>>> @Radoslaw Zarzynski  will give final approval
>>>>>> for rados.
>>>>>>
>>>>>> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
>>>>>> wrote:
>>>>>>
>>>>>>> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Details of this release are updated here:
>>>>>>>>
>>>>>>>> https://tracker.ceph.com/issues/59070#note-1
>>>>>>>> Release Notes - TBD
>>>>>>>>
>>>>>>>> The slowness we experienced seemed to be self-cured.
>>>>>>>> Neha, Radek, and Laura please provide any findings if you have them.
>>>>>>>>
>>>>>>>> Seeking approvals/reviews for:
>>>>>>>>
>>>>>>>> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2
>>>>>>> with
>>>>>>>> PRs merged on top of quincy-release)
>>>>>>>> rgw - Casey (rerun on Build 2 with PRs merged on top of
>>>>>>> quincy-release)
>>>>>>>
>>>>>>> rgw approved
>>>>>>>
>>>>>>>> fs - Venky
>>>>>>>>
>>>>>>>> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
>>>>>>> updates?)
>>>>>>>> upgrade/pacific-x - Neha, Laura, Ilya see
>>>>>>> https://tracker.ceph.com/issues/58914
>>>>>>>> upgrade/quincy-p2p - Neha, Laura
>>>>>>>> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue
>>>>>>> Adam
>>>>>>>> Kraitman any updates?)
>>>>>>>> powercycle - Brad
>>>>>>>>

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Yuri Weinstein]

Thx, Josh!

We will start releasing now.

Release notes - https://github.com/ceph/ceph/pull/50721

On Wed, Apr 5, 2023 at 7:16 AM Josh Durgin  wrote:

> The LRC upgraded with no problems, so this release is good to go!
>
> Josh
>
> On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein  wrote:
>
>> Josh, the release is ready for your review and approval.
>>
>> Adam, can you please update the LRC upgrade to 17.2.6 RC?
>>
>> Thx
>>
>>
>> On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein 
>> wrote:
>>
>>> The release has been approved.
>>>
>>> And the gibba cluster upgraded.
>>>
>>> We are awaiting the LRC upgrade and then/or in parallel will publish an
>>> RC for testing.
>>>
>>> ETA for the release publishing is 04/07/23
>>>
>>> On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:
>>>
 upgrades approved!

 Thanks,
 Neha

 On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <
 rzarz...@redhat.com> wrote:

> rados: approved!
>
> On Mon, Mar 27, 2023 at 7:02 PM Laura Flores 
> wrote:
>
>> Rados review, second round:
>>
>> Failures:
>> 1. https://tracker.ceph.com/issues/58560
>> 2. https://tracker.ceph.com/issues/58476
>> 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
>> 4. https://tracker.ceph.com/issues/49287
>> 5. https://tracker.ceph.com/issues/58585
>>
>> Details:
>> 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
>> Infrastructure
>> 2. test_non_existent_cluster: cluster does not exist - Ceph -
>> Orchestrator
>> 3. test_dashboard_e2e.sh: Conflicting peer dependency:
>> postcss@8.4.21 - Ceph - Mgr - Dashboard
>> 4. podman: setting cgroup config for procHooks process caused:
>> Unit libpod-$hash.scope not found - Ceph - Orchestrator
>> 5. rook: failed to pull kubelet image - Ceph - Orchestrator
>>
>> @Radoslaw Zarzynski  will give final approval
>> for rados.
>>
>> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
>> wrote:
>>
>>> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
>>> wrote:
>>> >
>>> > Details of this release are updated here:
>>> >
>>> > https://tracker.ceph.com/issues/59070#note-1
>>> > Release Notes - TBD
>>> >
>>> > The slowness we experienced seemed to be self-cured.
>>> > Neha, Radek, and Laura please provide any findings if you have
>>> them.
>>> >
>>> > Seeking approvals/reviews for:
>>> >
>>> > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2
>>> with
>>> > PRs merged on top of quincy-release)
>>> > rgw - Casey (rerun on Build 2 with PRs merged on top of
>>> quincy-release)
>>>
>>> rgw approved
>>>
>>> > fs - Venky
>>> >
>>> > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
>>> updates?)
>>> > upgrade/pacific-x - Neha, Laura, Ilya see
>>> https://tracker.ceph.com/issues/58914
>>> > upgrade/quincy-p2p - Neha, Laura
>>> > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue
>>> Adam
>>> > Kraitman any updates?)
>>> > powercycle - Brad
>>> >
>>> > Please reply to this email with approval and/or trackers of known
>>> > issues/PRs to address them.
>>> >
>>> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>>> > RC release - pending major suites approvals.
>>> >
>>> > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <
>>> ywein...@redhat.com> wrote:
>>> > >
>>> > > Details of this release are summarized here:
>>> > >
>>> > > https://tracker.ceph.com/issues/59070#note-1
>>> > > Release Notes - TBD
>>> > >
>>> > > The reruns were in the queue for 4 days because of some slowness
>>> issues.
>>> > > The core team (Neha, Radek, Laura, and others) are trying to
>>> narrow
>>> > > down the root cause.
>>> > >
>>> > > Seeking approvals/reviews for:
>>> > >
>>> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have
>>> to test
>>> > > and merge at least one PR
>>> https://github.com/ceph/ceph/pull/50575 for
>>> > > the core)
>>> > > rgw - Casey
>>> > > fs - Venky (the fs suite has an unusually high amount of failed
>>> jobs,
>>> > > any reason to suspect it in the observed slowness?)
>>> > > orch - Adam King
>>> > > rbd - Ilya
>>> > > krbd - Ilya
>>> > > upgrade/octopus-x - Laura is looking into failures
>>> > > upgrade/pacific-x - Laura is looking into failures
>>> > > upgrade/quincy-p2p - Laura is looking into failures
>>> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
>>> Kraitman
>>> > > is looking into it
>>> > > powercycle - Brad
>>> > > ceph-volume - needs a rerun on merged
>>> > > https://github.com/ceph/ceph-ansible/pull/7409
>>> > >
>>> > > Please reply to this email with 

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Thomas Widhalm]

Sorry for interfereing, but: Wh!! Thank you so much for the 
great work. Can't wait for the release with a good chance to get access 
to my data again.


On 05.04.23 16:15, Josh Durgin wrote:

The LRC upgraded with no problems, so this release is good to go!

Josh

On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein  wrote:


Josh, the release is ready for your review and approval.

Adam, can you please update the LRC upgrade to 17.2.6 RC?

Thx


On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein 
wrote:


The release has been approved.

And the gibba cluster upgraded.

We are awaiting the LRC upgrade and then/or in parallel will publish an
RC for testing.

ETA for the release publishing is 04/07/23

On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:


upgrades approved!

Thanks,
Neha

On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski 
wrote:


rados: approved!

On Mon, Mar 27, 2023 at 7:02 PM Laura Flores 
wrote:


Rados review, second round:

Failures:
 1. https://tracker.ceph.com/issues/58560
 2. https://tracker.ceph.com/issues/58476
 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
 4. https://tracker.ceph.com/issues/49287
 5. https://tracker.ceph.com/issues/58585

Details:
 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
Infrastructure
 2. test_non_existent_cluster: cluster does not exist - Ceph -
Orchestrator
 3. test_dashboard_e2e.sh: Conflicting peer dependency:
postcss@8.4.21 - Ceph - Mgr - Dashboard
 4. podman: setting cgroup config for procHooks process caused:
Unit libpod-$hash.scope not found - Ceph - Orchestrator
 5. rook: failed to pull kubelet image - Ceph - Orchestrator

@Radoslaw Zarzynski  will give final approval
for rados.

On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
wrote:


On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
wrote:


Details of this release are updated here:

https://tracker.ceph.com/issues/59070#note-1
Release Notes - TBD

The slowness we experienced seemed to be self-cured.
Neha, Radek, and Laura please provide any findings if you have them.

Seeking approvals/reviews for:

rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2

with

PRs merged on top of quincy-release)
rgw - Casey (rerun on Build 2 with PRs merged on top of

quincy-release)

rgw approved


fs - Venky

upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any

updates?)

upgrade/pacific-x - Neha, Laura, Ilya see

https://tracker.ceph.com/issues/58914

upgrade/quincy-p2p - Neha, Laura
client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue

Adam

Kraitman any updates?)
powercycle - Brad

Please reply to this email with approval and/or trackers of known
issues/PRs to address them.

Josh, Neha - gibba and LRC upgrades pending major suites approvals.
RC release - pending major suites approvals.

On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 

wrote:


Details of this release are summarized here:

https://tracker.ceph.com/issues/59070#note-1
Release Notes - TBD

The reruns were in the queue for 4 days because of some slowness

issues.

The core team (Neha, Radek, Laura, and others) are trying to

narrow

down the root cause.

Seeking approvals/reviews for:

rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to

test

and merge at least one PR https://github.com/ceph/ceph/pull/50575

for

the core)
rgw - Casey
fs - Venky (the fs suite has an unusually high amount of failed

jobs,

any reason to suspect it in the observed slowness?)
orch - Adam King
rbd - Ilya
krbd - Ilya
upgrade/octopus-x - Laura is looking into failures
upgrade/pacific-x - Laura is looking into failures
upgrade/quincy-p2p - Laura is looking into failures
client-upgrade-octopus-quincy-quincy - missing packages, Adam

Kraitman

is looking into it
powercycle - Brad
ceph-volume - needs a rerun on merged
https://github.com/ceph/ceph-ansible/pull/7409

Please reply to this email with approval and/or trackers of known
issues/PRs to address them.

Also, share any findings or hypnosis about the slowness in the
execution of the suite.

Josh, Neha - gibba and LRC upgrades pending major suites

approvals.

RC release - pending major suites approvals.

Thx
YuriW

___
Dev mailing list -- d...@ceph.io
To unsubscribe send an email to dev-le...@ceph.io

___
Dev mailing list -- d...@ceph.io
To unsubscribe send an email to dev-le...@ceph.io




--

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage 

Chicago, IL

lflo...@ibm.com | lflo...@redhat.com 
M: +17087388804


___

Dev mailing list -- d...@ceph.io
To unsubscribe send an email to dev-le...@ceph.io


___

Dev mailing list -- d...@ceph.io
To unsubscribe send an email to dev-le...@ceph.io


___
ceph-users mailing list -- ceph-users@ceph.io
To 

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Josh Durgin]

The LRC upgraded with no problems, so this release is good to go!

Josh

On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein  wrote:

> Josh, the release is ready for your review and approval.
>
> Adam, can you please update the LRC upgrade to 17.2.6 RC?
>
> Thx
>
>
> On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein 
> wrote:
>
>> The release has been approved.
>>
>> And the gibba cluster upgraded.
>>
>> We are awaiting the LRC upgrade and then/or in parallel will publish an
>> RC for testing.
>>
>> ETA for the release publishing is 04/07/23
>>
>> On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:
>>
>>> upgrades approved!
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski 
>>> wrote:
>>>
 rados: approved!

 On Mon, Mar 27, 2023 at 7:02 PM Laura Flores 
 wrote:

> Rados review, second round:
>
> Failures:
> 1. https://tracker.ceph.com/issues/58560
> 2. https://tracker.ceph.com/issues/58476
> 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
> 4. https://tracker.ceph.com/issues/49287
> 5. https://tracker.ceph.com/issues/58585
>
> Details:
> 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
> Infrastructure
> 2. test_non_existent_cluster: cluster does not exist - Ceph -
> Orchestrator
> 3. test_dashboard_e2e.sh: Conflicting peer dependency:
> postcss@8.4.21 - Ceph - Mgr - Dashboard
> 4. podman: setting cgroup config for procHooks process caused:
> Unit libpod-$hash.scope not found - Ceph - Orchestrator
> 5. rook: failed to pull kubelet image - Ceph - Orchestrator
>
> @Radoslaw Zarzynski  will give final approval
> for rados.
>
> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
> wrote:
>
>> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
>> wrote:
>> >
>> > Details of this release are updated here:
>> >
>> > https://tracker.ceph.com/issues/59070#note-1
>> > Release Notes - TBD
>> >
>> > The slowness we experienced seemed to be self-cured.
>> > Neha, Radek, and Laura please provide any findings if you have them.
>> >
>> > Seeking approvals/reviews for:
>> >
>> > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2
>> with
>> > PRs merged on top of quincy-release)
>> > rgw - Casey (rerun on Build 2 with PRs merged on top of
>> quincy-release)
>>
>> rgw approved
>>
>> > fs - Venky
>> >
>> > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
>> updates?)
>> > upgrade/pacific-x - Neha, Laura, Ilya see
>> https://tracker.ceph.com/issues/58914
>> > upgrade/quincy-p2p - Neha, Laura
>> > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue
>> Adam
>> > Kraitman any updates?)
>> > powercycle - Brad
>> >
>> > Please reply to this email with approval and/or trackers of known
>> > issues/PRs to address them.
>> >
>> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>> > RC release - pending major suites approvals.
>> >
>> > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 
>> wrote:
>> > >
>> > > Details of this release are summarized here:
>> > >
>> > > https://tracker.ceph.com/issues/59070#note-1
>> > > Release Notes - TBD
>> > >
>> > > The reruns were in the queue for 4 days because of some slowness
>> issues.
>> > > The core team (Neha, Radek, Laura, and others) are trying to
>> narrow
>> > > down the root cause.
>> > >
>> > > Seeking approvals/reviews for:
>> > >
>> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
>> test
>> > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
>> for
>> > > the core)
>> > > rgw - Casey
>> > > fs - Venky (the fs suite has an unusually high amount of failed
>> jobs,
>> > > any reason to suspect it in the observed slowness?)
>> > > orch - Adam King
>> > > rbd - Ilya
>> > > krbd - Ilya
>> > > upgrade/octopus-x - Laura is looking into failures
>> > > upgrade/pacific-x - Laura is looking into failures
>> > > upgrade/quincy-p2p - Laura is looking into failures
>> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
>> Kraitman
>> > > is looking into it
>> > > powercycle - Brad
>> > > ceph-volume - needs a rerun on merged
>> > > https://github.com/ceph/ceph-ansible/pull/7409
>> > >
>> > > Please reply to this email with approval and/or trackers of known
>> > > issues/PRs to address them.
>> > >
>> > > Also, share any findings or hypnosis about the slowness in the
>> > > execution of the suite.
>> > >
>> > > Josh, Neha - gibba and LRC upgrades pending major suites
>> approvals.
>> > > RC release - pending major suites approvals.
>> 

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Yuri Weinstein]

Josh, the release is ready for your review and approval.

Adam, can you please update the LRC upgrade to 17.2.6 RC?

Thx


On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein  wrote:

> The release has been approved.
>
> And the gibba cluster upgraded.
>
> We are awaiting the LRC upgrade and then/or in parallel will publish an RC
> for testing.
>
> ETA for the release publishing is 04/07/23
>
> On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:
>
>> upgrades approved!
>>
>> Thanks,
>> Neha
>>
>> On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski 
>> wrote:
>>
>>> rados: approved!
>>>
>>> On Mon, Mar 27, 2023 at 7:02 PM Laura Flores  wrote:
>>>
 Rados review, second round:

 Failures:
 1. https://tracker.ceph.com/issues/58560
 2. https://tracker.ceph.com/issues/58476
 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
 4. https://tracker.ceph.com/issues/49287
 5. https://tracker.ceph.com/issues/58585

 Details:
 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
 Infrastructure
 2. test_non_existent_cluster: cluster does not exist - Ceph -
 Orchestrator
 3. test_dashboard_e2e.sh: Conflicting peer dependency:
 postcss@8.4.21 - Ceph - Mgr - Dashboard
 4. podman: setting cgroup config for procHooks process caused: Unit
 libpod-$hash.scope not found - Ceph - Orchestrator
 5. rook: failed to pull kubelet image - Ceph - Orchestrator

 @Radoslaw Zarzynski  will give final approval for
 rados.

 On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
 wrote:

> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
> wrote:
> >
> > Details of this release are updated here:
> >
> > https://tracker.ceph.com/issues/59070#note-1
> > Release Notes - TBD
> >
> > The slowness we experienced seemed to be self-cured.
> > Neha, Radek, and Laura please provide any findings if you have them.
> >
> > Seeking approvals/reviews for:
> >
> > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2
> with
> > PRs merged on top of quincy-release)
> > rgw - Casey (rerun on Build 2 with PRs merged on top of
> quincy-release)
>
> rgw approved
>
> > fs - Venky
> >
> > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
> updates?)
> > upgrade/pacific-x - Neha, Laura, Ilya see
> https://tracker.ceph.com/issues/58914
> > upgrade/quincy-p2p - Neha, Laura
> > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue
> Adam
> > Kraitman any updates?)
> > powercycle - Brad
> >
> > Please reply to this email with approval and/or trackers of known
> > issues/PRs to address them.
> >
> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > RC release - pending major suites approvals.
> >
> > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 
> wrote:
> > >
> > > Details of this release are summarized here:
> > >
> > > https://tracker.ceph.com/issues/59070#note-1
> > > Release Notes - TBD
> > >
> > > The reruns were in the queue for 4 days because of some slowness
> issues.
> > > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > > down the root cause.
> > >
> > > Seeking approvals/reviews for:
> > >
> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
> test
> > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
> for
> > > the core)
> > > rgw - Casey
> > > fs - Venky (the fs suite has an unusually high amount of failed
> jobs,
> > > any reason to suspect it in the observed slowness?)
> > > orch - Adam King
> > > rbd - Ilya
> > > krbd - Ilya
> > > upgrade/octopus-x - Laura is looking into failures
> > > upgrade/pacific-x - Laura is looking into failures
> > > upgrade/quincy-p2p - Laura is looking into failures
> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
> Kraitman
> > > is looking into it
> > > powercycle - Brad
> > > ceph-volume - needs a rerun on merged
> > > https://github.com/ceph/ceph-ansible/pull/7409
> > >
> > > Please reply to this email with approval and/or trackers of known
> > > issues/PRs to address them.
> > >
> > > Also, share any findings or hypnosis about the slowness in the
> > > execution of the suite.
> > >
> > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > > RC release - pending major suites approvals.
> > >
> > > Thx
> > > YuriW
> > ___
> > Dev mailing list -- d...@ceph.io
> > To unsubscribe send an email to dev-le...@ceph.io
> ___
> Dev mailing list -- d...@ceph.io
> To 

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Radoslaw Zarzynski]

rados: approved!

On Mon, Mar 27, 2023 at 7:02 PM Laura Flores  wrote:

> Rados review, second round:
>
> Failures:
> 1. https://tracker.ceph.com/issues/58560
> 2. https://tracker.ceph.com/issues/58476
> 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
> 4. https://tracker.ceph.com/issues/49287
> 5. https://tracker.ceph.com/issues/58585
>
> Details:
> 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
> Infrastructure
> 2. test_non_existent_cluster: cluster does not exist - Ceph -
> Orchestrator
> 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21
> - Ceph - Mgr - Dashboard
> 4. podman: setting cgroup config for procHooks process caused: Unit
> libpod-$hash.scope not found - Ceph - Orchestrator
> 5. rook: failed to pull kubelet image - Ceph - Orchestrator
>
> @Radoslaw Zarzynski  will give final approval for
> rados.
>
> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley  wrote:
>
>> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
>> wrote:
>> >
>> > Details of this release are updated here:
>> >
>> > https://tracker.ceph.com/issues/59070#note-1
>> > Release Notes - TBD
>> >
>> > The slowness we experienced seemed to be self-cured.
>> > Neha, Radek, and Laura please provide any findings if you have them.
>> >
>> > Seeking approvals/reviews for:
>> >
>> > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
>> > PRs merged on top of quincy-release)
>> > rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
>>
>> rgw approved
>>
>> > fs - Venky
>> >
>> > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
>> updates?)
>> > upgrade/pacific-x - Neha, Laura, Ilya see
>> https://tracker.ceph.com/issues/58914
>> > upgrade/quincy-p2p - Neha, Laura
>> > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
>> > Kraitman any updates?)
>> > powercycle - Brad
>> >
>> > Please reply to this email with approval and/or trackers of known
>> > issues/PRs to address them.
>> >
>> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>> > RC release - pending major suites approvals.
>> >
>> > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 
>> wrote:
>> > >
>> > > Details of this release are summarized here:
>> > >
>> > > https://tracker.ceph.com/issues/59070#note-1
>> > > Release Notes - TBD
>> > >
>> > > The reruns were in the queue for 4 days because of some slowness
>> issues.
>> > > The core team (Neha, Radek, Laura, and others) are trying to narrow
>> > > down the root cause.
>> > >
>> > > Seeking approvals/reviews for:
>> > >
>> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
>> > > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
>> > > the core)
>> > > rgw - Casey
>> > > fs - Venky (the fs suite has an unusually high amount of failed jobs,
>> > > any reason to suspect it in the observed slowness?)
>> > > orch - Adam King
>> > > rbd - Ilya
>> > > krbd - Ilya
>> > > upgrade/octopus-x - Laura is looking into failures
>> > > upgrade/pacific-x - Laura is looking into failures
>> > > upgrade/quincy-p2p - Laura is looking into failures
>> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
>> > > is looking into it
>> > > powercycle - Brad
>> > > ceph-volume - needs a rerun on merged
>> > > https://github.com/ceph/ceph-ansible/pull/7409
>> > >
>> > > Please reply to this email with approval and/or trackers of known
>> > > issues/PRs to address them.
>> > >
>> > > Also, share any findings or hypnosis about the slowness in the
>> > > execution of the suite.
>> > >
>> > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>> > > RC release - pending major suites approvals.
>> > >
>> > > Thx
>> > > YuriW
>> > ___
>> > Dev mailing list -- d...@ceph.io
>> > To unsubscribe send an email to dev-le...@ceph.io
>> ___
>> Dev mailing list -- d...@ceph.io
>> To unsubscribe send an email to dev-le...@ceph.io
>>
>
>
> --
>
> Laura Flores
>
> She/Her/Hers
>
> Software Engineer, Ceph Storage 
>
> Chicago, IL
>
> lflo...@ibm.com | lflo...@redhat.com 
> M: +17087388804
>
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Yuri Weinstein]

The release has been approved.

And the gibba cluster upgraded.

We are awaiting the LRC upgrade and then/or in parallel will publish an RC
for testing.

ETA for the release publishing is 04/07/23

On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha  wrote:

> upgrades approved!
>
> Thanks,
> Neha
>
> On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski 
> wrote:
>
>> rados: approved!
>>
>> On Mon, Mar 27, 2023 at 7:02 PM Laura Flores  wrote:
>>
>>> Rados review, second round:
>>>
>>> Failures:
>>> 1. https://tracker.ceph.com/issues/58560
>>> 2. https://tracker.ceph.com/issues/58476
>>> 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
>>> 4. https://tracker.ceph.com/issues/49287
>>> 5. https://tracker.ceph.com/issues/58585
>>>
>>> Details:
>>> 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
>>> Infrastructure
>>> 2. test_non_existent_cluster: cluster does not exist - Ceph -
>>> Orchestrator
>>> 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21
>>> - Ceph - Mgr - Dashboard
>>> 4. podman: setting cgroup config for procHooks process caused: Unit
>>> libpod-$hash.scope not found - Ceph - Orchestrator
>>> 5. rook: failed to pull kubelet image - Ceph - Orchestrator
>>>
>>> @Radoslaw Zarzynski  will give final approval for
>>> rados.
>>>
>>> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley 
>>> wrote:
>>>
 On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
 wrote:
 >
 > Details of this release are updated here:
 >
 > https://tracker.ceph.com/issues/59070#note-1
 > Release Notes - TBD
 >
 > The slowness we experienced seemed to be self-cured.
 > Neha, Radek, and Laura please provide any findings if you have them.
 >
 > Seeking approvals/reviews for:
 >
 > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
 > PRs merged on top of quincy-release)
 > rgw - Casey (rerun on Build 2 with PRs merged on top of
 quincy-release)

 rgw approved

 > fs - Venky
 >
 > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
 updates?)
 > upgrade/pacific-x - Neha, Laura, Ilya see
 https://tracker.ceph.com/issues/58914
 > upgrade/quincy-p2p - Neha, Laura
 > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
 > Kraitman any updates?)
 > powercycle - Brad
 >
 > Please reply to this email with approval and/or trackers of known
 > issues/PRs to address them.
 >
 > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
 > RC release - pending major suites approvals.
 >
 > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 
 wrote:
 > >
 > > Details of this release are summarized here:
 > >
 > > https://tracker.ceph.com/issues/59070#note-1
 > > Release Notes - TBD
 > >
 > > The reruns were in the queue for 4 days because of some slowness
 issues.
 > > The core team (Neha, Radek, Laura, and others) are trying to narrow
 > > down the root cause.
 > >
 > > Seeking approvals/reviews for:
 > >
 > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
 test
 > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
 for
 > > the core)
 > > rgw - Casey
 > > fs - Venky (the fs suite has an unusually high amount of failed
 jobs,
 > > any reason to suspect it in the observed slowness?)
 > > orch - Adam King
 > > rbd - Ilya
 > > krbd - Ilya
 > > upgrade/octopus-x - Laura is looking into failures
 > > upgrade/pacific-x - Laura is looking into failures
 > > upgrade/quincy-p2p - Laura is looking into failures
 > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
 Kraitman
 > > is looking into it
 > > powercycle - Brad
 > > ceph-volume - needs a rerun on merged
 > > https://github.com/ceph/ceph-ansible/pull/7409
 > >
 > > Please reply to this email with approval and/or trackers of known
 > > issues/PRs to address them.
 > >
 > > Also, share any findings or hypnosis about the slowness in the
 > > execution of the suite.
 > >
 > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
 > > RC release - pending major suites approvals.
 > >
 > > Thx
 > > YuriW
 > ___
 > Dev mailing list -- d...@ceph.io
 > To unsubscribe send an email to dev-le...@ceph.io
 ___
 Dev mailing list -- d...@ceph.io
 To unsubscribe send an email to dev-le...@ceph.io

>>>
>>>
>>> --
>>>
>>> Laura Flores
>>>
>>> She/Her/Hers
>>>
>>> Software Engineer, Ceph Storage 
>>>
>>> Chicago, IL
>>>
>>> lflo...@ibm.com | lflo...@redhat.com 
>>> M: +17087388804
>>>
>>>
>>> ___
>> Dev mailing list -- 

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Neha Ojha]

upgrades approved!

Thanks,
Neha

On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski 
wrote:

> rados: approved!
>
> On Mon, Mar 27, 2023 at 7:02 PM Laura Flores  wrote:
>
>> Rados review, second round:
>>
>> Failures:
>> 1. https://tracker.ceph.com/issues/58560
>> 2. https://tracker.ceph.com/issues/58476
>> 3. https://tracker.ceph.com/issues/58475 -- pending Q backport
>> 4. https://tracker.ceph.com/issues/49287
>> 5. https://tracker.ceph.com/issues/58585
>>
>> Details:
>> 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
>> Infrastructure
>> 2. test_non_existent_cluster: cluster does not exist - Ceph -
>> Orchestrator
>> 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21
>> - Ceph - Mgr - Dashboard
>> 4. podman: setting cgroup config for procHooks process caused: Unit
>> libpod-$hash.scope not found - Ceph - Orchestrator
>> 5. rook: failed to pull kubelet image - Ceph - Orchestrator
>>
>> @Radoslaw Zarzynski  will give final approval for
>> rados.
>>
>> On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley  wrote:
>>
>>> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
>>> wrote:
>>> >
>>> > Details of this release are updated here:
>>> >
>>> > https://tracker.ceph.com/issues/59070#note-1
>>> > Release Notes - TBD
>>> >
>>> > The slowness we experienced seemed to be self-cured.
>>> > Neha, Radek, and Laura please provide any findings if you have them.
>>> >
>>> > Seeking approvals/reviews for:
>>> >
>>> > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
>>> > PRs merged on top of quincy-release)
>>> > rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
>>>
>>> rgw approved
>>>
>>> > fs - Venky
>>> >
>>> > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
>>> updates?)
>>> > upgrade/pacific-x - Neha, Laura, Ilya see
>>> https://tracker.ceph.com/issues/58914
>>> > upgrade/quincy-p2p - Neha, Laura
>>> > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
>>> > Kraitman any updates?)
>>> > powercycle - Brad
>>> >
>>> > Please reply to this email with approval and/or trackers of known
>>> > issues/PRs to address them.
>>> >
>>> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>>> > RC release - pending major suites approvals.
>>> >
>>> > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 
>>> wrote:
>>> > >
>>> > > Details of this release are summarized here:
>>> > >
>>> > > https://tracker.ceph.com/issues/59070#note-1
>>> > > Release Notes - TBD
>>> > >
>>> > > The reruns were in the queue for 4 days because of some slowness
>>> issues.
>>> > > The core team (Neha, Radek, Laura, and others) are trying to narrow
>>> > > down the root cause.
>>> > >
>>> > > Seeking approvals/reviews for:
>>> > >
>>> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
>>> test
>>> > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
>>> for
>>> > > the core)
>>> > > rgw - Casey
>>> > > fs - Venky (the fs suite has an unusually high amount of failed jobs,
>>> > > any reason to suspect it in the observed slowness?)
>>> > > orch - Adam King
>>> > > rbd - Ilya
>>> > > krbd - Ilya
>>> > > upgrade/octopus-x - Laura is looking into failures
>>> > > upgrade/pacific-x - Laura is looking into failures
>>> > > upgrade/quincy-p2p - Laura is looking into failures
>>> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
>>> Kraitman
>>> > > is looking into it
>>> > > powercycle - Brad
>>> > > ceph-volume - needs a rerun on merged
>>> > > https://github.com/ceph/ceph-ansible/pull/7409
>>> > >
>>> > > Please reply to this email with approval and/or trackers of known
>>> > > issues/PRs to address them.
>>> > >
>>> > > Also, share any findings or hypnosis about the slowness in the
>>> > > execution of the suite.
>>> > >
>>> > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>>> > > RC release - pending major suites approvals.
>>> > >
>>> > > Thx
>>> > > YuriW
>>> > ___
>>> > Dev mailing list -- d...@ceph.io
>>> > To unsubscribe send an email to dev-le...@ceph.io
>>> ___
>>> Dev mailing list -- d...@ceph.io
>>> To unsubscribe send an email to dev-le...@ceph.io
>>>
>>
>>
>> --
>>
>> Laura Flores
>>
>> She/Her/Hers
>>
>> Software Engineer, Ceph Storage 
>>
>> Chicago, IL
>>
>> lflo...@ibm.com | lflo...@redhat.com 
>> M: +17087388804
>>
>>
>> ___
> Dev mailing list -- d...@ceph.io
> To unsubscribe send an email to dev-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Laura Flores]

Rados review, second round:

Failures:
1. https://tracker.ceph.com/issues/58560
2. https://tracker.ceph.com/issues/58476
3. https://tracker.ceph.com/issues/58475 -- pending Q backport
4. https://tracker.ceph.com/issues/49287
5. https://tracker.ceph.com/issues/58585

Details:
1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo -
Infrastructure
2. test_non_existent_cluster: cluster does not exist - Ceph -
Orchestrator
3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 -
Ceph - Mgr - Dashboard
4. podman: setting cgroup config for procHooks process caused: Unit
libpod-$hash.scope not found - Ceph - Orchestrator
5. rook: failed to pull kubelet image - Ceph - Orchestrator

@Radoslaw Zarzynski  will give final approval for
rados.

On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley  wrote:

> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein 
> wrote:
> >
> > Details of this release are updated here:
> >
> > https://tracker.ceph.com/issues/59070#note-1
> > Release Notes - TBD
> >
> > The slowness we experienced seemed to be self-cured.
> > Neha, Radek, and Laura please provide any findings if you have them.
> >
> > Seeking approvals/reviews for:
> >
> > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
> > PRs merged on top of quincy-release)
> > rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
>
> rgw approved
>
> > fs - Venky
> >
> > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any
> updates?)
> > upgrade/pacific-x - Neha, Laura, Ilya see
> https://tracker.ceph.com/issues/58914
> > upgrade/quincy-p2p - Neha, Laura
> > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
> > Kraitman any updates?)
> > powercycle - Brad
> >
> > Please reply to this email with approval and/or trackers of known
> > issues/PRs to address them.
> >
> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > RC release - pending major suites approvals.
> >
> > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 
> wrote:
> > >
> > > Details of this release are summarized here:
> > >
> > > https://tracker.ceph.com/issues/59070#note-1
> > > Release Notes - TBD
> > >
> > > The reruns were in the queue for 4 days because of some slowness
> issues.
> > > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > > down the root cause.
> > >
> > > Seeking approvals/reviews for:
> > >
> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> > > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> > > the core)
> > > rgw - Casey
> > > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > > any reason to suspect it in the observed slowness?)
> > > orch - Adam King
> > > rbd - Ilya
> > > krbd - Ilya
> > > upgrade/octopus-x - Laura is looking into failures
> > > upgrade/pacific-x - Laura is looking into failures
> > > upgrade/quincy-p2p - Laura is looking into failures
> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> > > is looking into it
> > > powercycle - Brad
> > > ceph-volume - needs a rerun on merged
> > > https://github.com/ceph/ceph-ansible/pull/7409
> > >
> > > Please reply to this email with approval and/or trackers of known
> > > issues/PRs to address them.
> > >
> > > Also, share any findings or hypnosis about the slowness in the
> > > execution of the suite.
> > >
> > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > > RC release - pending major suites approvals.
> > >
> > > Thx
> > > YuriW
> > ___
> > Dev mailing list -- d...@ceph.io
> > To unsubscribe send an email to dev-le...@ceph.io
> ___
> Dev mailing list -- d...@ceph.io
> To unsubscribe send an email to dev-le...@ceph.io
>


-- 

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage 

Chicago, IL

lflo...@ibm.com | lflo...@redhat.com 
M: +17087388804
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Casey Bodley]

On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein  wrote:
>
> Details of this release are updated here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The slowness we experienced seemed to be self-cured.
> Neha, Radek, and Laura please provide any findings if you have them.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
> PRs merged on top of quincy-release)
> rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)

rgw approved

> fs - Venky
>
> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?)
> upgrade/pacific-x - Neha, Laura, Ilya see 
> https://tracker.ceph.com/issues/58914
> upgrade/quincy-p2p - Neha, Laura
> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
> Kraitman any updates?)
> powercycle - Brad
>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein  wrote:
> >
> > Details of this release are summarized here:
> >
> > https://tracker.ceph.com/issues/59070#note-1
> > Release Notes - TBD
> >
> > The reruns were in the queue for 4 days because of some slowness issues.
> > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > down the root cause.
> >
> > Seeking approvals/reviews for:
> >
> > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> > the core)
> > rgw - Casey
> > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > any reason to suspect it in the observed slowness?)
> > orch - Adam King
> > rbd - Ilya
> > krbd - Ilya
> > upgrade/octopus-x - Laura is looking into failures
> > upgrade/pacific-x - Laura is looking into failures
> > upgrade/quincy-p2p - Laura is looking into failures
> > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> > is looking into it
> > powercycle - Brad
> > ceph-volume - needs a rerun on merged
> > https://github.com/ceph/ceph-ansible/pull/7409
> >
> > Please reply to this email with approval and/or trackers of known
> > issues/PRs to address them.
> >
> > Also, share any findings or hypnosis about the slowness in the
> > execution of the suite.
> >
> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > RC release - pending major suites approvals.
> >
> > Thx
> > YuriW
> ___
> Dev mailing list -- d...@ceph.io
> To unsubscribe send an email to dev-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Venky Shankar]

On Sat, Mar 25, 2023 at 1:17 AM Yuri Weinstein  wrote:
>
> Details of this release are updated here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The slowness we experienced seemed to be self-cured.
> Neha, Radek, and Laura please provide any findings if you have them.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
> PRs merged on top of quincy-release)
> rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
> fs - Venky

fs approved.

>
> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?)
> upgrade/pacific-x - Neha, Laura, Ilya see 
> https://tracker.ceph.com/issues/58914
> upgrade/quincy-p2p - Neha, Laura
> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
> Kraitman any updates?)
> powercycle - Brad
>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein  wrote:
> >
> > Details of this release are summarized here:
> >
> > https://tracker.ceph.com/issues/59070#note-1
> > Release Notes - TBD
> >
> > The reruns were in the queue for 4 days because of some slowness issues.
> > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > down the root cause.
> >
> > Seeking approvals/reviews for:
> >
> > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> > the core)
> > rgw - Casey
> > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > any reason to suspect it in the observed slowness?)
> > orch - Adam King
> > rbd - Ilya
> > krbd - Ilya
> > upgrade/octopus-x - Laura is looking into failures
> > upgrade/pacific-x - Laura is looking into failures
> > upgrade/quincy-p2p - Laura is looking into failures
> > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> > is looking into it
> > powercycle - Brad
> > ceph-volume - needs a rerun on merged
> > https://github.com/ceph/ceph-ansible/pull/7409
> >
> > Please reply to this email with approval and/or trackers of known
> > issues/PRs to address them.
> >
> > Also, share any findings or hypnosis about the slowness in the
> > execution of the suite.
> >
> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > RC release - pending major suites approvals.
> >
> > Thx
> > YuriW
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io



-- 
Cheers,
Venky
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Nizamudeen A]

Dashboard LGTM!

On Sat, Mar 25, 2023 at 1:16 AM Yuri Weinstein  wrote:

> Details of this release are updated here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The slowness we experienced seemed to be self-cured.
> Neha, Radek, and Laura please provide any findings if you have them.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
> PRs merged on top of quincy-release)
> rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
> fs - Venky
>
> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?)
> upgrade/pacific-x - Neha, Laura, Ilya see
> https://tracker.ceph.com/issues/58914
> upgrade/quincy-p2p
>  - Neha, Laura
> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
> Kraitman any updates?)
> powercycle - Brad
>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein 
> wrote:
> >
> > Details of this release are summarized here:
> >
> > https://tracker.ceph.com/issues/59070#note-1
> > Release Notes - TBD
> >
> > The reruns were in the queue for 4 days because of some slowness issues.
> > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > down the root cause.
> >
> > Seeking approvals/reviews for:
> >
> > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> > the core)
> > rgw - Casey
> > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > any reason to suspect it in the observed slowness?)
> > orch - Adam King
> > rbd - Ilya
> > krbd - Ilya
> > upgrade/octopus-x - Laura is looking into failures
> > upgrade/pacific-x - Laura is looking into failures
> > upgrade/quincy-p2p - Laura is looking into failures
> > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> > is looking into it
> > powercycle - Brad
> > ceph-volume - needs a rerun on merged
> > https://github.com/ceph/ceph-ansible/pull/7409
> >
> > Please reply to this email with approval and/or trackers of known
> > issues/PRs to address them.
> >
> > Also, share any findings or hypnosis about the slowness in the
> > execution of the suite.
> >
> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > RC release - pending major suites approvals.
> >
> > Thx
> > YuriW
> ___
> Dev mailing list -- d...@ceph.io
> To unsubscribe send an email to dev-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Brad Hubbard]

On Sat, Mar 25, 2023 at 5:46 AM Yuri Weinstein  wrote:
>
> Details of this release are updated here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The slowness we experienced seemed to be self-cured.
> Neha, Radek, and Laura please provide any findings if you have them.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
> PRs merged on top of quincy-release)
> rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
> fs - Venky
>
> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?)
> upgrade/pacific-x - Neha, Laura, Ilya see 
> https://tracker.ceph.com/issues/58914
> upgrade/quincy-p2p - Neha, Laura
> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
> Kraitman any updates?)
> powercycle - Brad

Still the selinux ping issues I've been trying to pin down. Approved.

>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein  wrote:
> >
> > Details of this release are summarized here:
> >
> > https://tracker.ceph.com/issues/59070#note-1
> > Release Notes - TBD
> >
> > The reruns were in the queue for 4 days because of some slowness issues.
> > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > down the root cause.
> >
> > Seeking approvals/reviews for:
> >
> > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> > the core)
> > rgw - Casey
> > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > any reason to suspect it in the observed slowness?)
> > orch - Adam King
> > rbd - Ilya
> > krbd - Ilya
> > upgrade/octopus-x - Laura is looking into failures
> > upgrade/pacific-x - Laura is looking into failures
> > upgrade/quincy-p2p - Laura is looking into failures
> > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> > is looking into it
> > powercycle - Brad
> > ceph-volume - needs a rerun on merged
> > https://github.com/ceph/ceph-ansible/pull/7409
> >
> > Please reply to this email with approval and/or trackers of known
> > issues/PRs to address them.
> >
> > Also, share any findings or hypnosis about the slowness in the
> > execution of the suite.
> >
> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > RC release - pending major suites approvals.
> >
> > Thx
> > YuriW
> ___
> Dev mailing list -- d...@ceph.io
> To unsubscribe send an email to dev-le...@ceph.io



-- 
Cheers,
Brad
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Yuri Weinstein]

Details of this release are updated here:

https://tracker.ceph.com/issues/59070#note-1
Release Notes - TBD

The slowness we experienced seemed to be self-cured.
Neha, Radek, and Laura please provide any findings if you have them.

Seeking approvals/reviews for:

rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with
PRs merged on top of quincy-release)
rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
fs - Venky

upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?)
upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914
upgrade/quincy-p2p - Neha, Laura
client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam
Kraitman any updates?)
powercycle - Brad

Please reply to this email with approval and/or trackers of known
issues/PRs to address them.

Josh, Neha - gibba and LRC upgrades pending major suites approvals.
RC release - pending major suites approvals.

On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein  wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The reruns were in the queue for 4 days because of some slowness issues.
> The core team (Neha, Radek, Laura, and others) are trying to narrow
> down the root cause.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> the core)
> rgw - Casey
> fs - Venky (the fs suite has an unusually high amount of failed jobs,
> any reason to suspect it in the observed slowness?)
> orch - Adam King
> rbd - Ilya
> krbd - Ilya
> upgrade/octopus-x - Laura is looking into failures
> upgrade/pacific-x - Laura is looking into failures
> upgrade/quincy-p2p - Laura is looking into failures
> client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> is looking into it
> powercycle - Brad
> ceph-volume - needs a rerun on merged
> https://github.com/ceph/ceph-ansible/pull/7409
>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Also, share any findings or hypnosis about the slowness in the
> execution of the suite.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> Thx
> YuriW
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Ken Dreyer]

How much more time do we need to get PR 50549 in if we delayed v17.2.6?

- Ken

On Thu, Mar 23, 2023 at 2:44 PM Laura Flores  wrote:

> We are all good on the Core end of things.
> https://github.com/ceph/ceph/pull/50549 is needed for downstream, but it
> should not block upstream.
>
> On Thu, Mar 23, 2023 at 12:59 PM Laura Flores  wrote:
>
>> https://github.com/ceph/ceph/pull/50575 was also merged.
>>
>> On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein 
>> wrote:
>>
>>> We are still working on core PRs:
>>>
>>> https://github.com/ceph/ceph/pull/50549
>>> https://github.com/ceph/ceph/pull/50625 - merged
>>> https://github.com/ceph/ceph/pull/50575
>>>
>>> Will update as soon as we are ready for the next steps.
>>>
>>> On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley 
>>> wrote:
>>> >
>>> > On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley 
>>> wrote:
>>> > >
>>> > > On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein 
>>> wrote:
>>> > > >
>>> > > > Details of this release are summarized here:
>>> > > >
>>> > > > https://tracker.ceph.com/issues/59070#note-1
>>> > > > Release Notes - TBD
>>> > > >
>>> > > > The reruns were in the queue for 4 days because of some slowness
>>> issues.
>>> > > > The core team (Neha, Radek, Laura, and others) are trying to narrow
>>> > > > down the root cause.
>>> > > >
>>> > > > Seeking approvals/reviews for:
>>> > > >
>>> > > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
>>> test
>>> > > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
>>> for
>>> > > > the core)
>>> > > > rgw - Casey
>>> > >
>>> > > there were some java_s3test failures related to
>>> > > https://tracker.ceph.com/issues/58554. i've added the fix to
>>> > > https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun
>>> > > should resolve those failures
>>> > > there were also some 'Failed to fetch package version' failures in
>>> the
>>> > > rerun that warranted another rerun anyway
>>> > >
>>> > > there's also an urgent priority bug fix in
>>> > > https://github.com/ceph/ceph/pull/50625 that i'd really like to add
>>> to
>>> > > this release; sorry for the late notice
>>> >
>>> > this fix merged, so rgw is now approved. thanks Yuri
>>> >
>>> > >
>>> > > > fs - Venky (the fs suite has an unusually high amount of failed
>>> jobs,
>>> > > > any reason to suspect it in the observed slowness?)
>>> > > > orch - Adam King
>>> > > > rbd - Ilya
>>> > > > krbd - Ilya
>>> > > > upgrade/octopus-x - Laura is looking into failures
>>> > > > upgrade/pacific-x - Laura is looking into failures
>>> > > > upgrade/quincy-p2p - Laura is looking into failures
>>> > > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
>>> Kraitman
>>> > > > is looking into it
>>> > > > powercycle - Brad
>>> > > > ceph-volume - needs a rerun on merged
>>> > > > https://github.com/ceph/ceph-ansible/pull/7409
>>> > > >
>>> > > > Please reply to this email with approval and/or trackers of known
>>> > > > issues/PRs to address them.
>>> > > >
>>> > > > Also, share any findings or hypnosis about the slowness in the
>>> > > > execution of the suite.
>>> > > >
>>> > > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>>> > > > RC release - pending major suites approvals.
>>> > > >
>>> > > > Thx
>>> > > > YuriW
>>> > > > ___
>>> > > > ceph-users mailing list -- ceph-users@ceph.io
>>> > > > To unsubscribe send an email to ceph-users-le...@ceph.io
>>> > > >
>>> >
>>> ___
>>> ceph-users mailing list -- ceph-users@ceph.io
>>> To unsubscribe send an email to ceph-users-le...@ceph.io
>>>
>>
>>
>> --
>>
>> Laura Flores
>>
>> She/Her/Hers
>>
>> Software Engineer, Ceph Storage 
>>
>> Chicago, IL
>>
>> lflo...@ibm.com | lflo...@redhat.com 
>> M: +17087388804
>>
>>
>>
>
> --
>
> Laura Flores
>
> She/Her/Hers
>
> Software Engineer, Ceph Storage 
>
> Chicago, IL
>
> lflo...@ibm.com | lflo...@redhat.com 
> M: +17087388804
>
>
> ___
> Dev mailing list -- d...@ceph.io
> To unsubscribe send an email to dev-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Laura Flores]

I will have a review of the rados suite ready soon.

On Thu, Mar 23, 2023 at 1:44 PM Laura Flores  wrote:

> We are all good on the Core end of things.
> https://github.com/ceph/ceph/pull/50549 is needed for downstream, but it
> should not block upstream.
>
> On Thu, Mar 23, 2023 at 12:59 PM Laura Flores  wrote:
>
>> https://github.com/ceph/ceph/pull/50575 was also merged.
>>
>> On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein 
>> wrote:
>>
>>> We are still working on core PRs:
>>>
>>> https://github.com/ceph/ceph/pull/50549
>>> https://github.com/ceph/ceph/pull/50625 - merged
>>> https://github.com/ceph/ceph/pull/50575
>>>
>>> Will update as soon as we are ready for the next steps.
>>>
>>> On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley 
>>> wrote:
>>> >
>>> > On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley 
>>> wrote:
>>> > >
>>> > > On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein 
>>> wrote:
>>> > > >
>>> > > > Details of this release are summarized here:
>>> > > >
>>> > > > https://tracker.ceph.com/issues/59070#note-1
>>> > > > Release Notes - TBD
>>> > > >
>>> > > > The reruns were in the queue for 4 days because of some slowness
>>> issues.
>>> > > > The core team (Neha, Radek, Laura, and others) are trying to narrow
>>> > > > down the root cause.
>>> > > >
>>> > > > Seeking approvals/reviews for:
>>> > > >
>>> > > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
>>> test
>>> > > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
>>> for
>>> > > > the core)
>>> > > > rgw - Casey
>>> > >
>>> > > there were some java_s3test failures related to
>>> > > https://tracker.ceph.com/issues/58554. i've added the fix to
>>> > > https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun
>>> > > should resolve those failures
>>> > > there were also some 'Failed to fetch package version' failures in
>>> the
>>> > > rerun that warranted another rerun anyway
>>> > >
>>> > > there's also an urgent priority bug fix in
>>> > > https://github.com/ceph/ceph/pull/50625 that i'd really like to add
>>> to
>>> > > this release; sorry for the late notice
>>> >
>>> > this fix merged, so rgw is now approved. thanks Yuri
>>> >
>>> > >
>>> > > > fs - Venky (the fs suite has an unusually high amount of failed
>>> jobs,
>>> > > > any reason to suspect it in the observed slowness?)
>>> > > > orch - Adam King
>>> > > > rbd - Ilya
>>> > > > krbd - Ilya
>>> > > > upgrade/octopus-x - Laura is looking into failures
>>> > > > upgrade/pacific-x - Laura is looking into failures
>>> > > > upgrade/quincy-p2p - Laura is looking into failures
>>> > > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
>>> Kraitman
>>> > > > is looking into it
>>> > > > powercycle - Brad
>>> > > > ceph-volume - needs a rerun on merged
>>> > > > https://github.com/ceph/ceph-ansible/pull/7409
>>> > > >
>>> > > > Please reply to this email with approval and/or trackers of known
>>> > > > issues/PRs to address them.
>>> > > >
>>> > > > Also, share any findings or hypnosis about the slowness in the
>>> > > > execution of the suite.
>>> > > >
>>> > > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>>> > > > RC release - pending major suites approvals.
>>> > > >
>>> > > > Thx
>>> > > > YuriW
>>> > > > ___
>>> > > > ceph-users mailing list -- ceph-users@ceph.io
>>> > > > To unsubscribe send an email to ceph-users-le...@ceph.io
>>> > > >
>>> >
>>> ___
>>> ceph-users mailing list -- ceph-users@ceph.io
>>> To unsubscribe send an email to ceph-users-le...@ceph.io
>>>
>>
>>
>> --
>>
>> Laura Flores
>>
>> She/Her/Hers
>>
>> Software Engineer, Ceph Storage 
>>
>> Chicago, IL
>>
>> lflo...@ibm.com | lflo...@redhat.com 
>> M: +17087388804
>>
>>
>>
>
> --
>
> Laura Flores
>
> She/Her/Hers
>
> Software Engineer, Ceph Storage 
>
> Chicago, IL
>
> lflo...@ibm.com | lflo...@redhat.com 
> M: +17087388804
>
>
>

-- 

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage 

Chicago, IL

lflo...@ibm.com | lflo...@redhat.com 
M: +17087388804
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Laura Flores]

We are all good on the Core end of things.
https://github.com/ceph/ceph/pull/50549 is needed for downstream, but it
should not block upstream.

On Thu, Mar 23, 2023 at 12:59 PM Laura Flores  wrote:

> https://github.com/ceph/ceph/pull/50575 was also merged.
>
> On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein 
> wrote:
>
>> We are still working on core PRs:
>>
>> https://github.com/ceph/ceph/pull/50549
>> https://github.com/ceph/ceph/pull/50625 - merged
>> https://github.com/ceph/ceph/pull/50575
>>
>> Will update as soon as we are ready for the next steps.
>>
>> On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley  wrote:
>> >
>> > On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley 
>> wrote:
>> > >
>> > > On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein 
>> wrote:
>> > > >
>> > > > Details of this release are summarized here:
>> > > >
>> > > > https://tracker.ceph.com/issues/59070#note-1
>> > > > Release Notes - TBD
>> > > >
>> > > > The reruns were in the queue for 4 days because of some slowness
>> issues.
>> > > > The core team (Neha, Radek, Laura, and others) are trying to narrow
>> > > > down the root cause.
>> > > >
>> > > > Seeking approvals/reviews for:
>> > > >
>> > > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
>> test
>> > > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
>> for
>> > > > the core)
>> > > > rgw - Casey
>> > >
>> > > there were some java_s3test failures related to
>> > > https://tracker.ceph.com/issues/58554. i've added the fix to
>> > > https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun
>> > > should resolve those failures
>> > > there were also some 'Failed to fetch package version' failures in the
>> > > rerun that warranted another rerun anyway
>> > >
>> > > there's also an urgent priority bug fix in
>> > > https://github.com/ceph/ceph/pull/50625 that i'd really like to add
>> to
>> > > this release; sorry for the late notice
>> >
>> > this fix merged, so rgw is now approved. thanks Yuri
>> >
>> > >
>> > > > fs - Venky (the fs suite has an unusually high amount of failed
>> jobs,
>> > > > any reason to suspect it in the observed slowness?)
>> > > > orch - Adam King
>> > > > rbd - Ilya
>> > > > krbd - Ilya
>> > > > upgrade/octopus-x - Laura is looking into failures
>> > > > upgrade/pacific-x - Laura is looking into failures
>> > > > upgrade/quincy-p2p - Laura is looking into failures
>> > > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
>> Kraitman
>> > > > is looking into it
>> > > > powercycle - Brad
>> > > > ceph-volume - needs a rerun on merged
>> > > > https://github.com/ceph/ceph-ansible/pull/7409
>> > > >
>> > > > Please reply to this email with approval and/or trackers of known
>> > > > issues/PRs to address them.
>> > > >
>> > > > Also, share any findings or hypnosis about the slowness in the
>> > > > execution of the suite.
>> > > >
>> > > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
>> > > > RC release - pending major suites approvals.
>> > > >
>> > > > Thx
>> > > > YuriW
>> > > > ___
>> > > > ceph-users mailing list -- ceph-users@ceph.io
>> > > > To unsubscribe send an email to ceph-users-le...@ceph.io
>> > > >
>> >
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-le...@ceph.io
>>
>
>
> --
>
> Laura Flores
>
> She/Her/Hers
>
> Software Engineer, Ceph Storage 
>
> Chicago, IL
>
> lflo...@ibm.com | lflo...@redhat.com 
> M: +17087388804
>
>
>

-- 

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage 

Chicago, IL

lflo...@ibm.com | lflo...@redhat.com 
M: +17087388804
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Laura Flores]

https://github.com/ceph/ceph/pull/50575 was also merged.

On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein  wrote:

> We are still working on core PRs:
>
> https://github.com/ceph/ceph/pull/50549
> https://github.com/ceph/ceph/pull/50625 - merged
> https://github.com/ceph/ceph/pull/50575
>
> Will update as soon as we are ready for the next steps.
>
> On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley  wrote:
> >
> > On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley  wrote:
> > >
> > > On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein 
> wrote:
> > > >
> > > > Details of this release are summarized here:
> > > >
> > > > https://tracker.ceph.com/issues/59070#note-1
> > > > Release Notes - TBD
> > > >
> > > > The reruns were in the queue for 4 days because of some slowness
> issues.
> > > > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > > > down the root cause.
> > > >
> > > > Seeking approvals/reviews for:
> > > >
> > > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to
> test
> > > > and merge at least one PR https://github.com/ceph/ceph/pull/50575
> for
> > > > the core)
> > > > rgw - Casey
> > >
> > > there were some java_s3test failures related to
> > > https://tracker.ceph.com/issues/58554. i've added the fix to
> > > https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun
> > > should resolve those failures
> > > there were also some 'Failed to fetch package version' failures in the
> > > rerun that warranted another rerun anyway
> > >
> > > there's also an urgent priority bug fix in
> > > https://github.com/ceph/ceph/pull/50625 that i'd really like to add to
> > > this release; sorry for the late notice
> >
> > this fix merged, so rgw is now approved. thanks Yuri
> >
> > >
> > > > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > > > any reason to suspect it in the observed slowness?)
> > > > orch - Adam King
> > > > rbd - Ilya
> > > > krbd - Ilya
> > > > upgrade/octopus-x - Laura is looking into failures
> > > > upgrade/pacific-x - Laura is looking into failures
> > > > upgrade/quincy-p2p - Laura is looking into failures
> > > > client-upgrade-octopus-quincy-quincy - missing packages, Adam
> Kraitman
> > > > is looking into it
> > > > powercycle - Brad
> > > > ceph-volume - needs a rerun on merged
> > > > https://github.com/ceph/ceph-ansible/pull/7409
> > > >
> > > > Please reply to this email with approval and/or trackers of known
> > > > issues/PRs to address them.
> > > >
> > > > Also, share any findings or hypnosis about the slowness in the
> > > > execution of the suite.
> > > >
> > > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > > > RC release - pending major suites approvals.
> > > >
> > > > Thx
> > > > YuriW
> > > > ___
> > > > ceph-users mailing list -- ceph-users@ceph.io
> > > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > > >
> >
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>


-- 

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage 

Chicago, IL

lflo...@ibm.com | lflo...@redhat.com 
M: +17087388804
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Yuri Weinstein]

We are still working on core PRs:

https://github.com/ceph/ceph/pull/50549
https://github.com/ceph/ceph/pull/50625 - merged
https://github.com/ceph/ceph/pull/50575

Will update as soon as we are ready for the next steps.

On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley  wrote:
>
> On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley  wrote:
> >
> > On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein  wrote:
> > >
> > > Details of this release are summarized here:
> > >
> > > https://tracker.ceph.com/issues/59070#note-1
> > > Release Notes - TBD
> > >
> > > The reruns were in the queue for 4 days because of some slowness issues.
> > > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > > down the root cause.
> > >
> > > Seeking approvals/reviews for:
> > >
> > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> > > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> > > the core)
> > > rgw - Casey
> >
> > there were some java_s3test failures related to
> > https://tracker.ceph.com/issues/58554. i've added the fix to
> > https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun
> > should resolve those failures
> > there were also some 'Failed to fetch package version' failures in the
> > rerun that warranted another rerun anyway
> >
> > there's also an urgent priority bug fix in
> > https://github.com/ceph/ceph/pull/50625 that i'd really like to add to
> > this release; sorry for the late notice
>
> this fix merged, so rgw is now approved. thanks Yuri
>
> >
> > > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > > any reason to suspect it in the observed slowness?)
> > > orch - Adam King
> > > rbd - Ilya
> > > krbd - Ilya
> > > upgrade/octopus-x - Laura is looking into failures
> > > upgrade/pacific-x - Laura is looking into failures
> > > upgrade/quincy-p2p - Laura is looking into failures
> > > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> > > is looking into it
> > > powercycle - Brad
> > > ceph-volume - needs a rerun on merged
> > > https://github.com/ceph/ceph-ansible/pull/7409
> > >
> > > Please reply to this email with approval and/or trackers of known
> > > issues/PRs to address them.
> > >
> > > Also, share any findings or hypnosis about the slowness in the
> > > execution of the suite.
> > >
> > > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > > RC release - pending major suites approvals.
> > >
> > > Thx
> > > YuriW
> > > ___
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > >
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Casey Bodley]

On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley  wrote:
>
> On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein  wrote:
> >
> > Details of this release are summarized here:
> >
> > https://tracker.ceph.com/issues/59070#note-1
> > Release Notes - TBD
> >
> > The reruns were in the queue for 4 days because of some slowness issues.
> > The core team (Neha, Radek, Laura, and others) are trying to narrow
> > down the root cause.
> >
> > Seeking approvals/reviews for:
> >
> > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> > the core)
> > rgw - Casey
>
> there were some java_s3test failures related to
> https://tracker.ceph.com/issues/58554. i've added the fix to
> https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun
> should resolve those failures
> there were also some 'Failed to fetch package version' failures in the
> rerun that warranted another rerun anyway
>
> there's also an urgent priority bug fix in
> https://github.com/ceph/ceph/pull/50625 that i'd really like to add to
> this release; sorry for the late notice

this fix merged, so rgw is now approved. thanks Yuri

>
> > fs - Venky (the fs suite has an unusually high amount of failed jobs,
> > any reason to suspect it in the observed slowness?)
> > orch - Adam King
> > rbd - Ilya
> > krbd - Ilya
> > upgrade/octopus-x - Laura is looking into failures
> > upgrade/pacific-x - Laura is looking into failures
> > upgrade/quincy-p2p - Laura is looking into failures
> > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> > is looking into it
> > powercycle - Brad
> > ceph-volume - needs a rerun on merged
> > https://github.com/ceph/ceph-ansible/pull/7409
> >
> > Please reply to this email with approval and/or trackers of known
> > issues/PRs to address them.
> >
> > Also, share any findings or hypnosis about the slowness in the
> > execution of the suite.
> >
> > Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> > RC release - pending major suites approvals.
> >
> > Thx
> > YuriW
> > ___
> > ceph-users mailing list -- ceph-users@ceph.io
> > To unsubscribe send an email to ceph-users-le...@ceph.io
> >
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Venky Shankar]

On Wed, Mar 22, 2023 at 1:36 AM Yuri Weinstein  wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The reruns were in the queue for 4 days because of some slowness issues.
> The core team (Neha, Radek, Laura, and others) are trying to narrow
> down the root cause.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> the core)
> rgw - Casey
> fs - Venky (the fs suite has an unusually high amount of failed jobs,
> any reason to suspect it in the observed slowness?)

Some of those failures can be attributed to slowness, but not all.

Can we have a re-run?

> orch - Adam King
> rbd - Ilya
> krbd - Ilya
> upgrade/octopus-x - Laura is looking into failures
> upgrade/pacific-x - Laura is looking into failures
> upgrade/quincy-p2p - Laura is looking into failures
> client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> is looking into it
> powercycle - Brad
> ceph-volume - needs a rerun on merged
> https://github.com/ceph/ceph-ansible/pull/7409
>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Also, share any findings or hypnosis about the slowness in the
> execution of the suite.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> Thx
> YuriW
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>


-- 
Cheers,
Venky
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Adam King]

orch approved.

After reruns, the only failed jobs in the orch run were orch/rook tests
(which are broken currently) and 2 instances of "Test failure:
test_non_existent_cluster". That failure is just a command expecting a zero
return code and an error message instead of a nonzero return code in a
failure case. I think the test got backported without the change of the
error code, either way it's not a big deal.

I also took a brief look at the orchestrator failure from the upgrade
tests  (https://tracker.ceph.com/issues/59121) that Laura saw. In the
instance of it I looked at, It seems like the test is running "orch upgrade
start" and then not running "orch upgrade pause" until about 20 minutes
later, at which point the upgrade has already completed (and I can see all
the daemons got upgraded to the new image in the logs). It looks like it
was waiting on a loop to see a minority of the mons had been upgraded
before pausing the upgrade, but even starting that loop took over 2
minutes, despite the only actions in between being a "ceph orch ps" call
and echoing out a value. Really not sure why it was so slow in running
those commands or why it happened 3 times in the initial run but never in
the reruns, but the failure came from that, and the upgrade itself seems to
still work fine.

 - Adam King
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Ilya Dryomov]

On Tue, Mar 21, 2023 at 9:06 PM Yuri Weinstein  wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The reruns were in the queue for 4 days because of some slowness issues.
> The core team (Neha, Radek, Laura, and others) are trying to narrow
> down the root cause.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> the core)
> rgw - Casey
> fs - Venky (the fs suite has an unusually high amount of failed jobs,
> any reason to suspect it in the observed slowness?)
> orch - Adam King
> rbd - Ilya
> krbd - Ilya

rbd and krbd approved.

Thanks,

Ilya
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Casey Bodley]

On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein  wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The reruns were in the queue for 4 days because of some slowness issues.
> The core team (Neha, Radek, Laura, and others) are trying to narrow
> down the root cause.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> the core)
> rgw - Casey

there were some java_s3test failures related to
https://tracker.ceph.com/issues/58554. i've added the fix to
https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun
should resolve those failures
there were also some 'Failed to fetch package version' failures in the
rerun that warranted another rerun anyway

there's also an urgent priority bug fix in
https://github.com/ceph/ceph/pull/50625 that i'd really like to add to
this release; sorry for the late notice

> fs - Venky (the fs suite has an unusually high amount of failed jobs,
> any reason to suspect it in the observed slowness?)
> orch - Adam King
> rbd - Ilya
> krbd - Ilya
> upgrade/octopus-x - Laura is looking into failures
> upgrade/pacific-x - Laura is looking into failures
> upgrade/quincy-p2p - Laura is looking into failures
> client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> is looking into it
> powercycle - Brad
> ceph-volume - needs a rerun on merged
> https://github.com/ceph/ceph-ansible/pull/7409
>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Also, share any findings or hypnosis about the slowness in the
> execution of the suite.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> Thx
> YuriW
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: quincy v17.2.6 QE Validation status

[Laura Flores]

I reviewed the upgrade tests.

I opened two new trackers:
1. https://tracker.ceph.com/issues/59121 - "No upgrade in progress" during
upgrade tests - Ceph - Orchestrator
2. https://tracker.ceph.com/issues/59124 - "Health check failed: 1/3 mons
down, quorum b,c (MON_DOWN)" during quincy p2p upgrade test - Ceph - RADOS

Starred (*) trackers occurred frequently throughout the suites.

https://pulpito.ceph.com/yuriw-2023-03-14_21:33:13-upgrade:octopus-x-quincy-release-distro-default-smithi/
Failures:
1. *https://tracker.ceph.com/issues/59121
 ***
2. *https://tracker.ceph.com/issues/56393
 ***
3. https://tracker.ceph.com/issues/53615
Details:
1.* "No upgrade in progress" during upgrade tests - Ceph - Orchestrator*
2. *thrash-erasure-code-big: failed to complete snap trimming before
timeout - Ceph - RADOS*
3. qa: upgrade test fails with "timeout expired in wait_until_healthy"
- Ceph - CephFS

https://pulpito.ceph.com/yuriw-2023-03-15_21:14:59-upgrade:pacific-x-quincy-release-distro-default-smithi/
Failures:
1. *https://tracker.ceph.com/issues/56393
 ***
2. https://tracker.ceph.com/issues/58914
3. *https://tracker.ceph.com/issues/59121
 ***
4. https://tracker.ceph.com/issues/59123
Details:
1. *thrash-erasure-code-big: failed to complete snap trimming before
timeout - Ceph - RADOS*
2. [ FAILED ] TestClsRbd.group_snap_list_max_read in
upgrade:quincy-x-reef - Ceph - RBD
3. *"No upgrade in progress" during upgrade tests*
4. Timeout opening channel - Tools - Teuthology

https://pulpito.ceph.com/yuriw-2023-03-14_21:36:24-upgrade:quincy-p2p-quincy-release-distro-default-smithi/
Failures:
1. https://tracker.ceph.com/issues/59124
Details:
1. "Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" during
quincy p2p upgrade test - Ceph - RADOS

https://pulpito.ceph.com/yuriw-2023-03-15_15:25:50-upgrade-clients:client-upgrade-octopus-quincy-quincy-release%C2%A0-distro-default-smithi/
1 test failed from failure to fetch package

https://pulpito.ceph.com/yuriw-2023-03-15_15:26:37-upgrade-clients:client-upgrade-pacific-quincy-quincy-release-distro-default-smithi/
All green

On Tue, Mar 21, 2023 at 3:06 PM Yuri Weinstein  wrote:

> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/59070#note-1
> Release Notes - TBD
>
> The reruns were in the queue for 4 days because of some slowness issues.
> The core team (Neha, Radek, Laura, and others) are trying to narrow
> down the root cause.
>
> Seeking approvals/reviews for:
>
> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test
> and merge at least one PR https://github.com/ceph/ceph/pull/50575 for
> the core)
> rgw - Casey
> fs - Venky (the fs suite has an unusually high amount of failed jobs,
> any reason to suspect it in the observed slowness?)
> orch - Adam King
> rbd - Ilya
> krbd - Ilya
> upgrade/octopus-x - Laura is looking into failures
> upgrade/pacific-x - Laura is looking into failures
> upgrade/quincy-p2p - Laura is looking into failures
> client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman
> is looking into it
> powercycle - Brad
> ceph-volume - needs a rerun on merged
> https://github.com/ceph/ceph-ansible/pull/7409
>
> Please reply to this email with approval and/or trackers of known
> issues/PRs to address them.
>
> Also, share any findings or hypnosis about the slowness in the
> execution of the suite.
>
> Josh, Neha - gibba and LRC upgrades pending major suites approvals.
> RC release - pending major suites approvals.
>
> Thx
> YuriW
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>

-- 

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage 

Chicago, IL

lflo...@ibm.com | lflo...@redhat.com 
M: +17087388804
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io