Re: [openstack-dev] [Fuel][library] Switching to external fixtures for integration Noop tests

[Bogdan Dobrelya]

An update.
The task is done, the noop tests framework
now has automatic docs builds to readthedocs [0] and a Fuel infra CI
gate that tests changes against the fuel-library master, by running
existing integration noop rspecs.

So the CI is now mutual: changes to the fuel library are tested against
the noop tests fixtures and vice versa. This means no manual testing
required, if one wants either update integration rspecs or related
astute.yaml fixtures.

Next step is to refactor tests coverage to reduce duplication of test
cases, which is binding rspecs to corresponding fixtures one to many or
many to one instead of all to all. Hopefully, this will reduce the
current set of ~6000 test cases to more reasonable amounts.

[0] http://fuel-noop-fixtures.readthedocs.org/en/latest/

On 17.02.2016 15:43, Bogdan Dobrelya wrote:
> Hello,
> an update inline!
> 
> On 27.01.2016 17:37, Bogdan Dobrelya wrote:
>> On 26.01.2016 22:18, Kyrylo Galanov wrote:
>>> Hello Bogdan,
>>>
>>> I hope I am not the one of the context. Why do we separate fixtures for
>>> Noop tests from the repo?
>>> I can understand if while noop test block was carried out to a separate
>>> repo.
>>>
>>
>> I believe fixtures normally are downloaded by the rake spec_prep.
>> Developers avoid to ship fixtures with tests.
>>
>> The astute.yaml data fixtures are supposed to be external to the
>> fuel-library as that data comes from the Nailgun backend and corresponds
>> to all known deploy paths.
>>
>> Later, the generated puppet catalogs (see [0]) shall be put to the
>> fixtures repo as well - as they will contain hundreds thousands of
>> auto-generate lines and are tightly related to the astute.yaml fixtures.
>>
>> While the Noop tests framework itself indeed may be moved to another
>> separate repo (later), we should keep our integration tests [1] in the
>> fuel-library repository, which is "under test" by those tests.
> 
> Dmitry Ilyin did a great job and reworked the Fuel-library Noop Tests
> Framework. He also provided docs to describe changes for developers.
> There is a patch [0] to move the astute.yaml fixtures, noop tests docs
> and the framework itself from the fuel-library to the fuel-noop-fixtures
> repo [1].
> 
> With the patch, full run for the Noop tests job shortens from 40 minutes
> to 5 (by 8 times!) as it supports multiple rspec processes running in
> parallel. It also provides advanced test reports. Please see details in
> the docs [2]. You can read as is or build locally with tox. Later, the
> docs will go to readthedocs.org as well.
> 
> Note, there is no impact for developers and all changes are backwards
> compatible to existing noop tests and Fuel jenkins CI jobs. Later we may
> start to add new features from the reworked framework to make things
> even better. So please take a look on the patch and new docs.
> 
> PS. Noop tests gate passed for the patch, though there is CI -1 as we
> disabled non related deployment gates by the "Fuel-CI: disable" tag.
> 
> [0] https://review.openstack.org/#/c/276816/
> [1] https://git.openstack.org/cgit/openstack/fuel-noop-fixtures
> [2] https://git.openstack.org/cgit/openstack/fuel-noop-fixtures/tree/doc
> 
>>
>> [0] https://blueprints.launchpad.net/fuel/+spec/deployment-data-dryrun
>> [1]
>> https://git.openstack.org/cgit/openstack/fuel-library/tree/tests/noop/spec/hosts
>>
>>> On Tue, Jan 26, 2016 at 1:54 PM, Bogdan Dobrelya >> > wrote:
>>>
>>> We are going to switch [0] to external astute.yaml fixtures for Noop
>>> tests and remove them from the fuel-library repo as well.
>>> Please make sure all new changes to astute.yaml fixtures will be
>>> submitted now to the new location. Related mail thread [1].
>>>
>>> [0]
>>> 
>>> https://review.openstack.org/#/c/272480/1/doc/noop-guide/source/noop_fixtures.rst
>>> [1]
>>> 
>>> http://lists.openstack.org/pipermail/openstack-dev/2015-December/082888.html
>>>
>>> --
>>> Best regards,
>>> Bogdan Dobrelya,
>>> Irc #bogdando
>>>
>>> 
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> 
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
> 
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [Fuel][library] Switching to external fixtures for integration Noop tests

[Bogdan Dobrelya]

Hello,
an update inline!

On 27.01.2016 17:37, Bogdan Dobrelya wrote:
> On 26.01.2016 22:18, Kyrylo Galanov wrote:
>> Hello Bogdan,
>>
>> I hope I am not the one of the context. Why do we separate fixtures for
>> Noop tests from the repo?
>> I can understand if while noop test block was carried out to a separate
>> repo.
>>
> 
> I believe fixtures normally are downloaded by the rake spec_prep.
> Developers avoid to ship fixtures with tests.
> 
> The astute.yaml data fixtures are supposed to be external to the
> fuel-library as that data comes from the Nailgun backend and corresponds
> to all known deploy paths.
> 
> Later, the generated puppet catalogs (see [0]) shall be put to the
> fixtures repo as well - as they will contain hundreds thousands of
> auto-generate lines and are tightly related to the astute.yaml fixtures.
> 
> While the Noop tests framework itself indeed may be moved to another
> separate repo (later), we should keep our integration tests [1] in the
> fuel-library repository, which is "under test" by those tests.

Dmitry Ilyin did a great job and reworked the Fuel-library Noop Tests
Framework. He also provided docs to describe changes for developers.
There is a patch [0] to move the astute.yaml fixtures, noop tests docs
and the framework itself from the fuel-library to the fuel-noop-fixtures
repo [1].

With the patch, full run for the Noop tests job shortens from 40 minutes
to 5 (by 8 times!) as it supports multiple rspec processes running in
parallel. It also provides advanced test reports. Please see details in
the docs [2]. You can read as is or build locally with tox. Later, the
docs will go to readthedocs.org as well.

Note, there is no impact for developers and all changes are backwards
compatible to existing noop tests and Fuel jenkins CI jobs. Later we may
start to add new features from the reworked framework to make things
even better. So please take a look on the patch and new docs.

PS. Noop tests gate passed for the patch, though there is CI -1 as we
disabled non related deployment gates by the "Fuel-CI: disable" tag.

[0] https://review.openstack.org/#/c/276816/
[1] https://git.openstack.org/cgit/openstack/fuel-noop-fixtures
[2] https://git.openstack.org/cgit/openstack/fuel-noop-fixtures/tree/doc

> 
> [0] https://blueprints.launchpad.net/fuel/+spec/deployment-data-dryrun
> [1]
> https://git.openstack.org/cgit/openstack/fuel-library/tree/tests/noop/spec/hosts
> 
>> On Tue, Jan 26, 2016 at 1:54 PM, Bogdan Dobrelya > > wrote:
>>
>> We are going to switch [0] to external astute.yaml fixtures for Noop
>> tests and remove them from the fuel-library repo as well.
>> Please make sure all new changes to astute.yaml fixtures will be
>> submitted now to the new location. Related mail thread [1].
>>
>> [0]
>> 
>> https://review.openstack.org/#/c/272480/1/doc/noop-guide/source/noop_fixtures.rst
>> [1]
>> 
>> http://lists.openstack.org/pipermail/openstack-dev/2015-December/082888.html
>>
>> --
>> Best regards,
>> Bogdan Dobrelya,
>> Irc #bogdando
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][library] Switching to external fixtures for integration Noop tests

[Bogdan Dobrelya]

On 26.01.2016 22:18, Kyrylo Galanov wrote:
> Hello Bogdan,
> 
> I hope I am not the one of the context. Why do we separate fixtures for
> Noop tests from the repo?
> I can understand if while noop test block was carried out to a separate
> repo.
> 

I believe fixtures normally are downloaded by the rake spec_prep.
Developers avoid to ship fixtures with tests.

The astute.yaml data fixtures are supposed to be external to the
fuel-library as that data comes from the Nailgun backend and corresponds
to all known deploy paths.

Later, the generated puppet catalogs (see [0]) shall be put to the
fixtures repo as well - as they will contain hundreds thousands of
auto-generate lines and are tightly related to the astute.yaml fixtures.

While the Noop tests framework itself indeed may be moved to another
separate repo (later), we should keep our integration tests [1] in the
fuel-library repository, which is "under test" by those tests.

[0] https://blueprints.launchpad.net/fuel/+spec/deployment-data-dryrun
[1]
https://git.openstack.org/cgit/openstack/fuel-library/tree/tests/noop/spec/hosts

> On Tue, Jan 26, 2016 at 1:54 PM, Bogdan Dobrelya  > wrote:
> 
> We are going to switch [0] to external astute.yaml fixtures for Noop
> tests and remove them from the fuel-library repo as well.
> Please make sure all new changes to astute.yaml fixtures will be
> submitted now to the new location. Related mail thread [1].
> 
> [0]
> 
> https://review.openstack.org/#/c/272480/1/doc/noop-guide/source/noop_fixtures.rst
> [1]
> 
> http://lists.openstack.org/pipermail/openstack-dev/2015-December/082888.html
> 
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][library] Switching to external fixtures for integration Noop tests

[Kyrylo Galanov]

Hello Bogdan,

I hope I am not the one of the context. Why do we separate fixtures for Noop
 tests from the repo?
I can understand if while noop test block was carried out to a separate repo
.

On Tue, Jan 26, 2016 at 1:54 PM, Bogdan Dobrelya 
wrote:

> We are going to switch [0] to external astute.yaml fixtures for Noop
> tests and remove them from the fuel-library repo as well.
> Please make sure all new changes to astute.yaml fixtures will be
> submitted now to the new location. Related mail thread [1].
>
> [0]
>
> https://review.openstack.org/#/c/272480/1/doc/noop-guide/source/noop_fixtures.rst
> [1]
>
> http://lists.openstack.org/pipermail/openstack-dev/2015-December/082888.html
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel][library] Switching to external fixtures for integration Noop tests

[Bogdan Dobrelya]

We are going to switch [0] to external astute.yaml fixtures for Noop
tests and remove them from the fuel-library repo as well.
Please make sure all new changes to astute.yaml fixtures will be
submitted now to the new location. Related mail thread [1].

[0]
https://review.openstack.org/#/c/272480/1/doc/noop-guide/source/noop_fixtures.rst
[1]
http://lists.openstack.org/pipermail/openstack-dev/2015-December/082888.html

-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev