Re: [ovirt-devel] [lago-devel] ovirt tests failing on missing libxml2-python

[Yaniv Kaul]

On Sun, Jul 3, 2016 at 3:31 PM, Barak Korren  wrote:

> >> Maybe we can take a middle ground, pre-fetch, but also enable external
> >> repos in CI (perhaps with some way to log and find out what was not
> >> pre-fetched).
> >
> > This is what the code is supposed to do, I suspect. reposync syncs
> between
> > what you already have and what you fetch, no?
> > Y.
> >
> I was referring to the deployment/test code.
> AFAIK right now the external repos are disabled before the test starts
>

Indeed, to save time not fetching their metadata, and ensure we have the
correct deps. I guess in some cases we can enable them - need to see how
much time it wastes.
Y.


>
>
> --
> Barak Korren
> bkor...@redhat.com
> RHEV-CI Team
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [lago-devel] ovirt tests failing on missing libxml2-python

[Barak Korren]

>> Maybe we can take a middle ground, pre-fetch, but also enable external
>> repos in CI (perhaps with some way to log and find out what was not
>> pre-fetched).
>
> This is what the code is supposed to do, I suspect. reposync syncs between
> what you already have and what you fetch, no?
> Y.
>
I was referring to the deployment/test code.
AFAIK right now the external repos are disabled before the test starts


-- 
Barak Korren
bkor...@redhat.com
RHEV-CI Team
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [lago-devel] ovirt tests failing on missing libxml2-python

[Yaniv Kaul]

On Wed, Jun 29, 2016 at 11:15 PM, Barak Korren  wrote:

> On 29 June 2016 at 21:45, Yaniv Kaul  wrote:
> > On Mon, Jun 27, 2016 at 9:45 AM, Barak Korren 
> wrote:
> >>
> >> >
> >> >
> >> > It means that packages will be fetched EVERY time from outside, which
> >> > may be
> >> > slow(er).
> >> > Y.
> >> >
> >>
> >> We can (and mostly already have) setup simple caches to prevent that.
> >
> >
> > How do you set up cache on a developer's laptop?
> >
> We may have been unclear in our intentions, we want to make the
> pre-syncing optional not remove it completely. It does make sense on
> the laptop (sometimes), but not so much in the CI env.
>
> > The repo directory does not need to be cleaned every time.
>
> This is an assumption that may break if we end up having any corrupt
> or failing packages in the cache. It also make it hard to "go back in
> time" if we want to test without some update.
> (Cleaning corrupt caches an re-running is easy in a local setting, in
> CI you end up dealing with angry devs getting false '-1's)
>

True, and we don't want that. Developers have to trust the CI system.
This is an important point.


>
> > It can also be
> > resync'ed from a central repo - which still going to be faster than any
> > other fetching.
> > (hopefully sync'ed into the slave /dev/shm btw).
>
> It could be faster, but could also be slower if you end up fetching
> more then you have to. (if engine setup fails on missing dependency,
> you just spent needless time fetching VDSM dpes)
> Also fetching by itself may not be the bottleneck in all cases, it is
> surely slow when fetching from PHX to TLV, but when fetching from the
> Squid proxy's RAM inside PHX it can actually end up being faster then
> copying from the local disk.
>

I always fetch and store on /dev/shm/repostore
It's faster than anything else.

I did copy its content once to the disk, so when the host reboots, it
rsync's this to /dev/shm/repostore , then tests begin.
That perhaps is indeed not very needed in CI.


> >> The existing solution looks more like premature optimization gone badly
> >> IMO.
> >
> > Try to run ovirt-system-tests, clean the repo and re-run - it's 20-30
> > minutes at least longer - which is far more than what it takes to run the
> > whole test suite.
>
> I wonder how many of those minutes are spend on fetching things we
> actually need, and how much is spent on overhead. I suspect that
> without a local cache, the test run will be longer, but not as long as
> the pre-fetching+tests takes currently. More importantly, this may
> allow the CI to fail faster. I think we should at least test that.
>
> > I completely agree the manual maintenance is an annoyance, wish we had
> > something in between.
>
> Maybe we can take a middle ground, pre-fetch, but also enable external
> repos in CI (perhaps with some way to log and find out what was not
> pre-fetched).
>

This is what the code is supposed to do, I suspect. reposync syncs between
what you already have and what you fetch, no?
Y.


> --
> Barak Korren
> bkor...@redhat.com
> RHEV-CI Team
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [lago-devel] ovirt tests failing on missing libxml2-python

[Barak Korren]

On 29 June 2016 at 21:45, Yaniv Kaul  wrote:
> On Mon, Jun 27, 2016 at 9:45 AM, Barak Korren  wrote:
>>
>> >
>> >
>> > It means that packages will be fetched EVERY time from outside, which
>> > may be
>> > slow(er).
>> > Y.
>> >
>>
>> We can (and mostly already have) setup simple caches to prevent that.
>
>
> How do you set up cache on a developer's laptop?
>
We may have been unclear in our intentions, we want to make the
pre-syncing optional not remove it completely. It does make sense on
the laptop (sometimes), but not so much in the CI env.

> The repo directory does not need to be cleaned every time.

This is an assumption that may break if we end up having any corrupt
or failing packages in the cache. It also make it hard to "go back in
time" if we want to test without some update.
(Cleaning corrupt caches an re-running is easy in a local setting, in
CI you end up dealing with angry devs getting false '-1's)

> It can also be
> resync'ed from a central repo - which still going to be faster than any
> other fetching.
> (hopefully sync'ed into the slave /dev/shm btw).

It could be faster, but could also be slower if you end up fetching
more then you have to. (if engine setup fails on missing dependency,
you just spent needless time fetching VDSM dpes)
Also fetching by itself may not be the bottleneck in all cases, it is
surely slow when fetching from PHX to TLV, but when fetching from the
Squid proxy's RAM inside PHX it can actually end up being faster then
copying from the local disk.

>> The existing solution looks more like premature optimization gone badly
>> IMO.
>
> Try to run ovirt-system-tests, clean the repo and re-run - it's 20-30
> minutes at least longer - which is far more than what it takes to run the
> whole test suite.

I wonder how many of those minutes are spend on fetching things we
actually need, and how much is spent on overhead. I suspect that
without a local cache, the test run will be longer, but not as long as
the pre-fetching+tests takes currently. More importantly, this may
allow the CI to fail faster. I think we should at least test that.

> I completely agree the manual maintenance is an annoyance, wish we had
> something in between.

Maybe we can take a middle ground, pre-fetch, but also enable external
repos in CI (perhaps with some way to log and find out what was not
pre-fetched).

-- 
Barak Korren
bkor...@redhat.com
RHEV-CI Team
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [lago-devel] ovirt tests failing on missing libxml2-python

[Yaniv Kaul]

On Mon, Jun 27, 2016 at 9:45 AM, Barak Korren  wrote:

> >
> >
> > It means that packages will be fetched EVERY time from outside, which
> may be
> > slow(er).
> > Y.
> >
>
> We can (and mostly already have) setup simple caches to prevent that.
>

How do you set up cache on a developer's laptop?


> AFAIK CI slaves are cleaned every time anyway, so in practice there
> wouldn`t be much difference except we will have less hard-coding and
> perhaps be more efficient (are we certain we only download what we
> need atm?)
>

The repo directory does not need to be cleaned every time. It can also be
resync'ed from a central repo - which still going to be faster than any
other fetching.
(hopefully sync'ed into the slave /dev/shm btw).


> The existing solution looks more like premature optimization gone badly
> IMO.
>

Try to run ovirt-system-tests, clean the repo and re-run - it's 20-30
minutes at least longer - which is far more than what it takes to run the
whole test suite.

I completely agree the manual maintenance is an annoyance, wish we had
something in between.
Y.


>
> --
> Barak Korren
> bkor...@redhat.com
> RHEV-CI Team
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [lago-devel] ovirt tests failing on missing libxml2-python

[Barak Korren]

>
>
> It means that packages will be fetched EVERY time from outside, which may be
> slow(er).
> Y.
>

We can (and mostly already have) setup simple caches to prevent that.
AFAIK CI slaves are cleaned every time anyway, so in practice there
wouldn`t be much difference except we will have less hard-coding and
perhaps be more efficient (are we certain we only download what we
need atm?)

The existing solution looks more like premature optimization gone badly IMO.

-- 
Barak Korren
bkor...@redhat.com
RHEV-CI Team
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel