Re: [Development] Decrease amounth of delivered src packages

[Jani Heikkinen]

Ok, it seems we have reached a consensus here: From now on we will deliver two 
different src package sets

- One set of src packages with ẃindows line endings (.zip, both single one & 
submodule specific, both enterprise and lgpl)
- One set of src packages with unix line endings (.tar.xz, both single one & 
submodule specific, both enterprise and lgpl)

With alpha we sill delivered all as earlier but we will change our system now 
to adapt the change

br,
Jani

From: Development <development-bounces+jani.heikkinen=qt...@qt-project.org> on 
behalf of Matthew Woehlke <mwoehlke.fl...@gmail.com>
Sent: Friday, February 17, 2017 6:20:20 PM
To: development@qt-project.org
Subject: Re: [Development] Decrease amounth of delivered src packages

On 2017-02-15 18:11, Mathias Hasselmann wrote:
> The actual value of gzip and the reason for its return seems to be its
> significantly lower CPU usage. This is useful to reduce server load on
> heavily utilized services like github. This is useful to reduce
> development roundtrip cycles.
>
> Just to illustrate this I've collected some numbers on my Thinkpad:
>
>   Command | Average  | Savings  | Archive | Savings
>   | CPU time | p. build | size| @100MBit
>  
>   gzip| 00:44.19 | 09:54.58 | 469 MiB | 00:00.00
>   gzip -9 | 01:55.58 | 08:43.18 | 465 MiB | 00:00.32
>   xz  | 08:46.16 | 01:52.60 | 354 MiB | 00:09.20
>   xz -9   | 10:38.77 | 00:00.00 | 333 MiB | 00:10.88
>
> Is it worth to spend 10 additional minutes per CI cycle just to save our
> users a very few seconds of download time?

Um... yes? 10 min * once = 10 s * 60 users. Do you imply that fewer than
60 users use the .xz packages? Remember, that's not just *user* benefit,
that is also 10 s *per download* less load on the servers. Compression
happens once; downloads happen many times.

OTOH...

A better metric is decompression, which also happens many times. On one
of my machines, it takes¹ about 8.5 s to uncompress the .gz (to
/dev/null) and about 19 s to uncompress the .xz (also to /dev/null).
So... it does cost about 10 s more CPU time to uncompress the .xz. That
being the case, I'll grant that *if* you're on a sufficiently high-speed
network, maybe it doesn't make sense to download the .xz.

(¹ I ran 20 trials each to reduce artifacts from caching, etc. I got
very consistent times, so I have high confidence that these numbers are
reasonable.)

--
Matthew
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Lisandro Damián Nicanor Pérez Meyer]

On miércoles, 15 de febrero de 2017 17:29:43 ART Thiago Macieira wrote:
> On quarta-feira, 15 de fevereiro de 2017 15:11:49 PST Mathias Hasselmann
> 
> wrote:
> > That's a somewhat limited point of view. Yes, xz archives are slightly
> > smaller, but to be honest: In the days of 4K video streaming saving
> > 100MiB of download size doesn't seem as important as it was.
> 
> There are still people with bad or slow connections.

And lots of bandwith usage: download from qt, upload to distro server, push to 
every mirror around, users downloading the source code from distros...

Granted, some of we distro maintainers could repack the source code, but it is 
always better to have the original whenever possible.

> > Is it worth to spend 10 additional minutes per CI cycle just to save our
> > users a very few seconds of download time?
> 
> Wait, CI cycle?
> 
> I thought we were talking about releases only, which are final as well as
> snapshot packages. Why is the CI creating tarballs?


If the problem is the CI then the CI might use gz and use xz for released 
tarballs.

-- 
 AlfaScorpii: podés converncerla a tu vieja que le esconda
el diccionario a el_machi? Cada vez que aprende una palabra nueva
busca cualquier excusa para usarla
 e-squizo: leí mi primer diccionario a los 5 años
[...]
 e-squizo: como mi vieja no aprenda lo suficiente de
hacking no veo como haria para bajar el sitio de la real academia
  Visto en #grulic, irc.freenode.net

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/


signature.asc
Description: This is a digitally signed message part.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Thiago Macieira]

On quinta-feira, 16 de fevereiro de 2017 07:06:51 PST Kevin Kofler wrote:
> It was found that in practice, requesting the same tag from GitHub multiple
> times gives byte-identical tar.gz output. (Unfortunately, GitHub cannot
> deliver better compressions.) Of course this might break if they upgrade
> their gzip, but nothing is perfect.

Right, but my point is that you can get the same uncompressed stream by 
running it archive in anyone's machine.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Kevin Kofler]

Thiago Macieira wrote:
> You can compare the checksum of the uncompressed tarball.

But this is not how the Fedora workflow works. And please also note that I 
did not make the policies, I am just trying to explain why they are the way 
they are. And I suspect that there are several other distributions that work 
similarly.

> Two people running git archive on the same commit or tag will produce the
> same uncompressed stream (including GitHub), but not necessarily the same
> compressed stream.

It was found that in practice, requesting the same tag from GitHub multiple 
times gives byte-identical tar.gz output. (Unfortunately, GitHub cannot 
deliver better compressions.) Of course this might break if they upgrade 
their gzip, but nothing is perfect.

Kevin Kofler

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Thiago Macieira]

On quarta-feira, 15 de fevereiro de 2017 15:11:49 PST Mathias Hasselmann 
wrote:
> That's a somewhat limited point of view. Yes, xz archives are slightly
> smaller, but to be honest: In the days of 4K video streaming saving
> 100MiB of download size doesn't seem as important as it was.

There are still people with bad or slow connections.

> Is it worth to spend 10 additional minutes per CI cycle just to save our
> users a very few seconds of download time?

Wait, CI cycle?

I thought we were talking about releases only, which are final as well as 
snapshot packages. Why is the CI creating tarballs?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Thiago Macieira]

On quarta-feira, 15 de fevereiro de 2017 16:14:30 PST Kevin Kofler wrote:
> Mathias Hasselmann wrote:
> > If distros care about size they can re-compress.
> > Well, and to my experience they do.
> 
> I cannot speak for other distributions, but it is Fedora policy to never
> modify an upstream tarball unless it is required to remove files that cannot
> be legally redistributed for some reason (e.g. patents). This includes
> recompression. The reason for that policy is that recompressing makes it
> harder to compare file checksums.

You can compare the checksum of the uncompressed tarball. Two people running 
git archive on the same commit or tag will produce the same uncompressed 
stream (including GitHub), but not necessarily the same compressed stream.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Kevin Kofler]

Mathias Hasselmann wrote:
> If distros care about size they can re-compress.
> Well, and to my experience they do.

I cannot speak for other distributions, but it is Fedora policy to never 
modify an upstream tarball unless it is required to remove files that cannot 
be legally redistributed for some reason (e.g. patents). This includes 
recompression. The reason for that policy is that recompressing makes it 
harder to compare file checksums.

Kevin Kofler

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Mathias Hasselmann]

Am 15.02.2017 um 18:08 schrieb Konstantin Tokarev:



15.02.2017, 20:07, "Thiago Macieira" :

On quarta-feira, 15 de fevereiro de 2017 09:04:12 PST Konstantin Podsvirov
wrote:

 This can be not quite on this topic, but I have a suggestion and MVP
 implementation for QtIFW.

 This may affect the development process, testing and create offline
 installers.


Most Valuable Player?


Most Valuable Professional?


Most Verbose Poser? ;-) :-D
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Mathias Hasselmann]

Am 15.02.2017 um 13:05 schrieb Dmitry Shachnev:

On Wed, Feb 15, 2017 at 12:28:11PM +0100, Kevin Kofler wrote:

I would kindly request you to at least use tar.xz (rather than tar.gz) for
the tarballs. (What you use as the Windows format is something you need to
sort out with the Windows people.) The fact that tar.gz is still the most
downloaded is probably mostly out of habit, or maybe your download site is
directing to them by default (which ought to be fixed anyway, even if you
were to keep both). tar.gz has no advantage over tar.xz, it is just a lot
larger. Switching to the tar.gz tarballs (from the tar.xz tarballs that are
currently used) would increase the size of distributions' source packages
(source RPM etc.) significantly.


If distros care about size they can re-compress.
Well, and to my experience they do.


It is sad that the legacy gzip compression is living a renaissance due to
automatic tarball exports from GitHub and the like producing only that
format. It should finally be retired now that there are algorithms that are
just as open and that compress significantly better. At least for projects
like Qt that produce their own tarballs and are already able to produce xz-
compressed ones, I see no reason whatsoever to switch back to the obsolete
gzip.


+1, please leave tar.xz instead of tar.gz.

Users of all modern UNIX-like systems are able to decompress tar.xz, so .gz
has really no advantage over .xz.


That's a somewhat limited point of view. Yes, xz archives are slightly 
smaller, but to be honest: In the days of 4K video streaming saving 
100MiB of download size doesn't seem as important as it was.


The actual value of gzip and the reason for its return seems to be its 
significantly lower CPU usage. This is useful to reduce server load on 
heavily utilized services like github. This is useful to reduce 
development roundtrip cycles.


Just to illustrate this I've collected some numbers on my Thinkpad:

  Command | Average  | Savings  | Archive | Savings
  | CPU time | p. build | size| @100MBit
 
  gzip| 00:44.19 | 09:54.58 | 469 MiB | 00:00.00
  gzip -9 | 01:55.58 | 08:43.18 | 465 MiB | 00:00.32
  xz  | 08:46.16 | 01:52.60 | 354 MiB | 00:09.20
  xz -9   | 10:38.77 | 00:00.00 | 333 MiB | 00:10.88

Is it worth to spend 10 additional minutes per CI cycle just to save our 
users a very few seconds of download time?


Looking at the problems behind providing an official 5.8.1. Looking at 
my personal experience with slow CI systems I clearly vote for speeding 
up the Q/A process and sticking with .zip and .tar.gz.


Besides: Does it really make sense to fully test the expensive .xz and 
.7z builds? In my opinion it would be fully sufficient to only give the 
inexpensive .zip and .gz archives full test coverage. At least the .xz 
could be generated on the fly after uploading to the web server by 
simply decompression:


  gunzip < qt-sources.tar.gz | xz -9 > qt-sources.tar.xz

Ciao,
Mathias
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[André Pönitz]

On Wed, Feb 15, 2017 at 08:54:40AM -0800, Thiago Macieira wrote:
> Em quarta-feira, 15 de fevereiro de 2017, às 15:05:06 PST, Dmitry Shachnev 
> escreveu:
> > On Wed, Feb 15, 2017 at 12:28:11PM +0100, Kevin Kofler wrote:
> > > I would kindly request you to at least use tar.xz (rather than tar.gz) for
> > > the tarballs. (What you use as the Windows format is something you need to
> [cut]
> > +1, please leave tar.xz instead of tar.gz.
> > 
> > Users of all modern UNIX-like systems are able to decompress tar.xz, so .gz
> > has really no advantage over .xz.
> 
> Adding my voice to that. If data volume is an issue, then let's by all means 
> standardise on .tar.xz.

Similarly here.

Download statistics are -- as any usage based statistics -- in first
approximation independent of user intent and capability as they do not
account for cognitive bias like "people take the first feasible option"
or "use something with familiar name" etc.

If the question is how to reduce server load (a reasonable goal in
my book) the question would be "which format is smallest without
denying access".

Andre'

[Not speaking for any company, strictly my own opinion, not having
any stake in any of the mentioned formats, etc...]
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Konstantin Podsvirov]

15.02.2017, 20:10, "Alexandru Croitor" :
> I think it's minimal viable product.

Yes! It'w work for me now. You can try QtIFW (with --repository option) from my 
experimental installer for Windows:

http://download.podsvirov.pro/installers/dad-0.3.1-windows-mingw64-unstable.exe

>
>>  On 15 Feb 2017, at 18:08, Konstantin Tokarev  wrote:
>>
>>  15.02.2017, 20:07, "Thiago Macieira" :
>>>  On quarta-feira, 15 de fevereiro de 2017 09:04:12 PST Konstantin Podsvirov
>>>  wrote:
   This can be not quite on this topic, but I have a suggestion and MVP
   implementation for QtIFW.

   This may affect the development process, testing and create offline
   installers.
>>>
>>>  Most Valuable Player?
>>
>>  Most Valuable Professional?
>>
>>>  --
>>>  Thiago Macieira - thiago.macieira (AT) intel.com
>>>    Software Architect - Intel Open Source Technology Center
>>>
>>>  ___
>>>  Development mailing list
>>>  Development@qt-project.org
>>>  http://lists.qt-project.org/mailman/listinfo/development
>>
>>  --
>>  Regards,
>>  Konstantin
>>  ___
>>  Development mailing list
>>  Development@qt-project.org
>>  http://lists.qt-project.org/mailman/listinfo/development
>
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin Podsvirov
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Alexandru Croitor]

I think it's minimal viable product.

> On 15 Feb 2017, at 18:08, Konstantin Tokarev  wrote:
> 
> 
> 
> 15.02.2017, 20:07, "Thiago Macieira" :
>> On quarta-feira, 15 de fevereiro de 2017 09:04:12 PST Konstantin Podsvirov
>> wrote:
>>>  This can be not quite on this topic, but I have a suggestion and MVP
>>>  implementation for QtIFW.
>>> 
>>>  This may affect the development process, testing and create offline
>>>  installers.
>> 
>> Most Valuable Player?
> 
> Most Valuable Professional?
> 
>> 
>> --
>> Thiago Macieira - thiago.macieira (AT) intel.com
>>   Software Architect - Intel Open Source Technology Center
>> 
>> ___
>> Development mailing list
>> Development@qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
> 
> -- 
> Regards,
> Konstantin
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Konstantin Tokarev]

15.02.2017, 20:07, "Thiago Macieira" :
> On quarta-feira, 15 de fevereiro de 2017 09:04:12 PST Konstantin Podsvirov
> wrote:
>>  This can be not quite on this topic, but I have a suggestion and MVP
>>  implementation for QtIFW.
>>
>>  This may affect the development process, testing and create offline
>>  installers.
>
> Most Valuable Player?

Most Valuable Professional?

>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel Open Source Technology Center
>
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

-- 
Regards,
Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Thiago Macieira]

On quarta-feira, 15 de fevereiro de 2017 09:04:12 PST Konstantin Podsvirov 
wrote:
> This can be not quite on this topic, but I have a suggestion and MVP
> implementation for QtIFW.
> 
> 
> This may affect the development process, testing and create offline
> installers.

Most Valuable Player?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Konstantin Podsvirov]

Hello all!This can be not quite on this topic, but I have a suggestion and MVP implementation for QtIFW.This may affect the development process, testing and create offline installers.19:55, 15 February 2017 г., Thiago Macieira :Em quarta-feira, 15 de fevereiro de 2017, às 15:05:06 PST, Dmitry Shachnev escreveu: On Wed, Feb 15, 2017 at 12:28:11PM +0100, Kevin Kofler wrote: > I would kindly request you to at least use tar.xz (rather than tar.gz) for > the tarballs. (What you use as the Windows format is something you need to[cut] +1, please leave tar.xz instead of tar.gz. Users of all modern UNIX-like systems are able to decompress tar.xz, so .gz has really no advantage over .xz.Adding my voice to that. If data volume is an issue, then let's by all means standardise on .tar.xz.-- Thiago Macieira - thiago.macieira (AT) intel.com  Software Architect - Intel Open Source Technology Center___Development mailing listDevelopment@qt-project.orghttp://lists.qt-project.org/mailman/listinfo/development--Regards,Konstantin Podsvirov
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Thiago Macieira]

Em quarta-feira, 15 de fevereiro de 2017, às 15:05:06 PST, Dmitry Shachnev 
escreveu:
> On Wed, Feb 15, 2017 at 12:28:11PM +0100, Kevin Kofler wrote:
> > I would kindly request you to at least use tar.xz (rather than tar.gz) for
> > the tarballs. (What you use as the Windows format is something you need to
[cut]
> +1, please leave tar.xz instead of tar.gz.
> 
> Users of all modern UNIX-like systems are able to decompress tar.xz, so .gz
> has really no advantage over .xz.

Adding my voice to that. If data volume is an issue, then let's by all means 
standardise on .tar.xz.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Dmitry Shachnev]

On Wed, Feb 15, 2017 at 12:28:11PM +0100, Kevin Kofler wrote:
> I would kindly request you to at least use tar.xz (rather than tar.gz) for
> the tarballs. (What you use as the Windows format is something you need to
> sort out with the Windows people.) The fact that tar.gz is still the most
> downloaded is probably mostly out of habit, or maybe your download site is
> directing to them by default (which ought to be fixed anyway, even if you
> were to keep both). tar.gz has no advantage over tar.xz, it is just a lot
> larger. Switching to the tar.gz tarballs (from the tar.xz tarballs that are
> currently used) would increase the size of distributions' source packages
> (source RPM etc.) significantly.
>
> It is sad that the legacy gzip compression is living a renaissance due to
> automatic tarball exports from GitHub and the like producing only that
> format. It should finally be retired now that there are algorithms that are
> just as open and that compress significantly better. At least for projects
> like Qt that produce their own tarballs and are already able to produce xz-
> compressed ones, I see no reason whatsoever to switch back to the obsolete
> gzip.

+1, please leave tar.xz instead of tar.gz.

Users of all modern UNIX-like systems are able to decompress tar.xz, so .gz
has really no advantage over .xz.

--
Dmitry Shachnev


signature.asc
Description: PGP signature
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Kevin Kofler]

Jani Heikkinen wrote:
> well, of course this would be possible as well but this would be a change
> to existing "behavior" : Currently 7z is with unix line endings. And as
> you can see from statistic tar.gz and .zip are most used ones and so on it
> would be quite reasonable to continue offering those

I would kindly request you to at least use tar.xz (rather than tar.gz) for 
the tarballs. (What you use as the Windows format is something you need to 
sort out with the Windows people.) The fact that tar.gz is still the most 
downloaded is probably mostly out of habit, or maybe your download site is 
directing to them by default (which ought to be fixed anyway, even if you 
were to keep both). tar.gz has no advantage over tar.xz, it is just a lot 
larger. Switching to the tar.gz tarballs (from the tar.xz tarballs that are 
currently used) would increase the size of distributions' source packages 
(source RPM etc.) significantly.

It is sad that the legacy gzip compression is living a renaissance due to 
automatic tarball exports from GitHub and the like producing only that 
format. It should finally be retired now that there are algorithms that are 
just as open and that compress significantly better. At least for projects 
like Qt that produce their own tarballs and are already able to produce xz-
compressed ones, I see no reason whatsoever to switch back to the obsolete 
gzip.

The bzip2 (tar.bz2) compression is also obsoleted by xz nowadays, I see no 
good reason to pick that one either.

Kevin Kofler

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Jani Heikkinen]

>>Why not use most efficient compression formats instead, i.e. keep 7z and 
>>tar.xz only?
well, of course this would be possible as well but this would be a change to 
existing "behavior" : Currently 7z is with unix line endings.
And as you can see from statistic tar.gz and .zip are most used ones and so on 
it would be quite reasonable to continue offering those

br,
Jani


From: Konstantin Tokarev <annu...@yandex.ru>
Sent: Wednesday, February 15, 2017 12:46 PM
To: Jani Heikkinen; development@qt-project.org
Cc: releas...@qt-project.org
Subject: Re: [Development] Decrease amounth of delivered src packages

15.02.2017, 13:39, "Jani Heikkinen" <jani.heikki...@qt.io>:
> Hi all,
>
> As you know we need to optimize our systems to be able to keep our plans in 
> the future. In CI side we are handling at least 4 different branches at same 
> time. Releasing side we should be able to do many releases/snapshots 
> parallel, test those releases/snapshots parallel etc. And all this needs lots 
> of hw capasity (both calculating power & disc space).
>
> We are trying to analyze the places where to do optimization and one place is 
> our src package creation and delivery: At the moment we are doing, testing & 
> delivering single src packages + separate submodule src packages, all these 
> with .zip, .7z, .tar.gz & .tar.xz! Huge amount of src 
> packages/snapshot/release. And currently doing these both for enterprise and 
> lgpl! In the future (5.10?) we should be able to use unified ones meaning no 
> separate LGPL & enterprise ones.
>
> But I think we could also drop those less used ones(*) as well with quicker 
> schedule so I propose:
>
> From 5.9 -> let's start doing & delivering
>
> - One set of src packages with ẃindows line endings (.zip, both single one & 
> submodule specific, both enterprise and lgpl)
> - One set of src packages with unix line endings (.tar.gz, both single one & 
> submodule specific, both enterprise and lgpl)

Why not use most efficient compression formats instead, i.e. keep 7z and tar.xz 
only?

>
> br,
> Jani
>
> *:Download statistic:
> .7z 7%
> .tar.gz 44%
> .tar.xz 12%
> .zip 37%
>
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Decrease amounth of delivered src packages

[Konstantin Tokarev]

15.02.2017, 13:39, "Jani Heikkinen" :
> Hi all,
>
> As you know we need to optimize our systems to be able to keep our plans in 
> the future. In CI side we are handling at least 4 different branches at same 
> time. Releasing side we should be able to do many releases/snapshots 
> parallel, test those releases/snapshots parallel etc. And all this needs lots 
> of hw capasity (both calculating power & disc space).
>
> We are trying to analyze the places where to do optimization and one place is 
> our src package creation and delivery: At the moment we are doing, testing & 
> delivering single src packages + separate submodule src packages, all these 
> with .zip, .7z, .tar.gz & .tar.xz! Huge amount of src 
> packages/snapshot/release. And currently doing these both for enterprise and 
> lgpl! In the future (5.10?) we should be able to use unified ones meaning no 
> separate LGPL & enterprise ones.
>
> But I think we could also drop those less used ones(*) as well with quicker 
> schedule so I propose:
>
> From 5.9 -> let's start doing & delivering
>
> - One set of src packages with ẃindows line endings (.zip, both single one & 
> submodule specific, both enterprise and lgpl)
> - One set of src packages with unix line endings (.tar.gz, both single one & 
> submodule specific, both enterprise and lgpl)

Why not use most efficient compression formats instead, i.e. keep 7z and tar.xz 
only?

>
> br,
> Jani
>
> *:Download statistic:
> .7z 7%
> .tar.gz 44%
> .tar.xz 12%
> .zip 37%
>
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

-- 
Regards,
Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

[Development] Decrease amounth of delivered src packages

[Jani Heikkinen]

Hi all,

As you know we need to optimize our systems to be able to keep our plans in the 
future. In CI side we are handling at least 4 different branches at same time. 
Releasing side we should be able to do many releases/snapshots parallel, test 
those releases/snapshots parallel etc. And all this needs lots of hw capasity 
(both calculating power & disc space).

We are trying to analyze the places where to do optimization and one place is 
our src package creation and delivery: At the moment we are doing, testing & 
delivering single src packages + separate submodule src packages, all these 
with .zip, .7z, .tar.gz & .tar.xz! Huge amount of src 
packages/snapshot/release. And currently doing these both for enterprise and 
lgpl! In the future (5.10?) we should be able to use unified ones meaning no 
separate LGPL & enterprise ones. 

But I think we could also drop those less used ones(*) as well with quicker 
schedule so I propose:

From 5.9 -> let's start doing & delivering 

- One set of src packages with ẃindows line endings (.zip, both single one & 
submodule specific, both enterprise and lgpl)
- One set of src packages with unix line endings (.tar.gz, both single one & 
submodule specific, both enterprise and lgpl)

br,
Jani

*:Download statistic:
.7z  7%
.tar.gz 44%
.tar.xz 12%
.zip 37%






___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development