Re: DNF and PackageKit background data usage

[Kevin Kofler]

Vít Ondruch wrote:
> While more convenient way is "dnf history undo last", which used to work
> with YUM, but is broken in DNF, since it does not keep the cache around,
> as Kevin pointed out.

YUM was also set up with keepcache=0 in the Fedora package, against the 
upstream default of keepcache=1.

In addition, even with keepcache=1, it would not find the package to revert 
to if it was no longer in the repos, I had to use rpm -Uvh --oldpackage by 
hand. No idea whether this is better in DNF, I didn't have the situation 
with DNF yet (and besides, I usually update with PackageKit, which has its 
own cache, and which doesn't do downgrades at all, leaving me with only the 
rpm -Uvh --oldpackage approach).

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Kevin Kofler]

Adam Williamson wrote:
> (Obviously the *right* way to do that general approach is some form of
> snapshotting...

No. If one update is broken, I want to revert that particular update, not my 
entire system. The Window$ "system restore" approach is just broken and a 
horrible idea to reimplement.

> hi, ostree).

And ostree is even worse, copying the smartphone approach of having to 
update the whole take-it-or-leave-it image at every change, and also losing 
the ability to customize the package set. And reverting is just as broken as 
in the client-side snapshot approach. There is no way I am going to use an 
ostree any time soon.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Kevin Fenzi]

On Sun, 06 Nov 2016 23:57:07 -
Samuel Rakitničan  wrote:

> > On Sat, 2016-11-05 at 05:01 +0100, Kevin Kofler wrote:
> > 
> > "hunt down"?
> > 
> > koji download-build (nvr) works fine.  

Just a side note: Kevin Kofler didn't say that. 
Please watch your attributions. 

> I think you are missing the point here. Reverting dnf history does
> not work if packages are missing from repository. Besides "koji"
> command is not installed on my machine, and hence not on default
> install. Your suggestion for users is to install "RPM Development
> Tools" group?
> 
> I don't see how this command would help in what I usually do when I
> need stuff that is missing from repo, other then a little bit more
> convenience perhaps. That is to locate the packages in
> koji.fedoraproject.org and pass the urls to curl.

This command does that part... locates the package on koji and passes
the urls to curl and downloads them for you. 

kevin


pgpMkr2Y6_Tq4.pgp
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Vít Ondruch]

Dne 5.11.2016 v 07:10 Adam Williamson napsal(a):
> On Sat, 2016-11-05 at 05:01 +0100, Kevin Kofler wrote:
>> Otherwise, you have to hunt down the old builds 
>> directly in Koji
> "hunt down"?
>
> koji download-build (nvr) works fine.

While more convenient way is "dnf history undo last", which used to work
with YUM, but is broken in DNF, since it does not keep the cache around,
as Kevin pointed out.

Vít
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Theodore Papadopoulo]

On 11/05/2016 05:01 AM, Kevin Kofler wrote:
> Theodore Papadopoulo wrote:
>> Add to this that these caches seem to be never cleaned, so that they
>> grow up very large  up to the point they prevent updating the system. I
>> just found that the packagekit cache on my machine is about 7Gb !!!
>> Probably because I do use packageit at all (dnf cache is much smaller at
>> ~700 Mb).
> 
> IMHO, this is a feature, and the DNF default behavior is the bug, a critical 
> data loss bug even! Given that Fedora does NOT keep old updates on its 
> mirrors, the only convenient way to revert a broken update is to keep ALL 
> old updates cached locally. Otherwise, you have to hunt down the old builds 
> directly in Koji. The version from the GA release is more often than not TOO 
> old, you really want the previous update.
> 
> I consider it very broken to delete files that I might still need without 
> even asking me. Setting keepcache=1 is the one of the very first things I do 
> to any new Fedora installation.

Well, if I understand well. You tell us that because fedora repos do not
keep versions of old packages, all fedora machine have to keep them
locally ?? And that just in case they want to revert a package.
Honestly, why on earth would one ask every fedora machine to have copies
of something that is not even kept in the original repo ?

To me, if the other solutions that were proposed are not OK for some
reason, it is the fedora repo policy of not keeping old updates that is
wrong

Given the usual thoughtfull posts you usually make in this list, I fear
I have not well understood what you say or I'm missing some point
Otherwise, let's not correct a bug by creating other bugs


Theo.



smime.p7s
Description: S/MIME Cryptographic Signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Petr Pisar]

On 2016-11-07, Adam Williamson  wrote:
> I don't believe the vast majority of people actually install old
> package versions *ever*, unless they get very explicit instructions to
> do so either because there was a big fail of some kind and we did our
> usual emergency drill, or they hit some kind of very specific bug and
> got very specific instructions to do so in Bugzilla or something.
>
Becuse they don't know how, they are lost without explicit
instructions.

> I'd say. I think the
> idea that everyone needs to have a gigantic cache of every package
> version they've ever installed stored locally

Only few (usually one) previous ones are enough. If a buggy package
reaches stable repository, downgrading this one package is usually the
easiest temporary fix until package maintainer provides a fix.

I recommend to look into repository of a rolling distribution.

> they need to install an old one some time is...wrong. (Obviously the
> *right* way to do that general approach is some form of
> snapshotting...hi, ostree).

Of course. User is waiting for a bug fix for his hot package, na atomic
update delivers the fixed package with unrelated broken package. The
result is as terrible as now. User get everything or nothing.

That's maybe fine for laic users who want only to use. But unaccaptable
for advanced users or developers. Is Fedora still for developers?

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Petr Pisar]

On 2016-11-05, Adam Williamson  wrote:
> On Sat, 2016-11-05 at 05:01 +0100, Kevin Kofler wrote:
>> Otherwise, you have to hunt down the old builds 
>> directly in Koji
>
> "hunt down"?
>
> koji download-build (nvr) works fine.

DNF history lists binary package NVRs. Koji works with source package
NVRS. You cannot programatically map from the first one to the second.

Moreover users use package manager. They do not know anything about
Koji.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Sun, 2016-11-06 at 23:57 +, Samuel Rakitničan wrote:
> > On Sat, 2016-11-05 at 05:01 +0100, Kevin Kofler wrote:
> > 
> > "hunt down"?
> > 
> > koji download-build (nvr) works fine.
> 
> I think you are missing the point here. Reverting dnf history does
> not work if packages are missing from repository. Besides "koji"
> command is not installed on my machine, and hence not on default
> install. Your suggestion for users is to install "RPM Development
> Tools" group?

I don't believe the vast majority of people actually install old
package versions *ever*, unless they get very explicit instructions to
do so either because there was a big fail of some kind and we did our
usual emergency drill, or they hit some kind of very specific bug and
got very specific instructions to do so in Bugzilla or something.

I should probably have said that too to make it more clear, but
basically, in the rather unusual case that you actually do need to get
an old package, the koji CLI does the job fine, I'd say. I think the
idea that everyone needs to have a gigantic cache of every package
version they've ever installed stored locally just on the off-chance
they need to install an old one some time is...wrong. (Obviously the
*right* way to do that general approach is some form of
snapshotting...hi, ostree).
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Samuel Rakitničan]

> On Sat, 2016-11-05 at 05:01 +0100, Kevin Kofler wrote:
> 
> "hunt down"?
> 
> koji download-build (nvr) works fine.

I think you are missing the point here. Reverting dnf history does not work if 
packages are missing from repository. Besides "koji" command is not installed 
on my machine, and hence not on default install. Your suggestion for users is 
to install "RPM Development Tools" group?

I don't see how this command would help in what I usually do when I need stuff 
that is missing from repo, other then a little bit more convenience perhaps. 
That is to locate the packages in koji.fedoraproject.org and pass the urls to 
curl.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Sat, 2016-11-05 at 05:01 +0100, Kevin Kofler wrote:
> Otherwise, you have to hunt down the old builds 
> directly in Koji

"hunt down"?

koji download-build (nvr) works fine.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Kevin Kofler]

Adam Williamson wrote:
> 1) Both dnf and GNOME Software / PackageKit default to performing
> fairly data-hungry transactions in the background, out of the box,
> without telling you about it. GNOME's is particularly bad, as it will
> happily download available updates in the background, which can be
> gigabytes worth of data. DNF only updates its metadata caches (on a
> systemd timer), but even that could be behaviour that users in certain
> circumstances really really do not want.

IMHO, both of these are misbehaviors:

For the dnf case: I am not convinced that dnf's cronjob that always updates 
metadata rather than updating it when actually needed is a win. Sure, it 
helps interactivity, but at the expense of a lot of unnecessary downloads, 
especially if you use dnf rarely or not at all (because you use PackageKit). 
IMHO, updating metadata periodically only makes sense in user interfaces 
that are actually offering updates to the user. Updating the metadata 
eagerly just in case somebody will run the command later is a bad idea.

For the GNOME Software case: IMHO, downloading entire packages in the 
background without asking, before even telling the user that there are 
updates at all, is totally unacceptable (on ANY type of Internet 
connection), and I am really glad that plasma-pk-updates does not do such a 
boneheaded thing. (I do think that updating metadata to offer updates is a 
helpful thing on an unmetered connection, but you are right that there needs 
to be a good way to avoid doing so without an explicit request on metered 
connections. I also don't think that the metered/unmetered property can 
reliably be autodetected as GNOME Software tries to do.)

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Kevin Kofler]

Theodore Papadopoulo wrote:
> Add to this that these caches seem to be never cleaned, so that they
> grow up very large  up to the point they prevent updating the system. I
> just found that the packagekit cache on my machine is about 7Gb !!!
> Probably because I do use packageit at all (dnf cache is much smaller at
> ~700 Mb).

IMHO, this is a feature, and the DNF default behavior is the bug, a critical 
data loss bug even! Given that Fedora does NOT keep old updates on its 
mirrors, the only convenient way to revert a broken update is to keep ALL 
old updates cached locally. Otherwise, you have to hunt down the old builds 
directly in Koji. The version from the GA release is more often than not TOO 
old, you really want the previous update.

I consider it very broken to delete files that I might still need without 
even asking me. Setting keepcache=1 is the one of the very first things I do 
to any new Fedora installation.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Honza Silhan]

On Thu, Nov 3, 2016 at 10:13 AM, Gerd Hoffmann  wrote:
>   Hi,
>
>> > The metalinks already provide checksums and timestamps for the
>> > metadata. So instead of dumbly going out and re-downloading the entire
>> > metadata at hardcoded intervals, couldn't we rather just check if the
>> > 'latest' metadata (according to the metalink) has changed since the
>> > last time we ran, or something along those lines, and only actually
>> > download the metadata if so? Wouldn't that save us a lot of wasted
>> > metadata downloads?
>>
>> I suppose that's what is supposed to happen. And of course, now that I
>> try to reproduce it, that is exactly what happens.
>>
>> I know I've seen it redownload the main repo occasionally but perhaps
>> its just been bugs since fixed. Or then the main repo actually changed.
>> But perhaps having bugs is the more likely explanation...

Yeah, it was bug in DNF handling metalinks. It should have been fixed.

> dnf seems to reload the complete metadata if you touch the repo file.
> I see this happening after running fedrepos (to configure mirror and
> proxy) on fresh installs.  Not that it bothers me much in that specific
> case as it re-downloads the metadata from local squid cache.  But just
> re-downloading repomd.xml surely should have been enough.

Is this fedrepos changing URL and/or type of URL (metalink, baseurl or
mirrorlist)?


Honza
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Gerd Hoffmann]

  Hi,

> > The metalinks already provide checksums and timestamps for the
> > metadata. So instead of dumbly going out and re-downloading the entire
> > metadata at hardcoded intervals, couldn't we rather just check if the
> > 'latest' metadata (according to the metalink) has changed since the
> > last time we ran, or something along those lines, and only actually
> > download the metadata if so? Wouldn't that save us a lot of wasted
> > metadata downloads?
> 
> I suppose that's what is supposed to happen. And of course, now that I 
> try to reproduce it, that is exactly what happens.
> 
> I know I've seen it redownload the main repo occasionally but perhaps 
> its just been bugs since fixed. Or then the main repo actually changed. 
> But perhaps having bugs is the more likely explanation...

dnf seems to reload the complete metadata if you touch the repo file.
I see this happening after running fedrepos (to configure mirror and
proxy) on fresh installs.  Not that it bothers me much in that specific
case as it re-downloads the metadata from local squid cache.  But just
re-downloading repomd.xml surely should have been enough.

cheers,
  Gerd
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Panu Matilainen]

On 11/02/2016 08:15 PM, Adam Williamson wrote:

On Sun, 2016-10-30 at 08:50 +0200, Panu Matilainen wrote:

On a related note, why on earth is the main Fedora repo set to expire
every two weeks? (and its -source and -debuginfo every week??) It's not
supposed to change *ever* for a released distro version now is it?


You know, this may be me being dumb, but this question prompted me to
wonder...can't we do something better than metadata expiry and complete
re-download for repos that use the mirrormanager metalinks?

The metalinks already provide checksums and timestamps for the
metadata. So instead of dumbly going out and re-downloading the entire
metadata at hardcoded intervals, couldn't we rather just check if the
'latest' metadata (according to the metalink) has changed since the
last time we ran, or something along those lines, and only actually
download the metadata if so? Wouldn't that save us a lot of wasted
metadata downloads?


I suppose that's what is supposed to happen. And of course, now that I 
try to reproduce it, that is exactly what happens.


I know I've seen it redownload the main repo occasionally but perhaps 
its just been bugs since fixed. Or then the main repo actually changed. 
But perhaps having bugs is the more likely explanation...


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Christian Stadelmann]

Yeah, sorry, looks like either hyperkitty or I messed up.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[William Moreno]

El 2/11/2016 12:16 p. m., "Adam Williamson" 
escribió:
>
> On Sun, 2016-10-30 at 08:50 +0200, Panu Matilainen wrote:
> > On a related note, why on earth is the main Fedora repo set to expire
> > every two weeks? (and its -source and -debuginfo every week??) It's not
> > supposed to change *ever* for a released distro version now is it?
>
> You know, this may be me being dumb, but this question prompted me to
> wonder...can't we do something better than metadata expiry and complete
> re-download for repos that use the mirrormanager metalinks?
>
> The metalinks already provide checksums and timestamps for the
> metadata. So instead of dumbly going out and re-downloading the entire
> metadata at hardcoded intervals, couldn't we rather just check if the
> 'latest' metadata (according to the metalink) has changed since the
> last time we ran, or something along those lines, and only actually
> download the metadata if so? Wouldn't that save us a lot of wasted
> metadata downloads?

+1 and +1 to have dnf and packagekit sharing the same metadata

Why do not include a note in Fedora docs or release notes alerting users
about the default configuration and posible high data download and
instructions to disable background process if the user want? This could
help in the very short time.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Wed, 2016-11-02 at 10:49 +, Christian Stadelmann wrote:
> > On 10/30/2016 03:26 AM, Adam Williamson wrote:
> > 
> > Fedora updates so often that attempts to pre-download anything updates 
> > related are pointless. Chances are you
> > a) waste gobs of bandwidth downloading that changing data over and over 
> > again without ever using it
> > b) when you actually *do* use it, it's typically already outdated and 
> > the update you're looking for isn't there, so you need to force manual 
> > refresh anyway

Your quoting is incorrect. I did not write any of this.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Sun, 2016-10-30 at 08:50 +0200, Panu Matilainen wrote:
> On a related note, why on earth is the main Fedora repo set to expire 
> every two weeks? (and its -source and -debuginfo every week??) It's not 
> supposed to change *ever* for a released distro version now is it?

You know, this may be me being dumb, but this question prompted me to
wonder...can't we do something better than metadata expiry and complete
re-download for repos that use the mirrormanager metalinks?

The metalinks already provide checksums and timestamps for the
metadata. So instead of dumbly going out and re-downloading the entire
metadata at hardcoded intervals, couldn't we rather just check if the
'latest' metadata (according to the metalink) has changed since the
last time we ran, or something along those lines, and only actually
download the metadata if so? Wouldn't that save us a lot of wasted
metadata downloads?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Theodore Papadopoulo]

On 10/31/2016 05:10 PM, Adam Williamson wrote:
> On Mon, 2016-10-31 at 15:09 +, Richard Hughes wrote:
>>
>> I think this kind of issue is really fixed the hard way, i.e. fixing
>> bugs and adding unimplemented features rather than just adding complex
>> UI workarounds.
> 
> I think making it work as best as possible without interaction is
> great, but I'd argue that fundamentally this needs interaction. At a
> bare minimum the UI for users to mark connections as metered or
> unmetered needs to be there. Is there actually UI for this already? I
> could not find it.
> 
> The question of what the default state should be is an interesting one,
> and I bet your take on it differs substantially depending on your level
> of 'internet connection privilege'. ;)

While all of this is true, let's not forget that having two completely
different mechanisms downloading the same information in cache is the
really bad thing...





smime.p7s
Description: S/MIME Cryptographic Signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Christian Stadelmann]

> On 10/30/2016 03:26 AM, Adam Williamson wrote:
> 
> Fedora updates so often that attempts to pre-download anything updates 
> related are pointless. Chances are you
> a) waste gobs of bandwidth downloading that changing data over and over 
> again without ever using it
> b) when you actually *do* use it, it's typically already outdated and 
> the update you're looking for isn't there, so you need to force manual 
> refresh anyway

From my experience, this is just plain wrong. Even with updates-testing enabled 
there is hardly any package getting updated twice a week. And updating your OS 
less than once a week is careless if you run it every single or second day.

So, no, automatic update downloads are the only right thing to do if you can't 
guarantee having no security bugs.

> On a related note, why on earth is the main Fedora repo set to expire every 
> two weeks? (and its -source and -debuginfo every week??) It's not supposed to 
> change *ever* for a released distro version now is it?

+1
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Honza Silhan]

On Tue, Nov 1, 2016 at 9:59 AM, Miroslav Suchý  wrote:
> Dne 31.10.2016 v 12:55 Tom Hughes napsal(a):
>> The problem, as I believe has been repeatedly explained, is that you don't 
>> really have any idea whether the connection
>> is metered. Yes you may be trying to guess by considering things like 3G 
>> connections as metered but that is an extremely
>> poor proxy for whether a connection is metered.
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1112230

Then new DNF config option ~sync_metadata_unmetred_conn_only, set to
True by default could be a solution. I am just skeptical about DNF
depending on NetworkManager.

In the long term we would like to share cache between PackageKit and
DNF + download just metadata changes [1] which is the root cause and
should reduce data transfer in general.


Honza

[1] 
https://github.com/rh-lab-q/deltametadata-prototype/wiki/Deltametadata-of-repo-md-files
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

NetworkManager's Metered property (was: Re: DNF and PackageKit background data usage)

[Thomas Haller]

On Mon, 2016-10-31 at 12:31 -0400, Matthew Miller wrote:
> On Mon, Oct 31, 2016 at 09:23:06AM -0700, Adam Williamson wrote:
> > 
> > For that matter, I'm fairly sure I've seen background update
> > downloading happen when I was using an Android wifi tether
> > connection. I'm pretty sure I remember it blowing my data cap one
> > month when I was using my laptop on the bus.
> 
> As far as I can see, the default is "unknown". You can set
> "CONNECTION_METERED=yes" in the
> /etc/sysconfig/network-scripts/ifcfg-[whatever] file to change it on
> a
> per-device basis.
> 
> The man page for NetworkManager.conf doesn't document this as one of
> the properties which can have its default overridden (and says that
> anything not documented can't be overridden). I took that at its
> word;
> possibly worth testing anyway. :)


Hi,


In NetworkManager this works as follows:


(1) There is the per-connection property "connection.metered" [1]. It
can be "yes", "no", "unknown". "unknown" is the default and prompts NM
to do some basic heuristics.

 $ nmcli -f connection.metered connection show $NAME
 $ nmcli connection update $NAME connection.metered yes


(2) Then there is the Device's metered property in the D-Bus API [2].
If you activate a connection that has "connection.metered" explicitly
set to "yes" or "no", that's it.
Otherwise, NM will set the device property to "guess-yes" or "guess-
no". For example,
  - WWAN connections are "guess-yes"
  - if there is a DHCP option ANDROID_METERED, it is "guess-yes".
See [4].

  $ nmcli -f GENERAL.METERED device show $DEVICE


(3) Finally, there is the Manager's metered property [3]. This combines
the metered property of multiple devices into one. PackageKit would
only care about this

  $ busctl get-property org.freedesktop.NetworkManager 
/org/freedesktop/NetworkManager org.freedesktop.NetworkManager Metered



best,
Thomas


[1] 
https://developer.gnome.org/NetworkManager/stable/nm-settings.html#nm-settings.property.connection.metered
[2] 
https://developer.gnome.org/NetworkManager/stable/gdbus-org.freedesktop.NetworkManager.Device.html#gdbus-property-org-freedesktop-NetworkManager-Device.Metered
[3] 
https://developer.gnome.org/NetworkManager/stable/gdbus-org.freedesktop.NetworkManager.html#gdbus-property-org-freedesktop-NetworkManager.Metered
[4] 
https://cgit.freedesktop.org/NetworkManager/NetworkManager/tree/src/devices/nm-device.c?id=58482a8feccc58db8d5a7eae9a004ce691e0c502#n10272


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Miroslav Suchý]

Dne 31.10.2016 v 12:55 Tom Hughes napsal(a):
> The problem, as I believe has been repeatedly explained, is that you don't 
> really have any idea whether the connection
> is metered. Yes you may be trying to guess by considering things like 3G 
> connections as metered but that is an extremely
> poor proxy for whether a connection is metered.

https://bugzilla.redhat.com/show_bug.cgi?id=1112230

-- 
Miroslav Suchy, RHCA
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Michael Catanzaro]

On Mon, 2016-10-31 at 18:57 -0700, Adam Williamson wrote:
> Huh, since F23 or so I've found the refresh button to be pretty
> reliable. It does seem to actually force Software to go refresh the
> metadata and download available updates, now. I think in the past it
> just 'forced' a run of the refresh timer, which doesn't always
> actually
> go and check for updates, it has a bunch of conditionals that decide
> whether it will. But that got changed at some point.

The problem is that if the update fails for any reason, it reports that
everything is up to date, last check for updates at the time the update
failed. It is https://bugzilla.gnome.org/show_bug.cgi?id=763566. Note
that bug was reported with F23, you can tell by the desktop background
in my screenshot. It's still broken today I think.

In practice, updates quite often fail for me. I think PackageKit gives
up if a mirror is missing a package, which is not an infrequent
occurrence, whereas dnf just moves on to the next mirror. (Or maybe
that got fixed?) I've seen a post-install update fail five times in a
row before. :(

Michael
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Mon, 2016-10-31 at 19:57 -0500, Michael Catanzaro wrote:

> I think it's a GNOME Software bug that it says up to date when it's
> not. I've complained about that for years. It will happily tell you
> that your freshly-installed Fedora system is up to date months after
> release, before the first update, often even after you manually check
> for updates by pressing the Refresh button. :)

Huh, since F23 or so I've found the refresh button to be pretty
reliable. It does seem to actually force Software to go refresh the
metadata and download available updates, now. I think in the past it
just 'forced' a run of the refresh timer, which doesn't always actually
go and check for updates, it has a bunch of conditionals that decide
whether it will. But that got changed at some point.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Michael Catanzaro]

On Mon, 2016-10-31 at 15:35 -0600, Chris Murphy wrote:
> The other thing I'm seeing is, I'll get a notification for software
> updates, click on it, see the Restart & Install blue button in GNOME
> Software, close/quite GNOME Software, and at some later time relaunch
> GNOME Software and it says everything is up to date even though the
> offline update wasn't run. And if I refresh, it appears to be
> downloading a lot of data all over again - I just don't know what and
> have no good way to troubleshoot this, but the refresh is taking a
> long time, maybe 30 minutes.

I think any time you run a dnf command, the prepared update gets
invalidated. (That may be an oversimplification.) I don't think it
should be downloading unchanged packages again, though.

I think it's a GNOME Software bug that it says up to date when it's
not. I've complained about that for years. It will happily tell you
that your freshly-installed Fedora system is up to date months after
release, before the first update, often even after you manually check
for updates by pressing the Refresh button. :)

Michael
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Rafal Luzynski]

31.10.2016 22:35 Chris Murphy  wrote:
> [...] And if I refresh, it appears to be
> downloading a lot of data all over again - I just don't know what and
> have no good way to troubleshoot this, but the refresh is taking a
> long time, maybe 30 minutes.

That's definitely not the answer for the end users but 'pkmon'
command may at least tell you what it's downloading and what's
the progress.

Regards,

Rafal
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Chris Murphy]

On Mon, Oct 31, 2016 at 3:35 PM, Chris Murphy  wrote:
> On Mon, Oct 31, 2016 at 5:02 AM, Michael Catanzaro  
> wrote:
>> On Sun, 2016-10-30 at 22:50 -0600, Chris Murphy wrote:
>>> Since offline updates are the default, and packagekit downloads
>>> everything currently needing updating, if the user doesn't ever do a
>>> Restart & Install to proceed with offline updates, i.e. they only
>>> ever
>>> use dnf for updates and never disable packagekit updates, I'm pretty
>>> sure the expected result is it just accumulates all of these
>>> unapplied
>>> updates.
>>
>> A 7 GB cache cannot be the expected result. PackageKit can only have
>> one update prepared at a time, so it should only cache one prepared
>> update at a time and should clear the cache before preparing the next
>> one. Otherwise it's just a cache leak.
>
> Then there must be a leak as I have multiple rpms with slightly
> different versions.
>
> https://paste.fedoraproject.org/466976/
>
> The other thing I'm seeing is, I'll get a notification for software
> updates, click on it, see the Restart & Install blue button in GNOME
> Software, close/quite GNOME Software, and at some later time relaunch
> GNOME Software and it says everything is up to date even though the
> offline update wasn't run. And if I refresh, it appears to be
> downloading a lot of data all over again - I just don't know what and
> have no good way to troubleshoot this, but the refresh is taking a
> long time, maybe 30 minutes.

After refreshing, and choosing Restart  & Install, I get an offline
update. Following reboot from that, all the RPMs that were in
/var/cache/PackageKit/25/metadata/updates-testing/packages before the
update are still in that path.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Chris Murphy]

On Mon, Oct 31, 2016 at 5:02 AM, Michael Catanzaro  wrote:
> On Sun, 2016-10-30 at 22:50 -0600, Chris Murphy wrote:
>> Since offline updates are the default, and packagekit downloads
>> everything currently needing updating, if the user doesn't ever do a
>> Restart & Install to proceed with offline updates, i.e. they only
>> ever
>> use dnf for updates and never disable packagekit updates, I'm pretty
>> sure the expected result is it just accumulates all of these
>> unapplied
>> updates.
>
> A 7 GB cache cannot be the expected result. PackageKit can only have
> one update prepared at a time, so it should only cache one prepared
> update at a time and should clear the cache before preparing the next
> one. Otherwise it's just a cache leak.

Then there must be a leak as I have multiple rpms with slightly
different versions.

https://paste.fedoraproject.org/466976/

The other thing I'm seeing is, I'll get a notification for software
updates, click on it, see the Restart & Install blue button in GNOME
Software, close/quite GNOME Software, and at some later time relaunch
GNOME Software and it says everything is up to date even though the
offline update wasn't run. And if I refresh, it appears to be
downloading a lot of data all over again - I just don't know what and
have no good way to troubleshoot this, but the refresh is taking a
long time, maybe 30 minutes.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Simo Sorce]

On Mon, 2016-10-31 at 11:54 -0500, Michael Catanzaro wrote:
> On Mon, 2016-10-31 at 18:15 +0200, Alexander Ploumistos wrote:
> > This is more of an issue for Workstation and the spins destined for
> > desktop use, so any possible solution needs to rely on
> > Anaconda/Initial Setup and each DE.
> 
> My $0.02 as a sometimes contributor to gnome-initial-setup: it already
> has too many options that can be changed post-installation. Users
> report confusion about how to answer the existing, very simple options
> (enable geolocation, enable automatic crash reporting). We don't want
> to add any more. Especially as this option needs to be network-
> specific. I do think we need a graphical option somewhere, but the
> right place is the Network system settings panel.

This is something that needs to be changed after install anyway, it
should be a setting that is available via network manager or similar as
it depends on the actual interface use.

I also experienced my smartphone tethered connection being clogged with
PkgKit trying to do updates, luckily for me T-Mobile has unlimited data
even on roaming ... or the bill could have been astronomical.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Michael Catanzaro]

On Mon, 2016-10-31 at 18:15 +0200, Alexander Ploumistos wrote:
> This is more of an issue for Workstation and the spins destined for
> desktop use, so any possible solution needs to rely on
> Anaconda/Initial Setup and each DE.

My $0.02 as a sometimes contributor to gnome-initial-setup: it already
has too many options that can be changed post-installation. Users
report confusion about how to answer the existing, very simple options
(enable geolocation, enable automatic crash reporting). We don't want
to add any more. Especially as this option needs to be network-
specific. I do think we need a graphical option somewhere, but the
right place is the Network system settings panel.

Michael
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Mon, 2016-10-31 at 12:25 -0400, Bastien Nocera wrote:

> Otherwise your description doesn't seem to square with the fact that
> > most everyone sees background update downloads OOTB.
> > 
> > Even still, in fact, kparal mentioned them happening on hotel wifi, so
> > why would that be the case if the default is 'metered'?
> 
> Looks like I switched metered and unmetered around in my description ;)
> 
> They're unmetered unless you either tag them as metered, or something
> (like a DHCP server) tells NM they're metered.
> 
> Sorry about that.

Ah, indeed, that lines up better with the observed data =) Still, it's
interesting that the DHCP metered indication thing doesn't seem to be
working, per my recall and kparal's nmcli output?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Matthew Miller]

On Mon, Oct 31, 2016 at 09:23:06AM -0700, Adam Williamson wrote:
> For that matter, I'm fairly sure I've seen background update
> downloading happen when I was using an Android wifi tether
> connection. I'm pretty sure I remember it blowing my data cap one
> month when I was using my laptop on the bus.

As far as I can see, the default is "unknown". You can set
"CONNECTION_METERED=yes" in the
/etc/sysconfig/network-scripts/ifcfg-[whatever] file to change it on a
per-device basis.

The man page for NetworkManager.conf doesn't document this as one of
the properties which can have its default overridden (and says that
anything not documented can't be overridden). I took that at its word;
possibly worth testing anyway. :)

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Bastien Nocera]

No, there's no UI:
https://bugzilla.gnome.org/show_bug.cgi?id=745747

- Original Message -
> On Mon, 2016-10-31 at 15:09 +, Richard Hughes wrote:
> > 
> > I think this kind of issue is really fixed the hard way, i.e. fixing
> > bugs and adding unimplemented features rather than just adding complex
> > UI workarounds.
> 
> I think making it work as best as possible without interaction is
> great, but I'd argue that fundamentally this needs interaction. At a
> bare minimum the UI for users to mark connections as metered or
> unmetered needs to be there. Is there actually UI for this already? I
> could not find it.
> 
> The question of what the default state should be is an interesting one,
> and I bet your take on it differs substantially depending on your level
> of 'internet connection privilege'. ;)
> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Bastien Nocera]

- Original Message -
> On Mon, 2016-10-31 at 08:18 -0400, Bastien Nocera wrote:
> 
> > > How does a connection become "unmetered"? It can't just be on interface
> > > type,
> > > as I have metered connections on all interface types, so presumably you
> > > use
> > > some form of web service to distinguish "metered" from "unmetered" based
> > > on
> > > a list of known IP blocks?
> > > 
> > > Or do you simply assume that all connections are metered until the user
> > > says
> > > otherwise?
> > 
> > They're metered unless you either tag them as unmetered, or hints are
> > provided
> > to NetworkManager by what you're connected to. For example, Android
> > tethering
> > is automatically tagged as metered as Android provides a hint in its DHCP
> > configuration.
> 
> Does NetworkManager 'hint' that wired connections are unmetered?

It doesn't say "definitely metered" or "definitely unmetered". We usually take
that to mean it's unmetered.

> Otherwise your description doesn't seem to square with the fact that
> most everyone sees background update downloads OOTB.
> 
> Even still, in fact, kparal mentioned them happening on hotel wifi, so
> why would that be the case if the default is 'metered'?

Looks like I switched metered and unmetered around in my description ;)

They're unmetered unless you either tag them as metered, or something
(like a DHCP server) tells NM they're metered.

Sorry about that.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Mon, 2016-10-31 at 08:18 -0400, Bastien Nocera wrote:

> > How does a connection become "unmetered"? It can't just be on interface 
> > type,
> > as I have metered connections on all interface types, so presumably you use
> > some form of web service to distinguish "metered" from "unmetered" based on
> > a list of known IP blocks?
> > 
> > Or do you simply assume that all connections are metered until the user says
> > otherwise?
> 
> They're metered unless you either tag them as unmetered, or hints are provided
> to NetworkManager by what you're connected to. For example, Android tethering
> is automatically tagged as metered as Android provides a hint in its DHCP
> configuration.

Does NetworkManager 'hint' that wired connections are unmetered?
Otherwise your description doesn't seem to square with the fact that
most everyone sees background update downloads OOTB.

Even still, in fact, kparal mentioned them happening on hotel wifi, so
why would that be the case if the default is 'metered'?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Matthew Miller]

On Mon, Oct 31, 2016 at 09:06:50AM -0700, Adam Williamson wrote:
> One of the bugs has a note I was not aware of before: there's
> actually a DHCP value that can be sent by the server to denote a
> metered connection. If that's actually widely respected, and set by
> phone wifi AP applications and the like, it could be pretty accurate.
> Not perfect, but better than heuristics. I've no idea how widely it's
> used in the real world, though.

For what it's worth, with wifi hotspot on my Nexus 6P running latest
Android N:

$ nmcli -f connection.metered connection show myhotspotssid
connection.metered: unknown


-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Mon, 2016-10-31 at 09:13 -0700, Adam Williamson wrote:
> On Mon, 2016-10-31 at 08:18 -0400, Bastien Nocera wrote:
> 
> > > How does a connection become "unmetered"? It can't just be on interface 
> > > type,
> > > as I have metered connections on all interface types, so presumably you 
> > > use
> > > some form of web service to distinguish "metered" from "unmetered" based 
> > > on
> > > a list of known IP blocks?
> > > 
> > > Or do you simply assume that all connections are metered until the user 
> > > says
> > > otherwise?
> > 
> > They're metered unless you either tag them as unmetered, or hints are 
> > provided
> > to NetworkManager by what you're connected to. For example, Android 
> > tethering
> > is automatically tagged as metered as Android provides a hint in its DHCP
> > configuration.
> 
> Does NetworkManager 'hint' that wired connections are unmetered?
> Otherwise your description doesn't seem to square with the fact that
> most everyone sees background update downloads OOTB.
> 
> Even still, in fact, kparal mentioned them happening on hotel wifi, so
> why would that be the case if the default is 'metered'?

For that matter, I'm fairly sure I've seen background update
downloading happen when I was using an Android wifi tether connection.
I'm pretty sure I remember it blowing my data cap one month when I was
using my laptop on the bus.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Mon, 2016-10-31 at 15:09 +, Richard Hughes wrote:
> 
> I think this kind of issue is really fixed the hard way, i.e. fixing
> bugs and adding unimplemented features rather than just adding complex
> UI workarounds.

I think making it work as best as possible without interaction is
great, but I'd argue that fundamentally this needs interaction. At a
bare minimum the UI for users to mark connections as metered or
unmetered needs to be there. Is there actually UI for this already? I
could not find it.

The question of what the default state should be is an interesting one,
and I bet your take on it differs substantially depending on your level
of 'internet connection privilege'. ;)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Alexander Ploumistos]

Hello,

I'm not sure I have all my facts straight -and by all means, please
help straighten them out-, but I would like to address Adam's original
proposals.

This is more of an issue for Workstation and the spins destined for
desktop use, so any possible solution needs to rely on
Anaconda/Initial Setup and each DE. In essence, we need a script that
runs with elevated privileges and disables/enables the makecache timer
and Packagit. This could then be wrapped in zenity or whatever similar
works in every DE. I think this would work regardless of
NetworkManager's presence, metered connection detection etc..

Would it be unfeasible to craft an Initial Setup add-on that runs
after the EULA screen, displays a notification explaining background
data usage and allows the user to turn it off (temporarily)? Since
having updates turned off indefinitely is a Bad Thing™, this add-on
could spawn a systemd service that checks if the makecache timer and
PackageKit have been disabled and if so, throw a nag screen (which
could be disabled, but not too easily ;) ) every n minutes/hours/days.
After the first run, every DE could have each own (control) panel
item/button/widget to enable toggling automatic updates on and off.

All of the above would require a) the Initial Setup add-on, b) a
systemd service and/or timer and c) a DE-specific package to alter the
state of things, which would have to be among the default packages in
each DE group. Having looked at the code of yumex, it doesn't look
that difficult to adapt the toggle to work with any underlying package
management system, at least the mainstream ones. Perhaps each DE could
enhance it even further, e.g. GNOME could integrate
gnome-online-accounts, so users could selectively disable specific
sync activities or any background data-consuming jobs altogether.

And now that I re-read what I have written, is this work that can be
done in Fedora or would we have to work with upstream from the get-go?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Adam Williamson]

On Mon, 2016-10-31 at 11:55 +, Tom Hughes wrote:
> On 31/10/16 11:41, Richard Hughes wrote:
> > On 30 October 2016 at 01:26, Adam Williamson  
> > wrote:
> > > 1) Both dnf and GNOME Software / PackageKit default to performing
> > > fairly data-hungry transactions in the background, out of the box,
> > > without telling you about it. GNOME's is particularly bad, as it will
> > > happily download available updates in the background, which can be
> > > gigabytes worth of data.
> > 
> > If you're on an "unmetered" connection type...
> 
> The problem, as I believe has been repeatedly explained, is that you 
> don't really have any idea whether the connection is metered. Yes you 
> may be trying to guess by considering things like 3G connections as 
> metered but that is an extremely poor proxy for whether a connection is 
> metered.
> 
> It's not something that can be tied to connection types anyway. A wifi 
> interface might be unmetered if I'm at home but metered if I'm in a cafe 
> or hotel. An ethernet interface might be metered at certain times of day 
> and not at others.
> 
> The idea that you can programatically determine, using a simple 
> heuristic, if a connection will be metered is just nonsense.

One of the bugs has a note I was not aware of before: there's actually
a DHCP value that can be sent by the server to denote a metered
connection. If that's actually widely respected, and set by phone wifi
AP applications and the like, it could be pretty accurate. Not perfect,
but better than heuristics. I've no idea how widely it's used in the
real world, though.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Richard Hughes]

On 31 October 2016 at 14:56, Bastien Nocera  wrote:
> I'd really like the kernel to do QoS on the user's own connections. We can 
> know
> whether downloads are interactive or not, so there is metadata available
> to make this better, and not cripple interactive downloads while background
> downloads are ongoing.

We actually do this with PackageKit. It used to work when we were
using yum, but I don't think the libdnf code does any kind of
throttling. Certainly PackageKit knows which transactions are
background and non-interactive, so it certainly makes sense to dribble
those down slowly.

I think this kind of issue is really fixed the hard way, i.e. fixing
bugs and adding unimplemented features rather than just adding complex
UI workarounds.

Richard.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Bastien Nocera]

- Original Message -
> > On 30 October 2016 at 01:26, Adam Williamson 
> > wrote:
> > > 1) Both dnf and GNOME Software / PackageKit default to performing
> > > fairly data-hungry transactions in the background, out of the box,
> > > without telling you about it. GNOME's is particularly bad, as it will
> > > happily download available updates in the background, which can be
> > > gigabytes worth of data.
> > 
> > If you're on an "unmetered" connection type...
> 
> This can be problematic even on an unmetered connection. An anecdotal
> experience: A few months back I was on a hotel wifi, I vitally needed some
> information quick, and the wifi simply didn't work - all web pages timed
> out. I was very disgruntled about a crappy hotel wifi (that used to work the
> day before), when in 5-10 minutes, I saw "Your updates were downloaded and
> are ready to install" popup. Then I realized... tried the web browser and
> web pages loaded normally. The wifi connection was so slow that while
> PackageKit was downloading updates in the background, I couldn't access the
> web at all.
> 
> My poor experience stemmed from:
> a) not being informed that updates were being downloaded in the background -
> so I assumed the problem was elsewhere
> b) not being able to pause/abort background downloads - even if I had
> realized/figured out PackageKit was hogging the network, there'd have been
> no way to stop the downloads (certainly no user accessible one, and even
> when I tried to kill the process some time in the past, it just kept
> respawning)
> 
> You can disregard this as a "slow hotel wifi problem only", but I live in a
> block of flats, the air is jammed with 20-30 wifi networks all around me,
> and I experience a similar situation (though not that severe) from time to
> time even at my home, a few meters from the AP - one full speed download can
> completely kill any other (my own) network traffic. Again, this would not be
> a problem if I a) knew about it b) could stop it.

I'd really like the kernel to do QoS on the user's own connections. We can know
whether downloads are interactive or not, so there is metadata available
to make this better, and not cripple interactive downloads while background
downloads are ongoing.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Bill Peck]

On Mon, 2016-10-31 at 10:32 -0400, Kamil Paral wrote:
> > 
> > On 30 October 2016 at 01:26, Adam Williamson  > t.org>
> > wrote:
> > > 
> > > 1) Both dnf and GNOME Software / PackageKit default to performing
> > > fairly data-hungry transactions in the background, out of the
> > > box,
> > > without telling you about it. GNOME's is particularly bad, as it
> > > will
> > > happily download available updates in the background, which can
> > > be
> > > gigabytes worth of data.
> > 
> > If you're on an "unmetered" connection type...
> 
> This can be problematic even on an unmetered connection. An anecdotal
> experience: A few months back I was on a hotel wifi, I vitally needed
> some information quick, and the wifi simply didn't work - all web
> pages timed out. I was very disgruntled about a crappy hotel wifi
> (that used to work the day before), when in 5-10 minutes, I saw "Your
> updates were downloaded and are ready to install" popup. Then I
> realized... tried the web browser and web pages loaded normally. The
> wifi connection was so slow that while PackageKit was downloading
> updates in the background, I couldn't access the web at all.
> 
> My poor experience stemmed from:
> a) not being informed that updates were being downloaded in the
> background - so I assumed the problem was elsewhere
> b) not being able to pause/abort background downloads - even if I had
> realized/figured out PackageKit was hogging the network, there'd have
> been no way to stop the downloads (certainly no user accessible one,
> and even when I tried to kill the process some time in the past, it
> just kept respawning)
> 
> You can disregard this as a "slow hotel wifi problem only", but I
> live in a block of flats, the air is jammed with 20-30 wifi networks
> all around me, and I experience a similar situation (though not that
> severe) from time to time even at my home, a few meters from the AP -
> one full speed download can completely kill any other (my own)
> network traffic. Again, this would not be a problem if I a) knew
> about it b) could stop it.


Some sort of notification stating that it is happening with a
[pause/cancel/never do this] button so you could control this in the
moment.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Kamil Paral]

> On 30 October 2016 at 01:26, Adam Williamson 
> wrote:
> > 1) Both dnf and GNOME Software / PackageKit default to performing
> > fairly data-hungry transactions in the background, out of the box,
> > without telling you about it. GNOME's is particularly bad, as it will
> > happily download available updates in the background, which can be
> > gigabytes worth of data.
> 
> If you're on an "unmetered" connection type...

This can be problematic even on an unmetered connection. An anecdotal 
experience: A few months back I was on a hotel wifi, I vitally needed some 
information quick, and the wifi simply didn't work - all web pages timed out. I 
was very disgruntled about a crappy hotel wifi (that used to work the day 
before), when in 5-10 minutes, I saw "Your updates were downloaded and are 
ready to install" popup. Then I realized... tried the web browser and web pages 
loaded normally. The wifi connection was so slow that while PackageKit was 
downloading updates in the background, I couldn't access the web at all.

My poor experience stemmed from:
a) not being informed that updates were being downloaded in the background - so 
I assumed the problem was elsewhere
b) not being able to pause/abort background downloads - even if I had 
realized/figured out PackageKit was hogging the network, there'd have been no 
way to stop the downloads (certainly no user accessible one, and even when I 
tried to kill the process some time in the past, it just kept respawning)

You can disregard this as a "slow hotel wifi problem only", but I live in a 
block of flats, the air is jammed with 20-30 wifi networks all around me, and I 
experience a similar situation (though not that severe) from time to time even 
at my home, a few meters from the AP - one full speed download can completely 
kill any other (my own) network traffic. Again, this would not be a problem if 
I a) knew about it b) could stop it.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Matthew Miller]

On Mon, Oct 31, 2016 at 01:38:17PM +, Richard Hughes wrote:
> I guess use a dconf override to change the default for all users, e.g.
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Desktop_Migration_and_Administration_Guide/custom-default-values-system-settings.html

So in this case, a file named `/etc/dconf/db/local.d/download-updates-false`
containing:

[org/gnome/software]
download-updates=false

and then `dconf update`?

Hmmm, wait -- need to create a profile file first, I guess — the
default is just "user's personal database is consulted and there are no
system settings"... 

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Richard Hughes]

On 31 October 2016 at 13:33, Matthew Miller  wrote:
> This is per-user, right?

Right.

> How do you do it for the whole system?

I guess use a dconf override to change the default for all users, e.g.

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Desktop_Migration_and_Administration_Guide/custom-default-values-system-settings.html

Richard.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Tom Hughes]

On 31/10/16 13:29, Richard Hughes wrote:

On 31 October 2016 at 12:55, Bastien Nocera  wrote:

So how do I tag a connection as unmetered with systemd-networkd?

You can't. You get a box of bits if you start replacing parts of the
Workstation experience


Right, in this case you'd just have to do "gsettings set
org.gnome.software download-updates false" -- we can't possibly cope
with every kind of configuration possible when you start removing key
bits of infrastructure that GNOME expects to be there.


Well I long ago removed gnome-software completely from my machines 
anyway so it doesn't really bother me but really the original point in 
respect of people using NM and Gnome Software is that trying to track 
whether the connection is metered at the connection level doesn't really 
make any sense.


On top of which I wonder how many people actually know that you can mark 
a connection as metered in NetworkManager - it's not like it asks when 
you connect to a new WiFi network if it is metered or not...


The DNF cache is a separate issue of course as I'm not sure that makes 
any attempt to check for metered connections.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Matthew Miller]

On Mon, Oct 31, 2016 at 01:29:45PM +, Richard Hughes wrote:
> > You can't. You get a box of bits if you start replacing parts of the
> > Workstation experience
> Right, in this case you'd just have to do "gsettings set
> org.gnome.software download-updates false" -- we can't possibly cope
> with every kind of configuration possible when you start removing key
> bits of infrastructure that GNOME expects to be there.

This is per-user, right? How do you do it for the whole system?

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Richard Hughes]

On 31 October 2016 at 12:55, Bastien Nocera  wrote:
>> So how do I tag a connection as unmetered with systemd-networkd?
> You can't. You get a box of bits if you start replacing parts of the
> Workstation experience

Right, in this case you'd just have to do "gsettings set
org.gnome.software download-updates false" -- we can't possibly cope
with every kind of configuration possible when you start removing key
bits of infrastructure that GNOME expects to be there.

Richard
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Bastien Nocera]

- Original Message -
> On 31/10/16 12:18, Bastien Nocera wrote:
> 
> > They're metered unless you either tag them as unmetered, or hints are
> > provided
> > to NetworkManager by what you're connected to. For example, Android
> > tethering
> > is automatically tagged as metered as Android provides a hint in its DHCP
> > configuration.
> 
> What if you're not even using NetworkManager?
> 
> Only my laptop, which needs the easy UI of Network Manager to connect to
> different networks, uses NM while all my non-mobile machines just use
> systemd-networkd to manage the network.
> 
> So how do I tag a connection as unmetered with systemd-networkd?

You can't. You get a box of bits if you start replacing parts of the
Workstation experience, eg. the network status icon also won't work
as expected. That's par for the course.

> More to the point how do I tag it as unmetered at certain times of day...

Right now, you'd script it.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Tom Hughes]

On 31/10/16 12:18, Bastien Nocera wrote:


They're metered unless you either tag them as unmetered, or hints are provided
to NetworkManager by what you're connected to. For example, Android tethering
is automatically tagged as metered as Android provides a hint in its DHCP
configuration.


What if you're not even using NetworkManager?

Only my laptop, which needs the easy UI of Network Manager to connect to 
different networks, uses NM while all my non-mobile machines just use 
systemd-networkd to manage the network.


So how do I tag a connection as unmetered with systemd-networkd?

More to the point how do I tag it as unmetered at certain times of day...

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Bastien Nocera]

- Original Message -
> On 31 Oct 2016, at 11:41, Richard Hughes  wrote:
> > 
> > On 30 October 2016 at 01:26, Adam Williamson 
> > wrote:
> >> 1) Both dnf and GNOME Software / PackageKit default to performing
> >> fairly data-hungry transactions in the background, out of the box,
> >> without telling you about it. GNOME's is particularly bad, as it will
> >> happily download available updates in the background, which can be
> >> gigabytes worth of data.
> > 
> > If you're on an "unmetered" connection type...
> > 
> 
> How does a connection become "unmetered"? It can't just be on interface type,
> as I have metered connections on all interface types, so presumably you use
> some form of web service to distinguish "metered" from "unmetered" based on
> a list of known IP blocks?
> 
> Or do you simply assume that all connections are metered until the user says
> otherwise?

They're metered unless you either tag them as unmetered, or hints are provided
to NetworkManager by what you're connected to. For example, Android tethering
is automatically tagged as metered as Android provides a hint in its DHCP
configuration.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Simon Farnsworth]

On 31 Oct 2016, at 11:41, Richard Hughes  wrote:
> 
> On 30 October 2016 at 01:26, Adam Williamson  
> wrote:
>> 1) Both dnf and GNOME Software / PackageKit default to performing
>> fairly data-hungry transactions in the background, out of the box,
>> without telling you about it. GNOME's is particularly bad, as it will
>> happily download available updates in the background, which can be
>> gigabytes worth of data.
> 
> If you're on an "unmetered" connection type...
> 

How does a connection become "unmetered"? It can't just be on interface type, 
as I have metered connections on all interface types, so presumably you use 
some form of web service to distinguish "metered" from "unmetered" based on a 
list of known IP blocks?

Or do you simply assume that all connections are metered until the user says 
otherwise?

-- 
Simon Farnsworth
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Tom Hughes]

On 31/10/16 11:41, Richard Hughes wrote:

On 30 October 2016 at 01:26, Adam Williamson  wrote:

1) Both dnf and GNOME Software / PackageKit default to performing
fairly data-hungry transactions in the background, out of the box,
without telling you about it. GNOME's is particularly bad, as it will
happily download available updates in the background, which can be
gigabytes worth of data.


If you're on an "unmetered" connection type...


The problem, as I believe has been repeatedly explained, is that you 
don't really have any idea whether the connection is metered. Yes you 
may be trying to guess by considering things like 3G connections as 
metered but that is an extremely poor proxy for whether a connection is 
metered.


It's not something that can be tied to connection types anyway. A wifi 
interface might be unmetered if I'm at home but metered if I'm in a cafe 
or hotel. An ethernet interface might be metered at certain times of day 
and not at others.


The idea that you can programatically determine, using a simple 
heuristic, if a connection will be metered is just nonsense.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Richard Hughes]

On 30 October 2016 at 01:26, Adam Williamson  wrote:
> 1) Both dnf and GNOME Software / PackageKit default to performing
> fairly data-hungry transactions in the background, out of the box,
> without telling you about it. GNOME's is particularly bad, as it will
> happily download available updates in the background, which can be
> gigabytes worth of data.

If you're on an "unmetered" connection type...

> 'hey, do you want to turn on background update downloads'?

This needs to go to the GNOME upstream designers as a proposal, rather
than to a downstream list. We're actually working on supporting a mode
like this for Endless (i.e. we download the metadata, but not the
packages themselves) but I'm not sure we've talked about any UI to
configure it.

Richard.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Michael Catanzaro]

On Sun, 2016-10-30 at 22:50 -0600, Chris Murphy wrote:
> Since offline updates are the default, and packagekit downloads
> everything currently needing updating, if the user doesn't ever do a
> Restart & Install to proceed with offline updates, i.e. they only
> ever
> use dnf for updates and never disable packagekit updates, I'm pretty
> sure the expected result is it just accumulates all of these
> unapplied
> updates.

A 7 GB cache cannot be the expected result. PackageKit can only have
one update prepared at a time, so it should only cache one prepared
update at a time and should clear the cache before preparing the next
one. Otherwise it's just a cache leak.

Michael
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[jack smith]

You can add this one https://bugzilla.gnome.org/show_bug.cgi?id=768632
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Till Maas]

On Sun, Oct 30, 2016 at 10:02:43AM -0500, Michael Catanzaro wrote:
> On Sun, 2016-10-30 at 09:15 +0100, Theodore Papadopoulo wrote:
> > I
> > just found that the packagekit cache on my machine is about 7Gb !!!
> 
> This has got to be a bug, please report it.

There are several bug reports about the bad behaviour:

https://bugs.freedesktop.org/show_bug.cgi?id=80053
https://bugzilla.redhat.com/show_bug.cgi?id=1306992
https://bugzilla.redhat.com/show_bug.cgi?id=1321319

Kind regards
Till
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Chris Murphy]

On Sun, Oct 30, 2016 at 9:02 AM, Michael Catanzaro  wrote:
> On Sun, 2016-10-30 at 09:15 +0100, Theodore Papadopoulo wrote:
>> I
>> just found that the packagekit cache on my machine is about 7Gb !!!
>
> This has got to be a bug, please report it.

Since offline updates are the default, and packagekit downloads
everything currently needing updating, if the user doesn't ever do a
Restart & Install to proceed with offline updates, i.e. they only ever
use dnf for updates and never disable packagekit updates, I'm pretty
sure the expected result is it just accumulates all of these unapplied
updates.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Chuck Anderson]

On Sun, Oct 30, 2016 at 10:02:43AM -0500, Michael Catanzaro wrote:
> On Sun, 2016-10-30 at 09:15 +0100, Theodore Papadopoulo wrote:
> > I
> > just found that the packagekit cache on my machine is about 7Gb !!!
> 
> This has got to be a bug, please report it.

My packagekit cache regularly fills up / on my systems.  I almost
never use packagekit to perform updates or installs, so maybe that's
why.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Michael Catanzaro]

On Sun, 2016-10-30 at 09:15 +0100, Theodore Papadopoulo wrote:
> I
> just found that the packagekit cache on my machine is about 7Gb !!!

This has got to be a bug, please report it.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Theodore Papadopoulo]

On 10/30/2016 02:26 AM, Adam Williamson wrote:
> There's two things I think are somewhat unfortunate here:
> 
> 1) Both dnf and GNOME Software / PackageKit default to performing
> fairly data-hungry transactions in the background, out of the box,
> without telling you about it. GNOME's is particularly bad, as it will
> happily download available updates in the background, which can be
> gigabytes worth of data. DNF only updates its metadata caches (on a
> systemd timer), but even that could be behaviour that users in certain
> circumstances really really do not want.
> 
> 2) There is no particularly obvious or visible mechanism for a 'typical
> user' (or, if you prefer, many of the target 'personas' for our
> flavors) to configure this behaviour...and you have to figure out two
> completely *different* configuration mechanisms in order to shut off
> both.

Add to this that these caches seem to be never cleaned, so that they
grow up very large  up to the point they prevent updating the system. I
just found that the packagekit cache on my machine is about 7Gb !!!
Probably because I do use packageit at all (dnf cache is much smaller at
~700 Mb).

A common cache, would be very welcome if possible. Less time, less
download volume, less used space.

Theo.



smime.p7s
Description: S/MIME Cryptographic Signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Theodore Papadopoulo]

On 10/30/2016 02:26 AM, Adam Williamson wrote:

> * We could have some kind of configuration interface appear on install
> / first boot. This would require integration with anaconda and/or
> initial-setup and gnome-initial-setup.
> 
> * We could invert the defaults and have the apps ask the user if they
> want to enable data-hungry background operations on first interaction:
> the first time you use dnf (without -y) it could say 'hey, do you want
> to turn on background cache refreshes?' Similarly, the first time you
> run GNOME Software or click on an update notification, it could say
> 'hey, do you want to turn on background update downloads'?
> 
> * We could at least make both of them respect one single config setting
> for 'do data hungry background operations' (this is kinda one small
> part of the bigger issue that is 'dnf and PK-based stuff don't share
> any configuration aside from repo definitions').

The first two points are necessary but not sufficient at least for
people with varying internet connexions (laptops). Ideally, you'd want
to configure it per internet interface. Specifically, this is typically
the thng I do not want while being connected thought a smartphone (even
more when abroad).

So not only a first time configuration, but also an easy way to disable
it Your third point seems crucial.

Theo.




smime.p7s
Description: S/MIME Cryptographic Signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Panu Matilainen]

On 10/30/2016 03:26 AM, Adam Williamson wrote:

Hi folks!

I kinda hate kicking off discussions like this without having a solid
solution to propose or being able to promise to work on one, but this
really seems important. Unfortunately I can't claim I'm gonna have time
to do any concrete work on it, though I'd really *like* to. But I
thought it would be worthwhile to kick around still; perhaps someone
else will be inspired.

I just read Hedayat's review of Fedora 25 Beta:
https://hedayatvk.wordpress.com/2016/10/30/fedora-25-beta
and this really jumped out at me:

"And, if you care about your internet usage, make sure that you disable
both dnf makecache timer, and stop PackageKit from downloading updates
automatically. I don’t allow a new Fedora installation to access
internet before doing these, as it might just eat a considerable amount
of data."

There's two things I think are somewhat unfortunate here:

1) Both dnf and GNOME Software / PackageKit default to performing
fairly data-hungry transactions in the background, out of the box,
without telling you about it. GNOME's is particularly bad, as it will
happily download available updates in the background, which can be
gigabytes worth of data. DNF only updates its metadata caches (on a
systemd timer), but even that could be behaviour that users in certain
circumstances really really do not want.

2) There is no particularly obvious or visible mechanism for a 'typical
user' (or, if you prefer, many of the target 'personas' for our
flavors) to configure this behaviour...and you have to figure out two
completely *different* configuration mechanisms in order to shut off
both.

I think this is kind of poor behaviour on our part and we should make
it better. Do I have a specific concrete proposal? No. But I do have
some vague ideas.

* We could have some kind of configuration interface appear on install
/ first boot. This would require integration with anaconda and/or
initial-setup and gnome-initial-setup.

* We could invert the defaults and have the apps ask the user if they
want to enable data-hungry background operations on first interaction:
the first time you use dnf (without -y) it could say 'hey, do you want
to turn on background cache refreshes?' Similarly, the first time you
run GNOME Software or click on an update notification, it could say
'hey, do you want to turn on background update downloads'?

* We could at least make both of them respect one single config setting
for 'do data hungry background operations' (this is kinda one small
part of the bigger issue that is 'dnf and PK-based stuff don't share
any configuration aside from repo definitions').

Anyone have thoughts on this? Any DNF or Software devs want to say I'm
totally wrong and an idiot? Anyone inspired to do something more
concrete than a mailing list post? :)


Fedora updates so often that attempts to pre-download anything updates 
related are pointless. Chances are you
a) waste gobs of bandwidth downloading that changing data over and over 
again without ever using it
b) when you actually *do* use it, it's typically already outdated and 
the update you're looking for isn't there, so you need to force manual 
refresh anyway


Just turn the automatic downloads off already. It's okay to have an 
option to enable such a thing if it actually makes sense for somebody in 
some context but in Fedora context, it makes no sense whatsoever. IMNSHO.


On a related note, why on earth is the main Fedora repo set to expire 
every two weeks? (and its -source and -debuginfo every week??) It's not 
supposed to change *ever* for a released distro version now is it?


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: DNF and PackageKit background data usage

[Andrew Lutomirski]

On Sat, Oct 29, 2016 at 5:26 PM, Adam Williamson
 wrote:
> Hi folks!
>
> I kinda hate kicking off discussions like this without having a solid
> solution to propose or being able to promise to work on one, but this
> really seems important. Unfortunately I can't claim I'm gonna have time
> to do any concrete work on it, though I'd really *like* to. But I
> thought it would be worthwhile to kick around still; perhaps someone
> else will be inspired.
>
> I just read Hedayat's review of Fedora 25 Beta:
> https://hedayatvk.wordpress.com/2016/10/30/fedora-25-beta
> and this really jumped out at me:
>
> "And, if you care about your internet usage, make sure that you disable
> both dnf makecache timer, and stop PackageKit from downloading updates
> automatically. I don’t allow a new Fedora installation to access
> internet before doing these, as it might just eat a considerable amount
> of data."
>
> There's two things I think are somewhat unfortunate here:
>

I'll add:

3) dnf and PackageKit still seem to *separately* download metadata.
Could they be made to share their metadata cache and download only one
copy?  Bonus points if they could share their cache of downloaded rpm
files, too.

4) Why is the metadata so big in the first place?  As far as I can
tell, it's a small number (3?) of xml files that don't change all that
fast.

As a simple but rather silly solution, imagine if the repodata XML
files were actually in a git repository and the local machine
maintained a clone of the git repo.  Then it would download deltas and
would probably update very quickly indeed.

--Andy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

DNF and PackageKit background data usage

[Adam Williamson]

Hi folks!

I kinda hate kicking off discussions like this without having a solid
solution to propose or being able to promise to work on one, but this
really seems important. Unfortunately I can't claim I'm gonna have time
to do any concrete work on it, though I'd really *like* to. But I
thought it would be worthwhile to kick around still; perhaps someone
else will be inspired.

I just read Hedayat's review of Fedora 25 Beta:
https://hedayatvk.wordpress.com/2016/10/30/fedora-25-beta
and this really jumped out at me:

"And, if you care about your internet usage, make sure that you disable
both dnf makecache timer, and stop PackageKit from downloading updates
automatically. I don’t allow a new Fedora installation to access
internet before doing these, as it might just eat a considerable amount
of data."

There's two things I think are somewhat unfortunate here:

1) Both dnf and GNOME Software / PackageKit default to performing
fairly data-hungry transactions in the background, out of the box,
without telling you about it. GNOME's is particularly bad, as it will
happily download available updates in the background, which can be
gigabytes worth of data. DNF only updates its metadata caches (on a
systemd timer), but even that could be behaviour that users in certain
circumstances really really do not want.

2) There is no particularly obvious or visible mechanism for a 'typical
user' (or, if you prefer, many of the target 'personas' for our
flavors) to configure this behaviour...and you have to figure out two
completely *different* configuration mechanisms in order to shut off
both.

I think this is kind of poor behaviour on our part and we should make
it better. Do I have a specific concrete proposal? No. But I do have
some vague ideas.

* We could have some kind of configuration interface appear on install
/ first boot. This would require integration with anaconda and/or
initial-setup and gnome-initial-setup.

* We could invert the defaults and have the apps ask the user if they
want to enable data-hungry background operations on first interaction:
the first time you use dnf (without -y) it could say 'hey, do you want
to turn on background cache refreshes?' Similarly, the first time you
run GNOME Software or click on an update notification, it could say
'hey, do you want to turn on background update downloads'?

* We could at least make both of them respect one single config setting
for 'do data hungry background operations' (this is kinda one small
part of the bigger issue that is 'dnf and PK-based stuff don't share
any configuration aside from repo definitions').

Anyone have thoughts on this? Any DNF or Software devs want to say I'm
totally wrong and an idiot? Anyone inspired to do something more
concrete than a mailing list post? :)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org