Quick question in relation to this. Is there any debug logging one can enable if one thinks that the browser is re-claiming files deleted from the temporary file system? i.e. Where the browser would log that is deleting file x because storage limit has been hit?
Thanks in advance On Thursday, August 5, 2021 at 11:18:30 PM UTC+1 Marijn Kruisselbrink wrote: > On Thu, Aug 5, 2021 at 2:41 PM Alex Russell <[email protected]> wrote: > >> Do we have stats on potential storage pressure evictions that would be >> changed as a result of this change (as that appears to be the only place >> behavior differs)? And is there any UI that we display (e.g. in >> cache/storage clearing) that will be affected? >> > Good questions. No, I don't think we have stats that would tell us > potential changes to storage pressure evictions as a result of this change. > Data stored in the persistent file system would indeed go from never being > evicted today to being evicted with all other data an origin stores. Having > said that, in more than 99.9% of the cases where we evict data, that data > has been last accessed years ago. > > As far as affected UI, the current UI isn't very good in displaying this > legacy persistent storage. I don't expect any of the storage management UI > to change, if anything getting rid of persistent quota will mean less > chance for bugs where we might inadvertently not count or include > persistent storage when displaying information about a site. The main thing > that will change UI wise is that there will no longer be a "foo.com wants > to permanently store data on your device" permission prompt that > accompanied usage of persistent quota. > > >> On Thursday, July 29, 2021 at 4:13:57 PM UTC-7 Chris Harrelson wrote: >> > LGTM2 >>> >>> On Wed, Jul 28, 2021 at 9:48 AM Yoav Weiss <[email protected]> wrote: >>> >> Thanks for clarifying! This seems like a low risk indeed. >>>> >>>> *LGTM1* to deprecate and remove (while keeping track of related issue, >>>> just in case) >>>> >>>> On Wed, Jul 28, 2021 at 6:41 PM Marijn Kruisselbrink <[email protected]> >>>> wrote: >>>> >>> To elaborate a bit more on the potential for breakage, I believe the >>>>> only way a website could be truly broken from this change is if it >>>>> somehow >>>>> relies on temporary and persistent quota being separate buckets. Not sure >>>>> what kind of logic they would have to employ to actually be broken by >>>>> that >>>>> no longer being the case though. As such my expectation is that it is >>>>> extremely unlikely that any site will be broken by this. Certainly none >>>>> of >>>>> the sites in httparchive I looked at. >>>>> >>>> On Mon, Jul 26, 2021 at 1:25 PM Marijn Kruisselbrink <[email protected]> >>>>> wrote: >>>>> >>>> On Mon, Jul 26, 2021 at 2:26 AM Yoav Weiss <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Since the 0.4% usage numbers are suspected to be a very loose >>>>>>> upper-bound, I wonder what's the best strategy here. >>>>>>> Is there a way to use-count cases where storage quota would run out >>>>>>> as temporary but not as persistent? >>>>>>> >>>>>> Good question. I think today we actually generally allow websites to >>>>>> store much more data in the temporary quota bucket than in the >>>>>> persistent >>>>>> quota bucket. The latter is capped at 10GB, while the former in our >>>>>> histograms is more than 10GB 97% of the time on desktop, and even on >>>>>> mobile >>>>>> is larger than that 80% of the time (and actual usage doesn't tend to >>>>>> get >>>>>> anywhere remotely near these amounts in the vast majority of cases). So >>>>>> even if websites that are using the persistent file system are on >>>>>> average >>>>>> using a lot more data than other websites, in the majority of cases that >>>>>> should keep working just the same way. >>>>>> >>>>>> >>>>>>> If not, are you planning to roll this out with a kill-switch while >>>>>>> monitoring filed issues for breakage? >>>>>>> >>>>>> The described approach was intentionally picked to minimize the risk >>>>>> of breakage, as such I'm fairly confident that just rolling out the >>>>>> change >>>>>> directly and relying on dev & beta coverage for issues should be safe >>>>>> enough. Having to support persistent quota for longer will make some >>>>>> other >>>>>> ongoing projects/refactoring in the area more challenging, so the sooner >>>>>> we >>>>>> could get rid of the code the better. But if you feel the risk might be >>>>>> too >>>>>> high we certainly could roll this out with a kill-switch, we'd just >>>>>> prefer >>>>>> not to. >>>>>> >>>>> On Fri, Jul 23, 2021 at 6:21 PM Marijn Kruisselbrink < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>> On Thu, Jul 22, 2021 at 11:52 PM Mike West <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hey folks! >>>>>>>>> >>>>>>>>> I'm quite supportive of deprecating the entire >>>>>>>>> `webkitRequestFileSystem` API, and this seems like a reasonable first >>>>>>>>> step. >>>>>>>>> I appreciate the data you've gathered thus far through HTTP Archive, >>>>>>>>> and >>>>>>>>> your analysis matches my suppositions: incognito detection drove much >>>>>>>>> of >>>>>>>>> the usage, and since that abuse doesn't even work anymore >>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/detail?id=1017120>, >>>>>>>>> the API serves little purpose. >>>>>>>>> >>>>>>>>> That said, 0.4% is a lot of percent. Is there a migration story >>>>>>>>> for folks who were using it in good faith? You pointed to one or two >>>>>>>>> sites >>>>>>>>> in your analysis that seemed to be doing so: would we just wipe >>>>>>>>> user's >>>>>>>>> previously-persistent data, or would it be transparently switched to >>>>>>>>> temporary storage? >>>>>>>>> >>>>>>>> >>>>>>>> It perhaps wasn't entirely clear from what we wrote (and reading >>>>>>>> the explainer/design doc again, I agree it is sort of unclear), but >>>>>>>> what >>>>>>>> we're proposing won't cause any API call that previously worked to no >>>>>>>> longer do anything. We will still (for now) support >>>>>>>> `webkitRequestFileSystem`, both with PERSISTENT and TEMPORARY options, >>>>>>>> and >>>>>>>> both those will still give you separate file systems. I.e. sites using >>>>>>>> either of both of these will keep working. What will change is the >>>>>>>> quota >>>>>>>> treatment of the PERSISTENT file system. I.e. we will start counting >>>>>>>> usage >>>>>>>> of the persistent file system against the same quota as all other >>>>>>>> storage >>>>>>>> APIs. Concretely for websites that means that >>>>>>>> `navigator.webkitPersistentStorage` will alias >>>>>>>> `navigator.webkitTemporaryStorage`, and similarly the methods on >>>>>>>> `window.webkitStorageInfo` will start ignoring the "type" parameter >>>>>>>> (as >>>>>>>> long as it is one of TEMPORARY or PERSISTENT). >>>>>>>> >>>>>>>> With this, the expectation is that the number of websites that >>>>>>>> would actually be broken by this should be much much less than 0.4%. >>>>>>>> It's >>>>>>>> hard to put an exact number on it, since we're not currently proposing >>>>>>>> to >>>>>>>> make any API not work anymore, rather we're slightly changing the >>>>>>>> behavior >>>>>>>> of some of the legacy methods. A hypothetical website that somehow >>>>>>>> relies >>>>>>>> on temporary and persistent quota being separate could be broken by >>>>>>>> this, >>>>>>>> although I'm not sure how such reliance would have to work to actually >>>>>>>> cause issues. >>>>>>>> >>>>>>>> We did consider more "aggressive" breakage of PERSISTENT, for >>>>>>>> example by making it an alias for TEMPORARY when passed to >>>>>>>> `webkitRequestFileSystem`. But as you point out, that would then >>>>>>>> require >>>>>>>> some kind of migration story, and would increase the risk of breakage >>>>>>>> of >>>>>>>> otherwise well-intentioned websites, without having significant extra >>>>>>>> benefits. I.e. merely changing how PERSISTENT is treated for quota >>>>>>>> purposes >>>>>>>> gets us all the same benefits for reducing code complexity and >>>>>>>> eliminating >>>>>>>> confusing UI. >>>>>>>> >>>>>>>> Also, I note that third-party usage of PERSISTENT is quite low, >>>>>>>>> hovering around 0.0003% ( >>>>>>>>> https://www.chromestatus.com/metrics/feature/timeline/popularity/3879). >>>>>>>>> >>>>>>>>> I wonder if there's an opportunity to simply break it in those >>>>>>>>> contexts? >>>>>>>>> Have you done any analysis on that subset of pages? >>>>>>>>> >>>>>>>> >>>>>>>> Good question, I don't think we've looked into that yet. While good >>>>>>>> for the web (in that we'd be getting rid of some portion of a >>>>>>>> chrome-only >>>>>>>> API) doing so wouldn't help us much with implementation complexity >>>>>>>> etc, so >>>>>>>> the benefits are fairly minimal. >>>>>>>> >>>>>>>> >>>>>>>>> Also also, I'm also somewhat dismayed to note that TEMPORARY usage >>>>>>>>> is at 10% of page views ( >>>>>>>>> https://www.chromestatus.com/metrics/feature/timeline/popularity/2996). >>>>>>>>> >>>>>>>>> That's high! Can I safely assume that that's overwhelmingly incognito >>>>>>>>> detection as well? If so, is there a path to removal? >>>>>>>>> >>>>>>>> >>>>>>>> A large part of that probably is incognito detection, yes. However >>>>>>>> there are a number of big websites/partners that are still relying on >>>>>>>> webkitRequestFileSystem as well. Long term we definitely would like to >>>>>>>> deprecate and remove the entire `webkitRequestFileSystem` API (and >>>>>>>> the >>>>>>>> various legacy quota APIs also involved in this intent), but >>>>>>>> realistically >>>>>>>> this is not something we expect to be able to make much headway on in >>>>>>>> the >>>>>>>> near future. On the deprecating storage APIs front we're first >>>>>>>> focussing on >>>>>>>> WebSQL, as we feel removing that has a much bigger impact on security >>>>>>>> and >>>>>>>> maintainability of the web. Getting rid of `webkitRequestFileSystem` >>>>>>>> will >>>>>>>> be nice to get to a more cross-browser interoperable web, but the vast >>>>>>>> majority of the implementation is shared with other features, so the >>>>>>>> additional maintenance burden cause by keeping it around a bit longer >>>>>>>> is >>>>>>>> fairly minimal. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> From a process perspective, I'm comfortable both with this not >>>>>>>>> requiring signals from other vendors (as they've clearly expressed >>>>>>>>> their >>>>>>>>> opinion that the spec is dead >>>>>>>>> <https://lists.w3.org/Archives/Public/public-webapps/2014AprJun/0010.html>), >>>>>>>>> >>>>>>>>> and the TAG's input is likewise unnecessary IMO. >>>>>>>>> >>>>>>>>> -mike >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Jul 22, 2021 at 11:01 PM Chris Fredrickson < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>> *Contact emails* >>>>>>>>>> >>>>>>>>>> [email protected], [email protected] >>>>>>>>>> >>>>>>>>>> *Explainer* >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://docs.google.com/document/d/1Cm2Hv_-RMqQPqruGL2rUyiTRgqMyHMJbNyNvhLH6KWU/edit >>>>>>>>>> >>>>>>>>>> *Specification* >>>>>>>>>> >>>>>>>>>> None >>>>>>>>>> >>>>>>>>>> *Summary* >>>>>>>>>> >>>>>>>>>> We plan to deprecate and remove support of the >>>>>>>>>> `window.PERSISTENT` quota type in `webkitRequestFileSystem`. >>>>>>>>>> `webkitRequestFileSystem` will still accept a `type` parameter and >>>>>>>>>> use of >>>>>>>>>> the PERSISTENT and TEMPORARY types will create filesystems with >>>>>>>>>> separate >>>>>>>>>> roots, but the PERSISTENT type will no longer grant access to a >>>>>>>>>> persistent >>>>>>>>>> filesystem. >>>>>>>>>> >>>>>>>>>> We will also modify `navigator.webkitPersistentStorage` to be an >>>>>>>>>> alias for `navigator.webkitTemporaryStorage`. >>>>>>>>>> >>>>>>>>>> Finally, we will also begin ignoring the `storageType` parameter >>>>>>>>>> (either `window.PERSISTENT` or `window.TEMPORARY`) in methods of the >>>>>>>>>> `webkitStorageInfo` API (which is deprecated). >>>>>>>>>> >>>>>>>>>> *Blink component* >>>>>>>>>> >>>>>>>>>> Blink>Storage >>>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EStorage> >>>>>>>>>> >>>>>>>>>> *Motivation* >>>>>>>>>> >>>>>>>>>> Support for the PERSISTENT quota type contributes some amount of >>>>>>>>>> complexity to the quota system, but `webkitRequestFileSystem` is the >>>>>>>>>> only >>>>>>>>>> consumer, with relatively low usage. Additionally, support for >>>>>>>>>> persistent >>>>>>>>>> quota will hinder work on storage partitioning, so it would be >>>>>>>>>> helpful to >>>>>>>>>> simplify the quota system prior to partitioning storage. >>>>>>>>>> >>>>>>>>>> *Initial public proposal* >>>>>>>>>> >>>>>>>>>> None >>>>>>>>>> >>>>>>>>>> *TAG review* >>>>>>>>>> >>>>>>>>>> None >>>>>>>>>> >>>>>>>>>> *TAG review status* >>>>>>>>>> >>>>>>>>>> Not applicable >>>>>>>>>> >>>>>>>>>> *Risks* >>>>>>>>>> >>>>>>>>>> Because we plan to just change the “lifetime” of the persistent >>>>>>>>>> filesystem (rather than, e.g., merging it with the temporary >>>>>>>>>> filesystem), >>>>>>>>>> risks are fairly minimal. Usage of the PERSISTENT type with >>>>>>>>>> `webkitRequestFileSystem` is low, about 0.4% of pageloads >>>>>>>>>> <https://www.chromestatus.com/metrics/feature/timeline/popularity/2997> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> Regarding making `navigator.webkitPersistentStorage` an alias for >>>>>>>>>> `navigator.webkitTemporaryStorage`, we believe this presents the >>>>>>>>>> least >>>>>>>>>> risk, since this way the quota for the `window.PERSISTENT` >>>>>>>>>> filesystem will >>>>>>>>>> still be accurately reported to sites that query it. >>>>>>>>>> >>>>>>>>>> Regarding ignoring `storageType`: since persistent quota will no >>>>>>>>>> longer exist, our choices are to ignore the storage type completely, >>>>>>>>>> or >>>>>>>>>> throw an exception if kPersistent is used. The former presents the >>>>>>>>>> least >>>>>>>>>> risk of breakage. >>>>>>>>>> >>>>>>>>>> *Interoperability and Compatibility* >>>>>>>>>> >>>>>>>>>> None. No other browser supports the APIs affected here. >>>>>>>>>> >>>>>>>>>> Gecko: No signal >>>>>>>>>> >>>>>>>>>> WebKit: No signal >>>>>>>>>> >>>>>>>>>> Web developers: No signals >>>>>>>>>> >>>>>>>>>> *Is this feature fully tested by web-platform-tests >>>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md>?* >>>>>>>>>> >>>>>>>>>> No >>>>>>>>>> >>>>>>>>>> *Flag name* >>>>>>>>>> >>>>>>>>>> None >>>>>>>>>> >>>>>>>>>> *Link to entry on the Chrome Platform Status* >>>>>>>>>> >>>>>>>>>> https://chromestatus.com/feature/5176235376246784 >>>>>>>>>> >>>>>>>>>> This intent message was generated by Chrome Platform Status >>>>>>>>>> <https://www.chromestatus.com/>. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "blink-dev" group. >>>>>>>>>> >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>> >>>>>>>>> >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c415cd1e-fbba-4cb6-9c83-62e7d608e04an%40chromium.org >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c415cd1e-fbba-4cb6-9c83-62e7d608e04an%40chromium.org?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "blink-dev" group. >>>>>>>>> >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>> >>>>>>>> >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3Dd_r83Lf4NDokigV%2BKCOs98b_EeJ9NcWASj1eVOD_4Mqg%40mail.gmail.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3Dd_r83Lf4NDokigV%2BKCOs98b_EeJ9NcWASj1eVOD_4Mqg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "blink-dev" group. >>>>>>>> >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>> >>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BOSsVa1KT2sc4eSJNvG9hc0LSHNc1YNJhgi7DwnqSHjZ7tvCQ%40mail.gmail.com >>>>>>>> >>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BOSsVa1KT2sc4eSJNvG9hc0LSHNc1YNJhgi7DwnqSHjZ7tvCQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "blink-dev" group. >>>> >>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfVjqW2hFpzhTzwj05P7AmvOKN%3DPFJq3pz5DwDSoqjOucw%40mail.gmail.com >>>> >>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfVjqW2hFpzhTzwj05P7AmvOKN%3DPFJq3pz5DwDSoqjOucw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/80b73287-a86b-4f15-93dd-1b2740726a3fn%40chromium.org.
