Quick question in relation to this.

Is there any debug logging one can enable if one thinks that the browser is 
re-claiming files deleted from the temporary file system?
i.e. Where the browser would log that is deleting file x because storage 
limit has been hit?

Thanks in advance



On Thursday, August 5, 2021 at 11:18:30 PM UTC+1 Marijn Kruisselbrink wrote:

> On Thu, Aug 5, 2021 at 2:41 PM Alex Russell <[email protected]> wrote:
>
>> Do we have stats on potential storage pressure evictions that would be 
>> changed as a result of this change (as that appears to be the only place 
>> behavior differs)? And is there any UI that we display (e.g. in 
>> cache/storage clearing) that will be affected?
>>
> Good questions. No, I don't think we have stats that would tell us 
> potential changes to storage pressure evictions as a result of this change. 
> Data stored in the persistent file system would indeed go from never being 
> evicted today to being evicted with all other data an origin stores. Having 
> said that, in more than 99.9% of the cases where we evict data, that data 
> has been last accessed years ago.
>
> As far as affected UI, the current UI isn't very good in displaying this 
> legacy persistent storage. I don't expect any of the storage management UI 
> to change, if anything getting rid of persistent quota will mean less 
> chance for bugs where we might inadvertently not count or include 
> persistent storage when displaying information about a site. The main thing 
> that will change UI wise is that there will no longer be a "foo.com wants 
> to permanently store data on your device" permission prompt that 
> accompanied usage of persistent quota.
>
>
>> On Thursday, July 29, 2021 at 4:13:57 PM UTC-7 Chris Harrelson wrote:
>>
> LGTM2
>>>
>>> On Wed, Jul 28, 2021 at 9:48 AM Yoav Weiss <[email protected]> wrote:
>>>
>> Thanks for clarifying! This seems like a low risk indeed.
>>>>
>>>> *LGTM1* to deprecate and remove (while keeping track of related issue, 
>>>> just in case)
>>>>
>>>> On Wed, Jul 28, 2021 at 6:41 PM Marijn Kruisselbrink <[email protected]> 
>>>> wrote:
>>>>
>>> To elaborate a bit more on the potential for breakage, I believe the 
>>>>> only way a website could be truly broken from this change is if it 
>>>>> somehow 
>>>>> relies on temporary and persistent quota being separate buckets. Not sure 
>>>>> what kind of logic they would have to employ to actually be broken by 
>>>>> that 
>>>>> no longer being the case though. As such my expectation is that it is 
>>>>> extremely unlikely that any site will be broken by this. Certainly none 
>>>>> of 
>>>>> the sites in httparchive I looked at.
>>>>>
>>>> On Mon, Jul 26, 2021 at 1:25 PM Marijn Kruisselbrink <[email protected]> 
>>>>> wrote:
>>>>>
>>>> On Mon, Jul 26, 2021 at 2:26 AM Yoav Weiss <[email protected]> 
>>>>>> wrote:
>>>>>>
>>>>>>> Since the 0.4% usage numbers are suspected to be a very loose 
>>>>>>> upper-bound, I wonder what's the best strategy here.
>>>>>>> Is there a way to use-count cases where storage quota would run out 
>>>>>>> as temporary but not as persistent? 
>>>>>>>
>>>>>> Good question. I think today we actually generally allow websites to 
>>>>>> store much more data in the temporary quota bucket than in the 
>>>>>> persistent 
>>>>>> quota bucket. The latter is capped at 10GB, while the former in our 
>>>>>> histograms is more than 10GB 97% of the time on desktop, and even on 
>>>>>> mobile 
>>>>>> is larger than that 80% of the time (and actual usage doesn't tend to 
>>>>>> get 
>>>>>> anywhere remotely near these amounts in the vast majority of cases). So 
>>>>>> even if websites that are using the persistent file system are on 
>>>>>> average 
>>>>>> using a lot more data than other websites, in the majority of cases that 
>>>>>> should keep working just the same way.
>>>>>>  
>>>>>>
>>>>>>> If not, are you planning to roll this out with a kill-switch while 
>>>>>>> monitoring filed issues for breakage?
>>>>>>>
>>>>>> The described approach was intentionally picked to minimize the risk 
>>>>>> of breakage, as such I'm fairly confident that just rolling out the 
>>>>>> change 
>>>>>> directly and relying on dev & beta coverage for issues should be safe 
>>>>>> enough. Having to support persistent quota for longer will make some 
>>>>>> other 
>>>>>> ongoing projects/refactoring in the area more challenging, so the sooner 
>>>>>> we 
>>>>>> could get rid of the code the better. But if you feel the risk might be 
>>>>>> too 
>>>>>> high we certainly could roll this out with a kill-switch, we'd just 
>>>>>> prefer 
>>>>>> not to.  
>>>>>>
>>>>> On Fri, Jul 23, 2021 at 6:21 PM Marijn Kruisselbrink <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>> On Thu, Jul 22, 2021 at 11:52 PM Mike West <[email protected]> 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey folks!
>>>>>>>>>
>>>>>>>>> I'm quite supportive of deprecating the entire 
>>>>>>>>> `webkitRequestFileSystem` API, and this seems like a reasonable first 
>>>>>>>>> step. 
>>>>>>>>> I appreciate the data you've gathered thus far through HTTP Archive, 
>>>>>>>>> and 
>>>>>>>>> your analysis matches my suppositions: incognito detection drove much 
>>>>>>>>> of 
>>>>>>>>> the usage, and since that abuse doesn't even work anymore 
>>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/detail?id=1017120>, 
>>>>>>>>> the API serves little purpose.
>>>>>>>>>
>>>>>>>>> That said, 0.4% is a lot of percent. Is there a migration story 
>>>>>>>>> for folks who were using it in good faith? You pointed to one or two 
>>>>>>>>> sites 
>>>>>>>>> in your analysis that seemed to be doing so: would we just wipe 
>>>>>>>>> user's 
>>>>>>>>> previously-persistent data, or would it be transparently switched to 
>>>>>>>>> temporary storage?
>>>>>>>>>
>>>>>>>>
>>>>>>>> It perhaps wasn't entirely clear from what we wrote (and reading 
>>>>>>>> the explainer/design doc again, I agree it is sort of unclear), but 
>>>>>>>> what 
>>>>>>>> we're proposing won't cause any API call that previously worked to no 
>>>>>>>> longer do anything. We will still (for now) support 
>>>>>>>> `webkitRequestFileSystem`, both with PERSISTENT and TEMPORARY options, 
>>>>>>>> and 
>>>>>>>> both those will still give you separate file systems. I.e. sites using 
>>>>>>>> either of both of these will keep working. What will change is the 
>>>>>>>> quota 
>>>>>>>> treatment of the PERSISTENT file system. I.e. we will start counting 
>>>>>>>> usage 
>>>>>>>> of the persistent file system against the same quota as all other 
>>>>>>>> storage 
>>>>>>>> APIs. Concretely for websites that means that 
>>>>>>>> `navigator.webkitPersistentStorage` will alias 
>>>>>>>> `navigator.webkitTemporaryStorage`, and similarly the methods on 
>>>>>>>> `window.webkitStorageInfo` will start ignoring the "type" parameter 
>>>>>>>> (as 
>>>>>>>> long as it is one of TEMPORARY or PERSISTENT).
>>>>>>>>
>>>>>>>> With this, the expectation is that the number of websites that 
>>>>>>>> would actually be broken by this should be much much less than 0.4%. 
>>>>>>>> It's 
>>>>>>>> hard to put an exact number on it, since we're not currently proposing 
>>>>>>>> to 
>>>>>>>> make any API not work anymore, rather we're slightly changing the 
>>>>>>>> behavior 
>>>>>>>> of some of the legacy methods. A hypothetical website that somehow 
>>>>>>>> relies 
>>>>>>>> on temporary and persistent quota being separate could be broken by 
>>>>>>>> this, 
>>>>>>>> although I'm not sure how such reliance would have to work to actually 
>>>>>>>> cause issues.
>>>>>>>>
>>>>>>>> We did consider more "aggressive" breakage of PERSISTENT, for 
>>>>>>>> example by making it an alias for TEMPORARY when passed to 
>>>>>>>> `webkitRequestFileSystem`. But as you point out, that would then 
>>>>>>>> require 
>>>>>>>> some kind of migration story, and would increase the risk of breakage 
>>>>>>>> of 
>>>>>>>> otherwise well-intentioned websites, without having significant extra 
>>>>>>>> benefits. I.e. merely changing how PERSISTENT is treated for quota 
>>>>>>>> purposes 
>>>>>>>> gets us all the same benefits for reducing code complexity and 
>>>>>>>> eliminating 
>>>>>>>> confusing UI.
>>>>>>>>
>>>>>>>> Also, I note that third-party usage of PERSISTENT is quite low, 
>>>>>>>>> hovering around 0.0003% (
>>>>>>>>> https://www.chromestatus.com/metrics/feature/timeline/popularity/3879).
>>>>>>>>>  
>>>>>>>>> I wonder if there's an opportunity to simply break it in those 
>>>>>>>>> contexts? 
>>>>>>>>> Have you done any analysis on that subset of pages?
>>>>>>>>>
>>>>>>>>
>>>>>>>> Good question, I don't think we've looked into that yet. While good 
>>>>>>>> for the web (in that we'd be getting rid of some portion of a 
>>>>>>>> chrome-only 
>>>>>>>> API) doing so wouldn't help us much with implementation complexity 
>>>>>>>> etc, so 
>>>>>>>> the benefits are fairly minimal.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Also also, I'm also somewhat dismayed to note that TEMPORARY usage 
>>>>>>>>> is at 10% of page views (
>>>>>>>>> https://www.chromestatus.com/metrics/feature/timeline/popularity/2996).
>>>>>>>>>  
>>>>>>>>> That's high! Can I safely assume that that's overwhelmingly incognito 
>>>>>>>>> detection as well? If so, is there a path to removal?
>>>>>>>>>
>>>>>>>>
>>>>>>>> A large part of that probably is incognito detection, yes. However 
>>>>>>>> there are a number of big websites/partners that are still relying on 
>>>>>>>> webkitRequestFileSystem as well. Long term we definitely would like to 
>>>>>>>> deprecate  and remove the entire `webkitRequestFileSystem` API (and 
>>>>>>>> the 
>>>>>>>> various legacy quota APIs also involved in this intent), but 
>>>>>>>> realistically 
>>>>>>>> this is not something we expect to be able to make much headway on in 
>>>>>>>> the 
>>>>>>>> near future. On the deprecating storage APIs front we're first 
>>>>>>>> focussing on 
>>>>>>>> WebSQL, as we feel removing that has a much bigger impact on security 
>>>>>>>> and 
>>>>>>>> maintainability of the web. Getting rid of `webkitRequestFileSystem` 
>>>>>>>> will 
>>>>>>>> be nice to get to a more cross-browser interoperable web, but the vast 
>>>>>>>> majority of the implementation is shared with other features, so the 
>>>>>>>> additional maintenance burden cause by keeping it around a bit longer 
>>>>>>>> is 
>>>>>>>> fairly minimal.
>>>>>>>>  
>>>>>>>>  
>>>>>>>>
>>>>>>> From a process perspective, I'm comfortable both with this not 
>>>>>>>>> requiring signals from other vendors (as they've clearly expressed 
>>>>>>>>> their 
>>>>>>>>> opinion that the spec is dead 
>>>>>>>>> <https://lists.w3.org/Archives/Public/public-webapps/2014AprJun/0010.html>),
>>>>>>>>>  
>>>>>>>>> and the TAG's input is likewise unnecessary IMO.
>>>>>>>>>
>>>>>>>>> -mike
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jul 22, 2021 at 11:01 PM Chris Fredrickson <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>> *Contact emails*
>>>>>>>>>>
>>>>>>>>>> [email protected], [email protected]
>>>>>>>>>>
>>>>>>>>>> *Explainer*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://docs.google.com/document/d/1Cm2Hv_-RMqQPqruGL2rUyiTRgqMyHMJbNyNvhLH6KWU/edit
>>>>>>>>>>
>>>>>>>>>> *Specification*
>>>>>>>>>>
>>>>>>>>>> None
>>>>>>>>>>
>>>>>>>>>> *Summary*
>>>>>>>>>>
>>>>>>>>>> We plan to deprecate and remove support of the 
>>>>>>>>>> `window.PERSISTENT` quota type in `webkitRequestFileSystem`. 
>>>>>>>>>> `webkitRequestFileSystem` will still accept a `type` parameter and 
>>>>>>>>>> use of 
>>>>>>>>>> the PERSISTENT and TEMPORARY types will create filesystems with 
>>>>>>>>>> separate 
>>>>>>>>>> roots, but the PERSISTENT type will no longer grant access to a 
>>>>>>>>>> persistent 
>>>>>>>>>> filesystem.
>>>>>>>>>>
>>>>>>>>>> We will also modify `navigator.webkitPersistentStorage` to be an 
>>>>>>>>>> alias for `navigator.webkitTemporaryStorage`. 
>>>>>>>>>>
>>>>>>>>>> Finally, we will also begin ignoring the `storageType` parameter 
>>>>>>>>>> (either `window.PERSISTENT` or `window.TEMPORARY`) in methods of the 
>>>>>>>>>> `webkitStorageInfo` API (which is deprecated). 
>>>>>>>>>>
>>>>>>>>>> *Blink component*
>>>>>>>>>>
>>>>>>>>>> Blink>Storage 
>>>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EStorage>
>>>>>>>>>>
>>>>>>>>>> *Motivation*
>>>>>>>>>>
>>>>>>>>>> Support for the PERSISTENT quota type contributes some amount of 
>>>>>>>>>> complexity to the quota system, but `webkitRequestFileSystem` is the 
>>>>>>>>>> only 
>>>>>>>>>> consumer, with relatively low usage. Additionally, support for 
>>>>>>>>>> persistent 
>>>>>>>>>> quota will hinder work on storage partitioning, so it would be 
>>>>>>>>>> helpful to 
>>>>>>>>>> simplify the quota system prior to partitioning storage.
>>>>>>>>>>
>>>>>>>>>> *Initial public proposal*
>>>>>>>>>>
>>>>>>>>>> None
>>>>>>>>>>
>>>>>>>>>> *TAG review*
>>>>>>>>>>
>>>>>>>>>> None
>>>>>>>>>>
>>>>>>>>>> *TAG review status*
>>>>>>>>>>
>>>>>>>>>> Not applicable
>>>>>>>>>>
>>>>>>>>>> *Risks*
>>>>>>>>>>
>>>>>>>>>> Because we plan to just change the “lifetime” of the persistent 
>>>>>>>>>> filesystem (rather than, e.g., merging it with the temporary 
>>>>>>>>>> filesystem), 
>>>>>>>>>> risks are fairly minimal. Usage of the PERSISTENT type with 
>>>>>>>>>> `webkitRequestFileSystem` is low, about 0.4% of pageloads 
>>>>>>>>>> <https://www.chromestatus.com/metrics/feature/timeline/popularity/2997>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>>> Regarding making `navigator.webkitPersistentStorage` an alias for 
>>>>>>>>>> `navigator.webkitTemporaryStorage`, we believe this presents the 
>>>>>>>>>> least 
>>>>>>>>>> risk, since this way the quota for the `window.PERSISTENT` 
>>>>>>>>>> filesystem will 
>>>>>>>>>> still be accurately reported to sites that query it.
>>>>>>>>>>
>>>>>>>>>> Regarding ignoring `storageType`: since persistent quota will no 
>>>>>>>>>> longer exist, our choices are to ignore the storage type completely, 
>>>>>>>>>> or 
>>>>>>>>>> throw an exception if kPersistent is used. The former presents the 
>>>>>>>>>> least 
>>>>>>>>>> risk of breakage.
>>>>>>>>>>
>>>>>>>>>> *Interoperability and Compatibility*
>>>>>>>>>>
>>>>>>>>>> None. No other browser supports the APIs affected here.
>>>>>>>>>>
>>>>>>>>>> Gecko: No signal
>>>>>>>>>>
>>>>>>>>>> WebKit: No signal
>>>>>>>>>>
>>>>>>>>>> Web developers: No signals
>>>>>>>>>>
>>>>>>>>>> *Is this feature fully tested by web-platform-tests 
>>>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md>?*
>>>>>>>>>>
>>>>>>>>>> No
>>>>>>>>>>
>>>>>>>>>> *Flag name*
>>>>>>>>>>
>>>>>>>>>> None
>>>>>>>>>>
>>>>>>>>>> *Link to entry on the Chrome Platform Status*
>>>>>>>>>>
>>>>>>>>>> https://chromestatus.com/feature/5176235376246784
>>>>>>>>>>
>>>>>>>>>> This intent message was generated by Chrome Platform Status 
>>>>>>>>>> <https://www.chromestatus.com/>.
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> You received this message because you are subscribed to the 
>>>>>>>>>> Google Groups "blink-dev" group.
>>>>>>>>>>
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>>>>>> send an email to [email protected].
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> To view this discussion on the web visit 
>>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c415cd1e-fbba-4cb6-9c83-62e7d608e04an%40chromium.org
>>>>>>>>>>  
>>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c415cd1e-fbba-4cb6-9c83-62e7d608e04an%40chromium.org?utm_medium=email&utm_source=footer>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> You received this message because you are subscribed to the Google 
>>>>>>>>> Groups "blink-dev" group.
>>>>>>>>>
>>>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>>>>> send an email to [email protected].
>>>>>>>>
>>>>>>>>
>>>>>>>>> To view this discussion on the web visit 
>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3Dd_r83Lf4NDokigV%2BKCOs98b_EeJ9NcWASj1eVOD_4Mqg%40mail.gmail.com
>>>>>>>>>  
>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3Dd_r83Lf4NDokigV%2BKCOs98b_EeJ9NcWASj1eVOD_4Mqg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>> -- 
>>>>>>>> You received this message because you are subscribed to the Google 
>>>>>>>> Groups "blink-dev" group.
>>>>>>>>
>>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>>>> send an email to [email protected].
>>>>>>>
>>>>>>>
>>>>>>>> To view this discussion on the web visit 
>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BOSsVa1KT2sc4eSJNvG9hc0LSHNc1YNJhgi7DwnqSHjZ7tvCQ%40mail.gmail.com
>>>>>>>>  
>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BOSsVa1KT2sc4eSJNvG9hc0LSHNc1YNJhgi7DwnqSHjZ7tvCQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>> .
>>>>>>>>
>>>>>>> -- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "blink-dev" group.
>>>>
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to [email protected].
>>>>
>>> To view this discussion on the web visit 
>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfVjqW2hFpzhTzwj05P7AmvOKN%3DPFJq3pz5DwDSoqjOucw%40mail.gmail.com
>>>>  
>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfVjqW2hFpzhTzwj05P7AmvOKN%3DPFJq3pz5DwDSoqjOucw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/80b73287-a86b-4f15-93dd-1b2740726a3fn%40chromium.org.

Reply via email to