[issue45452] Support crash tolerance feature for gdbm module

Change by Dong-hee Na : -- nosy: +vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: After discussion with Victor by using DM, I decided to provide high-level API instead of low-level APIs. - gdbm.open(filename, snapshots=(foo, bar)) will do everything at once. Regards, Dong-hee -- ___ Python trac

[issue45452] Support crash tolerance feature for gdbm module

Gregory P. Smith added the comment: The gdbm module's purpose is effectively one of just exposing the underlying C library APIs to Python as you said. Consider this a +1 in favor of exposing the new APIs in the Python gdbm module. I'm not concerned about anyone wanting these in older Pytho

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: Wow, long discussion than I expected, I wish that you don't feel uncomfortable with my opinion first :) > The main concern is that it is not clear how to use this feature, and if it > is not clear IMHO, this feature is similar usage level with gdbm.reorganize(

[issue45452] Support crash tolerance feature for gdbm module

Serhiy Storchaka added the comment: The main concern is that it is not clear how to use this feature, and if it is not clear, it will not be used. I am not even sure that it is Pythonic, because I do not know how to use it. For example, can it be used to implement transactions? How it works

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: if you think that this feature is not usable, I will let gdbms maintainers write the idea to the Python-dev mailing list if they want to enable this feature. -- ___ Python tracker

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: @serhiy what is the main concern about this feature? -- ___ Python tracker ___ ___ Python-bugs-list

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: So IMHO, those APIs are not that low-level API since they only need to create a file with 'x' flag and then calling gdbm_failure_atomic API. and if the user failed to save the file due to several accidents. They can easily restore the local file DB by using the

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: I've done my PoC in my local environment. ``` import dbm.gnu as dbm db = dbm.open('x.db', 'nx') db.gdbm_failure_atomic('even_snapshot.bin', 'odd_snapshot.bin') for k, v in zip('abcdef', 'ghijkl'): db[k] = v db.sync() db.close() ``` By doing this in local

[issue45452] Support crash tolerance feature for gdbm module

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue45452] Support crash tolerance feature for gdbm module

Serhiy Storchaka added the comment: Examples of using the new feature. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: > Please show examples. Sorry, I don't understand, do you mean caching example or recovery example? When we use rocksdb, we can get checkpoints files under local file system. And we can recover the database by using them. http://rocksdb.org/blog/2015/11/10/use-ch

[issue45452] Support crash tolerance feature for gdbm module

Serhiy Storchaka added the comment: Please show examples. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: Yeah, I agree with gdbm_latest_snapshot(), this API might be too low-level. But as I am one of the rocksdb user who directly uses rocksdb API for the production platform as caching purposes. so when I think about my and my team's rocksdb usage, gdbm_failure_atomi

[issue45452] Support crash tolerance feature for gdbm module

Serhiy Storchaka added the comment: I am interesting how these gdbm_failure_atomic() and gdbm_latest_snapshot() can be used in user code. Some real world examples. They look very low-level and requiring an additional boilerplate code to be useful if I understand it correctly. -- __

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: dbm.gdbm_failure_atomic('snapshot_path0', 'snapshot_path1') -> db.gdbm_failure_atomic('snapshot_path0', 'snapshot_path1') -- ___ Python tracker ___

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: Sorry, I don't know my answer is enough. Or do you have any ideas or the opposite ideas? -- ___ Python tracker ___ _

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: > And what's next? I may introduce following API which only available to use extension format. (I am PoC with this) > dbm.gdbm_failure_atomic('snapshot_path0', 'snapshot_path1') About gdbm_latest_snapshot(), I am still curious that Python module should provid

[issue45452] Support crash tolerance feature for gdbm module

Serhiy Storchaka added the comment: And what's next? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: > How would it be used from Python? What are scenarios? I am preparing PoC, I will share you once done. >From gdbm 1.21, gdbm provides 2 kinds of format. (Standard format / Extended >format.) To create a gdm format with the extended option, GDBM_NUMSYNC flag ne

[issue45452] Support crash tolerance feature for gdbm module

Serhiy Storchaka added the comment: How would it be used from Python? What are scenarios? -- ___ Python tracker ___ ___ Python-bugs

[issue45452] Support crash tolerance feature for gdbm module

Change by Dong-hee Na : -- keywords: +patch pull_requests: +27230 stage: -> patch review pull_request: https://github.com/python/cpython/pull/28942 ___ Python tracker ___

[issue45452] Support crash tolerance feature for gdbm module

Serhiy Storchaka added the comment: See also issue22035. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue45452] Support crash tolerance feature for gdbm module

Change by Dong-hee Na : -- title: Support crash tolerance for gdbm module -> Support crash tolerance feature for gdbm module ___ Python tracker ___ ___

[issue45452] Support crash tolerance feature for gdbm module

Dong-hee Na added the comment: FYI, I got a mail about this feature from Terence Kelly who design these amazing things :) -- ___ Python tracker ___ __