Re: [Firebird-devel] tool for encrypting database initially (andprobably decrypting it)
03.04.2012 10:44, Vlad Khorsun wrote: To not read whole database searching for the not encrypted pages after restart i offer to store last encrypted page number at header page (also, obviously, we need to store encription state on the header such as clear, encrypted, encryption is in progress, decryption is in progress). Do you suggest to write header page after writing of every single encrypted page, thus doubling I/O?.. We can a) don't write header page immediately to disk after change encrypted page number b) change encrypted page number every N pages, where N 1 Regards, Vlad PS Of course, change of encryption state should be written to disk immediately -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] tool for encrypting database initially (andprobably decrypting it)
On 04/03/12 12:49, Dimitry Sibiryakov wrote: 03.04.2012 10:44, Vlad Khorsun wrote: To not read whole database searching for the not encrypted pages after restart i offer to store last encrypted page number at header page (also, obviously, we need to store encription state on the header such as clear, encrypted, encryption is in progress, decryption is in progress). Do you suggest to write header page after writing of every single encrypted page, thus doubling I/O?.. This may be done once per N pages (with N = 1024 for example). To continue with encryption engine searches inside that N pages for not encrypted yet. -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] tool for encrypting database initially (andprobably decrypting it)
Hello, Vlad! Tuesday, April 3, 2012, 12:44:07 PM, you wrote: VK Encryption must be resistent to the database shutdown\server restart and so on. VK Therefore it must be restartable. As we going to add encrypted flag for each page VK we can know pages that already encrypted. To not read whole database searching VK for the not encrypted pages after restart i offer to store last encrypted page number VK at header page (also, obviously, we need to store encription state on the header VK such as clear, encrypted, encryption is in progress, decryption is in progress). disagree, because - not all db pages need to be encrypted. for example PIP, TIP, etc. Performance effect of decrypting/encrypting these pages can be disasterous. - I suspect that pages will not be encrypted as a whole. Otherwise, engine will need to understand page type only after decryption. And, we already have checksum field in the page header, that can indicate, it is decrypted or not. - if there were reset during encryption, this is the abnormal sitiation, and it can be fixed, for example, with header page flag encryption is in progress and encrypted. The mechanism can be same while engine checks TIP on first connect. If first connect see that DB is in progress, it just continues to encrypt (or decrypt) pages that are not finished yet. Pages itself does not need that flags - they are being written in atomic way, so they can't be in progress. Yes, this way FB will read (at least) the whole database. So? It was intended to do it when encryption/decryption was started. -- Dmitry Kuzmenko, www.ibase.ru, (495) 953-13-34 -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] tool for encrypting database initially (andprobably decrypting it)
On 04/03/12 13:03, Dmitry Kuzmenko wrote: Hello, Vlad! Tuesday, April 3, 2012, 12:44:07 PM, you wrote: VK Encryption must be resistent to the database shutdown\server restart and so on. VK Therefore it must be restartable. As we going to add encrypted flag for each page VK we can know pages that already encrypted. To not read whole database searching VK for the not encrypted pages after restart i offer to store last encrypted page number VK at header page (also, obviously, we need to store encription state on the header VK such as clear, encrypted, encryption is in progress, decryption is in progress). disagree, because - not all db pages need to be encrypted. for example PIP, TIP, etc. Performance effect of decrypting/encrypting these pages can be disasterous. Ceratinly, we should not encrypt something except data, index and blob pages. - I suspect that pages will not be encrypted as a whole. Otherwise, engine will need to understand page type only after decryption. And, we already have checksum field in the page header, that can indicate, it is decrypted or not. Skipping page header looks reasonable to me. - if there were reset during encryption, this is the abnormal sitiation, and it can be fixed, for example, with header page flag encryption is in progress and encrypted. The mechanism can be same while engine checks TIP on first connect. If first connect see that DB is in progress, it just continues to encrypt (or decrypt) pages that are not finished yet. Pages itself does not need that flags - they are being written in atomic way, so they can't be in progress. Yes, this way FB will read (at least) the whole database. So? It was intended to do it when encryption/decryption was started. Storing last encypted page sometimes is good compromise not to read whole DB I think. -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] tool for encrypting database initially(andprobably decrypting it)
VK Encryption must be resistent to the database shutdown\server restart and so on. VK Therefore it must be restartable. As we going to add encrypted flag for each page VK we can know pages that already encrypted. To not read whole database searching VK for the not encrypted pages after restart i offer to store last encrypted page number VK at header page (also, obviously, we need to store encription state on the header VK such as clear, encrypted, encryption is in progress, decryption is in progress). disagree, because - not all db pages need to be encrypted. for example PIP, TIP, etc. Performance effect of decrypting/encrypting these pages can be disasterous. Where it conflicts with my proposition ? - I suspect that pages will not be encrypted as a whole. Otherwise, engine will need to understand page type only after decryption. And, we already have checksum field in the page header, that can indicate, it is decrypted or not. a) There is no checksum field in ODS12, but it doesn't matter here because even if will return checksums it should be calculated for encrypted image *stored on disk* (because checksums are guard against disk errors). b) Page header sooner af all will not be encrypted. c) Where it conflicts with my proposition ? - if there were reset during encryption, this is the abnormal sitiation, and it can be fixed, for example, with header page flag encryption is in progress and encrypted. The mechanism can be same while engine checks TIP on first connect. If first connect see that DB is in progress, it just continues to encrypt (or decrypt) pages that are not finished yet. How do you going to detect still not encrypted pages if you against encrypted flag ? Pages itself does not need that flags - they are being written in atomic way, so they can't be in progress. This flag is *required* to distinguish encrypted pages. I don't understand why do you object it. Yes, this way FB will read (at least) the whole database. So? It was intended to do it when encryption/decryption was started. Why do we need to do a dumb work ? Sorry, but i see no arguments in your disagreement ;) Regards, Vlad -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] tool for encrypting database initially(andprobably decrypting it)
I meant 12345 checksum that was fixed since InterBase 5. So, page checksums are not guards of the pages for a long time. They are just indicators, that if there no 12345, page can be considered as crap. Anyway not good crypt indicator - in some rare cases it can be 12345 on encrypted page. As an indicator flag in header is much better. -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
