Re: MS Office 2010+ "Agile" encrypted OOXML documents are working now!

[Matthias Seidel]

Hi All,

Am 18.03.24 um 20:01 schrieb Matthias Seidel:

Hi Damjan,

I managed to get it cherry-picked to AOO42X (one older commit was missing) and do a build now.

Works great!



Anyone?

Regards,

   Matthias

That said, it would be fantastic to have that fix in AOO41X but I don't dare to cherry-pick it...

Thanks again for this work!

Regards,

   Matthias

Am 18.03.24 um 07:10 schrieb Damjan Jovanovic:

That's great. Thank you for testing.

My own tests also show it no longer crashes, both the Standard and Agile
encryption are working now :).


wrote:

On Sun, Mar 17, 2024 at 7:59 PM Matthias Seidel <matthias.sei...@hamburg.de>

Hi Damjan, All,

That seems to have fixed the crashes on Windows...

Regards,

     Matthias

Am 17.03.24 um 16:56 schrieb Matthias Seidel:

I am already doing a full build,but hopefully I could pull your last
commit soon enough.

Regards,

    Matthias

Am 17.03.24 um 15:20 schrieb Damjan Jovanovic:

If you haven't done "dmake clean" yet like me, try to "git pull" my
latest


This commit fixes at least one more issue:

commit 244f2bcc921bc5dc45e6c1970e27ac2409c44e17 (HEAD -> trunk,
origin/trunk, origin/HEAD)
Author: Damjan Jovanovic <dam...@apache.org>
Date:   Sun Mar 17 15:56:38 2024 +0200


      is initialized.

On Sun, Mar 17, 2024 at 1:24 PM Matthias Seidel
<matthias.sei...@hamburg.de>
wrote:

commit, and then in main/instsetoo_native "build --from oox -P2 -- -P2".       Don't allow calls to OpenSSLCipher::blockSize() before the cipher

WTH, shortly after I sent this mail I could open the files...

Matthias

Am 17.03.24 um 14:22 schrieb Matthias Seidel:

Hi Damjan,

That was fast! ;-)

I can confirm that it builds now on Windows.

When I try to open password protected MS Office 2020+ files (xlsx,
docx) I now get the password dialog

But when I enter the correct password AOO crashes.

Regards,

     Matthias

P.S.: This was a partial build, I will try a complete build now.

Am 17.03.24 um 12:23 schrieb Damjan Jovanovic:

I've fixed this now and it seems to build on Windows too. The fixes

are in

order:

f65b4e326d91bfe900dc1dd22ece69e3ddd8444a
f3025b08c40161265442c34e2b50bc05aa5388c6
42c0a318a970f6f7f43d26a8397448d5d5b8bd36

There were several problems on Windows. The Visual Studio 2008
compiler
doesn't have the ::std::vector::data() method (even though it
should, it

Also
that ancient OpenSSL version we use internally, 1.0.x, uses
EVP_MD_CTX_create()/destroy() instead of EVP_MD_CTX_new()/free().
Finally
some template function was unhappy about parameter type ambiguity
(even

to add
casts.


On Sun, Mar 17, 2024 at 11:15 AM Matthias Seidel
<matthias.sei...@hamburg.de>
wrote:

these commits, and if they work, should be cherry-picked in the given was part of the C++98/03 standard), I had to use &vector[0] instead. though superior compilers like Clang are perfectly happy), and I had

Hi Damjan,

You are right, the interesting part is at the beginning:

---

=============
Building module oox
=============

Entering /cygdrive/c/Source/openoffice/main/oox/prj

cd .. && make -s -r -j1   && make -s -r deliverlog
[ build CXX ] oox/source/core/encryption
encryption.cxx

c:/Source/openoffice/main/oox/inc\oox/helper/openssl_wrapper.hxx(44)

:
error C3861: 'EVP_MD_CTX_new': identifier not found

c:/Source/openoffice/main/oox/inc\oox/helper/openssl_wrapper.hxx(51)

:
error C3861: 'EVP_MD_CTX_free': identifier not found
c:/Source/openoffice/main/oox/source/core/encryption.cxx(100) :
error

ambiguous

declaration of 'oox::getFlag'
            could be 'unsigned int'
            or       'sal_uInt32'
c:/Source/openoffice/main/oox/source/core/encryption.cxx(133) :
error

ambiguous

declaration of 'oox::getFlag'
            could be 'unsigned int'
            or       'sal_uInt32'
c:/Source/openoffice/main/oox/source/core/encryption.cxx(134) :
error

ambiguous

declaration of 'oox::getFlag'
            could be 'unsigned int'
            or       'sal_uInt32'
c:/Source/openoffice/main/oox/source/core/encryption.cxx(210) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(214) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(217) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(299) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(362) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(700) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(701) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(702) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(710) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(711) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(753) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(754) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(756) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(780) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(781) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(783) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(809) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(833) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(833) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(833) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(852) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(854) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(882) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(882) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(882) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(888) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(888) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(888) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(930) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(932) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(932) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(938) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
c:/Source/openoffice/main/oox/source/core/encryption.cxx(946) :
error
C2039: 'data' : is not a member of 'std::vector<_Ty>'
            with
            [
                _Ty=sal_uInt8
            ]
make: *** No rule to make target
'/cygdrive/c/Source/openoffice/main/solver/450/
wntmsci12.pro/workdir/CxxObject/oox/source/core/encryption.o',
needed by
'/cygdrive/c/Source/openoffice/main/solver/450/
wntmsci12.pro/workdir/LinkTarget/Library/ioox.lib'.
Stop.
dmake:  Error code 2, while making 'all'

1 module(s):
            oox
need(s) to be rebuilt

Reason(s):

ERROR: error 65280 occurred while making
/cygdrive/c/Source/openoffice/main/oox/prj

When you have fixed the errors in that module you can resume the
build
by running:

            build --from oox

---

Regards,

       Matthias

Am 17.03.24 um 01:55 schrieb Damjan Jovanovic:

C2782: 'bool oox::getFlag(Type,Type)' : template parameter 'Type' is c:/Source/openoffice/main/oox/inc\oox/helper/helper.hxx(141) : see C2782: 'bool oox::getFlag(Type,Type)' : template parameter 'Type' is c:/Source/openoffice/main/oox/inc\oox/helper/helper.hxx(141) : see C2782: 'bool oox::getFlag(Type,Type)' : template parameter 'Type' is c:/Source/openoffice/main/oox/inc\oox/helper/helper.hxx(141) : see

Please check further up in the log, or run "build" in main/oox
again.

That "No rule to make target" happens after other errors break

compilation.

On Sat, Mar 16, 2024 at 8:38 PM Matthias Seidel <

matthias.sei...@hamburg.de>

wrote:

Hi Damjan,

Tried to build trunk on Windows but it stops here:

make: *** No rule to make target
'/cygdrive/c/Source/openoffice/main/solver/450/
wntmsci12.pro/workdir/CxxObject/oox/source/core/encryption.o',
needed by
'/cygdrive/c/Source/openoffice/main/solver/450/
wntmsci12.pro/workdir/LinkTarget/Library/ioox.lib'.
Stop.
dmake:  Error code 2, while making 'all'

1 module(s):
             oox
need(s) to be rebuilt

Reason(s):

ERROR: error 65280 occurred while making
/cygdrive/c/Source/openoffice/main/oox/prj


build
by running:

             build --from oox

Am 16.03.24 um 04:49 schrieb Damjan Jovanovic:

When you have fixed the errors in that module you can resume the

Hi


(encrypted) OOXML files from MS Office 2010+, is now fixed in
trunk :-)

---snip---

origin/trunk, origin/HEAD)
Author: Damjan Jovanovic <dam...@apache.org>
Date:   Sat Mar 2 18:47:05 2024 +0200

Bug 118236 with 7 votes, the inability to open password-protected commit 506fa58b1970084a0caacb50b3a805e469be4756 (HEAD -> trunk,          Implement the (MS Office 2010+) OOXML "Agile encryption"

support, so

that we
         can open such password-protected OOXML files.

         Adds all the Agile encryption XML tokens and
namespaces, and

parses

the

XML
         from EncryptionInfo stream, gets OpenOffice to
recognize the

file is

encrypted
         and ask for a password, and successfully decrypts the
file if

password

is
         correct.

         Also a number of other fixes and improvements:
         - Sorted main/oox/source/token/tokens.txt so it's in
alphabetical

order

           (wrong order might have broken certain tokens?).

It's now

         - Refactored how OOXML encryption is generally handled.

in

its

           own file.

         - Added logging to the FilterDetect class. It logs to the

office-wide

default
           logger.
         - Added a flush() method to the BinaryXOutputStream
class.

XComponentContext
           instead of the deprecated XMultiServiceFactory.
         - Error handling was generally improved.

instead

         - Changed FilterDetect to use XMultiComponentFactory and          - Exception safety and some memory safety (::std::vector

of

new[])
           in all the new code. Memory leaks should not be
possible.


from the
excellent
         Apache POI project, so it's been credited in our
NOTICE file.

         Patch by: me
---snip---


It took much longer than I expected:

The MS-OFFCRYPTO specification was unclear, and plain wrong in
some

         Much of the code involved in the decryption was ported

parts,

eg. "SHA-1" in the spec but "SHA1" in actual OOXML documents;
I've made

our

code support both.

The "Standard" encryption from MS Office 2007 that we already
supported

was

itself a mess, and much work was needed to refactor and clean up

that

code

before the "Agile" encryption could also be added.

Then XML parsing had to be added, since Agile encryption
specifies

settings

in XML instead of binary like Standard encryption did. XML
handling in
OpenOffice is pretty outdated, with no support for namespaces,
but at

least

the newer "FastParser" does support namespaces and is in fact
very fast
because it converts strings to unique integers, and packs
namespaces

into

bit fields, for faster comparisons. I ended up updating the
main/oox
FastParser to support the new Agile encryption namespaces and
elements.


since

MS-OFFCRYPTO also only describes encryption, not decryption, and

we

can only read OOXML, only decryption matters.

Apache POI code was tremendously helpful in figuring out the
decryption
process. Most of the decryption code I added was just ported
directly

from

theirs, and thus I've added Apache POI to our NOTICE file (please

check
that I've done it correctly). Also several bugs were figured
out by

simultaneously stepping through our code in gdb and their code in

NetBeans,

and comparing respective values. A big thank you to the Apache
POI

ways!

I used OpenSSL for all the message digest and encryption
stuff, both

developers, whose OOXML support is still better than ours in many because our MD5 and SHA1 algorithms are broken (bug 127661), and

because

Agile encryption requires many digests and ciphers that OpenSSL

supports

but we don't.


text
documents, spreadsheets, presentations, etc.

Other issues I am aware of:
- We only support password encrypted documents. Certificate
encrypted

maybe
that's something we should develop together.
- There are other variations of encryption we still don't
support, eg.

Anyway, it works now. All encrypted OOXML files should work, eg. documents: not yet. ODF 1.3 also added certificate encryption, so

the

"Extensible" encryption, the "RC4 CryptoAPI" encryption, "XOR

obfuscation",

etc. Apache POI would be a good source for those too. It's
unclear to

me

how widely those are used, and whether they are worth
implementing.
- It may need to be rearchitected when we add OOXML writing.
- A lot of other required cleanups to our code were
discovered, will
discuss those separately.

I've squashed all my work into a single commit, so it can be
easily
cherry-picked to AOO42X and maybe even AOO41X when people are
happy

with

it.

Regards
Damjan

smime.p7s
Description: Kryptografische S/MIME-Signatur