Also, FWIW the crypto transformation for WALs in 2.0 uses "AES/CBC/NoPadding". There was only minimal testing done with that mode in 2.0 but it does seem to work with the WALs.
>From my understanding, I think you may have issues with CFB mode and the block sizes. The crypto parameters are written out at the beginning of the WAL (since the file could be closed prematurely) which doesn't work for all cipher modes. On Mon, Oct 21, 2019 at 6:04 PM Christopher <[email protected]> wrote: > My understanding is that CFB mode may not be suitable for write-ahead > logs. Perhaps CBC is better? I'm not sure. For what it's worth, a lot > of work was done in Accumulo 2.0 to help support configuring separate > modes for RFiles and WALs > (https://accumulo.apache.org/docs/2.x/security/on-disk-encryption), > and the updates may better handle the failure conditions that can > happen. Of course, this won't help you in 1.9.3. > > Keep in mind the on-disk encryption features are still experimental. > With more feedback, and development time/effort, it'd be cool to see > this feature get better over time, but I'm not sure to what extent WAL > encryption has been tested in the case of failure scenarios requiring > recovery. > > One relatively easy thing to do to recover from this specific failure > (assuming your key is correct, and this is definitely caused by a > corrupt block in the WAL file), is to replace the WAL file with an > effectively blank one or to surgically delete the WAL entry from the > metadata. However, this will likely cause data loss, which is > obviously not a great outcome, but it depends on how critical the data > is to your application. It could be particularly bad if the WAL > contained writes to the root or metadata table. > > I don't know how much work it would be to try to recover the > non-corrupt blocks from the WAL file, but that's an option as well, if > the data is critical. > > On Mon, Oct 21, 2019 at 4:42 PM Cyrille Savelief <[email protected]> > wrote: > > > > We are using the built in crypto strategy : > CachingHDFSSecretKeyEncryptionStrategy with the cypher suite > AES/CFB/NoPadding > > > > > > > > > > > > > > > > De : Mike Miller > > Envoyé le :lundi 21 octobre 2019 22:38 > > À : [email protected] > > Objet :Re: javax.crypto.BadPaddingException > > > > > > > > Hi Cyrille - This could be a number of issues. The Key could be wrong, > the block could not finished writing to the WAL when you shutdown, etc... > > > > > > > > What type of crypto do you have setup? Are you using the built in > crypto strategies or did you write your own? > > > > > > > > On Mon, Oct 21, 2019 at 3:36 PM Cyrille Savelief <[email protected]> > wrote: > > > > Hi, > > > > > > > > Today, we stopped our Accumulo cluster (1.9.3, with encryption of data > at rest) but we are now unable to restart it due to the following error : > javax.crypto.BadPaddingException : Given final block not properly padded. > Any workaround ? > > > > > > > > Best, > > > > > > > > Cyrille > > > > > > > > >
