> On Feb. 17, 2022, 7:54 a.m., bhavik patel wrote: > > Thanks for the deatiled info and cleaning up the packing. > > > > can you please verify zone operations, import/export keys to jceks file and > > masterkey import/export operation. > > > > I just wan to make we are not breaking the existing functionality’s. > > Kirby Zhou wrote: > Tested: > > # Basic APIs passed > > 1. TestConnection by rangeradmin > 2. Browes existing keys by rangeradmin > 2. CreateKey by rangeradmin > 3. KeyRoll by rangeradmin > 4. generateeek by curl > 5. decrypteek by curl > > # export passed > ]$ ./exportKeysToJCEKS.sh mkeybak jceks > Enter Password for the keystore FILE : > Enter Password for the KEY(s) stored in the keystore: > Keys from Ranger KMS Database has been successfully exported into mkeybak > ]$ keytool -list -keystore mkeybak -storetype jceks > Enter keystore password: > Keystore type: JCEKS > Keystore provider: SunJCE > > Your keystore contains 12 entries > > hell, Feb 17, 2022, SecretKeyEntry, > hell-1-2-31, Feb 17, 2022, SecretKeyEntry, > hell-1-2-31@0, Feb 17, 2022, SecretKeyEntry, > hell-1-2-31@1, Feb 17, 2022, SecretKeyEntry, > hell@0, Feb 17, 2022, SecretKeyEntry, > hell@1, Feb 17, 2022, SecretKeyEntry, > hell@2, Feb 17, 2022, SecretKeyEntry, > hello-world, Feb 17, 2022, SecretKeyEntry, > hello-world@0, Feb 17, 2022, SecretKeyEntry, > hello-world@1, Feb 17, 2022, SecretKeyEntry, > paladin, Feb 17, 2022, SecretKeyEntry, > paladin@0, Feb 17, 2022, SecretKeyEntry, > > > > # import passed with a bit of accident > bash -c 'function java() { /usr/bin/java -Djceks.key.serialFilter= "$@" ; > } ; source ./importJCEKSKeys.sh mkeybak jceks' > Enter Password for the keystore FILE : > Enter Password for the KEY(s) stored in the keystore: > Keys from mkeybak has been successfully imported into RangerDB. > > # Why -Djceks.key.serialFilter appears? > > See > https://www.oracle.com/java/technologies/javase/8u171-relnotes.html#JDK-8189997 > > Since jdk-8u171, importJCEKSKeys is broken without > -Djceks.key.serialFilter=. > I tested it at tag-ranger-2.2, it is broken too. > But that is another story, I wont fix it in that patch here. > > Kirby Zhou wrote: > Let us check "RangerKeyStoreProvider.java" > > ``` > KeyVersion innerSetKeyVersion(...) { > if (azureKeyVaultEnabled || this.isGCPEnabled) { > // ... > } else { > dbStore.addKeyEntry(versionName, new > SecretKeySpec(material, > cipher), masterKey, cipher, > bitLength, description, > version, attribute); > } > } > > > public void flush(...) { > if (azureKeyVaultEnabled || > this.isGCPEnabled) { > // ... > } else { > > dbStore.addKeyEntry(entry.getKey(), new KeyMetadata( > metadata), > masterKey, metadata.getAlgorithm(), > > metadata.getBitLength(), metadata > > .getDescription(), metadata > > .getVersions(), attributes); > } > } > > ``` > > innerSetKeyVersion is called by createKey and rollNewVersion to create > versioned Key "xxxx@0" "xxxx@1", and flush is called after > createKey/rollNewVersion, it will create "xxxx". > > You can see innerSetKeyVersion call new SecretKeySpec(...) and flush call > new KeyMetadata(...). > And the SecretKeySpec can not be unsealed without > "-Djceks.key.serialFilter=" > > I donot know the reason of store type choice between KeyMetadata and > SecretKeySpec. > Maybe someone can tell me? > > The code is writteb by @dhavalshah9131 , commited by @mehulbparikh > > 02640d3cf520ee5a0dae13dd46b91ff266359e77 > > RANGER-2497 : Support Azure Key Vault for storing master keys of Ranger > KMS
Thanks for the verification. "-Djceks.key.serialFilte" is not the issue with kms code I think it's the issue with some Java version(Security policies) - bhavik ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73852/#review224066 ----------------------------------------------------------- On Feb. 16, 2022, 10:29 a.m., Kirby Zhou wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73852/ > ----------------------------------------------------------- > > (Updated Feb. 16, 2022, 10:29 a.m.) > > > Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, > Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Mehul > Parikh, pengjianhua, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal > Suvagia, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-3595 > https://issues.apache.org/jira/browse/RANGER-3595 > > > Repository: ranger > > > Description > ------- > > There are lots of .class files under ews/webapp/, and never used. > > > 1. place web.xml at correct location > 2. setup.sh want to patch hadoop-common.jar at runtime, it requires some file > inside ranger-kms.jar. But the patching of hadoop-common.jar is unnecessary. > > Regular webapp should have its own class files under > ews/webapp/WEB-INF/classes, and dependencies under ews/webapp/WEB-INF/lib, > and the Container should put its libraries under ews/lib. But at current, we > use directories sucn as ews/webapp/lib, ews/webapp/WEB-INF/classes/lib. It > looks dirty and ugly. > > > My patch here makes KMS no longer bring ranger-kms.jar, and place classes and > web.xml at correct location. as a alternative of > https://reviews.apache.org/r/73816/ > > > Now: > ews/lib contains ews bootstrap jars, > ews/webapp/WEB-INF/classes contains KMS app itself, > ews/webapp/WEB-INF/lib contains KMS dependencies, > ews/webapp/WEB-INF/lib/ranger-kms-plugin-impl contains ranger-kms-plugin. > > Additionaly, kms/pom.xml even depends on original hadoop-kms, which can > confuse developers, so I removed it. > > BTW: the bootstrap embedded server looks like too heavy and too much > dependeices. > > > Diffs > ----- > > distro/src/main/assembly/kms.xml 983a43e59 > kms/pom.xml 7a4f98df7 > kms/scripts/DBMK2HSM.sh 001199d97 > kms/scripts/DBMKTOAZUREKEYVAULT.sh cfe5a6b5e > kms/scripts/DBMKTOKEYSECURE.sh c0aa6e58c > kms/scripts/HSMMK2DB.sh 6c77f7340 > kms/scripts/KEYSECUREMKTOKMSDB.sh 340e05e2c > kms/scripts/VerifyIsDBMasterkeyCorrect.sh 1c9a2e148 > kms/scripts/exportKeysToJCEKS.sh f3205789b > kms/scripts/importJCEKSKeys.sh 5d4fe978f > kms/scripts/ranger-kms 429a31e5a > kms/scripts/setup.sh 2051df59a > kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java > c899bdf98 > kms/src/main/resources/META-INF/context.xml > kms/src/main/resources/WEB-INF/web.xml 5e2d489fe > > > Diff: https://reviews.apache.org/r/73852/diff/1/ > > > Testing > ------- > > mvn clean pacakge > fresh install and upgrade from 2.2.0 > > > Thanks, > > Kirby Zhou > >