> My cluster won't start now after this IT test runs. That is to be expected?
Yes, but it is ugly I agree. We can check the deploy-ability of encryption like we do with compression. Let me open an issue for doing that. > On Dec 1, 2014, at 2:29 AM, Stack <[email protected]> wrote: > >> On Sun, Nov 30, 2014 at 7:54 PM, Andrew Purtell <[email protected]> wrote: >> >> Yes, you have to set up cluster configuration for encryption or the IT >> won't work. > > > Ok. Thanks. I'll add note to the IT section that wildcard will run at least > this test that requires encryption setup. > > My cluster won't start now after this IT test runs. That is to be expected? > > > > >> Encryption requires creating a keystore, etc. If you run the >> test as all-localhost, using mvn verify, it should pass, because all >> configuration can be programatically set up and deployed since the >> environment is a mini cluster. You can see how the test sets up the >> necessary configuration using HTU for that case. > Thanks. Will add note on above and pointers to the security section of the > refguide (as per your note after this one). > > St.Ack > > > >>> On Sun, Nov 30, 2014 at 6:52 PM, Stack <[email protected]> wrote: >>> >>> I just tried running all tests and notice that at least >>> IntegrationTestIngestWithEcnryption fails reliably with the below. Is >> this >>> expected? Does this test require a particular context setup first to run, >>> one that is not present normally? Once it has run, I cannot get past it. >>> If I restart servers, they fail on log replay with same exception. Is >> this >>> expected? >>> >>> On IntegrationTests in general, I was thinking they should generally pass >>> and that it is a bug if they do not. Is that what others think? >>> >>> Has anyone been running IT tests regularly? If so, what has been your >>> experience? Or if you have been running it tests, do you run individual >>> tests? >>> >>> Thanks. Just trying to figure what current understanding of their state >> is >>> before digging in. >>> Yours, >>> St.Ack >>> >>> 2014-11-29 21:19:06,447 DEBUG >>> [B.defaultRpcServer.handler=14,queue=2,port=16020] regionserver.HRegion: >>> Flush requested on >> IntegrationTestIngestWithEncryption,cccccccc,1417324570806.325074071473eb28b662f7d694e8c609. >>> 2014-11-29 21:19:06,640 FATAL [MemStoreFlusher.1] >>> regionserver.HRegionServer: ABORTING region server >>> c2021.halxg.cloudera.com,16020,1417323325816: >>> Replay of HLog required. Forcing server shutdown >>> org.apache.hadoop.hbase.DroppedSnapshotException: region: >> IntegrationTestIngestWithEncryption,,1417324570806.55e317f49ddf8ed35323d9b675611548. >>> at >> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1964) >>> at >> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1730) >>> at >> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1662) >>> at >> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:434) >>> at >> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:407) >>> at >> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$800(MemStoreFlusher.java:69) >>> at >> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:225) >>> at java.lang.Thread.run(Thread.java:745) >>> Caused by: java.lang.RuntimeException: java.lang.RuntimeException: >>> KeyProvider scheme should specify KeyStore type >>> at >> org.apache.hadoop.hbase.io.crypto.Encryption.getKeyProvider(Encryption.java:535) >>> at >> org.apache.hadoop.hbase.io.crypto.Encryption.getSecretKeyForSubject(Encryption.java:425) >>> at >> org.apache.hadoop.hbase.io.crypto.Encryption.encryptWithSubjectKey(Encryption.java:449) >>> at >> org.apache.hadoop.hbase.security.EncryptionUtil.wrapKey(EncryptionUtil.java:92) >>> at >> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.finishClose(HFileWriterV3.java:127) >>> at >> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.close(HFileWriterV2.java:366) >>> at >> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.close(StoreFile.java:986) >>> at >> org.apache.hadoop.hbase.regionserver.StoreFlusher.finalizeWriter(StoreFlusher.java:67) >>> at >> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:80) >>> at >>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:871) >>> at >> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2080) >>> at >> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1918) >>> ... 7 more >>> Caused by: java.lang.RuntimeException: KeyProvider scheme should specify >>> KeyStore type >>> at >> org.apache.hadoop.hbase.io.crypto.KeyStoreKeyProvider.init(KeyStoreKeyProvider.java:142) >>> at >> org.apache.hadoop.hbase.io.crypto.Encryption.getKeyProvider(Encryption.java:528) >>> ... 18 more >> >> >> >> -- >> Best regards, >> >> - Andy >> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >> (via Tom White) >>
