Yes, I'll will create a new RC soon based on Enrico's -1.
Andor On 4/30/19 23:34, Flavio Junqueira wrote: > Thanks for the input, Andor and Norbert. I managed to get it to work. Using > "hostname -f" like in the instructions was not working for me. I tried adding > names to /etc/hosts, but it was not liking that either. It worked when I > used the loopback IP address. > > On the RC, could you please clarify the status? Is the plan to have a new RC? > > -Flavio > >> On 30 Apr 2019, at 13:37, Norbert Kalmar <nkal...@cloudera.com.INVALID> >> wrote: >> >> Hi Flavio, >> >> When I tested the TLS setup on a single machine, I also had issues, java >> errors. It turns out those error were completely unrelated to the problem, >> namely I didn't have the localhost setup in my hostname file. If I remember >> correctly, I needed to add localhost 127.0.0.1, because I only had the >> machine's name setup there. So it couldn't find "localhost". >> >> Not entirely sure the solution anymore, but it was definitely something >> with localhost. >> >> After that, it worked fine for me, but I also used a single key for all >> instances on the machine. >> >> Regards, >> Norbert >> >> On Tue, Apr 30, 2019 at 1:29 PM Andor Molnar <an...@apache.org> wrote: >> >>> Hi Flavio, >>> >>> Works for me on a single machine with the following keystores. >>> Aliases are the hostname of the machine (all the same). >>> >>> keystore1.jks >>> ~~~~~~~~~~ >>> Keystore type: JKS >>> Keystore provider: SUN >>> >>> Your keystore contains 1 entry >>> >>> andors-macbook-pro.local, Apr 30, 2019, PrivateKeyEntry, >>> Certificate fingerprint (SHA1): >>> 14:46:5F:92:D9:03:88:7E:C9:0A:95:9E:F5:74:08:F4:27:89:36:9D >>> >>> keystore2.jks >>> ~~~~~~~~~~ >>> Keystore type: JKS >>> Keystore provider: SUN >>> >>> Your keystore contains 1 entry >>> >>> andors-macbook-pro.local, Apr 30, 2019, PrivateKeyEntry, >>> Certificate fingerprint (SHA1): >>> 61:11:F3:FC:97:B1:3D:DB:6C:65:11:AE:FB:26:39:C0:4F:8E:A7:F7 >>> >>> keystore3.jks >>> ~~~~~~~~~~ >>> Keystore type: JKS >>> Keystore provider: SUN >>> >>> Your keystore contains 1 entry >>> >>> andors-macbook-pro.local, Apr 30, 2019, PrivateKeyEntry, >>> Certificate fingerprint (SHA1): >>> E0:84:2A:37:A0:8E:22:67:B3:50:21:43:34:D0:FD:E8:A4:50:C4:3F >>> >>> >>> …and a single truststore: >>> >>> $ keytool -list -keystore truststore.jks >>> Enter keystore password: >>> Keystore type: JKS >>> Keystore provider: SUN >>> >>> Your keystore contains 4 entries >>> >>> mycert3, Apr 30, 2019, trustedCertEntry, >>> Certificate fingerprint (SHA1): >>> E0:84:2A:37:A0:8E:22:67:B3:50:21:43:34:D0:FD:E8:A4:50:C4:3F >>> mycert2, Apr 30, 2019, trustedCertEntry, >>> Certificate fingerprint (SHA1): >>> 61:11:F3:FC:97:B1:3D:DB:6C:65:11:AE:FB:26:39:C0:4F:8E:A7:F7 >>> mycert1, Apr 30, 2019, trustedCertEntry, >>> Certificate fingerprint (SHA1): >>> 14:46:5F:92:D9:03:88:7E:C9:0A:95:9E:F5:74:08:F4:27:89:36:9D >>> >>> Aliases (mycert1, mycert2, mycert3) doesn’t matter here, ZooKeeper only >>> checks if the given certificate is included in the truststore or not. >>> >>> Regards, >>> Andor >>> >>> >>> >>>> On 2019. Apr 30., at 12:48, Andor Molnar <an...@apache.org> wrote: >>>> >>>> Makes sense. >>>> >>>> I’ve tested it on a single machine with the same cert/key for all >>> instances: keystore / truststore only contained a single entry and it >>> worked fine. >>>> We’ve also tested on multiple instances with multiple keys / instances >>> which also worked fine. >>>> Let me give it another go with single machine / multiple certs combo. >>>> >>>> I might need to modify the docs to emphasize keys must be generated on a >>> per machine basis, not ZK instance. >>>> Regards, >>>> Andor >>>> >>>> >>>> >>>>> On 2019. Apr 29., at 16:53, Flavio Junqueira <f...@apache.org> wrote: >>>>> >>>>> I'm also +1 for adding a comment to the release notes (thanks for the >>> suggestion, Ted). Updating the readme makes sense, but the release notes >>> will be the main source to indicate that we require a specific or later >>> version of Java from that particular release. My preference would be to >>> update the release notes. >>>>> As for running TLS on a single node, have you been able to do it? I >>> haven't had a chance to look further into it throughout my day, so if >>> anyone has successfully done it and can share some instructions, it would >>> help me. Otherwise, I'll keep investigating once I have a chance. To be >>> specific, I created the keystore, certificate and truststore files >>> according to instructions, but the instructions assume that the aliases are >>> different when it comes to populating the truststore. At that point, I had >>> to get creative and I have tried a couple of options that didn't work. >>> Either way, I think that being able to run locally and documenting is >>> desirable, although not a blocker. If I can get it right, then I can write >>> a gist describing it that we can use as a reference until we properly >>> document it. >>>>> -Flavio >>>>> >>>>>> On 29 Apr 2019, at 15:42, Andor Molnar <an...@apache.org> wrote: >>>>>> >>>>>> Thanks Flavio for the investigation. I’ll update the README file to >>> include instructions on supported Java 8 versions. >>>>>> I’m wondering if I have to update the admin docs based on your >>> problems running TLS quorum on a single machine. >>>>>> Andor >>>>>> >>>>>> >>>>>> >>>>>>> On 2019. Apr 29., at 15:06, Enrico Olivelli <eolive...@gmail.com> >>> wrote: >>>>>>> Il lun 29 apr 2019, 13:44 Ted Dunning <ted.dunn...@gmail.com> ha >>> scritto: >>>>>>>> Other changes in u211+ substantially improve how Java 8 applications >>> behave >>>>>>>> in containers. I am seeing this more and more with customers. >>>>>>>> >>>>>>>> Combined with the crypto issues, it might be worth a release note >>>>>>>> suggesting that if you are going to compile with Java 1.8, you >>> should use a >>>>>>>> recent release at u211 (u212?) Or above. >>>>>>>> >>>>>>> +1 for a note on release docs >>>>>>> >>>>>>> >>>>>>> Enrico >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Mon, Apr 29, 2019, 11:43 AM Flavio Junqueira <f...@apache.org> >>> wrote: >>>>>>>>> I did a bit more research and it turns out that the crypto.policy >>> option >>>>>>>>> was introduced u151: >>>>>>>>> >>>>>>>>> >>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html >>>>>>>>> < >>>>>>>>> >>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html >>>>>>>>> And started being defined by default with the "unlimited" option in >>> u161: >>>>>>>>> >>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html >>>>>>>>> < >>>>>>>>> >>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html >>>>>>>>> I have installed a more recent version, 1.8.0_211, and it builds >>> fine >>>>>>>> (all >>>>>>>>> tests pass consistently for me). >>>>>>>>> >>>>>>>>> >>>>>>>>> I'm now trying to start an ensemble with ssl enabled locally, but >>> it is >>>>>>>>> failing for me. It looks like the instructions in the admin doc >>> assumes >>>>>>>>> different hosts. I need to look more closely into it to determine >>> what >>>>>>>> not >>>>>>>>> is that I'm doing wrong, but in any case, the instructions do not >>> make it >>>>>>>>> very clear whether one can run locally. >>>>>>>>> >>>>>>>>> -Flavio >>>>>>>>> >>>>>>>>>> On 27 Apr 2019, at 19:28, Patrick Hunt <ph...@apache.org> wrote: >>>>>>>>>> >>>>>>>>>> Odd. I had done my testing on jdk11/macos which is fine. >>>>>>>>>> >>>>>>>>>> I just tried jdk8 and 3 times in a row it's failing with: >>>>>>>>>> [ERROR] >>> SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure:176 » >>>>>>>>>> Timeout Failed t... >>>>>>>>>> >>>>>>>>>> I don't see the error Flavio is seeing. I have never installed >>> special >>>>>>>>>> crypto libraries, etc... just vanilla jdk. >>>>>>>>>> >>>>>>>>>> ⌂102% [phunt:~/Downloads/z/apache-zookeeper-3.5.5] 3s $ mvn >>> --version >>>>>>>>>> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; >>>>>>>>>> 2019-04-04T12:00:29-07:00) >>>>>>>>>> Maven home: /usr/local/Cellar/maven/3.6.1/libexec >>>>>>>>>> Java version: 1.8.0_201, vendor: Oracle Corporation, runtime: >>>>>>>>>> >>> /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre >>>>>>>>>> Default locale: en_US, platform encoding: UTF-8 >>>>>>>>>> OS name: "mac os x", version: "10.14.4", arch: "x86_64", family: >>> "mac" >>>>>>>>>> >>>>>>>>>> 2019-04-27 10:11:51,635 [myid:] - INFO >>>>>>>>>> [NIOWorkerThread-6:SaslServerCallbackHandler@136] - Setting >>>>>>>>> authorizedID: >>>>>>>>>> super >>>>>>>>>> 2019-04-27 10:11:51,636 [myid:] - INFO >>>>>>>>>> [NIOWorkerThread-6:ZooKeeperServer@1170] - adding SASL >>> authorization >>>>>>>> for >>>>>>>>>> authorizationID: super >>>>>>>>>> 2019-04-27 10:11:51,813 [myid:] - INFO >>>>>>>>>> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl >>> exited >>>>>>>>> loop! >>>>>>>>>> 2019-04-27 10:12:21,596 [myid:] - INFO >>>>>>>>>> [main:JUnit4ZKTestRunner$LoggedInvokeMethod@98] - TEST METHOD >>> FAILED >>>>>>>>>> testZKOperationsAfterClientSaslAuthFailure >>>>>>>>>> java.util.concurrent.TimeoutException: Failed to connect to >>> ZooKeeper >>>>>>>>>> server. >>>>>>>>>> at >>>>>>>>>> >>> org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:150) >>>>>>>>>> at >>>>>>>>>> >>> org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure(SaslAuthTest.java:176) >>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>>>> at >>>>>>>>>> >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>>>>>> at >>>>>>>>>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>>>>>>>> at >>>>>>>>>> >>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) >>>>>>>>>> at >>>>>>>>>> >>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) >>>>>>>>>> >>>>>>>>>> Patrick >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sat, Apr 27, 2019 at 8:54 AM Enrico Olivelli < >>> eolive...@gmail.com> >>>>>>>>> wrote: >>>>>>>>>>> On my local tests I usually don't get the error because I am using >>>>>>>>>>> jdk11 and unlimited strength cryptography is enabled by default >>>>>>>>>>> >>>>>>>>>>> >>> https://www.oracle.com/technetwork/java/javase/documentation/jdk11-readme-5097204.html#jce >>>>>>>>>>> In production it depends on the configuration of SSL, if you >>> require >>>>>>>>>>> strong ciphers/big keys you will have problems, but the server >>> won't >>>>>>>>>>> start so you will find soon the problem. >>>>>>>>>>> I think this is not a real issue (for production I mean). >>>>>>>>>>> I see these ways: >>>>>>>>>>> 1) adapt the tests in order to make default jdk8 happy >>>>>>>>>>> 2) tweak the tests enabling "unlimited strenght cryptography" >>> using >>>>>>>>>>> reflection >>>>>>>>>>> 3) just write a line in documentation that says that in order to >>> make >>>>>>>>>>> tests pass you have to enable the flag >>>>>>>>>>> >>>>>>>>>>> That flag is deprecated and enabled by default in modern JREs, so >>> I >>>>>>>>>>> would go for 2) or 3) >>>>>>>>>>> >>>>>>>>>>> I guess that on ASF Jenkins if the JDK8 we are using has the flag >>>>>>>>> turned >>>>>>>>>>> on >>>>>>>>>>> >>>>>>>>>>> Enrico >>>>>>>>>>> >>>>>>>>>>> Il giorno sab 27 apr 2019 alle ore 17:48 Andor Molnar >>>>>>>>>>> <an...@apache.org> ha scritto: >>>>>>>>>>>> I’m running the tests fine without setting the policy to >>> unlimited: >>>>>>>>>>>> java version "1.8.0_161" >>>>>>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_161-b12) >>>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode) >>>>>>>>>>>> >>>>>>>>>>>> Have you tried to run it with a more recent version of Java? >>>>>>>>>>>> >>>>>>>>>>>> Andor >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 2019. Apr 27., at 17:33, Andor Molnar <an...@apache.org> >>> wrote: >>>>>>>>>>>>> Good catch, thanks Flavio for reporting this. We need to double >>>>>>>> check >>>>>>>>>>> the tests with Ilya I believe. >>>>>>>>>>>>> Having tests failure means that you were actually able to >>> _build_ >>>>>>>>>>> ZooKeeper successfully without changing the crypto policy setting. >>>>>>>> Have >>>>>>>>> you >>>>>>>>>>> tried to start an ensemble with Quorum TLS by any chance? That >>> would >>>>>>>> add >>>>>>>>>>> some more color to this issue. >>>>>>>>>>>>> This might be just a testing issue. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Andor >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 2019. Apr 27., at 16:09, Flavio Junqueira <f...@apache.org> >>>>>>>> wrote: >>>>>>>>>>>>>> Hi Enrico, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here is the info you are requesting: >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Java version* >>>>>>>>>>>>>> >>>>>>>>>>>>>> $ java -version >>>>>>>>>>>>>> java version "1.8.0_152" >>>>>>>>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_152-b16) >>>>>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed >>> mode) >>>>>>>>>>>>>> *Test case errors* >>>>>>>>>>>>>> >>>>>>>>>>>>>> I won’t post all of them, I get a good number of errors: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ================================ >>>>>>>>>>>>>> [ERROR] Tests run: 64, Failures: 0, Errors: 16, Skipped: 0, >>> Time >>>>>>>>>>> elapsed: 9.21 s <<< FAILURE! - in >>>>>>>>> org.apache.zookeeper.util.PemReaderTest >>>>>>>>>>>>>> [ERROR] >>> testLoadCertificateFromKeyStore[1](org.apache.zookeeper.util.PemReaderTest) >>>>>>>>>>> Time elapsed: 1.593 s <<< ERROR! >>>>>>>>>>>>>> java.io.IOException: >>>>>>>>>>> org.bouncycastle.operator.OperatorCreationException: Illegal key >>> size >>>>>>>> or >>>>>>>>>>> default parameters >>>>>>>>>>>>>> at >>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException: >>>>>>>>>>> Illegal key size or default parameters >>>>>>>>>>>>>> at >>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size >>> or >>>>>>>>>>> default parameters >>>>>>>>>>>>>> at >>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>>>>>>>> [ERROR] >>> testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword[1](org.apache.zookeeper.util.PemReaderTest) >>>>>>>>>>> Time elapsed: 0.004 s <<< ERROR! >>>>>>>>>>>>>> java.lang.Exception: Unexpected exception, >>>>>>>>>>> expected<java.security.GeneralSecurityException> but >>>>>>>>>>> was<java.io.IOException> >>>>>>>>>>>>>> at >>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException: >>>>>>>>>>> Illegal key size or default parameters >>>>>>>>>>>>>> at >>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size >>> or >>>>>>>>>>> default parameters >>>>>>>>>>>>>> at >>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> ================================ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Crypto policy* >>>>>>>>>>>>>> If I uncomment this configuration option: >>>>>>>>>>>>>> >>>>>>>>>>>>>> # Please see the JCA documentation for additional information >>> on >>>>>>>>> these >>>>>>>>>>>>>> # files and formats. >>>>>>>>>>>>>> # crypto.policy=unlimited >>>>>>>>>>>>>> >>>>>>>>>>>>>> in: >>>>>>>>>>>>>> >>>>>>>>>>>>>> $JAVA_HOME/jre/lib/security/java.security >>>>>>>>>>>>>> >>>>>>>>>>>>>> then it all works and I get no error at all. This option >>> controls >>>>>>>>>>> cryptographic strengths according to the documentation, and is >>> present >>>>>>>>>>> because of crypto regulations in different countries. >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> -Flavio >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 27 Apr 2019, at 15:52, Enrico Olivelli < >>> eolive...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>> Il sab 27 apr 2019, 14:18 Flavio Junqueira <f...@apache.org> >>> ha >>>>>>>>>>> scritto: >>>>>>>>>>>>>>>> I have a clarification question about the RC. To build the >>> RC, I >>>>>>>>>>> had to >>>>>>>>>>>>>>>> enable crypto.policy unlimited in the jre (I'm using build >>>>>>>>>>> 1.8.0_152-b16). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Flavio >>>>>>>>>>>>>>> What do you mean with 'build' ? >>>>>>>>>>>>>>> Make tests pass? >>>>>>>>>>>>>>> AFAIK we are not using tweaked jdks in CI builds, so in theory >>>>>>>> there >>>>>>>>>>> is no >>>>>>>>>>>>>>> need. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you please share your error? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Enrico >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm wondering if this is going to be an issue for some users >>> as >>>>>>>> this >>>>>>>>>>> option >>>>>>>>>>>>>>>> is related to import/export regulation. Has anyone looked >>> into it >>>>>>>>>>> and could >>>>>>>>>>>>>>>> clarify it to me, please? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> -Flavio >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 25 Apr 2019, at 15:10, Andor Molnar <an...@apache.org> >>>>>>>> wrote: >>>>>>>>>>>>>>>>> This is the first stable release of 3.5 branch: 3.5.5. It >>>>>>>> resolves >>>>>>>>>>> 117 >>>>>>>>>>>>>>>> issues, including Maven migration, Quorum TLS, TTL nodes and >>> lots >>>>>>>>>>> of other >>>>>>>>>>>>>>>> performance and stability improvements. >>>>>>>>>>>>>>>>> The full release notes is available at: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12343268 >>>>>>>>>>>>>>>>> *** Please download, test and vote by May 3rd 2019, 23:59 >>> UTC+0. >>>>>>>>>>> *** >>>>>>>>>>>>>>>>> Source files: >>>>>>>>>>>>>>>>> >>> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc5/ >>>>>>>>>>>>>>>>> Maven staging repos: >>>>>>>>>>>>>>>>> >>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/ >>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/ >>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/ >>>>>>>>>>>>>>>>> The release candidate tag in git to be voted upon: >>>>>>>>>>> release-3.5.5-rc5 >>>>>>>>>>>>>>>>> ZooKeeper's KEYS file containing PGP keys we use to sign the >>>>>>>>>>> release: >>>>>>>>>>>>>>>>> http://www.apache.org/dist/zookeeper/KEYS >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Should we release this candidate? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>> >>>
