Hi Marc, I didn't follow the full thread, but I get that you start from a letsencrypt PEM cert/key and try to create a keystore for James. I could show you how I do that in Java (maybe that could help since at the end, you will want to automate all that).
First, there is a json config file where there is the path to the Pem file that contains both the cert and private key parts: https://github.com/foilen/foilen-email-server/blob/master/src/main/java/com/foilen/email/server/config/EmailConfig.java#L31 For reading the PEM and converting it, I use Bouncycastle and an helper library (jl-smallstools). The dependencies are here: https://github.com/foilen/foilen-email-server/blob/master/build.gradle#L32-L34 https://github.com/foilen/foilen-email-server/blob/master/build.gradle#L38 https://github.com/foilen/foilen-email-server/blob/master/gradle.properties#L5 The part that reads the PEM file https://github.com/foilen/foilen-email-server/blob/master/src/main/java/com/foilen/email/server/james/JamesWorkDirManagement.java#L145-L151 and loads it in an RSACertificate Lastly, the part that creates the keystore with a password (always "james" in my case) https://github.com/foilen/foilen-email-server/blob/master/src/main/java/com/foilen/email/server/james/JamesWorkDirManagement.java#L60-L73 Last thing to not forget is that your James Server needs Bouncycastle, so do not forget to add it to its classpath (e.g, put the jars in conf/lib) I hope this helps. On Tue, 7 Jan 2020 at 00:03, Marc Chamberlin <[email protected]> wrote: > Thanks Garry for your reply, and I am sorry for the slowness of my > response, I been busy doing internet research trying to grok everything > you said in your suggestions. I think the best thing to do is to walk > you and other readers through the steps I took to create the LetsEncrypt > certificate and put it in the keystore. Then I will comment on each of > your suggestions and await for further clarification on what I should > try/do next. I am not at all familiar with the code and inner workings > of Apache James so to me a lot of this is like trying to understand > what is in a room by looking through a keyhole! LOL > > OK, these are the steps I took - > > To create the LetsEncrypt certificate I used the DNS challenge method to > create a wildcard certificate for all the domains and subdomains that I > provide email services for. This was done as follows using the certbot > command - > > certbot certonly --config-dir /etc/letsencrypt_forApacheJames > --dns-rfc2136 --dns-rfc2136-credentials > /etc/letsencrypt_forApacheJames/james/rfc2136.ini > --dns-rfc2136-propagation-seconds 10 --server > https://acme-v02.api.letsencrypt.org/directory > --preferred-challenges=dns --email [email protected] --agree-tos > -d domainname1.com -d *.domainname1.com -d domainname2.com -d > *.domainname2.com /etc.../ > > If I understand things correctly this produced 4 certificate and chain > files - > > cert.pem is the certificate containing the public key for my domains. > chain.pem is the certificate for my certificate authority - LetsEncrypt > fullchain.pem is also created by certbot = cert.pem + chain.pem > privkey.pem containing the private key for my domains. > > Next I created an empty keystore in the conf directory for Apache James - > > cd /mail/apache-james-3.4/james-server-app-3.4.0/conf > mkdir keystore.privateFiles > cd keystore.privateFiles > keytool -genkeypair -keyalg RSA -alias emptykeystore -keystore keystore.jks > > and I migrated the keystore to PKCS12 format and deleted the > emptykeystore alias using > > keytool -importkeystore -srckeystore keystore.jks -destkeystore > keystore.jks -deststoretype pkcs12 > keytool -delete -alias emptykeystore -keystore keystore.jks > > Next I converted the keys created by certbot into the proper format for > importing it into the keystore, using openssl to convert the keys - > > openssl pkcs12 -export -in fullchain.pem -inkey privkey.pem -out > pkcs.p12 -name james > > This creates a file called pkcs.p12 which I then import into the > keystore file - > > keytool -importkeystore -deststorepass mypassword -destkeypass > mypassword -destkeystore keystore.jks -srckeystore > /etc/letsencrypt_forApacheJames/live/mydomain.com-0001/pkcs.p12 > -srcstoretype PKCS12 -srcstorepass mypassword -alias james > > I left out a few details like sensitive info, cd'ing to various > different directories and setting up a link to the keystore file. The > passwords are simple alphanumeric character strings. I will intersperse > the rest of my reply in-between your comments - > > On 1/3/20 2:12 PM, Garry Hurley wrote: > > Okay. One thing I noticed before. The key took command will hash the > password with the appropriate algorithm and use it to access the keystone > file. James code is not guaranteed to use the appropriate hashing > algorithm. Try the following: > I am not sure what you mean and are referring to as the hashing > algorithm though I do know what hashing is. In the steps I showed you, > that I took to create the keystore, I see parameters that specify > encryption algorithms but nothing that specifies a hashing algorithm. > > 1 sift through the code, find the hashing algorithm used by James and > hard code the hashed key in the configuration file. > Oh boy, I am not set up to build James from source and suspect that will > be a steep learning curve. I will table this option for the moment and > consider it if option 2 doesn't work. > > 2, find out which hashing algorithm is used to encrypt the key and > specify it in the configuration file > This sounds promising except I don't know what hashing algorithm was > used in creating the keys. I suspect you are referring to the encryption > algorithm? (I know nothing about how the encryption is actually done, > but could imagine that it is based on a hash of some kind. I believe > that I am using the RSA encryption algorithm but check me on that... > Also I don't know how to specify the "hashing algorithm" in the > configuration files so an example would be very helpful. > > 3, create a new key store WITHOUT a password and use that one instead. > This sounds dangerous and I will try this approach also if all else > fails... But yeah it sounds like another path I could follow... > > For ease of use, #3 is the clear winner. For maximum security, #1 would > be preferred. I am almost certain you will find it is due to a hashing > algorithm or, alternatively, a character in the password that is expressly > probibited in an XML file (like an ampersand or greater/less than sign for > example). > The password is only composed of lower case letters and numbers. No > punctuation characters are used. > > I don't know if this is applicable but in my internet searches looking > for discussions about the same or similar error messages that I am > seeing, I am seeing some references to problems with the tools from > BouncyCastle. Here are a couple of links, perhaps you or some other > guru could check these out and see if I am perhaps experiencing the same > or similar issue as these folks are talking about - > > > https://stackoverflow.com/questions/53542198/adding-bouncycastle-provider-breaks-keystore-load > > https://github.com/bcgit/bc-java/issues/586 > > This is above my pay grade to understand so again many thanks and I > appreciate yours and any other help offered... Marc... > > > > > Sent from my iPhone > > > >> On Dec 28, 2019, at 9:35 PM, Marc Chamberlin > >> <[email protected]> > wrote: > >> > >> Hello again, It appears that I have managed to make a bit of progress > >> on my own since my original post. For some odd reason I got past the > >> FileNotFound exception by changing (adding) execute permission to the > >> directory containing the keystore file. So now it is set as follows - > >> > >> quasar:/mail/apache-james-3.4/james-server-app-3.4.0/conf # ll -d > >> keystore.privateFiles > >> drwxrwxrwx 2 james mail 4096 Dec 27 22:11 keystore.privateFiles > >> > >> It is a puzzler (at least to me) why adding execute permissions would > >> allow James to find my Keystore file. But I immediately hit another > >> snag, James is not correctly applying the password to access the > >> keystore file and throwing another exception. The smtpserver.xml is > >> configured as follows - > >> > >> <tls socketTLS="false" startTLS="true"> > >> > >> > <keystore>file:/mail/apache-james-3.4/james-server-app-3.4.0/conf/keystore</keystore> > >> <secret>mypassword</secret> > >> > >> <provider>org.bouncycastle.jce.provider.BouncyCastleProvider</provider> > >> <algorithm>SunX509</algorithm> > >> </tls> > >> > >> To test the keystore file I executed the following command and it worked > >> fine - > >> > >> quasar:/mail/apache-james-3.4/james-server-app-3.4.0/conf # keytool > >> -list -v -keystore > >> /mail/apache-james-3.4/james-server-app-3.4.0/conf/keystore -storetype > >> PKCS12 -storepass mypassword > >> > >> The exception and stack walkback I am getting from James is shown > >> below. I am continuing to Google for answers but no joy so far... > >> Anyone here got any ideas? Thanks again in advance... Marc. > >> > >> ---- > >> > >> INFO | jvm 1 | 2019/12/28 17:13:27 | WARN 17:13:26,963 | > >> org.apache.james.container.spring.context.JamesServerApplicationContext > >> | Exception encountered during context initialization - cancelling > >> refresh attempt: > >> org.springframework.beans.factory.BeanCreationException: Error creating > >> bean with name 'smtpserver': Invocation of init method failed; nested > >> exception is java.io.IOException: keystore password was incorrect > >> INFO | jvm 1 | 2019/12/28 17:13:27 | INFO 17:13:26,964 | > >> org.apache.james.mailetcontainer.impl.JamesMailSpooler | start > dispose() ... > >> INFO | jvm 1 | 2019/12/28 17:13:27 | INFO 17:13:26,964 | > >> org.apache.james.mailetcontainer.impl.JamesMailSpooler | thread shutdown > >> completed. > >> INFO | jvm 1 | 2019/12/28 17:13:27 | > >> INFO | jvm 1 | 2019/12/28 17:13:27 | WrapperSimpleApp: Encountered > >> an error running main: > >> org.springframework.beans.factory.BeanCreationException: Error creating > >> bean with name 'smtpserver': Invocation of init method failed; nested > >> exception is java.io.IOException: keystore password was incorrect > >> INFO | jvm 1 | 2019/12/28 17:13:27 | > >> org.springframework.beans.factory.BeanCreationException: Error creating > >> bean with name 'smtpserver': Invocation of init method failed; nested > >> exception is java.io.IOException: keystore password was incorrect > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:133) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:396) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1507) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:638) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:942) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:93) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.apache.james.container.spring.context.JamesServerApplicationContext.<init>(JamesServerApplicationContext.java:40) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.apache.james.app.spring.JamesAppSpringMain.init(JamesAppSpringMain.java:56) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.apache.james.app.spring.JamesAppSpringMain.main(JamesAppSpringMain.java:42) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> java.lang.reflect.Method.invoke(Method.java:498) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> java.lang.Thread.run(Thread.java:748) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | Caused by: > >> java.io.IOException: keystore password was incorrect > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2059) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:238) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:70) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> java.security.KeyStore.load(KeyStore.java:1445) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.apache.james.protocols.lib.netty.AbstractConfigurableAsyncServer.buildSSLContext(AbstractConfigurableAsyncServer.java:405) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.apache.james.protocols.lib.netty.AbstractConfigurableAsyncServer.init(AbstractConfigurableAsyncServer.java:263) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.apache.james.protocols.lib.netty.AbstractServerFactory.init(AbstractServerFactory.java:57) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> java.lang.reflect.Method.invoke(Method.java:498) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:344) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:295) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | at > >> > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130) > >> INFO | jvm 1 | 2019/12/28 17:13:27 | ... 22 more > >> INFO | jvm 1 | 2019/12/28 17:13:27 | Caused by: > >> java.security.UnrecoverableKeyException: failed to decrypt safe contents > >> entry: java.lang.IllegalStateException: password has been cleared > >> INFO | jvm 1 | 2019/12/28 17:13:27 | ... 36 more > >> > >> -- > >> > >> --... ...-- .----. ... -.. . .-- .- --... .--. -..- > .-- -- .- .-. -.-. > >> > >> > >> *Computers: the final frontier. These are the voyages of the user Marc. > >> His mission: to explore strange new hardware. To seek out new software > >> and new applications. > >> To boldly go where no Marc has gone before! > >> * > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > -- > > --... ...-- .----. ... -.. . .-- .- --... .--. -..- .-- > -- .- .-. -.-. > > > *Computers: the final frontier. These are the voyages of the user Marc. > His mission: to explore strange new hardware. To seek out new software > and new applications. > To boldly go where no Marc has gone before! > * >
