Hi Matt,

Let me answer you this time below in each of your points :)

Cheers,

Rene.

On 1/9/24 21:27, cryptearth wrote:
Hello Rene,

thank you for the kind words.

May let me answer your points:

- Spring vs. Guice: I use James since the Beta of 3.0 and if I recall correctly back then the Spring package was the one to use. By the usual "never change a running system" I not changed since. I had a quick look into it, but unfortunately the links on the install instructions https://james.apache.org/server/install.html only lead to a 404 - so there's something missing. Also I seem to require docker - which I don't have any experience with. I'd rather prefer just to build a package, unpack it and start it without having to deal with any extra containerization around it. It seems I require some help to setup a guice install.

Regarding the 404 on the install page, it looks like you are right. It's not normal and I might take a look at it, thanks for the feedback! However, docker is one way, but like Sprint you can still just run a jar with Guice as well. You can see the download page for example (where the links seem to work) : https://james.apache.org/download.cgi .

If you want to try to switch to Guice, the JPA james server is the one for you, that you can plug with MariaDB. The distributed version is for heavy deployments (which I don't think interests you here) with cassandra, opensearch, rabbitmq, ... => https://www.apache.org/dyn/closer.lua/james/server/3.8.0/james-server-jpa-guice.zip . You will see in the README as well in the zip some basic instructions to run it, with docker or manually with the jar like you are used to.

I totally understand the reason behind the fact that you started with Spring, so far it works, and it's safe to stick to it. Quite a fair statement actually. The problem is that though the main development efforts since a while are being done on Guice, and the Spring one is quickly maintained, as nobody seems fully willing to keep developing on it now. It works but could have potentially some vulnerabilities... I'm thinking for example the spring dependencies, that are old and outdated I think^^'


- as for using MariaDB: This goes back to 2015 when I started to rent my server and own domain. Back then I had basically no knowledge about server administration let alone set up a mail server. Although there're complete guides on how to setup postfix with dovecot and such on opensuse I not managed to get it working. So I looked for a simple complete solution and ended up using Citadel/UX. For some reason with some update it broke to a state I wasn't able to start it again. Hence I wasn't able to get any data out of it as for some reason I just was not able to access the BerkeleyDB data file. As I also wasn't able to get the regular stuff with maildir running I just wanted a setup using a regular MySQL not just for config- and user-data but also as the mail storage backend so that in the event I encountered such a situation again I can just use regular tools to restore a mysql database. As I'm also a hobbyist Java developer for me it was a simple idea of "there has to be some simple mail server implemented in Java" - and this is how I ended up discovering and using James ever since.

It also already came into play that for some reason I had to manually restore my database - exactly the reason why I chose MySQL as storage backend. As it was helpful for the reason I chose it I rather stick to it for that reason. The switch from MySQL to MariaDB was just something OpenSuSE did - so I followed it. As a user I not noticed any differences.

As for the database issues mentioned: For some reason I had to set up the MySQL- and later the MariaDB-repos instead of using what comes with the OpenSuSE repos. I don't know what the difference was but it prevented the initial setup of the database structure on first run. Hence I'm glad that this is now fixed - whoever is reponsible for it.
I get this :) I was just trying to inform you that these days we work also on a Postgresql exclusive implementation for James. JPA is good to be able to plug to a lot of different SQL databases with the same code, but performances can be a bit meh. We are working on a full postgresql implementation of James in parallel as most people using JPA in the community are using Postgresql as their database (as it is quite a popular one). The goal is to have better performance here than with JPA for this particular db. But likely JPA might be enough for your singular usage. Was just throwing that here if maybe it was interesting for you.


- sync between main server and backup: I guess the additional forward from the main server to the backup and handle them independently seems like a neat idea simple to implement. I mostly rely on my backup when doing maintenance on the root like OS updates which require a system reboot. It's not I want to try to build a distributed synchronized cluster. I guess for that there're better solutions out there. Looking thru the docs there's something pointing to https://github.com/apache/james-project/blob/james-project-3.8.0/docs/modules/servers/pages/distributed/run-docker.adoc - but that doesn't exists or isn't set to public, same as for the guice setup. So it seems there's documentation for what I want to do - but it's not available to me.

That new Antora documentation is still incomplete and yeah there is issues for some reasons with the distributed part that does not render each time during build... Might need tot ake an other look at it! For the temporary link of it though if it interests you: https://james.staged.apache.org/james-project/3.8.0/index.html

Distributed doc in the source code though if you feel like building it yourself: https://github.com/apache/james-project/tree/master/server/apps/distributed-app/docs

I think yes forwarding your mails to both your main and backup servers is simple and efficient :)


Have a good one.

Matt

On 09.01.24 08:44, Rene Cordier wrote:
Hello,

Interesting read thanks.

Few remarks though, but feel free to agree or not :) :

- any reason to still run the spring version of James? That version I believe is not always up-to-date with dependencies (specially the sprint dependencies). Truth is it might get deprecated and removed in later release if a consensus is reached, as all active dev is done on the Guice versions now. You might want to check that (JPA Guice here probably closest to the Spring version)

- note that there is active development at the moment (still WIP) regarding a Postgresql reactive implementation with james (Guice). I saw you seem to use MariaDB but might interest you for the future maybe?

- for sync between your main server and backup, maybe just redirect the mails to both platforms, and let them treat the mails separately? If not I guess db backup and sync between the two could do it too.

Good job on running latest James on the new OpenSuse though, thanks for sharing!

Rene.

On 1/9/24 10:17, cryptearth wrote:
Hello there everybody,

happy new year 2024 to all of you.
It's been quite some time since I was last active on this maillist. After upgrading my server to OpenSuSE 15.5 and James to 3.8 and after reading the recent topics on the web archive I decided to post a small round up how it
went this way.

Important word of warning and security:
This is only a personal write-down how I did my setup. I'm aware there're still some security improvements to be done. This is meant as an informative guide. I'm by no means responsible for any security issues that may arise by following this. I myself got hacked by the Log4J issue mostly due to not applying the available update fast enough. It's your own responsibility to make your setup secure. All I can offer are some guidelines from personal
experience.

That warning out now let's get to the real topic.

As several of the active devs here know I use James for quite some time now and over the years encountered several issues mostly specific to OpenSuSE and the packages provided by its repositories. Luckily this time everything went
smoothly.

I started from a fresh clean "server"-type install, but this should be
applicable to any existing system as well. So the first step is to fully update
the new system.

As OpenSuSE comes with postfix I replace it with sendmail for later. I guess
it's also possible with postfix or exim or any other MTA providing the
required sendmail dependency. Using sendmail was just the simplest option I
figured out when I first used it.
The keyword here is "nullclient": It configures sendmail to just forward everything to James without any handling itself. For this sendmail comes with an almost ready-to-use config which only needs one change in /etc/sendmail/
linux.nullclient.mc in line 27 by replacing the example
"mailhub.domain.notused" with "localhost". Then just generate the new
sendmail.cf from it (either as root or via sudo):
m4 /etc/mail/linux.nullclient.mc > /etc/sendmail.cf

I also have to edit the systemd script. For some reason the systemctl edit command works a bit unexpected for simple edits so by appending --full it
becomes a full override. The service file needs two changes:

1) moving the PIDFile from /var/run to /run as hinted by systemctl status
message:
PIDFile=/run/sendmail.pid

2) remove the -bd flag from the start args to prevent sendmail from blocking
port TCP/25:
Environment="SENDMAIL_ARGS=-L -Am -q30m -om"

Check if sendmail service is enabled and started: systemctl status sendmail At least enabled it so it gets started at reboot. As James isn't setup yet to take its mails don't start it yet - it could cause a hang into an infinite
loop.

Before get to James I prepare the rest of the system by install these packages
(can be installed along when switching postfix for sendmail):
- java-17-openjdk-devel
- git
- maven
- mariadb
- phpMyAdmin-apache
- yast2-http-server
The required dependencies get pulled in by the resolver. After installing the packages YaST tells you to secure MariaDB. So I do this right away by running mariadb-secure-installation as root/sudo and set a root password. All other
questions can be answered with thier default option (just hit enter).

To get phpMyAdmin running only the initial setup using "yast http-server" is required. Select enable PHP in the 2nd step and complete the setup by just
hitting F10 a couple times. If the Apache2 server is accessible via the
internet care should be taken by limiting phpMyAdmin to localhost. Edit /etc/ apache2/conf.d/phpMyAdmin.conf line 27 to "Require local" and use ssh tunnels. If you have services like a vpn or a tor hidden service running you may want to secure it even further with something like an .htaccess or similar. This way you prevent attackers from trying to bruteforce your database password.

Complete the pre-setup by use phpMyAdmin to add a new user for James.

Now let's build James:
git clone https://github.com/apache/james-project -b james-project-3.8.0
cd james-project
mvn clean package -DskipTests -T 1C
As I use the spring package I then unpack the tar in james-project/server/
apps/spring-app/target/james-server-spring-app-3.8.0-app.tar.gz.
I also need to get the mariadb-connector/J: https://mariadb.com/downloads/
connectors/ and place the .jar in the james-server-spring-app-3.8.0/lib
folder.
I'm not sure why this isn't distributed along or part of the package manifest
to pull from the repos - guess this is some licence nonesense.

It's config time now - and there's not that much to do.
As I don't use POP3 the first I do is disable the POP3 server.
Next I set up the database stuff by change the driver to
org.mariadb.jdbc.Driver
and the connector-string to
jdbc:mariadb://localhost:3306/james
Btw: As pretty much any modern JDBC driver should be a type 4.x there's no
need to specify the driver class name or do something like
Class.forName()
As long as the driver jar is in classpath on startup of the JVM it gets pulled
in via the ServiceLoader-API.

Next I set my domain and the greetings in the imap and smtp servers. I also enable StartTLS for both imap and smtp. As for outgoing smtp I also enable StartTLS by adding this to the RemoteDelievery in the mailetcontainer.xml:
<startTLS>true</startTLS>
This should be added by default as it's only mentioned in the RemoteDelievery
javadoc but not in the XML doc nor in the config doc.
Until now I used a Java keystore for the certificate and key. But using the new
PEM files directly skips this. Thanks for who added this.

The last part is to deal with sendmail and apache2/php mail. For this I first add two users to James (along with others like postmaster): webmaster and local-service. Then I add this to the mailetcontainer.xml right at the top of
the root processor:

         <matcher name="local-service"
match="org.apache.james.mailetcontainer.impl.matchers.And">
                 <matcher match="RemoteAddrInNetwork=127.0.0.0/8" />
                 <matcher
match="org.apache.james.mailetcontainer.impl.matchers.Not">
                         <matcher match="SMTPAuthSuccessful" />
                 </matcher>
                 <matcher
match="org.apache.james.mailetcontainer.impl.matchers.Not">
                         <matcher match="SenderIs=webmas...@cryptearth.de" />
                 </matcher>
                 <matcher
match="org.apache.james.mailetcontainer.impl.matchers.Not">
                         <matcher match="RecipientIs=local-
serv...@cryptearth.de" />
                 </matcher>
         </matcher>

         <mailet match="local-service" class="Redirect">
<to>local-serv...@cryptearth.de</to>
         </mailet>

To explain its purpose:
When starting James via crontab by root crond sends a start report via
sendmail. In order for sendmail to not get stuck in an endless loop trying to deliver the crond status I have to provide some drop off destination for it.
But as I also want to send e-mails by PHP I had to get a bit creative
filtering. It basically goes like this:

IF incoming mail comes from localhost
AND is NOT authenticated via smtp auth
AND is NOT sent by webmaster (from php)
AND is NOT already redirected to local-service
THEN redirect to local-service

This rule maybe can be written better - but it was the simplest I came up
with. The check for the recipient is required so the mail doesn't get
redirected to itself again. The check for the webmaster is required as
sendmail drops off php mails with wwwrun@localhost, which is overriden by the - f parameter to webmaster. Any other client that regular logs in via AUTH is
ignored anyway. And the whole thing is limited to localhost.

Finally, add a
@reboot /path/to/james/bin/james start
to roots crontab for james autostart.

If everything was done correctly james should start up just fine for the first time (it actually already had to for adding the users - but this can be done
now).
I do the first start via "console" to verify everything is done correctly. After all is done and james starts correctly I just reboot the system. Now when all comes up by itself the local-service user should have a mail confirm the successful start of James which also confirms that the sendmail nullclient
and the mailet redirect works.

A common failure can be an issue with name resolution. In this case you have to manually set /etc/hosts and reboot again. This can also be done early on as
one of the first few steps after updating the system.

To send mails from apache this can be done like this:

<?php
         $to="cryptea...@googlemail.com";
         $subject=$content="this is a apache webmail test";
         $headers = array(
                 'From' => 'CryptEarth.de Webmaster <webmas...@cryptearth.de>',
                 'Reply-To' => 'webmas...@cryptearth.de',
                 'X-Mailer' => 'PHP/' . phpversion()
         );
         mail($to, $subject, $content, $headers, "-fwebmas...@cryptearth.de");
?>

A few improvements that can (and should) be done:

- don't start james as root but rather move the ports to 8025 and 8143 and use a firewall rule to redirect 25 to 8025 and 143 to 8143 ... but I have to figure
that out
- maybe rethink about sendmail nullclient and do similar with postfix or exim
- maybe redo the mailet redirect for local-service and php-mail
- setting up DKIM on outgoing mails

Currently I have James running on two servers: my main root as primary and on a small sbc at home as a backup. Is there a way to synchronize the two servers on the application level. Or would it be enough to just synchronize the two datases secured by a ssh tunnel? How to handle the differences in the counters when both servers receive an e-mail each which have to get synchronized in
both ways?

Have a nice week.
Greetings from Germany

Matt



---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Reply via email to