I tried to use the instructions here: https://bugs.quassel-irc.org/projects/quassel-irc/wiki/Client-Core_SSL_support but could not get it to work.
I ended up having to remove all the /var/lib/quassel/ files and running dpkg-reconfigure quassel-core and manually recreating my accounts but that seems to have fixed it.