Comparing your setup with my notes, I noticed that I also do the
following on the *server*:
# sudo -u backup-user ssh client
And then abort after accepting the host key. Did you do this as well?
Otherwise try to run amrecover -o debug_auth=1 on the client and check
the log files. You should now
Amanda has been happily backing up a half dozen Linux machines for a week now
and I can even restore files using amrecover on the server. That part was easy.
Using the instructions I found on zmanda wiki -- with appropriate adjustments
for my environment -- I tried to get amrecover over ssh