RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

Thanks Eric, I'll make that change.

-Original Message-
From: Eric Broch [mailto:ebr...@whitehorsetc.com] 
Sent: Monday, August 17, 2020 9:21 AM
To: qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

In your .qmail-default file for the domain it's recommended to use 'delete' 
instead of 'bounce-no-mailbox'


On 8/17/2020 8:14 AM, Chas Hockenbarger wrote:
> Thanks, Angus.  I searched the whole system for a .forward and there aren't 
> any on the system I can find.
>
> I'm not seeing anything that is obvious here.  I appreciate all the feedback 
> and help, there were definitely suggestions made I hadn't chased yet.  I'm 
> perplexed to say the least.  I deleted all the messages from the bounce queue 
> and will see if that rectifies the situation or not.  I'm watching this 
> system like a hawk so hopefully if something that is more 'normal' looking is 
> going on I'll be able to catch it.
>
> If I find the culprit I'll absolutely update this thread.  If anyone has any 
> other ideas, I'd love to hear them as well.
>
> -Original Message-
> From: Angus McIntyre [mailto:an...@pobox.com]
> Sent: Monday, August 17, 2020 5:44 AM
> To: qmailtoaster-list@qmailtoaster.com; Chas Hockenbarger 
> Subject: Re: [qmailtoaster] Distressing strange behavior
>
> Check for a '.forward' file in '/root'?
>
> That could account for the status report going somewhere other than where 
> it's supposed to, but might not explain the other issues you're seeing.
>
> Angus
>
>
>
> Chas Hockenbarger wrote on 8/16/20 6:09 PM:
>> I just got another piece of information.  I got a failure message a
>> few hours ago to the postmaster account for this domain that a message
>> from root to root was not delivered to 5 different Gmail accounts.
>> The email was the cron.daily status report.  There is no way that
>> should be going to these Gmail accounts.  They are accounts I don’t
>> know and root at this server is supposed to go to postmaster.
>>
>> This just keeps getting weirder.
>>
>> *From:* Eric Broch [mailto:ebr...@whitehorsetc.com]
>> *Sent:* Sunday, August 16, 2020 4:13 PM
>> *To:* qmailtoaster-list@qmailtoaster.com
>> *Subject:* Re: [qmailtoaster] Distressing strange behavior
>>
>> Yes forwards can be in a .qmail file or in the vpopmail database.
>>
>> So, the bounces occurring presently, what's the originating account?
>>
>> Is there anything in your queue (# qmailctl queue)?
>>
>> On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:
>>
>>  As I understand the forwards setup in qmailadmin those are in the
>>  database, right?
>>
>>  The address that was compromised hasn't sent any email since the
>>  password change.
>>
>>  I hadn't thought about looking at qmail-inject. I'll dig into
>>  watching that part of the process.
>>
>>  Get TypeApp for Android <http://www.typeapp.com/r?b=15986>
>>
>>  On Aug 16, 2020, at 3:14 PM, Eric Broch >  <mailto:ebr...@whitehorsetc.com>> wrote:
>>
>>  How do you have your forwards set up?
>>
>>  Is there any mail in your queue?
>>
>>  If someone hacked an account on your server with forwards to
>>  gmail accounts they aren't limited to just these forwards, they
>>  also have the option in the email client to add gmail accounts
>>  in the "To:" field of the email they're sending, thus bounces
>>  from gmail accounts that aren't in your forwards file.
>>
>>  Also, qmail-inject puts mail in the queue and you'll see it in
>>  the send log.
>>
>>  On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:
>>
>>  I'm hoping someone has encountered this weird behavior or
>>  something like it before and can point me down a path,
>>  because all my research has turned up nothing so far.
>>
>>  I had an email account recently get breached due to a
>>  re-used password, and that account was used to send a bunch
>>  of spam out from a server I help manage.  We changed the
>>  password on the account as soon as we found it happening and
>>  the outbound flood stopped.
>>
>>  Shortly after that, however, I started seeing a very, very
>>  strange behavior.  Sometimes, and I haven’t yet been able to
>&

Re: [qmailtoaster] Distressing strange behavior

[Eric Broch]

In your .qmail-default file for the domain it's recommended to use 
'delete' instead of 'bounce-no-mailbox'



On 8/17/2020 8:14 AM, Chas Hockenbarger wrote:

Thanks, Angus.  I searched the whole system for a .forward and there aren't any 
on the system I can find.

I'm not seeing anything that is obvious here.  I appreciate all the feedback 
and help, there were definitely suggestions made I hadn't chased yet.  I'm 
perplexed to say the least.  I deleted all the messages from the bounce queue 
and will see if that rectifies the situation or not.  I'm watching this system 
like a hawk so hopefully if something that is more 'normal' looking is going on 
I'll be able to catch it.

If I find the culprit I'll absolutely update this thread.  If anyone has any 
other ideas, I'd love to hear them as well.

-Original Message-
From: Angus McIntyre [mailto:an...@pobox.com]
Sent: Monday, August 17, 2020 5:44 AM
To: qmailtoaster-list@qmailtoaster.com; Chas Hockenbarger 
Subject: Re: [qmailtoaster] Distressing strange behavior

Check for a '.forward' file in '/root'?

That could account for the status report going somewhere other than where it's 
supposed to, but might not explain the other issues you're seeing.

Angus



Chas Hockenbarger wrote on 8/16/20 6:09 PM:

I just got another piece of information.  I got a failure message a
few hours ago to the postmaster account for this domain that a message
from root to root was not delivered to 5 different Gmail accounts.
The email was the cron.daily status report.  There is no way that
should be going to these Gmail accounts.  They are accounts I don’t
know and root at this server is supposed to go to postmaster.

This just keeps getting weirder.

*From:* Eric Broch [mailto:ebr...@whitehorsetc.com]
*Sent:* Sunday, August 16, 2020 4:13 PM
*To:* qmailtoaster-list@qmailtoaster.com
*Subject:* Re: [qmailtoaster] Distressing strange behavior

Yes forwards can be in a .qmail file or in the vpopmail database.

So, the bounces occurring presently, what's the originating account?

Is there anything in your queue (# qmailctl queue)?

On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:

 As I understand the forwards setup in qmailadmin those are in the
 database, right?

 The address that was compromised hasn't sent any email since the
 password change.

 I hadn't thought about looking at qmail-inject. I'll dig into
 watching that part of the process.

 Get TypeApp for Android <http://www.typeapp.com/r?b=15986>

 On Aug 16, 2020, at 3:14 PM, Eric Broch mailto:ebr...@whitehorsetc.com>> wrote:

 How do you have your forwards set up?

 Is there any mail in your queue?

 If someone hacked an account on your server with forwards to
 gmail accounts they aren't limited to just these forwards, they
 also have the option in the email client to add gmail accounts
 in the "To:" field of the email they're sending, thus bounces
 from gmail accounts that aren't in your forwards file.

 Also, qmail-inject puts mail in the queue and you'll see it in
 the send log.

 On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:

 I'm hoping someone has encountered this weird behavior or
 something like it before and can point me down a path,
 because all my research has turned up nothing so far.

 I had an email account recently get breached due to a
 re-used password, and that account was used to send a bunch
 of spam out from a server I help manage.  We changed the
 password on the account as soon as we found it happening and
 the outbound flood stopped.

 Shortly after that, however, I started seeing a very, very
 strange behavior.  Sometimes, and I haven’t yet been able to
 identify the trigger or pattern, when users on this server
 send email to a forward that contains around 50 or so email
 addresses (they use it like a private distribution list)
 they will get anywhere from 1-10 bounces from Gmail.  Not
 every email sent to the forward has this happen, and not
 even every email from a particular user.

 The outbound spamming caused the server’s reputation to go
 in the tank with Google, and if it weren’t for that, I
 wouldn’t know this was happening, because they get the
 bounces from Gmail accounts that absolutely ARE NOT in the
 forward or part of the email chain AT ALL.

 I’m kind of freaking out here because while I haven’t found
 a breach of the actual server / OS, this feels like someone
 has been able to inject som

RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

Thanks, Angus.  I searched the whole system for a .forward and there aren't any 
on the system I can find.

I'm not seeing anything that is obvious here.  I appreciate all the feedback 
and help, there were definitely suggestions made I hadn't chased yet.  I'm 
perplexed to say the least.  I deleted all the messages from the bounce queue 
and will see if that rectifies the situation or not.  I'm watching this system 
like a hawk so hopefully if something that is more 'normal' looking is going on 
I'll be able to catch it.

If I find the culprit I'll absolutely update this thread.  If anyone has any 
other ideas, I'd love to hear them as well.

-Original Message-
From: Angus McIntyre [mailto:an...@pobox.com] 
Sent: Monday, August 17, 2020 5:44 AM
To: qmailtoaster-list@qmailtoaster.com; Chas Hockenbarger 
Subject: Re: [qmailtoaster] Distressing strange behavior

Check for a '.forward' file in '/root'?

That could account for the status report going somewhere other than where it's 
supposed to, but might not explain the other issues you're seeing.

Angus



Chas Hockenbarger wrote on 8/16/20 6:09 PM:
> I just got another piece of information.  I got a failure message a 
> few hours ago to the postmaster account for this domain that a message 
> from root to root was not delivered to 5 different Gmail accounts.  
> The email was the cron.daily status report.  There is no way that 
> should be going to these Gmail accounts.  They are accounts I don’t 
> know and root at this server is supposed to go to postmaster.
> 
> This just keeps getting weirder.
> 
> *From:* Eric Broch [mailto:ebr...@whitehorsetc.com]
> *Sent:* Sunday, August 16, 2020 4:13 PM
> *To:* qmailtoaster-list@qmailtoaster.com
> *Subject:* Re: [qmailtoaster] Distressing strange behavior
> 
> Yes forwards can be in a .qmail file or in the vpopmail database.
> 
> So, the bounces occurring presently, what's the originating account?
> 
> Is there anything in your queue (# qmailctl queue)?
> 
> On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:
> 
> As I understand the forwards setup in qmailadmin those are in the
> database, right?
> 
> The address that was compromised hasn't sent any email since the
> password change.
> 
> I hadn't thought about looking at qmail-inject. I'll dig into
> watching that part of the process.
> 
> Get TypeApp for Android <http://www.typeapp.com/r?b=15986>
> 
> On Aug 16, 2020, at 3:14 PM, Eric Broch  <mailto:ebr...@whitehorsetc.com>> wrote:
> 
> How do you have your forwards set up?
> 
> Is there any mail in your queue?
> 
> If someone hacked an account on your server with forwards to
> gmail accounts they aren't limited to just these forwards, they
> also have the option in the email client to add gmail accounts
> in the "To:" field of the email they're sending, thus bounces
> from gmail accounts that aren't in your forwards file.
> 
> Also, qmail-inject puts mail in the queue and you'll see it in
> the send log.
> 
> On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:
> 
> I'm hoping someone has encountered this weird behavior or
> something like it before and can point me down a path,
> because all my research has turned up nothing so far.
> 
> I had an email account recently get breached due to a
> re-used password, and that account was used to send a bunch
> of spam out from a server I help manage.  We changed the
> password on the account as soon as we found it happening and
> the outbound flood stopped.
> 
> Shortly after that, however, I started seeing a very, very
> strange behavior.  Sometimes, and I haven’t yet been able to
> identify the trigger or pattern, when users on this server
> send email to a forward that contains around 50 or so email
> addresses (they use it like a private distribution list)
> they will get anywhere from 1-10 bounces from Gmail.  Not
> every email sent to the forward has this happen, and not
> even every email from a particular user.
> 
> The outbound spamming caused the server’s reputation to go
> in the tank with Google, and if it weren’t for that, I
> wouldn’t know this was happening, because they get the
> bounces from Gmail accounts that absolutely ARE NOT in the
> forward or part of the email chain AT ALL.
> 
> I’m kind of fr

Re: [qmailtoaster] Distressing strange behavior

[Angus McIntyre]

Check for a '.forward' file in '/root'?

That could account for the status report going somewhere other than 
where it's supposed to, but might not explain the other issues you're 
seeing.


Angus



Chas Hockenbarger wrote on 8/16/20 6:09 PM:
I just got another piece of information.  I got a failure message a few 
hours ago to the postmaster account for this domain that a message from 
root to root was not delivered to 5 different Gmail accounts.  The email 
was the cron.daily status report.  There is no way that should be going 
to these Gmail accounts.  They are accounts I don’t know and root at 
this server is supposed to go to postmaster.


This just keeps getting weirder.

*From:* Eric Broch [mailto:ebr...@whitehorsetc.com]
*Sent:* Sunday, August 16, 2020 4:13 PM
*To:* qmailtoaster-list@qmailtoaster.com
*Subject:* Re: [qmailtoaster] Distressing strange behavior

Yes forwards can be in a .qmail file or in the vpopmail database.

So, the bounces occurring presently, what's the originating account?

Is there anything in your queue (# qmailctl queue)?

On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:

As I understand the forwards setup in qmailadmin those are in the
database, right?

The address that was compromised hasn't sent any email since the
password change.

I hadn't thought about looking at qmail-inject. I'll dig into
watching that part of the process.

Get TypeApp for Android <http://www.typeapp.com/r?b=15986>

On Aug 16, 2020, at 3:14 PM, Eric Broch mailto:ebr...@whitehorsetc.com>> wrote:

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to
gmail accounts they aren't limited to just these forwards, they
also have the option in the email client to add gmail accounts
in the "To:" field of the email they're sending, thus bounces
from gmail accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in
the send log.

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:

I'm hoping someone has encountered this weird behavior or
something like it before and can point me down a path,
because all my research has turned up nothing so far.

I had an email account recently get breached due to a
re-used password, and that account was used to send a bunch
of spam out from a server I help manage.  We changed the
password on the account as soon as we found it happening and
the outbound flood stopped.

Shortly after that, however, I started seeing a very, very
strange behavior.  Sometimes, and I haven’t yet been able to
identify the trigger or pattern, when users on this server
send email to a forward that contains around 50 or so email
addresses (they use it like a private distribution list)
they will get anywhere from 1-10 bounces from Gmail.  Not
every email sent to the forward has this happen, and not
even every email from a particular user.

The outbound spamming caused the server’s reputation to go
in the tank with Google, and if it weren’t for that, I
wouldn’t know this was happening, because they get the
bounces from Gmail accounts that absolutely ARE NOT in the
forward or part of the email chain AT ALL.

I’m kind of freaking out here because while I haven’t found
a breach of the actual server / OS, this feels like someone
has been able to inject something somewhere into my server
that I simply can’t find.  It is especially troubling
because a user who is not on this domain, but is part of the
group and therefore uses the forward from time to time, sent
something to the forward today and got Gmail bounces.

I don’t see anything in the send log that shows the server
even trying to send to Gmail, which only adds to the ghost
story.

Any ideas, paths to go down, anything would be greatly
appreciated here.  I’m about to just rebuild the whole thing
from scratch on a new VM, but if I’m overlooking something
simple don’t want to put the users through that.

Thanks in advance.

Chas



-
To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com
For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com

Re: [qmailtoaster] Distressing strange behavior

[Eric Broch]

okay, so no other seeming way of forwarding.

On 8/16/2020 7:10 PM, Chas Hockenbarger wrote:


There are only 2 files found.  One is in a user’s directory, and the 
file contains this line:


/Maildir

The other is in the top of the domain, labeled .qmail-default, which 
contains


| /home/vpopmail/bin/vdelivermail ‘’ bounce-no-mailbox

*From:*Eric Broch [mailto:ebr...@whitehorsetc.com]
*Sent:* Sunday, August 16, 2020 7:40 PM
*To:* qmailtoaster-list@qmailtoaster.com
*Subject:* Re: [qmailtoaster] Distressing strange behavior

Do this:

# ls -la /home/vpopmail/domains/'mydomain'/postmaster/

look for a .qmail file.

In fact you could do this

# find /home/vpopmail/domains/ -name ".qmail*"

The .qmail is also a way to forward.

On 8/16/2020 4:49 PM, Chas Hockenbarger wrote:

So I looked at a few of the files in the bounce folder and every
one of them is bounces back from Gmail for either bad addresses or
just the reputation bounce.

Is there a down side to just blowing those away?

*From:*Remo Mattei [mailto:r...@mattei.org]
*Sent:* Sunday, August 16, 2020 5:43 PM
*To:* qmailtoaster-list@qmailtoaster.com
<mailto:qmailtoaster-list@qmailtoaster.com>
*Subject:* Re: [qmailtoaster] Distressing strange behavior

BTW, I always use the -L on the qmHandle it should not change much
but my 2 cents.

Remo




On Aug 16, 2020, at 3:32 PM, Chas Hockenbarger
mailto:chash...@gmail.com>> wrote:

Yes, I did check those, that was my first thought is that the
server had been compromised and someone modified those files
to do some weird thing.   However,

.qmail-root has one line &postmaster@

.qmail-postmaster has one line &postmaster@

.qmail-mailer-daemon has one line &postmaster@

I see no other files in that directory.

One more piece of info I just discovered.  Even though
qmHandle –l reports 0 messages in either the remote or local
queue, the bounce queue directory has over 2000 messages in it.

Could that be a contributing factor here?  I don’t see how
that would create random emails going to Gmail accounts from
(seemingly) random other messages, but is it possible
something is borked up in the queue processing there since
Gmail is bouncing everything back to me?

*From:*Remo Mattei [mailto:r...@mattei.org]
*Sent:*Sunday, August 16, 2020 5:26 PM
*To:*qmailtoaster-list@qmailtoaster.com
<mailto:qmailtoaster-list@qmailtoaster.com>
    *Subject:*Re: [qmailtoaster] Distressing strange behavior

did you check your qmail aliases?

cd /var/qmail/alias/

what do those files say?





On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger
mailto:chash...@gmail.com>> wrote:

Thanks, Boheme, and yes that’s a problem, but it’s a
symptom of this problem. Emails are going to Gmail
accounts when users aren’t sending them.  Legit emails to
Gmail accounts are definitely getting bounced, too, which
I have to deal with later.  If I can’t stop this weird
spamming to them, I can’t recover the reputation.

*From:*Boheme [mailto:boh...@gmail.com]
*Sent:*Sunday, August 16, 2020 4:59 PM
*To:*qmailtoaster-list@qmailtoaster.com
<mailto:qmailtoaster-list@qmailtoaster.com>
    *Subject:*Re: [qmailtoaster] Distressing strange behavior

It doesn’t sound like you are being repeatedly hacked. It
sounds like your reputation dropped with google, and
certain emails trigger their anti-spam filtering now. Not
all of them, just some. I have problems with Google
accepting email regularly sometimes, and dropping other
emails into people’s spam folders, as a result of too many
of my users forwarding email to google and those forwards
passing along a lot of spam to their addresses on my server.

-Sent from my Pip-Boy 3000






On 17/08/2020, at 8:46 AM, Charles Hockenbarger
mailto:chash...@gmail.com>> wrote:



As I understand the forwards setup in qmailadmin those
are in the database, right?

The address that was compromised hasn't sent any email
since the password change.

I hadn't thought about looking at qmail-inject. I'll
dig into watching that part of the process.

GetTypeApp for Android <http://www.typeapp.com/r?b=15986>

On Aug 16, 2020, at 3:14 PM, Eric Broch
mailto:ebr...@whitehorsetc.com>> wrote:

How do you have your forwards set up?

Is ther

RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

There are only 2 files found.  One is in a user’s directory, and the file 
contains this line:

 

/Maildir

 

The other is in the top of the domain, labeled .qmail-default, which contains

 

| /home/vpopmail/bin/vdelivermail ‘’ bounce-no-mailbox

 

 

 

From: Eric Broch [mailto:ebr...@whitehorsetc.com] 
Sent: Sunday, August 16, 2020 7:40 PM
To: qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

Do this:

# ls -la /home/vpopmail/domains/'mydomain'/postmaster/

look for a .qmail file.

In fact you could do this

# find /home/vpopmail/domains/ -name ".qmail*"

 

The .qmail is also a way to forward.

On 8/16/2020 4:49 PM, Chas Hockenbarger wrote:

So I looked at a few of the files in the bounce folder and every one of them is 
bounces back from Gmail for either bad addresses or just the reputation bounce. 

 

Is there a down side to just blowing those away?  

 

From: Remo Mattei [mailto:r...@mattei.org] 
Sent: Sunday, August 16, 2020 5:43 PM
To: qmailtoaster-list@qmailtoaster.com 
<mailto:qmailtoaster-list@qmailtoaster.com> 
Subject: Re: [qmailtoaster] Distressing strange behavior

 

BTW, I always use the -L on the qmHandle it should not change much but my 2 
cents. 

 

Remo






On Aug 16, 2020, at 3:32 PM, Chas Hockenbarger mailto:chash...@gmail.com> > wrote:

 

Yes, I did check those, that was my first thought is that the server had been 
compromised and someone modified those files to do some weird thing.   However, 

 

.qmail-root has one line &postmaster@

.qmail-postmaster has one line &postmaster@

.qmail-mailer-daemon has one line &postmaster@

 

I see no other files in that directory.

 

One more piece of info I just discovered.  Even though qmHandle –l reports 0 
messages in either the remote or local queue, the bounce queue directory has 
over 2000 messages in it.   

 

Could that be a contributing factor here?  I don’t see how that would create 
random emails going to Gmail accounts from (seemingly) random other messages, 
but is it possible something is borked up in the queue processing there since 
Gmail is bouncing everything back to me?

 

From: Remo Mattei [mailto:r...@mattei.org] 
Sent: Sunday, August 16, 2020 5:26 PM
To: qmailtoaster-list@qmailtoaster.com 
<mailto:qmailtoaster-list@qmailtoaster.com> 
Subject: Re: [qmailtoaster] Distressing strange behavior

 

did you check your qmail aliases?

cd /var/qmail/alias/

 

what do those files say?

 







On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger < <mailto:chash...@gmail.com> 
chash...@gmail.com> wrote:

 

Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this problem.  
Emails are going to Gmail accounts when users aren’t sending them.  Legit 
emails to Gmail accounts are definitely getting bounced, too, which I have to 
deal with later.  If I can’t stop this weird spamming to them, I can’t recover 
the reputation.

 

From: Boheme [ <mailto:boh...@gmail.com> mailto:boh...@gmail.com] 
Sent: Sunday, August 16, 2020 4:59 PM
To:  <mailto:qmailtoaster-list@qmailtoaster.com> 
qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

It doesn’t sound like you are being repeatedly hacked. It sounds like your 
reputation dropped with google, and certain emails trigger their anti-spam 
filtering now. Not all of them, just some. I have problems with Google 
accepting email regularly sometimes, and dropping other emails into people’s 
spam folders, as a result of too many of my users forwarding email to google 
and those forwards passing along a lot of spam to their addresses on my server. 

-Sent from my Pip-Boy 3000








On 17/08/2020, at 8:46 AM, Charles Hockenbarger < <mailto:chash...@gmail.com> 
chash...@gmail.com> wrote:



As I understand the forwards setup in qmailadmin those are in the database, 
right?

The address that was compromised hasn't sent any email since the password 
change. 

I hadn't thought about looking at qmail-inject. I'll dig into watching that 
part of the process. 

Get  <http://www.typeapp.com/r?b=15986> TypeApp for Android

On Aug 16, 2020, at 3:14 PM, Eric Broch < <mailto:ebr...@whitehorsetc.com> 
ebr...@whitehorsetc.com> wrote:

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail accounts 
they aren't limited to just these forwards, they also have the option in the 
email client to add gmail accounts in the "To:" field of the email they're 
sending, thus bounces from gmail accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in the send log.

 

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 

I'm hoping someone has encountered this weird behavior or something like it 
before and can point me down a path, 

Re: [qmailtoaster] Distressing strange behavior

[Eric Broch]

Do this:

# ls -la /home/vpopmail/domains/'mydomain'/postmaster/

look for a .qmail file.

In fact you could do this

# find /home/vpopmail/domains/ -name ".qmail*"


The .qmail is also a way to forward.

On 8/16/2020 4:49 PM, Chas Hockenbarger wrote:


So I looked at a few of the files in the bounce folder and every one 
of them is bounces back from Gmail for either bad addresses or just 
the reputation bounce.


Is there a down side to just blowing those away?

*From:*Remo Mattei [mailto:r...@mattei.org]
*Sent:* Sunday, August 16, 2020 5:43 PM
*To:* qmailtoaster-list@qmailtoaster.com
*Subject:* Re: [qmailtoaster] Distressing strange behavior

BTW, I always use the -L on the qmHandle it should not change much but 
my 2 cents.


Remo



On Aug 16, 2020, at 3:32 PM, Chas Hockenbarger mailto:chash...@gmail.com>> wrote:

Yes, I did check those, that was my first thought is that the
server had been compromised and someone modified those files to do
some weird thing.   However,

.qmail-root has one line &postmaster@

.qmail-postmaster has one line &postmaster@

.qmail-mailer-daemon has one line &postmaster@

I see no other files in that directory.

One more piece of info I just discovered.  Even though qmHandle –l
reports 0 messages in either the remote or local queue, the bounce
queue directory has over 2000 messages in it.

Could that be a contributing factor here?  I don’t see how that
would create random emails going to Gmail accounts from
(seemingly) random other messages, but is it possible something is
borked up in the queue processing there since Gmail is bouncing
everything back to me?

*From:*Remo Mattei [mailto:r...@mattei.org]
*Sent:*Sunday, August 16, 2020 5:26 PM
*To:*qmailtoaster-list@qmailtoaster.com
<mailto:qmailtoaster-list@qmailtoaster.com>
    *Subject:*Re: [qmailtoaster] Distressing strange behavior

did you check your qmail aliases?

cd /var/qmail/alias/

what do those files say?




On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger
mailto:chash...@gmail.com>> wrote:

Thanks, Boheme, and yes that’s a problem, but it’s a symptom
of this problem.  Emails are going to Gmail accounts when
users aren’t sending them.  Legit emails to Gmail accounts are
definitely getting bounced, too, which I have to deal with
later.  If I can’t stop this weird spamming to them, I can’t
recover the reputation.

*From:*Boheme [mailto:boh...@gmail.com]
*Sent:*Sunday, August 16, 2020 4:59 PM
*To:*qmailtoaster-list@qmailtoaster.com
<mailto:qmailtoaster-list@qmailtoaster.com>
    *Subject:*Re: [qmailtoaster] Distressing strange behavior

It doesn’t sound like you are being repeatedly hacked. It
sounds like your reputation dropped with google, and certain
emails trigger their anti-spam filtering now. Not all of them,
just some. I have problems with Google accepting email
regularly sometimes, and dropping other emails into people’s
spam folders, as a result of too many of my users forwarding
email to google and those forwards passing along a lot of spam
to their addresses on my server.

-Sent from my Pip-Boy 3000





On 17/08/2020, at 8:46 AM, Charles Hockenbarger
mailto:chash...@gmail.com>> wrote:



As I understand the forwards setup in qmailadmin those are
in the database, right?

The address that was compromised hasn't sent any email
since the password change.

I hadn't thought about looking at qmail-inject. I'll dig
into watching that part of the process.

GetTypeApp for Android <http://www.typeapp.com/r?b=15986>

On Aug 16, 2020, at 3:14 PM, Eric Broch
mailto:ebr...@whitehorsetc.com>>
wrote:

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with
forwards to gmail accounts they aren't limited to just
these forwards, they also have the option in the email
client to add gmail accounts in the "To:" field of the
email they're sending, thus bounces from gmail
accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll
see it in the send log.

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:

I'm hoping someone has encountered this weird
behavior or something like it before and can point
me down a path, because all my research has turned

Re: [qmailtoaster] Distressing strange behavior

[Remo Mattei]

well if qmailctl queue and or qmHandle do not see them I could zip the folder 
and remove those files. 

Remo 

> On Aug 16, 2020, at 3:49 PM, Chas Hockenbarger  wrote:
> 
> So I looked at a few of the files in the bounce folder and every one of them 
> is bounces back from Gmail for either bad addresses or just the reputation 
> bounce. 
>  
> Is there a down side to just blowing those away?  
>   <>
> From: Remo Mattei [mailto:r...@mattei.org] 
> Sent: Sunday, August 16, 2020 5:43 PM
> To: qmailtoaster-list@qmailtoaster.com
> Subject: Re: [qmailtoaster] Distressing strange behavior
>  
> BTW, I always use the -L on the qmHandle it should not change much but my 2 
> cents. 
>  
> Remo
> 
> 
>> On Aug 16, 2020, at 3:32 PM, Chas Hockenbarger > <mailto:chash...@gmail.com>> wrote:
>>  
>> Yes, I did check those, that was my first thought is that the server had 
>> been compromised and someone modified those files to do some weird thing.   
>> However, 
>>  
>> .qmail-root has one line &postmaster@
>> .qmail-postmaster has one line &postmaster@
>> .qmail-mailer-daemon has one line &postmaster@
>>  
>> I see no other files in that directory.
>>  
>> One more piece of info I just discovered.  Even though qmHandle –l reports 0 
>> messages in either the remote or local queue, the bounce queue directory has 
>> over 2000 messages in it.   
>>  
>> Could that be a contributing factor here?  I don’t see how that would create 
>> random emails going to Gmail accounts from (seemingly) random other 
>> messages, but is it possible something is borked up in the queue processing 
>> there since Gmail is bouncing everything back to me?
>>  
>> From: Remo Mattei [mailto:r...@mattei.org <mailto:r...@mattei.org>] 
>> Sent: Sunday, August 16, 2020 5:26 PM
>> To: qmailtoaster-list@qmailtoaster.com 
>> <mailto:qmailtoaster-list@qmailtoaster.com>
>> Subject: Re: [qmailtoaster] Distressing strange behavior
>>  
>> did you check your qmail aliases?
>> cd /var/qmail/alias/
>>  
>> what do those files say?
>>  
>> 
>> 
>> 
>>> On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger >> <mailto:chash...@gmail.com>> wrote:
>>>  
>>> Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this 
>>> problem.  Emails are going to Gmail accounts when users aren’t sending 
>>> them.  Legit emails to Gmail accounts are definitely getting bounced, too, 
>>> which I have to deal with later.  If I can’t stop this weird spamming to 
>>> them, I can’t recover the reputation.
>>>  
>>> From: Boheme [mailto:boh...@gmail.com <mailto:boh...@gmail.com>] 
>>> Sent: Sunday, August 16, 2020 4:59 PM
>>> To: qmailtoaster-list@qmailtoaster.com 
>>> <mailto:qmailtoaster-list@qmailtoaster.com>
>>> Subject: Re: [qmailtoaster] Distressing strange behavior
>>>  
>>> It doesn’t sound like you are being repeatedly hacked. It sounds like your 
>>> reputation dropped with google, and certain emails trigger their anti-spam 
>>> filtering now. Not all of them, just some. I have problems with Google 
>>> accepting email regularly sometimes, and dropping other emails into 
>>> people’s spam folders, as a result of too many of my users forwarding email 
>>> to google and those forwards passing along a lot of spam to their addresses 
>>> on my server. 
>>> 
>>> -Sent from my Pip-Boy 3000
>>> 
>>> 
>>> 
>>> 
>>>> On 17/08/2020, at 8:46 AM, Charles Hockenbarger >>> <mailto:chash...@gmail.com>> wrote:
>>>> 
>>>> 
>>>> As I understand the forwards setup in qmailadmin those are in the 
>>>> database, right?
>>>> 
>>>> The address that was compromised hasn't sent any email since the password 
>>>> change. 
>>>> 
>>>> I hadn't thought about looking at qmail-inject. I'll dig into watching 
>>>> that part of the process. 
>>>> 
>>>> Get TypeApp for Android <http://www.typeapp.com/r?b=15986>
>>>> On Aug 16, 2020, at 3:14 PM, Eric Broch >>> <mailto:ebr...@whitehorsetc.com>> wrote:
>>>>> How do you have your forwards set up?
>>>>> Is there any mail in your queue?
>>>>> If someone hacked an account on your server with forwards to gmail 
>>>>> accounts they aren't limited to just these forwards, they also have the 
>>&g

RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

So I looked at a few of the files in the bounce folder and every one of them is 
bounces back from Gmail for either bad addresses or just the reputation bounce. 

 

Is there a down side to just blowing those away?  

 

From: Remo Mattei [mailto:r...@mattei.org] 
Sent: Sunday, August 16, 2020 5:43 PM
To: qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

BTW, I always use the -L on the qmHandle it should not change much but my 2 
cents. 

 

Remo





On Aug 16, 2020, at 3:32 PM, Chas Hockenbarger mailto:chash...@gmail.com> > wrote:

 

Yes, I did check those, that was my first thought is that the server had been 
compromised and someone modified those files to do some weird thing.   However, 

 

.qmail-root has one line &postmaster@

.qmail-postmaster has one line &postmaster@

.qmail-mailer-daemon has one line &postmaster@

 

I see no other files in that directory.

 

One more piece of info I just discovered.  Even though qmHandle –l reports 0 
messages in either the remote or local queue, the bounce queue directory has 
over 2000 messages in it.   

 

Could that be a contributing factor here?  I don’t see how that would create 
random emails going to Gmail accounts from (seemingly) random other messages, 
but is it possible something is borked up in the queue processing there since 
Gmail is bouncing everything back to me?

 

From: Remo Mattei [mailto:r...@mattei.org] 
Sent: Sunday, August 16, 2020 5:26 PM
To: qmailtoaster-list@qmailtoaster.com 
<mailto:qmailtoaster-list@qmailtoaster.com> 
Subject: Re: [qmailtoaster] Distressing strange behavior

 

did you check your qmail aliases?

cd /var/qmail/alias/

 

what do those files say?

 






On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger < <mailto:chash...@gmail.com> 
chash...@gmail.com> wrote:

 

Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this problem.  
Emails are going to Gmail accounts when users aren’t sending them.  Legit 
emails to Gmail accounts are definitely getting bounced, too, which I have to 
deal with later.  If I can’t stop this weird spamming to them, I can’t recover 
the reputation.

 

From: Boheme [ <mailto:boh...@gmail.com> mailto:boh...@gmail.com] 
Sent: Sunday, August 16, 2020 4:59 PM
To:  <mailto:qmailtoaster-list@qmailtoaster.com> 
qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

It doesn’t sound like you are being repeatedly hacked. It sounds like your 
reputation dropped with google, and certain emails trigger their anti-spam 
filtering now. Not all of them, just some. I have problems with Google 
accepting email regularly sometimes, and dropping other emails into people’s 
spam folders, as a result of too many of my users forwarding email to google 
and those forwards passing along a lot of spam to their addresses on my server. 

-Sent from my Pip-Boy 3000







On 17/08/2020, at 8:46 AM, Charles Hockenbarger < <mailto:chash...@gmail.com> 
chash...@gmail.com> wrote:



As I understand the forwards setup in qmailadmin those are in the database, 
right?

The address that was compromised hasn't sent any email since the password 
change. 

I hadn't thought about looking at qmail-inject. I'll dig into watching that 
part of the process. 

Get  <http://www.typeapp.com/r?b=15986> TypeApp for Android

On Aug 16, 2020, at 3:14 PM, Eric Broch < <mailto:ebr...@whitehorsetc.com> 
ebr...@whitehorsetc.com> wrote:

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail accounts 
they aren't limited to just these forwards, they also have the option in the 
email client to add gmail accounts in the "To:" field of the email they're 
sending, thus bounces from gmail accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in the send log.

 

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 

I'm hoping someone has encountered this weird behavior or something like it 
before and can point me down a path, because all my research has turned up 
nothing so far. 

  

I had an email account recently get breached due to a re-used password, and 
that account was used to send a bunch of spam out from a server I help manage.  
We changed the password on the account as soon as we found it happening and the 
outbound flood stopped. 

  

Shortly after that, however, I started seeing a very, very strange behavior.  
Sometimes, and I haven’t yet been able to identify the trigger or pattern, when 
users on this server send email to a forward that contains around 50 or so 
email addresses (they use it like a private distribution list) they will get 
anywhere from 1-10 bounces from Gmail.  Not every email sent to the forward has 
this happen, and not even every email from a particula

Re: [qmailtoaster] Distressing strange behavior

[Remo Mattei]

BTW, I always use the -L on the qmHandle it should not change much but my 2 
cents. 

Remo

> On Aug 16, 2020, at 3:32 PM, Chas Hockenbarger  wrote:
> 
> Yes, I did check those, that was my first thought is that the server had been 
> compromised and someone modified those files to do some weird thing.   
> However, 
>  
> .qmail-root has one line &postmaster@
> .qmail-postmaster has one line &postmaster@
> .qmail-mailer-daemon has one line &postmaster@
>  
> I see no other files in that directory.
>  
> One more piece of info I just discovered.  Even though qmHandle –l reports 0 
> messages in either the remote or local queue, the bounce queue directory has 
> over 2000 messages in it.   
>  
> Could that be a contributing factor here?  I don’t see how that would create 
> random emails going to Gmail accounts from (seemingly) random other messages, 
> but is it possible something is borked up in the queue processing there since 
> Gmail is bouncing everything back to me?
>   <>
> From: Remo Mattei [mailto:r...@mattei.org] 
> Sent: Sunday, August 16, 2020 5:26 PM
> To: qmailtoaster-list@qmailtoaster.com
> Subject: Re: [qmailtoaster] Distressing strange behavior
>  
> did you check your qmail aliases?
> cd /var/qmail/alias/
>  
> what do those files say?
>  
> 
> 
>> On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger > <mailto:chash...@gmail.com>> wrote:
>>  
>> Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this 
>> problem.  Emails are going to Gmail accounts when users aren’t sending them. 
>>  Legit emails to Gmail accounts are definitely getting bounced, too, which I 
>> have to deal with later.  If I can’t stop this weird spamming to them, I 
>> can’t recover the reputation.
>>  
>> From: Boheme [mailto:boh...@gmail.com <mailto:boh...@gmail.com>] 
>> Sent: Sunday, August 16, 2020 4:59 PM
>> To: qmailtoaster-list@qmailtoaster.com 
>> <mailto:qmailtoaster-list@qmailtoaster.com>
>> Subject: Re: [qmailtoaster] Distressing strange behavior
>>  
>> It doesn’t sound like you are being repeatedly hacked. It sounds like your 
>> reputation dropped with google, and certain emails trigger their anti-spam 
>> filtering now. Not all of them, just some. I have problems with Google 
>> accepting email regularly sometimes, and dropping other emails into people’s 
>> spam folders, as a result of too many of my users forwarding email to google 
>> and those forwards passing along a lot of spam to their addresses on my 
>> server. 
>> 
>> -Sent from my Pip-Boy 3000
>> 
>> 
>> 
>>> On 17/08/2020, at 8:46 AM, Charles Hockenbarger >> <mailto:chash...@gmail.com>> wrote:
>>> 
>>> 
>>> As I understand the forwards setup in qmailadmin those are in the database, 
>>> right?
>>> 
>>> The address that was compromised hasn't sent any email since the password 
>>> change. 
>>> 
>>> I hadn't thought about looking at qmail-inject. I'll dig into watching that 
>>> part of the process. 
>>> 
>>> Get TypeApp for Android <http://www.typeapp.com/r?b=15986>
>>> On Aug 16, 2020, at 3:14 PM, Eric Broch >> <mailto:ebr...@whitehorsetc.com>> wrote:
>>>> How do you have your forwards set up?
>>>> Is there any mail in your queue?
>>>> If someone hacked an account on your server with forwards to gmail 
>>>> accounts they aren't limited to just these forwards, they also have the 
>>>> option in the email client to add gmail accounts in the "To:" field of the 
>>>> email they're sending, thus bounces from gmail accounts that aren't in 
>>>> your forwards file.
>>>> Also, qmail-inject puts mail in the queue and you'll see it in the send 
>>>> log.
>>>>  
>>>> On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 
>>>>> I'm hoping someone has encountered this weird behavior or something like 
>>>>> it before and can point me down a path, because all my research has 
>>>>> turned up nothing so far. 
>>>>>   
>>>>> I had an email account recently get breached due to a re-used password, 
>>>>> and that account was used to send a bunch of spam out from a server I 
>>>>> help manage.  We changed the password on the account as soon as we found 
>>>>> it happening and the outbound flood stopped. 
>>>>>   
>>>>> Shortly after that, however, I started seeing a very, ve

Re: [qmailtoaster] Distressing strange behavior

[Remo Mattei]

do a tree in /var/qmail/queue 
that shows all the msg you got even though the qmHandle does not show it. I 
could check that and read some of those files. 

> On Aug 16, 2020, at 3:32 PM, Chas Hockenbarger  wrote:
> 
> Yes, I did check those, that was my first thought is that the server had been 
> compromised and someone modified those files to do some weird thing.   
> However, 
>  
> .qmail-root has one line &postmaster@
> .qmail-postmaster has one line &postmaster@
> .qmail-mailer-daemon has one line &postmaster@
>  
> I see no other files in that directory.
>  
> One more piece of info I just discovered.  Even though qmHandle –l reports 0 
> messages in either the remote or local queue, the bounce queue directory has 
> over 2000 messages in it.   
>  
> Could that be a contributing factor here?  I don’t see how that would create 
> random emails going to Gmail accounts from (seemingly) random other messages, 
> but is it possible something is borked up in the queue processing there since 
> Gmail is bouncing everything back to me?
>   <>
> From: Remo Mattei [mailto:r...@mattei.org] 
> Sent: Sunday, August 16, 2020 5:26 PM
> To: qmailtoaster-list@qmailtoaster.com
> Subject: Re: [qmailtoaster] Distressing strange behavior
>  
> did you check your qmail aliases?
> cd /var/qmail/alias/
>  
> what do those files say?
>  
> 
> 
>> On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger > <mailto:chash...@gmail.com>> wrote:
>>  
>> Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this 
>> problem.  Emails are going to Gmail accounts when users aren’t sending them. 
>>  Legit emails to Gmail accounts are definitely getting bounced, too, which I 
>> have to deal with later.  If I can’t stop this weird spamming to them, I 
>> can’t recover the reputation.
>>  
>> From: Boheme [mailto:boh...@gmail.com <mailto:boh...@gmail.com>] 
>> Sent: Sunday, August 16, 2020 4:59 PM
>> To: qmailtoaster-list@qmailtoaster.com 
>> <mailto:qmailtoaster-list@qmailtoaster.com>
>> Subject: Re: [qmailtoaster] Distressing strange behavior
>>  
>> It doesn’t sound like you are being repeatedly hacked. It sounds like your 
>> reputation dropped with google, and certain emails trigger their anti-spam 
>> filtering now. Not all of them, just some. I have problems with Google 
>> accepting email regularly sometimes, and dropping other emails into people’s 
>> spam folders, as a result of too many of my users forwarding email to google 
>> and those forwards passing along a lot of spam to their addresses on my 
>> server. 
>> 
>> -Sent from my Pip-Boy 3000
>> 
>> 
>> 
>>> On 17/08/2020, at 8:46 AM, Charles Hockenbarger >> <mailto:chash...@gmail.com>> wrote:
>>> 
>>> 
>>> As I understand the forwards setup in qmailadmin those are in the database, 
>>> right?
>>> 
>>> The address that was compromised hasn't sent any email since the password 
>>> change. 
>>> 
>>> I hadn't thought about looking at qmail-inject. I'll dig into watching that 
>>> part of the process. 
>>> 
>>> Get TypeApp for Android <http://www.typeapp.com/r?b=15986>
>>> On Aug 16, 2020, at 3:14 PM, Eric Broch >> <mailto:ebr...@whitehorsetc.com>> wrote:
>>>> How do you have your forwards set up?
>>>> Is there any mail in your queue?
>>>> If someone hacked an account on your server with forwards to gmail 
>>>> accounts they aren't limited to just these forwards, they also have the 
>>>> option in the email client to add gmail accounts in the "To:" field of the 
>>>> email they're sending, thus bounces from gmail accounts that aren't in 
>>>> your forwards file.
>>>> Also, qmail-inject puts mail in the queue and you'll see it in the send 
>>>> log.
>>>>  
>>>> On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 
>>>>> I'm hoping someone has encountered this weird behavior or something like 
>>>>> it before and can point me down a path, because all my research has 
>>>>> turned up nothing so far. 
>>>>>   
>>>>> I had an email account recently get breached due to a re-used password, 
>>>>> and that account was used to send a bunch of spam out from a server I 
>>>>> help manage.  We changed the password on the account as soon as we found 
>>>>> it happening and the outbound flood stopped. 
>>>>>   
>>>>&

RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

Yes, I did check those, that was my first thought is that the server had been 
compromised and someone modified those files to do some weird thing.   However, 

 

.qmail-root has one line &postmaster@

.qmail-postmaster has one line &postmaster@

.qmail-mailer-daemon has one line &postmaster@

 

I see no other files in that directory.

 

One more piece of info I just discovered.  Even though qmHandle –l reports 0 
messages in either the remote or local queue, the bounce queue directory has 
over 2000 messages in it.   

 

Could that be a contributing factor here?  I don’t see how that would create 
random emails going to Gmail accounts from (seemingly) random other messages, 
but is it possible something is borked up in the queue processing there since 
Gmail is bouncing everything back to me?

 

From: Remo Mattei [mailto:r...@mattei.org] 
Sent: Sunday, August 16, 2020 5:26 PM
To: qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

did you check your qmail aliases?

cd /var/qmail/alias/

 

what do those files say?

 





On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger mailto:chash...@gmail.com> > wrote:

 

Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this problem.  
Emails are going to Gmail accounts when users aren’t sending them.  Legit 
emails to Gmail accounts are definitely getting bounced, too, which I have to 
deal with later.  If I can’t stop this weird spamming to them, I can’t recover 
the reputation.

 

From: Boheme [mailto:boh...@gmail.com] 
Sent: Sunday, August 16, 2020 4:59 PM
To: qmailtoaster-list@qmailtoaster.com 
<mailto:qmailtoaster-list@qmailtoaster.com> 
Subject: Re: [qmailtoaster] Distressing strange behavior

 

It doesn’t sound like you are being repeatedly hacked. It sounds like your 
reputation dropped with google, and certain emails trigger their anti-spam 
filtering now. Not all of them, just some. I have problems with Google 
accepting email regularly sometimes, and dropping other emails into people’s 
spam folders, as a result of too many of my users forwarding email to google 
and those forwards passing along a lot of spam to their addresses on my server. 

-Sent from my Pip-Boy 3000






On 17/08/2020, at 8:46 AM, Charles Hockenbarger < <mailto:chash...@gmail.com> 
chash...@gmail.com> wrote:



As I understand the forwards setup in qmailadmin those are in the database, 
right?

The address that was compromised hasn't sent any email since the password 
change. 

I hadn't thought about looking at qmail-inject. I'll dig into watching that 
part of the process. 

Get  <http://www.typeapp.com/r?b=15986> TypeApp for Android

On Aug 16, 2020, at 3:14 PM, Eric Broch < <mailto:ebr...@whitehorsetc.com> 
ebr...@whitehorsetc.com> wrote:

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail accounts 
they aren't limited to just these forwards, they also have the option in the 
email client to add gmail accounts in the "To:" field of the email they're 
sending, thus bounces from gmail accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in the send log.

 

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 

I'm hoping someone has encountered this weird behavior or something like it 
before and can point me down a path, because all my research has turned up 
nothing so far. 

  

I had an email account recently get breached due to a re-used password, and 
that account was used to send a bunch of spam out from a server I help manage.  
We changed the password on the account as soon as we found it happening and the 
outbound flood stopped. 

  

Shortly after that, however, I started seeing a very, very strange behavior.  
Sometimes, and I haven’t yet been able to identify the trigger or pattern, when 
users on this server send email to a forward that contains around 50 or so 
email addresses (they use it like a private distribution list) they will get 
anywhere from 1-10 bounces from Gmail.  Not every email sent to the forward has 
this happen, and not even every email from a particular user. 

  

The outbound spamming caused the server’s reputation to go in the tank with 
Google, and if it weren’t for that, I wouldn’t know this was happening, because 
they get the bounces from Gmail accounts that absolutely ARE NOT in the forward 
or part of the email chain AT ALL. 

  

I’m kind of freaking out here because while I haven’t found a breach of the 
actual server / OS, this feels like someone has been able to inject something 
somewhere into my server that I simply can’t find.  It is especially troubling 
because a user who is not on this domain, but is part of the group and 
therefore uses the forward from time to time, sent something to the forward 
today and got Gmail bounces. 

Re: [qmailtoaster] Distressing strange behavior

[Remo Mattei]

did you check your qmail aliases?
cd /var/qmail/alias/

what do those files say?


> On Aug 16, 2020, at 3:10 PM, Chas Hockenbarger  wrote:
> 
> Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this problem. 
>  Emails are going to Gmail accounts when users aren’t sending them.  Legit 
> emails to Gmail accounts are definitely getting bounced, too, which I have to 
> deal with later.  If I can’t stop this weird spamming to them, I can’t 
> recover the reputation.
>   <>
> From: Boheme [mailto:boh...@gmail.com] 
> Sent: Sunday, August 16, 2020 4:59 PM
> To: qmailtoaster-list@qmailtoaster.com
> Subject: Re: [qmailtoaster] Distressing strange behavior
>  
> It doesn’t sound like you are being repeatedly hacked. It sounds like your 
> reputation dropped with google, and certain emails trigger their anti-spam 
> filtering now. Not all of them, just some. I have problems with Google 
> accepting email regularly sometimes, and dropping other emails into people’s 
> spam folders, as a result of too many of my users forwarding email to google 
> and those forwards passing along a lot of spam to their addresses on my 
> server. 
> 
> -Sent from my Pip-Boy 3000
> 
> 
>> On 17/08/2020, at 8:46 AM, Charles Hockenbarger > <mailto:chash...@gmail.com>> wrote:
>> 
>> 
>> As I understand the forwards setup in qmailadmin those are in the database, 
>> right?
>> 
>> The address that was compromised hasn't sent any email since the password 
>> change. 
>> 
>> I hadn't thought about looking at qmail-inject. I'll dig into watching that 
>> part of the process. 
>> 
>> Get TypeApp for Android <http://www.typeapp.com/r?b=15986>
>> On Aug 16, 2020, at 3:14 PM, Eric Broch > <mailto:ebr...@whitehorsetc.com>> wrote:
>>> How do you have your forwards set up?
>>> 
>>> Is there any mail in your queue?
>>> 
>>> If someone hacked an account on your server with forwards to gmail accounts 
>>> they aren't limited to just these forwards, they also have the option in 
>>> the email client to add gmail accounts in the "To:" field of the email 
>>> they're sending, thus bounces from gmail accounts that aren't in your 
>>> forwards file.
>>> 
>>> Also, qmail-inject puts mail in the queue and you'll see it in the send log.
>>> 
>>>  
>>> 
>>> On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 
>>>> I'm hoping someone has encountered this weird behavior or something like 
>>>> it before and can point me down a path, because all my research has turned 
>>>> up nothing so far. 
>>>>   
>>>> 
>>>> I had an email account recently get breached due to a re-used password, 
>>>> and that account was used to send a bunch of spam out from a server I help 
>>>> manage.  We changed the password on the account as soon as we found it 
>>>> happening and the outbound flood stopped. 
>>>>   
>>>> 
>>>> Shortly after that, however, I started seeing a very, very strange 
>>>> behavior.  Sometimes, and I haven’t yet been able to identify the trigger 
>>>> or pattern, when users on this server send email to a forward that 
>>>> contains around 50 or so email addresses (they use it like a private 
>>>> distribution list) they will get anywhere from 1-10 bounces from Gmail.  
>>>> Not every email sent to the forward has this happen, and not even every 
>>>> email from a particular user. 
>>>>   
>>>> 
>>>> The outbound spamming caused the server’s reputation to go in the tank 
>>>> with Google, and if it weren’t for that, I wouldn’t know this was 
>>>> happening, because they get the bounces from Gmail accounts that 
>>>> absolutely ARE NOT in the forward or part of the email chain AT ALL. 
>>>>   
>>>> 
>>>> I’m kind of freaking out here because while I haven’t found a breach of 
>>>> the actual server / OS, this feels like someone has been able to inject 
>>>> something somewhere into my server that I simply can’t find.  It is 
>>>> especially troubling because a user who is not on this domain, but is part 
>>>> of the group and therefore uses the forward from time to time, sent 
>>>> something to the forward today and got Gmail bounces.
>>>>   
>>>> 
>>>> I don’t see anything in the send log that shows the server even trying to 
>>>> send to Gmail, which only adds to the ghost story. 
>>>>   
>>>> 
>>>> Any ideas, paths to go down, anything would be greatly appreciated here.  
>>>> I’m about to just rebuild the whole thing from scratch on a new VM, but if 
>>>> I’m overlooking something simple don’t want to put the users through that. 
>>>>   
>>>> 
>>>> Thanks in advance. 
>>>>   
>>>> 
>>>> Chas 

RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

Thanks, Boheme, and yes that’s a problem, but it’s a symptom of this problem.  
Emails are going to Gmail accounts when users aren’t sending them.  Legit 
emails to Gmail accounts are definitely getting bounced, too, which I have to 
deal with later.  If I can’t stop this weird spamming to them, I can’t recover 
the reputation.

 

From: Boheme [mailto:boh...@gmail.com] 
Sent: Sunday, August 16, 2020 4:59 PM
To: qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

It doesn’t sound like you are being repeatedly hacked. It sounds like your 
reputation dropped with google, and certain emails trigger their anti-spam 
filtering now. Not all of them, just some. I have problems with Google 
accepting email regularly sometimes, and dropping other emails into people’s 
spam folders, as a result of too many of my users forwarding email to google 
and those forwards passing along a lot of spam to their addresses on my server. 

-Sent from my Pip-Boy 3000





On 17/08/2020, at 8:46 AM, Charles Hockenbarger mailto:chash...@gmail.com> > wrote:



As I understand the forwards setup in qmailadmin those are in the database, 
right?

The address that was compromised hasn't sent any email since the password 
change. 

I hadn't thought about looking at qmail-inject. I'll dig into watching that 
part of the process. 

Get TypeApp for Android <http://www.typeapp.com/r?b=15986>  

On Aug 16, 2020, at 3:14 PM, Eric Broch mailto:ebr...@whitehorsetc.com> > wrote:

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail accounts 
they aren't limited to just these forwards, they also have the option in the 
email client to add gmail accounts in the "To:" field of the email they're 
sending, thus bounces from gmail accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in the send log.

 

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 

I'm hoping someone has encountered this weird behavior or something like it 
before and can point me down a path, because all my research has turned up 
nothing so far. 

  

I had an email account recently get breached due to a re-used password, and 
that account was used to send a bunch of spam out from a server I help manage.  
We changed the password on the account as soon as we found it happening and the 
outbound flood stopped. 

  

Shortly after that, however, I started seeing a very, very strange behavior.  
Sometimes, and I haven’t yet been able to identify the trigger or pattern, when 
users on this server send email to a forward that contains around 50 or so 
email addresses (they use it like a private distribution list) they will get 
anywhere from 1-10 bounces from Gmail.  Not every email sent to the forward has 
this happen, and not even every email from a particular user. 

  

The outbound spamming caused the server’s reputation to go in the tank with 
Google, and if it weren’t for that, I wouldn’t know this was happening, because 
they get the bounces from Gmail accounts that absolutely ARE NOT in the forward 
or part of the email chain AT ALL. 

  

I’m kind of freaking out here because while I haven’t found a breach of the 
actual server / OS, this feels like someone has been able to inject something 
somewhere into my server that I simply can’t find.  It is especially troubling 
because a user who is not on this domain, but is part of the group and 
therefore uses the forward from time to time, sent something to the forward 
today and got Gmail bounces.

  

I don’t see anything in the send log that shows the server even trying to send 
to Gmail, which only adds to the ghost story. 

  

Any ideas, paths to go down, anything would be greatly appreciated here.  I’m 
about to just rebuild the whole thing from scratch on a new VM, but if I’m 
overlooking something simple don’t want to put the users through that. 

  

Thanks in advance. 

  

Chas 

RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

I just got another piece of information.  I got a failure message a few hours 
ago to the postmaster account for this domain that a message from root to root 
was not delivered to 5 different Gmail accounts.  The email was the cron.daily 
status report.  There is no way that should be going to these Gmail accounts.  
They are accounts I don’t know and root at this server is supposed to go to 
postmaster.

 

This just keeps getting weirder.

 

From: Eric Broch [mailto:ebr...@whitehorsetc.com] 
Sent: Sunday, August 16, 2020 4:13 PM
To: qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

Yes forwards can be in a .qmail file or in the vpopmail database.

So, the bounces occurring presently, what's the originating account?

Is there anything in your queue (# qmailctl queue)?

 

On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:

As I understand the forwards setup in qmailadmin those are in the database, 
right?

The address that was compromised hasn't sent any email since the password 
change. 

I hadn't thought about looking at qmail-inject. I'll dig into watching that 
part of the process. 

Get TypeApp for Android <http://www.typeapp.com/r?b=15986>  

On Aug 16, 2020, at 3:14 PM, Eric Broch mailto:ebr...@whitehorsetc.com> > wrote: 

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail accounts 
they aren't limited to just these forwards, they also have the option in the 
email client to add gmail accounts in the "To:" field of the email they're 
sending, thus bounces from gmail accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in the send log.

 

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 

I'm hoping someone has encountered this weird behavior or something like it 
before and can point me down a path, because all my research has turned up 
nothing so far. 

  

I had an email account recently get breached due to a re-used password, and 
that account was used to send a bunch of spam out from a server I help manage.  
We changed the password on the account as soon as we found it happening and the 
outbound flood stopped. 

  

Shortly after that, however, I started seeing a very, very strange behavior.  
Sometimes, and I haven’t yet been able to identify the trigger or pattern, when 
users on this server send email to a forward that contains around 50 or so 
email addresses (they use it like a private distribution list) they will get 
anywhere from 1-10 bounces from Gmail.  Not every email sent to the forward has 
this happen, and not even every email from a particular user. 

  

The outbound spamming caused the server’s reputation to go in the tank with 
Google, and if it weren’t for that, I wouldn’t know this was happening, because 
they get the bounces from Gmail accounts that absolutely ARE NOT in the forward 
or part of the email chain AT ALL. 

  

I’m kind of freaking out here because while I haven’t found a breach of the 
actual server / OS, this feels like someone has been able to inject something 
somewhere into my server that I simply can’t find.  It is especially troubling 
because a user who is not on this domain, but is part of the group and 
therefore uses the forward from time to time, sent something to the forward 
today and got Gmail bounces.

  

I don’t see anything in the send log that shows the server even trying to send 
to Gmail, which only adds to the ghost story. 

  

Any ideas, paths to go down, anything would be greatly appreciated here.  I’m 
about to just rebuild the whole thing from scratch on a new VM, but if I’m 
overlooking something simple don’t want to put the users through that. 

  

Thanks in advance. 

  

Chas 

Re: [qmailtoaster] Distressing strange behavior

[remo]

For that i had to fix my ptr and make sure the name matches. Once i fixed that 
and I also added dmark and added google to my trust dns records looks like it’s 
not going into spam and mail gets delivered just fine. 
> Il giorno 16 ago 2020, alle ore 14:59, Boheme  ha scritto:
> 
> It doesn’t sound like you are being repeatedly hacked. It sounds like your 
> reputation dropped with google, and certain emails trigger their anti-spam 
> filtering now. Not all of them, just some. I have problems with Google 
> accepting email regularly sometimes, and dropping other emails into people’s 
> spam folders, as a result of too many of my users forwarding email to google 
> and those forwards passing along a lot of spam to their addresses on my 
> server. 
> 
> -Sent from my Pip-Boy 3000
> 
>>> On 17/08/2020, at 8:46 AM, Charles Hockenbarger  wrote:
>>> 
>> 
>> As I understand the forwards setup in qmailadmin those are in the database, 
>> right?
>> 
>> The address that was compromised hasn't sent any email since the password 
>> change. 
>> 
>> I hadn't thought about looking at qmail-inject. I'll dig into watching that 
>> part of the process. 
>> 
>> Get TypeApp for Android
>>> On Aug 16, 2020, at 3:14 PM, Eric Broch  wrote:
>>> How do you have your forwards set up?
>>> 
>>> Is there any mail in your queue?
>>> 
>>> If someone hacked an account on your server with forwards to gmail accounts 
>>> they aren't limited to just these forwards, they also have the option in 
>>> the email client to add gmail accounts in the "To:" field of the email 
>>> they're sending, thus bounces from gmail accounts that aren't in your 
>>> forwards file.
>>> 
>>> Also, qmail-inject puts mail in the queue and you'll see it in the send log.
>>> 
>>> 
>>> 
 On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 
 I'm hoping someone has encountered this weird behavior or something like 
 it before and can point me down a path, because all my research has turned 
 up nothing so far.
  
 
 I had an email account recently get breached due to a re-used password, 
 and that account was used to send a bunch of spam out from a server I help 
 manage.  We changed the password on the account as soon as we found it 
 happening and the outbound flood stopped.
  
 
 Shortly after that, however, I started seeing a very, very strange 
 behavior.  Sometimes, and I haven’t yet been able to identify the trigger 
 or pattern, when users on this server send email to a forward that 
 contains around 50 or so email addresses (they use it like a private 
 distribution list) they will get anywhere from 1-10 bounces from Gmail.  
 Not every email sent to the forward has this happen, and not even every 
 email from a particular user.
  
 
 The outbound spamming caused the server’s reputation to go in the tank 
 with Google, and if it weren’t for that, I wouldn’t know this was 
 happening, because they get the bounces from Gmail accounts that 
 absolutely ARE NOT in the forward or part of the email chain AT ALL.
  
 
 I’m kind of freaking out here because while I haven’t found a breach of 
 the actual server / OS, this feels like someone has been able to inject 
 something somewhere into my server that I simply can’t find.  It is 
 especially troubling because a user who is not on this domain, but is part 
 of the group and therefore uses the forward from time to time, sent 
 something to the forward today and got Gmail bounces.   
  
 
 I don’t see anything in the send log that shows the server even trying to 
 send to Gmail, which only adds to the ghost story.
  
 
 Any ideas, paths to go down, anything would be greatly appreciated here.  
 I’m about to just rebuild the whole thing from scratch on a new VM, but if 
 I’m overlooking something simple don’t want to put the users through that.
  
 
 Thanks in advance.
  
 
 Chas
-
To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com
For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com

Re: [qmailtoaster] Distressing strange behavior

[Boheme]

It doesn’t sound like you are being repeatedly hacked. It sounds like your 
reputation dropped with google, and certain emails trigger their anti-spam 
filtering now. Not all of them, just some. I have problems with Google 
accepting email regularly sometimes, and dropping other emails into people’s 
spam folders, as a result of too many of my users forwarding email to google 
and those forwards passing along a lot of spam to their addresses on my server. 

-Sent from my Pip-Boy 3000

> On 17/08/2020, at 8:46 AM, Charles Hockenbarger  wrote:
> 
> 
> As I understand the forwards setup in qmailadmin those are in the database, 
> right?
> 
> The address that was compromised hasn't sent any email since the password 
> change. 
> 
> I hadn't thought about looking at qmail-inject. I'll dig into watching that 
> part of the process. 
> 
> Get TypeApp for Android
>> On Aug 16, 2020, at 3:14 PM, Eric Broch  wrote:
>> How do you have your forwards set up?
>> 
>> Is there any mail in your queue?
>> 
>> If someone hacked an account on your server with forwards to gmail accounts 
>> they aren't limited to just these forwards, they also have the option in the 
>> email client to add gmail accounts in the "To:" field of the email they're 
>> sending, thus bounces from gmail accounts that aren't in your forwards file.
>> 
>> Also, qmail-inject puts mail in the queue and you'll see it in the send log.
>> 
>> 
>> 
>>> On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 
>>> I'm hoping someone has encountered this weird behavior or something like it 
>>> before and can point me down a path, because all my research has turned up 
>>> nothing so far.
>>>  
>>> 
>>> I had an email account recently get breached due to a re-used password, and 
>>> that account was used to send a bunch of spam out from a server I help 
>>> manage.  We changed the password on the account as soon as we found it 
>>> happening and the outbound flood stopped.
>>>  
>>> 
>>> Shortly after that, however, I started seeing a very, very strange 
>>> behavior.  Sometimes, and I haven’t yet been able to identify the trigger 
>>> or pattern, when users on this server send email to a forward that contains 
>>> around 50 or so email addresses (they use it like a private distribution 
>>> list) they will get anywhere from 1-10 bounces from Gmail.  Not every email 
>>> sent to the forward has this happen, and not even every email from a 
>>> particular user.
>>>  
>>> 
>>> The outbound spamming caused the server’s reputation to go in the tank with 
>>> Google, and if it weren’t for that, I wouldn’t know this was happening, 
>>> because they get the bounces from Gmail accounts that absolutely ARE NOT in 
>>> the forward or part of the email chain AT ALL.
>>>  
>>> 
>>> I’m kind of freaking out here because while I haven’t found a breach of the 
>>> actual server / OS, this feels like someone has been able to inject 
>>> something somewhere into my server that I simply can’t find.  It is 
>>> especially troubling because a user who is not on this domain, but is part 
>>> of the group and therefore uses the forward from time to time, sent 
>>> something to the forward today and got Gmail bounces.   
>>>  
>>> 
>>> I don’t see anything in the send log that shows the server even trying to 
>>> send to Gmail, which only adds to the ghost story.
>>>  
>>> 
>>> Any ideas, paths to go down, anything would be greatly appreciated here.  
>>> I’m about to just rebuild the whole thing from scratch on a new VM, but if 
>>> I’m overlooking something simple don’t want to put the users through that.
>>>  
>>> 
>>> Thanks in advance.
>>>  
>>> 
>>> Chas

Re: [qmailtoaster] Distressing strange behavior

[remo]

I have created an Ansible playbook to check queue :) which i run when needed. 
Now lot less but with the older version it was more useful. 

Remo
> Il giorno 16 ago 2020, alle ore 14:55, Chas Hockenbarger  
> ha scritto:
> 
> 
> Thanks Eric and Remo, I appreciate the assistance.
>  
> I’d forgotten about the simscan setting for the cdb to up the logging, it’s 
> been a LONG time since I’ve had to do that.
>  
> My queue is empty.  Nothing clogged up, it’s not residual stuff; that said, 
> I’m watching it pretty closely right now.
>  
> No .qmail files.  I logged into the db and looked – the forward is all in the 
> database, and I don’t have any .qmail files that I can find outside of the 
> skel folder.  My users aren’t ‘real’ users on the system, they’re all virtual 
> users.
>  
> Part of the problem is that the bouncing from Gmail has happened to different 
> users at different times, and at other times it doesn’t happen to them.  It 
> is so very bizarre.  Hopefully with an increased logging level I can find 
> enough to trace this down to its actual origins. 
>  
> From: Eric Broch [mailto:ebr...@whitehorsetc.com] 
> Sent: Sunday, August 16, 2020 4:13 PM
> To: qmailtoaster-list@qmailtoaster.com
> Subject: Re: [qmailtoaster] Distressing strange behavior
>  
> Yes forwards can be in a .qmail file or in the vpopmail database.
> 
> So, the bounces occurring presently, what's the originating account?
> 
> Is there anything in your queue (# qmailctl queue)?
> 
>  
> 
> On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:
> As I understand the forwards setup in qmailadmin those are in the database, 
> right?
> 
> The address that was compromised hasn't sent any email since the password 
> change.
> 
> I hadn't thought about looking at qmail-inject. I'll dig into watching that 
> part of the process.
> 
> Get TypeApp for Android
> On Aug 16, 2020, at 3:14 PM, Eric Broch  wrote:
> How do you have your forwards set up?
> 
> Is there any mail in your queue?
> 
> If someone hacked an account on your server with forwards to gmail accounts 
> they aren't limited to just these forwards, they also have the option in the 
> email client to add gmail accounts in the "To:" field of the email they're 
> sending, thus bounces from gmail accounts that aren't in your forwards file.
> 
> Also, qmail-inject puts mail in the queue and you'll see it in the send log.
> 
>  
> 
> On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:
> I'm hoping someone has encountered this weird behavior or something like it 
> before and can point me down a path, because all my research has turned up 
> nothing so far.
>  
> 
> I had an email account recently get breached due to a re-used password, and 
> that account was used to send a bunch of spam out from a server I help 
> manage.  We changed the password on the account as soon as we found it 
> happening and the outbound flood stopped.
>  
> 
> Shortly after that, however, I started seeing a very, very strange behavior.  
> Sometimes, and I haven’t yet been able to identify the trigger or pattern, 
> when users on this server send email to a forward that contains around 50 or 
> so email addresses (they use it like a private distribution list) they will 
> get anywhere from 1-10 bounces from Gmail.  Not every email sent to the 
> forward has this happen, and not even every email from a particular user.
>  
> 
> The outbound spamming caused the server’s reputation to go in the tank with 
> Google, and if it weren’t for that, I wouldn’t know this was happening, 
> because they get the bounces from Gmail accounts that absolutely ARE NOT in 
> the forward or part of the email chain AT ALL.
>  
> 
> I’m kind of freaking out here because while I haven’t found a breach of the 
> actual server / OS, this feels like someone has been able to inject something 
> somewhere into my server that I simply can’t find.  It is especially 
> troubling because a user who is not on this domain, but is part of the group 
> and therefore uses the forward from time to time, sent something to the 
> forward today and got Gmail bounces.   
>  
> 
> I don’t see anything in the send log that shows the server even trying to 
> send to Gmail, which only adds to the ghost story.
>  
> 
> Any ideas, paths to go down, anything would be greatly appreciated here.  I’m 
> about to just rebuild the whole thing from scratch on a new VM, but if I’m 
> overlooking something simple don’t want to put the users through that.
>  
> 
> Thanks in advance.
>  
> 
> Chas
-
To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com
For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com

RE: [qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

Thanks Eric and Remo, I appreciate the assistance. 

 

I’d forgotten about the simscan setting for the cdb to up the logging, it’s 
been a LONG time since I’ve had to do that.

 

My queue is empty.  Nothing clogged up, it’s not residual stuff; that said, I’m 
watching it pretty closely right now.

 

No .qmail files.  I logged into the db and looked – the forward is all in the 
database, and I don’t have any .qmail files that I can find outside of the skel 
folder.  My users aren’t ‘real’ users on the system, they’re all virtual users.

 

Part of the problem is that the bouncing from Gmail has happened to different 
users at different times, and at other times it doesn’t happen to them.  It is 
so very bizarre.  Hopefully with an increased logging level I can find enough 
to trace this down to its actual origins.  

 

From: Eric Broch [mailto:ebr...@whitehorsetc.com] 
Sent: Sunday, August 16, 2020 4:13 PM
To: qmailtoaster-list@qmailtoaster.com
Subject: Re: [qmailtoaster] Distressing strange behavior

 

Yes forwards can be in a .qmail file or in the vpopmail database.

So, the bounces occurring presently, what's the originating account?

Is there anything in your queue (# qmailctl queue)?

 

On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:

As I understand the forwards setup in qmailadmin those are in the database, 
right?

The address that was compromised hasn't sent any email since the password 
change. 

I hadn't thought about looking at qmail-inject. I'll dig into watching that 
part of the process. 

Get TypeApp for Android <http://www.typeapp.com/r?b=15986>  

On Aug 16, 2020, at 3:14 PM, Eric Broch mailto:ebr...@whitehorsetc.com> > wrote: 

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail accounts 
they aren't limited to just these forwards, they also have the option in the 
email client to add gmail accounts in the "To:" field of the email they're 
sending, thus bounces from gmail accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in the send log.

 

On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 

I'm hoping someone has encountered this weird behavior or something like it 
before and can point me down a path, because all my research has turned up 
nothing so far. 

  

I had an email account recently get breached due to a re-used password, and 
that account was used to send a bunch of spam out from a server I help manage.  
We changed the password on the account as soon as we found it happening and the 
outbound flood stopped. 

  

Shortly after that, however, I started seeing a very, very strange behavior.  
Sometimes, and I haven’t yet been able to identify the trigger or pattern, when 
users on this server send email to a forward that contains around 50 or so 
email addresses (they use it like a private distribution list) they will get 
anywhere from 1-10 bounces from Gmail.  Not every email sent to the forward has 
this happen, and not even every email from a particular user. 

  

The outbound spamming caused the server’s reputation to go in the tank with 
Google, and if it weren’t for that, I wouldn’t know this was happening, because 
they get the bounces from Gmail accounts that absolutely ARE NOT in the forward 
or part of the email chain AT ALL. 

  

I’m kind of freaking out here because while I haven’t found a breach of the 
actual server / OS, this feels like someone has been able to inject something 
somewhere into my server that I simply can’t find.  It is especially troubling 
because a user who is not on this domain, but is part of the group and 
therefore uses the forward from time to time, sent something to the forward 
today and got Gmail bounces.

  

I don’t see anything in the send log that shows the server even trying to send 
to Gmail, which only adds to the ghost story. 

  

Any ideas, paths to go down, anything would be greatly appreciated here.  I’m 
about to just rebuild the whole thing from scratch on a new VM, but if I’m 
overlooking something simple don’t want to put the users through that. 

  

Thanks in advance. 

  

Chas 

Re: [qmailtoaster] Distressing strange behavior

[Eric Broch]

Yes forwards can be in a .qmail file or in the vpopmail database.

So, the bounces occurring presently, what's the originating account?

Is there anything in your queue (# qmailctl queue)?


On 8/16/2020 2:46 PM, Charles Hockenbarger wrote:
As I understand the forwards setup in qmailadmin those are in the 
database, right?


The address that was compromised hasn't sent any email since the 
password change.


I hadn't thought about looking at qmail-inject. I'll dig into watching 
that part of the process.


Get TypeApp for Android 
On Aug 16, 2020, at 3:14 PM, Eric Broch > wrote:


How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail
accounts they aren't limited to just these forwards, they also
have the option in the email client to add gmail accounts in the
"To:" field of the email they're sending, thus bounces from gmail
accounts that aren't in your forwards file.

Also, qmail-inject puts mail in the queue and you'll see it in the
send log.


On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:


I'm hoping someone has encountered this weird behavior or
something like it before and can point me down a path, because
all my research has turned up nothing so far.

I had an email account recently get breached due to a re-used
password, and that account was used to send a bunch of spam out
from a server I help manage.  We changed the password on the
account as soon as we found it happening and the outbound flood
stopped.

Shortly after that, however, I started seeing a very, very
strange behavior. Sometimes, and I haven’t yet been able to
identify the trigger or pattern, when users on this server send
email to a forward that contains around 50 or so email addresses
(they use it like a private distribution list) they will get
anywhere from 1-10 bounces from Gmail. Not every email sent to
the forward has this happen, and not even every email from a
particular user.

The outbound spamming caused the server’s reputation to go in the
tank with Google, and if it weren’t for that, I wouldn’t know
this was happening, because they get the bounces from Gmail
accounts that absolutely ARE NOT in the forward or part of the
email chain AT ALL.

I’m kind of freaking out here because while I haven’t found a
breach of the actual server / OS, this feels like someone has
been able to inject something somewhere into my server that I
simply can’t find.  It is especially troubling because a user who
is not on this domain, but is part of the group and therefore
uses the forward from time to time, sent something to the forward
today and got Gmail bounces.

I don’t see anything in the send log that shows the server even
trying to send to Gmail, which only adds to the ghost story.

Any ideas, paths to go down, anything would be greatly
appreciated here.  I’m about to just rebuild the whole thing from
scratch on a new VM, but if I’m overlooking something simple
don’t want to put the users through that.

Thanks in advance.

Chas

Re: [qmailtoaster] Distressing strange behavior

[Remo Mattei]

here are some steps to do

1) enable more debugs 
:allow,SIMSCAN_DEBUG=“5” to /etc/tcprules.d/tcp.smtp 
then make sure you run the 
qmailctl cdb
Reloaded /etc/tcprules.d/tcp.smtp
Reloaded /var/qmail/control/badmimetypes.cdb
Reloaded /var/qmail/control/badloadertypes.cdb
Reloaded /var/qmail/control/simversions.cdb
Reloaded /var/qmail/control/simcontrol.cdb

I would also check if you have something in the queue

qmHandle -L 
will show if you do. I would probably remove what’s not valide with the 
qmHandle or all with qmHandle -D :) careful since that will delete all in the 
queue so some valid msg will be erased. 

then once you enable the logs I have mine in the diff folder under the 
/var/log/qmail and I normally check both the submission and the send 
here is what mine looks like:

# ls -al
total 28
drwxr-x---   7 qmaill qmail 4096 Sep 14  2019 .
drwxr-xr-x. 23 root   root  4096 Aug 16 03:26 ..
drwxr-x---   2 qmaill qmail 4096 Aug 15 10:38 send
drwxr-x---   2 qmaill qmail 4096 Aug 15 21:01 smtp
drwxr-x---   2 qmaill qmail 4096 Aug 16 10:02 smtps
drwxr-x---   2 qmaill qmail 4096 Aug  5 05:44 submission
drwx--   2 qmaill qmail 4096 Aug  5 05:44 vpopmaild
so less, tail you pick which one to use. I have also used multitail which makes 
life much easier ;) 

Remo
 

> On Aug 16, 2020, at 1:46 PM, Charles Hockenbarger  wrote:
> 
> As I understand the forwards setup in qmailadmin those are in the database, 
> right?
> 
> The address that was compromised hasn't sent any email since the password 
> change. 
> 
> I hadn't thought about looking at qmail-inject. I'll dig into watching that 
> part of the process. 
> 
> Get TypeApp for Android 
> On Aug 16, 2020, at 3:14 PM, Eric Broch  > wrote:
> How do you have your forwards set up?
> 
> Is there any mail in your queue?
> 
> If someone hacked an account on your server with forwards to gmail accounts 
> they aren't limited to just these forwards, they also have the option in the 
> email client to add gmail accounts in the "To:" field of the email they're 
> sending, thus bounces from gmail accounts that aren't in your forwards file.
> 
> Also, qmail-inject puts mail in the queue and you'll see it in the send log.
> 
> 
> 
> On 8/16/2020 10:05 AM, Chas Hockenbarger wrote: 
>> I'm hoping someone has encountered this weird behavior or something like it 
>> before and can point me down a path, because all my research has turned up 
>> nothing so far.
>> 
>>  
>> I had an email account recently get breached due to a re-used password, and 
>> that account was used to send a bunch of spam out from a server I help 
>> manage.  We changed the password on the account as soon as we found it 
>> happening and the outbound flood stopped.
>> 
>>  
>> Shortly after that, however, I started seeing a very, very strange behavior. 
>>  Sometimes, and I haven’t yet been able to identify the trigger or pattern, 
>> when users on this server send email to a forward that contains around 50 or 
>> so email addresses (they use it like a private distribution list) they will 
>> get anywhere from 1-10 bounces from Gmail.  Not every email sent to the 
>> forward has this happen, and not even every email from a particular user.
>> 
>>  
>> The outbound spamming caused the server’s reputation to go in the tank with 
>> Google, and if it weren’t for that, I wouldn’t know this was happening, 
>> because they get the bounces from Gmail accounts that absolutely ARE NOT in 
>> the forward or part of the email chain AT ALL.
>> 
>>  
>> I’m kind of freaking out here because while I haven’t found a breach of the 
>> actual server / OS, this feels like someone has been able to inject 
>> something somewhere into my server that I simply can’t find.  It is 
>> especially troubling because a user who is not on this domain, but is part 
>> of the group and therefore uses the forward from time to time, sent 
>> something to the forward today and got Gmail bounces.   
>> 
>>  
>> I don’t see anything in the send log that shows the server even trying to 
>> send to Gmail, which only adds to the ghost story.
>> 
>>  
>> Any ideas, paths to go down, anything would be greatly appreciated here.  
>> I’m about to just rebuild the whole thing from scratch on a new VM, but if 
>> I’m overlooking something simple don’t want to put the users through that.
>> 
>>  
>> Thanks in advance.
>> 
>>  
>> Chas

Re: [qmailtoaster] Distressing strange behavior

[Charles Hockenbarger]

As I understand the forwards setup in qmailadmin those are in the database, 
right?

The address that was compromised hasn't sent any email since the password 
change.

I hadn't thought about looking at qmail-inject. I'll dig into watching that 
part of the process.

⁣Get TypeApp for Android ​

On Aug 16, 2020, 3:14 PM, at 3:14 PM, Eric Broch  
wrote:
>How do you have your forwards set up?
>
>Is there any mail in your queue?
>
>If someone hacked an account on your server with forwards to gmail
>accounts they aren't limited to just these forwards, they also have the
>
>option in the email client to add gmail accounts in the "To:" field of
>the email they're sending, thus bounces from gmail accounts that aren't
>
>in your forwards file.
>
>Also, qmail-inject puts mail in the queue and you'll see it in the send
>log.
>
>
>On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:
>>
>> I'm hoping someone has encountered this weird behavior or something
>> like it before and can point me down a path, because all my research
>> has turned up nothing so far.
>>
>> I had an email account recently get breached due to a re-used
>> password, and that account was used to send a bunch of spam out from
>a
>> server I help manage.  We changed the password on the account as soon
>
>> as we found it happening and the outbound flood stopped.
>>
>> Shortly after that, however, I started seeing a very, very strange
>> behavior.  Sometimes, and I haven’t yet been able to identify the
>> trigger or pattern, when users on this server send email to a forward
>
>> that contains around 50 or so email addresses (they use it like a
>> private distribution list) they will get anywhere from 1-10 bounces
>> from Gmail.  Not every email sent to the forward has this happen, and
>
>> not even every email from a particular user.
>>
>> The outbound spamming caused the server’s reputation to go in the
>tank
>> with Google, and if it weren’t for that, I wouldn’t know this was
>> happening, because they get the bounces from Gmail accounts that
>> absolutely ARE NOT in the forward or part of the email chain AT ALL.
>>
>> I’m kind of freaking out here because while I haven’t found a breach
>> of the actual server / OS, this feels like someone has been able to
>> inject something somewhere into my server that I simply can’t find. 
>> It is especially troubling because a user who is not on this domain,
>> but is part of the group and therefore uses the forward from time to
>> time, sent something to the forward today and got Gmail bounces.
>>
>> I don’t see anything in the send log that shows the server even
>trying
>> to send to Gmail, which only adds to the ghost story.
>>
>> Any ideas, paths to go down, anything would be greatly appreciated
>> here.  I’m about to just rebuild the whole thing from scratch on a
>new
>> VM, but if I’m overlooking something simple don’t want to put the
>> users through that.
>>
>> Thanks in advance.
>>
>> Chas
>>

Re: [qmailtoaster] Distressing strange behavior

[Eric Broch]

How do you have your forwards set up?

Is there any mail in your queue?

If someone hacked an account on your server with forwards to gmail 
accounts they aren't limited to just these forwards, they also have the 
option in the email client to add gmail accounts in the "To:" field of 
the email they're sending, thus bounces from gmail accounts that aren't 
in your forwards file.


Also, qmail-inject puts mail in the queue and you'll see it in the send log.


On 8/16/2020 10:05 AM, Chas Hockenbarger wrote:


I'm hoping someone has encountered this weird behavior or something 
like it before and can point me down a path, because all my research 
has turned up nothing so far.


I had an email account recently get breached due to a re-used 
password, and that account was used to send a bunch of spam out from a 
server I help manage.  We changed the password on the account as soon 
as we found it happening and the outbound flood stopped.


Shortly after that, however, I started seeing a very, very strange 
behavior.  Sometimes, and I haven’t yet been able to identify the 
trigger or pattern, when users on this server send email to a forward 
that contains around 50 or so email addresses (they use it like a 
private distribution list) they will get anywhere from 1-10 bounces 
from Gmail.  Not every email sent to the forward has this happen, and 
not even every email from a particular user.


The outbound spamming caused the server’s reputation to go in the tank 
with Google, and if it weren’t for that, I wouldn’t know this was 
happening, because they get the bounces from Gmail accounts that 
absolutely ARE NOT in the forward or part of the email chain AT ALL.


I’m kind of freaking out here because while I haven’t found a breach 
of the actual server / OS, this feels like someone has been able to 
inject something somewhere into my server that I simply can’t find.  
It is especially troubling because a user who is not on this domain, 
but is part of the group and therefore uses the forward from time to 
time, sent something to the forward today and got Gmail bounces.


I don’t see anything in the send log that shows the server even trying 
to send to Gmail, which only adds to the ghost story.


Any ideas, paths to go down, anything would be greatly appreciated 
here.  I’m about to just rebuild the whole thing from scratch on a new 
VM, but if I’m overlooking something simple don’t want to put the 
users through that.


Thanks in advance.

Chas

Re: [qmailtoaster] Distressing strange behavior

[Charles Hockenbarger]

Thanks, Remo. I don't see any http logins for the compromised account but I'll 
try there.

Sorry for the stupid question, but how do we up the logging level for qmail 
logs? I've never had to do that and my searching hasn't shown me anything. I've 
got debug on for dovecot, though that's not really where the problem seems to 
be.

⁣Get TypeApp for Android ​

On Aug 16, 2020, 11:17 AM, at 11:17 AM, r...@mattei.org wrote:
>I would suggest to stop httpd normally when I saw something like that
>in one of my old server that i now displaced and replaced with CentOS 7
>the user found that loop to send. I would enable debug on all outgoing
>which is how I found that hole.
>
>It sucks I know. For google that’s something more work for reputation.
>
>Ciao
>> Il giorno 16 ago 2020, alle ore 09:05, Chas Hockenbarger
> ha scritto:
>>
>> 
>> I'm hoping someone has encountered this weird behavior or something
>like it before and can point me down a path, because all my research
>has turned up nothing so far.
>>
>> I had an email account recently get breached due to a re-used
>password, and that account was used to send a bunch of spam out from a
>server I help manage.  We changed the password on the account as soon
>as we found it happening and the outbound flood stopped.
>>
>> Shortly after that, however, I started seeing a very, very strange
>behavior.  Sometimes, and I haven’t yet been able to identify the
>trigger or pattern, when users on this server send email to a forward
>that contains around 50 or so email addresses (they use it like a
>private distribution list) they will get anywhere from 1-10 bounces
>from Gmail.  Not every email sent to the forward has this happen, and
>not even every email from a particular user.
>>
>> The outbound spamming caused the server’s reputation to go in the
>tank with Google, and if it weren’t for that, I wouldn’t know this was
>happening, because they get the bounces from Gmail accounts that
>absolutely ARE NOT in the forward or part of the email chain AT ALL.
>>
>> I’m kind of freaking out here because while I haven’t found a breach
>of the actual server / OS, this feels like someone has been able to
>inject something somewhere into my server that I simply can’t find.  It
>is especially troubling because a user who is not on this domain, but
>is part of the group and therefore uses the forward from time to time,
>sent something to the forward today and got Gmail bounces.
>>
>> I don’t see anything in the send log that shows the server even
>trying to send to Gmail, which only adds to the ghost story.
>>
>> Any ideas, paths to go down, anything would be greatly appreciated
>here.  I’m about to just rebuild the whole thing from scratch on a new
>VM, but if I’m overlooking something simple don’t want to put the users
>through that.
>>
>> Thanks in advance.
>>
>> Chas
>
>
>
>
>-
>To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com
>For additional commands, e-mail:
>qmailtoaster-list-h...@qmailtoaster.com

Re: [qmailtoaster] Distressing strange behavior

[remo]

I would suggest to stop httpd normally when I saw something like that in one of 
my old server that i now displaced and replaced with CentOS 7 the user found 
that loop to send. I would enable debug on all outgoing which is how I found 
that hole. 

It sucks I know. For google that’s something more work for reputation. 

Ciao
> Il giorno 16 ago 2020, alle ore 09:05, Chas Hockenbarger  
> ha scritto:
> 
> 
> I'm hoping someone has encountered this weird behavior or something like it 
> before and can point me down a path, because all my research has turned up 
> nothing so far.
>  
> I had an email account recently get breached due to a re-used password, and 
> that account was used to send a bunch of spam out from a server I help 
> manage.  We changed the password on the account as soon as we found it 
> happening and the outbound flood stopped.
>  
> Shortly after that, however, I started seeing a very, very strange behavior.  
> Sometimes, and I haven’t yet been able to identify the trigger or pattern, 
> when users on this server send email to a forward that contains around 50 or 
> so email addresses (they use it like a private distribution list) they will 
> get anywhere from 1-10 bounces from Gmail.  Not every email sent to the 
> forward has this happen, and not even every email from a particular user.
>  
> The outbound spamming caused the server’s reputation to go in the tank with 
> Google, and if it weren’t for that, I wouldn’t know this was happening, 
> because they get the bounces from Gmail accounts that absolutely ARE NOT in 
> the forward or part of the email chain AT ALL.
>  
> I’m kind of freaking out here because while I haven’t found a breach of the 
> actual server / OS, this feels like someone has been able to inject something 
> somewhere into my server that I simply can’t find.  It is especially 
> troubling because a user who is not on this domain, but is part of the group 
> and therefore uses the forward from time to time, sent something to the 
> forward today and got Gmail bounces.   
>  
> I don’t see anything in the send log that shows the server even trying to 
> send to Gmail, which only adds to the ghost story.
>  
> Any ideas, paths to go down, anything would be greatly appreciated here.  I’m 
> about to just rebuild the whole thing from scratch on a new VM, but if I’m 
> overlooking something simple don’t want to put the users through that.
>  
> Thanks in advance.
>  
> Chas
-
To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com
For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com

[qmailtoaster] Distressing strange behavior

[Chas Hockenbarger]

I'm hoping someone has encountered this weird behavior or something like it
before and can point me down a path, because all my research has turned up
nothing so far.

 

I had an email account recently get breached due to a re-used password, and
that account was used to send a bunch of spam out from a server I help
manage.  We changed the password on the account as soon as we found it
happening and the outbound flood stopped.

 

Shortly after that, however, I started seeing a very, very strange behavior.
Sometimes, and I haven't yet been able to identify the trigger or pattern,
when users on this server send email to a forward that contains around 50 or
so email addresses (they use it like a private distribution list) they will
get anywhere from 1-10 bounces from Gmail.  Not every email sent to the
forward has this happen, and not even every email from a particular user.

 

The outbound spamming caused the server's reputation to go in the tank with
Google, and if it weren't for that, I wouldn't know this was happening,
because they get the bounces from Gmail accounts that absolutely ARE NOT in
the forward or part of the email chain AT ALL.

 

I'm kind of freaking out here because while I haven't found a breach of the
actual server / OS, this feels like someone has been able to inject
something somewhere into my server that I simply can't find.  It is
especially troubling because a user who is not on this domain, but is part
of the group and therefore uses the forward from time to time, sent
something to the forward today and got Gmail bounces.   

 

I don't see anything in the send log that shows the server even trying to
send to Gmail, which only adds to the ghost story.

 

Any ideas, paths to go down, anything would be greatly appreciated here.
I'm about to just rebuild the whole thing from scratch on a new VM, but if
I'm overlooking something simple don't want to put the users through that.

 

Thanks in advance.

 

Chas