Re: smtpd_recipient_restrictions in mongodb?
The user database is set up because my clients, some bad users, send mass mailings to non-existent accounts, such as hotmail, gmail, which usually measure the number of IP bounces. Also sometimes there are cases where the PC or mobile device is infected with a virus and they use the email account to send spam, so I add those accounts that they sent to my list because they can see spam trap. the database has maintenance, the accounts within it are cleaned as I check them with another system, I only clean those that exist. here I add some info from the table mysql> desc virtual_sender_access; +---+-+--+-+-+-+ | Field| Type | Null | Key | Default| Extra | +---+-+--+-+-+-+ | id | int(11)| NO | PRI | NULL | auto_increment | | source | varchar(64) | NO | MUL || | | access | varchar(64) | NO ||| | | created_on | timestamp | NO || -00-00 00:00:00 | | | check_bounce | int(11)| NO || NULL | | | last_modified | timestamp | NO || CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP | +---+-+--+-+-+-+ 6 rows in set (0.00 sec) mysql> show index from virtual_sender_access ; +---++--+--+-+---+-+--++--++-+ ---+ | Table| Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | +---++--+--+-+---+-+--++--++-+ ---+ | virtual_sender_access | 0 | PRIMARY | 1 | id | A|4225535 |NULL | NULL | | BTREE || | | virtual_sender_access | 1 | source | 1 | source | A|4225535 |NULL | NULL | | BTREE || | +---++--+--+-+---+-+--++--++-+ ---+ 2 rows in set (0.00 sec) I would like to know if I can use mongo in the smtpd_recipient_restrictions parameter anyway. Thank you. Regards, El lun., 13 de jul. de 2020 a la(s) 18:05, Ralph Seichter ( ra...@ml.seichter.de) escribió: > * SysAdmin EM: > > > I use a suppression list where I block domains and email accounts that > > don't exist to prevent the reputation of my IP addresses from going > > down. [...] The table currently has 4 million rows, so I am looking > > for a faster database engine than MySql. > > Can you clarify for me: You try to maintain a database for domains that > do not exist and email accounts that do not exist? How would that work? > Both the number of nonexistent domains and and accounts are, while not > infinite, huge. Why not list the domains and accounts that do exist? > These numbers are definitely finite. > > As for your choice of database: I am a fan of MongoDB because of its > capability of storing unstructured data, but here your data does have a > well-defined, uniform structure. If you use a proper index, MySQL or > similar relational databases should be quick as blazes. > > -Ralph >
Re: smtpd_recipient_restrictions in mongodb?
* SysAdmin EM: > I use a suppression list where I block domains and email accounts that > don't exist to prevent the reputation of my IP addresses from going > down. [...] The table currently has 4 million rows, so I am looking > for a faster database engine than MySql. Can you clarify for me: You try to maintain a database for domains that do not exist and email accounts that do not exist? How would that work? Both the number of nonexistent domains and and accounts are, while not infinite, huge. Why not list the domains and accounts that do exist? These numbers are definitely finite. As for your choice of database: I am a fan of MongoDB because of its capability of storing unstructured data, but here your data does have a well-defined, uniform structure. If you use a proper index, MySQL or similar relational databases should be quick as blazes. -Ralph
smtpd_recipient_restrictions in mongodb?
Hello, I use a suppression list where I block domains and email accounts that don't exist to prevent the reputation of my IP addresses from going down. smtpd_recipient_restrictions = check_recipient_access mysql:/etc/postfix/ mysql-virtual-recipient-access.cf # connection to mysql hosts = 172.x.x.x user = myuser password = mypass dbname = blacklist query = SELECT access FROM virtual_sender_access WHERE source='%s' The table currently has 4 million rows, so I am looking for a faster database engine than MySql. I read this postshttp://www.postfix.org/DATABASE_README.html and http://www.postfix.org/access.5.html but I didn't find any information. Does postfix have support for mongo database? Or what method do you recommend to host a large number of unknown users and invalid domains? Regards,
Re: Postfix MySql result in bolean
SysAdmin EM skrev den 2020-07-13 20:43: SELECT COUNT(1) FROM virtual_sender_access WHERE source = 'u...@gmail.com'? valid sql query, its just not meaningfull to postfix, and you did not provide how postfix should use (1)
Re: Postfix MySql result in bolean
SysAdmin EM: > Hello, > > I am using a suppression list in MySql with Postfix. > > smtpd_recipient_restrictions = check_recipient_access mysql:/etc/postfix/ > mysql-virtual-recipient-access.cf > > query = SELECT access FROM virtual_sender_access WHERE source='%s' > > the question is, is it possible to use boolean data? > > like this: > > SELECT COUNT(1) FROM virtual_sender_access WHERE source = 'u...@gmail.com'? The check_recipient_access lookup result must be as described in http://www.postfix.org/access.5.html Wietse
Postfix MySql result in bolean
Hello, I am using a suppression list in MySql with Postfix. smtpd_recipient_restrictions = check_recipient_access mysql:/etc/postfix/ mysql-virtual-recipient-access.cf query = SELECT access FROM virtual_sender_access WHERE source='%s' the question is, is it possible to use boolean data? like this: SELECT COUNT(1) FROM virtual_sender_access WHERE source = 'u...@gmail.com'? Regards,
Re: Nothing in /var/log/maillog under stress
On 2020-07-13 02:08, Greg Sims wrote: I updated my maillog processing tool to make use of journalctl. This is working well and I can now see the "missing" maillog entries with my tool. This is a great step in the right direction. That sounds great. I have rsyslog running which looks like it might be redundant -- based on the serverfault post you supplied. I will try running without rsyslog and see what happens. I am aware of the systemd journal rate limits from CentOS 7. I will do additional research to know when I hit these limits and make needed adjustments if I do. I added this to /etc/system/journal.conf.d/journald.conf and it works for me. [Journal] RateLimitIntervalSec=1s RateLimitBurst=0 Thanks for your help Christian! I am now able to accomplish my goals using journalctl. I am more than willing to collect data to help determine why the three minutes of log data is not making it to /var/log/maillog. To be honest, I do not know how to "... find out how your syslog daemon gets the messages from the systemd journal.". Greg Sims On Sun, Jul 12, 2020 at 3:51 PM Christian Kivalo wrote: On 2020-07-13 00:10, Greg Sims wrote: Thank you Christian. I am running on CentOS 8.2 and the name of the service is "postfix.service". When I enter: journalctl -u postfix.service --since="2020-07-12 03:06:00" --until="2020-07-12 03:11:00" I see all of the missing data that should be in /var/log/maillog -- almost 50,000 records. You discovered a way to gain access to the missing data! The big question for me continues to be, why did this data not make it to /var/log/maillog? You'd have to find out how your syslog daemon get the messages from the systemd journal. What syslog daemon do you have installed? Be aware that systemd journal has some rate limits which can lead to loss of log messages, see the man 5 journald.conf I found this https://serverfault.com/questions/959982/is-rsyslog-redundant-on-when-using-journald which covers rsyslog on centos 7. There is an import module for systemd journal. On my server rsyslog is configured to create a log socket at /var/spool/postfix/dev/log and ignore systemd journal and that works well for my use case. Greg Sims On Sun, Jul 12, 2020 at 2:40 PM Christian Kivalo wrote: On 2020-07-12 23:01, Greg Sims wrote: Nothing Christian: [root@mail0 postfix]# journalctl -u postfix@-.service --since="2020-07-12 03:06:00" --until="2020-07-12 03:11:00" -- Logs begin at Sat 2020-07-11 09:35:28 CDT, end at Sun 2020-07-12 15:50:00 CDT. -- -- No entries -- Maybe your systemd unit is named slightly different as in debian, postfix@-.service is what tab completion makes for me... Is there anything in journalctl? What does systemctl status postfix show? You can have postfix log to a file as described in http://www.postfix.org/MAILLOG_README.html first and then fix your logging. -- Christian Kivalo -- Christian Kivalo -- Christian Kivalo
Re: Nothing in /var/log/maillog under stress
On 2020-07-13 03:57, Greg Sims wrote: I removed rsyslog using yum, rebooted the VM and made sure postfix was running. I then sent five emails from a remote VM using SMTP. I can see the postfix logs using journalctl. This set of postfix logs do not make it to /var/log/maillog. The five emails were delivered. I'm not sure if this is the expected behavior. This is expected as rsyslog writes to /var/log/maillog. Now you only have the journal except for those services that write to their own logfile directly... Apache is also running on this VM. I performed "tail /var/log/httpd/access_log" and can see Apache logging. ... like apache does. Greg Sims www.RayStedman.org [1] On Sun, Jul 12, 2020 at 5:08 PM Greg Sims wrote: I updated my maillog processing tool to make use of journalctl. This is working well and I can now see the "missing" maillog entries with my tool. This is a great step in the right direction. I have rsyslog running which looks like it might be redundant -- based on the serverfault post you supplied. I will try running without rsyslog and see what happens. I am aware of the systemd journal rate limits from CentOS 7. I will do additional research to know when I hit these limits and make needed adjustments if I do. Thanks for your help Christian! I am now able to accomplish my goals using journalctl. I am more than willing to collect data to help determine why the three minutes of log data is not making it to /var/log/maillog. To be honest, I do not know how to "... find out how your syslog daemon gets the messages from the systemd journal.". Greg Sims On Sun, Jul 12, 2020 at 3:51 PM Christian Kivalo wrote: On 2020-07-13 00:10, Greg Sims wrote: Thank you Christian. I am running on CentOS 8.2 and the name of the service is "postfix.service". When I enter: journalctl -u postfix.service --since="2020-07-12 03:06:00" --until="2020-07-12 03:11:00" I see all of the missing data that should be in /var/log/maillog -- almost 50,000 records. You discovered a way to gain access to the missing data! The big question for me continues to be, why did this data not make it to /var/log/maillog? You'd have to find out how your syslog daemon get the messages from the systemd journal. What syslog daemon do you have installed? Be aware that systemd journal has some rate limits which can lead to loss of log messages, see the man 5 journald.conf I found this https://serverfault.com/questions/959982/is-rsyslog-redundant-on-when-using-journald which covers rsyslog on centos 7. There is an import module for systemd journal. On my server rsyslog is configured to create a log socket at /var/spool/postfix/dev/log and ignore systemd journal and that works well for my use case. Greg Sims On Sun, Jul 12, 2020 at 2:40 PM Christian Kivalo wrote: On 2020-07-12 23:01, Greg Sims wrote: Nothing Christian: [root@mail0 postfix]# journalctl -u postfix@-.service --since="2020-07-12 03:06:00" --until="2020-07-12 03:11:00" -- Logs begin at Sat 2020-07-11 09:35:28 CDT, end at Sun 2020-07-12 15:50:00 CDT. -- -- No entries -- Maybe your systemd unit is named slightly different as in debian, postfix@-.service is what tab completion makes for me... Is there anything in journalctl? What does systemctl status postfix show? You can have postfix log to a file as described in http://www.postfix.org/MAILLOG_README.html first and then fix your logging. -- Christian Kivalo -- Christian Kivalo Links: -- [1] https://www.RayStedman.org -- Christian Kivalo
Re: Nothing in /var/log/maillog under stress
I removed rsyslog using yum, rebooted the VM and made sure postfix was running. I then sent five emails from a remote VM using SMTP. I can see the postfix logs using journalctl. This set of postfix logs do not make it to /var/log/maillog. The five emails were delivered. I'm not sure if this is the expected behavior. Apache is also running on this VM. I performed "tail /var/log/httpd/access_log" and can see Apache logging. Greg Sims www.RayStedman.org On Sun, Jul 12, 2020 at 5:08 PM Greg Sims wrote: > I updated my maillog processing tool to make use of journalctl. This is > working well and I can now see the "missing" maillog entries with my tool. > This is a great step in the right direction. > > I have rsyslog running which looks like it might be redundant -- based on > the serverfault post you supplied. I will try running without rsyslog and > see what happens. > > I am aware of the systemd journal rate limits from CentOS 7. I will do > additional research to know when I hit these limits and make needed > adjustments if I do. > > Thanks for your help Christian! I am now able to accomplish my goals > using journalctl. > > I am more than willing to collect data to help determine why the three > minutes of log data is not making it to /var/log/maillog. To be honest, I > do not know how to "... find out how your syslog daemon gets the messages > from the systemd journal.". > > Greg Sims > > On Sun, Jul 12, 2020 at 3:51 PM Christian Kivalo > wrote: > >> >> >> On 2020-07-13 00:10, Greg Sims wrote: >> > Thank you Christian. I am running on CentOS 8.2 and the name of the >> > service is "postfix.service". When I enter: >> > >> >> journalctl -u postfix.service --since="2020-07-12 03:06:00" >> >> --until="2020-07-12 03:11:00" >> > I see all of the missing data that should be in /var/log/maillog -- >> > almost 50,000 records. You discovered a way to gain access to the >> > missing data! >> > >> > The big question for me continues to be, why did this data not make it >> > to /var/log/maillog? >> You'd have to find out how your syslog daemon get the messages from the >> systemd journal. What syslog daemon do you have installed? >> Be aware that systemd journal has some rate limits which can lead to >> loss of log messages, see the man 5 journald.conf >> >> I found this >> >> https://serverfault.com/questions/959982/is-rsyslog-redundant-on-when-using-journald >> which covers rsyslog on centos 7. There is an import module for systemd >> journal. >> >> On my server rsyslog is configured to create a log socket at >> /var/spool/postfix/dev/log and ignore systemd journal and that works >> well for my use case. >> >> > Greg Sims >> > >> > On Sun, Jul 12, 2020 at 2:40 PM Christian Kivalo >> > wrote: >> > >> >> On 2020-07-12 23:01, Greg Sims wrote: >> >>> Nothing Christian: >> >>> >> [root@mail0 postfix]# journalctl -u postfix@-.service >> --since="2020-07-12 03:06:00" --until="2020-07-12 03:11:00" >> -- Logs begin at Sat 2020-07-11 09:35:28 CDT, end at Sun >> >> 2020-07-12 >> 15:50:00 CDT. -- >> -- No entries -- >> >> Maybe your systemd unit is named slightly different as in debian, >> >> postfix@-.service is what tab completion makes for me... >> >> >> >> Is there anything in journalctl? What does systemctl status postfix >> >> show? >> >> >> >> You can have postfix log to a file as described in >> >> http://www.postfix.org/MAILLOG_README.html first and then fix your >> >> logging. >> >> >> >> -- >> >> Christian Kivalo >> >> -- >> Christian Kivalo >> >
