hmm - given that after "header_cache_readonly = yes" no new records are added into "dbmail_headername" i think the existing ones should not be purged by "dbmail-util"
"newsgroups", "x-msmail-priority", "x-msoesrec", "x-priority"
and "x-ref" obviously are not in any of my test-messages and
so never come back while they are in Paul's "keep list"
the question now is: is it's worth to make sure in dbmail-util
that "dbmail_headername" values are not removed in read-only
mode or is it very likely that in real production always
messages with them are present or as "fallback" a generic
message containing them all should be created
>> So at the very least keep:
>> Cc
>> Content-Type
>> Date
>> From
>> In-Reply-To
>> Message-ID
>> Newsgroups
>> Priority
>> References
>> Subject
>> To
>> X-MSMail-Priority
>> X-MSOESRec
>> X-Priority
>> X-Ref
MariaDB [dbmail]> select * from dbmail_headername order by headername;
+-----+-------------------+
| id | headername |
+-----+-------------------+
| 21 | cc |
| 1 | content-type |
| 2 | date |
| 3 | from |
| 220 | importance |
| 22 | in-reply-to |
| 4 | message-id |
| 5 | mime-version |
| 377 | newsgroups |
| 378 | priority |
| 24 | references |
| 179 | reply-to |
| 229 | status |
| 9 | subject |
| 188 | thread-index |
| 187 | thread-topic |
| 10 | to |
| 386 | x-msmail-priority |
| 387 | x-msoesrec |
| 388 | x-priority |
| 389 | x-ref |
+-----+-------------------+
21 rows in set (0.00 sec)
[root@testserver:~]$
/usr/sbin/dbmail-util -y -c -t -u -b -p -d -r 24h
MariaDB [dbmail]> select * from dbmail_headername order by headername;
+-----+--------------+
| id | headername |
+-----+--------------+
| 21 | cc |
| 1 | content-type |
| 2 | date |
| 3 | from |
| 220 | importance |
| 22 | in-reply-to |
| 4 | message-id |
| 5 | mime-version |
| 24 | references |
| 179 | reply-to |
| 229 | status |
| 9 | subject |
| 188 | thread-index |
| 187 | thread-topic |
| 10 | to |
+-----+--------------+
15 rows in set (0.00 sec)
Am 17.03.2014 18:46, schrieb Reindl Harald:
> caution: local testing shows "like 'x_%'" needs to be escaped as "like 'x\_%'"
> other wise the by Paul statet x-headers without a underscore are also removed
>
> i think i will prefer the other direction around and at the same
> time make sure that the 21 headers below are really present
>
> delete from dbmail_headername where headername != 'cc'
> and headername != 'content-type'
> and headername != 'date'
> and headername != 'from'
> and headername != 'importance'
> and headername != 'in-reply-to'
> and headername != 'message-id'
> and headername != 'mime-version'
> and headername != 'newsgroups'
> and headername != 'priority'
> and headername != 'references'
> and headername != 'reply-to'
> and headername != 'status'
> and headername != 'subject'
> and headername != 'thread-index'
> and headername != 'thread-topic'
> and headername != 'to'
> and headername != 'x-msmail-priority'
> and headername != 'x-msoesrec'
> and headername != 'x-priority'
> and headername != 'x-ref';
>
> insert into dbmail_headername (headername) values ('cc');
> insert into dbmail_headername (headername) values ('content-type');
> insert into dbmail_headername (headername) values ('date');
> insert into dbmail_headername (headername) values ('from');
> insert into dbmail_headername (headername) values ('importance');
> insert into dbmail_headername (headername) values ('in-reply-to');
> insert into dbmail_headername (headername) values ('message-id');
> insert into dbmail_headername (headername) values ('mime-version');
> insert into dbmail_headername (headername) values ('newsgroups');
> insert into dbmail_headername (headername) values ('priority');
> insert into dbmail_headername (headername) values ('references');
> insert into dbmail_headername (headername) values ('reply-to');
> insert into dbmail_headername (headername) values ('status');
> insert into dbmail_headername (headername) values ('subject');
> insert into dbmail_headername (headername) values ('thread-index');
> insert into dbmail_headername (headername) values ('thread-topic');
> insert into dbmail_headername (headername) values ('to');
> insert into dbmail_headername (headername) values ('x-msmail-priority');
> insert into dbmail_headername (headername) values ('x-msoesrec');
> insert into dbmail_headername (headername) values ('x-priority');
> insert into dbmail_headername (headername) values ('x-ref');
>
> Am 17.03.2014 17:14, schrieb Reindl Harald:
>> i am aware, thanks for the warning!
>>
>> most important: does anybody see something in
>> that list which may make better to keep
>>
>> on my testserver i start now playing around with stress-testing
>> http://git.dbmail.eu/paul/dbmail/commit/?h=dbmail_3_1&id=3dc82d0455b6e87f3f6b63ba101a16ca1faddf70
>>
>> Am 17.03.2014 17:11, schrieb Paul J Stevens:
>>> Like I said, below query is going to take a very, very long time to run.
>>> And while it's running it's keeping a write lock on the
>>> dbmail_headervalue table, essentially blocking all message insertions.
>>>
>>> On 17-03-14 16:54, Reindl Harald wrote:
>>>>
>>>> use dbmail; delete from dbmail_headername where
>>>> headername like 'x_%'
>>>> or headername like '%abuse%'
>>>> or headername like '%apparently%'
>>>> or headername like '%bounce%'
>>>> or headername like '%campaign%'
>>>> or headername like '%exchange%'
>>>> or headername like 'helpdesk%'
>>>> or headername like '%java%'
>>>> or headername like '%linkedin%'
>>>> or headername like '%list-%'
>>>> or headername like '%newsticker%'
>>>> or headername like '%original%'
>>>> or headername like '%pool%'
>>>> or headername like '%precedence%'
>>>> or headername like '%presedence%'
>>>> or headername like '%resent%'
>>>> or headername like '%return%'
>>>> or headername like '%scanner%'
>>>> or headername like '%spam%'
>>>> or headername like '%univie%'
>>>> or headername like '%virus%'
>>>> or headername like 'attn%'
>>>> or headername like 'bounces%'
>>>> or headername like 'breadcrumbid%'
>>>> or headername like 'mailing-%'
>>>> or headername like 'mail-reply-%'
>>>> or headername like 'recipient_%'
>>>> or headername like 'resent-%'
>>>> or headername like 'x-abuse%'
>>>> or headername like 'x-account-%'
>>>> or headername like 'x-amavis-%'
>>>> or headername like 'x-amazon-%'
>>>> or headername like 'x-aol%'
>>>> or headername like 'x-apple-%'
>>>> or headername like 'x-assp-%'
>>>> or headername like 'x-authenticated-%'
>>>> or headername like 'x-av-%'
>>>> or headername like 'x-banan%'
>>>> or headername like 'x-bawag%'
>>>> or headername like 'x-bigfish%'
>>>> or headername like 'x-bounce%'
>>>> or headername like 'x-broadcast-%'
>>>> or headername like 'x-bugzilla-%'
>>>> or headername like 'x-byjm-data%'
>>>> or headername like 'x-cam%'
>>>> or headername like 'x-care2-%'
>>>> or headername like 'x-cloud%'
>>>> or headername like 'x-clx%'
>>>> or headername like 'x-cm-%'
>>>> or headername like 'x-copfilter%'
>>>> or headername like 'x-cron-%'
>>>> or headername like 'x-ctch-%'
>>>> or headername like 'x-direct%'
>>>> or headername like 'x-disclaim%'
>>>> or headername like 'x-dynect%'
>>>> or headername like 'x-ec-%'
>>>> or headername like 'x-egroups%'
>>>> or headername like 'x-email%'
>>>> or headername like 'x-emv-%'
>>>> or headername like 'x-enigmail%'
>>>> or headername like 'x-envelope%'
>>>> or headername like 'x-envelope-helo%'
>>>> or headername like 'x-epwebmail%'
>>>> or headername like 'x-eventbrite%'
>>>> or headername like 'x-evnnet-%'
>>>> or headername like 'x-face%'
>>>> or headername like 'x-fb- %'
>>>> or headername like 'x-fbridge%'
>>>> or headername like 'x-feedback-%'
>>>> or headername like 'x-felis-%'
>>>> or headername like 'x-forefront%'
>>>> or headername like 'x-forwarded%'
>>>> or headername like 'x-gfi-%'
>>>> or headername like 'x-gmx%'
>>>> or headername like 'x-grey%'
>>>> or headername like 'x-http-user-agent%'
>>>> or headername like 'x-iadb%'
>>>> or headername like 'x-in-dnswl%'
>>>> or headername like 'x-inf-%'
>>>> or headername like 'x-ing-%'
>>>> or headername like 'x-ironport%'
>>>> or headername like 'x-klm%'
>>>> or headername like 'x-kse%'
>>>> or headername like 'x-linetec%'
>>>> or headername like 'x-linkedin%'
>>>> or headername like 'x-m1-%'
>>>> or headername like 'x-mail2%'
>>>> or headername like 'x-mailer%'
>>>> or headername like 'x-mailgun%'
>>>> or headername like 'x-mailing%'
>>>> or headername like 'x-mailman%'
>>>> or headername like 'x-mailru%'
>>>> or headername like 'x-mdr%'
>>>> or headername like 'x-mimedefang%'
>>>> or headername like 'x-ms-exchange%'
>>>> or headername like 'x-mw-%'
>>>> or headername like 'x-nai%'
>>>> or headername like 'x-no-%'
>>>> or headername like 'x-notification%'
>>>> or headername like 'x-organization%'
>>>> or headername like 'x-origin%'
>>>> or headername like 'x-ovh%'
>>>> or headername like 'x-perlmx-%'
>>>> or headername like 'x-php%'
>>>> or headername like 'x-pmx-%'
>>>> or headername like 'x-qmail%'
>>>> or headername like 'x-qq-%'
>>>> or headername like 'x-redhat%'
>>>> or headername like 'x-report%'
>>>> or headername like 'x-rm-%'
>>>> or headername like 'x-rocket%'
>>>> or headername like 'x-roving%'
>>>> or headername like 'x-sa-%'
>>>> or headername like 'x-sanitize%'
>>>> or headername like 'x-secure%'
>>>> or headername like 'x-smtp%'
>>>> or headername like 'x-smtpcom-%'
>>>> or headername like 'x-source%'
>>>> or headername like 'x-spf-%'
>>>> or headername like 'x-spread%'
>>>> or headername like 'x-starscan%'
>>>> or headername like 'x-stayfriends%'
>>>> or headername like 'x-tbone%'
>>>> or headername like 'x-tele2-%'
>>>> or headername like 'x-tm_%'
>>>> or headername like 'x-tm-%'
>>>> or headername like 'x-touris%'
>>>> or headername like 'x-tpg-%'
>>>> or headername like 'x-tpg%'
>>>> or headername like 'x-track%'
>>>> or headername like 'x-tug%'
>>>> or headername like 'x-tumblr%'
>>>> or headername like 'x-typo3%'
>>>> or headername like 'x-ui%'
>>>> or headername like 'x-universal%'
>>>> or headername like 'x-unsent%'
>>>> or headername like 'x-unsubscribe%'
>>>> or headername like 'x-uri%'
>>>> or headername like 'x-user%'
>>>> or headername like 'x-virtual%'
>>>> or headername like 'x-vr-%'
>>>> or headername like 'x-webhouse%'
>>>> or headername like 'x-willhaben%'
>>>> or headername like 'x-yahoo%'
>>>> or headername like 'x-ymail%'
>>>> or headername like 'x-yml%';
signature.asc
Description: OpenPGP digital signature
_______________________________________________ DBmail mailing list [email protected] http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
