On Wed, 9 Sep 2026, Crystal Kolipe wrote: > On Wed, Sep 09, 2026 at 12:10:17PM +0100, Mark Hills wrote: > > On Wed, 9 Sep 2026, Crystal Kolipe wrote: > > > > > On Tue, Sep 08, 2026 at 08:42:18PM +0100, Mark Hills wrote: > > > > Here's an example: I would like our outbound SMTP servers to strip > > > > "Received" headers, to avoid exposing some gnarly details of our > > > > internal > > > > network (eg. prototype script below) > > > > > > > > There needs quite a lot of code to sit between smtpd and this script. > > > > > > About 300 lines of C. Possibly less. > > [...] > > > all that is required is field-splitting the lines that you receive via > > > the filter API, session management, and error handling. > > > > This is "all"(!) > > > > A bunch of state, a persistent process, and due care around field > > splitting, escaping and buffer handling. > > > > This is "and now draw the rest of the owl" > > > > That ~15 line example requires ~2,000% overhead with no discernable upside > > (but corresponding exponential increase in bugs) > > Well, adding a second filter API to smtpd wouldn't be 'free'. It would just > shift the burden of potential new bugs away from your code and towards the > smtpd code.
I agree, but this is exactly the benefit though. I (and others) are not smtpd developers and therefore this is _absolutely_ the best place place to shift the burden of bugs. And the closer to the core, the more there is a single implementation rather than repeating the burdern on each author/filter. > > > Written like that, it might sound ominous, but about 98% of this has > > > already been done for you, (we published a guide to writing smtpd > > > filters which includes examples of such logic). > > > > Please link to this guide? It does not appear to be in the docs. > > https://research.exoticsilicon.com/articles/mail_filters Just the length of this guide makes my point. The intro is not agreeable. The _point_ is to leverage high level languages appropriately to achieve things quickly. A user with C skills akin to demo_filter.c (pasted below, formatting preserved) and the time to code like this would be better patching a feature into smtpd itself. I may like to think I'm a C ninja, but I code in C to get me to high level constructs like awk where I can do my work quickly (and still without "bloated third-party dependencies") The truth here is that example shows enormous scope for fragility or bugs (at a glance, malloc() and write() results are unchecked, so at best the example is incomplete; and it omits bounds checking on parts of the protocol with smtpd) More will emerge if "session management" is added to match my trivial header example. > > If it exists then the project would be well placed to deploy this -- a > > "wrapper" filter which forks scripts on stdin/stdout, maintaned and > > installed as part of OpenSMTPd, bug-free. > > I guess you're looking for something equivalent to slowcgi for httpd. > > Well, it could be done. Patches are welcome :-). I'll add to my long to-do list... -- Mark /* * Copyright 2023, Exotic Silicon, all rights reserved. * * Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following * conditions are met: * * 1. This software is licensed exclusively under this specific license text. The license text may not be changed, and the software * including modified versions may not be re-licensed under any other license text. * 2. Redistributions of source code must retain the above copyright notice, this list of conditions, and the following disclaimer. * 3. Redistributions in binary form must reproduce the above copyright notice, this list of conditions, and the following * disclaimer in the documentation and/or other materials provided with the distribution. * 4. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product * includes software developed by Exotic Silicon. * 5. The name of Exotic Silicon must not be used to endorse or promote products derived from this software without specific prior * written permission. * 6. Redistributions of modified versions of the source code must be clearly identified as having been modified from the original. * 7. Redistributions in binary form that have been created from modified versions of the source code must clearly state in the * documentation and/or other materials provided with the distribution that the source code has been modified from the original. * * THIS SOFTWARE IS PROVIDED 'AS IS' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL EXOTIC SILICON BE LIABLE FOR ANY DIRECT, * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE * GOODS OR SERVICES; LOSS OF USE, DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER * IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * Demonstration email filter implementing timezone modification in date headers. * * This source code is supporting material for a programming tutorial. For more information, please visit: * * https://research.exoticsilicon.com/articles/mail_filters */ #include <unistd.h> #include <stdlib.h> #include <string.h> #include <time.h> #define RAW_BUFFER_SIZE 65536 #define LINE_BUFFER_SIZE 66560 #define DEBUG_OUT(x) { write (STDERR_FILENO, x, sizeof(x)-1); } /* * Global table of session IDs and single bytes of data area that the corresponding session can use. * * Format: * In memory, each record is 17 bytes long, (SR_SIZE), and stored as follows: * 16 bytes of ASCII characters for the session ID, without a terminating null. * 1 byte of memory that this session can use. */ #define MAX_SESSIONS 1024 #define SR_SIZE 17 struct st_pipeline { unsigned char * value[8]; unsigned int len[8]; unsigned int totlen[8]; unsigned int lastfield; }; /* * Parse the supplied line of input, and fill an array of pointers. * For each field: start of the field, length of the field, length to end of raw line, (including all subsequent fields). * * The last event specific or phase specific field is typically read as raw data, in which the pipe character is no longer considered a field delimiter. * For this purpose the length to end of raw line value is useful. */ int parse_pipeline(unsigned char * buffer_line_in, unsigned int buffer_line_in_len, struct st_pipeline * pipeline) { unsigned int i; unsigned int pipecount; pipecount=0; memset(pipeline, 0, sizeof(struct st_pipeline)); /* * We check for the first character being 0x00 even though this function doesn't otherwise parse null terminators. Since the first field shouldn't * ever begin with a null, this would almost certainly indicate a programming error elsewhere sending a valid line length but an incorrect pointer. */ if (buffer_line_in_len==0 || *buffer_line_in==0) { return (1); } pipeline->value[0]=buffer_line_in; for (i=1; i<buffer_line_in_len && pipecount < 7; i++) { if (*(buffer_line_in+i)=='|') { pipeline->len[pipecount]=i-(pipeline->value[pipecount]-buffer_line_in); pipeline->totlen[pipecount]=buffer_line_in_len-(pipeline->value[pipecount]-buffer_line_in); pipecount++; pipeline->value[pipecount]=buffer_line_in+i+1; } } pipeline->len[pipecount]=buffer_line_in_len-(pipeline->value[pipecount]-buffer_line_in); pipeline->totlen[pipecount]=buffer_line_in_len-(pipeline->value[pipecount]-buffer_line_in); pipeline->lastfield=pipecount; return (0); } /* * Fill the supplied output buffer with one line from the input raw buffer. * Read data in to the raw buffer as required. * The trailing newline in the output buffer will be replaced with a NULL, making the returned output buffer NULL terminated. */ int line_in(unsigned char * line_buffer, unsigned int * len, unsigned char * raw_buffer, unsigned int * raw_buffer_readpos, unsigned int * raw_buffer_writepos) { int bytesin; unsigned int outpos; outpos=0; while (1) { if (*raw_buffer_readpos==*raw_buffer_writepos) { *raw_buffer_readpos=0; bytesin=read(STDIN_FILENO, raw_buffer, RAW_BUFFER_SIZE); /* * If we get end of file, or an error reading stdin, just exit. */ if (bytesin==0) { DEBUG_OUT ("Error - unexpected EOF on STDIN\n"); return(1); } if (bytesin==-1 ) { DEBUG_OUT ("Error - got non-EOF error reading STDIN\n"); return(1); } *raw_buffer_writepos = bytesin; } /* * If we have already reached the end of the output buffer then exit, as this shouldn't happen during normal operation. */ if (outpos==LINE_BUFFER_SIZE) { DEBUG_OUT ("Error - buffer size exceeded\n"); return(1); } /* * Read a character from the global buffer, and put it in the output buffer. If it's 10, null terminate and return, else loop. */ if ((*(line_buffer+outpos++) = *(raw_buffer+(*raw_buffer_readpos)++)) == 0x0a) { *(line_buffer+outpos-1)=0; *len=outpos-1; return (0); } } } int session_get_add(unsigned char * session_table, unsigned int * sessions, unsigned char * sid, unsigned char ** data_area) { unsigned int n; for (n=0; n<*(sessions); n++) { if (memcmp(sid, session_table+(n * SR_SIZE), 16)==0) { *data_area=session_table+(n * SR_SIZE + 16); return (0); } } /* * No existing entry matching the session ID and token was found, so we add a new one. */ if (*(sessions) == MAX_SESSIONS) { return(2); } *data_area=session_table+(n * SR_SIZE + 16); memcpy(session_table+(n * SR_SIZE), sid, 16); (*sessions)++; /* * In more complex code we would probably zero the data area here. */ return (1); } int session_delete(unsigned char * session_table, unsigned int * sessions, unsigned char * sid) { unsigned int n; for (n=0; n<*(sessions); n++) { if (memcmp(sid, session_table+(n * SR_SIZE), 16)==0) { if (n==((*sessions)-1)) { (*sessions)--; return (0); } memcpy(session_table+(n * SR_SIZE), session_table+((n + 1) * SR_SIZE), ((*sessions) - n - 1) * SR_SIZE); (*sessions)--; return (0); } } DEBUG_OUT ("Session doesn't exist\n"); return (1); } int main() { unsigned char * session_table; unsigned char * raw_buffer; unsigned char * line_buffer; unsigned char * data_area; unsigned char * ascii_time_buffer; unsigned int raw_buffer_readpos; unsigned int raw_buffer_writepos; unsigned int line_buffer_len; unsigned int sessions; unsigned int result; unsigned int i; unsigned int flag_skip_line; unsigned int ascii_time_buffer_len; int offset; struct st_pipeline pipeline; struct tm time_tm; time_t timestamp; pledge ("stdio", NULL); raw_buffer_readpos=0; raw_buffer_writepos=0; raw_buffer=malloc_conceal(RAW_BUFFER_SIZE); line_buffer=malloc_conceal(LINE_BUFFER_SIZE); ascii_time_buffer=malloc_conceal(128); session_table=malloc(MAX_SESSIONS * SR_SIZE); sessions=0; line_buffer_len=0; while (line_buffer_len != 12 || memcmp("config|ready", line_buffer, 12) != 0) { if (line_in(line_buffer, &line_buffer_len, raw_buffer, &raw_buffer_readpos, &raw_buffer_writepos) != 0) { DEBUG_OUT ("I/O error on STDIN whilst waiting for config|ready\n"); return (1); } } write (STDOUT_FILENO,"register|filter|smtp-in|data-line\n",34); write (STDOUT_FILENO,"register|report|smtp-in|link-disconnect\n",40); write (STDOUT_FILENO,"register|ready\n",15); while (line_in(line_buffer, &line_buffer_len, raw_buffer, &raw_buffer_readpos, &raw_buffer_writepos)==0) { flag_skip_line=0; parse_pipeline(line_buffer, line_buffer_len, &pipeline); if (pipeline.len[4]==15 && memcmp(pipeline.value[4], "link-disconnect", 15)==0) { session_delete(session_table, &sessions, pipeline.value[5]); } if (pipeline.len[4]==9 && memcmp(pipeline.value[4], "data-line", 9)==0) { result=session_get_add(session_table, &sessions, pipeline.value[5], &data_area); if (result==2) { DEBUG_OUT ("Maximum number of concurrent sessions reached\n"); return (1); } /* * If this is a new session, then set the 'within headers' flag. */ if (result==1) { *data_area=1; } if (pipeline.totlen[7]==0) { *data_area=0; } if (*data_area==1 && pipeline.len[7] > 5 && memcmp(pipeline.value[7], "Date:", 5) == 0 ) { /* * Skip extraneous spaces and tabs. * These will be collapsed to a single space in the output if we modify the date header. */ for (i=5; i < pipeline.len[7] && (*(pipeline.value[7]+i)==' ' || *(pipeline.value[7]+i)==9); i++) { } if (i < pipeline.len[7] && strptime(pipeline.value[7]+i, "%a, %d %b %Y %H:%M:%S %z", &time_tm)!=NULL) { flag_skip_line=1; offset=(time_tm.tm_gmtoff); timestamp=timegm(&time_tm); timestamp-=offset; time_tm=*gmtime(×tamp); ascii_time_buffer_len=strftime(ascii_time_buffer, 128, "%a, %d %b %Y %H:%M:%S %z", &time_tm); write (STDOUT_FILENO, "filter-dataline|", 16); write (STDOUT_FILENO, pipeline.value[5], 34); write (STDOUT_FILENO, "Date: ", 6); write (STDOUT_FILENO, ascii_time_buffer, ascii_time_buffer_len); write (STDOUT_FILENO, "\n", 1); } } if (flag_skip_line==0) { write (STDOUT_FILENO, "filter-dataline|", 16); write (STDOUT_FILENO, pipeline.value[5], pipeline.totlen[5]); write (STDOUT_FILENO, "\n", 1); } if (pipeline.len[7] == 1 && *pipeline.value[7]=='.') { *data_area=1; } } } return (1); }
