Re: silly quesiton [ot]
On Mon, Jan 31, 2022 at 12:04:57PM +0100, Wojciech Puchar wrote: >>> mbox is multiple emails in single file, maildir is single email in >>> single file >> >> Exactly my point! > > which is good (mbox) in mail archiving. And not much else. Exactly. -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
Re: silly quesiton [ot]
files". mbox is multiple emails in single file, maildir is single email in single file Exactly my point! which is good (mbox) in mail archiving. And not much else.
Re: silly quesiton [ot]
On Mon, Jan 31, 2022 at 11:26:26AM +0100, Benny Pedersen wrote: > On 2022-01-31 07:23, Sam Kuper wrote: >>DJB developed Maildir to gain performance and reliability improvements >>over mbox files. Unlike Maildirs, mbox files *are* "large flat >>files". > > mbox is multiple emails in single file, maildir is single email in > single file Exactly my point! -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
RE: silly quesiton [ot]
> > Commercial Dovecot has had the ability to store mails & indexes in > > Object Storage for years now, we are not "working on it" anymore. > > so no one is using dovecot-pro ? (dead sources) :D
Re: silly quesiton [ot]
On 2022-01-31 11:29, Marc wrote: This 'performance statement' is not even correct under some conditions. I have tested mbox against mdbox, and mbox performed better in those tests. so you are now ready to make a postfix patch ? its called progress on make ...
Re: silly quesiton [ot]
> On 31/01/2022 12:29 Benny Pedersen wrote: > > > On 2022-01-31 10:45, Aki Tuomi wrote: > > > Commercial Dovecot has had the ability to store mails & indexes in > > Object Storage for years now, we are not "working on it" anymore. > > so no one is using dovecot-pro ? (dead sources) Ha ha. What I mean is that it is already being used in production, not something we are trying to get working. Of course we are still developing it. Aki
Re: silly quesiton [ot]
On 2022-01-31 10:45, Aki Tuomi wrote: Commercial Dovecot has had the ability to store mails & indexes in Object Storage for years now, we are not "working on it" anymore. so no one is using dovecot-pro ? (dead sources)
RE: silly quesiton [ot]
> > Removing or deleting a single message from near the beginning of a > > large flat file takes an inordinate amount of time because the > > remainder of the flat file has to be rewritten all the way from the > > point of the deleted message to the end of the file and then > > truncated. > > maybe why postfix only support maildir++ > > all other formats is properitary formats This 'performance statement' is not even correct under some conditions. I have tested mbox against mdbox, and mbox performed better in those tests.
Re: silly quesiton [ot]
On 2022-01-31 07:23, Sam Kuper wrote: DJB developed Maildir to gain performance and reliability improvements over mbox files. Unlike Maildirs, mbox files *are* "large flat files". mbox is multiple emails in single file, maildir is single email in single file
Re: silly quesiton [ot]
On 2022-01-31 05:49, justina colmena ~biz wrote: Just ideas. Removing or deleting a single message from near the beginning of a large flat file takes an inordinate amount of time because the remainder of the flat file has to be rewritten all the way from the point of the deleted message to the end of the file and then truncated. maybe why postfix only support maildir++ all other formats is properitary formats
RE: silly quesiton [ot]
> > I see. People make money outsourcing, consulting, and hooking up companies > with the best solutions for email, office collaboration, CRM, etc., etc., > which is great, but I didn't quite realize that look like a paid offering > on the table and this isn't the right list to discuss potential free > market competition... What are you on about?
RE: silly quesiton [ot]
I see. People make money outsourcing, consulting, and hooking up companies with the best solutions for email, office collaboration, CRM, etc., etc., which is great, but I didn't quite realize that look like a paid offering on the table and this isn't the right list to discuss potential free market competition... On January 31, 2022 12:45:48 AM AKST, Aki Tuomi wrote: > >> On 31/01/2022 10:36 Marc wrote: >> >> >> > >> > Just ideas. >> >> Maybe an idea to participate on a Microsoft forum? They like to use db's for >> email, and they are removing everything what is nice in order to push people >> into their cloud. So lots to change for the better there. >> >> It's so crappy that I recently wrote Bill Gates that he should not whine so >> much about the environment, because if he used only half of his profits to >> optimize code/designs in ms products, this would result in a significant >> reduction in energy use. Think of what global effect that has. >> >> FYI T-mobile (and the commercial version of dovecot?) is working on storing >> emails in object storage, that is the future. > >Commercial Dovecot has had the ability to store mails & indexes in Object >Storage for years now, we are not "working on it" anymore. > >Aki -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
RE: silly quesiton [ot]
> On 31/01/2022 10:36 Marc wrote: > > > > > > Just ideas. > > Maybe an idea to participate on a Microsoft forum? They like to use db's for > email, and they are removing everything what is nice in order to push people > into their cloud. So lots to change for the better there. > > It's so crappy that I recently wrote Bill Gates that he should not whine so > much about the environment, because if he used only half of his profits to > optimize code/designs in ms products, this would result in a significant > reduction in energy use. Think of what global effect that has. > > FYI T-mobile (and the commercial version of dovecot?) is working on storing > emails in object storage, that is the future. Commercial Dovecot has had the ability to store mails & indexes in Object Storage for years now, we are not "working on it" anymore. Aki
RE: silly quesiton [ot]
> > Just ideas. Maybe an idea to participate on a Microsoft forum? They like to use db's for email, and they are removing everything what is nice in order to push people into their cloud. So lots to change for the better there. It's so crappy that I recently wrote Bill Gates that he should not whine so much about the environment, because if he used only half of his profits to optimize code/designs in ms products, this would result in a significant reduction in energy use. Think of what global effect that has. FYI T-mobile (and the commercial version of dovecot?) is working on storing emails in object storage, that is the future.
Re: silly quesiton [ot]
On Mon, Jan 31, 2022 at 06:23:28AM +, Sam Kuper wrote:
> On Sun, Jan 30, 2022 at 07:49:56PM -0900, justina colmena ~biz wrote:
> > On January 30, 2022 6:30:44 PM AKST, Sam Kuper wrote:
> >> On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote:
> >>> On January 30, 2022 5:46:53 PM AKST, dove...@ptld.com wrote:
> Storing mail in a db... at the end of the day isn't it still just a
> file (.db file) on the drive?
>
> Aren't you just adding bloat and complexity vs just storing the
> mail directly (maildir format) to a file on the drive? [...]
> >>>
> >>> You'll get better indexing and fast full text search by storing your
> >>> emails in a database rather than a flat file, hopefully after
> >>> decoding any attachments. Especially for spam scoring, analysis, and
> >>> classification. Much better performance deleting or moving specific
> >>> messages, too.
> >>
> >> Do you have evidence to back up these claims, specifically re: mail
> >> servers?
> >>
> >> Like-for-like benchmarks, for instance?
> >
> > Just ideas.
>
> OK, no then.
>
>
> > Removing or deleting a single message from near the beginning of a
> > large flat file takes an inordinate amount of time because the
> > remainder of the flat file has to be rewritten all the way from the
> > point of the deleted message to the end of the file and then
> > truncated.
>
> You might want to look up what Maildir is before making bold but
> apparently unfounded claims about it.
>
> Maildir is not a "large flat file". It is a set of conventions that
> amount to a database specification, in the traditional sense of the word
> "database": a system for storing data. (Not a relational database.)
>
Many people haven't ever had to deal with the old "database" style of
files instead of tables and columns.
Maildir does show it's age with the little complexities it has.
> DJB developed Maildir to gain performance and reliability improvements
> over mbox files. Unlike Maildirs, mbox files *are* "large flat files".
Corrupt your mbox file and bad things happen!
I also like being able to throw in some older backed up email when I
find I need a few more to fill out that important thread from 3 years
ago with Maildir.
Maildir does not have the relational database problem of needing to keep
up with updates to the database software.
And nothing works very well when you suddenly discover that the company
you are renting servers from decides to close up and turn everything
off. While you are in another country with internet cafes only and don't
even have a laptop with you! Happened to me once. 8-{
--
Chris Bennett
Re: silly quesiton [ot]
On Sun, Jan 30, 2022 at 07:49:56PM -0900, justina colmena ~biz wrote: > On January 30, 2022 6:30:44 PM AKST, Sam Kuper wrote: >> On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote: >>> On January 30, 2022 5:46:53 PM AKST, dove...@ptld.com wrote: Storing mail in a db... at the end of the day isn't it still just a file (.db file) on the drive? Aren't you just adding bloat and complexity vs just storing the mail directly (maildir format) to a file on the drive? [...] >>> >>> You'll get better indexing and fast full text search by storing your >>> emails in a database rather than a flat file, hopefully after >>> decoding any attachments. Especially for spam scoring, analysis, and >>> classification. Much better performance deleting or moving specific >>> messages, too. >> >> Do you have evidence to back up these claims, specifically re: mail >> servers? >> >> Like-for-like benchmarks, for instance? > > Just ideas. OK, no then. > Removing or deleting a single message from near the beginning of a > large flat file takes an inordinate amount of time because the > remainder of the flat file has to be rewritten all the way from the > point of the deleted message to the end of the file and then > truncated. You might want to look up what Maildir is before making bold but apparently unfounded claims about it. Maildir is not a "large flat file". It is a set of conventions that amount to a database specification, in the traditional sense of the word "database": a system for storing data. (Not a relational database.) DJB developed Maildir to gain performance and reliability improvements over mbox files. Unlike Maildirs, mbox files *are* "large flat files". Best wishes, Sam -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
Re: silly quesiton [ot]
Just ideas. Removing or deleting a single message from near the beginning of a large flat file takes an inordinate amount of time because the remainder of the flat file has to be rewritten all the way from the point of the deleted message to the end of the file and then truncated. On January 30, 2022 6:30:44 PM AKST, Sam Kuper wrote: >On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote: >> On January 30, 2022 5:46:53 PM AKST, dove...@ptld.com wrote: >>> Storing mail in a db... at the end of the day isn't it still just a >>> file (.db file) on the drive? >>> >>> Aren't you just adding bloat and complexity vs just storing the mail >>> directly (maildir format) to a file on the drive? [...] >> >> You'll get better indexing and fast full text search by storing your >> emails in a database rather than a flat file, hopefully after decoding >> any attachments. Especially for spam scoring, analysis, and >> classification. Much better performance deleting or moving specific >> messages, too. > >Do you have evidence to back up these claims, specifically re: mail >servers? > >Like-for-like benchmarks, for instance? > >Thanks, > >Sam > >-- >A: When it messes up the order in which people normally read text. >Q: When is top-posting a bad thing? > >() ASCII ribbon campaign. Please avoid HTML emails & proprietary >/\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: silly quesiton [ot]
On Sun, Jan 30, 2022 at 09:46:53PM -0500, dove...@ptld.com wrote: > Storing mail in a db... at the end of the day isn't it still just a file (.db > file) on the drive? > Aren't you just adding bloat and complexity vs just storing the mail directly > (maildir format) to a file on the drive? > > What do you think you are saving? Security? > If someone can read files on your server, they can equally read a maildir or > a .db file. > K.I.S.S. I gain modularity for a system. The database is the foundation. I am working with: 1. Dovecot 2. Neomutt 3. OpenSMTPD Now, if I decide to drop or addon some new program, I can just adjust and/or add some new tables. Write a new stored procedure. Drop in a new Perl module or subroutine. 1. Dovecot 2. Neomutt 3. OpenSMTPD 4. Xyz 5. Abc 6. SuperDuperMail-ThingyPlus So what I am working for is a system that is united. Add a new user and email, CLI program, bang. All done. Change a password with a web interface. Click. All done. I'm in no rush. This is a fun side project. I have already done this type of work successfully for other kinds of projects, so it's different, but not really outside of my past experience. Secure today is wide open tomorrow. File, memory, etc. all get broken eventually. I'm much more worried about my own mistakes than that of others. :-* -- Chris Bennett
Re: silly quesiton [ot]
On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote: > On January 30, 2022 5:46:53 PM AKST, dove...@ptld.com wrote: >> Storing mail in a db... at the end of the day isn't it still just a >> file (.db file) on the drive? >> >> Aren't you just adding bloat and complexity vs just storing the mail >> directly (maildir format) to a file on the drive? [...] > > You'll get better indexing and fast full text search by storing your > emails in a database rather than a flat file, hopefully after decoding > any attachments. Especially for spam scoring, analysis, and > classification. Much better performance deleting or moving specific > messages, too. Do you have evidence to back up these claims, specifically re: mail servers? Like-for-like benchmarks, for instance? Thanks, Sam -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
Re: silly quesiton [ot]
You'll get better indexing and fast full text search by storing your emails in a database rather than a flat file, hopefully after decoding any attachments. Especially for spam scoring, analysis, and classification. Much better performance deleting or moving specific messages, too. On January 30, 2022 5:46:53 PM AKST, dove...@ptld.com wrote: >Storing mail in a db... at the end of the day isn't it still just a file (.db >file) on the drive? >Aren't you just adding bloat and complexity vs just storing the mail directly >(maildir format) to a file on the drive? > >What do you think you are saving? Security? >If someone can read files on your server, they can equally read a maildir or a >.db file. >K.I.S.S. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: silly quesiton [ot]
Storing mail in a db... at the end of the day isn't it still just a file (.db file) on the drive? Aren't you just adding bloat and complexity vs just storing the mail directly (maildir format) to a file on the drive? What do you think you are saving? Security? If someone can read files on your server, they can equally read a maildir or a .db file. K.I.S.S.
Re: silly quesiton [ot]
On 2022-01-31 02:30, Sean Kamath wrote: On Jan 30, 2022, at 10:55, Chris Bennett wrote: On Tue, Jan 25, 2022 at 03:50:12AM -0900, justina colmena ~biz wrote: Maybe a future programming project idea: I want a system that will store all mail messages and user account info in, say, a postgresql transactional database, a little more manageable and reliable than ad hoc databasing with those flat files all over the place cluttering up the system. I am in progress moving towards something like that. As of right now, perl, dovecot for IMAP, neomutt and OpenSMTPD. Right now, .neomuttrc files *only* exist during the usage of neomutt. They have random names, cannot be written to and are immediately erased after neomutt starts (not quits). That is a very small window of threat. I would very much like to put all of the messages into PostgreSQL also instead of file folders under the user vmail. This is just a side project. As I have been advised, there is no need to even write a configuration file at all, but there are some issues with dbh that I need to solve with a different database module. If someone can read files that never exist, well... At some point you have to at least consider trusting something. That or just turn it all off and get another career. -- Chris Bennett At some point you gotta ask yourself why you’re trusting your database more than your OS. And why you don’t trust the OS to handle files in a trusted way, but do for memory. dbmail exists, runs fine on sqlite3 :=) but that joke, why try ? how huge would that sqlite3 file be ?, i say no to one sqlite3 file, but yes if each mail user have there own sqlite3 tree with seperate sqlite3 file pr folder and user if more huge setup is meeded, then postgresql with replication, but this is not needed with dovecot, its more solid and with performance with imap protocol, and load balanced i would not wish for disaster with sqlite3, but it could be done, also sqlite cluster exists dream on, its monday where noting works :=)
Re: silly quesiton [ot]
> On Jan 30, 2022, at 10:55, Chris Bennett > wrote: > > On Tue, Jan 25, 2022 at 03:50:12AM -0900, justina colmena ~biz wrote: >> Maybe a future programming project idea: I want a system that will store all >> mail messages and user account info in, say, a postgresql transactional >> database, a little more manageable and reliable than ad hoc databasing with >> those flat files all over the place cluttering up the system. >> > > I am in progress moving towards something like that. > As of right now, perl, dovecot for IMAP, neomutt and OpenSMTPD. > > Right now, .neomuttrc files *only* exist during the usage of neomutt. > They have random names, cannot be written to and are immediately erased > after neomutt starts (not quits). That is a very small window of threat. > > I would very much like to put all of the messages into PostgreSQL also > instead of file folders under the user vmail. > > This is just a side project. > As I have been advised, there is no need to even write a configuration > file at all, but there are some issues with dbh that I need to solve > with a different database module. > > If someone can read files that never exist, well... > At some point you have to at least consider trusting something. > That or just turn it all off and get another career. > > -- > Chris Bennett At some point you gotta ask yourself why you’re trusting your database more than your OS. And why you don’t trust the OS to handle files in a trusted way, but do for memory. Sean
