Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On 07/18/2010 03:50 AM Brandon Lamb wrote: So I had another thought, if I have a user that has 500 megs of mail but its mostly large attachments, with the default mailfile size at 2 megs this means a 25 meg attachment is going to have to write to and read from 13 files every time? Interesting theory. ;-) But happily wrong. Messages with a size 2m will get their own storage file. Regards, Pascal -- The trapper recommends today: http://kopfkrebs.de/mitarbeiter/mitarbeiter_der_woche.html
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On 17.7.2010, at 0.07, Brandon Lamb wrote: sdbox is really simple. mdbox is more complex. A few people have used sdbox with v1.x, but it's more or less rewritten with v2.0. With v2.0 I'm not aware of any sdbox installations, but there is at least one somewhat large mdbox installation. They've had some problems, but I think those are mostly gone now. Any recommendation of one over the other? Depends on the filesystem and disks I guess. mdbox should reduce mailbox fragmentation because it uses larger files. But maybe with SSDs that doesn't matter and the extra metadata just slows down. Or maybe not. I don't have any real world benchmarks. I came across a 2009 mailing list entry where you showed some benchmark tests that had mdbox as the fastest. They were benchmarks of some unrealistic tests. Without any experience yet, my only possible hangup is being able to restore individual (lost, whatever other reason) mails for customers. This is more difficult with sdbox and mdbox than with maildir. I haven't really thought about this before. I guess with both you could get the one file (or the entire mailbox, either way) and then force a index rebuild and then copy/extract the mail in some way (e.g. doveadm fetch, imap client, etc). dsync is awesome and preserves everything. But test it anyway just in case. I'd anyway recommend setting pop3_save_uidl=yes and sometimes after that (when you think everyone's logged in at least once) switching to pop3_uidl_format=%g while still running maildir. With Maildir the new UIDLs are then the filenames and with dbox the UIDLs will be message GUIDs. Those are much more permanent than anything based on IMAP UIDs. Not a requirement, but still :) Cool, if anything I could just have a script login to every users account and logout and that would do the same thing right? That's all. Any main/major/big reasons to try to go with mdbox over sdbox? sdbox being really simple = less prone to typos and other bad things happening? There is much less potential for mailbox corruption with sdbox, since it's so much simpler. Oh, what about if i migrate to sdbox, then decide to go with mdbox, will I want to know for sure ahead of time? Granted I guess thats kind of an obvious yes, but any big gotchas to know about if I did? You can use dsync to easily switch between them.
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On 17.7.2010, at 4.46, Brandon Lamb wrote:
Ok I got all migrated over to dovecot2 configuration, and started to
dive into this converting business... one thing I immediately fell
into is that the wiki doesnt say anything about namespace issues. I
return maildir:~/maildir in my current setup via prefetch/sql, and I
got errors trying to do
dsync -u bran...@olypen.com mirror mdbox:~/mdbox
What errors did you get?
Do I need some special namespace configuation/setup?
No..
I think I may have got it, I added the following to my 10-mail.conf
# maildir
namespace {
type = private
separator = /
prefix =
# location = maildir:~/maildir
inbox = yes
hidden = no
}
# mdbox
namespace {
type = private
separator = /
prefix = #mdbox/
location = mdbox:~/mdbox
inbox = no
hidden = no
}
This shouldn't be necessary.
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On Sat, Jul 17, 2010 at 11:03 AM, Timo Sirainen t...@iki.fi wrote:
On 17.7.2010, at 4.46, Brandon Lamb wrote:
Ok I got all migrated over to dovecot2 configuration, and started to
dive into this converting business... one thing I immediately fell
into is that the wiki doesnt say anything about namespace issues. I
return maildir:~/maildir in my current setup via prefetch/sql, and I
got errors trying to do
dsync -u bran...@olypen.com mirror mdbox:~/mdbox
What errors did you get?
Do I need some special namespace configuation/setup?
No..
I think I may have got it, I added the following to my 10-mail.conf
# maildir
namespace {
type = private
separator = /
prefix =
# location = maildir:~/maildir
inbox = yes
hidden = no
}
# mdbox
namespace {
type = private
separator = /
prefix = #mdbox/
location = mdbox:~/mdbox
inbox = no
hidden = no
}
This shouldn't be necessary.
Without any namespaces it told me that I had to define the heirarchy
separator. Then I added the namespaces initially with both of them
specifying a location. But then when I tried to run dsync again, it
told me the namespace of the one to be mirrored should have no
location so I had to remove that. Then it worked.
I just wiped out and rebuilt my raid on my second box im working on so
once thats back and i restore my base maildir i'll run it again to get
the actual error output =P
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On 17.7.2010, at 20.39, Brandon Lamb wrote: Without any namespaces it told me that I had to define the heirarchy separator. Yes, that you need to set. But you need only one namespace.
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On Sat, Jul 17, 2010 at 1:09 PM, Timo Sirainen t...@iki.fi wrote: On 17.7.2010, at 20.39, Brandon Lamb wrote: Without any namespaces it told me that I had to define the heirarchy separator. Yes, that you need to set. But you need only one namespace. Oh! I'll try that instead soon thanks!
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On Sat, Jul 17, 2010 at 1:10 PM, Brandon Lamb brandonl...@gmail.com wrote: On Sat, Jul 17, 2010 at 1:09 PM, Timo Sirainen t...@iki.fi wrote: On 17.7.2010, at 20.39, Brandon Lamb wrote: Without any namespaces it told me that I had to define the heirarchy separator. Yes, that you need to set. But you need only one namespace. Oh! I'll try that instead soon thanks! So I had another thought, if I have a user that has 500 megs of mail but its mostly large attachments, with the default mailfile size at 2 megs this means a 25 meg attachment is going to have to write to and read from 13 files every time? And if so I guess this is just a tradeoff of the usual case of mails being 100k so fitting lots into the 2m files?
[Dovecot] Questions about dbox (sdbox?), mdbox
Hello, So ive spent the last 45 minutes or so searching the list for dbox stuff, but Im not seeing anything real definitive. My question is basically whether the dbox format is ready for production environment, is sdbox or mdbox more stable/ready than the other or does it matter? What are people's experience/review with running this format (and if you reply can you tell me which one you're using). Is there any docs/wiki page for migrating from maildir - dbox? Searching for dbox on the wiki only returns two results, and only one is actually for dbox. What I have (maildir format) is working, but who doesnt like shiny new toys? =P I have ~450gigs / 14k pop accounts I would have to migrate. Oh thats another question, I have seen hybrid throw around, can I take this to mean dbox/mdbox/both support both formats, maybe only new mail goes into the dbox stuff? And if so (or even if not) is there any way to actually convert from maildir to dbox? If anyone has any links that I cant seem to find by consulting the oracle (google) that would be fantastic =)
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On Fri, Jul 16, 2010 at 2:19 PM, Brandon Lamb brandonl...@gmail.com wrote: Hello, So ive spent the last 45 minutes or so searching the list for dbox stuff, but Im not seeing anything real definitive. My question is basically whether the dbox format is ready for production environment, is sdbox or mdbox more stable/ready than the other or does it matter? What are people's experience/review with running this format (and if you reply can you tell me which one you're using). Is there any docs/wiki page for migrating from maildir - dbox? Searching for dbox on the wiki only returns two results, and only one is actually for dbox. What I have (maildir format) is working, but who doesnt like shiny new toys? =P I have ~450gigs / 14k pop accounts I would have to migrate. Oh thats another question, I have seen hybrid throw around, can I take this to mean dbox/mdbox/both support both formats, maybe only new mail goes into the dbox stuff? And if so (or even if not) is there any way to actually convert from maildir to dbox? If anyone has any links that I cant seem to find by consulting the oracle (google) that would be fantastic =) Another question, if there is such a conversion tool, does it break UIDLs and require pop3 clients to redownload messages?
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On 16.7.2010, at 22.19, Brandon Lamb wrote: My question is basically whether the dbox format is ready for production environment, is sdbox or mdbox more stable/ready than the other or does it matter? What are people's experience/review with running this format (and if you reply can you tell me which one you're using). sdbox is really simple. mdbox is more complex. A few people have used sdbox with v1.x, but it's more or less rewritten with v2.0. With v2.0 I'm not aware of any sdbox installations, but there is at least one somewhat large mdbox installation. They've had some problems, but I think those are mostly gone now. Is there any docs/wiki page for migrating from maildir - dbox? dsync migrates between any two mailbox formats and does it as perfectly as it can. http://wiki2.dovecot.org/Tools/Dsync I have ~450gigs / 14k pop accounts I would have to migrate. Oh thats another question, I have seen hybrid throw around, can I take this to mean dbox/mdbox/both support both formats, maybe only new mail goes into the dbox stuff? And if so (or even if not) is there any way to actually convert from maildir to dbox? v1.x had some kind of a maildir/dbox hybrid.. until I realized it was just a bad idea. You can have per-user mail location defined in userdb though, so you can convert users a few at a time. Another question, if there is such a conversion tool, does it break UIDLs and require pop3 clients to redownload messages? dsync is awesome and preserves everything. But test it anyway just in case. I'd anyway recommend setting pop3_save_uidl=yes and sometimes after that (when you think everyone's logged in at least once) switching to pop3_uidl_format=%g while still running maildir. With Maildir the new UIDLs are then the filenames and with dbox the UIDLs will be message GUIDs. Those are much more permanent than anything based on IMAP UIDs. Not a requirement, but still :)
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On Fri, Jul 16, 2010 at 2:31 PM, Timo Sirainen t...@iki.fi wrote: On 16.7.2010, at 22.19, Brandon Lamb wrote: My question is basically whether the dbox format is ready for production environment, is sdbox or mdbox more stable/ready than the other or does it matter? What are people's experience/review with running this format (and if you reply can you tell me which one you're using). sdbox is really simple. mdbox is more complex. A few people have used sdbox with v1.x, but it's more or less rewritten with v2.0. With v2.0 I'm not aware of any sdbox installations, but there is at least one somewhat large mdbox installation. They've had some problems, but I think those are mostly gone now. Any recommendation of one over the other? I came across a 2009 mailing list entry where you showed some benchmark tests that had mdbox as the fastest. Without any experience yet, my only possible hangup is being able to restore individual (lost, whatever other reason) mails for customers. Doesnt really happen but being able to restore individual versus their whole mailbox is nice, but it happens so rarely it may be a non issue. Is there any docs/wiki page for migrating from maildir - dbox? dsync migrates between any two mailbox formats and does it as perfectly as it can. http://wiki2.dovecot.org/Tools/Dsync Awesome, I'll go do some reading and mess with that I have ~450gigs / 14k pop accounts I would have to migrate. Oh thats another question, I have seen hybrid throw around, can I take this to mean dbox/mdbox/both support both formats, maybe only new mail goes into the dbox stuff? And if so (or even if not) is there any way to actually convert from maildir to dbox? v1.x had some kind of a maildir/dbox hybrid.. until I realized it was just a bad idea. You can have per-user mail location defined in userdb though, so you can convert users a few at a time. Another question, if there is such a conversion tool, does it break UIDLs and require pop3 clients to redownload messages? dsync is awesome and preserves everything. But test it anyway just in case. I'd anyway recommend setting pop3_save_uidl=yes and sometimes after that (when you think everyone's logged in at least once) switching to pop3_uidl_format=%g while still running maildir. With Maildir the new UIDLs are then the filenames and with dbox the UIDLs will be message GUIDs. Those are much more permanent than anything based on IMAP UIDs. Not a requirement, but still :) Cool, if anything I could just have a script login to every users account and logout and that would do the same thing right? Or does it have to perform some kind of list or other action? Any main/major/big reasons to try to go with mdbox over sdbox? sdbox being really simple = less prone to typos and other bad things happening? Oh, what about if i migrate to sdbox, then decide to go with mdbox, will I want to know for sure ahead of time? Granted I guess thats kind of an obvious yes, but any big gotchas to know about if I did? Oh yea and dovecot kicks ass btw =P
Re: [Dovecot] Questions about dbox (sdbox?), mdbox
On Fri, Jul 16, 2010 at 8:34 PM, Brandon Lamb brandonl...@gmail.com wrote:
On Fri, Jul 16, 2010 at 4:07 PM, Brandon Lamb brandonl...@gmail.com wrote:
On Fri, Jul 16, 2010 at 2:31 PM, Timo Sirainen t...@iki.fi wrote:
On 16.7.2010, at 22.19, Brandon Lamb wrote:
My question is basically whether the dbox format is ready for
production environment, is sdbox or mdbox more stable/ready than the
other or does it matter? What are people's experience/review with
running this format (and if you reply can you tell me which one you're
using).
sdbox is really simple. mdbox is more complex. A few people have used sdbox
with v1.x, but it's more or less rewritten with v2.0. With v2.0 I'm not
aware of any sdbox installations, but there is at least one somewhat large
mdbox installation. They've had some problems, but I think those are mostly
gone now.
Any recommendation of one over the other? I came across a 2009 mailing
list entry where you showed some benchmark tests that had mdbox as the
fastest. Without any experience yet, my only possible hangup is being
able to restore individual (lost, whatever other reason) mails for
customers. Doesnt really happen but being able to restore individual
versus their whole mailbox is nice, but it happens so rarely it may be
a non issue.
Is there any docs/wiki page for migrating from maildir - dbox?
dsync migrates between any two mailbox formats and does it as perfectly as
it can. http://wiki2.dovecot.org/Tools/Dsync
Awesome, I'll go do some reading and mess with that
I have ~450gigs / 14k pop accounts I would have to migrate. Oh thats
another question, I have seen hybrid throw around, can I take this
to mean dbox/mdbox/both support both formats, maybe only new mail goes
into the dbox stuff? And if so (or even if not) is there any way to
actually convert from maildir to dbox?
v1.x had some kind of a maildir/dbox hybrid.. until I realized it was just
a bad idea. You can have per-user mail location defined in userdb though,
so you can convert users a few at a time.
Another question, if there is such a conversion tool, does it break
UIDLs and require pop3 clients to redownload messages?
dsync is awesome and preserves everything. But test it anyway just in case.
I'd anyway recommend setting pop3_save_uidl=yes and sometimes after that
(when you think everyone's logged in at least once) switching to
pop3_uidl_format=%g while still running maildir. With Maildir the new UIDLs
are then the filenames and with dbox the UIDLs will be message GUIDs. Those
are much more permanent than anything based on IMAP UIDs. Not a
requirement, but still :)
Cool, if anything I could just have a script login to every users
account and logout and that would do the same thing right? Or does it
have to perform some kind of list or other action?
Any main/major/big reasons to try to go with mdbox over sdbox? sdbox
being really simple = less prone to typos and other bad things
happening?
Oh, what about if i migrate to sdbox, then decide to go with mdbox,
will I want to know for sure ahead of time? Granted I guess thats kind
of an obvious yes, but any big gotchas to know about if I did?
Oh yea and dovecot kicks ass btw =P
Ok I got all migrated over to dovecot2 configuration, and started to
dive into this converting business... one thing I immediately fell
into is that the wiki doesnt say anything about namespace issues. I
return maildir:~/maildir in my current setup via prefetch/sql, and I
got errors trying to do
dsync -u bran...@olypen.com mirror mdbox:~/mdbox
Do I need some special namespace configuation/setup?
I think I may have got it, I added the following to my 10-mail.conf
# maildir
namespace {
type = private
separator = /
prefix =
# location = maildir:~/maildir
inbox = yes
hidden = no
}
# mdbox
namespace {
type = private
separator = /
prefix = #mdbox/
location = mdbox:~/mdbox
inbox = no
hidden = no
}
