Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-27 Thread Baptiste Malguy
Hello,

Le 27/02/10 09:45, Stan Hoeppner a écrit :
 Brian Hayden put forth on 2/25/2010 3:10 PM:
 
 To add insult to injury, mailbox migration from Exch 5.5 IMAP to Dovecot
 IMAP was done using TB as the conduit, simply copying folders from one
 account to another.  Took forever, but it was the only reliable way I
 could think of at the time to do it.  It actually went very smoothly.
 

imapsync is a _very_ useful tool to accomplish this. I don't know
Exchange, but if it allows you to have a kind of admin account having
access to all accounts or at least one at a time (and automatically
select which one), you can even automate the migration. On Dovecot, I
set a non-real user account, with the mailbox path being the same as the
user under migration.

A few scripting and you're done.

I've used it to move accounts from Gmail to Dovecot and from Cyrus IMAP
to Dovecot. Only pay attention with Gmail special IMAP folders: All
Messages and so forth.

Regards,

-- 
Baptiste MALGUY - http://www.malguy.net
PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF  9267 0F65 6C1C C473 6EC2



signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-27 Thread Brian Hayden

On Feb 27 2010, Stan Hoeppner wrote:


Brian Hayden put forth on 2/25/2010 3:10 PM:


particularly if the subscriptions file is in an unreliable state (as it
sounds like yours is--as most of them almost always are).


I guess I'm lucky so far Brian. I've had no subscription problems with TB 
+

Squirrelmail + Exchange 5.5 IMAP, or with TB + Roundcube + Dovecot IMAP.

To add insult to injury, mailbox migration from Exch 5.5 IMAP to Dovecot
IMAP was done using TB as the conduit, simply copying folders from one
account to another.  Took forever, but it was the only reliable way I
could think of at the time to do it.  It actually went very smoothly.


How large is the user base? Are the clients managed? And are you sure you 
haven't had any problems... only a very small number of users generally 
complain to relevant parties about a given issue, they grouse about it to 
their friends and colleagues.


It's definitely possible to make it work with tight control. In a large, 
unmanaged environment, it really isn't. Unless you want to posit that the 
80k users I dealt with for ten years on this were exceptionally stupid. ;)


-Brian


Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-27 Thread Stan Hoeppner
Brian Hayden put forth on 2/27/2010 8:28 AM:
 On Feb 27 2010, Stan Hoeppner wrote:

 To add insult to injury, mailbox migration from Exch 5.5 IMAP to Dovecot
 IMAP was done using TB as the conduit, simply copying folders from one
 account to another.  Took forever, but it was the only reliable way I
 could think of at the time to do it.  It actually went very smoothly.
 
 How large is the user base? Are the clients managed? And are you sure
 you haven't had any problems... only a very small number of users
 generally complain to relevant parties about a given issue, they grouse
 about it to their friends and colleagues.

Very small.  Manual client management.  No problems after the migration.  If
I complain, I'm sure I'll hear myself. ;)  The migration I mentioned was my
vanity server.  If I were supporting an IMAP infrastructure at a $dayjob
environment I would have never gone this route.  And I probably wouldn't
have been upgrading directly from Exchange 5.5 (which was EOL'd in like 1999
or 2000) to Dovecot 1.0.15.  ;)  I can't imagine an org that would have held
onto Exch 5.5 into 2009, given that service packs and hotfixes ceased around
a decade ago.

 It's definitely possible to make it work with tight control. In a large,
 unmanaged environment, it really isn't. Unless you want to posit that
 the 80k users I dealt with for ten years on this were exceptionally
 stupid. ;)

That would depend on who those 80K users were.  If they were federal
employees, worse yet, HUD or USPS employees, I'd guess the stupidity meter
would run pretty high. ;)

-- 
Stan


Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-27 Thread Jerry
On Sat, 27 Feb 2010 09:11:52 -0600
Stan Hoeppner s...@hardwarefreak.com articulated:

 Brian Hayden put forth on 2/27/2010 8:28 AM:
  On Feb 27 2010, Stan Hoeppner wrote:
 
  To add insult to injury, mailbox migration from Exch 5.5 IMAP to
  Dovecot IMAP was done using TB as the conduit, simply copying
  folders from one account to another.  Took forever, but it was
  the only reliable way I could think of at the time to do it.  It
  actually went very smoothly.
  
  How large is the user base? Are the clients managed? And are you
  sure you haven't had any problems... only a very small number of
  users generally complain to relevant parties about a given issue,
  they grouse about it to their friends and colleagues.
 
 Very small.  Manual client management.  No problems after the
 migration.  If I complain, I'm sure I'll hear myself. ;)  The
 migration I mentioned was my vanity server.  If I were supporting an
 IMAP infrastructure at a $dayjob environment I would have never gone
 this route.  And I probably wouldn't have been upgrading directly
 from Exchange 5.5 (which was EOL'd in like 1999 or 2000) to Dovecot
 1.0.15.  ;)  I can't imagine an org that would have held onto Exch
 5.5 into 2009, given that service packs and hotfixes ceased around a
 decade ago.

January 31, 2006
http://www.microsoft.com/exchange/2007/support/lifecycle/changes.mspx

The newest version is MS Exchange 2010

  It's definitely possible to make it work with tight control. In a
  large, unmanaged environment, it really isn't. Unless you want to
  posit that the 80k users I dealt with for ten years on this were
  exceptionally stupid. ;)
 
 That would depend on who those 80K users were.  If they were federal
 employees, worse yet, HUD or USPS employees, I'd guess the stupidity
 meter would run pretty high. ;)

-- 
Jerry
ges...@yahoo.com

|===
|===
|===
|===
|

For three days after death hair and fingernails
continue to grow, but phone calls taper off.


Johnny Carson



Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Carlos Williams
On Thu, Feb 25, 2010 at 2:19 PM, Rick Romero r...@havokmon.com wrote:
 The entire structure doesn't look correct for Maildir.
 The folders you listed, puts you already in the INBOX, so your MUA should be
 seeing:
 INBOX\
      Drafts
      INBOX\
            ClamAV
            Dell.Quotes
            Dell.Warranty
            etc.etc.etc..
      Sent
      Spam
      Trash

 If you are not seeing the folders in that order, then I think your
 dovecot.conf isn't quite right.  Please post the output of dovecot -n

No I don't appear to be seeing this and I think the fact that I have
used different mail clients (Thunderbird, Evolution, etc etc etc) and
think that their default folder structure may have made a mess on my
IMAP Maildir structure over the years.

Below is my dovecot -n:

[r...@mail Maildir]# dovecot -n
# 1.0.7: /etc/dovecot.conf
protocols: imap
ssl_cert_file: /srv/ssl/ghost.crt
ssl_key_file: /srv/ssl/ghost.key
login_dir: /var/run/dovecot/login
login_executable: /usr/libexec/dovecot/imap-login
mail_location: maildir:~/Maildir
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd
  socket:
type: listen
client:
  path: /var/spool/postfix/private/auth
  mode: 432
  user: postfix
  group: postfix

---

Beyond that I don't know why it's messed up and more importantly, how
I can fix it. Any suggestions?


Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Rick Romero

Quoting Carlos Williams carlosw...@gmail.com:


On Thu, Feb 25, 2010 at 2:19 PM, Rick Romero r...@havokmon.com wrote:

The entire structure doesn't look correct for Maildir.
The folders you listed, puts you already in the INBOX, so your MUA should be
seeing:
INBOX\
     Drafts
     INBOX\
           ClamAV
           Dell.Quotes
           Dell.Warranty
           etc.etc.etc..
     Sent
     Spam
     Trash

If you are not seeing the folders in that order, then I think your
dovecot.conf isn't quite right.  Please post the output of dovecot -n


No I don't appear to be seeing this and I think the fact that I have
used different mail clients (Thunderbird, Evolution, etc etc etc) and
think that their default folder structure may have made a mess on my
IMAP Maildir structure over the years.

Below is my dovecot -n:

[r...@mail Maildir]# dovecot -n
# 1.0.7: /etc/dovecot.conf
protocols: imap
ssl_cert_file: /srv/ssl/ghost.crt
ssl_key_file: /srv/ssl/ghost.key
login_dir: /var/run/dovecot/login
login_executable: /usr/libexec/dovecot/imap-login
mail_location: maildir:~/Maildir
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd
  socket:
type: listen
client:
  path: /var/spool/postfix/private/auth
  mode: 432
  user: postfix
  group: postfix

---

Beyond that I don't know why it's messed up and more importantly, how
I can fix it. Any suggestions?


Try adding a namespace.

namespace private {
  separator = .
  prefix = INBOX.
  inbox = yes
}





Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Rick Romero


Quoting Carlos Williams carlosw...@gmail.com:


On Thu, Feb 25, 2010 at 2:07 PM, Frank Elsner
fr...@moltke28.b.shuttle.de wrote:

It might be useful to tell Thunderbird not only to show subscribed folders.
Edit - Account settings - Server settings - Advanced, first check box.

There are some MUAs which create folders but do not add subscriptions
for the folders. The sylpheed I currently use is such a beast :-)


In Thunderbird that option is already checked and it wouldn't explain
why webmail (RoundCube) shows the exact same thing. I can't delete the
folder from Thunderbird and I don't see it when I login to the mail
server via shell. I am thinking there is something wrong with the
folder listing or something like that.



The entire structure doesn't look correct for Maildir.
The folders you listed, puts you already in the INBOX, so your MUA  
should be seeing:

INBOX\
  Drafts
  INBOX\
ClamAV
Dell.Quotes
Dell.Warranty
etc.etc.etc..
  Sent
  Spam
  Trash

If you are not seeing the folders in that order, then I think your  
dovecot.conf isn't quite right.  Please post the output of dovecot -n


Rick



Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Carlos Williams
On Thu, Feb 25, 2010 at 2:41 PM, Rick Romero r...@havokmon.com wrote:
 Try adding a namespace.

 namespace private {
  separator = .
  prefix = INBOX.
  inbox = yes
 }

I made that change and reloaded Dovecot and all my users on the mail
server lost the folders...


Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Rick Romero

Quoting Carlos Williams carlosw...@gmail.com:


On Thu, Feb 25, 2010 at 2:41 PM, Rick Romero r...@havokmon.com wrote:

Try adding a namespace.

namespace private {
 separator = .
 prefix = INBOX.
 inbox = yes
}


I made that change and reloaded Dovecot and all my users on the mail
server lost the folders...


Yikes!  I thought this was a private install. :(  The namespace would  
be a pretty radical change.  It tells dovecot how to present folders.


I'm not a namespace guru - but I'm fairly sure that's the issue.
http://wiki.dovecot.org/Namespaces

I would suggest creating an alternate config file, using different  
ports and system directories, to test a namespace change.


Rick




Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Rick Romero

Quoting Carlos Williams carlosw...@gmail.com:

On Thu, Feb 25, 2010 at 2:55 PM, Carlos Williams  
carlosw...@gmail.com wrote:

On Thu, Feb 25, 2010 at 2:41 PM, Rick Romero r...@havokmon.com wrote:

Try adding a namespace.

namespace private {
 separator = .
 prefix = INBOX.
 inbox = yes
}


I made that change and reloaded Dovecot and all my users on the mail
server lost the folders...


Sorry - I was wrong. This didn't remove anything from my server. I
guess it helps when you issue the ls -la command. I was just being
paranoid. My question now is how do I clean up my Maildir/ to how it
should be?


No problem - it's the display that was changed due to the namespace config.


Right now after the change and successfully restarting IMAP, I have
the following directory when I login to Linux. I haven't opened up any
mail clients yet...


Do your users have their mailboxes back?  That's the first thing.  If  
you're the only one with a funky layout, then forget yours, and get  
theirs back to normal. :)
So instead of changing the namespace, I suppose it would be better to  
conform your mailbox to what the server expects.  So I guess I'd do:


mv .INBOX.CDW .CDW
mv .INBOX.Dell .Dell
mv .INBOX.Dell.Certification .Dell.Certification
etc etc etc

Your .INBOX/ directory should contain cur/ new/ and tmp/ directories,  
where you can move/copy the individual emails into your  
~home/Maildir/cur/   new/tmp/


Maybe you had a MUA with a .INBOX prefix set, and when you created  
your folders, it stuck that in there... ?


Rick




Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Rick Romero

Quoting Carlos Williams carlosw...@gmail.com:


On Thu, Feb 25, 2010 at 3:08 PM, Rick Romero r...@havokmon.com wrote:

Yikes!  I thought this was a private install. :(  The namespace would be a
pretty radical change.  It tells dovecot how to present folders.

I'm not a namespace guru - but I'm fairly sure that's the issue.
http://wiki.dovecot.org/Namespaces

I would suggest creating an alternate config file, using different ports and
system directories, to test a namespace change.


It's OK. It appears to not have impacted any existing data. Perhaps
just changes how the new folders are managed.


Yep.


I did see a file in my
Maildir called 'subscriptions' and when I opened it with a text
editor, it had the old invalid IMAP directory structure...should I
delete or rename this file on my mailbox and then restart the client
to force it to re-build this subscriptions file?


The subscriptions file is only used by the MUAs, and you can set them  
to ignore it.  I would just tell the MUAs to ignore it.  You can  
safely delete it - except if you have an MUA that is using it then the  
folders will disappear...


I suppose if you have a PDA and a huge folder structure you might have  
the PDA use subscriptions and trim down the folder list...


I would 'start fresh' and remove it.




Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Carlos Williams
On Thu, Feb 25, 2010 at 3:21 PM, Rick Romero r...@havokmon.com wrote:
 The subscriptions file is only used by the MUAs, and you can set them to
 ignore it.  I would just tell the MUAs to ignore it.  You can safely delete
 it - except if you have an MUA that is using it then the folders will
 disappear...

 I suppose if you have a PDA and a huge folder structure you might have the
 PDA use subscriptions and trim down the folder list...

 I would 'start fresh' and remove it.

When I remove this file, my mail clients (webmail  Thunderbird) no
longer see any of my sub folders. How can I remove this file if it
holds obsolete data and still be able to see my Inbox sub folders?

I reverted back to my original dovecot.conf file as when I posted my
original. No changes basically have been made. When I add your
suggestion for namespace to my config, I for some reason then have a
main Inbox folder in my mail client and then it has a subdirectory
called inbox and that has all my IMAP folders listed. So it's a bit
redundant for the Inbox folder only.


Re: [Dovecot] IMAP Folders Don't Make Sense

2010-02-25 Thread Leeman Strout

On 2/25/2010 4:35 PM, Carlos Williams wrote:

On Thu, Feb 25, 2010 at 4:10 PM, Brian Haydenbhay...@umn.edu  wrote:

You need to:

1. Remove your subscriptions file.


This I did. Now I can no longer see from the client side any of my IMAP folders.


2. Set your client to ignore subscriptions and view all folders.


I logged into my webmail and in the main mailbox view all my IMAP
folders except for Inbox and the default ones were missing. I then
opened up the folders settings option and 'subscribed' to all the
folders I didn't see in my main mailbox view. Once I enabled
'subscription' via webmail, I could then see all my folders again. Is
this wrong? You stated the opposite should be done. I would think if
they were not selected for subscription, I would see them but it
appears to be the other way around. How can I see my messages on my
email client with out the 'subscriptions' file in my Maildir? I am
assuming this questions' answer varies depending on what client we're
talking about, no?


3. Then, and only then, settle on a server configuration (including any
namespaces you may choose to use), and then re-subscribe to folders in your
client (if you are going to insist on using subscriptions).

It's impossible to get your server configuration correct if you're judging
the user-visible side by a client using legacy subscriptions, particularly
if the subscriptions file is in an unreliable state (as it sounds like yours
is--as most of them almost always are).


I just can't see how to get any IMAP client to read IMAP folders w/o
that 'subscriptions' file. It appears that it looks for it or is in
some way dependent on this file. Am I wrong?


The subscriptions file is technically optional.  For instance, 
Thunderbird has a setting to either use nor not use it.  If TB uses 
subscriptions then you need to subscribe to all folders that you want to 
see.  Otherwise it will show all folders.


Dunno about your webmail, however as an example SquirrelMail uses 
subscriptions, no way around it.


With this combination, I'd enable webmail subscriptions and disable TB's 
use of subscriptions.


For TB if you want to ignore subscriptions, go into your Account 
Settings, select Server Settings under the correct account, click the 
Advanced button in the Server Settings sub-box, and uncheck Show only 
subscribed folders.




Leeman