Hi -
Having just recently struggled with a
still-not-100%-clear-how-it-happened mailboxes.db corruption issue that
Bron ended up having to help me fix, I have some thoughts on this. And
the recent bout of banging my head against the wall over this issue for
days has left me motivated to write up some of this documentation
myself, assuming I can find the information necessary. Here is my list
1. Suppose the /var/imap folder (the one containing the critical *.db
files) disappears completely: A step by step tutorial on how to get
your mail system back up and running. Such a tutorial alone would go a
long way towards clearing up the mysteries of how indexing is related
user mail folders. This tutorial should include explanations of what
can't be recovered (e.g. which messages have been viewed, possibly sieve
scripts, etc.).
2. Documentation on safely backing up and restoring a cyrus mail system.
There was a discussion about this on the list a year ago:
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2014-December/037849.html
(Further up the thread)
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2014-December/037799.html
Which I consider inconclusive. While Nic's suggestion to use
replication is probably the safest approach, there still needs to be
something for people who run a single mail server and want to keep it
that way. There's something to be said for simplicity. Also, the
process of setting up a replication server (with and without murder)
needs to be documented better. In the thread above, Nic suggested that
I look at this documentation:
http://cyrusimap.web.cmu.edu/docs/cyrus-imapd/2.4.17/ag.php
which I found to be mostly unhelpful. Tutorials are good even when they
dont match your exact use case. Too much abstract hand-waving is just
confusing to most people. I'd rather read a common use case tutorial
and then extrapolate to my own situation from there. Way too many
critical details end up getting left out when you talk about these
issues absractly.
3. The creation, use, and maintenance of sieve scripts. I dug into this
earlier this month and found almost nothing available online. I did get
some helpful responses on the cyrus list:
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2015-December/038701.html
but this should be in the documentation; including a tutorial on how to
work with sieve scripts and pointers to 3rd party tools, with perhaps
even a description of how to set up and use them.
4. Organization of the documentation. It looks like the worst case
scenario of having docs spread out over 2 domains with completely
different organizational schemes, style, etc.. is being addressed, but
some thought should nevertheless be given to this very difficult task.
My suggestion is to look at the organization of the documentation from
different perspectives:
- a new or potentially new cyrus user who knows absolutely nothing
- a somewhat experienced cyrus admin looking to quickly get an answer
to a specific question
- a cyrus admin who wants to implement some new functionality, say
moving from a single server to a murder, or start using sieve scripts
I use both cyrus and dovecot (the first by choice, the latter because I
got outvoted in a larger organization). Arch, Red Hat, and Ubuntu all
package dovecot as the default IMAP server; on Arch I have to install
cyrus from the AUR. I'm convinced that the one and only one reason for
this is that the dovecot guy has had pretty good and thorough
documentation from the very beginning. (That, and the cyrus project was
dormant for a while.) Technically, cyrus is cleaner, faster, more
scalable, etc..
Soapbox: having worked with open source software for decades, it's very
clear to me that open source projects become successful not on the
quality of the code, but rather the quality of the documentation. Good
projects with poor documentation generally die if there is an
alternative, poorly conceived projects with good documentation succeed.
Django is (in my opinion) pretty crappy software compared to say,
bottle.py (which most people haven't even heard of). Why is Django so
popular? Because they took the time to write good documentation. In
fact, if you want to use an MVC python web framework, the trick is to
read the Django documentation first, and then use something like
bottle.py or flask for your web app. Most developers don't bother to
follow up with step 2. They read the Django documentation ... and then
use Django.
On 12/30/2015 03:02 AM, Nicola Nye via Cyrus-devel wrote:
Hi Patrick,
Just before Christmas we got all the access sorted out and there's now a
copy of docs.cyrus.foundation on cyrusimap.org/new, and it's
auto-updating. It all took longer than we had hoped given FastMail's
exciting few weeks with the DDoS situation.
When I'm back in the office after the New Year I'll look at what's the
best way to go moving the old site out of the way and whether we
maintain any redirects so any old bookmarks still work for common entry
points.
Then we can start on the larger task of integrating the docs source back
into the cyrus code source.
If there's areas that you think need immediate and obvious updating, do
let me know (especially if you can provide pointers to where I can get
the information from, even if it's just links to list threads); I'd
rather ensure that my time is spent easing definite pain points.
Cheers,
Nicola
On Mon, Dec 28, 2015, at 01:45 AM, Patrick Goetz via Cyrus-devel wrote:
Was this issue ever resolved? I'm still finding the documentation split
between cyrus.foundation and cyrusimap.org, and it's incomplete in both
places.
On 09/16/2015 08:50 PM, Nicola Nye wrote:
Dear Cyrus folks,
After nominally completing the migration of the docs from the wiki at
cyrusimap.org onto the sphinx repository at docs.cyrus.foundation, I
worked out there was a whole bunch of existing services at cyrusimap.org
that were non-trivial to relocate. Which, after some discussion at the
Cyrus hangout, we felt that retaining cyrusimap.org as our primary (and
sole) presence made more sense.
Equally, there are some documentation issues that are affected by our
current straddling of two worlds. Read on for a discussion of the issues
and down to the options and recommendation.
I'd love to have your input (particularly Nic Bernstein on the docs
angle!) if you agree or disagree so we can make for a more unified Cyrus
presence.
*Issues*
1) Branding
We are currently split across two internet presences: cyrus.foundation
and cyrusimap.org. This is confusing. We should only have one.
The cyrus.foundation domain name is not of real use as there is a
pre-existing entity (The Cyrus Foundation) of the same name.
Swapping our internet domain name presence from cyrusimap.org to
cyrus.foundation is no longer a significant driver as a result,
especially as cyrusimap.org is already indexed and linked to around the
net and is unambiguous. (Yes, Cyrus does so much more than just imap,
but it's the primary association)
Currently services are split across the two domains - git repos,
phabricator and authoritative online docs on cyrus.foundation, while
downloads, bugzilla and generated docs are on cyrusimap.org.
2) Docs logistics
The docs repo is separate to source which is causing friction keeping
man pages updated.
We want man pages in two formats: *nix for actual man pages to ship and
install with cyrus, and html for web presentation.
Nice to have: ship a snapshot of all current docs (not just man pages)
along with source distribution.
3) Sphinx vs wiki
We have been working towards making the content on cyrus.foundation to
be the most up to date. This is using Sphinx, which allows us to
generate, from the same source, man pages as well as web pages. (And
even pdf or single page html). So we get nicer output format options.
Clearer history in the git log than you get from wiki.
Cyrusimap docs are held in mediawiki. Not so great for man pages. But
it's much easier for third party contributor to pitch in and help out:
they don't need to learn rst, they don't need a phabricator account,
they don't need to navigate git.
*Possible solutions*
In all cases it seems clear that we should stop using the
cyrus.foundation domain and ideally services that are hosted there be
shifted to live alongside cyrusimap.org for ease of access.
There are a number of @cyrus.foundation email accounts that would need
to be dealt with for Chris Davies' testing.
Option 1: Single domain, unify git repos, use sphinx everywhere
--------------------------------------------------------------------------------
Moving the git repo to cyrusimap (or anywhere) isn't hard. (a job for Bron)
Get rid of separate cyrus-docs repo. Put cyrus-docs stuff back inside
cyrus-imapd/docs for easier man page generation and tagging of docs
versions with source, and shipping of current docs with source.
Put sphinx on cyrusimap to replace the wiki. This requires either:
1.
working out how to set up the existing generated docs for use with
sphinx,
2.
or make all the stuff in cyrus-imapd/doc into rst so it works with
sphinx. We can still ship the built html.
Option 2: Single domain, unify git repos, use wiki for docs
-----------------------------------------------------------------------------
Put sphinx on cyrusimap.org just for generating man page html. Leave the
existing wiki where it is for documentation. (Requires porting the page
updates I made from cyrus.foundation onto the wiki).
This means the only thing left in the cyrus-docs git repo would be the
man pages, at which point it makes more sense to put them back into the
cyrus-source repo.
We can still ship docs with the release if we tie in a wiki export into
the release-building process. (A job for Ellie and I)
Option 3: Single domain, separate git repos, use sphinx for docs
----------------------------------------------------------------------------------
Move all services to cyrusimap.org, but still leaves us with all the
docs logistics and sphinx and wiki chafe points. Not recommended.
*Recommendation*
I am leaning towards suggesting option 2. Anything that makes
documentation support easier is a good! But we'd still like to retain
the usefulness of sphinx for generating two output formats from single
source man pages.
*What Now?
*
Discuss! I imagine those folks who come to the Hangout in the next few
meetings will kick it around and we'll come to a decision next week.
Nicola
