Log mail access
Hello, is it somehow possible to log the access to a mail? The log entry should be written, if a user fetches a mail via imap or pop3. It would be convenient for debugging. I've checked the Logging-Wiki and the docs for the mail_log plugin, but they seem to log any event but mail access. Best regards, Philipp
Re: stacking istreams and ostreams
Well, I've found the bug. I've got confused with the stream-stacking function pointers. The encryption istream was stacked on top of the parent and the ostream below the parent. That caused this very confusing bug. Best, Philipp Am 11.12.2014 um 12:16 schrieb Philipp Brüll: Hello, I'm developing an encryption plugin for dovecot and ran into a problem with the stacking of i/o-streams. The encryption i/o-streams are working fine on any kind of mail the test suite is passing through them. But as soon as the zlib plugin is enabled the logs show an cache error: failed: Cached message size larger than expected (214 206, box=INBOX, UID=1) I've already double-checked the return values of ostream's sendv and istream's read function. They seem correct (and equal). If the order of the streams are changed (by changing the number in the lib-filename libxx_scrambler.so); meaning that the encryption is done before the compression (which isn't efficient) both streams are working correct without any errors. Is there some way the zlib plugin changes the cached message size? Is there some behaviour of the zlib plugin that I'm missing? Any help would be very welcome. Best regards, Philipp signature.asc Description: OpenPGP digital signature
Re: stacking istreams and ostreams
Here is my ugly solution for this:
static int plugin_mail_save_begin(
struct mail_save_context *context,
struct istream *input
) {
...
if (mbox-super.save_begin(context, input) 0)
return -1;
output = scrambler_ostream_create(
context-data.output-real_stream-parent,
suser-public_key);
o_stream_unref(context-data.output-real_stream-parent);
context-data.output-real_stream-parent = output;
return 0;
}
The solution is ugly, because it's only working if there is another
ostream (in my case the zlib ostream). It would be better to add the
scrambler ostream to the other side of the ostream chain, instead of
messing with the ostream's parent.
I've tried to re-order the plugin itself - which brings the ostreams in
order - but than the istream order is messed up.
What would be the right solution here? Maybe someone with deeper
insights into dovecot's architecture can help.
Kind regards,
Philipp
Am 12.12.2014 um 15:25 schrieb Philipp Brüll:
Well, I've found the bug. I've got confused with the stream-stacking
function pointers. The encryption istream was stacked on top of the
parent and the ostream below the parent. That caused this very confusing
bug.
Best,
Philipp
Am 11.12.2014 um 12:16 schrieb Philipp Brüll:
Hello,
I'm developing an encryption plugin for dovecot and ran into a problem
with the stacking of i/o-streams.
The encryption i/o-streams are working fine on any kind of mail the test
suite is passing through them. But as soon as the zlib plugin is enabled
the logs show an cache error:
failed: Cached message size larger than expected (214 206, box=INBOX,
UID=1)
I've already double-checked the return values of ostream's sendv and
istream's read function. They seem correct (and equal).
If the order of the streams are changed (by changing the number in the
lib-filename libxx_scrambler.so); meaning that the encryption is done
before the compression (which isn't efficient) both streams are working
correct without any errors.
Is there some way the zlib plugin changes the cached message size? Is
there some behaviour of the zlib plugin that I'm missing? Any help would
be very welcome.
Best regards,
Philipp
signature.asc
Description: OpenPGP digital signature
stacking istreams and ostreams
Hello, I'm developing an encryption plugin for dovecot and ran into a problem with the stacking of i/o-streams. The encryption i/o-streams are working fine on any kind of mail the test suite is passing through them. But as soon as the zlib plugin is enabled the logs show an cache error: failed: Cached message size larger than expected (214 206, box=INBOX, UID=1) I've already double-checked the return values of ostream's sendv and istream's read function. They seem correct (and equal). If the order of the streams are changed (by changing the number in the lib-filename libxx_scrambler.so); meaning that the encryption is done before the compression (which isn't efficient) both streams are working correct without any errors. Is there some way the zlib plugin changes the cached message size? Is there some behaviour of the zlib plugin that I'm missing? Any help would be very welcome. Best regards, Philipp -- simia.tech GbR http://simiatech.com
[Dovecot] Finding memory leaks
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I try to find a memory leak in a dovecot plugin that I develop. In order to find it, it would be helpful to print the total amount of memory that is currently allocated. This print could than spread over the source code and the memory consumption can be tracked. I've tried i_debug(pool size %u, (unsigned int)pool_alloconly_get_total_alloc_size(system_pool)); But that failed. Does someone has a good advice? I would like to avoid complex solutions like valgrind. Best regards, Philipp -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEbBAEBAgAGBQJTIEJjAAoJEMxVSu8VsNAvlnUH+Nf9ueeaer8rpJ3gfaaz9hk8 KlKb0O0pamgwfNy3rqWR0jU0BRKhZaYUeVRv+9tk1qV2cMv11prcB+P0DojKbVuQ VgqBCYjeHbRXAasmm/39fieJ/c33y67RGKzVR50oNnhmQ/tbtUXwFdNGs5vGKzJs fZwD7EaoBQd5VFWfOu5oUrKCbIv4YVTIt8rcBah04oDV0nWilpgvFFUHo3+nKNpc SjSbhoXU/uRsrIPp4PsndCaRdZl4UfadFehu9xBsuZpZfamODozx5oEJhR4ArYMK gJYZGvSxGURgad/yRkkn3AIKHiJk0/MencjGcmi9ijdoKF4IhZL+k4w4oqjwRw== =/jzf -END PGP SIGNATURE- smime.p7s Description: S/MIME Cryptographic Signature
[Dovecot] Finding memory leaks
Hello, I try to find a memory leak in a dovecot plugin that I develop. In order to find it, it would be helpful to print the total amount of memory that is currently allocated. This print could than spread over the source code and the memory consumption can be tracked. I've tried i_debug(pool size %u, (unsigned int)pool_alloconly_get_total_alloc_size(system_pool)); But that failed. Does someone has a good advice? I would like to avoid complex solutions like valgrind. Best regards, Philipp
Re: [Dovecot] Order of istream and ostream chains
On 11/02/14 21:32, Timo Sirainen wrote: On 12.2.2014, at 2.27, Philipp Brüll philippbru...@gmail.com wrote: I'm creating a scrambler plugin, that adds an istream and an ostream to the stream-chain for the mail input/output. It works well until the zlib plugin is added to the configuration. The scrambler should run before the zlib and encrypt the mail before it's compressed. Since, the plugin is named lib18_scrambler_... (and the other lib20_zlib), that works well when a mail is received. When a mail is read via IMAP, the plugins should handle the mail in the reverse order. So first, the zlib should decompress it and afterwards the scrambler should decrypt it. But it seems, that they work the other way around. The scrambler istream gets compressed data as input. It's hooked in the chain of istream as the following... In your previous mail you mentioned you're using v2.1.17. Have you tried with v2.2.10? I think this is already fixed (at least I've successfully used zlib + mail encryption plugin). I just tried version 2.2.11, but the problem seems still to exists. As soon as the zlib is activated, the scrambler receives 0x1f, 0x8b, 0x08, ... which is exactly the gz header. The mail doesn't seem to get decompressed before it's passed to the scrambler. I assume that the order of istreams is messed up. I've checked already the istreams of the mail_filter and zlib plugins, but can't find any differences to my implementation. Do you know any other example code, where I can see how ostreams/istreams are chained in the correct order? Best regards, Philipp smime.p7s Description: S/MIME Cryptographic Signature
[Dovecot] Order of istream and ostream chains
Hi,
I'm creating a scrambler plugin, that adds an istream and an ostream to
the stream-chain for the mail input/output. It works well until the zlib
plugin is added to the configuration. The scrambler should run before
the zlib and encrypt the mail before it's compressed. Since, the plugin
is named lib18_scrambler_... (and the other lib20_zlib), that works
well when a mail is received.
When a mail is read via IMAP, the plugins should handle the mail in the
reverse order. So first, the zlib should decompress it and afterwards
the scrambler should decrypt it.
But it seems, that they work the other way around. The scrambler istream
gets compressed data as input. It's hooked in the chain of istream as
the following...
static int scrambler_istream_opened(struct mail *_mail, struct istream
**stream) {
struct mail_private *mail = (struct mail_private *)_mail;
union mail_module_context *mmail = SCRAMBLER_MAIL_CONTEXT(mail);
struct istream *input, *inputs[2];
input = *stream;
*stream = scrambler_istream_create(input);
i_stream_unref(input);
return mmail-super.istream_opened(_mail, stream);
}
static void scrambler_mail_allocated(struct mail *_mail) {
struct mail_private *mail = (struct mail_private *)_mail;
struct mail_vfuncs *v = mail-vlast;
union mail_module_context *mmail;
mmail = p_new(mail-pool, union mail_module_context, 1);
mmail-super = *v;
mail-vlast = mmail-super;
v-istream_opened = scrambler_istream_opened;
MODULE_CONTEXT_SET_SELF(mail, scrambler_mail_module, mmail);
}
How can I reverse the order of istreams? Should I use another hook or vfuns?
I'm stuck in the problem for a while now, so any help would be very welcome.
Best regards,
Philipp
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [Dovecot] Order of istream and ostream chains
On 11/02/14 21:32, Timo Sirainen wrote: On 12.2.2014, at 2.27, Philipp Brüll philippbru...@gmail.com wrote: I'm creating a scrambler plugin, that adds an istream and an ostream to the stream-chain for the mail input/output. It works well until the zlib plugin is added to the configuration. The scrambler should run before the zlib and encrypt the mail before it's compressed. Since, the plugin is named lib18_scrambler_... (and the other lib20_zlib), that works well when a mail is received. When a mail is read via IMAP, the plugins should handle the mail in the reverse order. So first, the zlib should decompress it and afterwards the scrambler should decrypt it. But it seems, that they work the other way around. The scrambler istream gets compressed data as input. It's hooked in the chain of istream as the following... In your previous mail you mentioned you're using v2.1.17. Have you tried with v2.2.10? I think this is already fixed (at least I've successfully used zlib + mail encryption plugin). Thanks for the fast reply. Yes, I'm using v2.1.17. If this is a bug in 2.1, is there a workaround existing? I'm already pushing the admin to upgrade to 2.2, but I don't know is this is happening soon ;-) Kind regards, Philipp smime.p7s Description: S/MIME Cryptographic Signature
[Dovecot] Plugin architecture
Hi, I'm currently developing a scrambler plugin which should be used to store the mails encrypted on the disk. It uses a special ostream for encryption and an istream for decryption. The idea is, that if a mail arrives via LMTP it goes through the ostream and is written encrypted to disk. If an mail is accessed via IMAP, the istream is used to decrypt the data from disk. That all works fine, but if I add the zlib plugin it all gets mixed up. For some reason, the lmtp process tries to read something from disk using the chain of istreams. That behaviour is a little bit bizzare to me, because my understanding is, that the zlib plugin at this point should only write (compressed) data to disk. Why is the istream used here? And what is read? If I remove zlib from the configuration again, it all works as expected. The dovecot version is 2.1.17. Best regards, Philipp
Re: [Dovecot] Accessing plain text password from memory
Hi Timo,
thanks for the answer. I'm working on a plug in with an similar
architecture.
Is there also a way to pass that plain password to a mail filter script?
Obviously, the %w option as mail filter script argument does not work.
Kind regards,
Philipp
On 13/12/13 15:47, Timo Sirainen wrote:
On 13.12.2013, at 16.37, Stanislas SABATIER s.sabat...@pobox.com wrote:
Is there a way to retrieve the client's password in plain text from memory ?
I don't store the password in plain text in my postgreSQL but I need it
when the client is connected to make crypto computation.
If I write a plugin to do the job, how could I retrieve the plain text
password from master ?
Assuming you you're using passdb sql and userdb prefetch and you want to access
the password in imap/pop3/etc process, you can do:
password_query = '%w' as userdb_password, ...
Then the password will be available the same way as plugin { password } would
be available (mail_user_plugin_getenv()).
You could also write a passdb plugin you could access the password directly from
auth_request-mech_password.
smime.p7s
Description: S/MIME Cryptographic Signature
[Dovecot] Plugin development - ostream and index cache size
Hi, I'm new to this list, so first thanks for the dovecot tool! So far, it has been great to work with. Currently, I'm developing a scrambler plugin for dovecot, that should stores the mails encrypted. To do that, I've created the scrambler_ostream and scrambler_istream that does the encryption and that I hook in the mail storage process. The scrambler_ostream is stacked on top of the output stream of the mail_save_context (mail_save_context-data.output) and the scrambler_istream is attached to the input stream when the virtual function istream_opened in the mail_private struct is called. Basically, the encrypt/decrypt works, but during the process the data size changes slightly. And when I'm doing a FETCH BINARY[] via imap, it results in the error... Cached message size smaller than expected (204 208) Error: Corrupted index cache file /home/.../mail/mailboxes/INBOX/dbox-Mails/dovecot.index.cache: Broken physical size for mail UID 1 My guess is, that the FETCH function tries to deliver the unencrypted cache version of the mail (204 bytes) and not the stored version (208 bytes). How can I make dovecot cache the output of my scrambler_ostream or at least disable this caching for testing? I've read already a lot of code, but I've found a hint yet. How does - for example - the zlib plugin handle this issue? I have read the code a couple of times, but still have no clue. Best regards, Philipp
