It goes without saying. In a production system with large queues, every little
detail must be addressed. A suitable indexed filesystem (i.e. ReiserFS) should
be used for the spool.
Cheers,
Nikos
----- Original Message -----
From: Alejandro Guerrieri
To: Nikos Balkanas
Cc: Tony Kirkham ; users@kannel.org
Sent: Thursday, December 04, 2008 2:44 AM
Subject: Re: Removing an erroring message stuck in queue / file store
Yes, most of the time. The only exception seems to be if you're using some
journaled filesystems (e.g. ext3), and your spool holds a lot of messages. The
system would take some time to come up if there's a lot of messages enqueued
when starting. We're talking about tenths of thousands messages on the queue,
not just a few hundreds.
Using xfs or ext2 avoids that.
Other than that, at least in my own experience the spool folder is quite
easier to maintain imho.
Regards,
Alejandro
2008/12/3 Nikos Balkanas <[EMAIL PROTECTED]>
Thanks for pointing that out. Still it is easy to preview in "vim -b" (not
hex) and can identify *any* of the fields Sender, recipient, text, messageID,
so if anyone is missing there are a lot of matches in ASCII. The binary text is
easy and straightforward. One can reasonably find the borders (patterns) of the
previous and next message (presumably intact) and delete the in between.
Tony, if the file is small enough to send through mail (zipped) and don't
feel like trying it yourself, why don't you send it to me along with the
message details to see what I can do about it.
Alej from your words it seems that a spool directory is more efficient than
a storefile. If anytime kannel sends a message has to delete and rewrite almost
the whole store file, it is a big waste. Isn't so?
BR,
Nikos
----- Original Message -----
From: Alejandro Guerrieri
To: Nikos Balkanas
Cc: Tony Kirkham ; users@kannel.org
Sent: Thursday, December 04, 2008 2:18 AM
Subject: Re: Removing an erroring message stuck in queue / file store
The 20 nulls are probably a lot of empty/unused fields, that doesn't
necessary means that they'll be always empty... so don't assume that there
should be that much. It's not a separator, just a pattern in your particular
message queue set.
When you edit, you have to make sure that you're actually cutting on the
right places. Alas, if the message is in fact corrupted maybe it's not
complete, so YMMV.
Using the spool file is way easier, of course. Again, if you have a
"broken" message (why is that it's another question) it's just a matter of
determining which one is causing the problem and removing it.
Regards,
Alejandro
2008/12/3 Nikos Balkanas <[EMAIL PROTECTED]>
Hi,
Just out of curiosity I openned a store file and it doesn't seem too
difficult. You can edit it easily with "vim -b" and there are a lot of ASCII
fields (sender, recipient, messageid), so you can search for your message
easily. Each message seems to be bordered by 20 '\0' (NULLS). Can't miss them.
Find offending message and delete by hand (assuming that you know which message
is the offending one).
This again assumes that there is no CRC or indexing of the messages as
discussed by Alejandro. In other words, you can safely remove any message.
Make a backup, try it and let us know of the result. Stop kannel, and
repeat for both files.
Cheers,
Nikos
----- Original Message -----
From: Alejandro Guerrieri
To: Tony Kirkham
Cc: users@kannel.org
Sent: Wednesday, December 03, 2008 7:14 PM
Subject: Re: Removing an erroring message stuck in queue / file store
Ah, you're using the "file" store, not "spool" maybe?
AFAIK, there's no document on the storage format. It's mainly a dump
from the "Msg" structure, but the "Octstr" objects can have nulls in the
middle, so it's not as easy to parse and, it doesn't have a fixed length and
(if you're still willing to venture at it) you'll need an hex editor if you
don't want to mess the whole file (the nulls in the middle of the file will
probably render any text editor unusable).
If you use the "spool" store, each message is stored on an
independent file on a directory tree. Removing the file removes the message.
The only problem is that kannel only uses the files as a backup
mechanism and it doesn't refresh it during runtime. To clarify:
* When a message arrives (or is enqueued to be sent), kannel adds it
to an in-memory queue and stores a copy on disk. An internal uuid is used to
keep track of messages on both places.
* When kannel ends processing the message, it removes it from the
in-memory queue and deletes the message from disk. Depending on the store type
being used, this means removing it from the store file and rewriting the file,
or remove an entire file from the spool tree.
* If kannel crashes or is shutdown with messages pending, the
messages are kept on the file/spool queue. During a shutdown, the in-memory
queue is dumped to file again afaik.
* When kannel starts, it first load the messages from the file/spool
queue.
* On the particular case of the file store, a backup file is kept at
all times in case the main file is deleted or corrupted.
So, from the above, it's clear why you need to stop kannel before
deleting the file(s).
You can stop kannel, delete the store and store.bak files, or remove
the whole spool tree and start again. It will remove ALL your pending messages.
If you can figure out which message is causing the problem, you can remove it
from the spool (if you're using the spool store) as well. With the file store,
you'd need to figure out from the binary, which is near to impossible without
some specially crafted tools imho.
Regards,
Alejandro
On Wed, Dec 3, 2008 at 1:15 PM, Tony Kirkham <[EMAIL PROTECTED]>
wrote:
How? By hand, in my favorite editor? I have tried this, with bad
results. There are special characters, at least vi displays them that way,
that I can't interpret so I do not know where, exactly, one message stops and
the next starts. Is there a document that describes the store file in detail?
-Tony
On Tue, Dec 2, 2008 at 4:08 PM, Alejandro Guerrieri <[EMAIL
PROTECTED]> wrote:
And do it without kannel running, of course ;)
If you're using the spool store file, you can remove the
individual message (assuming that you can find out what's the message causing
the error).
Regards,
Alejandro
On Tue, Dec 2, 2008 at 5:49 PM, info.ubichip <[EMAIL PROTECTED]>
wrote:
Hello,
when you trash the .store file, don't forget to remove the
.store.bak as well otherwise, it will retry to send all the remaining message.
regards
----------------------------------------------------------------
From: Tony Kirkham [mailto:[EMAIL PROTECTED]
Sent: lundi 1 dĪ¹cembre 2008 21:32
To: users@kannel.org
Subject: Removing an erroring message stuck in queue / file
store
Is there a way to remove a message from the file store that is
erroring continuously allowing the other messages to be sent?
If I simply restart Kannel sometimes the other messages get
sent and sometimes they do not. Why is that?
If they all get sent but the erroring message I can simply
replace the file store with another file with a restart of Kannel, but I would
like to know if there is a way to remove the message by some other means.
Thanks,
-Tony
