Re: Logfiles above 2G
Hi,
The problem is not the kernel - 32-bit linux kernels have no problem
handling files larger than 2GB. But if the program is not compiled with
support for large files, it will receive an interrupt when it attempts
to write past the 2GB barrier. The solution is to compile with the
CFLAGS -D_LARGE_FILES -D_FILE_OFFSET_BITS=64. Actually this just makes
the program call open64() and lseek64() instead of open() and lseek().
Compile kannel in this way, and kannel can handle enormous logs without
crashing.
I thought that large file support was actually enabled by default now
(in the latest CVS) - I believe I saw a note about that some time ago.
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
Mai: [EMAIL PROTECTED]
www: www.coolsystems.dk
Andreas Fink wrote:
Vincent says:
- logrotate is your friend... :)
- or use 64 bits arch
1. Logrotate is already my friend (it was misconfigured though...)
2. 64bit architecture is not a limit for having larger files. MacOS X on
intel has no problem creating such large files.
On 06.09.2006, at 02:34, Mi Reflejo wrote:
The 2gb limit is in 32-bit cpu: Most exactly in libc, kernel fs
drivers and VFS layer.
This is not correct anymore. The kernel is running 32bit (but is
actually a AMD Opteron with EMT64 support) but can deal with large
filesystems. The filesystem also supports file larger than 2GB (people
using VMWAre would cry otherwise). So the only location left is in libc.
See this:
Let me create a file larger than 2G
$ dd if=/dev/zero of=test bs=1M count=2050
2050+0 records in
2050+0 records out
2149580800 bytes (2.1 GB) copied, 48.2032 seconds, 44.6 MB/s
[EMAIL PROTECTED] ~]$ ls -l test
-rw-rw-r-- 1 afink afink 2149580800 Sep 6 05:42 test
[EMAIL PROTECTED] ~]$
So we can create files larger than 2GB on that filesystem. This was not
the case in 2001 when I run into that problem too where the filesystem
was really the limit. So it can not be kernel, not filesystem. So it
must be a libc limitation but for appending data to a existing file, it
sounds rather strange to me that fprintf has this limit. Lets test this
with a small piece of C:
#include stdio.h
#include errno.h
main()
{
FILE *f;
int i;
f = fopen(test.bin,a);
if(f==NULL)
{
fprintf(stderr,could not open file\nerrno= %d\n,errno);
return -1;
}
fseek(f,0,SEEK_END);
for(i=0;i1000;i++)
{
fprintf(f,The world is nice\n);
}
fclose(f);
}
This code shows that when I do a fopen on the 2.1GB file, I get an error
27 back. ( EFBIG). So it must be libc issue. However libc is used on my
other platforms too and doesnt show this behaviours.
As dd has no probelm, Kannel can be made not having a problem too.
Furthermore, Kannel should not crash in that scenario which it currently
does.
And Kannel doesnt recover from that crash after a restart because the
logfile doesnt change its size (we dont truncate it). So using the box
around the box to autorestart it, wont solve it.
Re: [PATH] convert internal charset to UTF-8
Hi Alex,
Awesome initiative! I've been hoping for this to happen for quite a
while. There are a few issues though:
1. In the gwlib/latin1_to_gsm.h, SP (space) is replaced with ESC
(0x1B), and ESC is mapped to NRP instead of just ESC. (If you follow me)
2. For some odd reason, smsbox trims the message to 160 characters,
while it is in utf-8 format... My usual charset test message which
contains all GSM characters except the Greek ones (wasn't possible
before now), looks like this:
Test: @£$¥èéùìòÇ
Øø
Åå_ÆæßÉ
!#¤%'()*+,-./0123456789:;=?¡ABCDEFGHIJKLMNOPQRSTUVWXYZÄÖÑܧ¿abcdefghijklmnopqrstuvwxyzäöñüà^{}\[~]|€
Which in UTF-8 takes up 163 octets, but only 141 septets in GSM. When
transmitting, the € is omitted, and judging from a ngrep of data
transfered from smsbox to bearerbox, it is smsbox which does the
trimming. For the record, the string is exactly 160 octets long when €
is omitted.
Apparently it uses the size of the GSM string to determine when to
split, but the trimming/splitting is done on the UTF-8 string. Obviously
it is sms_split, which is to blame, but why is this function used at all
if splitting is done in bearerbox (according to comments in source) -
this problem is probably not directly related to the utf-8 patch.
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
Mai: [EMAIL PROTECTED]
www: www.coolsystems.dk
Alexander Malysh wrote:
Hi all,
at http://www.kannel.org/~amalysh/kannel-utf8.patch is a not so huge
patch that converts internal kannel charset to UTF-8. Please note that I
didn't add smsbox compatibility code, means smsbox expect text body to
be encoded in UTF-8 as default also MOs will be forwarded in UTF-8. It
could be workarounded with charset cgi variable.
Please test it and send feedback/patches.
I will maintain this patch for a while as long as we don't decide to
commit it to CVS.
--Thanks,
Alex
Re: [PATH] convert internal charset to UTF-8
Hi,
Alexander Malysh wrote:
Hi,
Am 20.07.2006, 12:48 Uhr, schrieb Peter Christensen [EMAIL PROTECTED]:
Hi Alex,
Awesome initiative! I've been hoping for this to happen for quite a
Thanks!
while. There are a few issues though:
1. In the gwlib/latin1_to_gsm.h, SP (space) is replaced with ESC
(0x1B), and ESC is mapped to NRP instead of just ESC. (If you
follow me)
ok, here was a typo, changed SP to 0x20 but ESC should be NRP
because it's non representable in GSM.
I see your point. Assuming that kannel is updated if and when the GSM
charset is extended further in the future, the ESC really should be
NRP, but then again, I've experienced a few gateways which required you
to transmit the escape sign yourself for some reason... They probably
used iso-8559-1 charset and I needed € or whatever, and in such cases
the charset_utf8_to_gsm wouldn't be called anyway. My thought was
primarily in case the GSM charset was changed further. (In short, I can
live without the ESC :D)
2. For some odd reason, smsbox trims the message to 160 characters,
while it is in utf-8 format... My usual charset test message which
contains all GSM characters except the Greek ones (wasn't possible
before now), looks like this:
Test: @£$¥èéùìòÇ
Øø
Åå_ÆæßÉ
!#¤%'()*+,-./0123456789:;=?¡ABCDEFGHIJKLMNOPQRSTUVWXYZÄÖÑܧ¿abcdefghijklmnopqrstuvwxyzäöñüà^{}\[~]|€
Which in UTF-8 takes up 163 octets, but only 141 septets in GSM. When
transmitting, the € is omitted, and judging from a ngrep of data
transfered from smsbox to bearerbox, it is smsbox which does the
trimming. For the record, the string is exactly 160 octets long when €
is omitted.
Apparently it uses the size of the GSM string to determine when to
split, but the trimming/splitting is done on the UTF-8 string.
Obviously it is sms_split, which is to blame, but why is this function
used at all if splitting is done in bearerbox (according to comments
in source) - this problem is probably not directly related to the
utf-8 patch.
hmm, strange... I will look in smsbox code if you don't beat me ;)
The smsbox check max allowed messages from config and try to split
message sms_split. If there more as allowed smsbox send only allowed count.
Heh, beating you probably wouldn't solve anything :o)
Actually I would have looked at it myself, if it wasn't because it
apparently split the message just to combine the lot again, which seemed
kinda silly.
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
Mai: [EMAIL PROTECTED]
www: www.coolsystems.dk
Alexander Malysh wrote:
Hi all,
at http://www.kannel.org/~amalysh/kannel-utf8.patch is a not so huge
patch that converts internal kannel charset to UTF-8. Please note
that I didn't add smsbox compatibility code, means smsbox expect text
body to be encoded in UTF-8 as default also MOs will be forwarded in
UTF-8. It could be workarounded with charset cgi variable.
Please test it and send feedback/patches.
I will maintain this patch for a while as long as we don't decide to
commit it to CVS.
--Thanks,
Alex
--Thanks,
Alex
Re: [PATCH] Internal charset
Hi Alex,
Euro sign is 0x80 in windows-1252. It is iso-8859-15 which have euro at
the same place as the currency sign.
gsm_to_latin1:
static const struct {
int gsmesc;
int latin1;
} gsm_escapes[] = {
{ 10, 12 }, /* ASCII page break */
{ 20, '^' },
{ 40, '{' },
{ 41, '}' },
{ 47, '\\' },
{ 60, '[' },
{ 61, '~' },
{ 62, ']' },
{ 64, '|' },
{ 101, 128 },
{ -1, -1 }
};
101, 128 is €.. 0x1B 0x65 as GSM, 0x80 as windows-1252
latin1_to_gsm:
'x', 'y', 'z', -40, -64, -41, -61, NRP, /* 120 - 127 */
-101, NRP, NRP, NRP, NRP, NRP, NRP, NRP, /* 128 - 135 */
Once again, 0x80 is encoded into 0x1B 0x65
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
Alexander Malysh wrote:
Hi Peter,
I'm may be blind but I don't see where gsm_to_latin1 and latin1_to_gsm
process euro sign?
gsm_to_latin1:
- euro sign is not in gsm_escapes
- euro sign will be converted to 'e' because esc will be deleted
latin1_to_gsm:
- euro sign has the same code as CURRENCY SIGN and will be mapped to it (see
latin1_to_gsm array). here is snipplet:
/* 160 - 167 */
' ',
64, /* Inverted ! */
'c', /* approximation of cent marker */
1, /* Pounds sterling */
36, /* International currency symbol */
3, /* Yen */
64, /* approximate broken bar as inverted ! */
95, /* Section marker */
Does it all make sense for you or I'm overlooked anything?
Thanks,
Alex
Peter Christensen wrote:
Hi,
The GSM charset have € as an escaped character (0x1B 0x65) and
latin1_to_gsm() and gsm_to_latin1() assume windows-1252 character set.
So while I do admit that the patch i focused on SMPP, I doubt that it
breaks any of the other protocols.
If I go through each SMSC module:
smsc_at.c: Never does any charset conversion, but uses latin1_to_gsm and
gsm_to_latin1. So actually, this one already assumes windows-1252.
smsc_cgw.c: Apparently already assumes windows-1252 (0x80 = €). Does no
generic charset conversion.
smsc_cimd.c: Uses iso-8859-1. This one will need patching.
smsc_cmid2.c: iso-8859-1. Needs patching.
smsc_emi.c: Uses latin1_to_gsm/gsm_to_latin1
smsc_emi_x25.c: Uses its own gsm_to_iso function. The code looks kinda
deprecated. No support for extended chars at all, apparently.
smsc_fake.c:
smsc_http.c: Seems to do no charset conversion
smsc_ois.c: Uses latin1_to_gsm some places, but a simplified
gsm_to_iso88591 conversion elsewhere.
smsc_oisd.c: Uses latin1_to_gsm/gsm_to_latin1
smsc_sema.c: Uses a simplified gsm conversion like the one in smsc_ois.c
smsc_smasi.c: Not sure what charset this assumes. There are no apparent
charset conversions in place
smsc_smpp.c: Uses latin1_to_gsm/gsm_to_latin1 and charset conversion.
Currently originator string is windows-1252 and body is iso-8859-1.
smsc_soap.c: Uses iso-8859-1
smsc_wrapper.c: No apparent charset conversion
My point is, that while some protocols currently assume iso-8859-1, many
uses the latin1_to_gsm/gsm_to_latin1 which is ALREADY windows-1252.
Receipted messages from these gateways are windows-1252 as we speak,
although documentation says otherwise. But as long as smsbox uses
iso-8859-1 and not windows-1252, no gateway can transmit the € character
without manual escaping which I think is lame. If the charset in smsbox
was changed, at least some would have the possibility.
All this being said, I do agree that using UTF-8 internally is the best
way to go (but I assume that it will take a while before this is done).
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
Alexander Malysh wrote:
Hi,
I don't see how your patch should help with euro sign if SMSC supports
only GSM charset? and your patch is incomplete because it changes only
SMPP module.
What would be more suitable to support all GSM chars, is to switch
internal kannel charset to UTF-8. I have patch somewhere but it will take
some time to rebase it against current CVS and it's too intrusive (not
1.4.1 material).
For now it would be easy to keep latin1 as default but allow ESC (27) to
go through (in gwlib/charset.c change it from NRP to 27) and then you
should be able to send euro sign via sendsms interface.
Thanks,
Alex
Peter Christensen wrote:
Hi,
At the request of Hillel, I have agreed to update my patch for the
internal character set of smsbox/smpp, and post it here, hoping for it
to be committed to CVS.
It:
* Changes the default 7-bit character set of smsbox to windows-1252
instead of iso-8859-1, adding support for the euro-sign. (remember that
the latin1/gsm conversion functions already assumes windows-1252)
* smsbox uses charset_convert instead of octstr_recode, because the
latter will convert the euro-sign into a HTML
[PATCH] Revised smsbox leak/segfault patch
Hi,
I know it is long ago, but back in the beginning of February, I posted a
patch which resolved a number of memory leaks and potential segmentation
faults in the smsbox_req_handle. The patch was not accepted, apparently
primary due to a lot of ugly labels and stuff. So now, 3 months later,
I've taken the time to do a prettier patch (hopefully).
To summarize:
* Relevant List and Octstr variables are initialized to NULL
* All field errors jump to field_error label
* Transmission failure jump to transmission_error label
* Regardless of the outcome, memory cleanup and function termination is
the same piece of code (labeled cleanup)
Without this patch, newfrom, denied, allowed, receiver, and failed_id
might not be freed, depending on the error condition. When I worked with
the old patch, segmentation faults was also possible under the right
circumstances, but apparently this is not the case anymore (or x86-64
behave different than x86 - I've changed architecture since then)
I apologize for the long delay, but my mind have been elsewhere. And
since my own installation of kannel is patched with my old patch, I
haven't experienced leaks and segmentation faults since, and have thus
not been reminded of the issue. (This is actually also the case with the
MySQL escaping in dlr_mysql.c)
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
--- gateway.old/gw/smsbox.c 2006-04-28 17:33:54.0 +0200
+++ gateway.new/gw/smsbox.c 2006-05-06 19:25:44.0 +0200
@@ -1973,9 +1973,13 @@
List *receiver, Octstr *binfo, int priority)
{
Msg *msg = NULL;
-Octstr *newfrom, *returnerror, *receiv;
+Octstr *newfrom = NULL;
+Octstr *returnerror = NULL;
+Octstr *receiv;
Octstr *stored_uuid = NULL;
-List *failed_id, *allowed, *denied;
+List *failed_id = NULL;
+List *allowed = NULL;
+List *denied = NULL;
int no_recv, ret = 0, i;
long del;
@@ -1996,11 +2000,11 @@
*/
if (udh != NULL (octstr_len(udh) != octstr_get_char(udh, 0) + 1)) {
returnerror = octstr_create(UDH field misformed, rejected);
-goto fielderror2;
+goto field_error;
}
if (udh != NULL octstr_len(udh) MAX_SMS_OCTETS) {
returnerror = octstr_create(UDH field is too long, rejected);
-goto fielderror2;
+goto field_error;
}
/*
@@ -2114,6 +2118,12 @@
del = gwlist_delete_matching(allowed, receiv, octstr_item_match);
}
+/* have all receivers been denied by list rules?! */
+if (gwlist_len(allowed) == 0) {
+returnerror = octstr_create(Number(s) has/have been denied by white-
and/or black-lists.);
+goto field_error;
+}
+
if (urltrans_faked_sender(t) != NULL) {
/* discard previous from */
newfrom = octstr_duplicate(urltrans_faked_sender(t));
@@ -2125,7 +2135,7 @@
newfrom = octstr_duplicate(global_sender);
} else {
returnerror = octstr_create(Sender missing and no global set,
rejected);
- goto fielderror2;
+ goto field_error;
}
info(0, sendsms sender:%s:%s (%s) to:%s msg:%s,
@@ -2150,7 +2160,7 @@
msg-sms.account = account ? octstr_duplicate(account) : NULL;
} else {
returnerror = octstr_create(Account field misformed, rejected);
- goto fielderror;
+ goto field_error;
}
}
msg-sms.msgdata = text ? octstr_duplicate(text) : octstr_create();
@@ -2162,7 +2172,7 @@
if(octstr_len(dlr_url)) {
if(octstr_len(dlr_url) 8) { /* http(s):// */
returnerror = octstr_create(DLR-URL field misformed, rejected);
- goto fielderror;
+ goto field_error;
} else {
Octstr *tmp;
tmp = octstr_copy(dlr_url, 0, 7);
@@ -2174,7 +2184,7 @@
if(octstr_case_compare(tmp, octstr_imm(https://;)) != 0) {
returnerror = octstr_create(DLR-URL field misformed,
rejected);
O_DESTROY(tmp);
- goto fielderror;
+ goto field_error;
}
#ifdef HAVE_LIBSSL
msg-sms.dlr_url = octstr_duplicate(dlr_url);
@@ -2193,49 +2203,49 @@
if ( dlr_mask -1 || dlr_mask 31 ) { /* 0001 */
returnerror = octstr_create(DLR-Mask field misformed, rejected);
- goto fielderror;
+ goto field_error;
}
msg-sms.dlr_mask = dlr_mask;
if ( mclass -1 || mclass 3 ) {
returnerror = octstr_create(MClass field misformed, rejected);
- goto fielderror;
+ goto field_error;
}
msg-sms.mclass = mclass;
if ( pid -1 || pid 255 ) {
returnerror = octstr_create(PID field misformed, rejected);
- goto fielderror;
+ goto field_error
[PATCH] Leaks and segfaults in smsbox
Hi,
In relation to the thread Memory leak in smsbox, here's a patch which
corrects some of the errors.
The patch reorganizes the error goto-fields so they are as followed:
fielderror1:
At this point, only receiver have been allocated
fielderror2:
receiver, allowed, and denied have been allocated (allowed and denied
may contain elements from receiver)
fielderror3:
receiver, allowed, denied, newfrom, and msg have been allocated
(allowed and denied contains elements from receiver)
error:
receiver, allowed, denied, newfrom, and failed_id have been allocated
(allowed is empty, and denied and failed_id may contain elements from
receiver)
Besides, the Number(s) has/have been denied by white- and/or
black-lists. error have been moved to before SMS transmission is
attempted, and the logic is now gwlist_len(allowed) == 0 (No recipient
have been allowed)
This patch overrules my original patch for the memory leak.
For the record; While monitoring its memory consumption, I've flooded
smsbox with messages for each possible error condition (except
conditions where recipients are added to the failed_id list. Don't know
how to provoke that condition), and there seems to be no memory leaks,
double frees, or segmentation faults.
--
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
diff -Nru gateway/gw/smsbox.c gateway.smsbox/gw/smsbox.c
--- gateway/gw/smsbox.c 2005-12-09 03:14:31.0 +0100
+++ gateway.smsbox/gw/smsbox.c 2006-01-23 17:51:55.0 +0100
@@ -1990,11 +1990,11 @@
*/
if (udh != NULL (octstr_len(udh) != octstr_get_char(udh, 0) + 1)) {
returnerror = octstr_create(UDH field misformed, rejected);
-goto fielderror2;
+goto fielderror1;
}
if (udh != NULL octstr_len(udh) MAX_SMS_OCTETS) {
returnerror = octstr_create(UDH field is too long, rejected);
-goto fielderror2;
+goto fielderror1;
}
/*
@@ -2108,6 +2108,12 @@
del = gwlist_delete_matching(allowed, receiv, octstr_item_match);
}
+/* have all receivers been denied by list rules?! */
+if (gwlist_len(allowed) == 0) {
+returnerror = octstr_create(Number(s) has/have been denied by white- and/or black-lists.);
+goto fielderror2;
+}
+
if (urltrans_faked_sender(t) != NULL) {
/* discard previous from */
newfrom = octstr_duplicate(urltrans_faked_sender(t));
@@ -2144,7 +2150,7 @@
msg-sms.account = account ? octstr_duplicate(account) : NULL;
} else {
returnerror = octstr_create(Account field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
}
msg-sms.msgdata = text ? octstr_duplicate(text) : octstr_create();
@@ -2156,7 +2162,7 @@
if(octstr_len(dlr_url)) {
if(octstr_len(dlr_url) 8) { /* http(s):// */
returnerror = octstr_create(DLR-URL field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
} else {
Octstr *tmp;
tmp = octstr_copy(dlr_url, 0, 7);
@@ -2168,7 +2174,7 @@
if(octstr_case_compare(tmp, octstr_imm(https://;)) != 0) {
returnerror = octstr_create(DLR-URL field misformed, rejected);
O_DESTROY(tmp);
- goto fielderror;
+ goto fielderror3;
}
#ifdef HAVE_LIBSSL
msg-sms.dlr_url = octstr_duplicate(dlr_url);
@@ -2187,49 +2193,49 @@
if ( dlr_mask -1 || dlr_mask 31 ) { /* 0001 */
returnerror = octstr_create(DLR-Mask field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.dlr_mask = dlr_mask;
if ( mclass -1 || mclass 3 ) {
returnerror = octstr_create(MClass field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.mclass = mclass;
if ( pid -1 || pid 255 ) {
returnerror = octstr_create(PID field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.pid = pid;
if ( rpi -1 || rpi 2) {
returnerror = octstr_create(RPI field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.rpi = rpi;
if ( alt_dcs -1 || alt_dcs 1 ) {
returnerror = octstr_create(Alt-DCS field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.alt_dcs = alt_dcs;
if ( mwi -1 || mwi 7 ) {
returnerror = octstr_create(MWI field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.mwi = mwi;
if ( coding -1 || coding 2 ) {
returnerror = octstr_create(Coding field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.coding = coding;
if ( compress -1 || compress 1 ) {
returnerror = octstr_create(Compress field misformed, rejected);
- goto fielderror;
+ goto fielderror3;
}
msg-sms.compress = compress;
@@ -2244,19 +2250,19 @@
if ( validity -1 ) {
returnerror = octstr_create(Validity field
Re: Memory leak in smsbox
Stipe Tolj wrote:
Peter Christensen wrote:
diff -Nru gateway/gw/smsbox.c gateway.leak/gw/smsbox.c
--- gateway/gw/smsbox.c2005-12-09 03:14:31.0 +0100
+++ gateway.leak/gw/smsbox.c2006-01-20 21:30:57.0 +0100
@@ -2119,7 +2119,7 @@
newfrom = octstr_duplicate(global_sender);
} else {
returnerror = octstr_create(Sender missing and no global set,
rejected);
-goto fielderror2;
+goto fielderror3;
}
can we revise this to keep the present goto labels instead of
introducing new ones? The flow-control-logic in that function get's too
complicated.
info(0, sendsms sender:%s:%s (%s) to:%s msg:%s,
@@ -2365,6 +2365,11 @@
octstr_destroy(newfrom);
msg_destroy(msg);
+fielderror3:
+gwlist_destroy(allowed, NULL);
+gwlist_destroy(denied, NULL);
+gwlist_destroy(receiver, octstr_destroy_item);
+
why don't we gwlist_destroy([allowed|denied], octstr_destroy_item) too?
At this point some elements are present in both the allowed or denied
and the receiver list. Here's how I interpret the flow (reduced list):
1. At first, we have a list of recipients (receiver)
2. Empty allowed and denied lists are created
3. Each recipient is added to either allowed or denied depending on
blacklisting. (The lists contains only pointers to the Octstr, so all
pointers in allowed and denied are also present in receiver)
4. Every recipient both present in denied and allowed are removed from
allowed
5. Empty failed_id list is created
6. Each recipient is extracted from allowed and sent to bearerbox with
message. Failed recipients are added to failed_id list.
7. With each recipient removed from allowed, reipients are now only
present in both receiver and denied or receiver and failed_id at worst)
8. Message structure is freed
9. Receiver is freed with octstr_destroy_item
10. Now, every recipient in denied (if any) are actually invalid as they
have been freed.
11. Allowed is freed with octstr_destroy_item
12. Now, every recipient in failed_id (if any) are also invalid
13. If all recipients was denied (no_recv == gwlist_len(denied)),
terminate with error
14. If failed_id list is not empty, goto error (no. 18)
15. failed_id is freed with octstr_destroy_item (no need to use
octstr_destroy_item here, as failed_id is ALWAYS empty at this point. If
there WAS any element in the list, they would have been freed with
allowed was freed, and thus generate an error)
16. Each element in denied is extracted and appended to output (but at
this point, every string in the list is invalid. I foresee a
segmentation fault here)
17. denied is freed with octstr_destroy_item (Again, no need. Is always
empty at this point)
18. Terminate
19. Each failed recipient is extracted from failed_id and appended to
output (But every element is invalid and we'll get a segmentation fault)
20. failed_id is freed with octstr_destroy_item (but since list is
empty, octstr_destroy_item is unnecessary)
21. denied is freed with octstr_destroy_item (double free, as every
element have already been freed)
22. Terminate
Just as expected... Did a little test to prove whether my segfault
theory was accurate. I've added 234 to my black list:
http://localhost:13003/cgi-bin/sendsms?username=testuserpassword=testpassfrom=123to=123
smsbox says Sent. (denied and failed_id is empty)
http://localhost:13003/cgi-bin/sendsms?username=testuserpassword=testpassfrom=123to=123%20123
smsbox says Sent. (denied and failed_id is empty)
http://localhost:13003/cgi-bin/sendsms?username=testuserpassword=testpassfrom=123to=234
smsbox says Number(s) has/have been denied by white- and/or
black-lists. (denied have one item and failed_id is empty. Since ALL
recipients failed, they are not shown in output and we get no segfault.
In fact denied, failed_id, and newfrom are not freed. Slow, but
potential memory leak?)
http://localhost:13003/cgi-bin/sendsms?username=testuserpassword=testpassfrom=123to=123%20234
smsbox dies with no reply. (smsbox attempts to print out that 234 have
failed, but this string have been freed although it is present in the
denied list)
http://localhost:13003/cgi-bin/sendsms?username=testuserpassword=testpassfrom=123to=234%20234
smsbox dies with no reply. (Both recipients are rejected, but as they
are identical, no_recv is 2, but denied have only one element. Therefore
smsbox will fail as it attempts to return a list of denied recipients.
http://localhost:13003/cgi-bin/sendsms?username=testuserpassword=testpassfrom=123to=234%20123A
smsbox says Number(s) has/have been denied by white- and/or
black-lists. (At this point the two rejected numbers are NOT identical
and no_recv becomes gwlist_len(denied) and we won't crash)
So, smsbox definitely doesn't handle errors well when there are multiple
recipients.
fielderror2:
alog(send-SMS request failed - %s,
octstr_get_cstr(returnerror));
Stipe
Re: user guide
Hi, Alexander Malysh wrote: Hi, Peter Christensen schrieb: Hi, You're right, I probably should check the length before converting from GSM to latin1 (after all, the 11 char limit is in the GSM charset, not the latin1). SMPP_DEFAULT_CHARSET is not defined in alpha_gsm either, do you want me to define that (to windows-1252) so that you can use alpha_gsm without the cp1252 patch, or what was your intentions? Intention was really simply: If you just apply alpa_gsm patch w/o the other one than it just not compile because define is not defined. Attached patch defines and uses SMPP_DEFAULT_CHARSET. As for replacing @, it IS an ugly hack, but it is a necessary hack, but ONLY on SMPP gateways, and ONLY when using GSM charset in alphanumeric addresses (since SMPP uses zero-terminated strings. Although UCP also uses GSM, it is not a problem as it uses hexadecimals), so I think it is the right place to do it. I ded consider replacing @ with another printable character such as $ though, just to show that there should have been something. After all [EMAIL PROTECTED] would become ab.c and the recipient wouldn't be able to know where the @ ought to have been. how about replacing with '?' then? so at least you will see on handset that there was a char. Excellent idea. The new patch does exactly that. Anything else, now we're at it?! :p Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Thanks, Alex Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Alexander Malysh wrote: Hi, thanks for patches! Some notes: - please check addr length before any charset converts. Maybe I'm wrong there, opinions from others? - in a patch alpha_gsm patch your use SMPP_CHARSET that is not defined in this patch. Please rename it to something like SMPP_DEFAULT_CHARSET. - don't you think that this is a ugly hack to replace '@'? I don't think it belongs into smpp module. Opinions from others? Thanks, Alex Peter Christensen schrieb: Hi again, I've attached revised versions of the cp1252 patch and the alpha_gsm patch. Changes to cp1252 patch: * Change default character set from ISO-8859-1 to WINDOWS-1252 in user guide Changed to alpha_gsm patch: * Feature is controlled with alt-addr-charset. Possible values are gsm or any iconv charset. Default is windows-1252. * convert_addr_from_pdu() now converts FROM alt-addr-charset to windows-1252 and not the other way around (are used for incoming messages) * Added documentation of alt-addr-charset to user guide Just a side note: Kannel rejects messages with alphanumeric addresses longer than 11 characters, which have its origin in the GSM charset. Basically, when default assumed charset is windows-1252 (or previously iso-8859-1), the string test [test] is actually still invalid since [ and ] both takes up two septets in the GSM charset. Anyway, since kannel can now be configured to assume utf-8 or whatever, an alphanumeric address could actually be 44 characters long and still be valid (11 greek letters). Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Hi, I apologise for not waiting for your response. If you had not replied I would have assumed you were not going to submit the patch. Unfortunately from previous experience a developer was emailing the list and me about a patch, which got people excited, he then never completed it. This patch is fortunately exactly at the time I need it and for that I thank you. I was hoping you would follow this to completion, but if not then I needed to know. A suggested format: sender-charset = charset (The charset for the sender address) msg-charset = charset (The charset for the message) With the default current charsets as per Alex's email (sender: latin1, msg: gsm) Maybe others on the Kannel Devel list have better ideas. Rgds -Original Message- From: Peter Christensen [mailto:[EMAIL PROTECTED] Sent: 18 January 2006 15:38 To: [EMAIL PROTECTED] Cc: Kannel Devel Subject: Re: user guide Hi, Yes, I know that. No need to tell me that three times. But please be patient, although I WOULD like to have the work done quickly, I don't have an infinite amount of spare time, and some things are more urgent. I hope that I will have revised patches (including user guide documentation) done within this week, but I cannot promise anything. Btw., any suggestions to what I should call the alpha_gsm option, and exactly how it should work? One way would be addr-charset = charset In this way the user will be able to use practically any charset, but the consequence is that GSM charset becomes the default
Re: dlr_mysql_add and internal charset
Hi, Just wondering. What became the final conclusion on MySQL escaping? Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Andreas Fink wrote: On 10.01.2006, at 21:01, Stipe Tolj wrote: Peter Christensen wrote: Hi, I'd like to address a couple things now that a new kannel release near: 1. Some while ago, I reported problems within the dlr_mysql_add function. If the entry-timestamp, entry-source, or entry-url contains some unfortunate characters (most significantly '), the SQL query gets broken and the DLRs are wasted. After a while, the first patch was submitted, but as it used mysql_real_escape_string, it would potentially require an additional MySQL connection (or something - don't remember what the exact problem was), so it was not committed, and another patch was promised in a near future. Apparently this patch never came, however, and I see that the current CVS is still not escaping the strings. correct, I have that escaping version of mysql here and it's scheduled for commit to 1.5.0 devel, since I won't have the time to test it that extensively to ensure stability for 1.4.1 stable. how about taking a dumb and simple approach and simply escape all characters to hexadecimal (ie. \x30 for a 0). this will always work and not break anything existing. I could post the patch and let the list confirm via votes that it should or should not go to 1.4.1 stable? I think it should, its not a new feature but a important bugfix and not a critial to break anything in my eyes. Andreas Fink Fink Consulting GmbH --- Tel: +41-61-332 Fax: +41-61-331 Mobile: +41-79-2457333 Address: Clarastrasse 3, 4058 Basel, Switzerland E-Mail: [EMAIL PROTECTED] Homepage: http://www.finkconsulting.com --- ICQ: 101946485 MSN: [EMAIL PROTECTED] AIM: smsrelay Skype: andreasfink Yahoo: finkconsulting SMS: +41792457333 PGP9: 0714 DF2B A189 A760 6201 5CBD D040 3E71 4DAF 68BB
Memory leak in smsbox
Hi,
Earlier today I noticed a significant increase of memory consumption by
smsbox. Over a period of 5½ hour it had gone from around 1.8MB to
209MB. When I examined the log of smsbox, two messages were rejected by
smsbox due to bad character formatting, but a minute later the client
attempted to transfer both messages again and again etc.
Judging from this there must be a memory leak in smsbox which is only
triggered upon errors.
I went through some memory dumps and smsbox_req_handle and eventually
concluded that upon errors the allowed, denied, and receiver lists were
not freed.
I've attached a patch which will free the lists on errors in between the
creation and destruction of the lists.
As far as I could see, the elements within the three lists are actually
the same (that is, everything in allowed and denied are definitely also
in received), and since we have not extracted the elements from the
receiver yet, this patch only free the elements in the receiver list.
Not the allowed and denied lists. Is this assumption correct? One thing
is certain, freeing both receiver and allowed with gwlist_destroy(list,
octstr_destroy_item) ended in a PANIC since the first destruction freed
the elements in the other list.
A brute spamming with invalid messages made an unpatched newly started
kannel go from 1.3MB to 10MB in two seconds. The same thing with a
patched kannel went from 1.3MB to 1.5MB where it stayed (I assume that
the first bunch was first-time single-shot allocations). The memory
sizes are RSS.
Another unrelated thing... I've been bombing the mailing list with
patches and stuff the last few weeks, but what exactly is the preferred
way of getting patches? In the mailing list, or by creating a bug report
with an attached patch?
--
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
diff -Nru gateway/gw/smsbox.c gateway.leak/gw/smsbox.c
--- gateway/gw/smsbox.c 2005-12-09 03:14:31.0 +0100
+++ gateway.leak/gw/smsbox.c 2006-01-20 21:30:57.0 +0100
@@ -2119,7 +2119,7 @@
newfrom = octstr_duplicate(global_sender);
} else {
returnerror = octstr_create(Sender missing and no global set, rejected);
- goto fielderror2;
+ goto fielderror3;
}
info(0, sendsms sender:%s:%s (%s) to:%s msg:%s,
@@ -2365,6 +2365,11 @@
octstr_destroy(newfrom);
msg_destroy(msg);
+fielderror3:
+gwlist_destroy(allowed, NULL);
+gwlist_destroy(denied, NULL);
+gwlist_destroy(receiver, octstr_destroy_item);
+
fielderror2:
alog(send-SMS request failed - %s,
octstr_get_cstr(returnerror));
Re: SMPP reconnection on invalid username/password
Hi, Yes, I agree that the behavior is the ideal one. But I think it is confusing to the user that the log says Re-connecting and bearerbox reports the state of the gateway as reconnecting (When requesting status/status.xml/status.html), while it should actually be dead. Alexander Malysh wrote: Hi, yes it really intentional. if SMSC returns such error and your system_id/password are correct then it's SMSC bug! Imagine SMSC could not retrieve your login data for some reason it should never claim your system_id or password is wrong (for this purpose exists Bind Failed)! Why kannel has such behavior: because some SMSCs suspend (security issue) your account when you unintentionally supplied wrong system_id or password. And then you have to contact support and explain something etc. etc. It's just good behavior to not to abuse SMSC... Thanks, Alex Peter Christensen schrieb: Hi, I've been experiencing problems where kannel at some time no longer attempts to reconnect a gateway, which is down. Here's what basically happens (gateway name and address is replaced with fake name/address): ERROR: System error 111: Connection refused ERROR: error connecting to server `1.2.3.4' at port `2775' ERROR: SMPP[gateway]: Couldn't connect to server. ERROR: SMPP[gateway]: Couldn't connect to SMS center (retrying in 10 seconds). ERROR: connect failed ERROR: System error 111: Connection refused ERROR: error connecting to server `1.2.3.4' at port `2775' ERROR: SMPP[gateway]: Couldn't connect to server. ERROR: SMPP[gateway]: Couldn't connect to SMS center (retrying in 10 seconds). WARNING: SMPP: PDU NULL terminated string has no NULL. ERROR: SMPP[gateway]: SMSC rejected login to transmit, code 0x000f (Invalid System ID). ERROR: SMPP[gateway]: I/O error or other error. Re-connecting. Although kannel claim to be reconnect when the login was invalid, this never occurs. The thread terminates, but bearerbox still report the state of the gateway as reconnecting. When examining the code, it is very obvious that the SMPP gateway code is meant to terminate when login is rejected: gateway/gw/smsc/smsc_smpp.c:1468 if (pdu-u.bind_transmitter_resp.command_status == SMPP_ESME_RINVSYSID || pdu-u.bind_transmitter_resp.command_status == SMPP_ESME_RINVPASWD) smpp-quitting = 1; gateway/gw/smsc/smsc_smpp.c:1488 if (pdu-u.bind_transceiver_resp.command_status == SMPP_ESME_RINVSYSID || pdu-u.bind_transceiver_resp.command_status == SMPP_ESME_RINVPASWD) smpp-quitting = 1; Is this really intentional? And if so, isn't it then wrong to say claim that kannel will reconnect? Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk
Re: user guide
Hi, You're right, I probably should check the length before converting from GSM to latin1 (after all, the 11 char limit is in the GSM charset, not the latin1). SMPP_DEFAULT_CHARSET is not defined in alpha_gsm either, do you want me to define that (to windows-1252) so that you can use alpha_gsm without the cp1252 patch, or what was your intentions? As for replacing @, it IS an ugly hack, but it is a necessary hack, but ONLY on SMPP gateways, and ONLY when using GSM charset in alphanumeric addresses (since SMPP uses zero-terminated strings. Although UCP also uses GSM, it is not a problem as it uses hexadecimals), so I think it is the right place to do it. I ded consider replacing @ with another printable character such as $ though, just to show that there should have been something. After all [EMAIL PROTECTED] would become ab.c and the recipient wouldn't be able to know where the @ ought to have been. Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Alexander Malysh wrote: Hi, thanks for patches! Some notes: - please check addr length before any charset converts. Maybe I'm wrong there, opinions from others? - in a patch alpha_gsm patch your use SMPP_CHARSET that is not defined in this patch. Please rename it to something like SMPP_DEFAULT_CHARSET. - don't you think that this is a ugly hack to replace '@'? I don't think it belongs into smpp module. Opinions from others? Thanks, Alex Peter Christensen schrieb: Hi again, I've attached revised versions of the cp1252 patch and the alpha_gsm patch. Changes to cp1252 patch: * Change default character set from ISO-8859-1 to WINDOWS-1252 in user guide Changed to alpha_gsm patch: * Feature is controlled with alt-addr-charset. Possible values are gsm or any iconv charset. Default is windows-1252. * convert_addr_from_pdu() now converts FROM alt-addr-charset to windows-1252 and not the other way around (are used for incoming messages) * Added documentation of alt-addr-charset to user guide Just a side note: Kannel rejects messages with alphanumeric addresses longer than 11 characters, which have its origin in the GSM charset. Basically, when default assumed charset is windows-1252 (or previously iso-8859-1), the string test [test] is actually still invalid since [ and ] both takes up two septets in the GSM charset. Anyway, since kannel can now be configured to assume utf-8 or whatever, an alphanumeric address could actually be 44 characters long and still be valid (11 greek letters). Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Hi, I apologise for not waiting for your response. If you had not replied I would have assumed you were not going to submit the patch. Unfortunately from previous experience a developer was emailing the list and me about a patch, which got people excited, he then never completed it. This patch is fortunately exactly at the time I need it and for that I thank you. I was hoping you would follow this to completion, but if not then I needed to know. A suggested format: sender-charset = charset (The charset for the sender address) msg-charset = charset (The charset for the message) With the default current charsets as per Alex's email (sender: latin1, msg: gsm) Maybe others on the Kannel Devel list have better ideas. Rgds -Original Message- From: Peter Christensen [mailto:[EMAIL PROTECTED] Sent: 18 January 2006 15:38 To: [EMAIL PROTECTED] Cc: Kannel Devel Subject: Re: user guide Hi, Yes, I know that. No need to tell me that three times. But please be patient, although I WOULD like to have the work done quickly, I don't have an infinite amount of spare time, and some things are more urgent. I hope that I will have revised patches (including user guide documentation) done within this week, but I cannot promise anything. Btw., any suggestions to what I should call the alpha_gsm option, and exactly how it should work? One way would be addr-charset = charset In this way the user will be able to use practically any charset, but the consequence is that GSM charset becomes the default, which might not be desirable. Alternatively the option could be boolean: addr-gsm = boolean Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Hi, As per email from Alexander Malysh [EMAIL PROTECTED] Date: Fri, 13 Jan 2006 17:24:01 +0100 From: Alexander Malysh [EMAIL PROTECTED] Subject: Re: please reconsider the +0 you gave the patches Both your patches kannel.smpp.cp1252.diff and kannel.smpp.alpha_gsm.diff.have been accepted with added configurable parameters for the body source_addr
Re: SMPP reconnection on invalid username/password
Hi, Apparently fixing that was quite easy. When the SMPP thread escapes the while (!smpp-quitting) loop, I set state to SMSCCONN_DEAD Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Peter Christensen wrote: Hi, Yes, I agree that the behavior is the ideal one. But I think it is confusing to the user that the log says Re-connecting and bearerbox reports the state of the gateway as reconnecting (When requesting status/status.xml/status.html), while it should actually be dead. Alexander Malysh wrote: Hi, yes it really intentional. if SMSC returns such error and your system_id/password are correct then it's SMSC bug! Imagine SMSC could not retrieve your login data for some reason it should never claim your system_id or password is wrong (for this purpose exists Bind Failed)! Why kannel has such behavior: because some SMSCs suspend (security issue) your account when you unintentionally supplied wrong system_id or password. And then you have to contact support and explain something etc. etc. It's just good behavior to not to abuse SMSC... Thanks, Alex Peter Christensen schrieb: Hi, I've been experiencing problems where kannel at some time no longer attempts to reconnect a gateway, which is down. Here's what basically happens (gateway name and address is replaced with fake name/address): ERROR: System error 111: Connection refused ERROR: error connecting to server `1.2.3.4' at port `2775' ERROR: SMPP[gateway]: Couldn't connect to server. ERROR: SMPP[gateway]: Couldn't connect to SMS center (retrying in 10 seconds). ERROR: connect failed ERROR: System error 111: Connection refused ERROR: error connecting to server `1.2.3.4' at port `2775' ERROR: SMPP[gateway]: Couldn't connect to server. ERROR: SMPP[gateway]: Couldn't connect to SMS center (retrying in 10 seconds). WARNING: SMPP: PDU NULL terminated string has no NULL. ERROR: SMPP[gateway]: SMSC rejected login to transmit, code 0x000f (Invalid System ID). ERROR: SMPP[gateway]: I/O error or other error. Re-connecting. Although kannel claim to be reconnect when the login was invalid, this never occurs. The thread terminates, but bearerbox still report the state of the gateway as reconnecting. When examining the code, it is very obvious that the SMPP gateway code is meant to terminate when login is rejected: gateway/gw/smsc/smsc_smpp.c:1468 if (pdu-u.bind_transmitter_resp.command_status == SMPP_ESME_RINVSYSID || pdu-u.bind_transmitter_resp.command_status == SMPP_ESME_RINVPASWD) smpp-quitting = 1; gateway/gw/smsc/smsc_smpp.c:1488 if (pdu-u.bind_transceiver_resp.command_status == SMPP_ESME_RINVSYSID || pdu-u.bind_transceiver_resp.command_status == SMPP_ESME_RINVPASWD) smpp-quitting = 1; Is this really intentional? And if so, isn't it then wrong to say claim that kannel will reconnect? Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk diff -Nru gateway/gw/smsc/smsc_smpp.c gateway.dead/gw/smsc/smsc_smpp.c --- gateway/gw/smsc/smsc_smpp.c 2005-12-18 21:21:16.0 +0100 +++ gateway.dead/gw/smsc/smsc_smpp.c 2006-01-19 10:48:50.0 +0100 @@ -1840,6 +1840,8 @@ gwthread_sleep(smpp-conn-reconnect_delay); } } + +smpp-conn-status = SMSCCONN_DEAD; }
Re: SMPP reconnection on invalid username/password
I've done exactly that too :D Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Benjamin Lee wrote: For what it's worth... this behaviour occurs on an SMSC operated by a large entity that sounds like Saferway wink and also has another name with *365 in it. ;-P We basically wrap a cron script around the cgi/status of smsbox. And then ?stop and ?start the smsc=your_smsc when the connection is detected as dead.
Re: user guide
Hi, Yes, I know that. No need to tell me that three times. But please be patient, although I WOULD like to have the work done quickly, I don't have an infinite amount of spare time, and some things are more urgent. I hope that I will have revised patches (including user guide documentation) done within this week, but I cannot promise anything. Btw., any suggestions to what I should call the alpha_gsm option, and exactly how it should work? One way would be addr-charset = charset In this way the user will be able to use practically any charset, but the consequence is that GSM charset becomes the default, which might not be desirable. Alternatively the option could be boolean: addr-gsm = boolean Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Hi, As per email from Alexander Malysh [EMAIL PROTECTED] Date: Fri, 13 Jan 2006 17:24:01 +0100 From: Alexander Malysh [EMAIL PROTECTED] Subject: Re: please reconsider the +0 you gave the patches Both your patches kannel.smpp.cp1252.diff and kannel.smpp.alpha_gsm.diff.have been accepted with added configurable parameters for the body source_addr independently. Please when you submit your patch, which I'd like to start using, will you send through some details to the user guide on which char set corresponds to which configurable option for the body and source independently and what options to set to send the @, pound,Euro and ¡ symbols. Thanks for adding these patches. Rgds
Re: user guide
Hi again, I've attached revised versions of the cp1252 patch and the alpha_gsm patch. Changes to cp1252 patch: * Change default character set from ISO-8859-1 to WINDOWS-1252 in user guide Changed to alpha_gsm patch: * Feature is controlled with alt-addr-charset. Possible values are gsm or any iconv charset. Default is windows-1252. * convert_addr_from_pdu() now converts FROM alt-addr-charset to windows-1252 and not the other way around (are used for incoming messages) * Added documentation of alt-addr-charset to user guide Just a side note: Kannel rejects messages with alphanumeric addresses longer than 11 characters, which have its origin in the GSM charset. Basically, when default assumed charset is windows-1252 (or previously iso-8859-1), the string test [test] is actually still invalid since [ and ] both takes up two septets in the GSM charset. Anyway, since kannel can now be configured to assume utf-8 or whatever, an alphanumeric address could actually be 44 characters long and still be valid (11 greek letters). Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Hi, I apologise for not waiting for your response. If you had not replied I would have assumed you were not going to submit the patch. Unfortunately from previous experience a developer was emailing the list and me about a patch, which got people excited, he then never completed it. This patch is fortunately exactly at the time I need it and for that I thank you. I was hoping you would follow this to completion, but if not then I needed to know. A suggested format: sender-charset = charset (The charset for the sender address) msg-charset = charset (The charset for the message) With the default current charsets as per Alex's email (sender: latin1, msg: gsm) Maybe others on the Kannel Devel list have better ideas. Rgds -Original Message- From: Peter Christensen [mailto:[EMAIL PROTECTED] Sent: 18 January 2006 15:38 To: [EMAIL PROTECTED] Cc: Kannel Devel Subject: Re: user guide Hi, Yes, I know that. No need to tell me that three times. But please be patient, although I WOULD like to have the work done quickly, I don't have an infinite amount of spare time, and some things are more urgent. I hope that I will have revised patches (including user guide documentation) done within this week, but I cannot promise anything. Btw., any suggestions to what I should call the alpha_gsm option, and exactly how it should work? One way would be addr-charset = charset In this way the user will be able to use practically any charset, but the consequence is that GSM charset becomes the default, which might not be desirable. Alternatively the option could be boolean: addr-gsm = boolean Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Hi, As per email from Alexander Malysh [EMAIL PROTECTED] Date: Fri, 13 Jan 2006 17:24:01 +0100 From: Alexander Malysh [EMAIL PROTECTED] Subject: Re: please reconsider the +0 you gave the patches Both your patches kannel.smpp.cp1252.diff and kannel.smpp.alpha_gsm.diff.have been accepted with added configurable parameters for the body source_addr independently. Please when you submit your patch, which I'd like to start using, will you send through some details to the user guide on which char set corresponds to which configurable option for the body and source independently and what options to set to send the @, pound,Euro and ¡ symbols. Thanks for adding these patches. Rgds diff -Nru gateway.cp1252/doc/userguide/userguide.xml gateway.alpha_gsm/doc/userguide/userguide.xml --- gateway.cp1252/doc/userguide/userguide.xml 2006-01-18 17:39:36.0 +0100 +++ gateway.alpha_gsm/doc/userguide/userguide.xml 2006-01-18 17:39:50.0 +0100 @@ -3177,6 +3177,17 @@ should be presented for this directive. /entry/row + rowentryliteralalt-addr-charset/literal/entry +entryliteralstring/literal/entry +entry valign=bottom +Defines which character encoding is used for alphanumeric addresses. +When set to literalGSM/literal, addresses are converted into the +GSM 03.38 charset (Since @ is translated into 0x00 which will break + the SMPP PDU, @ are stripped from addresses). If set to another value, +literaliconv()/literal is used. +(Defaults to latin1) +/entry/row + rowentryliteralconnection-timeout/literal/entry entryliteralnumber (seconds)/literal/entry entry valign=bottom diff -Nru gateway.cp1252/gw/smsc/smsc_smpp.c gateway.alpha_gsm/gw/smsc/smsc_smpp.c --- gateway.cp1252/gw/smsc/smsc_smpp.c 2006-01-18 17:39:36.0 +0100 +++ gateway.alpha_gsm/gw/smsc/smsc_smpp.c 2006-01-18 17:59:47.0
Re: please reconsider the +0 you gave the patches
Hi, I find it odd that you have seen no SMSC which required that option. I personally tried transmitting a¡b as originator over 7 different gateways with the patch on. 5 of them assumed GSM in both source_addr and short_message, and the originator stayed a¡b (The actual transmitted source_addr was [EMAIL PROTECTED]). One required iso-8859-1 in both source_addr and short_message and the originator became a¡b (Transfered as a¡b as alt_charset=iso-8859-1). The last one required iso-8859-1 in source_addr and GSM in short_message and the resulting originator was [EMAIL PROTECTED]. So yes, it ought to be configurable (although I hear complaints of too many configuration options) since it clearly differ from SMSC to SMSC, but I am really surprised that I magically have found all the gateways which DID expect GSM charset. I work in a Danish company, and so several of our customers are Scandinavian and would like to have the letters Æ,Ø,Å in the source_addr, and without the patch, these characters are replaced with spaces or something random by the SMSC. I took a closer look at the actual SMPP spec, and they stated that C-octet-strings can only have ASCII characters, so if the protocol is followed exactly as written, Scandinavia have lost :( If I create a new patch where the option i selectable, would it then have a better chance of entering the CVS, or shouldn't I bother at all? Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Alexander Malysh wrote: Hi, sorry for delay... I'm -1 for the kannel.smpp.apha_gsm.patch in it form as is. The fact is really simple. With the patch in it's form it's not possible to restore default behavior (sender: latin1, msg: gsm) because it's depend on alt_charset that also used for message body. And as I don't know any SMSC that need this feature I would say: default behavior must stay. Thanks, Alex Hillel wrote: Dear Alex, Please if the diffs from Peter are in the correct format will you consider installing into CVS the kannel.smpp.cp1252.diff and kannel.smpp.alpha_gsm.diff. I've been emailing Peter and he explained the two patches gives support for € in both alphanumeric numbers as well as the actual message body. Its also helps to send many other characters. This is very useful. Please re-consider your +0 you gave the patch. Thanks
Re: dlr_mysql_add and internal charset
;
+ +octstr_insert(pdu-u.submit_sm.short_message,
udh_septets, 0);
+
+if (((octstr_len(pdu-u.submit_sm.short_message) + 1)
7) == 0) {
+/* message length will result in an @ in the end.
Add a space instead */
+octstr_append_char(pdu-u.submit_sm.short_message, '
');
+}
+
+octstr_destroy(udh_septets);
+message =
smpp_pack_septets(pdu-u.submit_sm.short_message);
+
+octstr_destroy(pdu-u.submit_sm.short_message);
+pdu-u.submit_sm.short_message = message;
+}
+else
+octstr_insert(pdu-u.submit_sm.short_message,
msg-sms.udhdata, 0);
}
pdu-u.submit_sm.sm_length =
octstr_len(pdu-u.submit_sm.short_message);
@@ -1937,6 +1998,7 @@
int autodetect_addr;
Octstr *alt_charset;
long connection_timeout, wait_ack, wait_ack_action;
+int pack_udh;
my_number = alt_charset = NULL;
transceiver_mode = 0;
@@ -2021,6 +2083,9 @@
if (cfg_get_bool(autodetect_addr, grp,
octstr_imm(source-addr-autodetect)) == -1)
autodetect_addr = 1; /* default is autodetect if no option
defined */
+if (cfg_get_bool(pack_udh, grp, octstr_imm(pack-udh)) == -1)
+pack_udh = 0;
+
/* check for any specified interface version */
if (cfg_get_integer(version, grp,
octstr_imm(interface-version)) == -1)
version = SMPP_DEFAULT_VERSION;
@@ -2074,7 +2139,7 @@
dest_addr_npi, enquire_link_interval,
max_pending_submits, version, priority,
validity, my_number, smpp_msg_id_type,
autodetect_addr, alt_charset, - service_type,
connection_timeout, wait_ack, wait_ack_action);
+ service_type, connection_timeout, wait_ack,
wait_ack_action, pack_udh);conn-data = smpp; conn-name
= octstr_format(SMPP:%S:%d/%d:%S:%S, diff -Nru
gateway/gwlib/cfg.def gateway.new/gwlib/cfg.def
--- gateway/gwlib/cfg.def2005-12-13 01:11:04.0 +0100
+++ gateway.new/gwlib/cfg.def2006-01-10 16:24:59.0 +0100
@@ -365,6 +365,7 @@
OCTSTR(denied-prefix-regex)
OCTSTR(preferred-prefix-regex)
OCTSTR(max-error-count)
+OCTSTR(pack-udh)
)
hmm, I never heard about a need to do udh packing. SMSC buggy? It
would be ok to pack whole message + udh but only udh???
as long as I'm not convinienced that SMPP spec require it: -1.
yep, agree'ing to Alex here.
@Peter: can you point out in detail why this is needed? Maybe a
practical example should enlight us?
Stipe
The operator which require this behavior is Cellebrum, if I remember
correct, but NowSMS expect this behavior by default (but users can
disable it). What it does is really transmitting the text in raw binary
form when UDH is present.
Consider the UDH
06 00 03 56 02 02 (UDHL=6, last part of a concatenated SMS with ref. 0x56)
and message body:
Test msg (54 65 73 74 20 6D 73 67)
In binary form, split into septets, the body will look like:
06 00 0C 30 25 40 00 (this is the UDH) 54 65 73 74 20 6D 73 67
If we pack this into octets, we get:
06 00 03 56 02 02 A8 E5 39 1D D4 9E 9F 01
which will then be transmitted to the operator. The operator is now
forced to guess the actual length in septets based on the length in
octets, and in this particular example, the operator will assume 9
septets instead of the correct 8 (14*8/7 = 16 - 7 (udh) = 9).
The mobile will decode the message as:
Test msg@ 54 65 73 74 20 6D 73 67 00)
So to avoid this (unless actually intentional), the message is padded
with a space in case the total length of UDH + binary encoded message is
unfortunate. So the patch would transmit
06 00 03 56 02 02 A8 E5 39 1D D4 9E 9F 61
which is decoded into
Test msg (54 65 73 74 20 6D 73 67 20)
I very much hope that gateways of this sort is very rare, but I posted
the patch anyway. My vote for CVS commit is very much -1 (even though
the feature is selectable through configuration)
--
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
Re: dlr_mysql_add and internal charset
Hi,
Revised patches.
kannel.mysql.escape2.diff:
All relevant queries are escaped now and indents are changed to 4
spaces instead of tabs.
kannel.smpp.cp1252.diff:
Indents are changed to 4 spaces instead of tabs
kannel.smpp.alpha_gsm.diff:
Indents are changed to 4 spaces instead of tabs. This patch require
that kannel.smpp.cp1252.diff is already present. (The patch converts
source_addr to GSM unless alt_charset is set. Got no response to the
original patch, but here is a revised version anyway)
As for mBlox and pack_udh patches, I didn't expect you to commit it to
the CVS anyway. I just thought that I would attach every part of my
patch, which was not from the original mBlox patch, in case anyone would
be interested. I believe that is was Cellebrum or something, which
required that we packed the UDH or something. AFAIK NowSMS even have the
feature set by default.
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
Alexander Malysh wrote:
Hi Peter,
please find comments bellow...
Thanks,
Alex
Peter Christensen wrote:
Hi again,
Here's the lot split into logical pieces (and synchronized with todays
CVS).
kannel.mysql.escape.diff:
* Escapes strings in dlr_mysql_add INSERT query.
kannel.smpp.cp1252.diff:
* Changed internal non-unicode character set to CP1252 in smsbox and
SMPP
kannel.smpp.errors.diff:
* Added recognition of all possible SMPP errors (including SMPPv5)
kannel.mblox.errors.diff:
* Added recognition of all mBlox SMPP errors
kannel.smpp.pack_udh.diff
* Added pack_udh option to SMPP. When set, Message is packed with the
UDH into binary data when sent.
Everything but the pack_udh patch have been thoroughly tested with
millions of messages for seveal months. pack_udh have only been tested
with some 100-200 different messages.
Med venlig hilsen / Best regards
Peter Christensen
Developer
--
Cool Systems ApS
Tel: +45 2888 1600
@ : [EMAIL PROTECTED]
www: www.coolsystems.dk
[snip]
diff -Nru gateway/gw/dlr_mysql.c gateway.new/gw/dlr_mysql.c
--- gateway/gw/dlr_mysql.c2006-01-10 13:53:29.0 +0100
+++ gateway.new/gw/dlr_mysql.c2006-01-10 16:05:29.0 +0100
@@ -140,9 +140,38 @@
dlr_db_fields_destroy(fields);
}
+char *_mysql_escape_string(const char *string) {
please fix coding style:
funct_name(...)
{
...
}
+unsigned long int size, length;
+char *buffer;
+length = strlen(string);
+size = (strlen(string) 1) + 1;
+if ((buffer = (char *)gw_malloc(size)) == NULL) {
+error(0, MYSQL: Error allocating buffer for string);
+return NULL;
+}
+else {
+mysql_escape_string(buffer, string, length);
+return buffer;
+}
+}
+
please fix coding style: indentation are 4 spaces.
static void dlr_mysql_add(struct dlr_entry *entry)
{
Octstr *sql;
+char *safe_ts, *safe_src, *safe_url;
+
+if ((safe_ts =
_mysql_escape_string(octstr_get_cstr(entry-timestamp))) == NULL)
+return;
+if ((safe_src =
_mysql_escape_string(octstr_get_cstr(entry-source))) == NULL) {
+gw_free(safe_ts);
+return;
+}
+if ((safe_url =
_mysql_escape_string(octstr_get_cstr(entry-url))) == NULL) {
+gw_free(safe_ts);
+gw_free(safe_src);
+return;
+}
+
indentation...
sql = octstr_format(INSERT INTO %s (%s, %s, %s, %s, %s, %s, %s,
%s, %s) VALUES
('%s', '%s', '%s', '%s', '%s', '%s', '%d',
'%s', '%d');,
@@ -152,10 +181,14 @@
octstr_get_cstr(fields-field_serv),
octstr_get_cstr(fields-field_url),
octstr_get_cstr(fields-field_mask),
octstr_get_cstr(fields-field_boxc),
octstr_get_cstr(fields-field_status),
-octstr_get_cstr(entry-smsc),
octstr_get_cstr(entry-timestamp), octstr_get_cstr(entry-source),
-octstr_get_cstr(entry-destination),
octstr_get_cstr(entry-service), octstr_get_cstr(entry-url),
+octstr_get_cstr(entry-smsc), safe_ts, safe_src,
+octstr_get_cstr(entry-destination),
octstr_get_cstr(entry-service), safe_url,
entry-mask, octstr_get_cstr(entry-boxc_id),
0);
+gw_free(safe_ts);
+gw_free(safe_src);
+gw_free(safe_url);
+
ditto
mysql_update(sql);
patch looks good. Please fix coding style and it could go into cvs.
Ahh I think you missing update/select case. Unescaped values could also
break update/select statements.
diff -Nru gateway/gw/smsbox.c gateway.new/gw/smsbox.c
--- gateway/gw/smsbox.c2005-12-09 03:14:31.0 +0100
+++ gateway.new/gw/smsbox.c2006-01-10 15:57
dlr_mysql_add and internal charset
Hi, I'd like to address a couple things now that a new kannel release near: 1. Some while ago, I reported problems within the dlr_mysql_add function. If the entry-timestamp, entry-source, or entry-url contains some unfortunate characters (most significantly '), the SQL query gets broken and the DLRs are wasted. After a while, the first patch was submitted, but as it used mysql_real_escape_string, it would potentially require an additional MySQL connection (or something - don't remember what the exact problem was), so it was not committed, and another patch was promised in a near future. Apparently this patch never came, however, and I see that the current CVS is still not escaping the strings. So, question is: What is the cause of the delay, and are you interested in my own patch, which uses mysql_escape_string instead of mysql_real_escape_string? I admit that problems related to this is somewhat rare (most likely cause for errors would be an SMS with ' in the originator string), but since it can be fixed relatively easy, I see no reason why NOT to do it. Although mysql_escape_string does not look at the character set (unlike mysql_real_escape_string), I still believe that it will be better to use it than to do nothing. 2. I've noticed that the charset_gsm_to_latin1 and charset_latin1_to_gsm functions actually uses windows-1252 as character set, but at least the SMPP gateway uses iso-8859-1 internally, which practically removes support for the € (euro) sign. This is probably also a problem within other gateways. If interesseted, I can supply a patch. But in relation to that, there is one thing which have begun to annoy me. By restricting to the windows-1252 character set when transmitting with the GSM character set, you remove support for the 10 Greek characters which is support by GSM. And as it always is when something is potentially possible, there will always be some large annoying customer who wants support for that particular feature. This last bit is only an observation and nothing more. I realize that fixing this (preferable by using UTF-8 or UCS-2BE as internal character set regardless of the output character set), would require a significantly amount of recoding within almost every part of the kannel software package. The reason while I have not attached any patches to this mail (although I have made quite a few changes), is that my own patch is really a combination of several patches, which is not in sync with the CVS. But if people are interested, I will update the patch and post the relevant bits. The whole patch does the following: SMPP: * Add support for mBlox operator and billing identifier (Not my own work) * Add support for ALL SMPPv5 error codes, including mBlox specific codes. (That is, recognize them and translate to human readable text) * Use CP1252 instead of ISO-8859-1 as internal charset * Added pack_udh parameter. When set, messages are sent as packed GSM data when UDH is present. (A few gateways require this) MySQL: * Escapes strings in dlr_mysql_add run_kannel_box: * Added waitpid after kill to avoid false terminations (the init scripts reports that kannel is terminated, while bearerbox is actually stuck within a connect call. - happens relatively often). With later CVS releases this didn't seem to work, so instead I've added some functionaly to the init script instead. -- Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk
Re: [Kannel-Devel] Re: dlr_mysql_add and internal charset
Sure thing, this is also why I made one for the actual official SMPP errors and one for the mBlox errors, in case anyone should be interested. The mBlox patches I've seen around did not add recognition of the errors. Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Peter Beckman wrote: On Tue, 10 Jan 2006, Peter Christensen wrote: kannel.mblox.errors.diff: * Added recognition of all mBlox SMPP errors It was my understanding that we weren't going to add any aggregator-specific code to kannel? I had done some exploratory work to figure out how we could add optional parameters to the config file to accept and handle them, but never finished. * Add support for ALL SMPPv5 error codes, including mBlox specific codes. (That is, recognize them and translate to human readable text) While I think this is valuable to mBlox customers, I don't think it applies to the kannel world as a whole, and should be left out of the code. I do think, however, that there should be a place to upload aggregator specific patches on kannel.org. Beckman --- Peter Beckman Internet Guy [EMAIL PROTECTED] http://www.purplecow.com/ ---
Re: Help to understand the encoding types
Hi Hillel, 1) coding=1 is for binary messages. Neither GSM nor unicode. 2) With coding=1, charset should have no effect, since the message is binary data (would be quite disastrous if Kannel parsed the binary data as character data and did some charset conversions) 3) With alt-dcs=1, the DCS field in the SMS is encoded as 00cmffmm (binary), where c is compression, m is message class and f is message coding (GSM/UCS2/8-bit data). With alt-dcs=2, DCS is encoded as 0fmm, which will always set a message class and can only choose between GSM charset and 8-bit data. At least, this is how I have understood the setting. Check out GSM TS 03.38 for more info. -- Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Dear Kannel Developers. From the user guide: http://kannel.org/download/kannel-userguide-snapshot/userguide.html#AEN4179 I'd like learn to send these coding types using Kannel. Any help on understanding the spec correctly appreciated. I sent this to the users list, but have not yet received any response. 1)Is this correct? For GSM alphabet which has coding of 7bits - use coding = 0 charset = ISO-8859-1 For ISO-8859-1 (Latin1) which has coding of 7bits - use coding = 0 charset = ISO-8859-1 For Unicode which has coding of 8bits - use coding = 1 charset = UTF-16BE For UCS2 which has coding UCS-2 - use coding = 2 charset = UTF-16BE 2)For ringtones or Logos is this correct? set udh for the ringtone or logo and coding is set to 8bits - use coding = 1 charset = UTF-16BE 3)When do you use alt-dcs? When do you use FX and 0X? From the User Guide Optional. If unset, Kannel uses the alt-dcs defined on smsc configuration, or 0X per default. If equals to 1, uses FX. If equals to 0, force 0X. Wishing the Kannel developers list seasons greetings and a prosperous 2006. I'd also like to thank the Kannel developers for giving the Kannel community such a great product. Rgds
Re: Help to understand the encoding types
Oh - my bad. alt-dcs=1 was 0fmm and alt-dcs=2 was 00cmffmm. Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Peter Christensen wrote: Hi Hillel, 1) coding=1 is for binary messages. Neither GSM nor unicode. 2) With coding=1, charset should have no effect, since the message is binary data (would be quite disastrous if Kannel parsed the binary data as character data and did some charset conversions) 3) With alt-dcs=1, the DCS field in the SMS is encoded as 00cmffmm (binary), where c is compression, m is message class and f is message coding (GSM/UCS2/8-bit data). With alt-dcs=2, DCS is encoded as 0fmm, which will always set a message class and can only choose between GSM charset and 8-bit data. At least, this is how I have understood the setting. Check out GSM TS 03.38 for more info. -- Med venlig hilsen / Best regards Peter Christensen Developer -- Cool Systems ApS Tel: +45 2888 1600 @ : [EMAIL PROTECTED] www: www.coolsystems.dk Hillel wrote: Dear Kannel Developers. From the user guide: http://kannel.org/download/kannel-userguide-snapshot/userguide.html#AEN4179 I'd like learn to send these coding types using Kannel. Any help on understanding the spec correctly appreciated. I sent this to the users list, but have not yet received any response. 1)Is this correct? For GSM alphabet which has coding of 7bits - use coding = 0 charset = ISO-8859-1 For ISO-8859-1 (Latin1) which has coding of 7bits - use coding = 0 charset = ISO-8859-1 For Unicode which has coding of 8bits - use coding = 1 charset = UTF-16BE For UCS2 which has coding UCS-2 - use coding = 2 charset = UTF-16BE 2)For ringtones or Logos is this correct? set udh for the ringtone or logo and coding is set to 8bits - use coding = 1 charset = UTF-16BE 3)When do you use alt-dcs? When do you use FX and 0X? From the User Guide Optional. If unset, Kannel uses the alt-dcs defined on smsc configuration, or 0X per default. If equals to 1, uses FX. If equals to 0, force 0X. Wishing the Kannel developers list seasons greetings and a prosperous 2006. I'd also like to thank the Kannel developers for giving the Kannel community such a great product. Rgds
Re: SMPP Maximum Password Length
Hi, What the SMPP 3.4 specs say is that it is a C-Octet string with a maximum of 9 octets. Also in the specs are defined that a C-Octet string may consists of any ASCII character and is terminated by a NULL terminator. This is also why almost all C-Octet String lengths limitations are odd numbers. Quote from the spec: In the case of all C-Octet String formats, the maximum field size is shown as a combination of string length and the NULL terminator, i.e., an 8-character C-Octet String is encoded in 9 octets when the NULL terminator is included. In other words, passwords longer than 8 characters are not suitable for SMPP. -- Peter Christensen M Amedeo Alaimo wrote: Hello, I have a question about Kannel and the smsc-password length The smpp v3.4 states the maximum length is 9 characters. The password i was supplied from our provider is 9 characters. I put a line similar to this in my kannel.conf - smsc-username = onetwothr smsc-password = onetwothr And then i get this error message when starting the bearerbox [27034] [6] WARNING: SMPP: PDU element password too long (length is 9, should be 8) 2005-12-19 14:27:07 [27034] [6] WARNING: SMPP: PDU NULL terminated string has no NULL. 2005-12-19 14:27:07 [27034] [6] ERROR: SMPP[SMPP:host:8080/700:onetwothr:]: SMSC rejected login to transmit, code 0x000e (Invalid Password). I was wondering if I am doing something wrong, or is Kannel's maximum password length 8? Also I am using today's cvs version. Thanks a Bunch! M Amedeo Alaimo
Re: [Fwd: [Kannel 0000278]: cannot build kannel with mysql support]
Hi,
Personally I have to configure kannel with:
CFLAGS=-I/usr/include/mysql ./configure --with-mysql --with-mysql-dir=/usr
to get the desired result. Have you tried that, Rene?
--
Peter Christensen
Stipe Tolj wrote:
Rene Kluwen wrote:
Rene Kluwen wrote:
This bug can be solved by applying the sqlbox patch.
??? how did you know that?,
Because I am smart :)...
And also because I always have a problem compiling mysql on a clean
checkout.
now, agree'ing to both ;)
What are the specific problems for compiling mysql support, can you guys
please profile bugs reports, so we have a track on this.
Thanks.
Stipe
mailto:stolj_{at}_wapme-group.de
---
Wapme Systems AG
Vogelsanger Weg 80
40470 Düsseldorf, NRW, Germany
phone: +49.211.74845.0
fax: +49.211.74845.299
mailto:info_{at}_wapme-systems.de
http://www.wapme-systems.de/
---
Re: How to pickup that you have run out of credits
Hi, Just for the record - IF kannel did call the dlr-url, the state of the message would be NACK/xxx where xxx is the error code (as a decimal number, if I remember correct) Best regards Peter Christensen Andreas Fink wrote: On 30.11.2005, at 15:55, Hillel wrote: Hi, Thanks for your response. Some SMSC providers indicate you have run out of credits by giving back in the bearerbox.log: 2005-11-28 23:19:10 [27849] [6] ERROR: SMPP[smsc1]: SMSC returned error code 0x0058 (Throttling error) in response to submit_sm. 2005-11-28 23:19:10 [27849] [6] DEBUG: command_status: 88 = 0x0058 Their reasoning is there is no other parameter in the SMPP spec to indicate you have run out of credits. When this happens the SMS is not sent and I wondered if there is any way to get back the command_status when sending an SMS. there are other messages the SMSC can send back. Throttling is definitively the wrong response. A real SMSC would not do that (most real SMSC's dont have credit limits and only work postpaid) And frankly having a DLR url for that would mean you get this being called even withouth being out of credit but simply because you send a lot of data quickly. Talk to your provider and ask him to send you back some other error message which is more to the spec. I'm sure you talk to some aggregator software, not a real SMSC. I just checked what our SMSC does in this case and it does return ESME_RMSGQFUL if you have a connection bound. If you try to connect, it returns ESME_RBINDFAIL. The problem with throttling is that its only a TEMPORARY failure and Kannel MUST step back and wait a second or two and then continue sending, meaning retrying the SMS over and over again. Thats the purpose of the throttling message. Its there for flow control. So it doesn't make any sense to generate any kind of DLR for it. It would not result in what you are looking for. Andreas Fink Fink Consulting GmbH --- Tel: +41-61-332 Fax: +41-61-331 Mobile: +41-79-2457333 Address: Clarastrasse 3, 4058 Basel, Switzerland E-Mail: [EMAIL PROTECTED] Homepage: http://www.finkconsulting.com --- ICQ: 101946485 MSN: [EMAIL PROTECTED] AIM: smsrelay Skype: andreasfink Yahoo: finkconsulting SMS: +41792457333 PGP9: 0714 DF2B A189 A760 6201 5CBD D040 3E71 4DAF 68BB
Re: devel Digest, Vol 21, Issue 7 (Stress Test SMPP)
Hi Mario,
It seems to me that the problem is the operator. Some operators allow
you to transmit messages to them as fast as you want, others don't.
Without the complete dump (for instance, beginning with sequence number
1), it is difficult to see whether kannel actually has a bug, or if it
is the operator who either is very slow at responding to submit_sm, or
is simply messing up with the sequence numbers.
Since enquire_link packages share the same sequence numbers as
submit_sm, it IS possible to get such a big gab between submit_sm and
submit_sm_resp. And if the operator takes more than 60 seconds to send a
submit_sm_resp, kannel will by default invalidate the old submit_sm and
send a new. This is why, when the operator responds, say 10 minutes
later, the sequence_number is way off, and it is no longer recognized by
kannel.
You could try to increase wait-ack or simply disable retransmission by
setting wait-ack to 0x02.
My personal experience tell me that the max-pending-submits work
flawless in both version 1.4.0 and the latests CVS, but there are quite
many SMPP gateways out there, who are not capable of replying within the
1 minute barrier. I once witness a gateway who simply was only capable
of replying with a submit_sm_resp every minute. So if you sent 10
msg/min, the first submit_sm_resp would come 1 minute later and the 10th
would come 10 minutes later. (by then the message has been retransmitted
several times by kannel)
Hope this helped.
Peter
Mario Noboa wrote:
Hi Stipe,
I made the stress test again with my operator. This time I set loglevel to
0, I changed throughput and max-pending-submits; but this did not resolve
the issue, the only difference is the problem takes place in different
messages, for example:
sequence number: 17074
sequence number: 17075
sequence number: 17076
sequence number: 17077 sequence number (_resp) : 4775
***error***
sequence number: 17078 sequence number (_resp) : 4776
***error***
sequence number: 17079
sequence number: 17191
sequence number: 17192
sequence number: 17193 sequence number (_resp) : 4777
***error***
sequence number: 17194
sequence number: 17195 sequence number (_resp) : 4778
***error***
(I´m attaching part of the log showing where the problem starts)
The values I used in my test where:
Test 1
Throughput = 7
max-pending-submits = 60
Test 2
Throughput = 20
max-pending-submits = 60
Test 1
Throughput = 10
max-pending-submits = 60
The only difference is the sequence number where the problem occurs, with
the different throughput and max-pending-submits combination.
After doing several tests with the operator, I decided to test kannel with a
SMSC simulator (Logica's SMSC Simulator), sending even more messages than
the operator sent me in the test (Actually I sent up to 200 msgs/sec),
kannel didn't have any problem.
I compared the sequences numbers, and checked that Kannel accepted and
answered correctly, this discard a race-condition, I think it's a problem in
the SMSC side, but they insist that it's in my side, I'd like to ask you
what can I ask the operator to check in their SMSC, and solve this issue?
Any clue will be highly appreciated.
Thanks for your help.
Mario
-Mensaje original-
De: Stipe Tolj [mailto:[EMAIL PROTECTED]
Enviado el: Jueves, 15 de Septiembre de 2005 8:50
Para: [EMAIL PROTECTED]
CC: devel@kannel.org
Asunto: Re: devel Digest, Vol 21, Issue 7
Mario Noboa wrote:
Hi Stipe, few days ago i did a stress test with an operator, kannel worked
very well with 30 msg/s. But i had problems with more than 40 msg/s in a
smpp conexion. I set throughput=6 and max-pending-submits=50 but didn't
work.
This error appears for each messange sent:
2005-09-02 11:15:51 [11943] [11] WARNING: SMPP[COM]: SMSC sent
submit_sm_resp with wrong sequence number 0x0001001d
I have a CPU system with 2 Gb and 1200 MHz. How would i configure kannel
for
this stress test?
Hi Mario,
now in terms of this WARNING, it means, Kannel received an submit_sm_resp
PDU
from the SMPP SMSC with a sequence number that was not within the dictonary
of
still open sequence number PDUs.
Obviously there may be 2 possible reasons:
a) a race-condition within the Kannel handling. Actually bearerbox
transmitting
the submit_sm PDU and receiving the _resp PDU faster then being able to
drop
the message in the still open dictonary. Which is almost uncertain, due to
the
amount of only 50 msg/sec.
b) a problem on the SMSC side itself. Which seems more reasonable.
I'd suggest that you do various tests on there again and try to circle in
the
problem scope. I do assume that you may not be able to reproduce the effect
on a
determenestic base, right?
Stipe
mailto:stolj_{at}_wapme-group.de
---
Wapme Systems AG
Vogelsanger Weg 80
40470 Düsseldorf, NRW, Germany
phone: +49.211.74845.0
fax: +49.211.74845.299
Character sets in SMPP
Hi,
During my quite extensive use of the SMPP client in kannel, I have
stumbled upon a few bugs related to the character handling in smsbox and
the SMPP sources.
1. When sending an SMS consisting of a '€' (euro) sign through SMPP, the
character is replaced with #8364; which is the unicode representation
of the character. Investigating the bug, I came to realize that
characters are converted into ISO-8859-1 before they are converted into
the GSM 03.38 charset, which obviously excludes the possibility for
transmitting the € sign. An interessting note however, is that the
latin1 to GSM conversion function uses CP1252 and thus HAVE support for
the €-sign.
2. As seen above, characters not convertable is translated into an XML
entity instead of a similar character, ?, or simply just being exluded
from the string. It seems that this problem is located in the smsbox code.
3. When sending an SMS with an alphanumeric originator, kannel does no
translation into any alternative character set, although the common
practice is to translate it into GSM 03.38 (1 char/octet).
Since the company I work at rely on these fuctionalities in kannel, it
is urgent that they are solved, and since I haven't got the luxury to
dig entirely into the kannel source, my attached patch (which solves
problem 1 and 3) is probably somewhat crude.
I've made a macro, defining the internal SMPP character set (just in
case), which is then used throughout the smsc_smpp.c file whenever
characters are converted. Besides, source_addr is parsed through the
charset_latin1_to_gsm function if is detected as being alphanumeric.
I will probably look at the xml entity stuff in a near future, but for
now, here is the patch fixing the SMPP charset probs. (for the latest CVS)
Best regards,
Peter Christensen
--- gateway/gw/smsc/smsc_smpp.c 2005-08-12 18:12:58.0 +0200
+++ gateway.new/gw/smsc/smsc_smpp.c 2005-08-18 18:01:57.055871145 +0200
@@ -78,6 +78,8 @@
#include sms.h
#include dlr.h
+#define SMPP_CHARSET CP1252
+
/*
* Select these based on whether you want to dump SMPP PDUs as they are
* sent and received or not. Not dumping should be the default in at least
@@ -389,6 +391,7 @@
break;
case GSM_ADDR_TON_ALPHANUMERIC:
+ charset_latin1_to_gsm(pdu-u.deliver_sm.source_addr);
if (octstr_len(pdu-u.deliver_sm.source_addr) 11) {
/* alphanum sender, max. allowed length is 11 (according to GSM specs) */
error(0, SMPP[%s]: Mallformed source_addr `%s', alphanum length greater 11 chars.
@@ -493,9 +496,9 @@
* unless it was specified binary, ie. UDH indicator was detected
*/
if (smpp-alt_charset msg-sms.coding != DC_8BIT) {
-if (charset_convert(msg-sms.msgdata, octstr_get_cstr(smpp-alt_charset), ISO-8859-1) != 0)
+if (charset_convert(msg-sms.msgdata, octstr_get_cstr(smpp-alt_charset), SMPP_CHARSET) != 0)
error(0, Failed to convert msgdata from charset %s to %s, will leave as is.,
- octstr_get_cstr(smpp-alt_charset), ISO-8859-1);
+ octstr_get_cstr(smpp-alt_charset), SMPP_CHARSET);
msg-sms.coding = DC_7BIT;
} else { /* assume GSM 03.38 7-bit alphabet */
charset_gsm_to_latin1(msg-sms.msgdata);
@@ -602,6 +605,7 @@
if (!octstr_check_range(pdu-u.submit_sm.source_addr, 1, 256, gw_isdigit)) {
pdu-u.submit_sm.source_addr_ton = GSM_ADDR_TON_ALPHANUMERIC; /* alphanum */
pdu-u.submit_sm.source_addr_npi = GSM_ADDR_NPI_UNKNOWN;/* short code */
+charset_latin1_to_gsm(pdu-u.submit_sm.source_addr);
} else {
/* numeric sender address with + in front - international (remove the +) */
octstr_delete(pdu-u.submit_sm.source_addr, 0, 1);
@@ -611,6 +615,7 @@
if (!octstr_check_range(pdu-u.submit_sm.source_addr,0, 256, gw_isdigit)) {
pdu-u.submit_sm.source_addr_ton = GSM_ADDR_TON_ALPHANUMERIC;
pdu-u.submit_sm.source_addr_npi = GSM_ADDR_NPI_UNKNOWN;
+charset_latin1_to_gsm(pdu-u.submit_sm.source_addr);
}
}
}
@@ -700,10 +705,10 @@
/*
* convert to the given alternative charset
*/
-if (charset_convert(pdu-u.submit_sm.short_message, ISO-8859-1,
+if (charset_convert(pdu-u.submit_sm.short_message, SMPP_CHARSET,
octstr_get_cstr(smpp-alt_charset)) != 0)
error(0, Failed to convert msgdata from charset %s to %s, will send as is.,
- ISO-8859-1, octstr_get_cstr(smpp-alt_charset));
+ SMPP_CHARSET, octstr_get_cstr(smpp-alt_charset));
}
}
