Re: multiple bearer box - Netikos?
On Sun, 23 Feb 2003, Stipe Tolj wrote: Nisan Bloch wrote: I think rolling NMGW into Kannel would be worthwhile. At the same time come up with a combined bbox+smsbox version, with the same HTTP interface. In addition an API to build XXXboxs with. Smsbox would be the first such app and maybe the smppbox and emibox that have been mentioned on the list. definetly. We should merge things from Netikos version into official Kannel. Main question would be how many efforts Netikos is *still* doing in continued development of their branch? Kalle? Practically nothing has been done since I pusblished that code. You guys could switch off to the official tree and get your great things included there. This would cause a benefit for all. That's why I published them, altought I knew that some things are a bit too radical. As we have no resources to do any development on it right now, I hope that you can scavenge useful things out from it and this way improve the Kannel project. -- kalle marjola
Re: multiple bearer box ?
On Thu, 20 Feb 2003, Aarno Syvänen wrote: I once stress tested stand-alone bearerbox. (Standalone meaning I was using smsboxes that did not do any fetch, but returned a standard text message instead.) I got value 1200 msg/s and must use ~20 smsboxes before I reached 100 % cpu utilisation. Cpu was AMD Athlon, 800 MHz. With this 'a bit optimized' merged bearerbox and smsbox of ours, I recall values of 1000-2000 msg/s in modern workstation (fakesmsc/simple answer). It is good to notice that main things that create bottlenecks are: - smsc drivers/smsc itself - smsbox utilities (http library) The current Kannel is, however, made in that way that it gets trashed if it receives messages faster than it can handle, so beware :] -- kalle marjola
Re: smscconn cimd2
On Thu, 6 Feb 2003, Stipe Tolj wrote: Kalle Marjola wrote: The attached patch changes cimd2 to use SMSCConn instead of old SMSCenter. However, notes: - diffed against latest CVS version (just did cvs udpate) - not tested! We do not have cimd2 currently available. Compiles, however =] - had some problem with merge, so it could have been made more efficiently (in original version, only PrivData was moved around, not entire SMSCConn, but DLR merge required modifications) - does not fully utilize SMSCConn possibilities, it is more like smsc_wrapper moved to cimd2. But it is easier to start using them after this works (and get rid of smsc_wrapper...) - includes some notes of various extra code which I couldn't include in the patch as then it would require patching of yet another big pile of files. However, if this can be merged, they might come next. can you resend the patch please in diff -u format against current cvs tree?! Thanks in advance. No. The patch required some work to do from our CVS version and that final version is only at my desktop computer - which currently does not boot (disk crash) and I'm about 3000 kms away from fixing it. -- kalle marjola
Re: Reloading configuration
On Wed, 11 Dec 2002, Stipe Tolj wrote: Shridhar Raju wrote: Does it possible to reload configuration without restarting, either by isolating and suspending kannel gateway? no, unfortunatly now. Kalle told us that the netikos version does this. Kalle, is there any efforts from your side to integrate these thins into the official tree?! No, too much work, too many differences, we have no resources to do that. As I have stated, most of our things require rework of many internal parts and if that is not done/used, integrating those other things are very close to rewriting them entirely. But we can 'release' that code to be freely used by the Kannel project as it sees fit, apart from unicode part (distribution rights to that part are still owned by Wapit estate), if someone wants to start hacking. -- kalle marjola
Re: Kannel archtitecture
The original modified version is available for you to test (but for possible copyright reasons in some parts I won't announce the place here, so send me email) so investigate it and think about this issue. Feel free to say your opinion how easily some things can be ported to main Kannel tree, too. Some days have passed and so far I have received some comments about this; however, more feedback would be nice to have and especially on this list - what do you people think? -- kalle marjola
Kannel archtitecture
Yet again I return to this subject of Kannel internal architecture thing. This time with something very concrete; available from my web page is now the entire modified version of Kannel we have made mainly during last Fall and Winter, with list of features we would like to include to Kannel. However, as told earlier, many of these changes are quite radical and thus not very easily portable. Now, instead of trying to port them neverthless, I propose something more radical: Move the entire Kannel to use our core engine, then port all new features in Kannel to that core engine (SQL interface, new AT2, all kind of special messages, DLR's (see notes below) etc.) Why this? Because we feel that the new design is more robust, stabile and faster. It has better SMS storing system, faster SMS handling, condiguration loading etc. And as most new features in Kannel are just new modules which use the interfaces I defined 2 or so years ago, so they should be easily movable (I dunno about DLRs, but it is mainly in smscconn, after all.. we also have our system to handle DLR's, it is a bit different) The biggest system-wise difference is that we have merged smsbox and bearerbox, while Kannel has separate. For this I propose that we can have BOTH: allow that it can be configured to use external smsboxes or just use internal handling (or maybe both?). We also have a bit different configuration file syntax for sms-services but that is just details.. Yes, this sounds heavy and very drastic. But I hope you can at least think about it, and think hard. Old Kannel developers at least can somehow trust that I and Uoti know what we have been doing :] The original modified version is available for you to test (but for possible copyright reasons in some parts I won't announce the place here, so send me email) so investigate it and think about this issue. Feel free to say your opinion how easily some things can be ported to main Kannel tree, too. -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: Kannel archtitecture
I liked having a seperate smsbox though. Even with the option of keeping the external smsbox or using the internal, we won't have the new features. These features are not heavily related to smsbox/features, and that is just details.. Also I have question marks about the copyright thingy. This is the infamous Unicode thing. (opens the Dusty Book of Kannel History and starts to tell a tale... ;) Long, long time ago we at Wapit did Unicode support for Kannel. However, for certain reasons we were not allowed to publish it immediately. And then the bankcrupty hit us (WapIT), and thus it (Unicode support) was never released as open source. After that, various parties bought non-exclusive rights to that Unicode from bankcrupty estate, including NETikos. So, in theory, someone MIGHT get mad if that code is released as Open Source. People at MatchEm were investigating the possibility to publish it neverthless, but then the development in MatchEm was dropped (afaik). On the other hand, that Unicode code is quite simple, any good programmer would do it in a day or two, at least internal parts (not SMSC), the main work was in decising HOW to do it -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
smscconn cimd2
The attached patch changes cimd2 to use SMSCConn instead of old SMSCenter. However, notes: - diffed against latest CVS version (just did cvs udpate) - not tested! We do not have cimd2 currently available. Compiles, however =] - had some problem with merge, so it could have been made more efficiently (in original version, only PrivData was moved around, not entire SMSCConn, but DLR merge required modifications) - does not fully utilize SMSCConn possibilities, it is more like smsc_wrapper moved to cimd2. But it is easier to start using them after this works (and get rid of smsc_wrapper...) - includes some notes of various extra code which I couldn't include in the patch as then it would require patching of yet another big pile of files. However, if this can be merged, they might come next. -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi) Index: gw/smscconn.c === RCS file: /home/cvs/gateway/gw/smscconn.c,v retrieving revision 1.32 diff -r1.32 smscconn.c 74a75,76 else if (octstr_compare(smsc_type, octstr_imm(cimd2)) == 0) ret = smsc_cimd2_create(conn, grp); Index: gw/smscconn_p.h === RCS file: /home/cvs/gateway/gw/smscconn_p.h,v retrieving revision 1.32 diff -r1.32 smscconn_p.h 180a181,183 /* Responsible file: smsc/smsc_cimd2.c */ int smsc_cimd2_create(SMSCConn *conn, CfgGroup *cfg); Index: gw/smsc/smsc.c === RCS file: /home/cvs/gateway/gw/smsc/smsc.c,v retrieving revision 1.1 diff -r1.1 smsc.c 76,87d75 /* CIMD 2 */ smsc-cimd2_hostname = NULL; smsc-cimd2_port = -1; smsc-cimd2_username = NULL; smsc-cimd2_password = NULL; smsc-cimd2_send_seq = 1; smsc-cimd2_receive_seq = 0; smsc-cimd2_inbuffer = NULL; smsc-cimd2_received = NULL; smsc-cimd2_error = 0; smsc-cimd2_next_ping = 0; 158,165d145 /* CIMD 2 */ octstr_destroy(smsc-cimd2_hostname); octstr_destroy(smsc-cimd2_username); octstr_destroy(smsc-cimd2_password); octstr_destroy(smsc-cimd2_inbuffer); octstr_destroy(smsc-sender_prefix); list_destroy(smsc-cimd2_received, NULL); 216,220d195 case SMSC_TYPE_CIMD2: if (cimd2_submit_msg(smsc, msg) == -1) goto error; break; 272,277d246 case SMSC_TYPE_CIMD2: ret = cimd2_receive_msg(smsc, msg); if (ret == -1) goto error; break; 336,341d304 case SMSC_TYPE_CIMD2: ret = cimd2_pending_smsmessage(smsc); if (ret == -1) goto error; break; 503,504d465 else if (octstr_compare(type, octstr_imm(cimd2)) == 0) typeno = SMSC_TYPE_CIMD2; 579,590d539 case SMSC_TYPE_CIMD2: if (host == NULL || port == 0 || username == NULL || password == NULL) error(0, Required field missing for CIMD 2 center.); else smsc = cimd2_open(host, port, username, password, keepalive, sender_prefix); break; 708,710d656 case SMSC_TYPE_CIMD2: ret = cimd2_reopen(smsc); break; 754,758d699 errors = 1; break; case SMSC_TYPE_CIMD2: if (cimd2_close(smsc) == -1) Index: gw/smsc/smsc_p.h === RCS file: /home/cvs/gateway/gw/smsc/smsc_p.h,v retrieving revision 1.1 diff -r1.1 smsc_p.h 26d25 SMSC_TYPE_CIMD2, 92,104d90 /* CIMD 2 */ Octstr *cimd2_hostname; int cimd2_port; Octstr *cimd2_username; Octstr *cimd2_password; int cimd2_send_seq; int cimd2_receive_seq; Octstr *cimd2_inbuffer; List *cimd2_received; int cimd2_error; time_t cimd2_next_ping; Octstr *sender_prefix; 199,208d184 /* * Interface to Nokia SMS centers using CIMD 2. */ SMSCenter *cimd2_open(Octstr *hostname, int port, Octstr *username, Octstr *password, int keepalive, Octstr *sender_prefix); int cimd2_reopen(SMSCenter *smsc); int cimd2_close(SMSCenter *smsc); int cimd2_pending_smsmessage(SMSCenter *smsc); int cimd2_submit_msg(SMSCenter *smsc, Msg *msg); int cimd2_receive_msg(SMSCenter *smsc, Msg **msg); Index: gw/smsc/smsc_cimd2.c === RCS file: /home/cvs/gateway/gw/smsc/smsc_cimd2.c,v retrieving revision 1.2 diff -r1.2 smsc_cimd2.c 3a4,5 * * Merge of SMSC Wrapper 2002 Kalle Marjola 31c33,37 #include smsc_p.h --- #include smscconn.h #include smscconn_p.h #include bb_smscconn_cb.h
Re: merging smsbox and bearerbox
This is going to be a bit long mail. Background: As I have now and then announced, I (long time ago) merged smsbox and bearerbox together, to make Kannel faster, easier to program (DLR's, store-files, dynamic configurations etc.). As a part of this project, we made list of reliability improvements. The main aims of this project were: - to make Kannel as fast as possible, so that the speed _always_ depends on external things (SMSC, HTT server) speed - to make system as reliable as possible (reliable store-file, steady memory use etc.) As a result, there is extend modifications to some parts (sms services) while some are as they were (newer smsc connections, wap parts). Now the politics here at NETikos have changed (well, not the only thing that have changed) so we look forward to integrate various features to Kannel CVS tree and partake more in the project. Of course there is that problem that most of these 'improvements' are tied to that new architecture (merged smsbox) and doing them otherwise might not be possible or not worth it. So, is the merging worth it? Well I finally found time to run some benchmarks, here is results briefly (see the attachment for more results): The benchmarking was not easy thing, and it is NOT a very good one either, as the problem is how to benchmark just one thing. As the merged Kannel (our version) has in addition to that merge, a better store-file and throttle system for fakesmsc, it resulted in level results regardless what I threw to it - it did not eat the resources of the system nor slowed down. With Vanilla Kannel (fresh CVS) I have to put store-file away etc. So this benchmark not only tested the speed difference of merged smsbox and bearerbox versus separate boxes, but other parts, too. I tried to minimize the effect of other parts and thus ran several tests with various settings. Well, the full results are in the attachment. Short conclusion: It is clearly faster to use merged system. However, as a sidenote, the inter-box speed etc. are not the main issue here, unless building up a very, very special system (where there is more than 1000-2000 msg/s of incoming traffic). However, the merged system made it far more easier to implement throttle system (for incoming messages and HTTP) and improved store-file, which resulted that this merged version could handle any loads without slowing down or eating all the resources. Vanilla Kannel was killed by OS after swap ran out or it got bogged down by the store-file. But that is another store does that happen in real world or just in benchmarks :] As last words, yes we have various little bits and in upcoming weeks, we investigate how we can merge them into CVS tree and then provide patches. Hopefully as many as possible, but some might become difficult because of the above differences and limited time to do bigger modifications to make them work.. -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi) Setup: Vanilla Kannel (fresh CVS) vs. modified Kannel (merged bearer- and smsbox, throttle system to fakesmsc driver, improved HTTP) System: Linux on tabletop Linux. No store-file. Merged Kannel has better store-system. Vanilla Kannel got completely frozen if store-file was used with high incoming message traffic. --- 1. fakesmsc and bearerbox/smsbox all in same machine, sending 'nop' messages at interval of 0.0001 (1/s) (note: fakesmsc binary of own distribution. But I guess they are the same) Vanilla Kannel: (1) 2002-09-17 09:17:42 [0] INFO: fakesmsc: from first to last sent message 1.4 s, 7308.5 msgs/s 2002-09-17 09:17:42 [0] INFO: fakesmsc: from first to last received message 4.7 s, 2120.6 msgs/s (10) 2002-09-17 09:04:29 [0] INFO: fakesmsc: from first to last sent message 13.4 s, 7441.1 msgs/s 2002-09-17 09:04:29 [0] INFO: fakesmsc: from first to last received message 50.0 s, 1999.2 msgs/s NOTE: memory growth (to 25 megabytes) (100) Couldn't handle, too much memory growth. Killed by operating system. Same happened even with interval = 0.0004 (2500 msg/s) Merged Kannel: (1) 2002-09-17 07:02:09 [0] INFO: fakesmsc: from first to last sent message 1.0 s, 9864.4 msgs/s 2002-09-17 07:02:09 [0] INFO: fakesmsc: from first to last received message 2.7 s, 3711.5 msgs/s NOTE: with store-file, ~2800 msg/s (10) 2002-09-17 07:03:12 [0] INFO: fakesmsc: from first to last sent message 17.7 s, 5650.2 msgs/s 2002-09-17 07:03:12 [0] INFO: fakesmsc: from first to last received message 27.6 s, 3623.6 msgs/s NOTE: with store-file, ~2830 msg/s (100) 2002-09-17 07:08:24 [0] INFO: fakesmsc: from first to last sent message 252.8 s, 3955.7 msgs/s 2002-09-17 07:08:24 [0] INFO: fakesmsc: from first to last received message 262.7 s, 3807.2 msgs/s NOTE: with store-file, ~2870 msg/s
Re: [PATCH] AT2 rewrites (resubmit)
idents are 4, tabs are 8. This is probably because editors like vi have a default of 8 but you really wana have 4 to write the code. So you use soft tabs which make you edit like you have tabs at 4 but save it like tabs are at 8. Personally I would prefer to have tabstops at 4 and idents at 4. Would make life much easier in any editor I guess. My editor (BBEdit) doesnt have virtual tabstops so I have to do it manually and I hate it. Would it be too keen to decide to change that rule? The idea in that is that most of the computer systems in the world has been configured so that tab is 8. If we change that, then all kind of cat, more, less, print etc. outputs of the source code get f*d up unless tab size of those is also modified to 8. Why can't you people just get an editor that auto-indents with spaces? :] -- kalle marjola NETikos finland (http://www.netikos.fi)
smsbox
How many of you really use the feature of independent smsbox in different machine than the bearerbox? (or several smsboxes). Just thinking about the old 'thread smsbox' which was in old Kannel... -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Stipe Tolj wrote: How many of you really use the feature of independent smsbox in different machine than the bearerbox? (or several smsboxes). Just thinking about the old 'thread smsbox' which was in old Kannel... we do, at least in some extend. Are you planing to incorporate smsbox as bearerbox thread?! Well, several threads... (and this is just speculation. Last time the idea was torpedoed down because of load distribution reasons, after I have done some tests etc.) If yes, we dislike this because of several reasons, one of them is the abilitiy to make config changes 'on-the-fly' to smsbox related groups, while bearerbox is still running. Do you have some nifty way to take smsbox down for that use, so that no messages are lost while they are being transfered between bearerbox and smsbox? (or open http connections) Dynamic configuration loading is doable (been there...), so what are those other reasons? (it is amazing how much it simplifies things to have smsbox in same process. And makes Kannel much faster) (for distribution, the possibility to run distinct smsbox could be left, too, alhought I do not see much use for that...) -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Harrie Hazewinkel wrote: Do you have some nifty way to take smsbox down for that use, so that no messages are lost while they are being transfered between bearerbox and smsbox? (or open http connections) (pure speculating here) A way could be to have the smsbox doing some gracefull restart by reading the configuration while the connections (sockets) stay connected. Then messages would get queue up. Unless the restart takes very long you would no loose messages. Not sure how the timing/speed is for this. Well my question was not about how it can be done but how they do do it right now :] (it is amazing how much it simplifies things to have smsbox in same process. And makes Kannel much faster) Just curious, but what about the wapbox?? Would that go faster too?? Possibly, but with wap, you do not need to say 'ack' to each message like with smsc Yes, dynamic configuration is possible via various ways, SNMP, HTTP or even just some kind of gracefull restart that reads the configuration and setups the new environment. What I meant here is that dynamic configuration loading inside Kannel without stopping the smsc connections can be done (quite easily), thus effectually same thing as never stopping bearerbox but restarting smsbox (for distribution, the possibility to run distinct smsbox could be left, too, alhought I do not see much use for that...) Only when the load can go so sky high that a singl machine cannot keep up Naturally. But show me the single SMSC connection that can do that. And if you have several SMS center connections, you can have several bearerboxes, too (yes, I know that if you need to reoute things between them, this can still cause problems. That is just what I'm asking here, does anyone really USE that or is it just speculation that it CAN be done and used? Is it worth all the pain of message/ack/dlr transfer between smsbox and bearerbox?) Usually the speed of applications (oh, the java servlets) and SMSC connections is the only factor of the speed. The core Kannel can handle like 1000-2000 msg/s of fixed answers in normal modern tabletop, like 500-1000 HTTP based services (if the server is fast enough) SMS centers usually do something between 10-50/s, java servlets less. Correct me if I'm horribly wrong -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Stipe Tolj wrote: Well my question was not about how it can be done but how they do do it right now :] we do terminate and restart smsbox to re-read the updated configuration. Okay, just curious about possible open HTTP connections etc. so they do not cause problems? (like lost replies later on requeed with store system) What I meant here is that dynamic configuration loading inside Kannel without stopping the smsc connections can be done (quite easily), thus effectually same thing as never stopping bearerbox but restarting smsbox which is what we do. But you still have to stop and restart bearerbox if you change smsc groups (adding or modifying) and hence those who are not affected have to go down and up too. Yes, the same way it goes with the integrated one. I'm also thinking of problems arrising from the planed modularization. As far as I see we will have modularization (which is already there) for SMSC modules and SMS services (i.e. sendsms, sendota, sendringtone, sendlogo etc). which means if someone links in a SMS service module (into smsbox scope) and smsbox is implemented as threads inside bearerbox and the new code dumps core, the whole systems blows up. Which is not the way we would like to see it. That is a valid point. If I start to speak ideally, I would get rid of all such services and instead implement just one simple http sender/receiver and all those services would be done by external filter/proxy/whatever. No idea to make basic _gateway_ an application server. But that is of course 'if I was the king of the world and would do redo everything' -talk. Things that aleady exist are hard to move away (like those prefix-suffix things in sms-service groups.. one in 200 uses them and thus they cannot be removed and left to be application problem..) -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Stipe Tolj wrote: reload of configuration on the fly?! Hmm, that's new to me, what signal are you using and for what box? I mean that can be done, I have done some tests, and it used admin command. AFAIK, you can not re-read the config file on the fly?!?! Not with CVS Kannel. With patched, yes. (but do not ask for it, at least yer, it is quite a mess and has all these toher thigns I'm talking about here, so it is not compatible) if we astract the logical bearerbox - smsbox communication so the configure switch does select then the accurate implementation (function or tcp calling) this would work. Think of MySQL-4's new way to link the whole mysqld engine to your client executable, I think they use the same approach. Hm, that could work.. All smsboxes are equal, and thus bearerbox sends the message to random one, or the one with lowest load (cannot recall the exact algorithm, use the source...) Hmm, that's bad in some sence. Well, it has been like this for all these 3 or so years and no one has ever complained... (but I doubt that anyone has used more than one smsbox except for testing purposes :) PS: Now that I at last remember: please do not send me double copies each time you reply to my emails. I'm on devel list, too, no need to send personal reply (hm, am I starting to sound like Lars?) -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: old smsc's and smsc_wrapper
On Tue, 9 Jul 2002, Kaido Karner wrote: does anybody use any smsc's which use old format smsc modules (CIMD, CIMD2, EMI, EMI_IP, SEMA, OIS, AT)? I use CIMD2. any volunteers to convert at least CIMD2 to new implementation? :) I have some porting of cimd2 to smscconn (and at) and have discarded rest. Has there been lots of changes to cimd2 recently? I havenät checked that version for some time, so there might be some updates needed. And I'm not guarenteening its behaviour, but so far no problems. -- kalle marjola
Re: Problem with SMPP smsc-password being too long
On Mon, 8 Jul 2002, Stipe Tolj wrote: contiue? why? the password is logically longer than 8 octects, but the SMPP spec does not accept it to put it in the appropriate PDUs. So what should you do then? Cut the Password to 8 chars long and send the login sequence anyway? No, I don't think so. The assertion check fails, and a failing assertion check means a non recoverable state within the system, hence panic! This is my opinion about the functional logic. But if you want to implement reloading of the configuration, non-fatal errors should not cause panic (in fact, no error except running out of memory, perhaps. Just revert to old configuration if the new one is unacceptable). And this error is not fatal, it simply means that this SMPP connection cannot be used (well, sort of). If you want to exit because of faulty configuration that is okay, but panic should be avoided as long as possible (and it is horror to debug panics ;) -- kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
RE: Message store (Was: EMI: Serious Problem PANIC: Too manyconcurrentallocations)
On Thu, 13 Jun 2002, Oded Arbel wrote: You also don't need to store acks in the store, as only unacked messages are stored there. But note that if you do not store them, you have to write the entire store each time you get ack or if your system goes down between two writes, it will resend all messages ackked after the last write. -- kalle marjola
Re: Message store (Was: EMI: Serious Problem PANIC: Too manyconcurrent allocations)
On Wed, 12 Jun 2002, Aarno Syvänen wrote: Speaking of store files - After really looking into the store logic, it The better way to implement the store file (than the old way I did quickly use - you have to remember that original store file was just a quick hack to concentrate more on SMS side that almost-useless-WAP ;) - like earlier, write all messages and acks and nacks to file - keep copy of all non-acked messages in memory - when Kannel receives an ack (or nack), write a new store file out of still remaining (non-acked) messages to replace the old. Naturally only do this now and then, like if there has not been writing in last half a minute. (and of course first write to new file and then rename etc.) ..this way the store file normally keeps empty or only some odd messages, unless lots of messages do not get (n)acks. Of course this means that Kannel will grow in memory if messages do not get ackked, too. -- kalle marjola
Re: EMI: Serious Problem PANIC: Too many concurrent allocations
On Sat, 1 Jun 2002, Jari Juslin wrote: So, panic to Too many concurrent allocations can happen if a) Kannel just has so high load, that is needs huge amounts of memory or b) if Kannel leaks too much memory. Am I right? Yes. The reason a) is encountered when incoming traffic is faster than the outgoign handle (namely SMS centers) and usually results in that 'too many concurrent allocations'. This can be avoided by increasing the size of the checking malloc buffers, but if things keep going like that it will eventually panic. Moreover, store files of several megabytes is more than a problem. This entire thing should be fixed so that more priority is given to outgoing messages, i.e. just make it impossible to grow queues inside Kannel - just make the queues to build up into sockets/other programs. Of course this is far from trivial with some SMS centers. -- kalle marjola
Re: EMI: Serious Problem PANIC: Too many concurrent allocations
On 24 May 2002, Abd Rahman Johari wrote: I'm trying to send out SMS Broadcast and after a while kannel just die because of PANIC: Too many concurrent allocations You need to compile Kannel with non-checking malloc, i.e. use native malloc. ./configure --with-defaults=speed or --with-malloc=native should help. Hm, this question needs to be a s a first one in FAQ, it is so often asked.. (in fact I could not find it from FAQ section at all..) -- kalle marjola
RE: [BUG] list implementation too slow.
On Thu, 21 Mar 2002, Oded Arbel wrote: I think this problem occurs as the bearerbox can't send messages as fast as they are delivered (probably due to a large burst and then sustained high load). I see messages queued on the module's queue, and then it slows doesn the module more, so it can handle less messages (while high load continues) and so the queue grows longer. The solution to this is to fix all code so that outgoing messages have higher priority than incoming ones, i.e. do not read anything from bufefrs before outgoing messages have been sent. NOTE: You need to set conn() buffer size, otherwise this does not help as things get added to its internal buffer... I did this for my modified version and got rid of growing size (in memory) and slowdown. (however, I do not know how SMSC would react if the socket gets filled up because the other end is not reading it fast enough..) -- kalle marjola
Re: loadable modules ?
On Fri, 22 Feb 2002, Stipe Tolj wrote: Alexei Pashkovsky wrote: I think idea of having the listening http interface inside bearerbox instead of running two executables could be usefull too. Obviously this should be configurable .. that's not how Lars designed it. No, but he has also said that the entire architecture should be re-thinked now when more the exact 'goal' is better known (ref: a conversation with him a few minutes ago) - similar reason why I did merge smsbox and bearerbox last summer, but which was a bit too drastic modification to the main project. As a sidenote, old Kannel had an option of having thread smsbox, i.e. instead of sending messages to seperate executable they were handled inside bearerbox. When the message queue/flow system was redesigned, I was too lazy to re-write that thread version... (althought actually it was so simple that instead of callin some write_to_box function the message was just put directly to receive queue...) So, it should be quite simple to re-implement 'thread' smsbox even if a separate smsbox is kept... (but I still feel that the advantages gained from that is far less than the extra problems caused, especially with DLR's etc... Kannel is anyway faster than normal HTTP servers or SMS center connections.. and that protocol between bearerbox and smsbox is not that faster than HTTP protocol...) PS: Loadable modules: a great idea -- kalle marjola
RE: EMI2 fix for multi recipients
On Tue, 13 Nov 2001, Jacob Vennervald Madsen wrote: So when I use kannel I should make one http request per recipient? Yes, but see below. Has anybody tried to send messages to more than 10.000 recipients? How does it perform? I mean it does create a huge network overhead. If you have this kind of mass-sending needs, I recommend adding a new content-type like application/Kannel-MassSending ;) to sendsms POST interface. Then just use some format in body, like receiver:content or whatever and this way submit them with just one HTTP call. -- Kalle
RE: Incoming concatenation..
On Fri, 26 Oct 2001, Anders Lindh wrote: Ok.. What would be required to implement the functionalty? Basically, I'd need to atleast get the contents of the messages. A strip-udh setting in kannel.conf could be implemented, this way the dirrent parts of long messages could be saved. Any thoughts? Whenever an SMS with concatenation set on arrives, put it aside until all parts have been received. Then when the entire message has been received, do whatever merge/etc. is needed. In this case, a suitable plcae for bundling would be smsbox, I guess. Of course if some parts never arrive the rest are forever queued, but that is more like a fundamental question in multipart messages. Kalle
Re: AW: PANIC: Too many concurrent allocations
On Wed, 24 Oct 2001, Jörg Pommnitz wrote: It's either a bug in Kannel (memory leak) or a heavily stressed system running in debug limits. If it's not a memory leak, you can prevent the emergency shutdown by disabling debug malloc while calling configure. The checking malloc system can only handle limited number of memory allocations at once; if there is more, it thinks it is a memory leak. Several months ago I multiplied the limit by 10x and thus the limit was very seldom reached. However, it was still met and it means that somewhere a queue is built up. And in normal situation this is not good, as queue should be done in sockets/similar places, not inside Kannel. I tinkered a bit with that and we fixed smsc_fake a bit so that queue is not built inside Kannel anymore. That way a benchmark/test of million or even 10 million messages can be tested out. BTW, the test showed that checking_malloc Kannel has speed of about 20-25% of native malloc.
Re: [patch] Flash, MWI and Unicode
On Thu, 19 Jul 2001, Bruno David Simões Rodrigues wrote: but this way you have to have code to do the utf-8 encoding inside kannel. I don't know if it's easy or not. My patch only activates the unicode bit and pass the text in raw mode. This way, something outside kannel could encode the unicode and pass it to kannel. as long as we use libxml it includes code to convert from utf-8 to other sets. I think wapit had some code to unicode but we can't use it anymore. Yup. Let's see what happens as soon as the bankcrupty thing is cleared. -- Kalle || http://iki.fi/rpr/
RE: [RFC] mysql interface to kannel?
On Fri, 20 Jul 2001, Paul Keogh wrote: - logging facility: access.log data may be stored to mysql db, especialy for automatic billing support. - configuration facility: multi-groups like sms-user, sms-service, ota-config may be stored in mysql db, so there may be added or manipulated while runtime and the service would not have to be restarted to get notice of an update. This would make thinks particular slower, but may add incredible functionality gains. I think the DB interface would be quite useful but I would echo comments from a previous post to ensure that there is a gw_ style wrapper approach so that use is optional. Updating running configurations is much tricky and doesn't depend on holding the configuration in a DB. There was some discussion lead by Lars some time ago about doing this so the archives might hold some interesting comments. Unfortunately, all Kannel code assumes that config file is read once at start-up and would require major surgery to move to a dynamic model. My custom tests can handle reload of configuration files via HTTP administration interfaces. For sendsms-users and sms-services, it does it 'on fly', but for other groups it restarts Kannel (which also can now be done via HTTP, i.e. without external start/stop script). Even for those groups which can be dynamically loaded it currently means: - lock system down - reload - release system In any case, it is something and usable for most needs, and can be tweaked quite easily (just split reloading into small parts), I just have had no time to do that yet. However, alas, it is based on that modification I did that smsbox is now merged to bearerbox etc. i.e. quite drastic modifications. So I simply cannot send a patch. (I have also replaced urltrans with own modules for sms-services and sendsms-users as they are practically different thing and removed some things rarely used and making things complicated) But a database interface in serial to http:, fixed-text: and file: is a very interesting idea. In any case, as soon as I really get things cleared I will start to check out again what can be applies as patch.. PS: who controls Kannel releases now? -- Kalle || http://iki.fi/rpr/
Re: CPU-usage
On Wed, 4 Jul 2001, Frederik Ammitzbøll wrote: It's always the same thread (whether it's smsbox or bearerbox) that gets hungry. I believe it to be thread 1 (gwlib/fdset:poller) so that might be where the problem is. Any suggestions? Hm sounds like a problem in fd_set.. so gwlib is a good place to double-check (conn.c) I have a solution that stops smsbox eating CPU, but it is a bit complicated and not yet ready - it is called 'merging bearerbox and smsbox', which have a list of advantages (easier to make that no messages are lost, faster roundtrip time, faster overall speed...) - honestly, how many have run more than one smsbox (not counting tests) -- Kalle || http://iki.fi/rpr/
Re: SMPP
On Wed, 27 Jun 2001, Nisan wrote: Hi I was wondering why the SMPP interface (smsc_smpp.c) is not built around smsc.c and smsc_p.h? because smsc.c and smsc_p.h are depricated and should not be used by any SMSC module anymore - all new ones should be done by smscconn.h as stated in README (hm, there should be indicator in smsc.h about that but I do not have cvs rights anymore so I cannot write that into it :) When talking about smsc vs smscconn, smsc_at should be converted at some point to use smscconn, no other 'major' protocol relies on smsc anymore.. (emi2 replaces old emi, cimd2 replaces cimd1.2..) -- Kalle || http://iki.fi/rpr/
