I'm back
I'm back after 2-3 weeks, this time with my private email address for a while. Soon I will be with my new work address, and I just wanted you to know that I'm returning to develop Kannel (I'm actually working at turbo-kannel with lots of little and larger modifications on it), and patches will start to appear if they fit into generic Kannel development But more about that later. Keep up the project. -- &Kalle || http://iki.fi/rpr/
Re: SMSC connection
On Wed, 27 Jun 2001, Alexei Pashkovsky wrote: > Right, I saw the new interface in SMPP module. > Does anyone have docs for it, can you supply us a list of modules that use > new interface ? see gw/smscconn_p.h modules smsc_emi2, smsc_fake and smsc_smpp should use it. > > > is there someone who have some free time to explain me when Kannel uses > > > those : > > > SMSCCONN_FAILED_SHUTDOWN, > > > SMSCCONN_FAILED_REJECTED, > > > SMSCCONN_FAILED_MALFORMED, > > > SMSCCONN_FAILED_TEMPORARILY > > > and what exactly happens if the failure is temporarily ? > > > Does bearerbox try to resend the message than ? see bb_smscconn.c - in a case of shutdown and temporarily, message is rerouted and retried with other smsc connection (if any), while rejected/malformed is discarded. -- &Kalle || http://iki.fi/rpr/
Re: SMPP
On Wed, 27 Jun 2001, Nisan wrote: > Hi > > I was wondering why the SMPP interface (smsc_smpp.c) is not built around > smsc.c and smsc_p.h? because smsc.c and smsc_p.h are depricated and should not be used by any SMSC module anymore - all new ones should be done by smscconn.h as stated in README (hm, there should be indicator in smsc.h about that but I do not have cvs rights anymore so I cannot write that into it :) When talking about smsc vs smscconn, smsc_at should be converted at some point to use smscconn, no other 'major' protocol relies on smsc anymore.. (emi2 replaces old emi, cimd2 replaces cimd1.2..) -- &Kalle || http://iki.fi/rpr/
Re: CPU-usage
On Wed, 4 Jul 2001, Frederik Ammitzbøll wrote: > It's always the same thread (whether it's smsbox or bearerbox) that gets > hungry. I believe it to be "thread 1 (gwlib/fdset:poller)" so that might be > where the problem is. Any suggestions? Hm sounds like a problem in fd_set.. so gwlib is a good place to double-check (conn.c) I have a solution that stops smsbox eating CPU, but it is a bit complicated and not yet ready - it is called 'merging bearerbox and smsbox', which have a list of advantages (easier to make that no messages are lost, faster roundtrip time, faster overall speed...) - honestly, how many have run more than one smsbox (not counting tests) -- &Kalle || http://iki.fi/rpr/
RE: smsc_at question
On Fri, 6 Jul 2001, Alex Judd wrote: > Ideally the whole of the smsc_at module could do with a good overhall to > make it asyncronous and to learn from the behaviour patterns that everyone > in this group has worked out (as well as to make it upgraded to the new smsc > code style). Best plan would be to put a team together to work on this part > of the code and it's architecture - I suggest Nick Clarey, myself, yourself, > any other volunteers - and then we can ideally produce a fully working at > smsc gateway (*smslink and all the others in the market that I have found > have various other limitations so it would do everyone a favour to do this). Remember to write it to use smscconn.c, not smsc.c, when you do this :) (it should be even easier to write it to use smscconn than smsc as there is no more pending_message etc. calls. After that only sema uses smsc.c and it needs rewriting, too, and then we can finally get rid of that old relic causing problems) I was also looking to update smsc_at but I am completely filled up with other little fixes so if someone else does this that would be excellent. -- &Kalle || http://iki.fi/rpr/
Re: [patch] Flash, MWI and Unicode
On Thu, 19 Jul 2001, Bruno David Simões Rodrigues wrote: > but this way you have to have code to do the utf-8 encoding inside > kannel. I don't know if it's easy or not. My patch only activates the > unicode bit and pass the text in raw mode. This way, something outside > kannel could encode the unicode and pass it to kannel. as long as we use libxml it includes code to convert from utf-8 to other sets. > I think wapit had some code to unicode but we can't use it anymore. Yup. Let's see what happens as soon as the bankcrupty thing is cleared. -- &Kalle || http://iki.fi/rpr/
RE: [RFC] mysql interface to kannel?
On Fri, 20 Jul 2001, Paul Keogh wrote: > > - logging facility: access.log data may be stored to mysql db, > > especialy for automatic billing support. > > - configuration facility: multi-groups like sms-user, sms-service, > > ota-config may be stored in mysql db, so there may be added or > > manipulated while runtime and the service would not have to be > > restarted to get notice of an update. This would make thinks > > particular slower, but may add incredible functionality gains. > > > > I think the DB interface would be quite useful but I would echo comments > from a previous post to ensure that there is a gw_ style wrapper approach > so that use is optional. > > Updating running configurations is much tricky and doesn't depend > on holding the configuration in a DB. There was some discussion > lead by Lars some time ago about doing this so the archives > might hold some interesting comments. Unfortunately, all Kannel > code assumes that config file is read once at start-up and would > require major surgery to move to a dynamic model. My custom tests can handle reload of configuration files via HTTP administration interfaces. For sendsms-users and sms-services, it does it 'on fly', but for other groups it restarts Kannel (which also can now be done via HTTP, i.e. without external start/stop script). Even for those groups which can be dynamically loaded it currently means: - lock system down - reload - release system In any case, it is something and usable for most needs, and can be tweaked quite easily (just split reloading into small parts), I just have had no time to do that yet. However, alas, it is based on that modification I did that smsbox is now merged to bearerbox etc. i.e. quite drastic modifications. So I simply cannot send a patch. (I have also replaced urltrans with own modules for sms-services and sendsms-users as they are practically different thing and removed some things rarely used and making things complicated) But a database interface in serial to http:, fixed-text: and file: is a very interesting idea. In any case, as soon as I really get things cleared I will start to check out again what can be applies as patch.. PS: who controls Kannel releases now? -- &Kalle || http://iki.fi/rpr/
RE: DLR support
Just as a short comment to this DLR thread and possibly to give ideas; as I have been tweaking with Kannel for a while (including some drastic changes that were rejected on the list :) as one operation I planned one way to implement delivery reports in Kannel It is divided into following parts: - sms-service or sendsms has 'delivery-reports' set to true - when SMS is delivered to SMSC, delivery report is asked for, and before even ACK from original delivery is written to bb_store (thus eliminating original message) the sent message is also written to separate report-file along with delivery report 'receipt' given by the SMSC - whenever delivery report is returned from SMSC, that is written to that same report-file ..any logic to match original message to received report is handled by external program that reads that report-file. Kannel simply writes to it. That external program can then re-send message if no report has been issued (after a long time), or send a notification to other program or whatever. Or it can be simply a small perl-script to extract delivered messages so that they can be billed. Whatever is suitable. Thus, the system would mean: a) Kannel does not need to store any messages after they have been successfully delivered to SMSC (complete original messages are only stored in file, and disk is cheap..) b) Kannel does not have any built-in logic to handle expiring reports etc. (NOTE: however, I was thinking about a separate function to read that report-file upon (HTTP) request and re-send messages that has not received delivery report..) c) (3rd party) applications only interested to get the message delivered does not need to store any information; that external program will handle re-send and/or only bill delivered messages My original idea has delivery reports to be written into store-file (bb_store) but then it would result in expanding file - now it keeps empty as long as nothing goes wrong -- &Kalle || http://iki.fi/rpr/
Re: Problems again..
> > 2001-08-31 11:20:12 [6] WARNING: smsc_fake: invalid message syntax from > client, ignored There seems to be mixture of docs refering to old fakesmsc and new one. In new one, you have to define message contenty type, like "100 200 text foobar" instead of just "100 200 foobar" -- &Kalle || http://iki.fi/rpr/
Re: usage of smsc_http.c
On Tue, 2 Oct 2001, Stipe Tolj wrote: > Hi Bruno, > > > http smsc could be used to create a kannel proxy where first kannel forwards > > sms to the second > > It is too a template to build code to interface with some future smsc that > > may appear that talks http > > (I've heard about sonera?) > > can soneone give a working example setup? My imagination is still > lacking here to see what we may do using SMSC HTTP. Consider scenario, where you want your application to send SMS'es to 2 different carriers. The other one has CIMD2 and the other hand only provides HTTP-based "content gateway" (Sonera in Finland, for example) So, instead that your application talks with 2 different protocols, you hide them both behind Kannel. Which talks both CIMD2 and HTTP. (of course, you have to code that specific HTTP based protocol onto it, but then it is coded just to one place, not all your applications)
Re: AW: PANIC: Too many concurrent allocations
On Wed, 24 Oct 2001, Jörg Pommnitz wrote: > It's either a bug in Kannel (memory leak) or a heavily > stressed system running in debug limits. If it's not a > memory leak, you can prevent the emergency shutdown > by disabling debug malloc while calling configure. The checking malloc system can only handle limited number of memory allocations at once; if there is more, it thinks it is a memory leak. Several months ago I multiplied the limit by 10x and thus the limit was very seldom reached. However, it was still met and it means that somewhere a queue is built up. And in normal situation this is not good, as queue should be done in sockets/similar places, not inside Kannel. I tinkered a bit with that and we fixed smsc_fake a bit so that queue is not built inside Kannel anymore. That way a benchmark/test of million or even 10 million messages can be tested out. BTW, the test showed that checking_malloc Kannel has speed of about 20-25% of native malloc.
RE: Incoming concatenation..
On Fri, 26 Oct 2001, Anders Lindh wrote: > > Ok.. What would be required to implement the functionalty? Basically, > I'd need to atleast get the contents of the messages. A "strip-udh" > setting in kannel.conf could be implemented, this way the dirrent parts > of long messages could be saved. Any thoughts? Whenever an SMS with concatenation set on arrives, put it aside until all parts have been received. Then when the entire message has been received, do whatever merge/etc. is needed. In this case, a suitable plcae for bundling would be smsbox, I guess. Of course if some parts never arrive the rest are forever queued, but that is more like a fundamental question in multipart messages. &Kalle
RE: EMI2 fix for multi recipients
On Tue, 13 Nov 2001, Jacob Vennervald Madsen wrote: > So when I use kannel I should make one http request per recipient? Yes, but see below. > Has anybody tried to send messages to more than 10.000 recipients? > How does it perform? > I mean it does create a huge network overhead. If you have this kind of mass-sending needs, I recommend adding a new content-type like application/Kannel-MassSending ;) to sendsms POST interface. Then just use some format in body, like receiver:content or whatever and this way submit them with just one HTTP call. -- &Kalle
Re: pthread/poll problem
On Fri, 11 Jan 2002, Stipe Tolj wrote:
> Nevertheless both sms and wap functionality seems very stable on
> Cygwin. Andrea, can you expience the poll error messages on your
> productive environment?
Speaking about Cygwin, how fast it is? I mean, is there any big
performance difference in linux version against cygwin in W2k?
--
&kalle marjola
Re: loadable modules ?
On Fri, 22 Feb 2002, Stipe Tolj wrote: > Alexei Pashkovsky wrote: > > > > I think idea of having the listening http interface inside bearerbox instead > > of running two executables could be usefull too. > > Obviously this should be configurable .. > > that's not how Lars designed it. No, but he has also said that the entire architecture should be re-thinked now when more the exact 'goal' is better known (ref: a conversation with him a few minutes ago) - similar reason why I did merge smsbox and bearerbox last summer, but which was a bit too drastic modification to the main project. As a sidenote, old Kannel had an option of having "thread" smsbox, i.e. instead of sending messages to seperate executable they were handled inside bearerbox. When the message queue/flow system was redesigned, I was too lazy to re-write that thread version... (althought actually it was so simple that instead of callin some write_to_box function the message was just put directly to receive queue...) So, it should be quite simple to re-implement 'thread' smsbox even if a separate smsbox is kept... (but I still feel that the advantages gained from that is far less than the extra problems caused, especially with DLR's etc... Kannel is anyway faster than normal HTTP servers or SMS center connections.. and that protocol between bearerbox and smsbox is not that faster than HTTP protocol...) PS: Loadable modules: a great idea -- &kalle marjola
RE: [BUG] list implementation too slow.
On Thu, 21 Mar 2002, Oded Arbel wrote:
> I think this problem occurs as the bearerbox can't send messages as fast
> as they are delivered (probably due to a large burst and then sustained
> high load). I see messages queued on the module's queue, and then it
> slows doesn the module more, so it can handle less messages (while high
> load continues) and so the queue grows longer.
The solution to this is to fix all code so that outgoing messages
have higher priority than incoming ones, i.e. do not read anything
from bufefrs before outgoing messages have been sent. NOTE: You need
to set conn() buffer size, otherwise this does not help as things
get added to its internal buffer... I did this for my modified
version and got rid of growing size (in memory) and slowdown.
(however, I do not know how SMSC would react if the socket gets filled up
because the other end is not reading it fast enough..)
--
&kalle marjola
Windows Kannel?
Yeah, I can easily remember that this question has been arisen here quite a couple times, but perhaps someone has some updated info; Has anybody ported Kannel to Windows, without using Cygwin but using native window solutions for threads etc.? I guess that would need at least rewriting some parts of the gwlib, mainly thread implementation. Then smsc_at would also need some serious hacking. Yeah, I guess cygwin would be sufficient but some people keep asking, can it be done without cygwin there - and I guess cygwin isn't the most efficient one if we want to have high throughput -- &kalle marjola
Re: Windows Kannel?
On Wed, 15 May 2002, Stipe Tolj wrote: > Kalle Marjola wrote: > > > > Yeah, I guess cygwin would be sufficient but some people keep asking, can > > it be done without cygwin there - and I guess cygwin isn't the most > > efficient one if we want to have high throughput > > Cygwin is of course not as fast as a native port would be, but > comparing the burdons in porting this may be a good trade-off IMO. Naturally, but those people keep asking me so I'd love to have some estimation/etc. to base that opinion on :] (like: a rought guess is that 3 man-months would be needed to do the porting, do you still think it is worth it?) (I personally know nothing about windows programming) -- &kalle marjola
Re: EMI: Serious Problem PANIC: Too many concurrent allocations
On 24 May 2002, Abd Rahman Johari wrote: > I'm trying to send out SMS Broadcast and after a while kannel just die > because of PANIC: Too many concurrent allocations You need to compile Kannel with non-checking malloc, i.e. use native malloc. ./configure --with-defaults=speed or --with-malloc=native should help. Hm, this question needs to be a s a first one in FAQ, it is so often asked.. (in fact I could not find it from FAQ section at all..) -- &kalle marjola
Re: EMI: Serious Problem PANIC: Too many concurrent allocations
On Sat, 1 Jun 2002, Jari Juslin wrote:
> So, panic to "Too many concurrent allocations" can happen if a) Kannel
> just has so high load, that is needs huge amounts of memory or b) if
> Kannel leaks too much memory. Am I right?
Yes. The reason a) is encountered when incoming traffic is faster than the
outgoign handle (namely SMS centers) and usually results in that
'too many concurrent allocations'. This can be avoided by increasing the
size of the checking malloc buffers, but if things keep going like that it
will eventually panic. Moreover, store files of several megabytes is more
than a problem. This entire thing should be fixed so that more priority is
given to outgoing messages, i.e. just make it impossible to grow queues
inside Kannel - just make the queues to build up into sockets/other
programs. Of course this is far from trivial with some SMS centers.
--
&kalle marjola
Re: Message store (Was: EMI: Serious Problem PANIC: Too manyconcurrent allocations)
On Wed, 12 Jun 2002, Aarno Syvänen wrote: > Speaking of store files - After really looking into the store logic, it The better way to implement the store file (than the old way I did quickly use - you have to remember that original store file was just a quick hack to concentrate more on SMS side that almost-useless-WAP ;) - like earlier, write all messages and acks and nacks to file - keep copy of all non-acked messages in memory - when Kannel receives an ack (or nack), write a new store file out of still remaining (non-acked) messages to replace the old. Naturally only do this now and then, like if there has not been writing in last half a minute. (and of course first write to new file and then rename etc.) ..this way the store file normally keeps empty or only some odd messages, unless lots of messages do not get (n)acks. Of course this means that Kannel will grow in memory if messages do not get ackked, too. -- &kalle marjola
RE: Message store (Was: EMI: Serious Problem PANIC: Too manyconcurrentallocations)
On Thu, 13 Jun 2002, Oded Arbel wrote: > You also don't need to store acks in the store, as only unacked messages > are stored there. But note that if you do not store them, you have to write the entire store each time you get ack or if your system goes down between two writes, it will resend all messages ackked after the last write. -- &kalle marjola
Re: Problem with SMPP smsc-password being too long
On Mon, 8 Jul 2002, Stipe Tolj wrote: > contiue? why? the password is logically longer than 8 octects, but the > SMPP spec does not accept it to put it in the appropriate PDUs. So > what should you do then? Cut the Password to 8 chars long and send the > login sequence anyway? No, I don't think so. > > The assertion check fails, and a failing assertion check means a non > recoverable state within the system, hence panic! This is my opinion > about the functional logic. But if you want to implement reloading of the configuration, non-fatal errors should not cause panic (in fact, no error except running out of memory, perhaps. Just revert to old configuration if the new one is unacceptable). And this error is not fatal, it simply means that this SMPP connection cannot be used (well, sort of). If you want to exit because of faulty configuration that is okay, but panic should be avoided as long as possible (and it is horror to debug panics ;) -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: old smsc's and smsc_wrapper
On Tue, 9 Jul 2002, Kaido Karner wrote: > does anybody use any smsc's which use old format smsc modules (CIMD, CIMD2, > EMI, EMI_IP, SEMA, OIS, AT)? I use CIMD2. any volunteers to convert at least > CIMD2 to new implementation? :) I have some porting of cimd2 to smscconn (and at) and have discarded rest. Has there been lots of changes to cimd2 recently? I havenät checked that version for some time, so there might be some updates needed. And I'm not guarenteening its behaviour, but so far no problems. -- &kalle marjola
smsbox
How many of you really use the feature of independent smsbox in different machine than the bearerbox? (or several smsboxes). Just thinking about the old 'thread smsbox' which was in old Kannel... -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: [PATCH] sms centers
On Fri, 9 Aug 2002, Stipe Tolj wrote: > > PPS: Anyone else noticed that Cygwin has some serious thread/signal > > problems? > > yeah, but it should be fixed in 1.3.12-2 and up. I claimed these to > the core Cygwin people. (I'm also the official Apache Cygwin port > maintainer, so the problems arrised from several sides) But they > didn't react up to now. Well, it is not fixed in 1.3.12-2, that is why I asked :] Hm, obviously I need to bug cygwin people -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Stipe Tolj wrote: > > How many of you really use the feature of independent smsbox in different > > machine than the bearerbox? (or several smsboxes). Just thinking about the > > old 'thread smsbox' which was in old Kannel... > > we do, at least in some extend. Are you planing to incorporate smsbox > as bearerbox thread?! Well, several threads... (and this is just speculation. Last time the idea was torpedoed down because of load distribution reasons, after I have done some tests etc.) > If yes, we dislike this because of several reasons, one of them is the > abilitiy to make config changes 'on-the-fly' to smsbox related groups, > while bearerbox is still running. Do you have some nifty way to take smsbox down for that use, so that no messages are lost while they are being transfered between bearerbox and smsbox? (or open http connections) Dynamic configuration loading is doable (been there...), so what are those other reasons? (it is amazing how much it simplifies things to have smsbox in same process. And makes Kannel much faster) (for distribution, the possibility to run distinct smsbox could be left, too, alhought I do not see much use for that...) -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Harrie Hazewinkel wrote: > > Do you have some nifty way to take smsbox down for that use, so that no > > messages are lost while they are being transfered between bearerbox and > > smsbox? (or open http connections) > > (pure speculating here) A way could be to have the smsbox doing some > gracefull restart by reading the configuration while the connections > (sockets) stay connected. Then messages would get queue up. Unless > the restart takes very long you would no loose messages. Not sure how > the timing/speed is for this. Well my question was not about how it can be done but how they do do it right now :] > > (it is amazing how much it simplifies things to have smsbox in same > > process. And makes Kannel much faster) > > Just curious, but what about the wapbox?? Would that go faster too?? Possibly, but with wap, you do not need to say 'ack' to each message like with smsc > Yes, dynamic configuration is possible via various ways, SNMP, HTTP > or even just some kind of gracefull restart that reads the configuration > and setups the new environment. What I meant here is that dynamic configuration loading inside Kannel without stopping the smsc connections can be done (quite easily), thus effectually same thing as never stopping bearerbox but restarting smsbox > > (for distribution, the possibility to run distinct smsbox could be left, > > too, alhought I do not see much use for that...) > > Only when the load can go so sky high that a singl machine cannot keep up Naturally. But show me the single SMSC connection that can do that. And if you have several SMS center connections, you can have several bearerboxes, too (yes, I know that if you need to reoute things between them, this can still cause problems. That is just what I'm asking here, does anyone really USE that or is it just speculation that it CAN be done and used? Is it worth all the pain of message/ack/dlr transfer between smsbox and bearerbox?) Usually the speed of applications (oh, the java servlets) and SMSC connections is the only factor of the speed. The core Kannel can handle like 1000-2000 msg/s of fixed answers in normal modern tabletop, like 500-1000 HTTP based services (if the server is fast enough) SMS centers usually do something between 10-50/s, java servlets less. Correct me if I'm horribly wrong -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Stipe Tolj wrote: > reload of configuration on the fly?! Hmm, that's new to me, what > signal are you using and for what box? I mean that can be done, I have done some tests, and it used admin command. > Do you have any accurate benchmarks comparing the two processes? Guess... ;] But I could arrange.. > Maybe we can keep the code that clean that we can have a configure > switch to incorporate the smsbox into bearerbox as thread?! I think > this would be perfect for all of us. But that wouldn't help for intra-Kannel logistics, sould it? :] > BTW, I didn't got the point how bearerbox knows to which smsbox it has > to route inbound messages? Or is it passing a copy to all connected > smsboxes?` All smsboxes are equal, and thus bearerbox sends the message to random one, or the one with lowest load (cannot recall the exact algorithm, use the source...) -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Stipe Tolj wrote: > > Well my question was not about how it can be done but how they do do it > > right now :] > > we do terminate and restart smsbox to re-read the updated > configuration. Okay, just curious about possible open HTTP connections etc. so they do not cause problems? (like lost replies later on requeed with store system) > > What I meant here is that dynamic configuration loading inside Kannel > > without stopping the smsc connections can be done (quite easily), thus > > effectually same thing as never stopping bearerbox but restarting smsbox > > which is what we do. > > But you still have to stop and restart bearerbox if you change smsc > groups (adding or modifying) and hence those who are not affected have > to go down and up too. Yes, the same way it goes with the integrated one. > I'm also thinking of problems arrising from the planed modularization. > As far as I see we will have modularization (which is already there) > for SMSC modules and SMS services (i.e. sendsms, sendota, > sendringtone, sendlogo etc). which means if someone links in a SMS > service module (into smsbox scope) and smsbox is implemented as > threads inside bearerbox and the new code dumps core, the whole > systems blows up. Which is not the way we would like to see it. That is a valid point. If I start to speak ideally, I would get rid of all such services and instead implement just one simple http sender/receiver and all those services would be done by external filter/proxy/whatever. No idea to make basic _gateway_ an application server. But that is of course 'if I was the king of the world and would do redo everything' -talk. Things that aleady exist are hard to move away (like those prefix-suffix things in sms-service groups.. one in 200 uses them and thus they cannot be removed and left to be application problem..) -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: smsbox
On Fri, 9 Aug 2002, Stipe Tolj wrote: > > > reload of configuration on the fly?! Hmm, that's new to me, what > > > signal are you using and for what box? > > > > I mean that can be done, I have done some tests, and it used > > admin command. > > AFAIK, you can not re-read the config file on the fly?!?! Not with CVS Kannel. With patched, yes. (but do not ask for it, at least yer, it is quite a mess and has all these toher thigns I'm talking about here, so it is not compatible) > if we astract the logical bearerbox <-> smsbox communication so the > configure switch does select then the accurate implementation > (function or tcp calling) this would work. Think of MySQL-4's new way > to link the whole mysqld engine to your client executable, I think > they use the same approach. Hm, that could work.. > > All smsboxes are equal, and thus bearerbox sends the message to random > > one, or the one with lowest load (cannot recall the exact algorithm, > > use the source...) > > Hmm, that's bad in some sence. Well, it has been like this for all these 3 or so years and no one has ever complained... (but I doubt that anyone has used more than one smsbox except for testing purposes :) PS: Now that I at last remember: please do not send me double copies each time you reply to my emails. I'm on devel list, too, no need to send personal reply (hm, am I starting to sound like Lars?) -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: [PATCH] AT2 rewrites (resubmit)
> idents are 4, tabs are 8. This is probably because editors like vi > have a default of 8 but you really wana have 4 to write the code. So > you use "soft tabs" which make you edit like you have tabs at 4 but > save it like tabs are at 8. > > Personally I would prefer to have tabstops at 4 and idents at 4. > Would make life much easier in any editor I guess. My editor (BBEdit) > doesnt have virtual tabstops so I have to do it manually and I hate > it. > > Would it be too keen to decide to change that rule? The idea in that is that most of the computer systems in the world has been configured so that tab is 8. If we change that, then all kind of cat, more, less, print etc. outputs of the source code get f*d up unless tab size of those is also modified to 8. Why can't you people just get an editor that auto-indents with spaces? :] -- &kalle marjola NETikos finland (http://www.netikos.fi)
RE: POST to /cgi-bin/sendsms
On Wed, 4 Sep 2002, Anarchy wrote: > i tried with a small page: > http://*.*.*.*:13013/cgi-bin/sendsms"; method="post"> > > > The problem here is that this creates an HTTP call of something like... (headers) (body) username = * password = * ..or whatever the browser POST does. The correct way to use Kannel POST is, as far as I recall, to provide all the additional information _in_headers_, thus: (headers) X-Kannel-Username: * X-Kannel-Password: * (body) At least this was how I coded it 1.5 years ago... (as a sidenote, there might be support for www-form-urlecoded nowayads, so any expert can correct me if I am only talking ancient history here...) -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: merging smsbox and bearerbox
This is going to be a bit long mail. Background: As I have now and then announced, I (long time ago) merged smsbox and bearerbox together, to make Kannel faster, easier to program (DLR's, store-files, dynamic configurations etc.). As a part of this project, we made list of reliability improvements. The main aims of this project were: - to make Kannel as fast as possible, so that the speed _always_ depends on external things (SMSC, HTT server) speed - to make system as reliable as possible (reliable store-file, steady memory use etc.) As a result, there is extend modifications to some parts (sms services) while some are as they were (newer smsc connections, wap parts). Now the politics here at NETikos have changed (well, not the only thing that have changed) so we look forward to integrate various features to Kannel CVS tree and partake more in the project. Of course there is that problem that most of these 'improvements' are tied to that new architecture (merged smsbox) and doing them otherwise might not be possible or not worth it. So, is the merging worth it? Well I finally found time to run some benchmarks, here is results briefly (see the attachment for more results): The benchmarking was not easy thing, and it is NOT a very good one either, as the problem is how to benchmark just one thing. As the merged Kannel (our version) has in addition to that merge, a better store-file and throttle system for fakesmsc, it resulted in level results regardless what I threw to it - it did not eat the resources of the system nor slowed down. With Vanilla Kannel (fresh CVS) I have to put store-file away etc. So this benchmark not only tested the speed difference of merged smsbox and bearerbox versus separate boxes, but other parts, too. I tried to minimize the effect of other parts and thus ran several tests with various settings. Well, the full results are in the attachment. Short conclusion: It is clearly faster to use merged system. However, as a sidenote, the inter-box speed etc. are not the main issue here, unless building up a very, very special system (where there is more than 1000-2000 msg/s of incoming traffic). However, the merged system made it far more easier to implement throttle system (for incoming messages and HTTP) and improved store-file, which resulted that this merged version could handle any loads without slowing down or eating all the resources. Vanilla Kannel was killed by OS after swap ran out or it got bogged down by the store-file. But that is another store does that happen in real world or just in benchmarks :] As last words, yes we have various little bits and in upcoming weeks, we investigate how we can merge them into CVS tree and then provide patches. Hopefully as many as possible, but some might become difficult because of the above differences and limited time to do bigger modifications to make them work.. -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi) Setup: Vanilla Kannel (fresh CVS) vs. modified Kannel (merged bearer- and smsbox, throttle system to fakesmsc driver, improved HTTP) System: Linux on tabletop Linux. No store-file. Merged Kannel has better store-system. Vanilla Kannel got completely frozen if store-file was used with high incoming message traffic. --- 1. fakesmsc and bearerbox/smsbox all in same machine, sending 'nop' messages at interval of 0.0001 (1/s) (note: fakesmsc binary of own distribution. But I guess they are the same) Vanilla Kannel: (1) 2002-09-17 09:17:42 [0] INFO: fakesmsc: from first to last sent message 1.4 s, 7308.5 msgs/s 2002-09-17 09:17:42 [0] INFO: fakesmsc: from first to last received message 4.7 s, 2120.6 msgs/s (10) 2002-09-17 09:04:29 [0] INFO: fakesmsc: from first to last sent message 13.4 s, 7441.1 msgs/s 2002-09-17 09:04:29 [0] INFO: fakesmsc: from first to last received message 50.0 s, 1999.2 msgs/s NOTE: memory growth (to 25 megabytes) (100) Couldn't handle, too much memory growth. Killed by operating system. Same happened even with interval = 0.0004 (2500 msg/s) Merged Kannel: (1) 2002-09-17 07:02:09 [0] INFO: fakesmsc: from first to last sent message 1.0 s, 9864.4 msgs/s 2002-09-17 07:02:09 [0] INFO: fakesmsc: from first to last received message 2.7 s, 3711.5 msgs/s NOTE: with store-file, ~2800 msg/s (10) 2002-09-17 07:03:12 [0] INFO: fakesmsc: from first to last sent message 17.7 s, 5650.2 msgs/s 2002-09-17 07:03:12 [0] INFO: fakesmsc: from first to last received message 27.6 s, 3623.6 msgs/s NOTE: with store-file, ~2830 msg/s (100) 2002-09-17 07:08:24 [0] INFO: fakesmsc: from first to last sent message 252.8 s, 3955.7 msgs/s 2002-09-17 07:08:24 [0] INFO: fakesmsc: from first to last received message 262.7 s, 3807.2 msgs/s
Re: merging smsbox and bearerbox
> > please make the patches as readable as possible, which means keep the > diff files semantically together, so that every patch does change one > semantical issue. The main problem here is the entirely updated inner structure. This means that we talk hundreds of lines of code here, which cannot be separated that easily - for example, we cannot use improved store-file without changing message queue system. Earlier when we talked about this bearerbox/smsbox issue, one reason to have separate smsbox was to keep unstable special message things (OTA etc.) out of bearerbox, so that if they fail, bearerbox is not affected. This arises again the very question 'what should Kannel (gateway) do?' In my opinion, the gateway should: - be as reliable as possible (no memory leak, no crashing, no panics, no lost messages) - act as fast tunnel between user agents and applications - provide minimal features (API) to make it usable In this sense, gateway could have following things: - SMSC connection handling (this is the main thing wanted from it) - simple message routing to SMSCes - virtual SMSC handling (for HTTP based content gateways, external SMSC connection systems etc.) - simple message routing to external HTTP servers - simple message concatenation (or support to force used SMSC) - OPTIONAL: fixed answers for time-critical things Following things should not be there: - complicated message handling (logos, icons, ringtones, OTA,..) -> these messages should be created by other process which then uses the simple interface of the gateway - message modifications (headers, footers, etc.) - unstable drivers etc. - wap stack (seperate wapbox is good) Thus, I would move basic SMS handling to bearerbox and then leave more complicated (push etc.) things to smsbox. Like make smsbox completely stand-alone, so that it is not needed but can be used if additional features are needed. In practise, to make things work 'like they were', bearerbox would be configured to send all SMS'es to this new smsbox (maybe over HTTP?) which would then do routing/modifications, and push any replies (or return as reply to bearerbox HTTP) to bearerbox as any external application. Bearerbox would not see difference between that stand-alone smsbox and any other application, making it simpler. But this stand-alone smsbox could be provided with Kannel, to make easier to develop applications. -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
smscconn cimd2
The attached patch changes cimd2 to use SMSCConn instead of old SMSCenter.
However, notes:
- diffed against latest CVS version (just did cvs udpate)
- not tested! We do not have cimd2 currently available. Compiles, however
=]
- had some problem with merge, so it could have been made more efficiently
(in original version, only PrivData was moved around, not entire
SMSCConn, but DLR merge required modifications)
- does not fully utilize SMSCConn possibilities, it is more like
smsc_wrapper moved to cimd2. But it is easier to start using them after
this works (and get rid of smsc_wrapper...)
- includes some notes of various extra code which I couldn't include in
the patch as then it would require patching of yet another big pile of
files. However, if this can be merged, they might come next.
--
&kalle marjola
product concept manager
NETikos finland (http://www.netikos.fi)
Index: gw/smscconn.c
===
RCS file: /home/cvs/gateway/gw/smscconn.c,v
retrieving revision 1.32
diff -r1.32 smscconn.c
74a75,76
> else if (octstr_compare(smsc_type, octstr_imm("cimd2")) == 0)
>ret = smsc_cimd2_create(conn, grp);
Index: gw/smscconn_p.h
===
RCS file: /home/cvs/gateway/gw/smscconn_p.h,v
retrieving revision 1.32
diff -r1.32 smscconn_p.h
180a181,183
> /* Responsible file: smsc/smsc_cimd2.c */
> int smsc_cimd2_create(SMSCConn *conn, CfgGroup *cfg);
>
Index: gw/smsc/smsc.c
===
RCS file: /home/cvs/gateway/gw/smsc/smsc.c,v
retrieving revision 1.1
diff -r1.1 smsc.c
76,87d75
< /* CIMD 2 */
< smsc->cimd2_hostname = NULL;
< smsc->cimd2_port = -1;
< smsc->cimd2_username = NULL;
< smsc->cimd2_password = NULL;
< smsc->cimd2_send_seq = 1;
< smsc->cimd2_receive_seq = 0;
< smsc->cimd2_inbuffer = NULL;
< smsc->cimd2_received = NULL;
< smsc->cimd2_error = 0;
< smsc->cimd2_next_ping = 0;
<
158,165d145
< /* CIMD 2 */
< octstr_destroy(smsc->cimd2_hostname);
< octstr_destroy(smsc->cimd2_username);
< octstr_destroy(smsc->cimd2_password);
< octstr_destroy(smsc->cimd2_inbuffer);
< octstr_destroy(smsc->sender_prefix);
< list_destroy(smsc->cimd2_received, NULL);
<
216,220d195
< case SMSC_TYPE_CIMD2:
< if (cimd2_submit_msg(smsc, msg) == -1)
< goto error;
< break;
<
272,277d246
< case SMSC_TYPE_CIMD2:
< ret = cimd2_receive_msg(smsc, msg);
< if (ret == -1)
< goto error;
< break;
<
336,341d304
< case SMSC_TYPE_CIMD2:
< ret = cimd2_pending_smsmessage(smsc);
< if (ret == -1)
< goto error;
< break;
<
503,504d465
< else if (octstr_compare(type, octstr_imm("cimd2")) == 0)
< typeno = SMSC_TYPE_CIMD2;
579,590d539
< case SMSC_TYPE_CIMD2:
< if (host == NULL || port == 0 || username == NULL || password == NULL)
< error(0, "Required field missing for CIMD 2 center.");
< else
< smsc = cimd2_open(host,
< port,
< username,
< password,
< keepalive,
< sender_prefix);
< break;
<
708,710d656
< case SMSC_TYPE_CIMD2:
< ret = cimd2_reopen(smsc);
< break;
754,758d699
< errors = 1;
< break;
<
< case SMSC_TYPE_CIMD2:
< if (cimd2_close(smsc) == -1)
Index: gw/smsc/smsc_p.h
===
RCS file: /home/cvs/gateway/gw/smsc/smsc_p.h,v
retrieving revision 1.1
diff -r1.1 smsc_p.h
26d25
< SMSC_TYPE_CIMD2,
92,104d90
< /* CIMD 2 */
< Octstr *cimd2_hostname;
< int cimd2_port;
< Octstr *cimd2_username;
< Octstr *cimd2_password;
< int cimd2_send_seq;
< int cimd2_receive_seq;
< Octstr *cimd2_inbuffer;
< List *cimd2_received;
< int cimd2_error;
< time_t cimd2_next_ping;
< Octstr *sender_prefix;
<
199,208d184
<
< /*
< * Interface to Nokia SMS centers using CIMD 2.
< */
< SMSCenter *cimd2_open(Octstr *hostname, int port, Octstr *username, Octstr
*password, int keepalive, Octstr *sender_prefix);
< int cimd2_reopen(SMSCenter *smsc);
< int cimd2_close(SMSCenter *smsc);
< int cimd2_pending_smsmessage(SMSCenter *smsc);
< int ci
Re: smscconn cimd2
On Thu, 26 Sep 2002, Stipe Tolj wrote: > Kalle, have you tested this any vote for commitment? Ok, I'm at least > 0+ while untested, because Kalle is an core developer and I think he > knows what he is patching :)) This have been used against CIMD2 without DLR parts. However, the new patch with DLR things merged to it have not been tested and that is where something might have gone wrong... (and I just removed charset (unicode) parts, as the legality of them is still unclear and needs patching to other parts, too, so that removal can also break things in CIMD2) -- &kalle
Kannel archtitecture
Yet again I return to this subject of Kannel internal architecture thing. This time with something very concrete; available from my web page is now the entire modified version of Kannel we have made mainly during last Fall and Winter, with list of features we would like to include to Kannel. However, as told earlier, many of these changes are quite radical and thus not very easily portable. Now, instead of trying to port them neverthless, I propose something more radical: Move the entire Kannel to use our core engine, then port all new features in Kannel to that core engine (SQL interface, new AT2, all kind of special messages, DLR's (see notes below) etc.) Why this? Because we feel that the new design is more robust, stabile and faster. It has better SMS storing system, faster SMS handling, condiguration loading etc. And as most new features in Kannel are just new modules which use the interfaces I defined 2 or so years ago, so they should be easily movable (I dunno about DLRs, but it is mainly in smscconn, after all.. we also have our system to handle DLR's, it is a bit different) The biggest system-wise difference is that we have merged smsbox and bearerbox, while Kannel has separate. For this I propose that we can have BOTH: allow that it can be configured to use external smsboxes or just use internal handling (or maybe both?). We also have a bit different configuration file syntax for sms-services but that is just details.. Yes, this sounds heavy and very drastic. But I hope you can at least think about it, and think hard. Old Kannel developers at least can somehow trust that I and Uoti know what we have been doing :] The original modified version is available for you to test (but for possible copyright reasons in some parts I won't announce the place here, so send me email) so investigate it and think about this issue. Feel free to say your opinion how easily some things can be ported to main Kannel tree, too. -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: Kannel archtitecture
> I liked having a seperate smsbox though. Even with the option of keeping the > external smsbox or using the internal, we won't have the new features. These features are not heavily related to smsbox/features, and that is just details.. > Also I have question marks about the copyright thingy. This is the infamous Unicode thing. (opens the Dusty Book of Kannel History and starts to tell a tale... ;) Long, long time ago we at Wapit did Unicode support for Kannel. However, for certain reasons we were not allowed to publish it immediately. And then the bankcrupty hit us (WapIT), and thus it (Unicode support) was never released as open source. After that, various parties bought "non-exclusive" rights to that Unicode from bankcrupty estate, including NETikos. So, in theory, someone MIGHT get mad if that code is released as Open Source. People at MatchEm were investigating the possibility to publish it neverthless, but then the development in MatchEm was dropped (afaik). On the other hand, that Unicode code is quite simple, any good programmer would do it in a day or two, at least internal parts (not SMSC), the main work was in decising HOW to do it.... -- &kalle marjola product concept manager NETikos finland (http://www.netikos.fi)
Re: Kannel archtitecture
> The original modified version is available for you to test (but for > possible copyright reasons in some parts I won't announce the place > here, so send me email) so investigate it and think about this issue. > Feel free to say your opinion how easily some things can be ported to > main Kannel tree, too. Some days have passed and so far I have received some comments about this; however, more feedback would be nice to have and especially on this list - what do you people think? -- &kalle marjola
Re: Reloading configuration
On Wed, 11 Dec 2002, Stipe Tolj wrote: > Shridhar Raju wrote: > > > > Does it possible to reload configuration without restarting, either by > > isolating and suspending kannel gateway? > > no, unfortunatly now. > > Kalle told us that the netikos version does this. Kalle, is there any > efforts from your side to integrate these thins into the official > tree?! No, too much work, too many differences, we have no resources to do that. As I have stated, most of our things require rework of many internal parts and if that is not done/used, integrating those other things are very close to rewriting them entirely. But we can 'release' that code to be freely used by the Kannel project as it sees fit, apart from unicode part (distribution rights to that part are still owned by Wapit "estate"), if someone wants to start hacking. -- &kalle marjola
Re: smscconn cimd2
On Thu, 6 Feb 2003, Stipe Tolj wrote: > Kalle Marjola wrote: > > > > The attached patch changes cimd2 to use SMSCConn instead of old SMSCenter. > > However, notes: > > > > - diffed against latest CVS version (just did cvs udpate) > > - not tested! We do not have cimd2 currently available. Compiles, however > > =] > > - had some problem with merge, so it could have been made more efficiently > > (in original version, only PrivData was moved around, not entire > >SMSCConn, but DLR merge required modifications) > > - does not fully utilize SMSCConn possibilities, it is more like > > smsc_wrapper moved to cimd2. But it is easier to start using them after > > this works (and get rid of smsc_wrapper...) > > - includes some notes of various extra code which I couldn't include in > > the patch as then it would require patching of yet another big pile of > > files. However, if this can be merged, they might come next. > > can you resend the patch please in diff -u format against current cvs > tree?! Thanks in advance. No. The patch required some work to do from our CVS version and that final version is only at my desktop computer - which currently does not boot (disk crash) and I'm about 3000 kms away from fixing it. -- &kalle marjola
Re: multiple bearer box ?
On Thu, 20 Feb 2003, Aarno Syvänen wrote: > I once stress tested stand-alone bearerbox. (Standalone meaning I was > using > smsboxes that did not do any fetch, but returned a standard text message > instead.) I got value 1200 msg/s and must use ~20 smsboxes before I > reached 100 % cpu utilisation. Cpu was AMD Athlon, 800 MHz. With this 'a bit optimized' merged bearerbox and smsbox of ours, I recall values of 1000-2000 msg/s in modern workstation (fakesmsc/simple answer). It is good to notice that main things that create bottlenecks are: - smsc drivers/smsc itself - smsbox utilities (http library) The current Kannel is, however, made in that way that it gets trashed if it receives messages faster than it can handle, so beware :] -- &kalle marjola
Re: multiple bearer box - Netikos?
On Sun, 23 Feb 2003, Stipe Tolj wrote: > Nisan Bloch wrote: > > > > I think rolling NMGW into Kannel would be worthwhile. At the same time come > > up with a combined bbox+smsbox version, with the same HTTP interface. In > > addition an API to build XXXboxs with. Smsbox would be the first such app > > and maybe the smppbox and emibox that have been mentioned on the list. > > definetly. > > We should merge things from Netikos version into official Kannel. > > Main question would be how many efforts Netikos is *still* doing in > continued development of their branch? Kalle? Practically nothing has been done since I pusblished that code. > You guys could switch off to the official tree and get your great > things included there. This would cause a benefit for all. That's why I published them, altought I knew that some things are a bit too radical. As we have no resources to do any development on it right now, I hope that you can scavenge useful things out from it and this way improve the Kannel project. -- &kalle marjola
