Re: merging smsbox and bearerbox

2002-09-19 Thread Kalle Marjola

> 
> please make the patches as readable as possible, which means keep the
> diff files semantically together, so that every patch does change one
> semantical issue.

The main problem here is the entirely updated inner structure. This means 
that we talk hundreds of lines of code here, which cannot be separated 
that easily - for example, we cannot use improved store-file without 
changing message queue system.

Earlier when we talked about this bearerbox/smsbox issue, one reason to 
have separate smsbox was to keep unstable special message things (OTA 
etc.) out of bearerbox, so that if they fail, bearerbox is not affected.
This arises again the very question 'what should Kannel (gateway) do?'
In my opinion, the gateway should:
 - be as reliable as possible (no memory leak, no crashing, no panics,
no lost messages)
 - act as fast tunnel between user agents and applications
 - provide minimal features (API) to make it usable

In this sense, gateway could have following things:
 - SMSC connection handling (this is the main thing wanted from it)
 - simple message routing to SMSCes
 - virtual SMSC handling (for HTTP based content gateways, external SMSC 
   connection systems etc.)
 - simple message routing to external HTTP servers
 - simple message concatenation (or support to force used SMSC)
 - OPTIONAL: fixed answers for time-critical things

Following things should not be there:
 - complicated message handling (logos, icons, ringtones, OTA,..)
-> these messages should be created by other process which then uses 
   the simple interface of the gateway
 - message modifications (headers, footers, etc.)
 - unstable drivers etc.
 - wap stack (seperate wapbox is good)

Thus, I would move basic SMS handling to bearerbox and then leave more 
complicated (push etc.) things to smsbox. Like make smsbox completely 
stand-alone, so that it is not needed but can be used if additional 
features are needed. In practise, to make things work 'like they were',
bearerbox would be configured to send all SMS'es to this new smsbox
(maybe over HTTP?) which would then do routing/modifications, and push any 
replies (or return as reply to bearerbox HTTP) to bearerbox as any 
external application. Bearerbox would not see difference between that 
stand-alone smsbox and any other application, making it simpler. But this 
stand-alone smsbox could be provided with Kannel, to make easier to 
develop applications.


-- 
&kalle marjola
product concept manager
NETikos finland (http://www.netikos.fi)





Re: merging smsbox and bearerbox

2002-09-17 Thread Nisan Bloch

Hi Kalle..

wow.. ;-).. i have been wanting to either merge smsbox and bearerbox or 
mere smsbox with our apps...  This sounds great!.. It would be great to 
have a list of the major differences, especially (well for me) with respect 
to the smsc modules and control thereof. How different are the 
implementations? Has Netikos kept up with the current CVS in anyway?

looking forward to this

nisan





Re: merging smsbox and bearerbox

2002-09-17 Thread Stipe Tolj

Kalle Marjola wrote:
> 
> As a result, there is extend modifications to some parts (sms services)
> while some are as they were (newer smsc connections, wap parts). Now
> the politics here at NETikos have changed (well, not the only thing that
> have changed) so we look forward to integrate various features to
> Kannel CVS tree and partake more in the project.

first of all we're happy to have you back again as active contributor
:)

> As last words, yes we have various little bits and in upcoming weeks, we
> investigate how we can merge them into CVS tree and then provide patches.
> Hopefully as many as possible, but some might become difficult because
> of the above differences and limited time to do bigger modifications to
> make them work..

please make the patches as readable as possible, which means keep the
diff files semantically together, so that every patch does change one
semantical issue.

I guess we should first observce the patches and then descide in the
group how to handle them concerning inclusion to the official tree.

BTW, we have implemented an SMPP server on Kannel's gwlib base that
does connect to bearerbox as a 'smsbox'. So I'm a bit critical
conerning having one big bearerbox+smsbox. But I guess the same that
applied for incorporating smsbox to bearerbox may be done to our
smppbox too. It's currently in experimental production state and we
have been planning to make this public too in some time.

Stipe

[EMAIL PROTECTED]
---
Wapme Systems AG

Vogelsanger Weg 80
40470 Düsseldorf

Tel: +49-211-74845-0
Fax: +49-211-74845-299

E-Mail: [EMAIL PROTECTED]
Internet: http://www.wapme-systems.de
---
wapme.net - wherever you are




Re: merging smsbox and bearerbox

2002-09-17 Thread Kalle Marjola


This is going to be a bit long mail.

Background: As I have now and then announced, I (long time ago) merged 
smsbox and bearerbox together, to make Kannel faster, easier to program 
(DLR's, store-files, dynamic configurations etc.). As a part of this 
project, we made list of reliability improvements. The main aims of this 
project were:
 - to make Kannel as fast as possible, so that the speed _always_ depends 
   on external things (SMSC, HTT server) speed
 - to make system as reliable as possible (reliable store-file, steady 
memory use etc.)

As a result, there is extend modifications to some parts (sms services) 
while some are as they were (newer smsc connections, wap parts). Now
the politics here at NETikos have changed (well, not the only thing that 
have changed) so we look forward to integrate various features to 
Kannel CVS tree and partake more in the project. 

Of course there is that problem that most of these 'improvements' are tied 
to that new architecture (merged smsbox) and doing them otherwise might 
not be possible or not worth it. So, is the merging worth it? Well I 
finally found time to run some benchmarks, here is results briefly
(see the attachment for more results):

The benchmarking was not easy thing, and it is NOT a very good one either, 
as the problem is how to benchmark just one thing. As the merged Kannel 
(our version) has in addition to that merge, a better store-file and
throttle system for fakesmsc, it resulted in level results regardless what 
I threw to it - it did not eat the resources of the system nor slowed 
down. With Vanilla Kannel (fresh CVS) I have to put store-file away etc.

So this benchmark not only tested the speed difference of merged smsbox 
and bearerbox versus separate boxes, but other parts, too. I tried to 
minimize the effect of other parts and thus ran several tests with various 
settings.

Well, the full results are in the attachment. Short conclusion:

It is clearly faster to use merged system. However, as a sidenote, the 
inter-box speed etc. are not the main issue here, unless building up a 
very, very special system (where there is more than 1000-2000 msg/s of 
incoming traffic). However, the merged system made it far more easier to 
implement throttle system (for incoming messages and HTTP) and improved 
store-file, which resulted that this merged version could handle any loads 
without slowing down or eating all the resources. Vanilla Kannel was
killed by OS after swap ran out or it got bogged down by the store-file.

But that is another store does that happen in real world or just in 
benchmarks :]
 

As last words, yes we have various little bits and in upcoming weeks, we 
investigate how we can merge them into CVS tree and then provide patches. 
Hopefully as many as possible, but some might become difficult because 
of the above differences and limited time to do bigger modifications to 
make them work..

-- 
&kalle marjola
product concept manager
NETikos finland (http://www.netikos.fi)


Setup:
 Vanilla Kannel (fresh CVS) vs. modified Kannel (merged bearer- and
 smsbox, throttle system to fakesmsc driver, improved HTTP)

 System: Linux on tabletop Linux.
 No store-file. Merged Kannel has better store-system. Vanilla
 Kannel got completely frozen if store-file was used with high
 incoming message traffic.


---
1. fakesmsc and bearerbox/smsbox all in same machine, sending 'nop'
   messages at interval of 0.0001 (1/s)
   (note: fakesmsc binary of own distribution. But I guess they are
   the same) 

Vanilla Kannel:
(1)
 2002-09-17 09:17:42 [0] INFO: fakesmsc: from first to last sent message 1.4 s, 7308.5 
msgs/s
 2002-09-17 09:17:42 [0] INFO: fakesmsc: from first to last received message 4.7 s, 
2120.6 msgs/s

(10)
 2002-09-17 09:04:29 [0] INFO: fakesmsc: from first to last sent message 13.4 s, 
7441.1 msgs/s
 2002-09-17 09:04:29 [0] INFO: fakesmsc: from first to last received message 50.0 s, 
1999.2 msgs/s
NOTE: memory growth (to 25 megabytes)

(100)
 Couldn't handle, too much memory growth. Killed by operating system.
 Same happened even with interval = 0.0004 (2500 msg/s)


Merged Kannel:
(1)
 2002-09-17 07:02:09 [0] INFO: fakesmsc: from first to last sent message 1.0 s, 9864.4 
msgs/s
 2002-09-17 07:02:09 [0] INFO: fakesmsc: from first to last received message 2.7 s, 
3711.5 msgs/s
NOTE: with store-file, ~2800 msg/s

(10)
 2002-09-17 07:03:12 [0] INFO: fakesmsc: from first to last sent message 17.7 s, 
5650.2 msgs/s
 2002-09-17 07:03:12 [0] INFO: fakesmsc: from first to last received message 27.6 s, 
3623.6 msgs/s
NOTE: with store-file, ~2830 msg/s

(100)
 2002-09-17 07:08:24 [0] INFO: fakesmsc: from first to last sent message 252.8 s, 
3955.7 msgs/s
 2002-09-17 07:08:24 [0] INFO: fakesmsc: from first to last received message 262.7 s, 
3807.2 msgs/s
NOTE: with store-file, ~2870 msg/s

--