Nikos,
It's great that you gave the numbers you obtained on your tests, so
that someone can have an idea on what's possible to achieve with
Kannel.
What we're saying, is that you are stating that because your tests
gave similar results on a few different server configurations that
means that those are the values that Kannel can handle, and that's not
the case.
It could be that maybe they are the average numbers on X
configuration, but on other environments it will be able to handle
much more traffic than that, and on other environments may be able to
handle much less traffic than that (and I'm not referring to
networking, nor application side, just Kannel).
So, it's cool to share your numbers, but just as a reference of your
specific case, and probably of others too.
But that doesn't mean that those are the numbers that Kannel can
achieve as a defacto.
2010/8/14 Nikos Balkanas <nbalka...@gmail.com>:
> I am not pretending to hold any holy grail. A user asked for some typical
> numbers and I gave him some averages. On the other hand you go talking on
> and on wihout anything to show for, and that's irresponsible.
>
> If they don't mean anything to you, that's fine, you can always get your own
> numbers.
>
> When asked about benchmarks *noone* ever gives real data. Real data is much
> too unreproducible and could even DOS attack unsuspected external servers.
> Not disregarding any cost from sending real SMS. Therefore it is impled that
> in these tests a simulator is used. Besides, real network is not part of
> kannel and irrelevant to benchmarks.
>
> BR,
> Nikos
> ----- Original Message ----- From: Alejandro Guerrieri
> To: Nikos Balkanas
> Cc: Juan Nin ; users@kannel.org
> Sent: Sunday, August 15, 2010 1:18 AM
> Subject: Re: Kannel profermance
>
>
> Hmm, nope, the fact is, you performed some benchmarks a on a few boxes and
> pretend it to be the holy grail of Kannel's performance benchmarks. It's
> not.
>
>
> Your tests might be valid for your setup, specially if you're optimizing
> things around and you repeat the tests to compare before and after the
> changes, but other than that, it doesn't mean much for the rest of us (other
> than ballpark figures of what to expect in general terms, in _very_ general
> terms).
>
>
> I believe that stating figures like "Kannel can do X MT per second" would be
> _very_ misleading, specially if the tests are performed in a lab environment
> against fake boxes. Those numbers are completely unrealistic and doesn't
> have any real-life use.
>
>
> Regards,
>
>
> Alex
>
>
> 2010/8/14 Nikos Balkanas <nbalka...@gmail.com>
>
> Sorry Alex,
>
> What you say is largely unsupported and unsubstantiated, unless you have
> some numbers to back it up. I do.
>
> 1 & 2) Results asked and tests given are for typical conditions (normal fs,
> spool) and a medium 64bit Ubuntu, kernel 2.6.31-22 box (dual-core Xeon
> @3.4GHz). Interestingly, widely different OS/hardware give similar results,
> ie a low end 64bit Solaris 10 box (dual-core AMD Opteron CPU @1218 uHz) is
> only 15% slower than the previous Ubuntu box.
>
> 3) Indeed using a DB for DLRs has a major effect, effectively cutting MT
> response to half, as shown in the tests. Choice of DB, will also have some
> minor effect, too. Fakesmpp can evaluate different DBs, index or other DB
> optimization choices (ie InnoDB vs MyIsam).
>
> As mentioned in the tests (check mail on fakesmpp submission) this is for
> 64bit architecture. Nowdays, typical conditions is 64bit compilation.
>
> 4) I am only benchmarking kannel. Applications are custom made and not part
> of kannel.
>
> 5) Nobody is trying to predict anything. You have an average figure of
> kannel's response, build your system and run some benchmarks to see where
> you fair.
>
> 6) Again, figures asked were for kannel. SMSc is not a kannel component.
>
> You really should read my mail about fakesmpp submission.
>
> Some useful conclusions that you can take home:
>
> 1) MOs, using default service, are ~30% faster than MTs using DLRs.
> 2) Internal DLRs are almost twice as fast as DB DLRs.
> 3) Throttling errors will slow kannel to a crawl. Kannel retries them every
> 30".
>
> BR,
> Nikos
> ----- Original Message ----- From: Alejandro Guerrieri
> To: Nikos Balkanas
> Cc: Juan Nin ; users@kannel.org
> Sent: Saturday, August 14, 2010 2:46 PM
> Subject: Re: Kannel profermance
>
>
> Your performance tests, while worth looking as a general reference, are
> probably near to useless as benchmarks for other people's platforms.
>
>
> 1. Variations in hardware could alter your figures greatly. Differences in
> CPU, memory and/or HDD speeds, as well as using RAID and/or SATA/SCSI/Fiber
> interfaces (among many other factors) obviously will alter the results, and
> it's _very_ difficult to predict the amount of variation, since it depends
> greatly on how do you really use Kannel.
>
>
> 2. Variations at the OS level also play a role: Using different Unix
> flavors, different disk partitioning types, ramdisks and a whole lot of
> other stuff that could be configured in a million different ways will also
> alter the results, and in a very hard to predict fashion.
>
>
> 3. Variations in Kannel configuration: the type of store, either using a DB
> or not for DLR's (and the type of DB engine), either if the DB engine and
> the application layer are local or remote and another thousand possible
> factors. Also if you compiled it as 32 or 64 bit and what kind of compiler
> optimizations you did.
>
>
> 4. Last but not least, the application layer: yo don't specify what you're
> doing with your messages when they're received, and here's when your
> benchmarks become unrealistic: The response time of the applications is KEY
> and can pose a severe bottleneck to Kannel, thus affecting the "raw
> performance" in very unpredictable ways: An application that takes time to
> respond will hold threads open, thus incrementing the "iowait" and "load" of
> the Kannel box accordingly. That degrades a great deal performance.
>
>
> So, since it's near impossible to predict all those factors and the
> application level can play a key role into determining the final
> performance, I wouldn't take your (nor anyone else's) performance benchmarks
> as a strict reference of what Kannel is capable of regarding performance in
> my particular platform.
>
>
> I guess a proper statement could be "my Kannel setup running on OS A is
> capable of handling B MO/s and C MT/s using this configuration {...}" but
> again, external factors like application's speed, smsc throttling and a
> thousand more factors could make very similar setups to perform in very
> dissimilar ways.
>
>
> A possible approach to provide a more relevant benchmark would be to
> standarize a series of tests with some fixed variables (specially at the
> application layer). That would provide at least a measure to compare against
> between installs, but again, it wouldn't be able to predict anyone's
> real-life scenario at all, as regular computer benchmarks are unable to
> predict how fast your computer will be for the work you'll use it for.
>
>
> In other words, don't take other people's benchmarks very seriously to
> predict what your system will be capable of.
>
>
> Regards,
>
>
> Alex
>
>
> 2010/8/14 Nikos Balkanas <nbalka...@gmail.com>
>
> Nope. The average between various servers that have been tested are
> approximately the ones cited. Standard deviation between optimized
> servers/OS is not that large (+/-50). This includes various
> conditions/architectures, each one with each own results. I just posted the
> most common ones. No optimized filesystem was used anywhere (spool, ext3 on
> linux, ufs on Solaris). Tests are very reproducible and discriminating with
> respect to DB, and possibly filesystem used for storage (not tested yet).
>
> If in doubt run your own tests.
>
>
> Nikos
> ----- Original Message ----- From: "Juan Nin" <jua...@gmail.com>
> To: <users@kannel.org>
>
> Sent: Saturday, August 14, 2010 8:23 AM
>
> Subject: Re: Kannel profermance
>
>
>
> So you're saying that on any server and/or architecture the results
> will be the same?
> Doesn't seem very reasonable...
>
>
> 2010/8/14 Nikos Balkanas <nbalka...@gmail.com>:
>
> Actually you have missed a couple of more emails. On fakesmpp submission I
> also posted results from a low-end Solaris 10 64bit box. Very similar to the
> results posted from the linux server. The averages seem pretty solid. So,
> contrary to your beliefs, it is not giving out the wrong impression.
>
> BR,
> Nikos
> ----- Original Message ----- From: "Juan Nin" <jua...@gmail.com>
> To: <users@kannel.org>
> Sent: Saturday, August 14, 2010 4:33 AM
> Subject: Re: Kannel profermance
>
>
>
> Ok, just saw on another thread where you got those values from, but
> again, that's very system specific.
>
> I guess you point was just to say that Kannel was not the issue for
> his bottleneck, but saying "It can handle ~1000 MO/s, 750 MT/s
> (internal DLRs) or 450 MT/s (DB DLRs)" may give the wrong impression
> that that's the maximum it can handle, or that it can always handle
> that load, while on low end servers it may not.
>
> So saying something like that in that way can confuse new users, IMHO.
>
>
> On Fri, Aug 13, 2010 at 10:26 PM, Juan Nin <jua...@gmail.com> wrote:
>
>
> 2010/8/13 Nikos Balkanas <nbalka...@gmail.com>:
>
>
> It is unlikely that kannel is your bottleneck. It can handle ~1000 MO/s,
> 750
> MT/s (internal DLRs) or 450 MT/s (DB DLRs).
>
>
>
> Just curious to know where do you get those values from...
> What Kannel supports depends on your hardware and architecture
>
>
>
>
>
> --
> Juan Nin
> 3Cinteractive / Mobilizing Great Brands
> http://www.3cinteractive.com
>
>
>
>
>
>
>
>
> --
> Juan Nin
> 3Cinteractive / Mobilizing Great Brands
> http://www.3cinteractive.com
>
--
Juan Nin
3Cinteractive / Mobilizing Great Brands
http://www.3cinteractive.com
