AFAICT most of the content of that document is rubbish. They have certainly
skewed our support costs heavily, using the most extreme examples.
The tpc-c performance figures look extremely suspect. Does anyone seriously
do tpc-c testing with 2 warehouses? It doesn't make sense to me.
Looking at the figures for the full table scan it looks as if they have
created a 16 warehouse database. This will occupy around 1.5 GB of disc space
and can easily be loaded in memory. This in itself has an impact on the
quality of the test results.
The InterBase tests were done with journaling and async writes compared to
Firebird with synchronous writes. We don't know the journalling
configuration. We also don't know how long the tests were run for. And
neither do we know anything about the speed and latency of disc sub-system. If
the run is short then journalling would have minimal overhead and InterBase
writes would just be cached while Firebird would write straight to disk. So
basically the comparison would be invalid. More realistic performance
comparisons would be
o with journalling off and sync vs sync
o journalling off and async vs async
o with a very large database that cannot fit in memory/file cache with a
realistic number of concurrent connections. This would effectively
bottleneck at the filesystem level, thus removing the advantage of
asynchronous file writes. Then we could see if their configuration
outperforms firebird.
Coming back to the data for the full table scans. How did they do that? It
certainly wasn't with a simple sql statement in isql. However, I created a
simple stored procedure that scanned the orders table in 0.261 seconds (10
warehouses, 300,000 rows, Firebird 3.0 SS.) The database was on a single,
old-fashioned HDD.
If anyone can let me have a copy of InterBase XE7 I would be happy to run
some proper comparisons on our test rig.
Paul
--
Paul Reeves
http://www.ibphoenix.com
Supporting users of Firebird
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
