Hi All, Sorry it's so long ;P
While testing TB! with heavy number of imported EML messages I had a few problems and thoughts. One of them is implementing a multicore use while making backup. At the moment TB! is using only 1 core while making backup so it takes long time if TB!'s database is big. Considering that only 1 core is used TB!'s results are quite good. However, it might be reconsidered to use an advantage of the multicores that are nowadays common :) A nice feature would be usage of GPU while compressing but it's probably too early at the moment to make it work. Works on that subject are still in early stage. TESTING I did some testing on concerning LZMA2 algorythm (and some more for comparision). LZMA is available on GPL license and probably LZMA can be implemented with usage of 7ZIP DLL file as well. Database size was 1.27GB (tests made on X6 1090T OC ; password used while compressing: jKojgrY8900)(75?.,<;"), results below (time ; archive size ; RAM usage ; CPU usage ; compression type): TB! Backup [1:30 ; 752.5MB ; 20MB ; 16-18% ; NA ] RAR [4:27 ; 679.0MB ; 37MB ; 22-48% ; Normal] RAR [4:21 ; 676.0MB ; 78MB ; 33-48% ; Best] RAR [4:27 ; 679.0MB ; 37MB ; 27-53% ; Normal (Encrypted)] 7ZIP [0:31 ; 756.0MB ; 20MB ; 76-92% ; Fastest, LZMA2 (6 cores)] 7ZIP [0:44 ; 756.0MB ; 15MB ; 59-66% ; Fastest, LZMA2 (4 cores)] 7ZIP [1:03 ; 756.0MB ; 10MB ; 30-36% ; Fastest, LZMA2 (2 cores)] 7ZIP [2:07 ; 756.0MB ; 8MB ; 17-18% ; Fastest, LZMA2 (1 core)] 7ZIP [2:13 ; 714.6MB ; 7MB ; 17% ; Fastest, BZip2 (1 core)] 7ZIP [2:03 ; 603.7MB ; 828MB ; 74-92% ; Normal, LZMA2 (6 cores)] 7ZIP [1:03 ; 714.6MB ; 62MB ; 74-92% ; Normal, BZip2 (6 cores)] 7ZIP [2:25 ; 607.0MB ; 550MB ; 59-66% ; Normal, LZMA2 (4 cores)] 7ZIP [4:23 ; 593.0MB ; 200MB ; 22-35% ; Normal, LZMA (2 cores)] 7ZIP [4:35 ; 593.0MB ; 200MB ; 26-35% ; Normal, LZMA (2 cores), Encrypted] 7ZIP [3:50 ; 574.4MB ; 2,93GB ; 76-96% ; Ultra, LZMA2 (6 cores) 7ZIP [4:21 ; 574.4MB ; 1.99GB ; 50-61% ; Ultra, LZMA2 (4 cores)]] 7ZIP [6:52 ; 574.4MB ; 700MB ; 29-50% ; Ultra, LZMA2 (2 cores)] 7ZIP [6:52 ; 573.0MB ; 700MB ; 24-31% ; Ultra, LZMA (2 cores)] As you can see from above: 1) if for instance only 2 cores are used and Fastest LZMA2 algorith is implemented a speed gain is about 30% with about the same file size. 2) if Fastest compresion (LZMA2) is used on all cores (X6 here) time is shorten to 31 seconds instead of 1m 30s (about same archive size). (faster 70%) 3) if Normal LZMA2 is selected on 4 cores (to leave 2 cores for a system) it's only about 1 minute longer (61% longer) than current TB!'s solution but archive size is 20% smaller than current one. EXAMPLE As for the database I got after importing all EML messages into TB! (10.7GB - TB!'s backup file size): - there would be a possibility to get a backup file 2.2GB smaller than a current one (LZMA2, Ultra - but it would take 30-37min longer). - it would be possible as well to get the same archive size in about 25-30min if only 4 cores are used (15-18min on 6 cores :) ) instead of 50-60min in current solution. IMPLEMENTATION I would see the implementation as an option in Backup menu allowing the user to select backup type: - current TB!'s solution, - LZMA/LZMA2 implementation. As for LZMA/LZMA2 in my opinion it would be good if user can assign number of cores and selection of compression method. User should be given estimated memory usage information as well - should apply to users that would like to use better compression method (more RAM needed - Ultra, LZMA2 needs 3GB RAM while compressing :) ) That could be another feature that would make TB! a heavy duty and better client :) If you like the idea you might be willing to give your feedback: https://www.ritlabs.com/bt/view.php?id=8708 SOME LINKS http://www.7-cpu.com/ http://www.7-zip.org/sdk.html http://www.7-zip.org/links.html http://tukaani.org/lzma/ -- Best regards, RS ________________________________________________________ Current beta is 5.0.12.3 | 'Using TBBETA' information: http://www.silverstones.com/thebat/TBUDLInfo.html
