Re[4]: [fpc-pascal] a suggestion...
I tried to benchmark a little. Archivers were limited to 512 Mb. Timings only in the second test. Precision is kept 1 Mb/10 seconds intentionally. There are many comprehensive benchmarks, but I tested nearly the last versions. OpenOffice 2.0.2 sources, 1209 Mb (there are several .gz and other binary files inside). (RAR and 7z had the advantage of managing files themselves to create solid archives, gzip and bzip2 were used on a 1209 Mb ball made by RAR.) Rar (3.60 beta 2): 212 Mb gzip (1.2.4): 276 Mb bzip2 (1.0.3): 230 Mb 7z PPMd (4.4.2):190 Mb 7z LZMA (): too slow (~100 Kb/s, more than three hours) After that I tried to compress FPC SVN (153 Mb), and the numbers were in different order (FPC ball was created in 2 minutes by Rar): Rar 21 Mb 5 minutes gzip29 Mb 1 minute 30 seconds bzip2 20 Mb 3 minutes 40 seconds (is it optimized for pascal?) 7z PPMd 17 Mb 8 minutes 20 seconds 7z LZMA 16 Mb 25 minutes (well, it would be better, I believe it had memory problems) I tried PPMonster archiver (open source, from the author of PPMd algorithm), while sources are less than 60 Kb in C, speed is often about 20 Kb/s. However, it promised 0.548 bits per byte of the FPC ball (but 1.5 for OpenOffice). Then try one of the last compressors as PAQ8, your OO 2.0.2 sources will take days in compress, but i think you can get 140-150 MB or even less and is opensource too. But it isn't used out of comp.compression world. Also, the WinRK3.03 is a lot faster and gets a bit more compression, but only works on windows. The tar.gz of FreeBSD (90MB decompressed) goes to 12 MB with medium compression, but takes 20 min in a 2GHz Athlon. Check http://www.maximumcompression.com/ for details and the newsgroup comp.compression or datacompression.info for more info. Just change the gzip to a better zip program for make zip files, as 7z, and you'll get an improvement from current compression, but the bzip2 suggestion is very good, even FreeBSD has changed the tar.gz system to tar.bz2 (tbz) and the difference is notable. A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re[5]: [fpc-pascal] a suggestion...
Correction: Rar produced 212 Mb on OO 2.0.2 sources, but using force text compression makes it produce 206 Mb with not really big time overhead. Then try one of the last compressors as PAQ8, your OO 2.0.2 sources will take days in compress, but i think you can get 140-150 MB or even less and is opensource too. But it isn't used out of First thing: I did neither think or believe, I just tested. Well, my benchmarks concerned bearable timings and memory requirements: no more than two hours and 512 Mb. I didn't test archivers, which were not mentioned in the thread (exception: PPMonstr, which actually was only an example of compression world only compressor). comp.compression world. Also, the WinRK3.03 is a lot faster and gets a bit more compression, but only works on windows. The tar.gz of All four archivers tested by me work under many systems (bzip2, gzip, rar, 7z). I believe, paq8h has only minor problems porting. Rar is commercial, but it has good GUI (well, I use it and I like it) and is supported and developed continuously (and my benchmarks showed, that its compression ratios under given circumstances are not so good: losing to 7z is due to dictionary size: rar uses no more than 128 Mb of memory, but losing to bzip2 on fpcSVN is freaky). http://www.maximumcompression.com/ for details and the newsgroup comp.compression or datacompression.info for more info. Oh, you forgot www.compression.ru :) Didn't you pay attention, that the these sites didn't test sources? (And www.compression.ru has really outdated benchmarks :( ) Just change the gzip to a better zip program for make zip files, as 7z, and you'll get an improvement from current compression, but the Gzip does not produce zip files. It just uses deflate algorithm. bzip2 suggestion is very good, even FreeBSD has changed the tar.gz system to tar.bz2 (tbz) and the difference is notable. Yes. And it uses only about 8 Mb memory. (But my test showed, that it loses 8.5% on OO 2.0.2 sources to Rar. And more than 11.5% with force text compression in rar.) WinRK is unacceptable, because it is not only commercial, but runs only under windows. While Rar offers command line support under several linuces and unrar support under many systems. AND WHILE ITS COMPRESSION ENGINE MAY BE GOOD, it simply does not work well (e.g. 3.0.3 b2 tells, that trial period has expired right after installation). IT'S PROGRAMMED BADLY. Paq8 may be good, because it's open source and free. (By the way, if I get it right and speak it out very roughly, it is PPMd based with some dictionary improvements and other tweaks added.) But it's incredibly slow (in PPMonstr's category). I understand, that experimental archivers are sometimes optimized for Calgary Challenge (not because of luaghable money prizes, but because of some sort of fame), so it's not that great for compressing OO 2.0.2 or FPC sources. Benchmarks addendum (FPCsvn): WinRK --- requires tweaks to run and may crash at many moments (GUI problems?). Could it be useful, High profile would be applied (512 Mb). And it would be too long (as 7z in LZMA mode). Paq-8H (on a rar ball): too long... Well, bzip2 or 7z do seem most efficient. Nothing has changed. Probably, this thread is useless: if the developers find some time, they will use bzip2 or something they like. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re[5]: [fpc-pascal] a suggestion...
On Thu, 1 Jun 2006, ??? wrote: Correction: Rar produced 212 Mb on OO 2.0.2 sources, but using force text compression makes it produce 206 Mb with not really big time overhead. Then try one of the last compressors as PAQ8, your OO 2.0.2 sources will take days in compress, but i think you can get 140-150 MB or even less and is opensource too. But it isn't used out of First thing: I did neither think or believe, I just tested. To save your time, you should limit your tests to: 1. Command-line tools. Creation of installs is automated. GUI tools cannot be used in automated builds. 2. Completely cross-platform. For obvious reasons. 3. Completely Open source. For obvious reasons. 4. Compress time should be definitely less than 10 minutes. Sometimes we do 4-5 builds one after the other. We simply cannot afford to wait 2 hours, and we never know whether the next build will be the final one. Any solution that falls outside of this is not acceptable in advance. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re[6]: [fpc-pascal] a suggestion...
To save your time, you should limit your tests to: It doesn't take much human time. 1. Command-line tools. Creation of installs is automated. GUI tools cannot be used in automated builds. 2. Completely cross-platform. For obvious reasons. 3. Completely Open source. For obvious reasons. 4. Compress time should be definitely less than 10 minutes. Sometimes we do 4-5 builds one after the other. We simply cannot afford to wait 2 hours, and we never know whether the next build will be the final one. Any solution that falls outside of this is not acceptable in advance. Michael. From first message: 1. rar,7z, bzip2+tar/bzip2+? (I used rar to create ball), gzip+tar/gzip+? (--) it depends on script writer 2. nothing is completely cross-platform, but all the four archivers provide executables (sources are great, but executables do work) for a good deal of platforms/systems (7z disclaims that executables are official). 3. 7z, bzip2, gzip 4. ??? well, if the thing is not bigger than FPCsvn directory, they fit. You didn't mention memory requirements. However, bzip2 is already great for FPC-sized installs. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re[6]: [fpc-pascal] a suggestion...
On Thu, 1 Jun 2006, ??? wrote: To save your time, you should limit your tests to: It doesn't take much human time. 1. Command-line tools. Creation of installs is automated. GUI tools cannot be used in automated builds. 2. Completely cross-platform. For obvious reasons. 3. Completely Open source. For obvious reasons. 4. Compress time should be definitely less than 10 minutes. Sometimes we do 4-5 builds one after the other. We simply cannot afford to wait 2 hours, and we never know whether the next build will be the final one. Any solution that falls outside of this is not acceptable in advance. Michael. From first message: 1. rar,7z, bzip2+tar/bzip2+? (I used rar to create ball), gzip+tar/gzip+? (--) it depends on script writer 2. nothing is completely cross-platform, but all the four archivers provide executables (sources are great, but executables do work) for a good deal of platforms/systems (7z disclaims that executables are official). 3. 7z, bzip2, gzip 4. ??? well, if the thing is not bigger than FPCsvn directory, they fit. ? I was talking about the time it takes to create the compressed file. This should not You didn't mention memory requirements. Because they are not an issue. Time, command-line use and availability are issues. However, bzip2 is already great for FPC-sized installs. Personally, I also think this can be used, but it should be checked that this is available for all platforms. Inno setup (for the windows installer) uses bz2, so that is already a start :-) Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Daemon doesn't work right at boottime.
Hi all, I have written a daemon as part of a project (I used the daemon.pp example as a skeleton), it's working quite perfectly if you start it from the commandline, including from the bootscript I wrote. But when started at boottime it doesn't operate correctly anymore. The timer still counts perfectly but it is as if it never hits the trigger times. By following the log files I can verify that the timer-triggered events never occur if it is started at boottime. These events, and their associated intervals are read from plugin configuration files in /etc/ All I can imagine is that for some reason when started from the bootscripts - the daemon suddenly doesn't find the files... but why would it then find it if you run the same script from a terminal later ? Any idea where I should start looking ? A.J. -- there's nothing as inspirational for a hacker as a cat obscuring a bug by sitting in front of the monitor - Boudewijn Rempt A.J. Venter Chief Software Architect OpenLab International www.getopenlab.com www.silentcoder.co.za +27 82 726 5103 ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Daemon doesn't work right at boottime.
Am Donnerstag, 1. Juni 2006 22:40 schrieb A.J. Venter: Hi all, I have written a daemon as part of a project (I used the daemon.pp example as a skeleton), it's working quite perfectly if you start it from the commandline, including from the bootscript I wrote. But when started at boottime it doesn't operate correctly anymore. The timer still counts perfectly but it is as if it never hits the trigger times. By following the log files I can verify that the timer-triggered events never occur if it is started at boottime. These events, and their associated intervals are read from plugin configuration files in /etc/ All I can imagine is that for some reason when started from the bootscripts - the daemon suddenly doesn't find the files... but why would it then find it if you run the same script from a terminal later ? Any idea where I should start looking ? Just a shot into the dark: maybe you start yout deamon, before your local timezone is set up? Does it change anything, when you move your bootscript to a higher position in rc.x, e.g. make it S99whatever? Burkhard ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Daemon doesn't work right at boottime.
Any idea where I should start looking ?
Just a shot into the dark: maybe you start yout deamon, before your
local timezone is set up?
Does it change anything, when you move your bootscript to a higher
position in rc.x, e.g. make it S99whatever?
Good theory, but my daemon is very nearly the last bootscript to run, quite a
long while after this system at least has set up the local timezone, ran
ntpdate once and then started ntpd.
The log on a normal run shows each plugin getting run on zero, then again on
whatever their interval dictates - but from bootup the only thing that shows
up in the log are the tick's
Here are the critical parts of the source:
snip
Procedure ReadPlugins;
Var Plugins, ThisPlugin : TstringList;
I,J : Integer;
CMD,Min : String;
Begin
Writeln('Woled started');
Assign(fLog,logname);
Append(Flog);
Writeln(Flog,'Woled started');
Actions := TStringList.Create;
ThisPLugin := TstringList.Create;
GetDirList('/etc/wole/plugins/*',Plugins);
If Plugins.Count = 0 then
Begin
Writeln(Flog,'No plugins configured, exciting');
close(Flog);
halt;
end;
For I := 0 to Plugins.Count -1 do
Begin
ThisPlugin.Clear;
ThisPlugin.LoadFromFile('/etc/wole/plugins/'+Plugins[I]);
for J := 0 to ThisPlugin.Count -1 do
Begin
If pos('ACTION=',uppercase(ThisPlugin[J])) 0 then
begin
CMD:=Copy(ThisPlugin[J],pos('=',ThisPlugin[J])+1,length(ThisPlugin[J]));
end;
If pos('INTERVAL=',uppercase(ThisPlugin[J])) 0 then
begin
Min:=Copy(ThisPlugin[J],pos('=',ThisPlugin[J])+1,length(ThisPlugin[J]));
end;
end;
If (Length(Min) 0) And (Length(CMD) 0) then
begin
Actions.Add(min+'='+cmd);
end else
Begin
Writeln(Flog,'Could not read ',Plugins[I],' format error.');
end;
end;
Close(FloG);
ThisPlugin.Free;
Plugins.Free;
Writeln('Handling ',actions.count,' plugins');
end;
snip
begin{MAIN}
snip
{Inside the fork}
Repeat
If bHup Then Begin
{$I-}
Close(fLog);
{$I+}
IOResult;
NewLog;
bHup := false;
End;
{--}
Assign(Flog,Logname);
Append(Flog);
Writeln (Flog,'Tick - ',minutes,' minutes');
For PCount := 0 to Actions.Count -1 do
Begin
Mint := StrToInt(Copy(Actions[Pcount],1,pos('=',Actions[Pcount]) -1));
If Minutes mod Mint = 0 then
Begin
Writeln(Flog,'Executing plugin action:');
Writeln(Flog,'
',copy(Actions[Pcount],pos('=',Actions[Pcount])+1,length(Actions[Pcount])));
shell(copy(Actions[Pcount],pos('=',Actions[Pcount])+1,length(Actions[Pcount])));
Writeln(Flog,'Action completed, resuming counter');
end;
end;
Close(fLog);
If bTerm Then
BREAK
Else
{ wait a while }
fpSelect(0,nil,nil,nil,6);
inc(Minutes);
Until bTerm;
Actions.Free;
--
there's nothing as inspirational for a hacker as a cat obscuring a bug
by sitting in front of the monitor - Boudewijn Rempt
A.J. Venter
Chief Software Architect
OpenLab International
www.getopenlab.com
www.silentcoder.co.za
+27 82 726 5103
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Daemon doesn't work right at boottime.
A.J. Venter wrote:
Any idea where I should start looking ?
Just a shot into the dark: maybe you start yout deamon, before your
local timezone is set up?
Does it change anything, when you move your bootscript to a higher
position in rc.x, e.g. make it S99whatever?
Good theory, but my daemon is very nearly the last bootscript to run, quite a
long while after this system at least has set up the local timezone, ran
ntpdate once and then started ntpd.
The log on a normal run shows each plugin getting run on zero, then again on
whatever their interval dictates - but from bootup the only thing that shows
up in the log are the tick's
Here are the critical parts of the source:
snip
Procedure ReadPlugins;
Var Plugins, ThisPlugin : TstringList;
I,J : Integer;
CMD,Min : String;
Begin
Writeln('Woled started');
Assign(fLog,logname);
Append(Flog);
Writeln(Flog,'Woled started');
Actions := TStringList.Create;
ThisPLugin := TstringList.Create;
GetDirList('/etc/wole/plugins/*',Plugins);
If Plugins.Count = 0 then
Begin
Writeln(Flog,'No plugins configured, exciting');
close(Flog);
halt;
end;
For I := 0 to Plugins.Count -1 do
Begin
ThisPlugin.Clear;
ThisPlugin.LoadFromFile('/etc/wole/plugins/'+Plugins[I]);
for J := 0 to ThisPlugin.Count -1 do
Begin
If pos('ACTION=',uppercase(ThisPlugin[J])) 0 then
begin
CMD:=Copy(ThisPlugin[J],pos('=',ThisPlugin[J])+1,length(ThisPlugin[J]));
end;
If pos('INTERVAL=',uppercase(ThisPlugin[J])) 0 then
begin
Min:=Copy(ThisPlugin[J],pos('=',ThisPlugin[J])+1,length(ThisPlugin[J]));
end;
end;
If (Length(Min) 0) And (Length(CMD) 0) then
begin
Actions.Add(min+'='+cmd);
end else
Begin
Writeln(Flog,'Could not read ',Plugins[I],' format error.');
end;
end;
Close(FloG);
ThisPlugin.Free;
Plugins.Free;
Writeln('Handling ',actions.count,' plugins');
end;
snip
begin{MAIN}
snip
{Inside the fork}
Repeat
If bHup Then Begin
{$I-}
Close(fLog);
{$I+}
IOResult;
NewLog;
bHup := false;
End;
{--}
Assign(Flog,Logname);
Append(Flog);
Writeln (Flog,'Tick - ',minutes,' minutes');
For PCount := 0 to Actions.Count -1 do
Begin
Mint := StrToInt(Copy(Actions[Pcount],1,pos('=',Actions[Pcount]) -1));
If Minutes mod Mint = 0 then
Begin
Writeln(Flog,'Executing plugin action:');
Writeln(Flog,'
',copy(Actions[Pcount],pos('=',Actions[Pcount])+1,length(Actions[Pcount])));
shell(copy(Actions[Pcount],pos('=',Actions[Pcount])+1,length(Actions[Pcount])));
Writeln(Flog,'Action completed, resuming counter');
end;
end;
Close(fLog);
If bTerm Then
BREAK
Else
{ wait a while }
fpSelect(0,nil,nil,nil,6);
inc(Minutes);
Until bTerm;
Actions.Free;
Just my 2 cents: If you're on linux look at the file rw permissions,
especially for I/O-interfaces like ttyS0, ttyS1, etc. In Fedora they
have only root write permissions.
Joep
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Daemon doesn't work right at boottime.
Just my 2 cents: If you're on linux look at the file rw permissions, especially for I/O-interfaces like ttyS0, ttyS1, etc. In Fedora they have only root write permissions. Joep Well as far as I know I'm not writing to anything except the log file, and the daemon is running as root anyway. /me considers recoding the whole thing as a cronjob... (A joke, in reality I simply don't have the diskspace to put cron where this is needed so it's not an option even if it COULD do the work). A.J. -- there's nothing as inspirational for a hacker as a cat obscuring a bug by sitting in front of the monitor - Boudewijn Rempt A.J. Venter Chief Software Architect OpenLab International www.getopenlab.com www.silentcoder.co.za +27 82 726 5103 ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
