Re: [fpc-pascal] TProcess usage and reading program output
Hi,
how would you handle large output and large stderr?
When you read from one and it writes to the other, the read blocks. Then
it keeps writing to the other buffer, till that is full, and then its
write is blocked, and it is deadlocked.
Probably check NumBytesAvailable before reading?
Bye,
Benito
On 02/28/2017 05:28 PM, Michael Van Canneyt wrote:
On Tue, 28 Feb 2017, Graeme Geldenhuys wrote:
Hi,
Can anybody see if there is something wrong with the code shown below.
The code is copied from one of my earlier projects where I call the FPC
compiler and it worked just fine in that project.
In the work I'm doing now, I'm calling the Delphi Command Line Compiler,
and made a few minor tweaks to the code, but this method never seems to
return when Delphi command line compiler is called, and I don't see any
output. If I call other utilities (eg: Git etc) then I do see output and
it works as expected. It's just the Delphi Command Line Compiler that
now seems to be giving troubles.
I would really appreciate it is anybody could spare a moment. Many
thanks.
I marked two areas with "// ???" where I am unsure if I'm doing the
right thing.
poWaitOnExit should not be needed, as this will cause Execute to wait
for process exit...
It seems likely that this will interfere with reading from output:
when the
output buffer is full, the executed process will block.
The rest seems fine.
Also:
It may be that Delphi somehow writes directly to the console buffer,
in that
case you will not catch any output. But then you should see the output on
the screen (just not caught by your process)
Michael.
==
function TBuildDelphiProject.RunTProcess(const Binary: string; args:
TStrings): boolean;
const
BufSize = 1024;
var
p: TProcess;
Buf: string;
Count: integer;
i: integer;
LineStart: integer;
OutputLine: string;
begin
p := TProcess.Create(nil);
try
p.Executable := Binary;
// ??? Is poWaitOnExit needed here, it is called later down the code
p.Options := [poUsePipes, poStdErrToOutPut, poWaitOnExit];
//p.CurrentDirectory := ExtractFilePath(p.Executable);
p.ShowWindow := swoShowNormal; // ??? Is this needed?
p.Parameters.Assign(args);
DoLog(etInfo,'Running command "%s" with arguments
"%s"',[p.Executable, p.Parameters.Text]);
p.Execute;
{ Now process the output }
OutputLine:='';
SetLength(Buf,BufSize);
repeat
if (p.Output<>nil) then
begin
Count:=p.Output.Read(Buf[1],Length(Buf));
end
else
Count:=0;
LineStart:=1;
i:=1;
while i<=Count do
begin
if Buf[i] in [#10,#13] then
begin
OutputLine:=OutputLine+Copy(Buf,LineStart,i-LineStart);
writeln(OutputLine);
OutputLine:='';
if (iBuf[i+1]) then
inc(i);
LineStart:=i+1;
end;
inc(i);
end;
OutputLine:=Copy(Buf,LineStart,Count-LineStart+1);
until Count=0;
if OutputLine <> '' then
writeln(OutputLine);
p.WaitOnExit;
Result := p.ExitStatus = 0;
if Not Result then
Writeln('Command ', p.Executable ,' failed with exit code: ',
p.ExitStatus);
finally
FreeAndNil(p);
end;
end;
==
Regards,
Graeme
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/
My public PGP key: http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] TProcess usage and reading program output
On 2017-02-28 20:03, nore...@z505.com wrote: > Did you end up resolving the issue? Yes, see my "closing thoughts" reply to Michael. What he suggested fixed the issue. > I'd be interested in creating a build tool that not only compiles > projects in FPC but also compiles with dcc32 (delphi compiler).. > > Is that something like what you are doing? I'm converting a *very* complex (and massive) multi-project Makefile system into a single "build binary". Configuration is controlled via project.json files. eg: build dependencies, build lists (if a project consists of more that one project), compiler parameters with macro support (macros are global configurations that can be injected in compiler parameters etc). This is not a general tool, but a specific commercial solution. Saying that, it should be possible building a general tool that does similar - reading project.json files to feed the build system. But then again, there are probably multiple such solutions around already (I think). Regards, Graeme -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] TProcess usage and reading program output
On 2017-02-28 16:28, Michael Van Canneyt wrote: > poWaitOnExit should not be needed, as this will cause Execute to wait for > process exit... This post is simply to close off this thread. Michael's suggestion was the solution to my problem. I removed the poWaitOnExit and set my buffer size to 2048 bytes and that seems to have resolved the issue. Regards, Graeme -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] TProcess usage and reading program output
On 2017-02-28 10:06, Graeme Geldenhuys wrote: Hi, Can anybody see if there is something wrong with the code shown below. The code is copied from one of my earlier projects where I call the FPC compiler and it worked just fine in that project. Did you end up resolving the issue? I'd be interested in creating a build tool that not only compiles projects in FPC but also compiles with dcc32 (delphi compiler).. Is that something like what you are doing? I have a lot of projects that compile both in fpc and delphi and compiling manually using to compilers is a pain. One step just compiling it with a build tool that does both compilers would be nice. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] TProcess usage and reading program output
On 2017-02-28 16:28, Michael Van Canneyt wrote: > poWaitOnExit should not be needed, as this will cause Execute to wait for > process exit... > > It seems likely that this will interfere with reading from output: when the > output buffer is full, the executed process will block. Ah. I've also been reading the docs and this FPC wiki page... http://wiki.freepascal.org/Executing_External_Programs In the wiki they talked about a dead-lock when poWaitOnExit is used and the output buffer is full. It seems that is exactly what I experienced. I'll increase the buffer size I use to 2048 and remove the poWaitOnExit and see how it goes. The "Reading large output" example also uses a TMemoryStream instead of a string buffer like I do, but I think we should have the same end result. http://wiki.freepascal.org/Executing_External_Programs#Reading_large_output If my code still doesn't work, I'll switch to using a TMemorySteam for storing the output and retest. Thanks for the extra pair of eyes. Regards, Graeme -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] TProcess usage and reading program output
On Tue, 28 Feb 2017, Graeme Geldenhuys wrote:
Hi,
Can anybody see if there is something wrong with the code shown below.
The code is copied from one of my earlier projects where I call the FPC
compiler and it worked just fine in that project.
In the work I'm doing now, I'm calling the Delphi Command Line Compiler,
and made a few minor tweaks to the code, but this method never seems to
return when Delphi command line compiler is called, and I don't see any
output. If I call other utilities (eg: Git etc) then I do see output and
it works as expected. It's just the Delphi Command Line Compiler that
now seems to be giving troubles.
I would really appreciate it is anybody could spare a moment. Many thanks.
I marked two areas with "// ???" where I am unsure if I'm doing the
right thing.
poWaitOnExit should not be needed, as this will cause Execute to wait for
process exit...
It seems likely that this will interfere with reading from output: when the
output buffer is full, the executed process will block.
The rest seems fine.
Also:
It may be that Delphi somehow writes directly to the console buffer, in that
case you will not catch any output. But then you should see the output on
the screen (just not caught by your process)
Michael.
==
function TBuildDelphiProject.RunTProcess(const Binary: string; args:
TStrings): boolean;
const
BufSize = 1024;
var
p: TProcess;
Buf: string;
Count: integer;
i: integer;
LineStart: integer;
OutputLine: string;
begin
p := TProcess.Create(nil);
try
p.Executable := Binary;
// ??? Is poWaitOnExit needed here, it is called later down the code
p.Options := [poUsePipes, poStdErrToOutPut, poWaitOnExit];
//p.CurrentDirectory := ExtractFilePath(p.Executable);
p.ShowWindow := swoShowNormal; // ??? Is this needed?
p.Parameters.Assign(args);
DoLog(etInfo,'Running command "%s" with arguments
"%s"',[p.Executable, p.Parameters.Text]);
p.Execute;
{ Now process the output }
OutputLine:='';
SetLength(Buf,BufSize);
repeat
if (p.Output<>nil) then
begin
Count:=p.Output.Read(Buf[1],Length(Buf));
end
else
Count:=0;
LineStart:=1;
i:=1;
while i<=Count do
begin
if Buf[i] in [#10,#13] then
begin
OutputLine:=OutputLine+Copy(Buf,LineStart,i-LineStart);
writeln(OutputLine);
OutputLine:='';
if (iBuf[i+1]) then
inc(i);
LineStart:=i+1;
end;
inc(i);
end;
OutputLine:=Copy(Buf,LineStart,Count-LineStart+1);
until Count=0;
if OutputLine <> '' then
writeln(OutputLine);
p.WaitOnExit;
Result := p.ExitStatus = 0;
if Not Result then
Writeln('Command ', p.Executable ,' failed with exit code: ',
p.ExitStatus);
finally
FreeAndNil(p);
end;
end;
==
Regards,
Graeme
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/
My public PGP key: http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] TProcess usage and reading program output
On 2017-02-28 16:06, Graeme Geldenhuys wrote: > //p.CurrentDirectory := ExtractFilePath(p.Executable); Just thought I would explain this. I don't change directory there (in the TProcess instance), because the program itself changes the current directory before calling RunTProcess(). As for the procedure not returning - from what I can see (I think) it seems like if Delphi fails to compile the project, that's when my code get stuck and never returns. Regards, Graeme -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
[fpc-pascal] TProcess usage and reading program output
Hi,
Can anybody see if there is something wrong with the code shown below.
The code is copied from one of my earlier projects where I call the FPC
compiler and it worked just fine in that project.
In the work I'm doing now, I'm calling the Delphi Command Line Compiler,
and made a few minor tweaks to the code, but this method never seems to
return when Delphi command line compiler is called, and I don't see any
output. If I call other utilities (eg: Git etc) then I do see output and
it works as expected. It's just the Delphi Command Line Compiler that
now seems to be giving troubles.
I would really appreciate it is anybody could spare a moment. Many thanks.
I marked two areas with "// ???" where I am unsure if I'm doing the
right thing.
==
function TBuildDelphiProject.RunTProcess(const Binary: string; args:
TStrings): boolean;
const
BufSize = 1024;
var
p: TProcess;
Buf: string;
Count: integer;
i: integer;
LineStart: integer;
OutputLine: string;
begin
p := TProcess.Create(nil);
try
p.Executable := Binary;
// ??? Is poWaitOnExit needed here, it is called later down the code
p.Options := [poUsePipes, poStdErrToOutPut, poWaitOnExit];
//p.CurrentDirectory := ExtractFilePath(p.Executable);
p.ShowWindow := swoShowNormal; // ??? Is this needed?
p.Parameters.Assign(args);
DoLog(etInfo,'Running command "%s" with arguments
"%s"',[p.Executable, p.Parameters.Text]);
p.Execute;
{ Now process the output }
OutputLine:='';
SetLength(Buf,BufSize);
repeat
if (p.Output<>nil) then
begin
Count:=p.Output.Read(Buf[1],Length(Buf));
end
else
Count:=0;
LineStart:=1;
i:=1;
while i<=Count do
begin
if Buf[i] in [#10,#13] then
begin
OutputLine:=OutputLine+Copy(Buf,LineStart,i-LineStart);
writeln(OutputLine);
OutputLine:='';
if (iBuf[i+1]) then
inc(i);
LineStart:=i+1;
end;
inc(i);
end;
OutputLine:=Copy(Buf,LineStart,Count-LineStart+1);
until Count=0;
if OutputLine <> '' then
writeln(OutputLine);
p.WaitOnExit;
Result := p.ExitStatus = 0;
if Not Result then
Writeln('Command ', p.Executable ,' failed with exit code: ',
p.ExitStatus);
finally
FreeAndNil(p);
end;
end;
==
Regards,
Graeme
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/
My public PGP key: http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
