Re: [fpc-pascal] Adding file to string to the RTL

[Jean SUZINEAU via fpc-pascal]

Le 09/10/2020 à 10:15, Santiago A. via fpc-pascal a écrit :

Just nitpicking.
Shouldn't "try" be after the "reset" and after the "rewrite"?
Why don't you allow to write an empty string?


Yes, you're right, the Reset/Rewrite should be after the try.
I admit this code is not general, just tailored to my own needs.
In my use-cases, writing a file of size 0 as placeholder is not useful.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 9, 2020, at 7:02 AM, Benito van der Zander via fpc-pascal 
>  wrote:
> 
> Writing a file should write the data in a temporary file and then rename the 
> temporary file to replace the target file. Otherwise it might destroy the 
> target file, without writing the new content, when there is an error. 
> Although not even renaming is always safe, some people say you need a 
> filesystem-specific transaction log.

I've seen in some API's they have an "atomic" flag for write file functions 
which do the swap you're describing. I agree it's a smart thing to include in 
the function but it's best to be an option since there is an additional 
operation being performed and affects performance.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Benito van der Zander via fpc-pascal]

They cannot be used on handles that do not support FileSeek() 
(sockets, pipes, stdin/stdout etc.).



Well, it would be better if it could

You can just incrementally resize the return array, when reading 
succeeds after seeking fails.


I have a string load function doing that:

https://github.com/benibela/bbutils/blob/master/bbutils.pas#L3652





  Is it possible to extend this same set of functions to writing? 
Naturally I need to write back to the file now and the same problem

presents itself.  I have to search through other code bases to find a
function or Google and find 



Writing a file should write the data in a temporary file and then rename 
the temporary file to replace the target file. Otherwise it might 
destroy the target file, without writing the new content, when there is 
an error. Although not even renaming is always safe, some people say you 
need a filesystem-specific transaction log.




Benito

On 06.10.2020 10:12, Michael Van Canneyt via fpc-pascal wrote:




No, we don't deal in magic, only bits and bytes :-)

I added the following functions to the sysutils unit (rev 47056):

// Read raw content as bytes

Function GetFileContents(Const aFileName : RawByteString) : TBytes;
Function GetFileContents(Const aFileName : UnicodeString) : TBytes;
Function GetFileContents(Const aHandle : THandle) : TBytes;

// Read content as string

// Assume TEncoding.SystemEncoding
Function GetFileAsString(Const aFileName : RawByteString) : 
RawByteString;

// Specify encoding
Function GetFileAsString(Const aFileName : RawByteString; aEncoding : 
TEncoding) : RawByteString;

// Assume TEncoding.Unicode contents
Function GetFileAsString(Const aFileName : UnicodeString) : 
UnicodeString;

// Specify encoding, return Unicode string.
Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : 
TEncoding) : UnicodeString;


These functions will raise an exception if the file cannot be opened 
or read.
They cannot be used on handles that do not support FileSeek() 
(sockets, pipes, stdin/stdout etc.).


I did some tests on encoding conversion but not extensively. If you 
find any errors, please report them through the bugtracker.


Michael.
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Santiago A. via fpc-pascal]

El 06/10/2020 a las 01:08, Jean SUZINEAU via fpc-pascal escribió:


In my own code I use BlockRead/BlockWrite, but I'm wondering if I've 
not seen this somewhere in RTL.


https://github.com/jsuzineau/pascal_o_r_mapping/blob/TjsDataContexte/pascal_o_r_mapping/02_Units/uuStrings.pas 




Just nitpicking.
Shouldn't "try" be after the "reset" and after the "rewrite"?
Why don't you allow to write an empty string?


function String_from_File( _FileName: String): String;
var
    F: File;
    Longueur: Integer;
begin
  Result:= '';
  if not FileExists( _FileName) then exit;
  AssignFile( F, _FileName);
  try
     Reset( F, 1);
     Longueur:= FileSize( F);
     if 0 = Longueur then exit;
     SetLength( Result, Longueur);
     BlockRead( F, Result[1], Longueur);
  finally
     CloseFile( F);
     end;
end;
procedure String_to_File( _FileName: String; _S: String);
var
    F: File;
begin
  if '' = _S then exit;
  AssignFile( F, _FileName);
  try
     ReWrite( F, 1);
     BlockWrite( F, _S[1], Length( _S));
  finally
     CloseFile( F);
     end;
end;



___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal



--
Saludos

Santiago A.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 8, 2020, at 3:01 PM, Michael Van Canneyt via fpc-pascal 
>  wrote:
> 
> Should be easy enough.

Great, glad this is happening. Let me know if you want some help with these 
additions or if it's best for you to just blast through it on you own. Seems 
like you know what's best for the RTL though honestly.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

On Thu, 8 Oct 2020, Ryan Joseph via fpc-pascal wrote:





On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal 
 wrote:

// Read raw content as bytes

Function GetFileContents(Const aFileName : RawByteString) : TBytes;
Function GetFileContents(Const aFileName : UnicodeString) : TBytes;
Function GetFileContents(Const aHandle : THandle) : TBytes;

// Read content as string

// Assume TEncoding.SystemEncoding
Function GetFileAsString(Const aFileName : RawByteString) : RawByteString;
// Specify encoding
Function GetFileAsString(Const aFileName : RawByteString; aEncoding : 
TEncoding) : RawByteString;
// Assume TEncoding.Unicode contents
Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString;
// Specify encoding, return Unicode string.
Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : 
TEncoding) : UnicodeString;


Michael,  Is it possible to extend this same set of functions to writing? 
Naturally I need to write back to the file now and the same problem

presents itself.  I have to search through other code bases to find a
function or Google and find
https://wiki.freepascal.org/File_Handling_In_Pascal which offers a whole
host of options.  SaveStringToFile from the wiki link wraps TFileStream
but having a single low-level function would be best.


Should be easy enough.

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal 
>  wrote:
> 
> // Read raw content as bytes
> 
> Function GetFileContents(Const aFileName : RawByteString) : TBytes;
> Function GetFileContents(Const aFileName : UnicodeString) : TBytes;
> Function GetFileContents(Const aHandle : THandle) : TBytes;
> 
> // Read content as string
> 
> // Assume TEncoding.SystemEncoding
> Function GetFileAsString(Const aFileName : RawByteString) : RawByteString;
> // Specify encoding
> Function GetFileAsString(Const aFileName : RawByteString; aEncoding : 
> TEncoding) : RawByteString;
> // Assume TEncoding.Unicode contents
> Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString;
> // Specify encoding, return Unicode string.
> Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : 
> TEncoding) : UnicodeString;

Michael,  Is it possible to extend this same set of functions to writing? 
Naturally I need to write back to the file now and the same problem presents 
itself. I have to search through other code bases to find a function or Google 
and find https://wiki.freepascal.org/File_Handling_In_Pascal which offers a 
whole host of options. SaveStringToFile from the wiki link wraps TFileStream 
but having a single low-level function would be best. 

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

Here's another interesting option I considered. You can call Scandir which 
returns a record with an enumerator. You can then use this to drop right into 
the for loop. It doesn't allocate memory and you can break the loop to stop the 
iteration. The benefit is you can avoid costly memory allocations for large 
directories but I don't think it's a replacement for a function that returns an 
array of TStringList.

type  
TDirectoryEnumerator = record
  path: string;
  count: integer;
name: string;
info: TSearchRec;
function GetEnumerator: TDirectoryEnumerator;
function MoveNext: boolean;  
property Current: string read name;  
end;

function TDirectoryEnumerator.GetEnumerator: TDirectoryEnumerator;
begin
result := self;
end;

Function TDirectoryEnumerator.MoveNext: Boolean;  
begin  
  if count = 0 then
begin
if FindFirst(path, faAnyFile, info) = 0 then
begin
name := info.name;
inc(count);
result := true;
end
else
result := false;
end
  else if FindNext(info) <> 0 then
begin
FindClose(info);
result := false;
end
else
begin
name := info.name;
inc(count);
result := true;
end;
end;  

function ScanDir(path: string): TDirectoryEnumerator;
begin
if not DirectoryExists(path) then
raise Exception.Create('Directory "'+path+'"" doesn''t exist');
result.path := path+DirectorySeparator+'*';
result.count := 0;
end;



Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 7, 2020, at 2:19 AM, Michael Van Canneyt via fpc-pascal 
>  wrote:
> 
> I see Delphi has something similar, so I will add an implementation. But I 
> need to study the options they provide so we can make a reasonably
> compatible version :)

Excellent thanks. It looks like Lazarus has FindAllFiles but it doesn't return 
a dynamic array (the important part so it can be dropped into for..in loops). 
Many scripting languages have a similar function (I can think of C#, PHP, 
Python and Javascript off the top of my head).

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

On Tue, 6 Oct 2020, Ryan Joseph via fpc-pascal wrote:


Since we're on the topic how about another one-liner for reading all the files 
in directory into a dynamic array? This has the added benefit of getting 
enumeration for free. This is standard stuff for working with files in 
scripting languages so I think the FPC RTL should include something like this 
also.



type
 TStringArray = array of string;

function FindAllFiles(path: string): TStringArray;
var
 info: TSearchRec;
begin
 if not DirectoryExists(path) then
   raise Exception.Create('Directory "'+path+'"" doesn''t exist');
 path := path+DirectorySeparator+'*';
 result := [];
 if FindFirst(path, faAnyFile, info) = 0 then
   begin
 repeat
   result += [info.name];
 until FindNext(info) <> 0;
 FindClose(info);
   end;
end;

begin
 // fastest way to read all text files in a directory.
 // only 2 lines doesn't leak memory
 for name in FindAllFiles('/usr/local/lib/fpc') do
   writeln(GetFileAsString('/usr/local/lib/fpc/'+name));
end.


I see Delphi has something similar, so I will add an implementation. 
But I need to study the options they provide so we can make a reasonably

compatible version :)

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 6, 2020, at 1:42 AM, Jer Haan via fpc-pascal 
>  wrote:
> 
> I use this function to read a file into a string:

Not sure how Michael implemented it but we have this in the RTL now, so we can 
throw away our ReadFile functions. I have many of those littered around in 
various projects so I'm looking forward to killing it. ;)

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

Since we're on the topic how about another one-liner for reading all the files 
in directory into a dynamic array? This has the added benefit of getting 
enumeration for free. This is standard stuff for working with files in 
scripting languages so I think the FPC RTL should include something like this 
also.



type
  TStringArray = array of string;

function FindAllFiles(path: string): TStringArray;
var
  info: TSearchRec;
begin
  if not DirectoryExists(path) then
raise Exception.Create('Directory "'+path+'"" doesn''t exist');
  path := path+DirectorySeparator+'*';
  result := [];
  if FindFirst(path, faAnyFile, info) = 0 then
begin
  repeat
result += [info.name];
  until FindNext(info) <> 0;
  FindClose(info);
end;
end;

begin
  // fastest way to read all text files in a directory.
  // only 2 lines doesn't leak memory
  for name in FindAllFiles('/usr/local/lib/fpc') do  
writeln(GetFileAsString('/usr/local/lib/fpc/'+name));
end.


Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

On Tue, 6 Oct 2020, Bart via fpc-pascal wrote:


On Tue, Oct 6, 2020 at 10:12 AM Michael Van Canneyt via fpc-pascal
 wrote:



// Assume TEncoding.SystemEncoding
Function GetFileAsString(Const aFileName : RawByteString) : RawByteString;
// Specify encoding
Function GetFileAsString(Const aFileName : RawByteString; aEncoding : 
TEncoding) : RawByteString;
// Assume TEncoding.Unicode contents
Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString;
// Specify encoding, return Unicode string.
Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : 
TEncoding) : UnicodeString;


Now I want to get the contents of a file, which contains either UTF8
or some single-byte encoding, and the filename is NOT in my locale
(e.g. it may be chinese, which I cannot represent in RawByteString),


Of course you can. That is why we use RawByteString for file opening
functions.  You specify it as UTF8 in that case, the RTL will do the rest.

The whole of lazarus works on this principle.

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

On Tue, 6 Oct 2020, Ryan Joseph via fpc-pascal wrote:





On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal 
 wrote:

I added the following functions to the sysutils unit (rev 47056):


Great, thanks Michael.

I've always used AnsiString so why is UnicodeString preferable here?


If the original file contains UTF16, it will save you a conversion.

I usually do not use UTF16, you clearly also not, but other people do, 
and we try to cater for them as well :-)



So is the idea we need to specify an UTF-8 encoding for unicode otherwise
it assumes ASCII format?


It will assume the system unit default encoding. What that is depends on your 
OS.
Mac, Linux: UTF8, windows: ANSI, unless you use Lazarus in which case it
will also be UTF8.

(in fact you can set it to any codepage you want, but the above are the 
defaults)


I didn't specify the encoding with TStringList
and I always seemed to get back what I wanted (maybe it sniffed the
encoding from the file?).


Depending on your environment TStringList will use ANSI or UTF8, just as
above.

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Bart via fpc-pascal]

On Tue, Oct 6, 2020 at 10:12 AM Michael Van Canneyt via fpc-pascal
 wrote:


> // Assume TEncoding.SystemEncoding
> Function GetFileAsString(Const aFileName : RawByteString) : RawByteString;
> // Specify encoding
> Function GetFileAsString(Const aFileName : RawByteString; aEncoding : 
> TEncoding) : RawByteString;
> // Assume TEncoding.Unicode contents
> Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString;
> // Specify encoding, return Unicode string.
> Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : 
> TEncoding) : UnicodeString;

Now I want to get the contents of a file, which contains either UTF8
or some single-byte encoding, and the filename is NOT in my locale
(e.g. it may be chinese, which I cannot represent in RawByteString),
but when I specify the filename as UnicodeString I am forced to
receive the contents Unicode as well.
From my perspective that sucks.


-- 
Bart
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

Well, the answer is twofold:

1. IOUtils does not exist yet. Now the OP has a solution.

2. I don't use IOUtils and don't plan to.
   Its .NET-like approach has no added value for me.
   Just like I don't use rtii unit, but always use the more low-level
   typinfo unit.

But once TFile will be implemented, it can obviously reuse the functions from
Sysutils. (like it presumably will for 99% of all other functions in it). 
If need be the API can be extended in SysUtils to match what IOUtils needs..


Michael.

On Tue, 6 Oct 2020, Nico Neumann wrote:


Wouldn't it be better to implement it directly as *IOUtils.TFile* (like
Deplhi)?




*class function ReadAllBytes(const Path: string): TBytes; static;class
function ReadAllLines(const Path: string): TStringDynArray;overload;
static;class function ReadAllLines(const Path: string; const Encoding:
TEncoding): TStringDynArray; overload; static;class function
ReadAllText(const Path: string): string; overload; inline; static;class
function ReadAllText(const Path: string; const Encoding: TEncoding):
string; overload; inline; static;*
Souce:
http://docwiki.embarcadero.com/Libraries/Sydney/en/System.IOUtils.TFile_Methods
Then there would be only one way in the RTL for this task and not two once
*IOUtils* (foundation webpage says it's planned anyway) is implemented.

Am Di., 6. Okt. 2020 um 10:12 Uhr schrieb Michael Van Canneyt via
fpc-pascal :




On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote:





On Oct 5, 2020, at 5:08 PM, Jean SUZINEAU via fpc-pascal <

fpc-pascal@lists.freepascal.org> wrote:


In my own code I use BlockRead/BlockWrite, but I'm wondering if I've

not seen this somewhere in RTL.





This looks good to me what about the concerns raised by Michael?  I don't
know enough about text formats but using AnsiString always just seems to
work for me (I assume the compiler did something magic behind the

scenes).

No, we don't deal in magic, only bits and bytes :-)

I added the following functions to the sysutils unit (rev 47056):

// Read raw content as bytes

Function GetFileContents(Const aFileName : RawByteString) : TBytes;
Function GetFileContents(Const aFileName : UnicodeString) : TBytes;
Function GetFileContents(Const aHandle : THandle) : TBytes;

// Read content as string

// Assume TEncoding.SystemEncoding
Function GetFileAsString(Const aFileName : RawByteString) : RawByteString;
// Specify encoding
Function GetFileAsString(Const aFileName : RawByteString; aEncoding :
TEncoding) : RawByteString;
// Assume TEncoding.Unicode contents
Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString;
// Specify encoding, return Unicode string.
Function GetFileAsString(Const aFileName : UnicodeString; aEncoding :
TEncoding) : UnicodeString;

These functions will raise an exception if the file cannot be opened or
read.
They cannot be used on handles that do not support FileSeek() (sockets,
pipes, stdin/stdout etc.).

I did some tests on encoding conversion but not extensively.
If you find any errors, please report them through the bugtracker.

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal




___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Nico Neumann via fpc-pascal]

Wouldn't it be better to implement it directly as *IOUtils.TFile* (like
Deplhi)?




*class function ReadAllBytes(const Path: string): TBytes; static;class
function ReadAllLines(const Path: string): TStringDynArray;overload;
static;class function ReadAllLines(const Path: string; const Encoding:
TEncoding): TStringDynArray; overload; static;class function
ReadAllText(const Path: string): string; overload; inline; static;class
function ReadAllText(const Path: string; const Encoding: TEncoding):
string; overload; inline; static;*
Souce:
http://docwiki.embarcadero.com/Libraries/Sydney/en/System.IOUtils.TFile_Methods
Then there would be only one way in the RTL for this task and not two once
*IOUtils* (foundation webpage says it's planned anyway) is implemented.

Am Di., 6. Okt. 2020 um 10:12 Uhr schrieb Michael Van Canneyt via
fpc-pascal :

>
>
> On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote:
>
> >
> >
> >> On Oct 5, 2020, at 5:08 PM, Jean SUZINEAU via fpc-pascal <
> fpc-pascal@lists.freepascal.org> wrote:
> >>
> >> In my own code I use BlockRead/BlockWrite, but I'm wondering if I've
> not seen this somewhere in RTL.
> >>
> >>
> >
> > This looks good to me what about the concerns raised by Michael?  I don't
> > know enough about text formats but using AnsiString always just seems to
> > work for me (I assume the compiler did something magic behind the
> scenes).
>
> No, we don't deal in magic, only bits and bytes :-)
>
> I added the following functions to the sysutils unit (rev 47056):
>
> // Read raw content as bytes
>
> Function GetFileContents(Const aFileName : RawByteString) : TBytes;
> Function GetFileContents(Const aFileName : UnicodeString) : TBytes;
> Function GetFileContents(Const aHandle : THandle) : TBytes;
>
> // Read content as string
>
> // Assume TEncoding.SystemEncoding
> Function GetFileAsString(Const aFileName : RawByteString) : RawByteString;
> // Specify encoding
> Function GetFileAsString(Const aFileName : RawByteString; aEncoding :
> TEncoding) : RawByteString;
> // Assume TEncoding.Unicode contents
> Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString;
> // Specify encoding, return Unicode string.
> Function GetFileAsString(Const aFileName : UnicodeString; aEncoding :
> TEncoding) : UnicodeString;
>
> These functions will raise an exception if the file cannot be opened or
> read.
> They cannot be used on handles that do not support FileSeek() (sockets,
> pipes, stdin/stdout etc.).
>
> I did some tests on encoding conversion but not extensively.
> If you find any errors, please report them through the bugtracker.
>
> Michael.
> ___
> fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
>
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Alexander Grotewohl via fpc-pascal]

Not 100% on this but I think the gist is that UnicodeString is compatible with 
AnsiString and a conversion is done on assignment.

--
Alexander Grotewohl
https://dcclost.com


From: fpc-pascal  on behalf of Ryan 
Joseph via fpc-pascal 
Sent: Tuesday, October 6, 2020 12:39:43 PM
To: FPC-Pascal users discussions 
Cc: Ryan Joseph 
Subject: Re: [fpc-pascal] Adding file to string to the RTL



> On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal 
>  wrote:
>
> I added the following functions to the sysutils unit (rev 47056):

Great, thanks Michael.

I've always used AnsiString so why is UnicodeString preferable here?

So is the idea we need to specify an UTF-8 encoding for unicode otherwise it 
assumes ASCII format? I didn't specify the encoding with TStringList and I 
always seemed to get back what I wanted (maybe it sniffed the encoding from the 
file?).

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal 
>  wrote:
> 
> I added the following functions to the sysutils unit (rev 47056):

Great, thanks Michael.

I've always used AnsiString so why is UnicodeString preferable here?

So is the idea we need to specify an UTF-8 encoding for unicode otherwise it 
assumes ASCII format? I didn't specify the encoding with TStringList and I 
always seemed to get back what I wanted (maybe it sniffed the encoding from the 
file?).

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote:





On Oct 5, 2020, at 5:08 PM, Jean SUZINEAU via fpc-pascal 
 wrote:

In my own code I use BlockRead/BlockWrite, but I'm wondering if I've not seen 
this somewhere in RTL.




This looks good to me what about the concerns raised by Michael?  I don't
know enough about text formats but using AnsiString always just seems to
work for me (I assume the compiler did something magic behind the scenes).


No, we don't deal in magic, only bits and bytes :-)

I added the following functions to the sysutils unit (rev 47056):

// Read raw content as bytes

Function GetFileContents(Const aFileName : RawByteString) : TBytes;
Function GetFileContents(Const aFileName : UnicodeString) : TBytes;
Function GetFileContents(Const aHandle : THandle) : TBytes;

// Read content as string

// Assume TEncoding.SystemEncoding
Function GetFileAsString(Const aFileName : RawByteString) : RawByteString;
// Specify encoding
Function GetFileAsString(Const aFileName : RawByteString; aEncoding : 
TEncoding) : RawByteString;
// Assume TEncoding.Unicode contents
Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString;
// Specify encoding, return Unicode string.
Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : 
TEncoding) : UnicodeString;

These functions will raise an exception if the file cannot be opened or read.
They cannot be used on handles that do not support FileSeek() (sockets, pipes, 
stdin/stdout etc.).

I did some tests on encoding conversion but not extensively. 
If you find any errors, please report them through the bugtracker.


Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Marco van de Voort via fpc-pascal]

Op 2020-10-05 om 20:45 schreef Ryan Joseph via fpc-pascal:

I often need to use a function which reads a file into a string, as is so 
common in so many scripting languages. Can it be considered to add something 
like this to the RTL? Since we have a refcounted Ansistring type it's natural 
for this to be a one-liner. Not saying we should use TStringList but that's the 
closest thing I found in the RTL.

function ReadFile(path: Ansistring): Ansistring;
var
   list: TStringList;
begin
   list := TStringList.Create;
   list.LoadFromFile(ExpandFileName(path));
   result := list.Text;
   list.Free;
end;

Stringlist static method loadstringfromfile ?
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Jer Haan via fpc-pascal]

I use this function to read a file into a string:

function ReadFile(const FileName: TFileName): String;
var
  InputFile: THandle;
  FileSize, BytesRead: Integer;
  Buffer: String='';
begin
  try
InputFile := FileOpen(FileName, fmOpenRead);
if InputFile = -1 then
  begin
WriteLn(Format('Error opening file "%s".', [FileName]));
Halt; // Exit?
  end;

FileSize := FileSeek(InputFile, 0, fsFromEnd);
SetLength(Buffer, FileSize);
FileSeek(InputFile, 0, fsFromBeginning);

BytesRead := FileRead(InputFile, Buffer[1], FileSize);

if BytesRead < FileSize then
  begin
WriteLn(Format('Error reading file "%s".', [FileName]));
Halt; // Exit?
  end;

Result := Buffer;
  finally
FileClose(InputFile);
  end;
end;


On 6 Oct 2020, at 09:25, Luca Olivetti via fpc-pascal 
 wrote:

El 6/10/20 a les 9:01, Michael Van Canneyt via fpc-pascal ha escrit:

> A simple filecreate, allocate buffer, fileread, fileclose will probably be 
> easiest.

Lazarus has a ReadFileToString in fileutil.

Bye
-- 
Luca

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Luca Olivetti via fpc-pascal]

El 6/10/20 a les 9:01, Michael Van Canneyt via fpc-pascal ha escrit:

A simple filecreate, allocate buffer, fileread, fileclose will probably 
be easiest.


Lazarus has a ReadFileToString in fileutil.

Bye
--
Luca

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote:





On Oct 5, 2020, at 3:23 PM, Michael Van Canneyt via fpc-pascal 
 wrote:

So I think you're looking at 6 or even 8 versions of this hypothetical 
function...


Ouch. :)

I'm sure this code already exists in the RTL though, right?  I assume it's
part of some classes like TStringList and we can just pull it up into a
collection of functions which are easily searchable in the RTL docs.


Using TStringList would be a mistake, since it will parse your file while
there is no need.

No, I think that if you want this, it must be built from the basic calls.



If not what is lowest level cross-platform layer which already exists and
can we build up from?  Maybe the "TextFile" type helps us?  (I know it's
ancient and I've never used it before honestly).


I don't think that would be a good strategy.

A simple filecreate, allocate buffer, fileread, fileclose will probably be 
easiest.

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 5, 2020, at 5:08 PM, Jean SUZINEAU via fpc-pascal 
>  wrote:
> 
> In my own code I use BlockRead/BlockWrite, but I'm wondering if I've not seen 
> this somewhere in RTL.
> 
> 

This looks good to me what about the concerns raised by Michael? I don't know 
enough about text formats but using AnsiString always just seems to work for me 
(I assume the compiler did something magic behind the scenes).

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Winfried Bartnick via fpc-pascal]

Am 06.10.20 um 01:08 schrieb Jean SUZINEAU via fpc-pascal:


In my own code I use BlockRead/BlockWrite, but I'm wondering if I've 
not seen this somewhere in RTL.


https://github.com/jsuzineau/pascal_o_r_mapping/blob/TjsDataContexte/pascal_o_r_mapping/02_Units/uuStrings.pas

function String_from_File( _FileName: String): String;
var
    F: File;
    Longueur: Integer;
begin
  Result:= '';
  if not FileExists( _FileName) then exit;
  AssignFile( F, _FileName);
  try
     Reset( F, 1);
     Longueur:= FileSize( F);
     if 0 = Longueur then exit;
     SetLength( Result, Longueur);
     BlockRead( F, Result[1], Longueur);
  finally
     CloseFile( F);
     end;
end;
procedure String_to_File( _FileName: String; _S: String);
var
    F: File;
begin
  if '' = _S then exit;
  AssignFile( F, _FileName);
  try
     ReWrite( F, 1);
     BlockWrite( F, _S[1], Length( _S));
  finally
     CloseFile( F);
     end;
end;



Hi!


Great idea!

Longueur  should be Int64


Winni





___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Jean SUZINEAU via fpc-pascal]

  
  
In my own code I use BlockRead/BlockWrite, but I'm wondering if
  I've not seen this somewhere in RTL.

https://github.com/jsuzineau/pascal_o_r_mapping/blob/TjsDataContexte/pascal_o_r_mapping/02_Units/uuStrings.pas
function String_from_File( _FileName: String): String;
var
   F: File;
   Longueur: Integer;
begin
 Result:= '';
 if not FileExists( _FileName) then exit;

 AssignFile( F, _FileName);
 try
    Reset( F, 1);
    Longueur:= FileSize( F);
    if 0 = Longueur then exit;
    SetLength( Result, Longueur);
    BlockRead( F, Result[1], Longueur);
 finally
    CloseFile( F);
    end;
end;

procedure String_to_File( _FileName: String; _S: String);
var
   F: File;
begin
 if '' = _S then exit;

 AssignFile( F, _FileName);
 try
    ReWrite( F, 1);
    BlockWrite( F, _S[1], Length( _S));
 finally
    CloseFile( F);
    end;
end;





  

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Ryan Joseph via fpc-pascal]

> On Oct 5, 2020, at 3:23 PM, Michael Van Canneyt via fpc-pascal 
>  wrote:
> 
> So I think you're looking at 6 or even 8 versions of this hypothetical 
> function...

Ouch. :)

I'm sure this code already exists in the RTL though, right? I assume it's part 
of some classes like TStringList and we can just pull it up into a collection 
of functions which are easily searchable in the RTL docs. 

If not what is lowest level cross-platform layer which already exists and can 
we build up from? Maybe the "TextFile" type helps us? (I know it's ancient and 
I've never used it before honestly).

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

[Michael Van Canneyt via fpc-pascal]

On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote:


I often need to use a function which reads a file into a string, as is so 
common in so many scripting languages. Can it be considered to add something 
like this to the RTL? Since we have a refcounted Ansistring type it's natural 
for this to be a one-liner. Not saying we should use TStringList but that's the 
closest thing I found in the RTL.

function ReadFile(path: Ansistring): Ansistring;
var
 list: TStringList;
begin
 list := TStringList.Create;
 list.LoadFromFile(ExpandFileName(path));
 result := list.Text;
 list.Free;
end;


I am inclined to agree, but there are some complications.

This will not work without some form of passing on the expected encoding. 
You will also need overloads for UnicodeString. Some people will want a

PChar or a buffer or a TBytes. Then you must allow the filename to be
specified as a single-byte string or 2-byte string.

So I think you're looking at 6 or even 8 versions of this hypothetical 
function...


Michael.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal