Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On Wed, 16 Aug 2017, Graeme Geldenhuys wrote: On 2017-07-25 09:54, Bo Berglund wrote: buffer in an old application, so I need to write efficient replacements for certain string functions (Delete, Insert, Copy etc). Now I am wondering if there is a better way to do the Insert() command than this: Reading that... All I can think of is the 1000's of Linked-List implementations I had to implement when I was studying Turbo Pascal all that years ago. Delete, Insert and Copy would be so easy to do. :-) What does TBytes use internally to store its data? It is an array. Hence a continuous block of memory. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 2017-07-25 09:54, Bo Berglund wrote: buffer in an old application, so I need to write efficient replacements for certain string functions (Delete, Insert, Copy etc). Now I am wondering if there is a better way to do the Insert() command than this: Reading that... All I can think of is the 1000's of Linked-List implementations I had to implement when I was studying Turbo Pascal all that years ago. Delete, Insert and Copy would be so easy to do. :-) What does TBytes use internally to store its data? Regards, Graeme -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 02.08.2017 09:29, Michael Van Canneyt wrote: FPC "forces" nothing. It offers a single-byte string TStrings implementation. If you want to use that for Unicode, you indeed have no choice but to use UTF8 or use/write a separate class that uses UTF16. That is why I said that it is less silly ! :) But it does force TStrings siblings to handle single-byte strings (While Delphi forces TStrings siblings to handle 16-bit-word Strings). As a huge count of classes in the RTL (and the LCL and MSEGUI and ..., and in user code) are or use TStrings siblings the compatibility issues and certain other problems are due to happen. (I hope I will not completely banned now, as I already am under monitoring for this kind of posts...) -Michael ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On Mon, 31 Jul 2017, Michael Schnell wrote: On 30.07.2017 12:37, Bo Berglund wrote: I asked about this problem over at Embarcadero too, but was flamed foreven thinking about using any kind of string for storing binary data. They are silly and defending their completely silly implementation of Code aware strings, forcing UTF-16 for any TStrings based classes. The fpc implementation (forcing UTF-8 for any TStrings based classes) is slightly less silly. Small correction: FPC "forces" nothing. It offers a single-byte string TStrings implementation. If you want to use that for Unicode, you indeed have no choice but to use UTF8 or use/write a separate class that uses UTF16. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 30.07.2017 12:37, Bo Berglund wrote: I asked about this problem over at Embarcadero too, but was flamed foreven thinking about using any kind of string for storing binary data. They are silly and defending their completely silly implementation of Code aware strings, forcing UTF-16 for any TStrings based classes. The fpc implementation (forcing UTF-8 for any TStrings based classes) is slightly less silly. -Michael ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 30.07.2017 08:06, Bo Berglund wrote: All of this is because I have found that using AnsiString is triggering data changes when the application is running in certaincountries (locales) and processing certain data values... I am rather sure that this can be avoided. -Michael ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
Am 30.07.2017 12:37 schrieb "Bo Berglund" : > > On Sun, 30 Jul 2017 09:33:59 +0200, Sven Barth via fpc-pascal > wrote: > > >> The application was started back in Delphi7 times when "string" > >> actually meant AnsiString and was a 1-byte per element container. > > > >You could always use RawByteString or a string with a fixed codepage > >instead if plain AnsiString. > > I asked about this problem over at Embarcadero too, but was flamed for > even thinking about using any kind of string for storing binary data. > THeyn did suggest RawByteString too, though so I went over the > application changing all buffers to RawByteString and it did work. > I also tried to set a fixed codepage for the application, but it is > hard to check if it actually does help. Not a fixed codepage for the application, but a string with fixed codepage, e.g. "String(CP_1242)" or so... > The root problem is what happens with string conversions on different > locales and some functions could not be easily modified to use > RawByteString. > So I decided to bite the bullet and convert the whole unit to TBytes > instead. Some 7000 code lines to go over... > > It has taken a good many hours now and I still have some intricate > problems to solve. It still won't compile without errors. > > > > >Just declare a type alias so that you can keep it compatible with older > >Delphi versions as well. > > > > Do you mean: > type > TMyString = RawByteString; Yes. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On Sun, 30 Jul 2017 09:33:59 +0200, Sven Barth via fpc-pascal wrote: >> The application was started back in Delphi7 times when "string" >> actually meant AnsiString and was a 1-byte per element container. > >You could always use RawByteString or a string with a fixed codepage >instead if plain AnsiString. I asked about this problem over at Embarcadero too, but was flamed for even thinking about using any kind of string for storing binary data. THeyn did suggest RawByteString too, though so I went over the application changing all buffers to RawByteString and it did work. I also tried to set a fixed codepage for the application, but it is hard to check if it actually does help. The root problem is what happens with string conversions on different locales and some functions could not be easily modified to use RawByteString. So I decided to bite the bullet and convert the whole unit to TBytes instead. Some 7000 code lines to go over... It has taken a good many hours now and I still have some intricate problems to solve. It still won't compile without errors. > >Just declare a type alias so that you can keep it compatible with older >Delphi versions as well. > Do you mean: type TMyString = RawByteString; -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
Am 30.07.2017 08:07 schrieb "Bo Berglund" : > > On Fri, 28 Jul 2017 16:35:05 +0200, Michael Schnell > wrote: > > >On 25.07.2017 10:54, Bo Berglund wrote: > >> so I need to write efficient replacements for certain string functions (Delete, Insert, Copy etc). > >Why do you think the string function (if using strictly just a single > >UTF-8 or RawByte branded String type) are not efficient ? > > > >If all string encoding brands in an operation are the same there will be > >no conversion. > > Sorry, > I was not clear in my question... > > What I meant to say is that I need to convert an existing application > that uses strings (AnsiString) as binary data buffers to instead use > TBytes buffers. Thus my quest is for functions operating like the > convenient string functions but on TBytes arrays. > > All of this is because I have found that using AnsiString is > triggering data changes when the application is running in certain > countries (locales) and processing certain data values... > > The application was started back in Delphi7 times when "string" > actually meant AnsiString and was a 1-byte per element container. You could always use RawByteString or a string with a fixed codepage instead if plain AnsiString. Just declare a type alias so that you can keep it compatible with older Delphi versions as well. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On Fri, 28 Jul 2017 16:35:05 +0200, Michael Schnell wrote: >On 25.07.2017 10:54, Bo Berglund wrote: >> so I need to write efficient replacements for certain string functions >> (Delete, Insert, Copy etc). >Why do you think the string function (if using strictly just a single >UTF-8 or RawByte branded String type) are not efficient ? > >If all string encoding brands in an operation are the same there will be >no conversion. Sorry, I was not clear in my question... What I meant to say is that I need to convert an existing application that uses strings (AnsiString) as binary data buffers to instead use TBytes buffers. Thus my quest is for functions operating like the convenient string functions but on TBytes arrays. All of this is because I have found that using AnsiString is triggering data changes when the application is running in certain countries (locales) and processing certain data values... The application was started back in Delphi7 times when "string" actually meant AnsiString and was a 1-byte per element container. By efficient I meant that I probably would write inefficient code if I had to create things like the mentioned functions myself and if the compiler did the job it would be more efficient. -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 25.07.2017 10:54, Bo Berglund wrote: so I need to write efficient replacements for certain string functions (Delete, Insert, Copy etc). Why do you think the string function (if using strictly just a single UTF-8 or RawByte branded String type) are not efficient ? If all string encoding brands in an operation are the same there will be no conversion. -Michael ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 25.07.2017 18:00, Martok wrote: ... FPC's intrinsics such as Insert() can already work with arrays: Nonetheless, IMHO using single-Byte Strings (UTF-8 or RawByte, as a proper "uncoded" string type brand does not exist), seems more convenient, especially, as here we have lazy copy on top of ref counting. -Michael ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On Tue, 25 Jul 2017 10:54:30 +0200, Bo Berglund wrote: >so I need to write efficient >replacements for certain string functions (Delete, Insert, Copy etc). What I am now up aginst after a few days work is to find how to code the Pos() function for strings... The comm buffer apparently contains the 2-char string as indication of the last and the existing code uses Pos() to find it. What about something like: function PosBytes(SearchItem, Target: TBytes): integer; begin ? end; What could I use inside the body of this function to mimik the string function Pos (for AnsiStrings)? -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 27/07/17 18:17, nore...@z505.com wrote: On 2017-07-25 11:00, Martok wrote: Ideally the function should be portable between FPC and Delphi XE5... You'd only need your own functions for Delphi, FPC's intrinsics such as Insert() can already work with arrays: var b, c: TBytes; begin b:= TBytes.Create(1,2,3); c:= TBytes.Create(10,11); Insert(c,b,2); -> b is now [1,2,10,11,3] The fpc wiki should probably be updated? I cannot find "insert" on this page: http://wiki.freepascal.org/Array I cannot find it here either: https://www.freepascal.org/docs-html/ref/refsu14.html For the last 10 years or so I've been criticizing occasionally the fact that everyone reinvents their own array algorithms for each array! So it seems that may solve it. In fact this problem goes back to the 1970's and 1980's when arrays were being used. Please list other functions other than insert that are available? Maybe someone add it to the wiki/docs ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal AFAIK, that is avaible only in the trunk version of FPC. You need to build the trunk version of the documents by yourself. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
Am 27.07.2017 17:18 schrieb : > > On 2017-07-25 11:00, Martok wrote: >>> >>> Ideally the function should be portable between FPC and Delphi XE5... >> >> You'd only need your own functions for Delphi, FPC's intrinsics such as Insert() >> can already work with arrays: >> >> var >> b, c: TBytes; >> begin >> b:= TBytes.Create(1,2,3); >> c:= TBytes.Create(10,11); >> Insert(c,b,2); >> >> -> b is now [1,2,10,11,3] >> >> > > The fpc wiki should probably be updated? > > I cannot find "insert" on this page: > http://wiki.freepascal.org/Array Then update it. > I cannot find it here either: > https://www.freepascal.org/docs-html/ref/refsu14.html The documentation is always for the latest release (currently 3.0.2, soon 3.0.4), but only trunk supports this as well as Delete() (Concat() is not yet implemented however). Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 2017-07-26 06:34, Sven Barth via fpc-pascal wrote: Am 26.07.2017 10:59 schrieb "Martok" : > Ideally the function should be portable between FPC and Delphi XE5... You'd only need your own functions for Delphi, FPC's intrinsics such as Insert() can already work with arrays: var b, c: TBytes; begin b:= TBytes.Create(1,2,3); c:= TBytes.Create(10,11); Insert(c,b,2); -> b is now [1,2,10,11,3] But only in trunk. And Delphi XE8 and newer also support them. Regards, Sven Oh, that would explain why it is not in the wiki/docs yet Only issue I can see, is with all Create's, programmers expect a matching Free. So that is a bit confusing, but so long as someone remembers arrays are garbage (reference rather) collected... ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On 2017-07-25 11:00, Martok wrote: Ideally the function should be portable between FPC and Delphi XE5... You'd only need your own functions for Delphi, FPC's intrinsics such as Insert() can already work with arrays: var b, c: TBytes; begin b:= TBytes.Create(1,2,3); c:= TBytes.Create(10,11); Insert(c,b,2); -> b is now [1,2,10,11,3] The fpc wiki should probably be updated? I cannot find "insert" on this page: http://wiki.freepascal.org/Array I cannot find it here either: https://www.freepascal.org/docs-html/ref/refsu14.html For the last 10 years or so I've been criticizing occasionally the fact that everyone reinvents their own array algorithms for each array! So it seems that may solve it. In fact this problem goes back to the 1970's and 1980's when arrays were being used. Please list other functions other than insert that are available? Maybe someone add it to the wiki/docs ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
Am 26.07.2017 10:59 schrieb "Martok" : > > > Ideally the function should be portable between FPC and Delphi XE5... > You'd only need your own functions for Delphi, FPC's intrinsics such as Insert() > can already work with arrays: > > var > b, c: TBytes; > begin > b:= TBytes.Create(1,2,3); > c:= TBytes.Create(10,11); > Insert(c,b,2); > > -> b is now [1,2,10,11,3] But only in trunk. And Delphi XE8 and newer also support them. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
> Ideally the function should be portable between FPC and Delphi XE5... You'd only need your own functions for Delphi, FPC's intrinsics such as Insert() can already work with arrays: var b, c: TBytes; begin b:= TBytes.Create(1,2,3); c:= TBytes.Create(10,11); Insert(c,b,2); -> b is now [1,2,10,11,3] -- Martok ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On Tue, 25 Jul 2017 11:15:31 +0200 (CEST), Michael Van Canneyt wrote: >There is no better way if you only want to use TBytes. Thanks, then I will continue using Move... -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Best way to insert bytes into a TBytes variable?
On Tue, 25 Jul 2017, Bo Berglund wrote: I am working on moving away from using AnsiString as communications buffer in an old application, so I need to write efficient replacements for certain string functions (Delete, Insert, Copy etc). Now I am wondering if there is a better way to do the Insert() command than this: procedure InsertBytes(var Dest: TBytes; Src: TBytes; Index: integer); var Len, LenA: integer; begin Len := Length(Dest); LenA := Length(Src); if LenA = 0 then exit; //Do nothing if Index > Len then Index := Len; //Make an append instead SetLength(Dest, Len + LenA); Move(Dest[Index], Dest[Index + LenA], LenA); //Create space Move(Src[0], Dest[Index], LenA); //Insert data end; Ideally the function should be portable between FPC and Delphi XE5... There is no better way if you only want to use TBytes. If you want less reallocations, you must reserve more memory for the array from the start (so length is a "capacity"), and keep a second count, the count of the number of bytes actually used. Then you're maybe better off working with a TMemory stream, which does all this for you. Or write a TBuffer class. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
