Dear Erich,
I already took note of the stream commands, they are in the pipeline (which is
becoming rather long now) of improvements. Thanks.
I know Unix (Mac is basically BSD Unix) type Rexx code run on Windows, I have
done that for my tests intensively. I just provided the Win version for easier
reading in case you are on Win and want to look at it with an editor. The one
is just a clone of the other. I have a ooRexx ppm the read the file (line by
line with ~lineIn :-)) and saves it under another name with the right <eol>.
Hope you can find something to improve the memory handling, twice as much
memory for the same thing seems like a waste :-)
For my understanding maybe you or someone else can explain how the garbage
collector in ooRexx works (in general terms). I notice that the memory
allocated to the rexx process only goes up and never comes down again (in my
test) and the total allocated memory is only released again to the OS when the
program lapses. I guess internally to the interpreter it is a different thing
but it puzzles me how the amount of allocated memory goes up at each iteration
of my little test program.
Hälsningar/Regards/Grüsse,
P.O. Jonsson
[email protected]
> Am 05.07.2017 um 22:04 schrieb Erich Steinböck <[email protected]>:
>
> Hi P.O.,
>
> This will be my last feedback for some time, promise.
> there's absolutely no need to apologize, or stop posting.
> Reporting bugs or (suspected) issues is always welcome!
>
>
> ooRexx5.0 need 7.6 seconds
> Don't use the lines() method repeatedly in a loop (please also see my other
> reply)
> The fastest way to read the lines of a file, is stream~arrayIn
> Instead of appending each line into a MutableBuffer, just read it with
> charIn(, size). This is of course even faster than arrayIn() and if you
> like, may even squeeze out another 10% if you use stream~open("read nobuffer")
>
>
> I haven't looked into the reported memory leak yet.
>
>
> I have tried to do a minimalistic test case
> Any Unix-style .rex program will also run on Windows .. no need to provide a
> separate version for Windows
>
> On Wed, Jul 5, 2017 at 6:44 PM, P.O. Jonsson <[email protected]
> <mailto:[email protected]>> wrote:
> Dear developers,
>
> This will be my last feedback for some time, promise.
>
> For reading files I have implemented mutable buffers that I convert only once
> the loading is finished. This takes me around the problems I get trying to
> read data directly into Stems or Arrays.
>
> Unfortunately there is still a high memory load for this. When I run my read
> routine to read 1 million lines input say 10 times the memory allocated to
> the rexx process increases for every iteration. This is the same if I do it
> in a routine or directly in the main program. Also DROPing everything between
> iterations does not help, for every iteration there is some 100 MB memory
> added to the rexx process. Since I had planned to open and read 1000 files or
> more in one go this will likely exhaust my memory :-(
>
> I have checked on different machines, this behavior is the same for WIN10,
> WIN7 (32 and 64 bit ooRexx) and for MAX OS (3 different machines tested).
>
> In ooRexx 4.2 the behavior is similar but not as grave, the increase in
> memory/iteration is only about half of what I see with ooRexx 5.0
>
> In addition to this problem I have noticed that reading files in the way I do
> it is around 3-4 times slower in ooRexx 5.0 compared to ooRexx 4.2. I only
> saw this on MAC OS, not on Windows, but I installed 4.2 and 5.0 on the same
> hardware several times after each other to confirm that I heard reindeers,
> not horses ;-)
>
> To make a „baseline“ test I wrote a C program that is doing roughly the same
> as the oorexx test. Reading in 1 Million lines in C byte by byte takes 1.3
> seconds, ooRexx 4.2 clocks in at 2.7 seconds (which is GREAT) and ooRexx5.0
> need 7.6 seconds. All on the same hardware. This can´t be right, can it?
>
> As before I have tried to do a minimalistic test case that should run on any
> platform
>
> https://www.dropbox.com/sh/f8hircvoz0ph6uh/AACQeoi8lyAB3aoVQV3ojRkPa?dl=0
> <https://www.dropbox.com/sh/f8hircvoz0ph6uh/AACQeoi8lyAB3aoVQV3ojRkPa?dl=0>
>
> If you are not interested in this kind of feedback just say so and I will
> remove myself from the developers list and go back to using 4.2.
>
> Hälsningar/Regards/Grüsse,
> P.O. Jonsson
> [email protected] <mailto:[email protected]>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> <http://sdm.link/slashdot>
> _______________________________________________
> Oorexx-devel mailing list
> [email protected] <mailto:[email protected]>
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org!
> http://sdm.link/slashdot_______________________________________________
> Oorexx-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
