Re: images eating memory

[Gerard van Enk]

Kees Jongenburger wrote:

Hi we have some memory problems concerning images

I have been running some tests and it seams there are 2 problems with the current 1.6 relase of mmbase (see TESTS in this message)

MagicFile still s*cks. it eats memory (luckely it gives it back after a while
but when the server is bussy this is enough to get OutOfMemory errors

secondly Node node = cloud.getNode(99)
results in
SELECT * FROM install_images WHERE number=99;

With mysql this results in returning all the data (even the handle fiels)
(causing out of memmory errors / maybe in the resultset/maybe in the node cache)

I think the memory problems come from the result set, not from the node cache, because the decodeDBnodeFiled method of the databasesupport classes are putting $SHORTED in a field when it's binary.
Or is the test your using not calling the decodeDBnodeField()?

and node.getByteValue("handle")
results in SELECT handle FROM install_images WHERE number=99
once a byte[] has been loaded it is kept in memmory

When the $SHORTED has been put in the node, this is needed, because the handle isn't already somewhere in memory.

While testing I have made some modifications to the MMObjectBuilder
(anywhere where there was a "SELECT *" I changed this to "SELECT " +getNonByteArrayFields(tableName) This had great inpact . i can list 1000 images with no problem (even if the node cache is set to 4000 nodes).

Is this something we want in 1.6 ?

I'm ok with this change. But maybe wait untill after the 1.6.1 release?

I can't work on the 1.7 branch because it's broken ( at least on mysql and postgress) but how wil this problem be fixed in 1.7 ?

Is this the result of the query project? And if so, when and how is this going to change?

I would like to know if you think this i a good idea?
(Images.java has it's onw handle cache for small images)


secondly the MagicFile. for the moment I have disabled it for Images
for the code is so "complex" that I realy don't have a clue how to fix it.
I would like to make the MagicFile configurable so that it's possible to use the unix "file" command.

Isn't there somekind of java-library available doing the same things?

Gerard