https://bz.apache.org/bugzilla/show_bug.cgi?id=64322
--- Comment #8 from sits <[email protected]> --- Hi - I don't think this issue is fixed. I still have massive performance issues with processing OLE2 files which have a very large number of small entries in them (for example - MSG files with huge numbers of recipients). The culprit is POIFSMiniStore which has this code below. You can see to get the data for a specific block, the for loop below reads all block data preceeding this block before returning the block's data. If you have code like mine which iterates over all document entries in the OLE2 file, you end up with O(N^2) complexity - reading massive parts of the file over and over again. Yes it is true using the InputStream argument to POIFSFileSystem dodges this since POI reads the entire file in memory, but this is not practical when processing massive OLE2 files. Can the code below be fixed in some way to just read the block requested rather than all preceeding it? It seems like a version of StreamBlockByteBufferIterator which doesn't read the current block's data but return's the block's offset would be a vast improvement? /** * Load the block at the given offset. */ protected ByteBuffer getBlockAt(final int offset) { // Which big block is this? int byteOffset = offset * POIFSConstants.SMALL_BLOCK_SIZE; int bigBlockNumber = byteOffset / _filesystem.getBigBlockSize(); int bigBlockOffset = byteOffset % _filesystem.getBigBlockSize(); // Now locate the data block for it Iterator<ByteBuffer> it = _mini_stream.getBlockIterator(); for(int i=0; i<bigBlockNumber; i++) { it.next(); } ByteBuffer dataBlock = it.next(); assert(dataBlock != null); // Position ourselves, and take a slice dataBlock.position( dataBlock.position() + bigBlockOffset ); ByteBuffer miniBuffer = dataBlock.slice(); miniBuffer.limit(POIFSConstants.SMALL_BLOCK_SIZE); return miniBuffer; } -- You are receiving this mail because: You are the assignee for the bug. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
