[ http://issues.apache.org/jira/browse/DERBY-606?page=comments#action_12450071 ] Mayuresh Nirhali commented on DERBY-606: ----------------------------------------
Thanks Mike and Suresh. Yes. It makes sense to only change the log record format for this special case. Looking at possible runtime fix, I feel, this approach would create some Major-Minor version checks in write/readExternal methods which would not be ideal. Approach of Extending the class would make the changes much simpler. So, currently this is what I have in mind. CompressSpacePageOperation will be extended to form a new class CompressSpacePageOperation10_3. This class over override readExternal, writeExternal and getTypeFormatId methods. read/writeExternal methods would write the new_highest_page field without compression and the other fields are compressed and written to the Log. A new ID StoreFormatIds.LOGOP_COMPRESS10_3_SPACE is created. Now, when LOggableAllocActions creates CompressSpacePageOperation, a new check will be added to identify whether this is a soft or hard upgrade. The only way to do this is by calling LogToFile.checkVersion(Major, Minor). Based on the flag returned either the old CompressSpacePageOperation or the new one will be called. LogFactory is not accessible at this level in the code and hence calling checkVersion is not possible. Xact has no interface to get XContext so there isnt any way to reach LogFactory. I need some inputs here. ------ While investigating, I also realized that even if you write the record with newHighestPage set to -1, the log is written successfully but the transaction does not commit because AllocPage.compressSpace has an assertion (only in DEBUG mode btw) which checks if the newHighestPage is not positive. Thus an Exception is thrown. I think this assertion check is invalid. Also it causes inconsistency in the behavior between sane-insane builds as it only happens when SanityManager.DEBUG is true. Will it be okay to remove this assertion ? ----- Lastly, If the compress space operation involving data pages spanned across multiple AllocExtents (which is the current case), the compress operation happens only on the last page. Ideally, it should clean up the last AllocPage (with referred datapages) and also clean up data pages which might be empty in the previous AllocationExtent. I realized this when I saw a very small decrease in physical size when I ran the reproducible testcase. Can someone please confirm this ?? So, I can file a JIRA for this. > SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails on (very) large tables > -------------------------------------------------------------------- > > Key: DERBY-606 > URL: http://issues.apache.org/jira/browse/DERBY-606 > Project: Derby > Issue Type: Bug > Components: Store > Affects Versions: 10.1.1.0 > Environment: Java 1.5.0_04 on Windows Server 2003 Web Edition > Reporter: Jeffrey Aguilera > Assigned To: Mayuresh Nirhali > Attachments: A606Test.java > > > SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails with one of the following error > messages when applied to a very large table (>2GB): > Log operation null encounters error writing itself out to the log stream, > this could be caused by an errant log operation or internal log buffer full > due to excessively large log operation. SQLSTATE: XJ001: Java exception: ': > java.io.IOException'. > or > The exception 'java.lang.ArrayIndexOutOfBoundsException' was thrown while > evaluating an expression. SQLSTATE: XJ001: Java exception: ': > java.lang.ArrayIndexOutOfBoundsException'. > In either case, no entry is written to the console log or to derby.log. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
