This seems like a reasonable 1st step, that would solve current upgrade issues. I suggest you submit this patch, close the current jira, and submit a new future improvement jira noting what would have to be done to eek out the last few bits (or documenting it somewhere appropriate in the code is also fine - just rather not see you have learned lost). Given that this soft upgrade stuff is new I would rather be safe this time.
/mikem
Suresh Thalamati wrote:
By looking at the code , it is kind of tricky to handle upgrade, because users can crash in the middle of recovery
and reboot again N times , which can generate new log for uncommitted transaction and also all the constant that manipulates LSN
are static now and are used by static routines in org.apache.derby.impl.store.raw.log.LogCounter.java, like :makeLogInstantAsLong().
To handle upgrade:
1) all static methods in LogCounter.java has to be replaced by instance methods , so that
in the same JVM, one can run a database that is in softupgrade mode and one that is new/upgraded..
2) modify the code to update the version number on disk(log control file ) to only when the recovery is complete;
currently it is updated in the boot code itself.
3) perform a checkpoint after recovery on upgrade, so that new log file limits can be uses, As it will make
sure that no old log instant format will be in use.
Instead of doing all of the above , I can reduce the limit to 2^31 -1 instead of 2^33-1 ;
this way I would not need to change LogCounter.FILE_NUMBER_SHIFT from 32 (old value) to 30,
which causes all upgrade issues.
with 2^31 -1 it will take approximately 68 years (2147483647/(24 * 60 * 60 * 365))
to hit the log file number limit if a new log file is generated every second., instead of 272 years
with 2^33-1.
If no one has objections , I would prefer to increase the log file number limit to 2^31 -1 and
avoid major upgrade issues. If any one strongly believes it should be 2^33 -1 , I will try
to do the necessary upgrade changes.
Any comments/suggestions ?
Thanks -suresht
Suresh Thalamati wrote:
I just realized , this patch actaully break's hard upgrade also, if log needs to be replayed.
Becuase I extract the log file number and log file postion from the instant that is on disk ,
differently after my change.
If any one else on the list is testing upgrade is hitting this problem, please let me know.
I can post a patch to backout the portion that cause upgrade issues.
Thanks -suresht
Mike Matrigali wrote:
I am looking at committing this. The changes look good to me, but I believe there are upgrade issues to handle.
For a hard upgrade either new or old databases are fine. For a soft upgrade I think there is a problem if the db generates enough log files to start using the new bits, and then the software is reverted to before the fix.
Seems like using the bits needs to be somehow only enabled for hard upgrade. It would be best if it was controlled just by hard upgrade, but if that is not possible then just doing it for databases created since this version would also work - but still leave problems with old hard upgraded databases.
I have reviewed the code and am running tests. I plan on committing this part of the fix and let you address upgrade issues with a follow on patch.
Suresh Thalamati wrote:
