Thanks Emmanuel for the sync and sharing. The approach looks pretty good, and is often seen in many mature products. The repair process is triggered when corruption is found while the server is running, or while restarting with a specific option? Or the both? If the repairing stuff is not easy to integrate, maybe a repair tool like the one Kiran worked out sounds good to use? Or during startup time checking the data/index, if not fine then Java system launching the tool process for the fixing? Just some thoughts, in case some useful.
I'm not very sure to rewrite JDBM though I know there're plenty of reasons to do so, as most software rewritings do. But if we have start with new, implementing something like B+ tree, that needs transaction support, I'm wondering if we could do it by leveraging already industry proven backend, because developing such backend may take long time effort and pretty much of resources. I'm wondering if Sqlite could serve the purpose well or not, or how it can be wrapped or adapted for the usage here. Again just a quick thought and in case somewhat useful. Regards, Kai -----Original Message----- From: Emmanuel Lécharny [mailto:[email protected]] Sent: Sunday, January 24, 2016 9:40 PM To: Apache Directory Developers List <[email protected]> Subject: JdbmPartition repair Hi guys, we have many users complaining about a corrupted JDBM database. As of today, we don't have another solution than telling them to reload their data, which is all be comfortable. First because it might take ages (reloading data is very slow) and also because they might not have a backup. Although this is not a frequent scenario, when it happens, it really take down any credibility ApacheDS can have. Here, we all know that Mavibot will be the solution, but until it's available with transaction support, we have to propose a tool that restore - of possible - the database. Hopefully, Kiran has worked on a tool that does that : the partition-plumber. The idea is to intergrate this tool into ApacheDS in order to allow users to restore their database in a simple way. Here is what I propose : - first, a way to start ApacheDS in a repair mode. That will drip all the indexes, and recreate them based on the master table. It might take some time, but it will be way better than any solution, and in any case, will be faster than a full reload, consider that we will bypass many checks. I suggest an option : apacheds -repair. When the server is started with that option, the server will restart after having cleaned up the database - the way to implement it is to add a method in the Partition interface : repair(). Not all the partition will need it, so only JdbmPartition will actually iumplement it. - that method will simply delete (or copy, if we want a backup) all the existing indexes (system and users). We then will recreate the indexes based on the master table content. There is still a remote risk that the master table can be corrupted, but it's unlikely, or at least very rare. Actually, the Rdn index is the one which get corrupted most of the time, because it get updated many times for each addition, move, rename or delete operations. I'm currently working on that, and it should be done fast enough (say, in less than a week, or even quicker if I have enough time this sundy and at night). The next step, and I'm also working on that, is to finish Mavibot. The problem is that it's a complex piece of code, and it's hard to work on it when I just have a couple of hours on evening or during the week-end. I'm sorry for that. But we eventually will get it ready ! Thanks !
