On 7/22/14, 12:51 PM, Keith Turner wrote:
Had some discussion w/ Dave Marion about the need to drop relatavie paths from internal metadata. From a user standpoint the requirement to possibly configure instance.dfs.uri and instance.dfs.dir if they might have relative paths is confusing over the long term. Also it places more of a maintenance burden on us if we need to ensure all bug fixes and new features work properly w/ relative paths.
Assuming that we squash relative paths by 1.7.0, we shouldn't have any additional burden on new feature work because there should be no new features in 1.6. Bug fixes are still potentially more complex.
I think everyone would agree that 1.6.0 should've nuked relative paths (I'm sorry if I squash anyone's opinions, but that was the impression I got before 1.6.0 came out). I think trying to eradicate them in 1.6 would just add even more confusion to an already sufficiently confusing situation. If a sufficiently simple approach came be thought of for a 1.6.x, I would be open to hear it.
What are our options and what should the timeline be? We could require the user to do something to remove all relative paths before before starting 1.7.0 for example. Some of the things we discussed * Provide a utility to rewrite all relative paths * Rework the volume replacement code to work w/ relative paths A stand alone utility is tricky. Don't want to modify tablet metadata if the table is loaded. Thats why the volume replacement code has the tablets themselves do the replacement.
I think I like the idea of writing a standalone utility as, while the "safe" conditions to run such a utility are harder, getting the rewrite correct is much easier. Didn't Sean already write some sort of check for an "is Accumulo off" environment?
I like the idea of reworking the volume replacement code, but I do not like the idea of it happening automatically (like the first time 1.6.2 is started). Could possibly have a boolean config instance.volume.replaceRelative. When this is set, as tablets are loaded and when the GC starts relative paths would be replaced using current instance.dfs.* config or hdfs config. Still uncertain about the best solution. Looking for the course of least user confusion and least maintenance. I think instance.volume.replaceRelative is a bit confusing from a user perspective. What other options are there to solve this problem? Any issue w/ the premise?
