> On April 15, 2016, 6:31 p.m., Suma Shivaprasad wrote: > > webapp/src/main/java/org/apache/atlas/web/setup/SetupSteps.java, line 59 > > <https://reviews.apache.org/r/45929/diff/3/?file=1344552#file1344552line59> > > > > what is the expected behaviour in unclean shutdown case when lock is > > not cleared? Is this an ephemeral node that will be cleared automatically? > > Hemanth Yamijala wrote: > I verified using Zookeeper Shell. The Lock nodes created by Curator are > ephemeral nodes. Hence, they are automatically cleared when the client > connection to Zookeeper breaks. The more important question though is what > happens if the setup is halfway through when this happens. To handle this > case, I took this approach: > > * It is critical to not have setup run multiple times in inconsistent > state as the results could be pretty much undefined. > * Given the above, it is safer, although very inconvenient to refuse to > run setup if something is detected as wrong. Manual fixing will be required > in this extreme scenario. > > To handle this, I have introduced a marker znode in Zookeeper to indicate > setup is in progress. This is deleted only when all setup steps are done. If > this znode is found on entering the critical section, Setup will fail with > SetupException and a message stating there was an unclean shutdown of setup > and this needs to be fixed manually. > > Like I said, in cases where the node failed to be cleared even when all > setup is done, this would be inconvenient. But the approach leans on safety > rather than convenience. Please let me know if this is OK. Leaving the issue > open until review is done.
Yes this makes sense since the user would get to know that there was an unclean shutdown during setup which could have caused other issues like disabling the hbase 'titan' table which means the next setup could fail. Should we add this as as a known issue in documentation - the hbase setup ? - Suma ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45929/#review129171 ----------------------------------------------------------- On April 18, 2016, 6:31 p.m., Hemanth Yamijala wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45929/ > ----------------------------------------------------------- > > (Updated April 18, 2016, 6:31 p.m.) > > > Review request for atlas. > > > Bugs: ATLAS-628 > https://issues.apache.org/jira/browse/ATLAS-628 > > > Repository: atlas > > > Description > ------- > > Implements capability in atlas to run a set of registered setup steps. > Currently the only step is setting up the backend graph repository. The > intent is to extend this later for setting up Kafka at least and maybe other > things like Solr. > > > Diffs > ----- > > distro/src/bin/atlas_start.py 7db74c0 > distro/src/conf/atlas-application.properties 4131240 > docs/src/site/twiki/Configuration.twiki 023f5a0 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphSchemaInitializer.java > PRE-CREATION > server-api/src/main/java/org/apache/atlas/setup/SetupException.java > PRE-CREATION > server-api/src/main/java/org/apache/atlas/setup/SetupStep.java PRE-CREATION > webapp/src/main/java/org/apache/atlas/Atlas.java 58c386d > webapp/src/main/java/org/apache/atlas/web/service/ActiveInstanceState.java > 6b39970 > > webapp/src/main/java/org/apache/atlas/web/service/AtlasZookeeperSecurityProperties.java > 40a6030 > webapp/src/main/java/org/apache/atlas/web/service/CuratorFactory.java > 773e04e > webapp/src/main/java/org/apache/atlas/web/setup/AtlasSetup.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/setup/AtlasSetupModule.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/setup/SetupSteps.java > PRE-CREATION > > webapp/src/test/java/org/apache/atlas/web/service/AtlasZookeeperSecurityPropertiesTest.java > 118587a > webapp/src/test/java/org/apache/atlas/web/setup/SetupStepsTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/45929/diff/ > > > Testing > ------- > > Existing ITs, UTs pass. > > * Manually verified setting up with both embedded HBase setup and external > HBase setup. > * Also ran atlas HA instances at exactly same time and verified the noted > exception does not happen. > > > Thanks, > > Hemanth Yamijala > >
