Amanda: it should be possible and desireable to make all these configuration changes in advance. They're totally valid HBase configuration settings that could very-well exist in advance.
[Amanda]: I would like to know the smallest number of these that must be set in order for Trafodion to be installer. Are all these settings mandatory for installation/start? This is what I want/need to know. Does a user have to go and set all those settings by hand in advance of installation Trafodion (assuming we 'remove' the traf_*mods scripts)? On Tue, Feb 9, 2016 at 9:32 AM, Gunnar Tapper <tapper.gun...@gmail.com> wrote: > Amanda: it should be possible and desireable to make all these > configuration changes in advance. They're totally valid HBase configuration > settings that could very-well exist in advance. > > The worse situation is if the Trafodion settings clash with some setting > that the customer already has for some other reason. For example, can we > really change the staging setting without affecting something else? This is > why I am documenting these settings as requirements in the upcoming > Provisioning Guide with the reasoning behind the setting. > > Further, I really recommend that we separate Trafodion installation from > configuration changes in the infrastructure to ensure that it's possible to > change the configuration changes, test their impact, and THEN install > Trafodion. The testing could consist of something as simple as verify the > restart worked or as advanced as an application regression test. Very user > specific and dependent what else HDFS and HBase is being used for. For > example, what's the impact on the rest of the Hadoop environment if turning > on HDFS ACLs? > > Thanks, > > Gunnar > > Gunnar > > On Tue, Feb 9, 2016 at 9:45 AM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > >> I agree with that :) Thanks Qifan! >> >> On Tue, Feb 9, 2016 at 8:43 AM, Qifan Chen <qifan.c...@esgyn.com> wrote: >> >>> Hi Amanda, >>> >>> I know the topic that I brought up is a little bit different :-). >>> >>> But the objectives of both thoughts are pretty much in line with the >>> improvement of install experiences. >>> >>> Thank you. >>> >>> --Qifan >>> >>> On Tue, Feb 9, 2016 at 10:36 AM, Amanda Moran <amanda.mo...@esgyn.com> >>> wrote: >>> >>>> Thanks for the response Qifan... but I am not sure we are talking about >>>> the same thing. >>>> >>>> I am talking about the traf_<hadoop distro>_mods98 script that is run >>>> as apart of the Trafodion Installation scripts. It is used to setup HBase >>>> settings, copy hbase-trx*.jar file, and restart HBase, HDFS, and Zookeeper. >>>> >>>> I would like to know what are the smallest number of settings that need >>>> to be set in advance for Trafodion to install/start. >>>> >>>> Examples of things set: >>>> hbase-site hbase.master.distributed.log.splitting false >>>> >>>> hbase-site hbase.coprocessor.region.classes >>>> "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" >>>> >>>> hbase-site hbase.hregion.impl >>>> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion >>>> >>>> hbase-site hbase.regionserver.region.split.policy >>>> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy >>>> >>>> hbase-site hbase.snapshot.enabled true >>>> >>>> hbase-site hbase.bulkload.staging.dir /hbase-staging >>>> >>>> hbase-site hbase.regionserver.region.transactional.tlog true >>>> >>>> hbase-site hbase.snapshot.master.timeoutMillis 600000 >>>> >>>> hbase-site hbase.snapshot.region.timeout 600000 >>>> >>>> hdfs-site dfs.namenode.acls.enabled true >>>> >>>> hbase-site hbase.client.scanner.timeout.period 600000 >>>> >>>> Thanks all! >>>> >>>> On Thu, Feb 4, 2016 at 5:40 AM, Qifan Chen <qifan.c...@esgyn.com> >>>> wrote: >>>> >>>>> Hi Amanda, >>>>> >>>>> I like your idea. >>>>> >>>>> On Trafodion's modification of config files for HBase, HDFS, and >>>>> Zookeeper, I wonder if it is possible to re-difine the port range for an >>>>> install, without going through all the install steps. This has the >>>>> practical value that due to port conflict, an install could not start >>>>> successfully, and one can easily switch to a different port range to retry >>>>> with this capability. >>>>> >>>>> In addition, some of the install steps, such as TPCDS data set >>>>> generation, and conversion to UTF8, could be made parallel to shorten the >>>>> install duration. They are done one table at a time in serial mode now. >>>>> >>>>> TPCDS table data is generated through the command "dsdgen -scale ..." >>>>> which could take the option "-parallel". >>>>> Data conversion to UTF8 is done through the iconv utility. >>>>> >>>>> Regards, --Qifan >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Feb 3, 2016 at 10:34 PM, Amanda Moran <amanda.mo...@esgyn.com> >>>>> wrote: >>>>> >>>>>> HI there All- >>>>>> >>>>>> The traf_*_mods scripts are used during installation to set HBase, >>>>>> HDFS, and Zookeeper settings needed by Trafodion. >>>>>> >>>>>> The script takes a long time to run (slowing down installation by a >>>>>> lot) because it needs to restart all these services for the changes to be >>>>>> picked up. >>>>>> >>>>>> My question is: What if I wanted to set the settings in advance of >>>>>> running the installer. What would be the absolute minimum set of settings >>>>>> that I need to get installed? I could set other settings later with a >>>>>> script or by hand. >>>>>> >>>>>> Thanks! >>>>>> -- >>>>>> Thanks, >>>>>> >>>>>> Amanda Moran >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, --Qifan >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> >>>> Amanda Moran >>>> >>> >>> >>> >>> -- >>> Regards, --Qifan >>> >>> >> >> >> -- >> Thanks, >> >> Amanda Moran >> > > > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* > -- Thanks, Amanda Moran
