On Tue, Dec 17, 2019 at 8:07 PM Aishwarya Thangappa <aishwarya.thanga...@gmail.com> wrote: > > Sorry, I wasn't aware that attachments are not allowed in ASF Mailing lists. > I have now created them as gists. Please have a look. > > master systemd unit: > https://gist.github.com/ata18/e8f7577c99cd08ba46544aacef26969f > accumulo-service: > https://gist.github.com/ata18/48014ea78b09e4febb88480ea48ed62c
These first two links don't work for me. I get a 404 error message. For reference, here's the basic unit files I wrote for Accumulo from Fedora 29: https://src.fedoraproject.org/rpms/accumulo/tree/f29 They used a /usr/bin/accumulo script generated using the %jpackage_script macro (see accumulo.spec file for that) which worked a lot like Accumulo 2.0's bin/accumulo file works (not a coincidence, since the 2.0 script was written with insight gained from the attempt to package in Fedora). > accumulo-cluster: > https://gist.github.com/ata18/234c2e63d2718aec65bd2037ec3125cd This appears to be based on an older version of our accumulo-cluster script (from 2.0?) rather than the current one in the master branch, but I think I got the sense of what was changed by glancing at the diff. Once you have systemd, I'm not convinced it's beneficial to use something like accumulo-cluster anymore, as it doesn't really serve any added value beyond what you would get with using systemctl via pssh or pdsh and a hostsfile. The accumulo-cluster script's purpose is for when you don't have an existing service management tool for the cluster, and its intent is to be very basic, to support the "deploy out of tarball" use case, with no other vendor or downstream packaging. Modifying it to wrap systemd seems a bit unnecessarily complex to me, since I don't think you need it when using systemd. It might be better to create a simpler script that makes it easy to run specific tasks using pdsh or pssh, a hostsfile, to be used when using systemd, rather than trying to put those features into the accumulo-cluster script. > > Thanks, > Aishwarya > > On 2019/12/15 16:16:56, Michael Wall <mjw...@gmail.com> wrote: > > Hi Aishwarya, > > > > I didn't get any attachments on this. > > > > Thanks > > > > Mike > > > > On Fri, Dec 13, 2019 at 5:46 PM Aishwarya Thangappa > > <aishwarya.thanga...@microsoft.com.invalid> wrote: > > > > > Hello everyone, > > > > > > I have not subscribed to the dev mailing list earlier and missed on some > > > of your questions. I will address them here. > > > > > > @Christopher > > > Most of the changes except the actual installation of the systemd units > > > could possibly go into Accumulo. These would be the systemd units for > > > various accumulo services, modification to cluster-wide scripts in > > > accumulo > > > to use systemd instead of directly starting/stopping the processes. We > > > would be happy to accommodate/answer any suggestions or follow-up > > > questions > > > you may have. > > > > > > Attached the accumulo_cluster and accumulo_service scripts with systemd > > > changes. > > > > > > > > > @Keith Turner > > > Once we determine where the different pieces should land, I can post PRs > > > accordingly. In our current setup, in muchos.properties file I have added > > > a > > > `use_systemd` flag which when set to true, will overwrite the accumulo > > > cluster-wide scripts in the nodes with the attached ones. These files > > > currently reside at ansible/roles/accumulo/files. If we determine that > > > these scripts and the systemd unit files will instead go to Accumulo > > > project, I will have to make changes accordingly. > > > > > > @Michael Wall > > > Systemd units internally call the same scripts that accumulo_cluster > > > commands currently use. The change is that accumulo_cluster commands would > > > call systemd start/stop which inturn would call accumulo_service commands. > > > Attached a sample systemd_unit template. Can you please elaborate if this > > > is still an issue? > > > > > > ------------------------------ > > > *From:* Aishwarya Thangappa > > > *Sent:* Thursday, December 12, 2019 11:25 AM > > > *To:* dev@fluo.apache.org <dev@fluo.apache.org> > > > *Cc:* Arvind Shyamsundar <arvin...@microsoft.com>; Billie Rinaldi < > > > billie.rina...@microsoft.com> > > > *Subject:* Run Accumulo and Hadoop services under systemd > > > > > > Hi everyone, > > > > > > While using fluo-muchos to deploy an Accumulo cluster, we recognized the > > > need for various Accumulo and Hadoop services to be run under a service > > > manager like systemd which will ensure that all these services are brought > > > up correctly in the event of VM / OS reboots / cold starts. We have made > > > the required changes for this and would like to contribute it back to the > > > community if there is any interest around it. > > > > > > Summarizing what we have done: > > > > > > - Crafted separate systemd unit files for Accumulo (master, monitor, > > > gc, traser, tserver), Hadoop (journalnode, namenode, datanode, > > > resourcemanager, nodemanager, zkfc) and Zookeeper services. > > > - All of these unit files will be copied to the respective nodes' > > > /etc/systemd/system directory; the services will then be started and > > > enabled by ansible systemd module. > > > - In case of num_tservers > 1, multiple tserver systemd units will be > > > copied to the node and each will be independently managed. > > > - Also made necessary changes to the existing cluster-wide scripts > > > including accumulo_cluster, accumulo_service, start_dfs, start_yarn > > > etc., > > > to have them work seamlessly with sytemd. > > > > > > Is there an appetite to look at the details? If so, we can post a PR or if > > > there are any feedbacks and other considerations, please let us know and > > > we > > > can discuss them. > > > > > > Thanks, > > > Aishwarya > > > > > > > >
