I do not think issuing START for HDFS from INSTALL of HAWQ is a good idea. It may not be the right level of dependency between services.
I think the right work around is what you described - Add Service for HAWQ after cluster is installed. The right solution would be to move logic to verify HDFS into START of HAWQ. This is also helpful if HDFS configuration changed and is no longer suitable for HAWQ. Start will fail and user will need to reconfigure HAWQ - think HA got enabled for NameNode. As a feature improvement to Ambari, you can file a JIRA to provide support for INITIALIZE which may be executed post INSTALL but before START. INITIALIZE may also demand that it be executed by only one component instance. -Sumit ________________________________ From: Brian de la Motte <[email protected]> Sent: Thursday, March 12, 2015 9:47 AM To: [email protected] Cc: [email protected] Subject: Re: Adding install priority to custom services Hi Sumit, I guess that explains the problem we're having. The custom service is HAWQ and requires HDFS to be started. During the install of HAWQ, it runs 'gpinitsystem' which initializes everything and verifies it can talk to HDFS correctly. It tries writing to HDFS and if it can't for any reason, errors out and stops the rest of the installation. One workaround I noticed is if I install just HDFS and ZOOKEEPER only, and then install HAWQ custom service, everything works correctly since HDFS is in a started state. Could I start a component from another component? So in the INSTALL function of the custom service, start HDFS if it's not started? Or would that be considered bad practice? For now, I will try moving some of the INSTALL code to the START section as you mentioned, and add some logic to only run if needed. Thank you for the help and explanation! Sincerely, Brian On Wed, Mar 11, 2015 at 5:47 PM, Sumit Mohanty <[email protected]<mailto:[email protected]>> wrote: Brian, All INSTALLs are scheduled before all STARTs. Does the install of your service require HDFS to be started? What operations do you perform on HDFS during the INSTALL? Could you move them to the START of the CUSTOM_MASTER/SLAVE and if you can the role_command_order you specified should take care of the dependencies. -Sumit ________________________________ From: Brian de la Motte <[email protected]<mailto:[email protected]>> Sent: Wednesday, March 11, 2015 4:34 PM To: [email protected]<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]> Subject: Re: Adding install priority to custom services Hello Sid, I believe I tried what you asked for, but it still didn't work. I separated it like this... "CUSTOM_MASTER-INSTALL": ["CUSTOM_SLAVE-INSTALL", "NAMENODE-INSTALL", "DATANODE-INSTALL"], "CUSTOM_SLAVE-START": ["CUSTOM_MASTER-START"], "CUSTOM_MASTER-START": ["NAMENODE-START"], I searched for "START" in the ambari-agent.log but there were not start commands issued for NAMENODE or DATANODE before the install began. Am I missing something here? Thank you for your help. Sincerely, Brian When On Fri, Mar 6, 2015 at 7:18 PM, Siddharth Wagle <[email protected]<mailto:[email protected]>> wrote: Hi Brian, Make sure to define the rules separately for install from start, since ambari creates separate stages for these commands. This should resolve the issue, do let us know the outcome. BR, Sid Sent by Outlook<http://taps.io/outlookmobile> for Android _____________________________ From:Brian de la Motte Subject:Re: Adding install priority to custom services To:[email protected]<mailto:to%[email protected]> Cc:Ben Barnett Hello, I believe we are having the same issue as Satya. We have a service that is dependent on HDFS being started and have declared it in our role_command_order.json, but Ambari is installing the custom service before HDFS has started and erring out. It does install HDFS before the custom service, just not start HDFS, even though I have this in my role_command_order.json: "CUSTOM_SERVICE-INSTALL": ["CUSTOM_SERVICE-INSTALL", "NAMENODE-INSTALL", "NAMENODE-START", "DATANODE-START"], When reviewing the ambari-agent.log, it installs the namenode and datanode, but doesn't start the namenode at all. It gets the status of the namenode, and shows it's not running, but doesn't start it. Is the role_command_order.json need anything else inside it to force a service to be in a started state before installing another service? Thanks, Brian On Thu, Mar 5, 2015 at 3:36 AM, Satyanarayana Jampa <[email protected]<mailto:[email protected]>> wrote: Hi Sid, Below are the steps I followed to keep the service installation in an order. 1. I have modified the below file and restarted ambari-server: vi /var/lib/ambari-server/resources/stacks/HDP/2.0.6/role_command_order.json #added below lines "A_HANDLER-INSTALL" : ["B_HANDLER-INSTALL", "C_HANDLER-INSTALL"], "A_HANDLER-START": ["B_HANDLER-START", "C_HANDLER-START"], 2. But, while installing the services the installation order was happening in Alphabetical order. 3. I want the order to be C, B and A. Am I missing something here. Thanks, Satya. From: Siddharth Wagle [mailto:[email protected]<mailto:[email protected]>] Sent: 05 March 2015 01:58 To: [email protected]<mailto:[email protected]> Subject: Re: Adding install priority to custom services Hi Satya, Take a look at the ambari-server/src/main/resources/role_command_order.json This json structure is used to build the dependency graph between components. Every stack overrides this file to add order between new components that the stack introduces, so that the base copy of this file remains unchanged. In your case, if you add INSTALL time or START time dependencies between service2 to service1 components, Ambari will re-order the commands automatically. Note: Format of the keys in the json is "ComponentName"-"Command" (Commands: START, INSTALL, UPGRADE, SERVICE_CHECK). BR, Sid ________________________________ From: Satyanarayana Jampa <[email protected]<mailto:[email protected]>> Sent: Wednesday, March 04, 2015 3:08 AM To: [email protected]<mailto:[email protected]> Subject: Adding install priority to custom services I have created some custom services, and in my case the "service2" needs "service1" to be installed first as it is needed for service2 to function properly. How can I specify the dependency or order in which the services should be installed. Thanks, Satya.
