This is the exception that I see -
16 Sep 2016 17:55:12,286 ERROR [qtp-ambari-client-25]
AbstractResourceProvider:1024 - Can't determine service state.
org.apache.ambari.server.HostNotFoundException: Host not found, cluster=c1,
hostname=bluedata-78.bdlocal
at
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getHostComponents(AmbariManagementControllerImpl.java:1182)
at
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getHostComponents(AmbariManagementControllerImpl.java:3055)
at
org.apache.ambari.server.controller.internal.ServiceResourceProvider$DefaultServiceState.getState(ServiceResourceProvider.java:947)
at
org.apache.ambari.server.controller.internal.ServiceResourceProvider.calculateServiceState(ServiceResourceProvider.java:904)
at
org.apache.ambari.server.controller.internal.ServiceResourceProvider.getResources(ServiceResourceProvider.java:198)
at
org.apache.ambari.server.controller.internal.ClusterControllerImpl$ExtendedResourceProviderWrapper.queryForResources(ClusterControllerImpl.java:945)
at
org.apache.ambari.server.controller.internal.ClusterControllerImpl.getResources(ClusterControllerImpl.java:132)
at
org.apache.ambari.server.api.query.QueryImpl.doQuery(QueryImpl.java:508)
at
org.apache.ambari.server.api.query.QueryImpl.queryForResources(QueryImpl.java:380)
at
org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:216)
at
org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:68)
at
org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:135)
at
org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:106)
at
org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:75)
at
org.apache.ambari.server.api.services.ServiceService.getServices(ServiceService.java:87)
at sun.reflect.GeneratedMethodAccessor178.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
On Fri, Sep 16, 2016 at 4:24 PM, Kartik Mathur <kar...@bluedata.com> wrote:
> ok sure ,
>
> I am doing a clean test again ,will send it to you sometime today
>
> Thanks
> Kartik
>
> On Fri, Sep 16, 2016 at 4:22 PM, Sumit Mohanty <smoha...@hortonworks.com>
> wrote:
>
>> The steps you have are correct to delete any host.
>>
>>
>> What is the internal server error that you see?
>>
>>
>> If possible, can you export Ambari database after server reports error.
>> You can send the db dump directly to me as Apache server may not deliver
>> the message with attachment or large attachments.
>> ------------------------------
>> *From:* Kartik Mathur <kar...@bluedata.com>
>> *Sent:* Friday, September 16, 2016 3:26 PM
>>
>> *To:* Sumit Mohanty; user@ambari.apache.org
>> *Subject:* Re: Unable to delete host from ambari using API (HDP2.4 with
>> Ambari 2.2)
>>
>> Hi Sumit,
>>
>> I don't see the host anymore but my delete never succeeded . This is the
>> third host I tried deleting form the same cluster , I don't see any of the
>> hosts for which my delete api calls failed however in ambari server logs I
>> still get exception for first host that I tried to delete.
>>
>> To give you a background , I am working on a python script to ingrate
>> with our product which decommissions and delete a host. Broadly these are
>> the steps I try to follow -
>>
>> 1) decommission data node
>> 2) stop all services running on the host
>> 3) delete all host components
>>
>> uptill this point everything is fine
>> 4) stop ambari agent on the host to be deleted
>> 5) issue delete for the host
>>
>> I am unable to do step 5 , it gives me internal server error and then
>> some random things on ambari UI where sometimes I see that host and
>> sometimes not. I tried trying the same after restarting ambari server .
>>
>>
>> Am I missing something ?
>>
>> I try again on a new cluster , if you thing cluster is in a messy state
>> right now.
>>
>> Thanks
>> Kartik
>>
>> On Fri, Sep 16, 2016 at 3:18 PM, Sumit Mohanty <smoha...@hortonworks.com>
>> wrote:
>>
>>> This is interesting. Can you try the steps after restart ambari and
>>> refreshing the browser?
>>> ------------------------------
>>> *From:* Kartik Mathur <kar...@bluedata.com>
>>> *Sent:* Friday, September 16, 2016 1:33 PM
>>> *To:* Sumit Mohanty; user@ambari.apache.org
>>> *Subject:* Re: Unable to delete host from ambari using API (HDP2.4 with
>>> Ambari 2.2)
>>>
>>> No api/v1/hosts give me the same error , host not found.
>>> Can you suggest how to debug this ?
>>>
>>> Thanks
>>> Kartik
>>>
>>> On Fri, Sep 16, 2016 at 12:22 PM, Sumit Mohanty <
>>> smoha...@hortonworks.com> wrote:
>>>
>>>> Can you delete from api/v1/hosts too?
>>>>
>>>> Get Outlook for iOS <https://aka.ms/o0ukef>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Sep 16, 2016 at 12:15 PM -0700, "Kartik Mathur" <
>>>> kar...@bluedata.com> wrote:
>>>>
>>>>
>>>> Hi Sumit thanks for replying , I stopped ambari-agent on the host and
>>>> restarted ambari server now I am getting host not found from UI as well as
>>>> from the api , however I can still see the host on ambari UI.
>>>>
>>>> This is the message from ambari UI
>>>>
>>>> 404 status codereceived on DELETE method for API:
>>>> /api/v1/clusters/h2/hosts/bluedata-68.bdlocal
>>>>
>>>> *Error message: *Not Found
>>>>
>>>> On Fri, Sep 16, 2016 at 8:44 AM, Sumit Mohanty <
>>>> smoha...@hortonworks.com> wrote:
>>>>
>>>>> Have you stopped the agent running on the host? If not then it will
>>>>> re-register.
>>>>>
>>>>>
>>>>> After stopping the agent, delete the host from api/v1/hosts/<hostname>
>>>>> as well.
>>>>> ------------------------------
>>>>> *From:* Kartik Mathur <kar...@bluedata.com>
>>>>> *Sent:* Thursday, September 15, 2016 11:43 PM
>>>>> *To:* user@ambari.apache.org
>>>>> *Subject:* Unable to delete host from ambari using API (HDP2.4 with
>>>>> Ambari 2.2)
>>>>>
>>>>>
>>>>> I have 7 nodes HDP 2.4 with ambari 2.2 cluster -
>>>>>
>>>>>
>>>>> This is the closest document that I am able to find for deleting host
>>>>> using APIs.
>>>>>
>>>>> My host has these services running -
>>>>>
>>>>> 1) Zookeeper_server
>>>>>
>>>>> 2)DATANODE
>>>>>
>>>>> 3)NODEMANAGER
>>>>>
>>>>> 4)RegionServer
>>>>>
>>>>> 5)FLUME
>>>>>
>>>>> 1) I first decommision data and node manager using this -
>>>>>
>>>>> curl -u admin:admin -i -H 'X-Requested-By: ambari' -X POST -d '{
>>>>> "RequestInfo":{ "context":"Decommission DataNode",
>>>>> "command":"DECOMMISSION", "parameters":{ "slave_type":"DATANODE",
>>>>> "excluded_hosts":"bluedata-69.bdlocal" }, "operation_level":{
>>>>> "level":"HOST_COMPONENT", "cluster_name":"h2" } },
>>>>> "Requests/resource_filters":[ { "service_name":"HDFS",
>>>>> "component_name":"NAMENODE" } ] }' http://10.35.229.2:8080/api
>>>>> /v1/clusters/h2/requests
>>>>>
>>>>> 2)Next I stop and delete all components (listed above, which I am
>>>>> successfully able to)
>>>>>
>>>>> 3) Strange thing happens after this -
>>>>>
>>>>> There is a random behavior on ambari UI where we sometimes see this
>>>>> host and sometimes not , besides now if I try to delete host from ambari
>>>>> API it gives me server error and tells host does not exist!! , however I
>>>>> can still go on UI and delete the host.
>>>>>
>>>>> All I wanted to do is decommision data node and delete a host in a
>>>>> clean way using ambari APIs , please assist
>>>>> Thanks
>>>>> Kartik
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>
