Seems there is a commit missing when merging the changes.
Setting the setKubernetesClusterId when creating the Kubenetes service
Now I will add that commit.
Thanks,
Gayan
On Mon, Sep 21, 2015 at 12:35 AM, Akila Ravihansa Perera <raviha...@wso2.com
> wrote:
> Great!
>
> I think I found the problem with K8s services not getting cleaned up. I
> see a NPE at removeKubernetesServices(KubernetesIaas.java:1025) when
> undeploying an application.
>
> [2015-09-21 00:31:16,750] ERROR
> {org.apache.stratos.messaging.message.receiver.cluster.status.ClusterStatusEventMessageDelegator}
> - Failed to retrieve cluster status event message
> java.lang.NullPointerException
> at
> org.apache.stratos.cloud.controller.iaases.kubernetes.KubernetesIaas.removeKubernetesServices(KubernetesIaas.java:1025)
> at
> org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder.handleClusterTerminatingEvent(TopologyBuilder.java:1068)
> at
> org.apache.stratos.cloud.controller.messaging.receiver.cluster.status.ClusterStatusTopicReceiver$5.onEvent(ClusterStatusTopicReceiver.java:86)
> at
> org.apache.stratos.messaging.listener.EventListener.update(EventListener.java:42)
> at java.util.Observable.notifyObservers(Observable.java:159)
> at
> org.apache.stratos.messaging.event.EventObservable.notifyEventListeners(EventObservable.java:51)
> at
> org.apache.stratos.messaging.message.processor.cluster.status.ClusterStatusClusterTerminatingMessageProcessor.process(ClusterStatusClusterTerminatingMessageProcessor.java:48)
> at
> org.apache.stratos.messaging.message.processor.cluster.status.ClusterStatusClusterTerminatedMessageProcessor.process(ClusterStatusClusterTerminatedMessageProcessor.java:55)
> at
> org.apache.stratos.messaging.message.processor.cluster.status.ClusterStatusClusterInactivateMessageProcessor.process(ClusterStatusClusterInactivateMessageProcessor.java:52)
> at
> org.apache.stratos.messaging.message.processor.cluster.status.ClusterStatusClusterActivatedMessageProcessor.process(ClusterStatusClusterActivatedMessageProcessor.java:52)
> at
> org.apache.stratos.messaging.message.processor.cluster.status.ClusterStatusClusterResetMessageProcessor.process(ClusterStatusClusterResetMessageProcessor.java:52)
> at
> org.apache.stratos.messaging.message.processor.MessageProcessorChain.process(MessageProcessorChain.java:61)
> at
> org.apache.stratos.messaging.message.receiver.cluster.status.ClusterStatusEventMessageDelegator.run(ClusterStatusEventMessageDelegator.java:71)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> On Mon, Sep 21, 2015 at 12:30 AM, Imesh Gunaratne <im...@apache.org>
> wrote:
>
>> I have now refined this logic to trim all labels to 62 characters and
>> added following as labels:
>>
>> Pod: Application-ID, Cluster-Instance-ID, Member-ID
>> Service: Application-ID, Cluster-ID, Port-Name
>>
>> Now I'm seeing a different problem. When an application is un-deployed,
>> services do not get deleted. We need to fix this.
>>
>> Thanks
>>
>> On Mon, Sep 21, 2015 at 12:12 AM, Gayan Gunarathne <gay...@wso2.com>
>> wrote:
>>
>>> Hi Akila,
>>>
>>> On Sun, Sep 20, 2015 at 11:30 PM, Akila Ravihansa Perera <
>>> raviha...@wso2.com> wrote:
>>>
>>>> Hi Gayan,
>>>>
>>>> +1 for properly identifying attributes to be used for labels and
>>>> annotations. As I mentioned earlier I tried to use them as labels but ran
>>>> into length limitation issues.
>>>>
>>>> memberId and clusterId cannot be used directly since they are generated
>>>> as follows;
>>>>
>>>> memberId = clusterId + UUID
>>>> clusterId = applicationId + "." + alias + "." + cartridgeType +
>>>> ".domain"
>>>>
>>>> As you can see those can go be beyond 63 characters since we don't have
>>>> a limitation for applicationId length. I'm -1 on imposing a character
>>>> limitation to applicationId only to get away with Kubernetes restriction.
>>>> What if some other provider comes out with a different limitation? It
>>>> should be restricted but only for good reasons concerning to Stratos.
>>>>
>>>
>>> Right. Let's say we are not going to restrict the application id length.
>>> So in that case we won't be able to use the application id as a pod label.
>>> IMO we used application id to identify the application. As it is a
>>> identifier I think it is OK to have a restriction.I think using a
>>> description for the application id is not practical.Also in most of the use
>>> cases identifiers comes up with some length restrictions.If we don't have a
>>> restriction on application id , users can enter description sort of a
>>> application id.
>>>
>>>
>>>>
>>>> I had an offline chat with Imesh and we were discussing about
>>>> implementing a sequence number based approach for generating clusterId and
>>>> memberId without using UUIDs. Any thoughts?
>>>>
>>>
>>> I am also +1 for this thinking. So how we are going to generate the
>>> cluster id? I think we need to generate the meaningful ids.
>>>
>>>
>>>>
>>>> Thanks.
>>>>
>>>>
>>>>
>>>> Thanks,
>>> Gayan
>>>
>>>>
>>>>
>>>> On Sun, Sep 20, 2015 at 10:55 PM, Gayan Gunarathne <gay...@wso2.com>
>>>> wrote:
>>>>
>>>>> As per Kubernetes definition I think we are using labels and
>>>>> annotation in a wrong way.As annotations are arbitrary non-identifying
>>>>> metadata , they might not shown them in their UI/CLI
>>>>>
>>>>> I think first we need to recognize what are the identifiers for the
>>>>> pod.In think we may need to use member id , application id
>>>>>
>>>>> Also what is the ration behind of using the md5Hex of the member id as
>>>>> a identifier. IMO we can directly use the member id. As member id is UUID
>>>>> it won't be more than 63 characters.
>>>>>
>>>>> Thanks,
>>>>> Gayan
>>>>>
>>>>> On Sun, Sep 20, 2015 at 10:13 PM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Akila,
>>>>>>
>>>>>> Right, I get your point on the length limitation, but this is how
>>>>>> Kubernetes has described Labels and Annotations:
>>>>>>
>>>>>> *We'll eventually index and reverse-index labels for efficient
>>>>>> queries and watches, use them to sort and group in UIs and CLIs, etc. We
>>>>>> don't want to pollute labels with non-identifying, especially large
>>>>>> and/or
>>>>>> structured, data. Non-identifying information should be recorded using
>>>>>> annotations.*
>>>>>>
>>>>>> The idea is to use labels for all identifiers.
>>>>>>
>>>>>> http://kubernetes.io/v1.0/docs/user-guide/labels.html
>>>>>>
>>>>>> On Sun, Sep 20, 2015 at 10:05 PM, Akila Ravihansa Perera <
>>>>>> raviha...@wso2.com> wrote:
>>>>>>
>>>>>>> Hi Imesh,
>>>>>>>
>>>>>>> Yes, I tried adding them as labels but ran into a problem since
>>>>>>> label length cannot be greater than 63 characters. I'm not sure why we
>>>>>>> don't see annotations in the UI for pods created. But I can see the
>>>>>>> annotations added for services. Might have to raise this in K8s mailing
>>>>>>> list.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> On Sun, Sep 20, 2015 at 10:00 PM, Imesh Gunaratne <im...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Akila,
>>>>>>>>
>>>>>>>> We do not query pod labels, its handled by the Kubernetes service
>>>>>>>> selector, at the moment this is called "name". I have now fixed this
>>>>>>>> issue
>>>>>>>> locally, will push in few minutes.
>>>>>>>>
>>>>>>>> In addition I think it would be better to use labels instead of
>>>>>>>> annotations because they are not visible in the UI:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Sun, Sep 20, 2015 at 9:52 PM, Akila Ravihansa Perera <
>>>>>>>> raviha...@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Imesh,
>>>>>>>>>
>>>>>>>>> Where do we query those pod labels?
>>>>>>>>>
>>>>>>>>> I'm in the process of moving label names to constants. I can apply
>>>>>>>>> the same in all the places. I think better to change that label to
>>>>>>>>> md5Hex-memberId to avoid any confusion when troubleshooting issues.
>>>>>>>>> One
>>>>>>>>> might mistakenly take that as the actual memberId when it is not the
>>>>>>>>> case.
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> On Sun, Sep 20, 2015 at 9:40 PM, Imesh Gunaratne <im...@apache.org
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> I'm seeing a problem, we have removed the "name" label in the pod
>>>>>>>>>> and added a new label called "md5Hex-memberId". Due to this load
>>>>>>>>>> balancing
>>>>>>>>>> would not work:
>>>>>>>>>>
>>>>>>>>>> Map<String, String> podLabels = new HashMap<>();
>>>>>>>>>> podLabels.put("md5Hex-memberId",
>>>>>>>>>> DigestUtils.md5Hex(memberContext.getMemberId()));
>>>>>>>>>>
>>>>>>>>>> Map<String, String> podAnnotations = new HashMap<>();
>>>>>>>>>> podAnnotations.put("memberId", memberContext.getMemberId());
>>>>>>>>>> podAnnotations.put("cartridgeType",
>>>>>>>>>> memberContext.getCartridgeType());
>>>>>>>>>> podAnnotations.put("applicationId",
>>>>>>>>>> memberContext.getApplicationId());
>>>>>>>>>> podAnnotations.put("clusterId", memberContext.getClusterId());
>>>>>>>>>> podAnnotations.put("clusterInstanceId",
>>>>>>>>>> memberContext.getClusterInstanceId());
>>>>>>>>>>
>>>>>>>>>> kubernetesApi.createPod(podId, podName, podLabels, podAnnotations,
>>>>>>>>>> dockerImage, cpu, memory, ports,
>>>>>>>>>> environmentVariables);
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Sep 20, 2015 at 2:51 PM, Akila Ravihansa Perera <
>>>>>>>>>> raviha...@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I've pushed release artifacts to staging repo at [1]. Please
>>>>>>>>>>> take few mins to verify the release. Also I've pushed
>>>>>>>>>>> cartridge-base, php
>>>>>>>>>>> and tomcat Docker images with 4.1.3 artifacts to DockerHub.
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachestratos-1057/
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Akila Ravihansa Perera
>>>>>>>>>>> WSO2 Inc.; http://wso2.com/
>>>>>>>>>>>
>>>>>>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>
>>>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Akila Ravihansa Perera
>>>>>>>>> WSO2 Inc.; http://wso2.com/
>>>>>>>>>
>>>>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Akila Ravihansa Perera
>>>>>>> WSO2 Inc.; http://wso2.com/
>>>>>>>
>>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Senior Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Gayan Gunarathne
>>>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>>>> Committer & PMC Member, Apache Stratos
>>>>> email : gay...@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Akila Ravihansa Perera
>>>> WSO2 Inc.; http://wso2.com/
>>>>
>>>> Blog: http://ravihansa3000.blogspot.com
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Gayan Gunarathne
>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>> Committer & PMC Member, Apache Stratos
>>> email : gay...@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
--
Gayan Gunarathne
Technical Lead, WSO2 Inc. (http://wso2.com)
Committer & PMC Member, Apache Stratos
email : gay...@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
