Re: [Architecture] Facility to delete an asset through the UI in ES Publisher

2014-08-13 Thread Sameera Jayaratna
Hi Udara, I think that will depend on how the lifecycle is defined and what states are declared as 'delete states'. i.e. Does the lifecycle contain a possible transition as 'Published' ---> 'Deleted'. If such a state transition is available we will face the issue you mentioned regarding subscript

Re: [Architecture] Carbonising Data Node

2014-08-13 Thread Poshitha Dabare
Hi, I am trying to spin up a carbonised datanode with authentication enabled in 'core-site.xml' file. When trying to spin the datanode by calling DataNode class in package org.apache.hadoop.hdfs.server.datanode it is unable. It we inspect the ''hadoop' CLI sh file it trys to spin up datanode usi

Re: [Architecture] [App Factory] Improvements to application wall

2014-08-13 Thread Gayan Dhanushka
Hi Dimuthu, Following are a set of actions that we want to give "started-inprogress-ended" messages and places where we would put the spinning wheel and messages. 1) Application Creation - User home near the application thumbnail (This is already implemented) 2) Application Upload - Application c

Re: [Architecture] [POC] Performance evaluation of Hive vs Shark

2014-08-13 Thread Anjana Fernando
Hi Niranda, Excellent analysis of Hive vs Shark! .. This gives a lot of insight into how both operates in different scenarios. As the next step, we will need to run this in an actual cluster of computers. Since you've used a subset of the dataset of 2014 DEBS challenge, we should use the full data

Re: [Architecture] [POC] Performance evaluation of Hive vs Shark

2014-08-13 Thread Sumedha Rubasinghe
On Aug 13, 2014 3:18 PM, "Anjana Fernando" wrote: > > Hi Niranda, > > Excellent analysis of Hive vs Shark! .. This gives a lot of insight into how both operates in different scenarios. As the next step, we will need to run this in an actual cluster of computers. Since you've used a subset of the d

Re: [Architecture] [POC] Performance evaluation of Hive vs Shark

2014-08-13 Thread Anjana Fernando
On Wed, Aug 13, 2014 at 3:51 PM, Sumedha Rubasinghe wrote: > > After these are done, we should also do a trial run of our own APIM Hive > scripts, migrated to Shark. > > Do we need to migrate?I thought existing Hive scripts can run as it is. > First of all we need to create a large data set of AP

Re: [Architecture] [App Factory] Improvements to application wall

2014-08-13 Thread Dimuthu Leelarathne
Hi Gayan, On Wed, Aug 13, 2014 at 2:28 PM, Gayan Dhanushka wrote: > Hi Dimuthu, > > Following are a set of actions that we want to give > "started-inprogress-ended" messages and places where we would put the > spinning wheel and messages. > > 1) Application Creation - User home near the applica

Re: [Architecture] [App Factory] Data Model for AppFactory

2014-08-13 Thread Dimuthu Leelarathne
On 14 Aug 2014 09:00, "Manisha Gayathri" wrote: > > > > > On Thu, Aug 14, 2014 at 3:47 AM, Dimuthu Leelarathne wrote: >> >> Hi Ajanthan and all, >> >> I see some meta data in these tables. The following are not runtime data, but rather meta data >> >> 1 - app_owner in AF_APP > > Also REPO_ACCESSA

Re: [Architecture] [App Factory] Data Model for AppFactory

2014-08-13 Thread Gayan Dhanushka
Hi Ajanthan, What about the forrk repository related data. We have builds for forked repos and we used to keep them in the rxts? Thanks GayanD On Thu, Aug 14, 2014 at 9:13 AM, Dimuthu Leelarathne wrote: > > On 14 Aug 2014 09:00, "Manisha Gayathri" wrote: > > > > > > > > > > On Thu, Aug 14, 2

Re: [Architecture] [App Factory] Improvements to application wall

2014-08-13 Thread Gayan Dhanushka
Hi Dimuthu, I tried a sample implementation to show the implementation on the application wall. This was done by looking at the content of the message. It was successful. But this may not be ideal since for different kind of start-inprogress-ended messages we may need to have different logic. Mea

Re: [Architecture] [POC] Performance evaluation of Hive vs Shark

2014-08-13 Thread Srinath Perera
This look great. We need to test Spark with multiple nodes? Did we do that. Please create few VMs in performance could (talk to Lakmal) and test with at least 5 nodes. We need to make sure it works OK with distributed setup as well. What does it take to change to spark? Anjana .. how much work is

Re: [Architecture] [POC] Performance evaluation of Hive vs Shark

2014-08-13 Thread Anjana Fernando
Hi Srinath, No, this has not been tested in multiple nodes. I told Niranda here in my last mail, to test a cluster with the same set of hardware we have, that we are using to test our large data set with Hive. As for the effort to make the change, we still have to figure out the MT aspects of Shar

Re: [Architecture] [App Factory] Improvements to application wall

2014-08-13 Thread Dmitry Sotnikov
Gayan and Dimuthu, >From my perspective, one-off UX exceptions are bad, because they make the UI inconsistent. There are two types of messages: A. High-level external notifications of someone taking some action (e.g. "John created a new app") and B. Low-level ongoing processes happening in AF as