Hi Srinath, Thanks a bunch for the feedback. We'll proceed as proposed.
Cheers, Prabath On Mon, Mar 28, 2016 at 3:11 PM, Srinath Perera <srin...@wso2.com> wrote: > Hi Prabath, > > Agree on your points. I am +1 to have analytics separate. > > --Srinath > > On Thu, Mar 24, 2016 at 5:06 PM, Prabath Abeysekera <praba...@wso2.com> > wrote: > >> Hi Srinath, >> >> On Thu, Mar 24, 2016 at 1:48 PM, Srinath Perera <srin...@wso2.com> wrote: >> >>> Hi All, >>> >>> Prabath, if analytics is not in the device type, where should it go? >>> >> >> IMO, we can have it as a separate project structure with a separate >> packaging as part of the underlying tooling platform. Just as how things >> are done when composing a typical enterprise integration scenario involving >> ESB configs, DSS configs, etc, one may create a multi-module maven project >> with device management plug-in skeleton as one module and analytics as >> another module. When built, there will be two resultant packagings one to >> be deployed into device management nodes and the other to DAS. If the same >> developer is expected to take care of developing the device management >> plugin implementation and the analytics story, then we can may be define a >> composite maven project artifact, when clicked, will create the >> aforementioned structure. >> >> WDYT? >> >> >>> >>> I do not agree with your observation that functional and >>> non-functional parts should be separate. It helps to have it separate if >>> non-functional parts can be reused with other devices. However, that is not >>> the case here. >>> >> >> IMO, it's not only reusability that defines it in this particular >> context. The fact that analytics can change and evolve independently from >> the device management plug-in implementation should also be considered as a >> valid requirement. This somewhat aligns with the first question you've >> raised below, I believe. >> >> The reason is, organizational analytics related requirements change over >> time depending on various business strategies that they might adopt, but >> the plug-in implementation might not. Also, the degree of analytics >> required too would change from one organization to another even if they >> deploy the same IoTS solution equipped with the same plug-in set. If we >> ship analytics bundled into the plug-in, we present things to the end-user >> as something tied into a plug-in imeplementation, which doesn't appear to >> be correct. So, we should have a mechanism that is not coupled to a plug-in >> implementation, yet functions in tandem with it, to cater that sort of >> requirements. >> >> For instance, let's imagine there exists Organizations Org1 and Org2. >> They both are interested in using IoTS, which has the capability to handle >> device DX, produced by a manufacturer ManX. There's a good chance that the >> analytics needed for Org1 and Org2 might be different? So, a single >> solution that is shipped with built-in analytics(bundled with the plug-in) >> might not be suitable for both organizations. Instead, they would want to >> customize the KPIs to suit their business requirements. One might think >> this is hypothetical, but I thought there's a good enough probability that >> this might be the case in a real deployment. >> >> >>> >>> I think, as sumedha said, we should ship what we have and change based >>> on how it is used. >>> >> >> Anyway, if everyone thinks, we should wait for industry feedback to make >> a decision on this, yeah of course, we'll do it that way. >> >> >>> >>> If we choose to ship analytics bundled, we need to answer two questions. >>> >>> >>> 1. Would plugin developer will do all analytics or do we have use >>> cases where end users will also create their own analytics? If they want >>> to >>> build their own analytics, we need think through how it work. >>> >>> To my understanding, both scenarios are possible. >> >> >>> >>> 1. There will be analytics that use data from multiple device types? >>> How would such cases handled? >>> >>> Well, in the case of mobile devices, this certainly is a valid scenario >> as people usually prefer to have an aggregated view of how devices as a >> whole behave within an organization adhering compliance rules, etc. Even in >> other typical IoT contexts, when it comes implementing monitoring/analytics >> upon groups that have multiple device types in them i.e. a smart home >> solution, you essentially need to have something similar, IMO. >> >> IoT experts, please do comment in and correct me if I'm wrong. >> >> Cheers, >> Prabath >> >> --Srinath >>> >>> >>> >>> On Thu, Mar 24, 2016 at 12:41 PM, Prabath Abeysekera <praba...@wso2.com> >>> wrote: >>> >>>> Hi Sumedha, >>>> >>>> IMO, the device management plug-in layer does show what features can be >>>> controlled, what device info to be captured, etc at a somewhat satisfactory >>>> level already. If this is not sufficient, we need to think further and see >>>> what needs to be added at a functional level. >>>> >>>> Also, I agree that there's no "huge" problem in having things bundled >>>> together. But, IMO, I feel that it's more logical to separate out >>>> functional bits and non-functional bits, that will not only make a clear >>>> demarcation between the two aspects, but also that will make the >>>> implementation as well as the positioning of the implementation more >>>> cleaner and cleanly presentable. >>>> >>>> Referring to the example you've brought in, which is around bringing up >>>> an analytics story upon the Greenhouse story, that's exactly what I tried >>>> to highlight too. Users would more often look at integrated analytics, >>>> particularly when it comes to real world systems, so just "plug-in >>>> contained" analytics will not be sufficient in majority of the cases, IMO. >>>> So, we need to think beyond that level and propose and approach that will >>>> be a better fit for majority of the real world use-cases. >>>> >>>> I totally agree on the fact that we just started getting things >>>> rolling. In fact, no one as yet seems to entirely know what exactly is IoT >>>> or how things will evolve in the future. As you have said, we can surely >>>> wait and see how the industry responds to things and then make things >>>> evolve at our end. But do we really have to rely on the industry feedback >>>> all the time to do what appears to be logical? :) >>>> >>>> Also, the intention of this mail is not about making changes to the >>>> current approach right at the last min causing possible delays in proposed >>>> release timelines. Rather this is something that I thought would be logical >>>> to be added to the product maybe in a future release. >>>> >>>> Cheers, >>>> Prabath >>>> >>>> On Thu, Mar 24, 2016 at 11:22 AM, Sumedha Rubasinghe <sume...@wso2.com> >>>> wrote: >>>> >>>>> My take on this is slightly different. >>>>> >>>>> First @ a conceptual level when we say IoT Server can be extended by a >>>>> plugin, we should have a clear definition of that plugin to have X, Y, Z >>>>> capabilities. So I feel having analytics @ device type definition level >>>>> (conceptually) is not a problem. >>>>> >>>>> Going by that thinking, I don't see a huge problem in **first version >>>>> of Maven archetype for creating device type plugin** having template >>>>> analytics scripts generated. >>>>> >>>>> (For anyone interested instructions are available @ >>>>> https://docs.wso2.com/display/IoTS100/Creating+a+New+Device+Plugin+via+the+Maven+Archetype >>>>> More detailed documentation available @ >>>>> https://docs.google.com/document/d/1ms8xJInbQlSt213Rh_cju7Amt27dLgf8twYWVDQPZ3E/edit >>>>> ) >>>>> >>>>> >>>>> On the other hand, I feel the level of analytics a device type plugin >>>>> can have is very limited compared to a real life view of a devices >>>>> deployment. >>>>> >>>>> Let me take an example. >>>>> >>>>> A Greenhouse has an air circulation system, humidity control & water >>>>> sprinklers. As per our IoT Server, these are instances of different device >>>>> types. Having per device analytics would not full fill the need of >>>>> Greenhouse operator. He should rather see an integrated analytics view of >>>>> all these subsystems to make decisions and control. >>>>> >>>>> Now where do we support this? Deploying a separate analytics bundle >>>>> can be a solution. But that cannot live on it's own without having a >>>>> reference to device types & respective even streams. So it's not a simple >>>>> solution. >>>>> >>>>> In a nutshell, I am trying to say, >>>>> - We have not even touched the tip of the iceberge yet >>>>> - There is a long way for us to go to get to the real level >>>>> >>>>> - Rather than trying to justify now about the best approach, let's >>>>> ship what we have now and evolve based on industry feedback. Cos, IMO what >>>>> we have is not 100% fail proof or completely bound to fail. >>>>> >>>>> >>>>> On Wed, Mar 23, 2016 at 10:37 PM, Ruwan Yatawara <ruw...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi Prabath, >>>>>> >>>>>> IoTS is basically the representation of WSO2's IoT Platform vision in >>>>>> the flesh (Well, not literally). The idea behind it, is to provide users >>>>>> with the necessary tools and components needed to get started with >>>>>> writing >>>>>> device types that will later go in actual deployments. So someone who is >>>>>> evaluating or test driving our platform per say, would find a more >>>>>> integrated and therefore appealing story by discovering our device >>>>>> management capabilities and analytics capabilities, integrate seamlessly >>>>>> with no additional effort. >>>>>> >>>>>> Analytics maybe a non-functional requirement when we try to view IoTS >>>>>> as just a device management framework. But in reality it is much bigger >>>>>> than that, the whole driving force behind IIOT is the need to improve >>>>>> operational efficiency of businesses through monitoring and analysis of >>>>>> data from an array of sensory networks. Therefore IMHO, by all means >>>>>> analytics becomes a primary functional requirement. We are just getting >>>>>> our >>>>>> beaks wet by making available a bunch of DAS scripts with the plugin. In >>>>>> the future, we may even find it necessary to put data models for machine >>>>>> learning frameworks in there. >>>>>> >>>>>> Furthermore, having all the information pertaining to a particular >>>>>> device type maybe advantageous when it comes to import/export of device >>>>>> types. For an example, lets for a moment assume there is a device cloud >>>>>> by >>>>>> the name of abc.com that runs on top of wso2's IOT framework. I as >>>>>> an independent hardware vendor have written a device type to manage and >>>>>> monitor my device on my laptop using the tooling available for IoTS. So >>>>>> with the current approach, all I have to do is grab that device type >>>>>> (Which >>>>>> should ideally be a deploy-able archive, contrary to our current feature >>>>>> based approach) and upload it to abc.com cloud. In one go, my device >>>>>> type will be deployed. >>>>>> >>>>>> Yes, this does bring to the table, the need to deploy various types >>>>>> of different artifacts in different systems. But I believe we have done >>>>>> something similar when it comes to deployment of imported API artifacts >>>>>> in >>>>>> APIM (Which are deploy-able zip archives, containing sequences, images, >>>>>> DB >>>>>> entries and what-not) >>>>>> >>>>>> Due to reasons mentioned above. I am of the opinion that we should >>>>>> bundle analytics scripts with the plugin. >>>>>> >>>>>> Thanks and Regards, >>>>>> >>>>>> Ruwan Yatawara >>>>>> >>>>>> Senior Software Engineer, >>>>>> WSO2 Inc. >>>>>> >>>>>> email : ruw...@wso2.com >>>>>> mobile : +94 77 9110413 >>>>>> blog : http://ruwansrants.blogspot.com/ >>>>>> www: :http://wso2.com >>>>>> >>>>>> >>>>>> On Wed, Mar 23, 2016 at 5:54 PM, Prabath Abeysekera < >>>>>> praba...@wso2.com> wrote: >>>>>> >>>>>>> Hi Everyone, >>>>>>> >>>>>>> As most of you know, IoTS is a unified platform that allows users to >>>>>>> come up with "plugins" and seamlessly add them to the framework to be >>>>>>> able >>>>>>> to manage different types of devices. For instance, if an organization >>>>>>> named "Org-X" manufactures a device type "Dev-X", they can simply adopt >>>>>>> IoTS and extend its plug-in model to come up with a plug-in to support >>>>>>> end-to-end device management and more upon "Dev-X" devices. To make >>>>>>> plug-in >>>>>>> generation process easier for the developers who are involved in the >>>>>>> same, >>>>>>> a maven archetype is used. The aforesaid archetype is simply used to >>>>>>> create >>>>>>> a skeleton of the plug-in, which can then be enriched with different >>>>>>> communication protocols, etc depending on the expectations of different >>>>>>> device manufacturers, etc. >>>>>>> >>>>>>> Currently, the maven archetype explained above has the following >>>>>>> structure. >>>>>>> >>>>>>> -- component >>>>>>> --*analytics* >>>>>>> --controller >>>>>>> --manager >>>>>>> --plugin >>>>>>> --ui >>>>>>> >>>>>>> Once the above skeleton is generated, then the developers customize >>>>>>> the same to come up with the desired plug-in implementations and package >>>>>>> things up to be able to push it to the CDM-F. >>>>>>> >>>>>>> Right now the model being used is that, all the aforementioned >>>>>>> resources such as the plugin implementation, analytics scripts, etc are >>>>>>> packed up as a single archive. IMO, having analytics scripts bundled up >>>>>>> with the rest of the bits is not a good approach depending on a few >>>>>>> reasons. >>>>>>> >>>>>>> 1. When it comes real production deployments, these analytics >>>>>>> scripts will be deployed in a separate DAS cluster as opposed to how >>>>>>> things >>>>>>> are when the same is done within the scope of a single server. So, if >>>>>>> the >>>>>>> two components are running separately, there's no point packaging >>>>>>> everything up as a single archive as we'd have to break things down to >>>>>>> two >>>>>>> separate packagings (a) to be deployed into IoTS (b) to be deployed into >>>>>>> DAS. >>>>>>> >>>>>>> 2. Analytics, I believe, primarily is a non-functional requirement. >>>>>>> Therefore, bundling non-functional bits with functional bits is not a >>>>>>> good >>>>>>> practice. >>>>>>> >>>>>>> 3. If we ship analytics scripts with a particular plug-in, the >>>>>>> natural expectation is that it would only cover specific bits bound to >>>>>>> the >>>>>>> scope of particular plug-in. However, in a practical environment, what >>>>>>> majority of the folks are looking at are much more advanced and >>>>>>> composite >>>>>>> KPIs and related analytics to be able to help organizational strategies, >>>>>>> etc. So it is highly likely that people might generally be interested in >>>>>>> brining in analytics as a separate step relieving the need to pack the >>>>>>> scripts, etc with the functional bits. i.e. plug-in implementation. >>>>>>> >>>>>>> >>>>>>> Instead what I propose is to have a separate "project type" or >>>>>>> something similar as part of the underlying tooling platform, which can >>>>>>> effectively be used to compose all analytics related scripts, etc. We >>>>>>> can >>>>>>> then package it up separately and deploy into the relevant DAS clusters >>>>>>> to >>>>>>> get the analytics up and running. >>>>>>> >>>>>>> >>>>>>> Some of the stuff I've brought up might be subjective, but I am of >>>>>>> the opinion that we need to remove analytics related scripts from the >>>>>>> plug-in skeleton. >>>>>>> >>>>>>> WDYT? >>>>>>> >>>>>>> >>>>>>> Cheers, >>>>>>> Prabath >>>>>>> -- >>>>>>> Prabath Abeysekara >>>>>>> Technical Lead >>>>>>> WSO2 Inc. >>>>>>> Email: praba...@wso2.com >>>>>>> Mobile: +94774171471 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> Architecture@wso2.org >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> Architecture@wso2.org >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> /sumedha >>>>> m: +94 773017743 >>>>> b : bit.ly/sumedha >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> Architecture@wso2.org >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Prabath Abeysekara >>>> Technical Lead >>>> WSO2 Inc. >>>> Email: praba...@wso2.com >>>> Mobile: +94774171471 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> ============================ >>> Srinath Perera, Ph.D. >>> http://people.apache.org/~hemapani/ >>> http://srinathsview.blogspot.com/ >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Prabath Abeysekara >> Technical Lead >> WSO2 Inc. >> Email: praba...@wso2.com >> Mobile: +94774171471 >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > ============================ > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://home.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Prabath Abeysekara Technical Lead WSO2 Inc. Email: praba...@wso2.com Mobile: +94774171471
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
