Re: [Architecture] [IoT] Load Distribution for Device + Server Communication.

[Harshan Liyanage]

Hi all,

Currently the bottleneck happens at getPendingOperations API. That method
takes a while to be executed as it involves both fetching the next
operations & updating the status of completed operations. The complete
getPendingOperations flow has more than 5 database calls (as I remember)
including a DB update. Hence most of the execution time would be spending
on DB queries. A proper distribution of load might help us to safeguard
server from sudden bursts for some extent. But IMO I think we should be
also focusing on making getPendingOperations flow more efficient so that
each device spends a lesser time in executing getPendingOperations API. For
that we'll have to use queues for both ways (fetching / updating) so that
we could fetch / update data from in-memory queue rather than from DB which
is more efficient. Hence +1 for using Queues.

Thanks,


Harshan Liyanage
EMM/IoT TG
Mobile: *+94765672894*
Email: hars...@wso2.com
Blog : http://harshanliyanage.blogspot.com/
*WSO2, Inc. :** wso2.com *
lean.enterprise.middleware.

On Thu, Mar 2, 2017 at 12:10 PM, Kamidu Punchihewa 
wrote:

> Hi All,
>
> Queuing the in and out communications seems to be the best option other
> than handling the loading using times outs. But we need to come up with the
> priority order when its come to the communication which is time sensitive
> temperature monitoring in fire alarm etc.
>
> +1 for utilizing queues for communications going both ways.
>
> Thanks and Best Regards,
>
> Kamidu Sachith Punchihewa
> *Software Engineer*
> WSO2, Inc.
> lean . enterprise . middleware
> Mobile : +94 (0) 770566749 <%2B94%20%280%29%20773%20451194>
>
> Please Note that I have dyslexia and it may results in few misspelled
> words in the content.
>
> Disclaimer: This communication may contain privileged or other
> confidential information and is intended exclusively for the addressee/s.
> If you are not the intended recipient/s, or believe that you may have
> received this communication in error, please reply to the sender indicating
> that fact and delete the copy you received and in addition, you should not
> print, copy, retransmit, disseminate, or otherwise use the information
> contained in this communication. Internet communications cannot be
> guaranteed to be timely, secure, error or virus-free. The sender does not
> accept liability for any errors or omissions.
>
> On Tue, Feb 28, 2017 at 7:41 AM, Charitha Goonetilleke  > wrote:
>
>> Hi All,
>>
>> I think it is better to have a solution which could be implemented in
>> server side to distribute operations among devices. Because as we know
>> device agents in IoTS devices are different to each other. In according to
>> our architecture, we are letting anyone to write their own device types as
>> well as the agents. Also we don't have our own agent for windows devices
>> but using default windows workspace configurations to provide MDM. So
>> implementing agent side solution would be more complex for some device
>> types and will not move forward with all device types.
>>
>> Anyway as according what to suggested by Geeth & Menaka, we could use
>> batch processing in server side to deliver operation to device batches
>> rather than sending it to all devices. So we can use operation queue to
>> deliver operations periodically to device batches. However we need to
>> figure out few more things in there.
>>
>>
>>1. How we figure out optimum number of devices we should call in a
>>batch?
>>2. What is the time span of delivering operation to all target
>>devices?
>>3. What we do if someone want to send operation immediately?
>>4. Can we assure guaranteed delivery and can avoid repeated
>>operations?
>>
>> For 1 & 2, we could use configuration to provide capability to set size
>> of a batch and time span. However for 3, we need to have additional option
>> to send operations immediately, probably using a priority queue. Also when
>> it comes to 4, we need to keep records about the operations we have
>> delivered and would be enough to add one additional operation status like
>> "Queued". So we can change operation status to pending, when operation is
>> delivered.
>>
>> WDYT?
>>
>> Thanks & Regards,
>> /charithag
>>
>> On Tue, Feb 28, 2017 at 12:39 AM, Menaka Jayawardena 
>> wrote:
>>
>>> Hi,
>>>
>>> IMO we can send a timeout with the operation which, once the operation/
>>> notification is received by the device, it waits for the given timeout and
>>> respond back to the server to perform the operation. All operations are
>>> sent to devices at ones and the timeout should be different to each device.
>>> So that each device gets its own time to contact the server and complete
>>> the operation.
>>>
>>> Also, this can be used for batch processing as well.
>>> As an example, if there are 1000 devices, for first 100 devices, the
>>> timeout will be 0s, for next 100, 10s and so on. Then the 

Re: [Architecture] [IoT] Load Distribution for Device + Server Communication.

[Ruwan Yatawara]

Hi All,

I think going forward, we need a queue to be present, for communications
going both ways. As Charitha highlighted, when multiple devices are
targeted, we need to be able to define a dynamic distribution strategy that
will gradually ramp up the rate of distribution, depending on success rate
and system throughput. Ofcourse in order to do this, we will need to have
guaranteed delivery.

I think maybe when we go forth, we may need to think about a combination of
MQTT and Kafka combination to cater all our delivery + connectivity needs.

Thanks and Regards,

Ruwan Yatawara

Associate Technical Lead,
WSO2 Inc.

email : ruw...@wso2.com
mobile : +94 77 9110413
blog :  http://ruwansrants.blogspot.com/

https://500px.com/ruwan_ace
https://medium.com/@ruwanyatawara

www: :http://wso2.com


On Thu, Mar 2, 2017 at 10:14 AM, Chathura Ekanayake 
wrote:

> Can we get all device requests through a message broker and let the broker
> handle spikes? Then the broker has to be scalable.. however, it anyway has
> to be scalable as devices can publish monitoring events to the broker
> (which can occur frequently by a large number of devices).
>
> Regards,
> Chathura
>
> On Wed, Mar 1, 2017 at 11:01 AM, Dilan Udara Ariyaratne 
> wrote:
>
>> Hi Geeth,
>>
>> IMO, all the current information retrieval and monitoring operations
>> could run in parallel from device to device.
>> In that case, cannot we maintain a task pool at server side where we
>> would allocate some defined set of operations to each task and dynamically
>> expand the pool based on demand.
>> It's more of a divide-and-concur approach that I am suggesting here.
>>
>> Thanks,
>> Dilan.
>>
>> *Dilan U. Ariyaratne*
>> Senior Software Engineer
>> WSO2 Inc. 
>> Mobile: +94766405580 <%2B94766405580>
>> lean . enterprise . middleware
>>
>>
>> On Mon, Feb 27, 2017 at 11:09 PM, Geeth Munasinghe 
>> wrote:
>>
>>> Hi all,
>>>
>>> We have encountered an issue with balancing load distribution of device
>>> + server communication. Device and server communication happens through
>>> following ways.
>>>
>>>1. Polling - Device keeps polling the server
>>>2. Push notification - Server sends a notification to device to come
>>>retrieve the operation. (wake up call)
>>>3. Push message - Whole message is pushed to device as notification.
>>>
>>> In 1 and 2 ways, device always comes to server to pick up the operation.
>>> (In MDM scenarios, we use only 1 and 2) We have found few scenarios where
>>> this would lead to a request overload to the server.
>>>
>>>1. Device information retrieval
>>>2. Device application list retrieval
>>>3. Device location retrieval
>>>4. Policy monitoring
>>>5. Installing/Uninstalling application (Role based)
>>>
>>> First four scenarios are handled by background tasks. These tasks are
>>> responsible for adding an operation against each of the device. This
>>> process would notify the device regarding operation or device will come to
>>> server at a pre-configured time to retrieve the operation. Specially with
>>> the notification process, when an operation is added to all the device,
>>> notification is send to all of them. In iOS, we send to the notification to
>>> APNS, for android - GCM/FCM and for Windows - WNS. These notification
>>> services normally deliver the messages to devices quickly. Therefore as
>>> soon as device receives the notification, it starts to contact the server.
>>> If this happens with a server which has 1,000s of devices enrolled, most of
>>> devices start retrieving the operation from the server which leads to a
>>> request a burst. 5th scenario happens when we install an application to a
>>> role which has 1,000s of users. This also starts sending notifications to
>>> all the devices of those users, which would result in a request burst.
>>>
>>> As a solution, we can limit the number of devices notified at given
>>> moment. Only set of devices will receive the notification at first, then
>>> after some time interval second set of devices will receive the
>>> notification and continue until all the devices are notified. This way, we
>>> can make sure only chunk of devices receives the notification, hence reduce
>>> the server load.
>>>
>>> Please give your feedbacks and suggestions.
>>>
>>> Thanks
>>> Geeth
>>>
>>> --
>>>
>>> *G. K. S. Munasinghe*
>>> *Senior Software Engineer,*
>>> *WSO2, Inc. http://wso2.com  *
>>> *lean.enterprise.middleware.*
>>>
>>> email: ge...@wso2.com
>>> phone:(+94) 777911226 <+94%2077%20791%201226>
>>>
>>> 
>>>
>>> ___
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> 

Re: [Architecture] [IoT] Load Distribution for Device + Server Communication.

[Chathura Ekanayake]

Can we get all device requests through a message broker and let the broker
handle spikes? Then the broker has to be scalable.. however, it anyway has
to be scalable as devices can publish monitoring events to the broker
(which can occur frequently by a large number of devices).

Regards,
Chathura

On Wed, Mar 1, 2017 at 11:01 AM, Dilan Udara Ariyaratne 
wrote:

> Hi Geeth,
>
> IMO, all the current information retrieval and monitoring operations could
> run in parallel from device to device.
> In that case, cannot we maintain a task pool at server side where we would
> allocate some defined set of operations to each task and dynamically expand
> the pool based on demand.
> It's more of a divide-and-concur approach that I am suggesting here.
>
> Thanks,
> Dilan.
>
> *Dilan U. Ariyaratne*
> Senior Software Engineer
> WSO2 Inc. 
> Mobile: +94766405580 <%2B94766405580>
> lean . enterprise . middleware
>
>
> On Mon, Feb 27, 2017 at 11:09 PM, Geeth Munasinghe  wrote:
>
>> Hi all,
>>
>> We have encountered an issue with balancing load distribution of device +
>> server communication. Device and server communication happens through
>> following ways.
>>
>>1. Polling - Device keeps polling the server
>>2. Push notification - Server sends a notification to device to come
>>retrieve the operation. (wake up call)
>>3. Push message - Whole message is pushed to device as notification.
>>
>> In 1 and 2 ways, device always comes to server to pick up the operation.
>> (In MDM scenarios, we use only 1 and 2) We have found few scenarios where
>> this would lead to a request overload to the server.
>>
>>1. Device information retrieval
>>2. Device application list retrieval
>>3. Device location retrieval
>>4. Policy monitoring
>>5. Installing/Uninstalling application (Role based)
>>
>> First four scenarios are handled by background tasks. These tasks are
>> responsible for adding an operation against each of the device. This
>> process would notify the device regarding operation or device will come to
>> server at a pre-configured time to retrieve the operation. Specially with
>> the notification process, when an operation is added to all the device,
>> notification is send to all of them. In iOS, we send to the notification to
>> APNS, for android - GCM/FCM and for Windows - WNS. These notification
>> services normally deliver the messages to devices quickly. Therefore as
>> soon as device receives the notification, it starts to contact the server.
>> If this happens with a server which has 1,000s of devices enrolled, most of
>> devices start retrieving the operation from the server which leads to a
>> request a burst. 5th scenario happens when we install an application to a
>> role which has 1,000s of users. This also starts sending notifications to
>> all the devices of those users, which would result in a request burst.
>>
>> As a solution, we can limit the number of devices notified at given
>> moment. Only set of devices will receive the notification at first, then
>> after some time interval second set of devices will receive the
>> notification and continue until all the devices are notified. This way, we
>> can make sure only chunk of devices receives the notification, hence reduce
>> the server load.
>>
>> Please give your feedbacks and suggestions.
>>
>> Thanks
>> Geeth
>>
>> --
>>
>> *G. K. S. Munasinghe*
>> *Senior Software Engineer,*
>> *WSO2, Inc. http://wso2.com  *
>> *lean.enterprise.middleware.*
>>
>> email: ge...@wso2.com
>> phone:(+94) 777911226 <+94%2077%20791%201226>
>>
>> 
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [IoT] Load Distribution for Device + Server Communication.

[Dilan Udara Ariyaratne]

Hi Geeth,

IMO, all the current information retrieval and monitoring operations could
run in parallel from device to device.
In that case, cannot we maintain a task pool at server side where we would
allocate some defined set of operations to each task and dynamically expand
the pool based on demand.
It's more of a divide-and-concur approach that I am suggesting here.

Thanks,
Dilan.

*Dilan U. Ariyaratne*
Senior Software Engineer
WSO2 Inc. 
Mobile: +94766405580 <%2B94766405580>
lean . enterprise . middleware


On Mon, Feb 27, 2017 at 11:09 PM, Geeth Munasinghe  wrote:

> Hi all,
>
> We have encountered an issue with balancing load distribution of device +
> server communication. Device and server communication happens through
> following ways.
>
>1. Polling - Device keeps polling the server
>2. Push notification - Server sends a notification to device to come
>retrieve the operation. (wake up call)
>3. Push message - Whole message is pushed to device as notification.
>
> In 1 and 2 ways, device always comes to server to pick up the operation.
> (In MDM scenarios, we use only 1 and 2) We have found few scenarios where
> this would lead to a request overload to the server.
>
>1. Device information retrieval
>2. Device application list retrieval
>3. Device location retrieval
>4. Policy monitoring
>5. Installing/Uninstalling application (Role based)
>
> First four scenarios are handled by background tasks. These tasks are
> responsible for adding an operation against each of the device. This
> process would notify the device regarding operation or device will come to
> server at a pre-configured time to retrieve the operation. Specially with
> the notification process, when an operation is added to all the device,
> notification is send to all of them. In iOS, we send to the notification to
> APNS, for android - GCM/FCM and for Windows - WNS. These notification
> services normally deliver the messages to devices quickly. Therefore as
> soon as device receives the notification, it starts to contact the server.
> If this happens with a server which has 1,000s of devices enrolled, most of
> devices start retrieving the operation from the server which leads to a
> request a burst. 5th scenario happens when we install an application to a
> role which has 1,000s of users. This also starts sending notifications to
> all the devices of those users, which would result in a request burst.
>
> As a solution, we can limit the number of devices notified at given
> moment. Only set of devices will receive the notification at first, then
> after some time interval second set of devices will receive the
> notification and continue until all the devices are notified. This way, we
> can make sure only chunk of devices receives the notification, hence reduce
> the server load.
>
> Please give your feedbacks and suggestions.
>
> Thanks
> Geeth
>
> --
>
> *G. K. S. Munasinghe*
> *Senior Software Engineer,*
> *WSO2, Inc. http://wso2.com  *
> *lean.enterprise.middleware.*
>
> email: ge...@wso2.com
> phone:(+94) 777911226 <+94%2077%20791%201226>
>
> 
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [IoT] Load Distribution for Device + Server Communication.

[Menaka Jayawardena]

Hi,

IMO we can send a timeout with the operation which, once the operation/
notification is received by the device, it waits for the given timeout and
respond back to the server to perform the operation. All operations are
sent to devices at ones and the timeout should be different to each device.
So that each device gets its own time to contact the server and complete
the operation.

Also, this can be used for batch processing as well.
As an example, if there are 1000 devices, for first 100 devices, the
timeout will be 0s, for next 100, 10s and so on. Then the first batch will
come to the server just after they received the operation and the next
batch will wait for 10s(or any given time) to respond back. This will
reduce the overhead of waiting for a set of devices to complete the
operation.

I think through this we can limit the number of devices which contact the
server at a given time and handle a reasonable amount.

Appreciate any feedback.

Thanks and regards,
Menaka


On Mon, Feb 27, 2017 at 11:09 PM, Geeth Munasinghe  wrote:

> Hi all,
>
> We have encountered an issue with balancing load distribution of device +
> server communication. Device and server communication happens through
> following ways.
>
>1. Polling - Device keeps polling the server
>2. Push notification - Server sends a notification to device to come
>retrieve the operation. (wake up call)
>3. Push message - Whole message is pushed to device as notification.
>
> In 1 and 2 ways, device always comes to server to pick up the operation.
> (In MDM scenarios, we use only 1 and 2) We have found few scenarios where
> this would lead to a request overload to the server.
>
>1. Device information retrieval
>2. Device application list retrieval
>3. Device location retrieval
>4. Policy monitoring
>5. Installing/Uninstalling application (Role based)
>
> First four scenarios are handled by background tasks. These tasks are
> responsible for adding an operation against each of the device. This
> process would notify the device regarding operation or device will come to
> server at a pre-configured time to retrieve the operation. Specially with
> the notification process, when an operation is added to all the device,
> notification is send to all of them. In iOS, we send to the notification to
> APNS, for android - GCM/FCM and for Windows - WNS. These notification
> services normally deliver the messages to devices quickly. Therefore as
> soon as device receives the notification, it starts to contact the server.
> If this happens with a server which has 1,000s of devices enrolled, most of
> devices start retrieving the operation from the server which leads to a
> request a burst. 5th scenario happens when we install an application to a
> role which has 1,000s of users. This also starts sending notifications to
> all the devices of those users, which would result in a request burst.
>
> As a solution, we can limit the number of devices notified at given
> moment. Only set of devices will receive the notification at first, then
> after some time interval second set of devices will receive the
> notification and continue until all the devices are notified. This way, we
> can make sure only chunk of devices receives the notification, hence reduce
> the server load.
>
> Please give your feedbacks and suggestions.
>
> Thanks
> Geeth
>
> --
>
> *G. K. S. Munasinghe*
> *Senior Software Engineer,*
> *WSO2, Inc. http://wso2.com  *
> *lean.enterprise.middleware.*
>
> email: ge...@wso2.com
> phone:(+94) 777911226 <+94%2077%20791%201226>
>
> 
>



-- 
*Menaka Jayawardena*
*Software Engineer - WSO2 Inc*
*Tel : 071 350 5470*
*LinkedIn: https://lk.linkedin.com/in/menakajayawardena
*
*Blog: https://menakamadushanka.wordpress.com/
*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [IoT] Load Distribution for Device + Server Communication.

[Charitha Goonetilleke]

Hi All,

I think it is better to have a solution which could be implemented in
server side to distribute operations among devices. Because as we know
device agents in IoTS devices are different to each other. In according to
our architecture, we are letting anyone to write their own device types as
well as the agents. Also we don't have our own agent for windows devices
but using default windows workspace configurations to provide MDM. So
implementing agent side solution would be more complex for some device
types and will not move forward with all device types.

Anyway as according what to suggested by Geeth & Menaka, we could use batch
processing in server side to deliver operation to device batches rather
than sending it to all devices. So we can use operation queue to deliver
operations periodically to device batches. However we need to figure out
few more things in there.


   1. How we figure out optimum number of devices we should call in a batch?
   2. What is the time span of delivering operation to all target devices?
   3. What we do if someone want to send operation immediately?
   4. Can we assure guaranteed delivery and can avoid repeated operations?

For 1 & 2, we could use configuration to provide capability to set size of
a batch and time span. However for 3, we need to have additional option to
send operations immediately, probably using a priority queue. Also when it
comes to 4, we need to keep records about the operations we have delivered
and would be enough to add one additional operation status like "Queued".
So we can change operation status to pending, when operation is delivered.

WDYT?

Thanks & Regards,
/charithag

On Tue, Feb 28, 2017 at 12:39 AM, Menaka Jayawardena 
wrote:

> Hi,
>
> IMO we can send a timeout with the operation which, once the operation/
> notification is received by the device, it waits for the given timeout and
> respond back to the server to perform the operation. All operations are
> sent to devices at ones and the timeout should be different to each device.
> So that each device gets its own time to contact the server and complete
> the operation.
>
> Also, this can be used for batch processing as well.
> As an example, if there are 1000 devices, for first 100 devices, the
> timeout will be 0s, for next 100, 10s and so on. Then the first batch will
> come to the server just after they received the operation and the next
> batch will wait for 10s(or any given time) to respond back. This will
> reduce the overhead of waiting for a set of devices to complete the
> operation.
>
> I think through this we can limit the number of devices which contact the
> server at a given time and handle a reasonable amount.
>
> Appreciate any feedback.
>
> Thanks and regards,
> Menaka
>
>
> On Mon, Feb 27, 2017 at 11:09 PM, Geeth Munasinghe  wrote:
>
>> Hi all,
>>
>> We have encountered an issue with balancing load distribution of device +
>> server communication. Device and server communication happens through
>> following ways.
>>
>>1. Polling - Device keeps polling the server
>>2. Push notification - Server sends a notification to device to come
>>retrieve the operation. (wake up call)
>>3. Push message - Whole message is pushed to device as notification.
>>
>> In 1 and 2 ways, device always comes to server to pick up the operation.
>> (In MDM scenarios, we use only 1 and 2) We have found few scenarios where
>> this would lead to a request overload to the server.
>>
>>1. Device information retrieval
>>2. Device application list retrieval
>>3. Device location retrieval
>>4. Policy monitoring
>>5. Installing/Uninstalling application (Role based)
>>
>> First four scenarios are handled by background tasks. These tasks are
>> responsible for adding an operation against each of the device. This
>> process would notify the device regarding operation or device will come to
>> server at a pre-configured time to retrieve the operation. Specially with
>> the notification process, when an operation is added to all the device,
>> notification is send to all of them. In iOS, we send to the notification to
>> APNS, for android - GCM/FCM and for Windows - WNS. These notification
>> services normally deliver the messages to devices quickly. Therefore as
>> soon as device receives the notification, it starts to contact the server.
>> If this happens with a server which has 1,000s of devices enrolled, most of
>> devices start retrieving the operation from the server which leads to a
>> request a burst. 5th scenario happens when we install an application to a
>> role which has 1,000s of users. This also starts sending notifications to
>> all the devices of those users, which would result in a request burst.
>>
>> As a solution, we can limit the number of devices notified at given
>> moment. Only set of devices will receive the notification at first, then
>> after some time interval second set of devices will receive the
>>