[Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Kasun Indrasiri]

Hi Venkat,

As per the benchmark results, the results that you get for LB seems to be
constrained by the performance of the backend. So, I would like suggest you
to use a backend that performs better.
@Ranwaka : Can you share the backend service that you used for benchmarking
our HTTP transport?

Lets do another iteration with above changes.

As the benchmarking too, please give a try on 'wrk'. We found this to be a
much more light weight and well suited for http load testing.
Sample :

wrk -t2 -d5s -c100 -s sample.lua  "
http://127.0.0.1:9000/services/SimpleStockQuoteService";

sample.lua
---
wrk.method = "POST"
wrk.body   = "payload..."
wrk.headers["Content-Type"] = "text/xml"
wrk.headers["SOAPAction"] = "urn:getQuote"

[1] https://github.com/wg/wrk

On Sun, Jul 24, 2016 at 9:24 PM, Venkat Raman  wrote:

> Hi Isuru,
>
> Please find 9th week's progress.
>
> 1) Customizing MSF4J's performance benchmark scripts for testing our LB.
>
> 2) Did preliminary TPS and Latency bench-marking between Nginx, GW-LB and
> Direct MSF4J BE.  I'm attaching it again for your reference.
>
> 3) Did few improvements on Callback pool's implementation.
>
> Kasun has  scheduled meeting today 10:30 to 11:30 PM IST.  Can we have our
> review tomorrow .?
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Sun, Jul 24, 2016 at 4:25 PM, Venkat Raman 
> wrote:
>
>> There Isuru..?
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Sat, Jul 23, 2016 at 10:17 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Kasun,
>>>
>>> PFA
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Sat, Jul 23, 2016 at 10:08 PM, Venkat Raman 
>>> wrote:
>>>
 Hi Kasun,

 It would be great if meeting is after 9 PM IST.




 *Thanks,*
 *Venkat.*

 On Sat, Jul 23, 2016 at 10:00 PM, Kasun Indrasiri 
 wrote:

> Hi Venkat,
>
> Scheduled a meeting on Monday.
>
> On Sat, Jul 23, 2016 at 5:06 AM, Venkat Raman 
> wrote:
>
>> I missed to mention about test environment.
>>
>> OS Ubuntu 16.04 64 bit - VM
>> 4 GB RAM, 4 Cores
>>
>> Host Machine is Macbook Pro 16GB RAM
>>
>> Also kindly find the attached ApacheBench results too.
>>
>>
>>
>>
>>
>> *Venkat.*
>>
>> On Sat, Jul 23, 2016 at 5:06 PM, Venkat Raman 
>> wrote:
>>
>>> Hi All,
>>>
>>> I've customized perf-benchmark scripts for our LB.  Kindly find it here.
>>>
>>> 
>>>
>>> You can also find Nginx config that I used for this bench-marking
>>> here.
>>> 
>>>
>>> I've also attached preliminary test results.  Kindly have a look at
>>> it.
>>>
>>> @IsuruR - We should have a discussion/code review ASAP.  You can see
>>> that test results are good and promising, but it can be improvised.  
>>> Kindly
>>> note that we have only few weeks left.  I understand that you are very
>>> busy, but kindly do allocate little time for this also Isuru.
>>>
>>> Once we do our code review, with Samiyuru's guidance I'll do memory
>>> test also. If you are free we can do it today or tomorrow.
>>>
>>> @Kasun - If you are free we can have demo today / tomorrow.
>>>
>>>
>>> Will be looking forward to hear from you.
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Thu, Jul 21, 2016 at 11:13 AM, Venkat Raman >> > wrote:
>>>
 Hi IsuruP and Samiyuru,

 Good morning.  Kindly find the project here.
 

 I also gave a demo to IsuruR and he is happy with the features
 implemented so far.

 Now we are in performance testing phase.  We need to do performance
 bench-marking similar to that of MSF4J.  For the project scope we have
 planned to compare with Nginx.

 Yesterday I tested Nginx and our LB with 1,000,000 requests and
 10,000 concurrency.  Performance was close.  Test was done with apache
 benchmark on Ubuntu 64 bit VM with 8 GB RAM and 3 CPU cores. I used 
 VMware
 Fusion as Hypervisor. Host machine is MacBook pro 16GB.

 I also tried MSF4J bench-marking scripts.  We need to do a similar
 comparison between Nginx and our LB.  It would be great if you could 
 help
 and guide me through it.

 Looking forward to hear from you.



 *Thanks,*
 *Venkat.*

 On Thu, Jul 21, 2016 at 9:43 AM, Venkat Raman >>> > wrote:

> Hi Isuru,
>
> Good morning.  Yesterday I tested Nginx and our LB with 1,000,000
> requests and 10,000 concurrency.  Performan

Re: [Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Isuru Ranawaka]

Hi Venkat
 I have scheduled a call on monday 9 30 p.m

On Sat, Jul 9, 2016 at 9:43 PM, Venkat Raman  wrote:

> Oh okay Kasun.  :)
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Sat, Jul 9, 2016 at 9:36 PM, Kasun Indrasiri  wrote:
>
>> That's fine. I was referring to demo that we were planning. You guys can
>> discuss the HTTPS related things at any convenient time.
>>
>> On Sat, Jul 9, 2016 at 9:03 AM, Venkat Raman 
>> wrote:
>>
>>> Hi Kasun,
>>>
>>> I'll be available on Tue 10 PM IST.   Isuru and I were planning to have
>>> discussion on HTTPS, Transports-statistics and also about proceeding with
>>> load and performance testing tomorrow.  That's why I was asking him about
>>> his availability.
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Sat, Jul 9, 2016 at 9:19 PM, Kasun Indrasiri  wrote:
>>>
 I'm in PDT time zone and might not able to make it on Sunday night. How
 about Tue night 10pm IST?

 On Sat, Jul 9, 2016 at 7:26 AM, Venkat Raman 
 wrote:

> Hi Isuru,
>
> At what time can we have our call tomorrow.. ?
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Fri, Jul 8, 2016 at 8:04 AM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> My bad.  I shared the URL of no SSL again.  It is available under
>> samples\conf\transports
>>
>> Thanks,
>> Venkat.
>> On Jul 8, 2016 7:58 AM, "Isuru Ranawaka"  wrote:
>>
>>> Hi Venkat,
>>>
>>> SSL OFFLOAD  thing I can't see https is enabled.
>>>
>>> On Wed, Jul 6, 2016 at 10:11 PM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 I'm blocked by this issue from yesterday.  We need have a small
 discussion ASAP.  I'll try to explain the scenario more clearly here 
 than I
 did in hangouts.

 For LB these are the 3 different types netty-transports.yml configs
 that we need to support.

 1) For no ssl : no_SSL
 

 2) SSL_OFFLOAD : (i.e) https listener and http sender ssl_offload
 

 3) END_TO_END : (i.e) https listener and https sender end_to_end
 

 Are these configs correct..?

 So far I have been using (1) as we were dealing only with http.  It
 was working fine.  Refer first screenshot for logs.  If you see, netty
 listener is starting in both 8290 and 9090.

 8290 is inboundEndpoint  port specified in router.iflow file and
 9090 is specified in netty-transports.yml

 Now, when I try (2)&(3), since we need to deal with https, netty
 listener is starting only in 9292.  Refer second screenshot for logs.  
 *We
 need a netty listener to be started in 8290 port also.  But it is not
 happening.*

 9292 is specified in netty-transports.yml & 8290 is specified in
 router.iflow

 But, when I change 8290 to 9292 in our router.iflow file, ssl is
 working fine.

 Kindly guide me through it.
 * It is critical as it is blocking my progress.*
 Looking forward to hear from you.


 *Thanks,*
 *Venkat.*

 On Mon, Jul 4, 2016 at 10:33 AM, Venkat Raman >>> > wrote:

> Hi Isuru,
>
> Good morning.  Please find 6th week's progress
>
> 1)  Demo / Discussion
> 2)  Change in Health Checking : Instead of caching and re-using
> carbonMessage's properties, we are establishing a socket connection 
> to see
> whether specified endpoint is alive or not. Find the issue
> 
> here.
> 3)  Active Health Checking.  Previously it was only passive mode
> available.  I've implemented active health checking as you asked 
> during our
> demo.
>
> Also kindly find my new post on mid-term evaluation here
> 
> .
>
> It would be great if you could share some SSL config samples with
> me.
>
> Looking forward for our code review today.
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Sun, Jul 3, 2016 at 4:49 PM, Venkat Raman  > wrote:
>
>> Hi Isuru,
>>
>> As discussed yesterday,

Re: [Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Kasun Indrasiri]

Hi Venkat,

Scheduled a meeting on Monday.

On Sat, Jul 23, 2016 at 5:06 AM, Venkat Raman  wrote:

> I missed to mention about test environment.
>
> OS Ubuntu 16.04 64 bit - VM
> 4 GB RAM, 4 Cores
>
> Host Machine is Macbook Pro 16GB RAM
>
> Also kindly find the attached ApacheBench results too.
>
>
>
>
>
> *Venkat.*
>
> On Sat, Jul 23, 2016 at 5:06 PM, Venkat Raman 
> wrote:
>
>> Hi All,
>>
>> I've customized perf-benchmark scripts for our LB.  Kindly find it here.
>> 
>>
>> You can also find Nginx config that I used for this bench-marking here.
>> 
>>
>> I've also attached preliminary test results.  Kindly have a look at it.
>>
>> @IsuruR - We should have a discussion/code review ASAP.  You can see that
>> test results are good and promising, but it can be improvised.  Kindly note
>> that we have only few weeks left.  I understand that you are very busy, but
>> kindly do allocate little time for this also Isuru.
>>
>> Once we do our code review, with Samiyuru's guidance I'll do memory test
>> also. If you are free we can do it today or tomorrow.
>>
>> @Kasun - If you are free we can have demo today / tomorrow.
>>
>>
>> Will be looking forward to hear from you.
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Thu, Jul 21, 2016 at 11:13 AM, Venkat Raman 
>> wrote:
>>
>>> Hi IsuruP and Samiyuru,
>>>
>>> Good morning.  Kindly find the project here.
>>> 
>>>
>>> I also gave a demo to IsuruR and he is happy with the features
>>> implemented so far.
>>>
>>> Now we are in performance testing phase.  We need to do performance
>>> bench-marking similar to that of MSF4J.  For the project scope we have
>>> planned to compare with Nginx.
>>>
>>> Yesterday I tested Nginx and our LB with 1,000,000 requests and 10,000
>>> concurrency.  Performance was close.  Test was done with apache benchmark
>>> on Ubuntu 64 bit VM with 8 GB RAM and 3 CPU cores. I used VMware Fusion as
>>> Hypervisor. Host machine is MacBook pro 16GB.
>>>
>>> I also tried MSF4J bench-marking scripts.  We need to do a similar
>>> comparison between Nginx and our LB.  It would be great if you could help
>>> and guide me through it.
>>>
>>> Looking forward to hear from you.
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Thu, Jul 21, 2016 at 9:43 AM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Good morning.  Yesterday I tested Nginx and our LB with 1,000,000
 requests and 10,000 concurrency.  Performance was close.

 As you suggested I tried MSF4J's perf benchmark.  I'm working
 customizing that script to test our LB.




 *Thanks,*
 *Venkat.*

 On Mon, Jul 18, 2016 at 9:16 AM, Venkat Raman 
 wrote:

> Hi Isuru,
>
> Good morning.  Please find 8th week's progress.
>
> 1) Removed SSL related config from LB as it is abstracted at
> gateway-framework level.
> 2) Added groups support as we discussed.  Now, LB will support
> multiple groups in a single .iflow file  (Designing and implementing this
> without   affecting any of the existing functionality took few days).
>
> *Thanks,*
> *Venkat.*
> Hi Isuru,
>
> Can we do review by 10 PM ..?
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Mon, Jul 11, 2016 at 10:52 PM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> Thanks for your time.  From today's discussion :
>>
>> 1) Remove SSL related config from LB level, as it is completely
>> dependent on transports level (carbon-transports).  LB sees as Inbound 
>> and
>> Outbound endpoints.  It is abstracted at gateway-framework level.
>>
>> 2) Add endpoints parameter to LB so that we can load balance multiple
>> groups specified in a single .iflow file.
>>
>> 3) Do Performance benchmark similar to MSF4J.
>>
>> 4) Transports statistics can be added at LB level, if needed.
>>
>> Will be looking forward for tomorrow's demo with Kasun. :)
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Mon, Jul 11, 2016 at 7:46 AM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> Good morning. Please find 7th week's progress.
>>>
>>> 1) Code Optimization - I've done some refactoring and optimization.
>>> 2) Though SSL support is not added in gateway-framework, I figured a
>>> work around and checked whether SSL offload and end to end encryption is
>>> working or not.. It is working fine. Once it is available in
>>> gateway-framework github, I'll take that.
>>> 3) Added statistics support and removed it. We have to discuss about
>>> this in our call today.
>>> 4) I've set up nginx LB in Ubuntu 64 bit VM. Did a load test using
>>> Apache Benchm

Re: [Dev] GSoC Project: HTTP Load Balancer on Top of WSO2 Gateway Discussion

[Isuru Ranawaka]

Hi venkat,

Yes we can. lets have a call today around 9.30 p.m

On Thu, Aug 4, 2016 at 9:34 AM, Venkat Raman  wrote:

> Hi Isuru,
>
> Good morning.  Yesterday night, I spoke with Kasun regarding the latest
> update on bench-mark results. Even without any locking performance is not
> good after concurrency of 5000.
>
> As you have done bench-mark till concurrency of 3000, we both would like
> to do bench-marking on raw carbon-transport upto concurrency of 10,000 and
> 1,00,000 requests so that we get an idea on this.
>
> How do we do that ? Will a simple response from engine suffice ?  Can I
> use LB to send simple response directly without doing any mediation ?
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Thu, Aug 4, 2016 at 12:19 AM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> Please find the attached bench-mark results.  As discussed,  I've
>> disabled health-checking and removed synchronized block and used atomic
>> Integer in one test and also did a test without any kind of lock or use of
>> atomic integers.
>>
>> Throughput and latency results are positive.  But, after concurrency
>> level of 5000 it is not that good.  So even If we use read-write lock or
>> stamped lock, we will get performance little performance gain only.
>>
>> I feel that If we can do bench-mark with integration-server upto 1
>> concurrent connections we'll get a better idea.  Is that okay ?
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Tue, Aug 2, 2016 at 9:39 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru & Kasun,
>>>
>>> Please find the findings from today's code review.
>>>
>>> 1) Locking in getNextLBOutboundEndpoint() method in algorithm
>>> implementation is causing over-head.  We have to find a way to efficiently
>>> handle communication between threads to reduce locking overhead.
>>>
>>> 2) Code repo freeze by August 15th for the sake of GSoC.  If we can find
>>> a way to overcome locking over-head before August 15th that changes will be
>>> added to code repo.  Otherwise it will be added after GSoC.
>>>
>>> 3) TPS, Latency and Memory graphs to be added.
>>>
>>> 4) Blog post and PDF documentation.
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Mon, Aug 1, 2016 at 9:25 AM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Good morning.  Please find 10th week's progress.

 1) Had discussion with Kasun.
 2) As suggested, did performance bench-marking using Netty BE, and it
 turns out that our LB is beating Nginx till concurrency level of 6000 after
 which it is not performing well.
 I've attached the results.

 I've started a new thread as Conversation arrangement is not good in
 previous one.

 It would be great if we can have a code review Isuru.  Based on your
 feedback I'll be abe to make changes and we can do bench-marking again.
 Can we do it today 9:30 PM ?  We have only 2 full weeks more.  The last
 week will be for documentation.



 *Thanks,*
 *Venkat.*

>>>
>>>
>>
>


-- 
Best Regards
Isuru Ranawaka
M: +94714629880
Blog : http://isurur.blogspot.com/
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] GSoC Project: HTTP Load Balancer on Top of WSO2 Gateway Discussion

[Kasun Indrasiri]

- Compare GW framework perf vs LB (need to identify if any perf impact from
the LB related code).
- Identify the reason for the apparent perf bottleneck with high
concurrency.

On Mon, Aug 8, 2016 at 10:55 AM, Venkat Raman  wrote:

> Hi Kasun,
>
> Please find the latest results after Saturday's code review.
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Mon, Aug 8, 2016 at 10:06 AM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> Good morning.  Please find 11th week's progress.
>>
>> 1) Had code reviews and made few suggested corrections.
>> 2) Did some groundwork for using JFR
>>
>> Will be continuing to work on performance tuning.
>>
>> @Kasun - Tomorrow is August 9th.  Can we have demo ?
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Sat, Aug 6, 2016 at 10:16 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> Here are the findings from today's review:
>>>
>>> 1) Change CallMediatorMap from ConcurrentHashMap to HashMap
>>> 2) Remove unnecessary Synchronized block while checking
>>> areAllEndpointsUnhealthy()
>>> 3) Rename LoadBalancerCallMediator to LBEndpointsCallMediator
>>> 4) Give a PR by adding getUri() method to gateway-framework
>>> 5) Use JavaFlightRecorder while doing benchmark to identify bottlenecks
>>>
>>>
>>>
>>>
>>> *Venkat.*
>>>
>>> On Fri, Aug 5, 2016 at 1:43 PM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Please find the attached latest bench-mark without synchronization,
 callBackpool, healthcheck.

 Throughput is just 1000 times faster than my current implementation.

 It is drastically falling because of some other reason.


 *Thanks,*
 *Venkat.*

 On Fri, Aug 5, 2016 at 10:09 AM, Venkat Raman 
 wrote:

> Hi IsuruU,
>
> FYI
>
>
>
> *Thanks,*
> *Venkat.*
>
> -- Forwarded message --
> From: Venkat Raman 
> Date: Thu, Aug 4, 2016 at 10:39 AM
> Subject: Re: GSoC Project: HTTP Load Balancer on Top of WSO2 Gateway
> Discussion
> To: Isuru Ranawaka , Kasun Indrasiri 
> Cc: DEV , Senduran Balasubramaniyam 
>
>
> Hi Isuru & Kasun,
>
> Please find the attached result document (raw-engine-transport.xlsx).
> I've done test with raw-engine-transport without any BE.  It is performing
> great  and is close to Netty based BE !!
>
> Problem is with LB only.
>
> My guess is that CallbackPool (using concurrent HashMap) that we are
> using to determine timeout is the bottle neck.  I'll disable Callback pool
> and do bench-mark and update you on that.
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Thu, Aug 4, 2016 at 9:48 AM, Venkat Raman 
> wrote:
>
>> Okay Isuru.
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Thu, Aug 4, 2016 at 9:42 AM, Isuru Ranawaka 
>> wrote:
>>
>>> Hi venkat,
>>>
>>> Yes we can. lets have a call today around 9.30 p.m
>>>
>>> On Thu, Aug 4, 2016 at 9:34 AM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Good morning.  Yesterday night, I spoke with Kasun regarding the
 latest update on bench-mark results. Even without any locking 
 performance
 is not good after concurrency of 5000.

 As you have done bench-mark till concurrency of 3000, we both would
 like to do bench-marking on raw carbon-transport upto concurrency of 
 10,000
 and 1,00,000 requests so that we get an idea on this.

 How do we do that ? Will a simple response from engine suffice ?
 Can I use LB to send simple response directly without doing any 
 mediation ?




 *Thanks,*
 *Venkat.*

 On Thu, Aug 4, 2016 at 12:19 AM, Venkat Raman >>> > wrote:

> Hi Isuru,
>
> Please find the attached bench-mark results.  As discussed,  I've
> disabled health-checking and removed synchronized block and used 
> atomic
> Integer in one test and also did a test without any kind of lock or 
> use of
> atomic integers.
>
> Throughput and latency results are positive.  But, after
> concurrency level of 5000 it is not that good.  So even If we use
> read-write lock or stamped lock, we will get performance little 
> performance
> gain only.
>
> I feel that If we can do bench-mark with integration-server upto
> 1 concurrent connections we'll get a better idea.  Is that okay ?
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Tue, Aug 2, 2016 at 9:39 PM, Venkat Raman  > wrote:
>
>> Hi Isuru & Kasun,
>>
>> Please find the findings from today's code review.
>>
>> 1) Locking in getNextLBOutboundEndpoint() method in algorithm
>> impl

Re: [Dev] GSoC Project: HTTP Load Balancer on Top of WSO2 Gateway Discussion

[Kasun Indrasiri]

Hi Venkat,

The drop of the performance of LB compared to GW Framework seems to be way
too much. I think we can't afford to lose nearly 50% of throughput because
of the LB components. Let's try to identify the bottlenecks related to LB
code.

On Tue, Aug 9, 2016 at 4:18 PM, Venkat Raman  wrote:

> Hi Isuru & Kasun,
>
> Please find the attached results document.  As discussed, I created a new
> VM for bench-marking.
>
> It seems like TPS of 20,000 (from yesterday's results) even at higher
> concurrency level is not accurate.  Sorry for the confusion caused.
> Most of the time endpoints were marked as unHealthy and direct error
> response is returned by LB Mediator which resulted in high TPS.  I tried
> multiple bench-marking from yesterday and I was never able to achieve that
> result.
>
> In this test, to avoid such cases, higher no of unHealthyRetries count has
> been configured.
>
> Also, I've bench-marked performance of GW-FMW using i-server with this
> simple
> 
> configuration.  It is a simple route even without if-else conditions.
>
> As you can see, It is performing 2X faster than LB.
>
> Next steps would be to do memory benchmark and plot graphs with these
> values.  Once repo, documentation and blog is ready, I'll be using JFR to
> identify bottle necks and on fine-tuning LB's performance.
>
> Will be looking forward to hear your feedback on this.
>
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Tue, Aug 9, 2016 at 1:13 AM, Venkat Raman  wrote:
>
>> Sure Kasun, I'll do a perf-benchmark between iserver and LB in a new VM
>> as discussed.
>>
>> Thanks,
>> Venkat.
>> On Aug 9, 2016 12:55 AM, "Kasun Indrasiri"  wrote:
>>
>>>
>>> - Compare GW framework perf vs LB (need to identify if any perf impact
>>> from the LB related code).
>>> - Identify the reason for the apparent perf bottleneck with high
>>> concurrency.
>>>
>>> On Mon, Aug 8, 2016 at 10:55 AM, Venkat Raman 
>>> wrote:
>>>
 Hi Kasun,

 Please find the latest results after Saturday's code review.




 *Thanks,*
 *Venkat.*

 On Mon, Aug 8, 2016 at 10:06 AM, Venkat Raman 
 wrote:

> Hi Isuru,
>
> Good morning.  Please find 11th week's progress.
>
> 1) Had code reviews and made few suggested corrections.
> 2) Did some groundwork for using JFR
>
> Will be continuing to work on performance tuning.
>
> @Kasun - Tomorrow is August 9th.  Can we have demo ?
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Sat, Aug 6, 2016 at 10:16 PM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> Here are the findings from today's review:
>>
>> 1) Change CallMediatorMap from ConcurrentHashMap to HashMap
>> 2) Remove unnecessary Synchronized block while checking
>> areAllEndpointsUnhealthy()
>> 3) Rename LoadBalancerCallMediator to LBEndpointsCallMediator
>> 4) Give a PR by adding getUri() method to gateway-framework
>> 5) Use JavaFlightRecorder while doing benchmark to identify
>> bottlenecks
>>
>>
>>
>>
>> *Venkat.*
>>
>> On Fri, Aug 5, 2016 at 1:43 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> Please find the attached latest bench-mark without synchronization,
>>> callBackpool, healthcheck.
>>>
>>> Throughput is just 1000 times faster than my current
>>> implementation.
>>>
>>> It is drastically falling because of some other reason.
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Fri, Aug 5, 2016 at 10:09 AM, Venkat Raman 
>>> wrote:
>>>
 Hi IsuruU,

 FYI



 *Thanks,*
 *Venkat.*

 -- Forwarded message --
 From: Venkat Raman 
 Date: Thu, Aug 4, 2016 at 10:39 AM
 Subject: Re: GSoC Project: HTTP Load Balancer on Top of WSO2
 Gateway Discussion
 To: Isuru Ranawaka , Kasun Indrasiri <
 ka...@wso2.com>
 Cc: DEV , Senduran Balasubramaniyam <
 sendu...@wso2.com>


 Hi Isuru & Kasun,

 Please find the attached result document
 (raw-engine-transport.xlsx).  I've done test with raw-engine-transport
 without any BE.  It is performing great  and is close to Netty based 
 BE !!

 Problem is with LB only.

 My guess is that CallbackPool (using concurrent HashMap) that we
 are using to determine timeout is the bottle neck.  I'll disable 
 Callback
 pool and do bench-mark and update you on that.



 *Thanks,*
 *Venkat.*

 On Thu, Aug 4, 2016 at 9:48 AM, Venkat Raman 
 wrote:

> Okay Isuru.
>
>
>
>
> *Thanks,*
> 

Re: [Dev] GSoC Project: HTTP Load Balancer on Top of WSO2 Gateway Discussion

[Kasun Indrasiri]

Venkat.. please check whether this is bounded by the contention in a map or
something... may be that's why it slows down when we have multiple
endpoints.

On Thu, Aug 11, 2016 at 2:43 AM, Venkat Raman  wrote:

> Hi Isuru,
>
> Please find the attached bench mark results that were done today.  I've
> used only one OutboundEP for Nginx, LB and i-server.  LB with HealthCheck
> and Timeout features is performing close to i-server.  So based on this..
> Is number of connections causing overhead ?
>
> In previous tests also, only one OutboundEP was used for benchmarking
> i-server against 5 OutboundEPs for Nginx and GW-LB.
>
> Even JFR results of  i-server and LB are similar.  Contentions and Latency
> occurring in LB are occurring in LB also.
>
> Also please find the attached JFR files.
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Wed, Aug 10, 2016 at 2:09 PM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> Please find the attached results with and without disruptor enablement.
>>
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Wed, Aug 10, 2016 at 9:23 AM, Venkat Raman 
>> wrote:
>>
>>> Sure Kasun.  Will try to find it.
>>>
>>> Thanks,
>>> Venkat.
>>> On Aug 10, 2016 5:30 AM, "Kasun Indrasiri"  wrote:
>>>
 Hi Venkat,

 The drop of the performance of LB compared to GW Framework seems to be
 way too much. I think we can't afford to lose nearly 50% of throughput
 because of the LB components. Let's try to identify the bottlenecks related
 to LB code.

 On Tue, Aug 9, 2016 at 4:18 PM, Venkat Raman 
 wrote:

> Hi Isuru & Kasun,
>
> Please find the attached results document.  As discussed, I created a
> new VM for bench-marking.
>
> It seems like TPS of 20,000 (from yesterday's results) even at higher
> concurrency level is not accurate.  Sorry for the confusion caused.
> Most of the time endpoints were marked as unHealthy and direct error
> response is returned by LB Mediator which resulted in high TPS.  I tried
> multiple bench-marking from yesterday and I was never able to achieve that
> result.
>
> In this test, to avoid such cases, higher no of unHealthyRetries count
> has been configured.
>
> Also, I've bench-marked performance of GW-FMW using i-server with this
> simple
> 
> configuration.  It is a simple route even without if-else conditions.
>
> As you can see, It is performing 2X faster than LB.
>
> Next steps would be to do memory benchmark and plot graphs with these
> values.  Once repo, documentation and blog is ready, I'll be using JFR to
> identify bottle necks and on fine-tuning LB's performance.
>
> Will be looking forward to hear your feedback on this.
>
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Tue, Aug 9, 2016 at 1:13 AM, Venkat Raman 
> wrote:
>
>> Sure Kasun, I'll do a perf-benchmark between iserver and LB in a new
>> VM as discussed.
>>
>> Thanks,
>> Venkat.
>> On Aug 9, 2016 12:55 AM, "Kasun Indrasiri"  wrote:
>>
>>>
>>> - Compare GW framework perf vs LB (need to identify if any perf
>>> impact from the LB related code).
>>> - Identify the reason for the apparent perf bottleneck with high
>>> concurrency.
>>>
>>> On Mon, Aug 8, 2016 at 10:55 AM, Venkat Raman 
>>> wrote:
>>>
 Hi Kasun,

 Please find the latest results after Saturday's code review.




 *Thanks,*
 *Venkat.*

 On Mon, Aug 8, 2016 at 10:06 AM, Venkat Raman >>> > wrote:

> Hi Isuru,
>
> Good morning.  Please find 11th week's progress.
>
> 1) Had code reviews and made few suggested corrections.
> 2) Did some groundwork for using JFR
>
> Will be continuing to work on performance tuning.
>
> @Kasun - Tomorrow is August 9th.  Can we have demo ?
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Sat, Aug 6, 2016 at 10:16 PM, Venkat Raman <
> vraman2...@gmail.com> wrote:
>
>> Hi Isuru,
>>
>> Here are the findings from today's review:
>>
>> 1) Change CallMediatorMap from ConcurrentHashMap to HashMap
>> 2) Remove unnecessary Synchronized block while checking
>> areAllEndpointsUnhealthy()
>> 3) Rename LoadBalancerCallMediator to LBEndpointsCallMediator
>> 4) Give a PR by adding getUri() method to gateway-framework
>> 5) Use JavaFlightRecorder while doing benchmark to identify
>> bottlenecks
>>
>>
>>
>>
>> *Venkat.*
>>
>> On Fri, Aug 5, 2016 at 1:43 PM, Venkat Raman <
>> vraman2...@gmail.com> wrote:
>>

Re: [Dev] GSoC Project: HTTP Load Balancer on Top of WSO2 Gateway Discussion

[Kasun Indrasiri]

On Thu, Aug 18, 2016 at 9:09 AM, Venkat Raman  wrote:

> Hi All,
>
> This project has been added to WSO2 Incubator
>  and I've been
> given membership to the same :D
>
> Would like to thank you all for your kind support and guidance throughout
> the course of this program.  I learned a lot and my programming and design
> skills have improved greatly.  I've also started blogging and I am part of
> VMware Open Source team !!
>
> All these were possible because of working in this wonderful project and
> amazing team ! :)
>
> Kindly have a look at my blog posts.  Would love to hear your suggestions.
>
> 1) Architecture : https://venkat2811.blogspot.
> in/2016/08/http-load-balancer-on-top-of-wso2.html
> 2) Performance: https://venkat2811.blogspot.in/2016/08/http-load-balancer-
> on-top-of-wso2_18.html
>
> I'll also be completing my final evaluations today.  @IsuruR and Kasun,
> It would be great if both of you could share your feedback with me also :)
>

Great job Venkat. You showed great passion and commitment througout the
project. It was a great pleassure to work with you and we would be more
than happy to work with you in the future. Please keep in touch!

Thanks,
Kasun



> Also have a look at this status updates and todo list
> 
> document which also available in README.md of the project.  As mentioned
> earlier will be working with you to make this project production ready.
>
> Once again thank you very much for giving me this wonderful opportunity !!
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Wed, Aug 17, 2016 at 9:09 PM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> I've written two blog posts:
>>
>> 1) One with Project repository and discussion on High level Architecture,
>> Engine Architecture, message flow and current features.
>> 2) Other on performance benchmarks.
>>
>> It would be great if you could create Incubation repo so that I can
>> update the URL in the blog post.
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Wed, Aug 17, 2016 at 8:15 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Kasun,
>>>
>>> PFA benchmarks done with OneOutboundEndpoint.
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Mon, Aug 15, 2016 at 4:41 PM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Please find 12th week's progress.

 1) Did performance benchmarks based on suggestions from code review.
 2) Did performance benchmarks on carbon gateway framework.
 3) Did JFR on gateway framework and LB.
 4) Found that more the no of Outbound Endpoints, lesser the performance.
 5) Currently writing blog post.
 6) Graphs added to repo.  Kindly find it here
 
 and here .




 *Thanks,*
 *Venkat.*

 On Thu, Aug 11, 2016 at 11:13 PM, Venkat Raman 
 wrote:

> Hi Isuru,
>
> In previous email, I've attached JFR for
>
> 1) LB (with healthcheck and timeout) with 5 OutboundEPs
> 2) i-server with 1 OutboundEP
>
> In this mail, I'm attaching
>
> 1) A very simple light weight mediator written to do load-balancing in
> round-robin fashion of 5 OutboundEPs
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Thu, Aug 11, 2016 at 10:21 PM, Venkat Raman 
> wrote:
>
>> Hi Kasun,
>>
>> Yes. I looked into JFR. With one endpoint disruptor is the most used.
>>
>> With 5 endpoints, HashMap is the most used.
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Thu, Aug 11, 2016 at 10:12 PM, Kasun Indrasiri 
>> wrote:
>>
>>> Venkat.. please check whether this is bounded by the contention in a
>>> map or something... may be that's why it slows down when we have 
>>> multiple
>>> endpoints.
>>>
>>> On Thu, Aug 11, 2016 at 2:43 AM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Please find the attached bench mark results that were done today.
 I've used only one OutboundEP for Nginx, LB and i-server.  LB with
 HealthCheck and Timeout features is performing close to i-server.  So 
 based
 on this.. Is number of connections causing overhead ?

 In previous tests also, only one OutboundEP was used for
 benchmarking i-server against 5 OutboundEPs for Nginx and GW-LB.

 Even JFR results of  i-server and LB are similar.  Contentions and
 Latency occurring in LB are occurring in LB also.

 Also please find the attached JFR files.




 *Thanks,*
 *Venkat.*

 On Wed, Aug 10, 2016 at 2:09 PM, Venkat Raman >>> > wrote:

> Hi Isuru,
>
> Please find the 

Re: [Dev] GSoC Project: HTTP Load Balancer on Top of WSO2 Gateway Discussion

[Isuru Ranawaka]

Hi Venkat,

As Kasun mentioned you have done great job and thanks for all the hard work
you did. We are looking forward to work with you in future as well.

Thanks


On Tue, Aug 23, 2016 at 2:05 AM, Kasun Indrasiri  wrote:

>
>
> On Thu, Aug 18, 2016 at 9:09 AM, Venkat Raman 
> wrote:
>
>> Hi All,
>>
>> This project has been added to WSO2 Incubator
>>  and I've been
>> given membership to the same :D
>>
>> Would like to thank you all for your kind support and guidance throughout
>> the course of this program.  I learned a lot and my programming and design
>> skills have improved greatly.  I've also started blogging and I am part of
>> VMware Open Source team !!
>>
>> All these were possible because of working in this wonderful project and
>> amazing team ! :)
>>
>> Kindly have a look at my blog posts.  Would love to hear your suggestions.
>>
>> 1) Architecture : https://venkat2811.blogspot.in
>> /2016/08/http-load-balancer-on-top-of-wso2.html
>> 2) Performance: https://venkat2811.blogspot.in
>> /2016/08/http-load-balancer-on-top-of-wso2_18.html
>>
>> I'll also be completing my final evaluations today.  @IsuruR and Kasun,
>> It would be great if both of you could share your feedback with me also :)
>>
>
> Great job Venkat. You showed great passion and commitment througout the
> project. It was a great pleassure to work with you and we would be more
> than happy to work with you in the future. Please keep in touch!
>
> Thanks,
> Kasun
>
>
>
>> Also have a look at this status updates and todo list
>> 
>> document which also available in README.md of the project.  As mentioned
>> earlier will be working with you to make this project production ready.
>>
>> Once again thank you very much for giving me this wonderful opportunity !!
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Wed, Aug 17, 2016 at 9:09 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> I've written two blog posts:
>>>
>>> 1) One with Project repository and discussion on High level
>>> Architecture, Engine Architecture, message flow and current features.
>>> 2) Other on performance benchmarks.
>>>
>>> It would be great if you could create Incubation repo so that I can
>>> update the URL in the blog post.
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Wed, Aug 17, 2016 at 8:15 PM, Venkat Raman 
>>> wrote:
>>>
 Hi Kasun,

 PFA benchmarks done with OneOutboundEndpoint.




 *Thanks,*
 *Venkat.*

 On Mon, Aug 15, 2016 at 4:41 PM, Venkat Raman 
 wrote:

> Hi Isuru,
>
> Please find 12th week's progress.
>
> 1) Did performance benchmarks based on suggestions from code review.
> 2) Did performance benchmarks on carbon gateway framework.
> 3) Did JFR on gateway framework and LB.
> 4) Found that more the no of Outbound Endpoints, lesser the
> performance.
> 5) Currently writing blog post.
> 6) Graphs added to repo.  Kindly find it here
> 
> and here .
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Thu, Aug 11, 2016 at 11:13 PM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> In previous email, I've attached JFR for
>>
>> 1) LB (with healthcheck and timeout) with 5 OutboundEPs
>> 2) i-server with 1 OutboundEP
>>
>> In this mail, I'm attaching
>>
>> 1) A very simple light weight mediator written to do load-balancing
>> in round-robin fashion of 5 OutboundEPs
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Thu, Aug 11, 2016 at 10:21 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Kasun,
>>>
>>> Yes. I looked into JFR. With one endpoint disruptor is the most used.
>>>
>>> With 5 endpoints, HashMap is the most used.
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Thu, Aug 11, 2016 at 10:12 PM, Kasun Indrasiri 
>>> wrote:
>>>
 Venkat.. please check whether this is bounded by the contention in
 a map or something... may be that's why it slows down when we have 
 multiple
 endpoints.

 On Thu, Aug 11, 2016 at 2:43 AM, Venkat Raman >>> > wrote:

> Hi Isuru,
>
> Please find the attached bench mark results that were done today.
> I've used only one OutboundEP for Nginx, LB and i-server.  LB with
> HealthCheck and Timeout features is performing close to i-server.  So 
> based
> on this.. Is number of connections causing overhead ?
>
> In previous tests also, only one OutboundEP was used for
> benchmarking i-server against 5 OutboundEPs for Nginx and GW-LB.
>
> Ev

Re: [Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Ravi Undupitiya]

Hi Venkat,

Please find answers inline below.

On Tue, May 24, 2016 at 9:29 PM, Venkat Raman  wrote:
>
>
> 1) Why are we pushing variables in carbon message variables stack..??
>

One of the features of the language is to have variables we can declare and
use throughout the mediation flow. We are able to populate variables during
loading and during runtime and so variable stack is kept inside the carbon
message so each thread has its own variable stack. Hope that clarifies.


> 2) Also could you kindly explain how AbstractFlowController and
> FlowControllerCallback are working together.  My understanding is not quite
> clear on this.
>

AbstractFlowControllers are mediators that require controlling the flow
(like filter mediator) where we the mediation flow branching out, in which
case we want to mediate only the branch we're in and FlowControllerCallback
is a CarbonCallback that we pass on to receive method of mediators of a
particular branch. This allows us to continue mediation from the parent
once a branch is complete. You can test this out by debugging the
FilterMediator, try passing on the usual callback and then check by passing
on a FlowControllerCallback.

@IsuruR, please correct me if I'm wrong. :)


Thanks,
-- 
*Ravi Undupitiya*
Senior Software Engineer; WSO2 http://wso2.com


*E-mail: r...@wso2.com **M: **+94 772 930 712
<%2B94%C2%A0772%20930%20712>*

Lean . Enterprise . Middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Isuru Ranawaka]

Hi Venkat.

Good.I will look at the implementation and setup a call for discuss next
steps.

thanks

On Mon, May 30, 2016 at 10:27 AM, Venkat Raman  wrote:

> Hi Isuru,
>
> Good morning :) . Kindly find the 1st week's progress.
>
> 1) Implementation of Round-Robin algorithm for Outbound Endpoints.
> 2) Round-Robin algorithm with group support, but one .iflow config file
> can have only one group as of now as mentioned earlier.
> 3) Few input validations.
>
> Kindly find the repo here.
> 
>
> In proposal I've mentioned that I'll be covering HTTPS 2nd week,since
> it'll be made available later in carbon-gateway-framework repo, I'm
> planning to proceed with adding session persistence.
>
> Will be looking forward for your feedback.
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Wed, May 25, 2016 at 8:57 PM, Venkat Raman 
> wrote:
>
>> Hi Ravi,
>>
>> Thank you for the clear explanation :) .  Yes, I added  appropriate log
>> statements and I was able to understand the flow and I am clear with it
>> now.
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Wed, May 25, 2016 at 8:09 PM, Ravi Undupitiya  wrote:
>>
>>> Hi Venkat,
>>>
>>> Please find answers inline below.
>>>
>>> On Tue, May 24, 2016 at 9:29 PM, Venkat Raman 
>>> wrote:


 1) Why are we pushing variables in carbon message variables stack..??

>>>
>>> One of the features of the language is to have variables we can declare
>>> and use throughout the mediation flow. We are able to populate variables
>>> during loading and during runtime and so variable stack is kept inside the
>>> carbon message so each thread has its own variable stack. Hope that
>>> clarifies.
>>>
>>>
 2) Also could you kindly explain how AbstractFlowController and
 FlowControllerCallback are working together.  My understanding is not quite
 clear on this.

>>>
>>> AbstractFlowControllers are mediators that require controlling the flow
>>> (like filter mediator) where we the mediation flow branching out, in which
>>> case we want to mediate only the branch we're in and FlowControllerCallback
>>> is a CarbonCallback that we pass on to receive method of mediators of a
>>> particular branch. This allows us to continue mediation from the parent
>>> once a branch is complete. You can test this out by debugging the
>>> FilterMediator, try passing on the usual callback and then check by passing
>>> on a FlowControllerCallback.
>>>
>>> @IsuruR, please correct me if I'm wrong. :)
>>>
>>>
>>> Thanks,
>>> --
>>> *Ravi Undupitiya*
>>> Senior Software Engineer; WSO2 http://wso2.com
>>>
>>>
>>> *E-mail: r...@wso2.com **M: **+94 772 930 712
>>> <%2B94%C2%A0772%20930%20712>*
>>>
>>> Lean . Enterprise . Middleware
>>>
>>
>>
>


-- 
Best Regards
Isuru Ranawaka
M: +94714629880
Blog : http://isurur.blogspot.com/
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Isuru Ranawaka]

Hi Venkat,

Good progress. I will schedule a call or code review within this week.

thanks

On Mon, Jun 20, 2016 at 9:18 AM, Venkat Raman  wrote:

> Hi Isuru,
>
> Good morning.  Kindly find the 4th week's progress.
>
> 1) Algorithm : a)  Least Response Time
>
> Response time of endpoints is calculated by Running Average of
> response time.  By this value, load will be balanced in such a way that,
> endpoints high Running average of response time will have less load
> directed to it and vice versa.  However, persistence will be maintained.
>
> b) Random
> Endpoints will be chosen Randomly. Persistence will be maintained
> though.
>
>
> 2) Health Checking : Healthy Endpoint detection.
>
>Once an Endpoint is back from unHealthy to Healthy state, LB will be
> able to detect it and direct further incoming traffic to it.
>
> Kindly note that there is a small deviation from my proposal.  In proposal
> I have mentioned that I'll also be implementing Least Connections
> Algorithm.  It is possible if we are load balancing on per service basis.
> With load balancing based on per endpoint basis, it will not be possible to
> implement it because, we will not have context outside of a specific iFlow
> file.  Instead, I have implemented Strict IP Hashing and Random algorithms
> which I have mentioned in IF TIME PERMITS section of my proposal.
>
> This week, I am planning to implement Weighted Round Robin.
>
> As mentioned earlier, I am waiting for HTTPS support to be included in
> carbon-gateway-framework repo.
>
> I'll also be sending you a document that has consolidated list of all the
> work that I have done so far, so that it'll be easier for you during
> mid-term evaluations.
>
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Mon, Jun 13, 2016 at 11:03 AM, Venkat Raman 
> wrote:
>
>> Hi Isusu,
>>
>> Good morning.  Kindly find the 3rd weeks progress.
>>
>> 1) Algorithm : StrictClientIPHashing
>>
>>This algorithm uses Client's IP address hashing technique to determine
>> OutboundEndpoint. If Client's IP address is not available in request
>> header, request will be discarded.
>>
>> 2) Persistence policy : ClientIPHahsing.
>>
>> Same as algorithm. But if Client's IP address is not available in
>> header, instead of discarding the request, OutboundEndpoint will be chosen
>> based on specified algorithm.
>>
>> 3) Implemented LBOutboundEndpoint. (as we discussed in call)
>>
>>   It'll be used inside LB. It will have reference for OutboundEndpoint
>> (HTTP or HTTPS) along with other attributes like healthyFlag,
>> unHealthyRetriesCount, healthyRetriesCount and weights (will be added in
>> future) for weighted  implementations of certain algorithms.
>>
>> 4) UnHealthy endpoint detection.
>>
>> I'm using ConcurrentHashMap to store RequestCallBacks (CallBackPool).
>> If response is received from OutboundEndpoint within specified timeOut, the
>> corresponding callBack will be removed from pool.  If callBack object is
>> still available in callBack pool, it means that it has been timed out. A
>> separate handler will be periodically checking for this.  In such cases
>> HTTP Error Code 504 : "Gateway Timeout" will be sent back to client. If
>> unHealthyRetries count has been reached, that endpoint will be marked as
>> unHealthy and it will not be chosen by algorithms till it is back to
>> healthy.  Kindly note that any other error response from BackEnd will be
>> directly sent back to client. LB will not handle those errors as it'll be
>> application specific.
>>
>> Now, I'm working Healthy Endpoint detection. i.e., when an endpoint is
>> back from unHealthy to Healthy state, LB should detect it automatically and
>> choose that endpoint for further requests.
>>
>> Also, I'm expecting this to be completed by this week.  It would be great
>> if you could add HTTPS support in Carbon-Gateway-Framework by next week so
>> that I can proceed with that.
>>
>>
>>
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Tue, Jun 7, 2016 at 12:14 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> I've completed client IP hashing based persistence.  Now I'm working on
>>> HealthCheck Implementation.  Kindly find the here.
>>> 
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Mon, Jun 6, 2016 at 11:18 AM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Good morning.  Kindly find the 2nd week's progress

 1) Implementation of Cookie based persistence
 a) LB cookie will be appended with the cookie in response to
 maintain persistence. So this has the same timeout and other properties of
 that cookie.
 b) There won't be any cookie in response from backend.  So, LB will
 insert its own cookie (session cookie).  It will be valid till client
 browser is closed.

 2) Laid ground work for Client IP Hashing based persistence.

 This week I'll be continuing to work on Client IP hashing.

 Your

Re: [Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Kasun Indrasiri]

Hi Venkat,

Good progress indeed!
Shall we do a demo/review on the current status and prioritize the features
that we expect for the final evaluation.
IsuruR will help you with arranging the meeting.

On Mon, Jun 27, 2016 at 9:19 PM, Venkat Raman  wrote:

> Hi Isuru,
>
> Good morning.  I'm currently working on few validations and writing
> unit-test cases.
>
> Kindly find the attached document of TODO list.  If there is anything more
> kindly let me know.
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Tue, Jun 28, 2016 at 1:09 AM, Venkat Raman 
> wrote:
>
>> Hi All,
>>
>> It's been a great journey with WSO2 community !!  I've passed my mid-term
>> evaluations. :D
>>
>> Thanks for your feedback IsuruR.  And sure, I will keep improving OOP
>> knowledge and analytical thinking.
>>
>> Will be working with same pace and dedication to make our project
>> successful.  It's been a great learning so far.
>>
>> Would also like to thank IsuruR, Senduran, IsuruU, Ravi and Kasun for
>> their valuable support and guidance :)
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Sat, Jun 25, 2016 at 3:10 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> Good morning.  Please find 5th week's progress.
>>>
>>> Algorithm : Weighted Round Robin and Weighted Random algorithms.  Kindly
>>> note that Client IP Hashing based is not supported for weighted algorithms.
>>>
>>> In the following weeks, I'm planning to write Unit Test cases.  Once
>>> HTTPS support is added, I'll add it to LB.
>>>
>>> Also find the attached Updated mid-term evaluation document.
>>>
>>> Kindly note that Tuesday early morning (12:30 A.M) is the last date, for
>>> submitting mid-term evaluations.
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Tue, Jun 21, 2016 at 12:43 PM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 I've submitted my Mid - Term evaluation.  It would be great if you
 could share your feedback with me once you complete it.

 Also, I'm looking forward for our code review.




 *Thanks,*
 *Venkat.*

 On Mon, Jun 20, 2016 at 9:28 AM, Venkat Raman 
 wrote:

> Ok Isuru. Looking forward to it :)
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Mon, Jun 20, 2016 at 9:21 AM, Isuru Ranawaka 
> wrote:
>
>> Hi Venkat,
>>
>> Good progress. I will schedule a call or code review within this
>> week.
>>
>> thanks
>>
>> On Mon, Jun 20, 2016 at 9:18 AM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> Good morning.  Kindly find the 4th week's progress.
>>>
>>> 1) Algorithm : a)  Least Response Time
>>>
>>> Response time of endpoints is calculated by Running Average of
>>> response time.  By this value, load will be balanced in such a way that,
>>> endpoints high Running average of response time will have less load
>>> directed to it and vice versa.  However, persistence will be maintained.
>>>
>>> b) Random
>>> Endpoints will be chosen Randomly. Persistence will be
>>> maintained though.
>>>
>>>
>>> 2) Health Checking : Healthy Endpoint detection.
>>>
>>>Once an Endpoint is back from unHealthy to Healthy state, LB will
>>> be able to detect it and direct further incoming traffic to it.
>>>
>>> Kindly note that there is a small deviation from my proposal.  In
>>> proposal I have mentioned that I'll also be implementing Least 
>>> Connections
>>> Algorithm.  It is possible if we are load balancing on per service 
>>> basis.
>>> With load balancing based on per endpoint basis, it will not be 
>>> possible to
>>> implement it because, we will not have context outside of a specific 
>>> iFlow
>>> file.  Instead, I have implemented Strict IP Hashing and Random 
>>> algorithms
>>> which I have mentioned in IF TIME PERMITS section of my proposal.
>>>
>>> This week, I am planning to implement Weighted Round Robin.
>>>
>>> As mentioned earlier, I am waiting for HTTPS support to be included
>>> in carbon-gateway-framework repo.
>>>
>>> I'll also be sending you a document that has consolidated list of
>>> all the work that I have done so far, so that it'll be easier for you
>>> during mid-term evaluations.
>>>
>>>
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Mon, Jun 13, 2016 at 11:03 AM, Venkat Raman >> > wrote:
>>>
 Hi Isusu,

 Good morning.  Kindly find the 3rd weeks progress.

 1) Algorithm : StrictClientIPHashing

This algorithm uses Client's IP address hashing technique to
 determine OutboundEndpoint. If Client's IP address is not available in
 request header, request will be discarded.

 2) Persistence policy : ClientIPHahsing.

 Same as algorithm. But if Client's IP address is not a

Re: [Dev] GSoC Project: HTTP Load Balancer On Top Of WSO2 Gateway Discussion

[Isuru Ranawaka]

Hi Venkat,

SSL OFFLOAD  thing I can't see https is enabled.

On Wed, Jul 6, 2016 at 10:11 PM, Venkat Raman  wrote:

> Hi Isuru,
>
> I'm blocked by this issue from yesterday.  We need have a small
> discussion ASAP.  I'll try to explain the scenario more clearly here than I
> did in hangouts.
>
> For LB these are the 3 different types netty-transports.yml configs that
> we need to support.
>
> 1) For no ssl : no_SSL
> 
>
> 2) SSL_OFFLOAD : (i.e) https listener and http sender ssl_offload
> 
>
> 3) END_TO_END : (i.e) https listener and https sender end_to_end
> 
>
> Are these configs correct..?
>
> So far I have been using (1) as we were dealing only with http.  It was
> working fine.  Refer first screenshot for logs.  If you see, netty listener
> is starting in both 8290 and 9090.
>
> 8290 is inboundEndpoint  port specified in router.iflow file and 9090 is
> specified in netty-transports.yml
>
> Now, when I try (2)&(3), since we need to deal with https, netty listener
> is starting only in 9292.  Refer second screenshot for logs.  *We need a
> netty listener to be started in 8290 port also.  But it is not happening.*
>
> 9292 is specified in netty-transports.yml & 8290 is specified in
> router.iflow
>
> But, when I change 8290 to 9292 in our router.iflow file, ssl is working
> fine.
>
> Kindly guide me through it.
> * It is critical as it is blocking my progress.*
> Looking forward to hear from you.
>
>
> *Thanks,*
> *Venkat.*
>
> On Mon, Jul 4, 2016 at 10:33 AM, Venkat Raman 
> wrote:
>
>> Hi Isuru,
>>
>> Good morning.  Please find 6th week's progress
>>
>> 1)  Demo / Discussion
>> 2)  Change in Health Checking : Instead of caching and re-using
>> carbonMessage's properties, we are establishing a socket connection to see
>> whether specified endpoint is alive or not. Find the issue
>>  here.
>> 3)  Active Health Checking.  Previously it was only passive mode
>> available.  I've implemented active health checking as you asked during our
>> demo.
>>
>> Also kindly find my new post on mid-term evaluation here
>> .
>>
>> It would be great if you could share some SSL config samples with me.
>>
>> Looking forward for our code review today.
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Sun, Jul 3, 2016 at 4:49 PM, Venkat Raman 
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> As discussed yesterday, I've added ACTIVE_HEALTH_CHECK feature.
>>>
>>> Do we have code review today..?  Also could you kindly share sample SSL
>>> config .?
>>>
>>>
>>>
>>> *Thanks,*
>>> *Venkat.*
>>>
>>> On Sat, Jul 2, 2016 at 11:31 PM, Venkat Raman 
>>> wrote:
>>>
 Hi Isuru,

 Here is the summary of our demo / discussion today.

 DEMO:

 1) Algorithms: ROUND_ROBIN and WEIGHTED_ROUND_ROBIN

 2) Persistence : NO_PERSISTENCE and CLIENT_IP_HASHING.

 3) Health Checking : PASSIVE

 DISCUSSION:

 1) Persistence : Difference between LB_COOKIE & APPLICATION_COOKIE.

 2) Support for HTTPS : We discussed regarding adding separate
 HTTPSEndpoints or not. Browser expects https when it is dealing with SSL.
 As of now, we are adding SSL configs as parameters.  We need to look into
 it.

 3) Active Health Checking : I'll add this new feature by this week.

 Also, kindly share sample SSL configs and also any PPT / documentation
 / architecture diagrams for carbon-transports with me, so that it would be
 helpful for me to add SSL.

 As discussed, if you are free we can do our code review tomorrow.


 *Thanks,*
 *Venkat.*

 On Sat, Jul 2, 2016 at 8:46 PM, Venkat Raman 
 wrote:

> Hi Isuru,
>
> If you are free, can we have review tomorrow..?
>
>
>
>
> *Thanks,*
> *Venkat.*
>
> On Wed, Jun 29, 2016 at 10:40 PM, Venkat Raman 
> wrote:
>
>>
>> I'm ready for our demo Isuru.  Awaiting your arrival.
>>
>>
>>
>> *Thanks,*
>> *Venkat.*
>>
>> On Wed, Jun 29, 2016 at 7:13 AM, Venkat Raman 
>> wrote:
>>
>>> Hi Kasun,
>>>
>>> Yea sure !
>>>
>>> Thanks,
>>> Venkat.
>>> On Jun 29, 2016 4:37 AM, "Kasun Indrasiri"  wrote:
>>>
 Hi Venkat,

 Good progress indeed!
 Shall we do a demo/review on the current status and prioritize the
 features that we expect for the final evaluation.
 IsuruR will help you with arranging the