Re: [Architecture] Methods to get currentSubject and currentStep in authentication script

[Harshan Liyanage]

Hi Senthalan,

What I understood by reading your description on the behavior of the
*context.currentSubject *method is that it always returns the subject of
the *last-completed-subject-identifier-step* rather than the subject of the
current subject identifier step. If my understanding is correct, I suggest
you change it to something more meaningful name such as
*context.lastSubject*.

I'm +1 with *context.currentStep.*

Thanks,

Harshan Liyanage
Mobile: *+94765672894*
Email: hars...@wso2.com
Blog: http://harshanliyanage.blogspot.com/
Medium: https://medium.com/@harshan.dll
*WSO2, Inc.:** wso2.com *
lean.enterprise.middleware.


On Tue, Jul 10, 2018 at 9:39 PM Senthalan Kanagalingam 
wrote:

> Hi all,
>
> I am working on to get the currently authenticated subject and currently
> executing step from the authentication script. Now, if the identity admin
> wants to get the authenticated subject, he/she has to know which step was
> set as the subject identifier step and have to call,
> "context.step[].subject".
>
> So, we have planned to implement a method as,
>
> *context.currentSubject *
>
> which will return the subject of the "subject identifier step", if that
> step is completed. Else return the subject of the last completed step.
>
> Another implementation is to have a method to get the currently exected
> method. Currently, identity admin has to specify the step number in order
> to get the details. "context.step[]". This will affect the
> reusability of the code.
>
> With this new implementation, the identity admin can use,
>
> *context.currentStep*
>
> which will return the executing step.
>
> Please share your comments on the naming of the methods.
>
> thanks,
> Senthalan.
> --
>
> *Senthalan Kanagalingam*
> *Software Engineer - WSO2 Inc.*
> *Mobile : +94 (0) 77 18 77 466*
> 
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Stream Processor HA improved archectecture

[Harshan Liyanage]

Hi Damith,

According to this architecture, the passive node will only fetch event
states and filter events only once it is active. This model might exhaust
the queue 2 very soon depending on the rate of events. What if we could add
another async task to check the state and filter events even if the node is
passive?

Thanks,

Harshan Liyanage
Mobile: *+94765672894*
Email: hars...@wso2.com
Blog: http://harshanliyanage.blogspot.com/
Medium: https://medium.com/@harshan.dll
*WSO2, Inc.:** wso2.com *
lean.enterprise.middleware.


On Fri, Jul 13, 2018 at 9:01 AM Tishan Dahanayakage  wrote:

> @Anoukh,
> We can configure a LB in front to handle failover. Actually that is the
> correct solution rather than publishing to both. But I think we need to
> figure out how to handle incoming databridge traffic as it cannot be load
> balanced. Yes we can utilize client side load balancing and directly
> publish to SP nodes. But then we will need to shutdown thrift/binary
> receiving servers of passive node for the failover to happen.
>
> @Damith,
> I have below questions/suggestions.
> 1. What is the expected behavior when queue is full in active node?
> First of all I suppose this is a in-memory queue. This is something we
> should think clearly and implement as even now we are facing quite a number
> of issues where server is stalled because queues are getting filled. When
> queue in active node is full
> - We can't let receiving threads to get block on the queue as it will
> cause server to freeze.
> - At the same time We can't also drop the event if exactly once processing
> or zero event drop is enforced.
> - Also we can't keep adding stuff to queue as it will go OOM.
> So IMO we need a way to persist the queue. In other words we need a proper
> broker queue to solve this issue. If user is fine with event loss then We
> can live with in-memory queue.
>
> 2. We need a automated purging task to clean queue2 to after every state
> persistance done by active node.
>
> /Tishan
>
> On Thu, Jul 12, 2018 at 10:41 PM, Anoukh Jayawardena 
> wrote:
>
>> Hi Damith,
>>
>> Just a few clarifications on the new architecture.
>> I'm assuming that "queue1" and "queue2" are actually per Source? Which
>> means if we have 3 sources, we'll have 3 queues in active node and 3 queues
>> in passive node? Does that mean there will be 3 threads working in async?
>> Also, the main difference with the current HA architecture is that both
>> nodes will not receive all the events. So at the moment the active node
>> goes down, how does the passive node start receiving events? Will we be
>> handling this or should the cluster be fronted by a separate load balancer?
>>
>> Thanks,
>> Anoukh
>>
>> On Thu, Jul 12, 2018 at 8:34 PM Damith Wickramasinghe 
>> wrote:
>>
>>> Hi all,
>>>
>>> We are in the process of refactor/improve the existing HA architecture
>>> due to various concerns found.
>>>
>>> Below is the high level design came up with. We will provide more
>>> in-depth details as the implementation carries on.
>>>
>>>
>>> ​
>>> ​
>>> As per above at a given point of time there will only be a one active
>>> node. Passive node will not consume any events. Electing active node will
>>> work as per current architecture via cluster coordination.
>>> A thread will work in the active node to put the data into a
>>> queue(queue1) and same thread will then publish the events to
>>> outside.Reason for having a queue here because to send events
>>> asynchronously to passive node. Here we are going to send events to passive
>>> node via TCP. (This we need to decide) Active node will persist the state
>>> periodically to the database.
>>>
>>> When active node goes down via cluster coordination passive node will
>>> become active. When active it will get the state from database(to sync the
>>> state) with latest event timestamp , and filter events from queue 2 (in
>>> order to stop processing already processed events) and send them out. Then
>>> open the ports in the newly active node and start receiving events from
>>> sources. In this architecture also at least one processing will be done.
>>>
>>> Nisala please add anything I missed.
>>>
>>> Regards,
>>> Damith
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Senior Software Engineer
>>> WSO2 Inc.; http://wso2.com
>>> 
>>> lean.enterprise.middleware
>>>
>>> mobile: *+94728671315*
>>>
>>>
>>
>> --
>> *Anoukh Jayawardena*
>> *Software Engineer*
>>
>> *WSO2 Lanka (Private) Limited: http://wso2.com
>> lean.enterprise.middle-ware*
>>
>>
>> *phone: (+94) 77 99 28932*
>> 
>>
>
>
>
> --
> Tishan Dahanayakage
> Associate Technical Lead
> WSO2, Inc.
> Mobile:+94 716481328
>
> Disclaimer: This communication may contain privileged or other
> confidential information and is intended exclusively for the addressee/s.
> If you are not the intended recipient/s, or believe that you may have

Re: [Architecture] [IAM] Introducing New Claim Properties to Control Claims Shown in Different Views

[Isura Karunaratne]

Hi Johann,



On Wed, Jun 27, 2018 at 8:52 AM Johann Nallathamby  wrote:

> Hi IAM Team,
>
> I think the following limitation in the WSO2 IS is causing some major
> usability issues.
>
> We have following views mainly where we display claims for a user:
> 1. Admin user profile view in management console
> 2. My user profile view in user portal
> 3. Self sign-up view
>
> The way we select the claims shown in these views is based on a single
> property which is "supported by default". This means that we can't control
> the behavior of a claim individually for each of these views and causing
> some major usability issues. This also means that to get that experience
> users have to do JSP customization to the product which I think we can
> easily avoid. And when it comes to IDaaS, customization is not a choice.
>
> For example a claim like "Account Locked" has to be shown in "admin user
> profile" view, but not for "self sign-up" view or "my user profile" view.
> Also there can be a use case where I want to show minimal vital information
> on the self sign-up page to make it easier for the user to sign-up, but
> have an extensive profile view in the user portal.
>
> I faced these issues when trying to demo the product to customers, where
> when I want to show the self sign-up flow with email verification, I enable
> to "supported by default" property for "account locked" claim to show in
> the "admin user profile" view that the account is actually getting locked
> once the user signs up and until he confirms the email. But while doing
> that the claim also gets visible in the "self sign-up" view  that makes no
> sense.
>
> We in fact have the capability to configure properties for each claim from
> IS 5.3.0 onwards. This issue explained in this mail was actually one of the
> reasons we added this capability to IS 5.3.0, but later it got diluted due
> to IS 6.0.0 efforts. Can we introduce some new claim properties for the 3
> profiles above to control the visibility of the claims individually for
> each of these views?
>
> We can easily do this without breaking backward compatibility. Since these
> properties can be added through the claim management UI, we can even fix
> the existing product versions in this way. We can continue to use the
> "supported by default" property as the default property to fallback on if
> the relevant new property is not configured. But if a user explicitly
> configures this property we can use it. For the new versions we can ship
> these properties with default values we agree on (may be not for all
> claims, but the most important ones only, and for the rest the users can
> explicitly create the properties through the UI).
>

Thanks for bringing this up. We will add this capability in a future
release. [1]

[1] https://github.com/wso2/product-is/issues/3412

Thanks
Isura.



> Thanks and Regards,
> Johann.
>
> --
>
> *Johann Dilantha Nallathamby*
> Senior Lead Solutions Engineer
> WSO2, Inc.
> lean.enterprise.middleware
>
> Mobile: *+94 77 7776950*
> LinkedIn: *http://www.linkedin.com/in/johann-nallathamby
> *
> Medium: *https://medium.com/@johann_nallathamby
> *
> Twitter: *@dj_nallaa*
>


-- 

*Isura Dilhara Karunaratne*
Associate Technical Lead | WSO2 
*lean.enterprise.middleware*
Email: is...@wso2.com
Mob : +94 772 254 810
Blog : http://isurad.blogspot.com/
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Stream Processor HA improved archectecture

[Tishan Dahanayakage]

@Anoukh,
We can configure a LB in front to handle failover. Actually that is the
correct solution rather than publishing to both. But I think we need to
figure out how to handle incoming databridge traffic as it cannot be load
balanced. Yes we can utilize client side load balancing and directly
publish to SP nodes. But then we will need to shutdown thrift/binary
receiving servers of passive node for the failover to happen.

@Damith,
I have below questions/suggestions.
1. What is the expected behavior when queue is full in active node?
First of all I suppose this is a in-memory queue. This is something we
should think clearly and implement as even now we are facing quite a number
of issues where server is stalled because queues are getting filled. When
queue in active node is full
- We can't let receiving threads to get block on the queue as it will cause
server to freeze.
- At the same time We can't also drop the event if exactly once processing
or zero event drop is enforced.
- Also we can't keep adding stuff to queue as it will go OOM.
So IMO we need a way to persist the queue. In other words we need a proper
broker queue to solve this issue. If user is fine with event loss then We
can live with in-memory queue.

2. We need a automated purging task to clean queue2 to after every state
persistance done by active node.

/Tishan

On Thu, Jul 12, 2018 at 10:41 PM, Anoukh Jayawardena 
wrote:

> Hi Damith,
>
> Just a few clarifications on the new architecture.
> I'm assuming that "queue1" and "queue2" are actually per Source? Which
> means if we have 3 sources, we'll have 3 queues in active node and 3 queues
> in passive node? Does that mean there will be 3 threads working in async?
> Also, the main difference with the current HA architecture is that both
> nodes will not receive all the events. So at the moment the active node
> goes down, how does the passive node start receiving events? Will we be
> handling this or should the cluster be fronted by a separate load balancer?
>
> Thanks,
> Anoukh
>
> On Thu, Jul 12, 2018 at 8:34 PM Damith Wickramasinghe 
> wrote:
>
>> Hi all,
>>
>> We are in the process of refactor/improve the existing HA architecture
>> due to various concerns found.
>>
>> Below is the high level design came up with. We will provide more
>> in-depth details as the implementation carries on.
>>
>>
>> ​
>> ​
>> As per above at a given point of time there will only be a one active
>> node. Passive node will not consume any events. Electing active node will
>> work as per current architecture via cluster coordination.
>> A thread will work in the active node to put the data into a
>> queue(queue1) and same thread will then publish the events to
>> outside.Reason for having a queue here because to send events
>> asynchronously to passive node. Here we are going to send events to passive
>> node via TCP. (This we need to decide) Active node will persist the state
>> periodically to the database.
>>
>> When active node goes down via cluster coordination passive node will
>> become active. When active it will get the state from database(to sync the
>> state) with latest event timestamp , and filter events from queue 2 (in
>> order to stop processing already processed events) and send them out. Then
>> open the ports in the newly active node and start receiving events from
>> sources. In this architecture also at least one processing will be done.
>>
>> Nisala please add anything I missed.
>>
>> Regards,
>> Damith
>>
>>
>>
>>
>>
>>
>> --
>> Senior Software Engineer
>> WSO2 Inc.; http://wso2.com
>> 
>> lean.enterprise.middleware
>>
>> mobile: *+94728671315*
>>
>>
>
> --
> *Anoukh Jayawardena*
> *Software Engineer*
>
> *WSO2 Lanka (Private) Limited: http://wso2.com
> lean.enterprise.middle-ware*
>
>
> *phone: (+94) 77 99 28932*
> 
>



-- 
Tishan Dahanayakage
Associate Technical Lead
WSO2, Inc.
Mobile:+94 716481328

Disclaimer: This communication may contain privileged or other confidential
information and is intended exclusively for the addressee/s. If you are not
the intended recipient/s, or believe that you may have received this
communication in error, please reply to the sender indicating that fact and
delete the copy you received and in addition, you should not print, copy,
re-transmit, disseminate, or otherwise use the information contained in
this communication. Internet communications cannot be guaranteed to be
timely, secure, error or virus-free. The sender does not accept liability
for any errors or omissions.
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Stream Processor HA improved archectecture

[Anoukh Jayawardena]

Hi Damith,

Just a few clarifications on the new architecture.
I'm assuming that "queue1" and "queue2" are actually per Source? Which
means if we have 3 sources, we'll have 3 queues in active node and 3 queues
in passive node? Does that mean there will be 3 threads working in async?
Also, the main difference with the current HA architecture is that both
nodes will not receive all the events. So at the moment the active node
goes down, how does the passive node start receiving events? Will we be
handling this or should the cluster be fronted by a separate load balancer?

Thanks,
Anoukh

On Thu, Jul 12, 2018 at 8:34 PM Damith Wickramasinghe 
wrote:

> Hi all,
>
> We are in the process of refactor/improve the existing HA architecture due
> to various concerns found.
>
> Below is the high level design came up with. We will provide more in-depth
> details as the implementation carries on.
>
>
> ​
> ​
> As per above at a given point of time there will only be a one active
> node. Passive node will not consume any events. Electing active node will
> work as per current architecture via cluster coordination.
> A thread will work in the active node to put the data into a queue(queue1)
> and same thread will then publish the events to outside.Reason for having a
> queue here because to send events asynchronously to passive node. Here we
> are going to send events to passive node via TCP. (This we need to decide)
> Active node will persist the state periodically to the database.
>
> When active node goes down via cluster coordination passive node will
> become active. When active it will get the state from database(to sync the
> state) with latest event timestamp , and filter events from queue 2 (in
> order to stop processing already processed events) and send them out. Then
> open the ports in the newly active node and start receiving events from
> sources. In this architecture also at least one processing will be done.
>
> Nisala please add anything I missed.
>
> Regards,
> Damith
>
>
>
>
>
>
> --
> Senior Software Engineer
> WSO2 Inc.; http://wso2.com
> 
> lean.enterprise.middleware
>
> mobile: *+94728671315*
>
>

-- 
*Anoukh Jayawardena*
*Software Engineer*

*WSO2 Lanka (Private) Limited: http://wso2.com
lean.enterprise.middle-ware*


*phone: (+94) 77 99 28932*

___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

[Architecture] Stream Processor HA improved archectecture

[Damith Wickramasinghe]

Hi all,

We are in the process of refactor/improve the existing HA architecture due
to various concerns found.

Below is the high level design came up with. We will provide more in-depth
details as the implementation carries on.


​
​
As per above at a given point of time there will only be a one active node.
Passive node will not consume any events. Electing active node will work as
per current architecture via cluster coordination.
A thread will work in the active node to put the data into a queue(queue1)
and same thread will then publish the events to outside.Reason for having a
queue here because to send events asynchronously to passive node. Here we
are going to send events to passive node via TCP. (This we need to decide)
Active node will persist the state periodically to the database.

When active node goes down via cluster coordination passive node will
become active. When active it will get the state from database(to sync the
state) with latest event timestamp , and filter events from queue 2 (in
order to stop processing already processed events) and send them out. Then
open the ports in the newly active node and start receiving events from
sources. In this architecture also at least one processing will be done.

Nisala please add anything I missed.

Regards,
Damith






-- 
Senior Software Engineer
WSO2 Inc.; http://wso2.com

lean.enterprise.middleware

mobile: *+94728671315*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture