Re: [Architecture] [IS] User Challenge question Internationalization

[Farasath Ahamed]

Hi Kasun,

"isPromoteQuestion" property is no longer used in our current
implementation. Therefore, we can get rid of the property and straight away
persist the challenge question as a registry resource.

Thanks,

Farasath Ahamed
Software Engineer,
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware


Email: farasa...@wso2.com
Mobile: +94777603866
Blog: blog.farazath.com
Twitter: @farazath619 

On Sat, Jun 4, 2016 at 12:26 AM, Kasun Bandara  wrote:

> Hi Farasath,
>
> +1 for storing the content of the questions as the registry resource
> values. But how are you planning to store the boolean values such as "
> *isPromoteQuestion*" ?. I think it's better to keep that sort of data as
> a property value, so the retrieval process will be easy. WDYT ?
>
> Thanks,
> Kasun.
>
> On Sat, Jun 4, 2016 at 12:16 AM, Farasath Ahamed 
> wrote:
>
>> Hi,
>>
>> In the current implementation, challenge questions are persisted to the
>> registry as registry resource properties as shown below.
>>
>>
>> I had a look at the discussion[1] on how persisting email templates for
>> different locale should be done. I am planning to follow a similar approach
>> in storing challenge questions for different locale. The idea explained in
>> [1] is to store the template/question as a registry resource rather than as
>> a registry property. So I will be following a structure similar to shown
>> below to store the challenge questions.
>>
>>
>> ​
>> So each question will be stored following a 
>> */system/config/challenge-questions///
>> *convention as shown above. The first step is to reuse most of the email
>> templates registry persistence code and make it generic (provide an
>> interface) to retrieve any registry resource(eg: email templates, challenge
>> questions, ) based on locale. The next step is to abstract the retrieval of
>> resources logic to support retrieval of resources from a DB, API etc.
>>
>>
>> [1] http://mail.wso2.org/mailarchive/architecture/2015-May/020188.html
>>
>> Thanks,
>> Farasath Ahamed
>> Software Engineer,
>> WSO2 Inc.; http://wso2.com
>> lean.enterprise.middleware
>>
>
>
>
> --
> Kasun Bandara
> *Software Engineer*
> Mobile : +94 (0) 718 338 360
> <%2B94%20%280%29%20773%20451194>
> kas...@wso2.com 
>
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Data Bridge Agent Publisher for C5 products

[Sajith Ravindra]

>
>
> We don't need to change from Thrift to HTTP, rather we need to have an
> intermediate (high-level) layer which does not depend on underneath
> transport protocol and relevant transport can be configured using a config
> file (Like we have stream terminology for CEP publisher) .
>

Ideally, we should have well a defined interfaces that components can use
to publish/receive data regardless of the underneath transport. However,
coming up with a unified interface for many transports will not be a
trivial task as we need to present semantics of different transports
through a single interface.

Thanks
*,Sajith Ravindra*
Senior Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware

mobile: +94 77 2273550
blog: http://sajithr.blogspot.com/


On Sat, Jun 4, 2016 at 3:06 AM, Mohanadarshan Vivekanandalingam <
mo...@wso2.com> wrote:

>
>
> On Thu, Jun 2, 2016 at 9:43 PM, Isuru Perera  wrote:
>
>> Hi Suho,
>>
>> In Metrics, I have written the DAS Reporter using the data publisher. Do
>> you think I should change the implementation to use an HTTP client?
>>
>
> We don't need to change from Thrift to HTTP, rather we need to have an
> intermediate (high-level) layer which does not depend on underneath
> transport protocol and relevant transport can be configured using a config
> file (Like we have stream terminology for CEP publisher) .
>
>
>>
>> We also need to release Metrics soon. So, if we are going to change the
>> implementation, we need to decide soon. However I think it'll have an
>> impact on the release schedule for Metrics.
>>
>> In Metrics, the DAS reporter is used to store the historical metrics data
>> in DAS. The reporter will send events periodically (1 minute, by default).
>> So, do you think we will encounter any issue with current Data Publisher.
>>
>
> Theoretically, I don't think there are any issues but we need tryout..
>
>
>>
>> Please let me know.
>>
>> Thanks!
>>
>> On Thu, Jun 2, 2016 at 3:18 PM, Sriskandarajah Suhothayan 
>> wrote:
>>
>>> Are we going to use data bridge in C5 ?
>>> C5 have Netty based transports, cant we use one of them to publish
>>> events to DAS, since DAS have the capability to receiving from any
>>> transport protocol via extensions this will not be a problem.
>>>
>>> In my opinion we should not be depending on data publisher as it have
>>> several issues like events can get out of ordered, dropped and its not
>>> reliable. We should have a publishing framework which is independent of the
>>> transport, so users can pic Thrift, HTTP or AMQP based on their use cases.
>>>
>>> WDYT?
>>>
>>> Suho
>>>
>>> On Thu, Jun 2, 2016 at 2:35 PM, Kishanthan Thangarajah <
>>> kishant...@wso2.com> wrote:
>>>
 A separate point to note.

 Thinking along the AS 6.0 and C5 aspect, what we need is a library,
 where we could use that in both OSGi env and non-OSGi environments. It
 should not have any direct dependency on carbon API's. Currently, with AS
 6.0, we are using the data publisher from C4 without any code changes and
 removing unwanted dependencies. If we planning to write this again, we
 should come up with the minimum library that could be used in a OSGi and
 non-OSGi env.


 On Thu, Jun 2, 2016 at 1:40 PM, Isuru Perera  wrote:

> Hi,
>
> On Thu, Jun 2, 2016 at 1:10 PM, Sinthuja Ragendran 
> wrote:
>
>> Hi IsuruP,
>>
>> Please find the comments inline.
>>
>> On Thu, Jun 2, 2016 at 12:18 PM, Isuru Perera 
>> wrote:
>>
>>> Hi,
>>>
>>> This is regarding $subject and the main problem we have is that
>>> there is no Carbon 5 compatible feature for data bridge agent.
>>>
>>
>> Data bridge agent publisher is not depends on carbon, and it has some
>> dependencies for carbon utils, and carbon base, which we can eliminate by
>> passing proper configurations in data-agent-config.xml.
>>
> Yes. This is what should be done.
>
> I think we should avoid carbon dependencies in data publisher. We can
> have some Carbon specific component to initialize data publisher in Carbon
> (OSGi) environment.
>
>> In that case, what do you mean by it's not compatible by Carbon 5 as
>> it's anyhow doesn't depend on carbon features?
>>
> As I mentioned earlier, the publisher has Carbon 4.x dependencies. So,
> we need to workaround problems like NoClassDefFoundError for
> CarbonUtils etc.
>
>>
>>
>>>
>>> Since Data Bridge Agent is already an OSGi bundle, we can use it
>>> within C5 products. But we have to include it with some feature.
>>>
>>> For example, Carbon Metrics needs to publish events to DAS. So, is
>>> it okay if I keep data bridge agent in Metrics feature?
>>>
>>
>> No, I don't think that is a good option. Because the publisher is a
>> generic feature, and it doesn't have any relation ship t

Re: [Architecture] [Dev] Common configuration for publishing events from carbon servers to DAS/CEP

[Buddhima Wijeweera]

Hi All,

We are in the process of introducing this to ESB. I have successfully added
relevant features and data publishing is working fine with ESB.
But I can see the following logs in a fresh ESB pack first start (those are
not appearing when restarting again).

[2016-06-04 21:57:10,253]  INFO - EventStreamDeployer Stream definition is
deployed successfully  : esb-config-entry-stream:1.0.0
[2016-06-04 21:57:10,257]  INFO - EventStreamDeployer Stream definition is
deployed successfully  : esb-flow-entry-stream:1.0.0
*[2016-06-04 21:57:10,261]  INFO -
EventPublisherConfigurationFilesystemInvoker Event Publisher configuration
deleted from the file system : MessageFlowConfigurationPublisher.xml*
*[2016-06-04 21:57:10,261]  INFO - EventPublisherDeployer Event Publisher
undeployed successfully : MessageFlowConfigurationPublisher.xml*
[2016-06-04 21:57:10,511]  INFO -
EventPublisherConfigurationFilesystemInvoker Event Publisher configuration
saved in the filesystem : MessageFlowConfigurationPublisher.xml
[2016-06-04 21:57:10,525]  INFO - EventJunction WSO2EventConsumer added to
the junction. Stream:esb-config-entry-stream:1.0.0
[2016-06-04 21:57:10,527]  INFO - EventPublisherDeployer Event Publisher
configuration successfully deployed and in active state :
MessageFlowConfigurationPublisher
*[2016-06-04 21:57:10,528]  INFO -
EventPublisherConfigurationFilesystemInvoker Event Publisher configuration
deleted from the file system : MessageFlowStatisticsPublisher.xml*
*[2016-06-04 21:57:10,528]  INFO - EventPublisherDeployer Event Publisher
undeployed successfully : MessageFlowStatisticsPublisher.xml*
[2016-06-04 21:57:10,532]  INFO -
EventPublisherConfigurationFilesystemInvoker Event Publisher configuration
saved in the filesystem : MessageFlowStatisticsPublisher.xml
[2016-06-04 21:57:10,542]  INFO - EventJunction WSO2EventConsumer added to
the junction. Stream:esb-flow-entry-stream:1.0.0
[2016-06-04 21:57:10,543]  INFO - EventPublisherDeployer Event Publisher
configuration successfully deployed and in active state :
MessageFlowStatisticsPublisher


In this case I have 2 publishers called; MessageFlowConfigurationPublisher
& MessageFlowStatisticsPublisher. And seems they get deleted and redeployed
for some reason.

Is this an expected behavior (for analytics-common : 5.0.12-beta2) ?

Thanks,

On Fri, May 20, 2016 at 3:47 PM, Niranjan Karunanandham 
wrote:

> Hi Pulasthi,
>
> Did you add the feature to the p2-repo generation section in the pom? Can
> you check if the feature is there in your p2-repo?
>
> Regards,
> Nira
>
> On Fri, May 20, 2016 at 2:46 PM, Pulasthi Mahawithana 
> wrote:
>
>> Hi Niranjan,
>>
>> It was failing with the p2 repo that is created in p2-profile-gen.
>> Looking at the pom at [1], this feature only imports the org.wso2.carbon.core
>> which is available in IS but as a different patch release version. Does the
>> difference in the patch release version matter here? Or is it due to
>> something else?
>>
>> [1]
>> https://github.com/wso2/carbon-analytics-common/blob/v5.0.11/features/event-publisher/org.wso2.carbon.event.publisher.aggregate.feature/pom.xml
>>
>> On Fri, May 20, 2016 at 2:10 PM, Niranjan Karunanandham <
>> niran...@wso2.com> wrote:
>>
>>> Hi Pulasthi,
>>>
>>> When building the product, are you referring to the p2-repo of the
>>> locally built carbon-feature-repository or to a p2-repo that you create in
>>> p2-profile-gen. If so this issue can happen if the feature that you are
>>> installing has import features which are not there in your p2-repo.
>>>
>>> @Malith: As per the previous mail, can you close this PR since it need
>>> to come from the PR from the product team which will be using the feature.
>>>
>>> Regards,
>>> Nira
>>>
>>> Regards,
>>> Nira
>>>
>>> On Fri, May 20, 2016 at 12:47 PM, Pulasthi Mahawithana <
>>> pulast...@wso2.com> wrote:
>>>
 I tried to add this feature to product-is 's p2-profile-gen. But it
 fails saying,

 Installation failed.
 The installable unit
 org.wso2.carbon.event.publisher.aggregate.feature.group/5.0.11 has not been
 found.

 However I can install it from feature management UI when I merged and
 built the carbon feature repository locally with the PR at [1] and point
 that as a local repository. Any possible causes for this?

 [1] https://github.com/wso2/carbon-feature-repository/pull/46

 On Wed, Apr 27, 2016 at 9:08 AM, Niranjan Karunanandham <
 niran...@wso2.com> wrote:

> Hi Malith,
>
> Since this feature will be included in the next ESB release, IMO it
> would be better to close the current PR and have it included in the same 
> PR
> after the product release.
>
> Regards,
> Nira
>
> On Mon, Apr 25, 2016 at 6:05 PM, Malith Dhanushka 
> wrote:
>
>> Hi Niranjan,
>>
>> Correction on my previous reply. we have to ship this feature by
>> default with ESB and APIM. So this needs to be released with the 
>> i

Re: [Architecture] [IS] User Challenge question Internationalization

[Isura Karunaratne]

Hi Farasath,

Currently, we are using "http://wso2.org/claims/challengeQuestionUris "
claim to identity user answered challenge question set Ids. Following is a
sample value for that claim.

*http://wso2.org/claims/challengeQuestion1!http://wso2.org/claims/challengeQuestion2
.*


user answered challenge questions are stored in different claims as follows.

Claim URI : http://wso2.org/claims/challengeQuestion1
Value: City where you were born
?!CCKWN1bOcOwXuJ3zPyThGZu3GZouQYGfw1tHWjUJdi4=

Claim URI : http://wso2.org/claims/challengeQuestion2
Value: What is your first pet's name
?!CCKWN1bOcOwXuJ3zPyThGZu3GZouQYGfw1tHWjUJdi4=


So, actual challenge question and encripted answer is stored as claim.
Since user may change his locale runtime, I think in new implementation we
have to store some question id rather than questoin text in claim.
Otherwise security question will display in previous locale.

Other improvement that we can do here is to use json string to store
question set ids and claim values rather than seperating from a seperator
("!")

Thanks
Isura.



On Sat, Jun 4, 2016 at 1:43 PM, Farasath Ahamed  wrote:

> Hi Kasun,
>
> "isPromoteQuestion" property is no longer used in our current
> implementation. Therefore, we can get rid of the property and straight away
> persist the challenge question as a registry resource.
>
> Thanks,
>
> Farasath Ahamed
> Software Engineer,
> WSO2 Inc.; http://wso2.com
> lean.enterprise.middleware
>
>
> Email: farasa...@wso2.com
> Mobile: +94777603866
> Blog: blog.farazath.com
> Twitter: @farazath619 
>
> On Sat, Jun 4, 2016 at 12:26 AM, Kasun Bandara  wrote:
>
>> Hi Farasath,
>>
>> +1 for storing the content of the questions as the registry resource
>> values. But how are you planning to store the boolean values such as "
>> *isPromoteQuestion*" ?. I think it's better to keep that sort of data as
>> a property value, so the retrieval process will be easy. WDYT ?
>>
>> Thanks,
>> Kasun.
>>
>> On Sat, Jun 4, 2016 at 12:16 AM, Farasath Ahamed 
>> wrote:
>>
>>> Hi,
>>>
>>> In the current implementation, challenge questions are persisted to the
>>> registry as registry resource properties as shown below.
>>>
>>>
>>> I had a look at the discussion[1] on how persisting email templates for
>>> different locale should be done. I am planning to follow a similar approach
>>> in storing challenge questions for different locale. The idea explained in
>>> [1] is to store the template/question as a registry resource rather than as
>>> a registry property. So I will be following a structure similar to shown
>>> below to store the challenge questions.
>>>
>>>
>>> ​
>>> So each question will be stored following a 
>>> */system/config/challenge-questions///
>>> *convention as shown above. The first step is to reuse most of
>>> the email templates registry persistence code and make it generic (provide
>>> an interface) to retrieve any registry resource(eg: email templates,
>>> challenge questions, ) based on locale. The next step is to abstract the
>>> retrieval of resources logic to support retrieval of resources from a DB,
>>> API etc.
>>>
>>>
>>> [1] http://mail.wso2.org/mailarchive/architecture/2015-May/020188.html
>>>
>>> Thanks,
>>> Farasath Ahamed
>>> Software Engineer,
>>> WSO2 Inc.; http://wso2.com
>>> lean.enterprise.middleware
>>>
>>
>>
>>
>> --
>> Kasun Bandara
>> *Software Engineer*
>> Mobile : +94 (0) 718 338 360
>> <%2B94%20%280%29%20773%20451194>
>> kas...@wso2.com 
>>
>
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
Isura Dilhara Karunaratne
Senior Software Engineer

Mob +94 772 254 810
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [IS] Supporting user information recovery scenarios in IS user portal

[Isura Karunaratne]

Hi Malithi,

These steps seem to be ok. However we are going to move from KAPCHA to
google reCaptcha in IS 5.3.0 and this captcha implementation is decoupled
with each flow you mentioned. So, in new implementation captcha validation
will happen before invoking to the backend APIs.

In "security questions based password recovery flow", we are going to add a
new parameter to parameterize the minimum number of questions that user
should be answered to recover the password. In the previous releases (IS 5,
IS5.1.0) this is an application side configuration and it was not a server
side configuration. With IS 5.3.0 onwards tenant admin can define the
minimum number of questions that user should be answered to recover the
password.

Thanks
Isura



On Fri, Apr 29, 2016 at 11:40 AM, Malithi Edirisinghe 
wrote:

> Hi All,
>
> We are working on $subject.
>
> So with this, we will be supporting below options in the login page of the
> end user dashboard in IS.
>
>1. Forgot Password (Password Recovery)
>2. Forgot Username (Username Recovery)
>3. Create an account (Self Signup)
>
> Below I have listed the user stories that we identified, to support each
> option above.
>
> *Password Recovery*
>
> With the existing information recovery service implementation we can
> support two options.
> Visibility of each option would be decided on the availability of required
> configurations and required user information.
>
>- Password recovery with an email notification (Email transport
>configuration are required and the user should have a valid email address)
>- Password recovery with security questions (User should have
>preconfigured security questions)
>
> *User story for password recovery with email notification*
>
>1. User views a generated captcha
>2. User enters the captcha answer and submit
>3. Server verifies the captcha answer and sends an email notification
>with the confirmation code
>4. User clicks on the received confirmation url
>5. User views a generated captcha
>6. User enters the captcha answer and submit
>7. Server verifies the captcha answer and let the user to reset
>password
>8. User resets the password
>9. Upon successful password reset user is notified and redirected to
>login page
>
>
> *User story for password recovery with security questions*
>
>1. User views a generated captcha
>2. User enters the captcha answer and submit
>3. User views the security questions (Based on the server
>configuration user may have to answer preconfigured security questions from
>each question set one by one or all at once)
>4. User answers the questions as requested by the server.
>5. Server verifies the answers and let the user to reset password
>6. User resets the password
>7. Upon successful password reset user is notified and redirected to
>login page
>
> *Username Recovery*
>
> Username recovery is supported only with an email notification.
> Thus, the visibility of this option would be decided on required email
> transport configurations
>
> *User story for username recovery with email notification*
>
>1. User views a page with a set of requested claims and captcha
>2. User enters the requested claim values and captcha answer
>3. Server verifies the captcha answer along with the claims
>4. Server sends an email notification with the username, notifies the
>user on the status and redirects to the login page
>
> *Self Registration*
>
> Ideally we have two services that supports self registration. One verifies
> the user account with an email notification and the other just registers
> the user.
> So if email transport configurations are done self registration will
> always verify the user with an email notification, other wise the user will
> be allowed to simply register.
>
> *User story for self registration with email verification*
>
>1. User views the registration page
>2. User enters the requested information and register
>3. Server sends an account confirmation to the user via email
>4. User clicks on the received confirmation url
>5. User views a generated captcha
>6. Server verifies the captcha answer
>7. User views security question configuration page (User should be
>allowed to skip this as well. Based on the preconfigured security questions
>it's decided if the user is allowed to recover password with security
>questions or not)
>8. User may skip or configure as desired
>9. Upon successful registration user is redirected to login page
>
> *User story for self registration without email verification*
>
>1. User views the registration page
>2. User enters the requested information and register
>3. User views security question configuration page (User should be
>allowed to skip this as well. Based on the preconfigured security questions
>it's decided if the user is allowed to recover password with security
>

Re: [Architecture] [CEP] Extending the Regression Function to support time window

[Seshika Fernando]

Hi,
The length ceiling is necessary along with the duration parameter. The
reason the batch size was originally implemented was to optimize
performance when large datasets are considered for regression. We need to
be able to give an upper bound. So for example in this case, if user uses a
large duration (24 hours)and there are millions of events, then if we put a
batch size of 1 million it will consider the last 1 million events in the
last 24 hours. Which is a valid use case.

For this reason, the ability to specify both duration and batch size is
important.

Seshi
On 2 Jun 2016 14:45, "Charini Nanayakkara"  wrote:

> Noted with thanks. Will proceed with the implementation likewise.
>
> Charini
>
> On Thu, Jun 2, 2016 at 2:28 PM, Sriskandarajah Suhothayan 
> wrote:
>
>> I think having batchSize & duration will be good as this will limit the
>> number of events considered, this can help to improve performance as well.
>>
>> Suho
>>
>> On Thu, Jun 2, 2016 at 1:59 PM, Charini Nanayakkara 
>> wrote:
>>
>>> Hi Tishan,
>>>
>>> For my requirement, having time window alone is adequate. So your point
>>> might be valid. However I'm concerned of the re-usability of the extension.
>>>
>>> @Srinath, WDYT? Which would be the better option? Having a single
>>> implementation or two different ones?
>>>
>>> Thanks
>>>
>>> On Thu, Jun 2, 2016 at 1:48 PM, Tishan Dahanayakage 
>>> wrote:
>>>
 Charini,

 My knowledge on the on this domain is sparse. Hence I do not know
 whether a scenario where time AND length is a valid business case. If it is
 a valid business case +1 for the design including both parameters in same
 implementation.

 Thanks
 /Tishan

 On Thu, Jun 2, 2016 at 12:54 PM, Charini Nanayakkara >>> > wrote:

> Hi Tishan,
>
> Yes. Assuming batch size is 5 and time window is 20 mins, only 5 out
> of 10 events which arrive within last 5 mins would be processed due to
> batch size constraint (even though all events must be processed if time
> alone was considered). Having separate implementations would work on the
> majority of the scenarios, since only time OR length is usually applicable
> but not both. However, having two implementations would cause trouble in
> the situations where both the time factor and length are important
> (equivalent to AND operation on the two constraints). If our requirement 
> is
> to have only one of the two constraints, we can use a very large value for
> the other parameter (i.e. if we only need to limit number of events based
> on time = 1 sec constraint, we can specify 1,000,000 for batch size
> assuming we have prior knowledge that 1,000,000 events would never arrive
> within 1 sec). IMHO neither of the two options (separate or single
> implementation) are perfect for every scenario. However having a single
> implementation would help address more cases as I understand. What's your
> opinion on this?
>
> Thanks
>
> On Thu, Jun 2, 2016 at 10:14 AM, Charini Nanayakkara <
> chari...@wso2.com> wrote:
>
>> Hi All,
>>
>> I have planned to extend the existent Regression Function by adding
>> time parameter. Regression is a functionality available for the Siddhi
>> stream processor extension known as timeseries. In the current
>> implementation, the regression function consumes two or more parameters 
>> and
>> performs regression as follows.
>>
>> The mandatory parameters to be given are the dependent attribute Y
>> and the independent attribute(s) X1, X2,Xn. For performing simple
>> linear regression, merely one independent attribute would be given. Two 
>> or
>> more independent attributes are consumed for executing multiple linear
>> regression.
>>
>> timeseries:regress(Y, X1, X2..,Xn)
>>
>> The other three optional parameters to be specified are calculation
>> interval, batch size and confidence interval (ci). In the case where 
>> those
>> are not specified, the default values would be assumed.
>>
>> timeseries:regress(calcInterval, batchSize, ci, Y, X1, X2..,Xn)
>>
>> Batch size works as a length window in this implementation, which
>> allows one to restrict the number of events considered when executing
>> regression in real time. For example, if length is 5, only the latest 5
>> events (current event and the 4 events prior to it) would be used for
>> performing regression.
>>
>> *This suggested extension would allow the user to restrict the number
>> of events based on a time window as well, apart from constraining based 
>> on
>> length only. Therefore regression function would consume duration as an
>> additional parameter, subsequent to the completion of my task. *
>>
>> *timeseries:regress(calcInterval, duration, batchSize, ci, Y, X1,
>> X2..,Xn).*
>>
>>