Re: [Architecture] Better Understanding Linux Tools for performance Monitoring

[Isuru Perera]

Hi Srinath,

I created "cheat sheet" for using some of the popular Linux Performance
Observability Tools.

See:
https://medium.com/@chrishantha/linux-performance-observability-tools-19ae2328f87f

On Wed, Jul 22, 2015 at 3:10 PM, Isuru Perera  wrote:

> Hi,
>
> I wrote a blog post on generating Java CPU Flame Graphs with linux perf
> [1]. With these flame graphs, we can see system code paths in addition to
> java code paths.
>
> Thanks!
>
> Best Regards,
>
> [1] http://isuru-perera.blogspot.com/2015/07/java-cpu-flame-graphs.html
> 
>
> On Tue, Jun 9, 2015 at 10:49 AM, Srinath Perera  wrote:
>
>> thanks and ack!
>>
>> On Mon, Jun 1, 2015 at 5:32 PM, Isuru Perera  wrote:
>>
>>> Hi Srinath,
>>>
>>> Flame Graphs help us to identify hot methods from sampled stack traces.
>>> There are examples to generate Java Flame Graphs from jstack and Google's
>>> lightweight-java-profiler.
>>>
>>> Since we primarily use Java Flight Recorder in development, I wrote a
>>> simple Java program to read a Java Flight Recording and create a Flame
>>> Graph compatible output [1]. I also wrote a blog post [2].
>>>
>>> I hope this Java program will help us in our performance testing.
>>>
>>> I have started looking in to other tools and I will update this thread
>>> later.
>>>
>>> Thanks!
>>>
>>> Best Regards,
>>>
>>> [1] https://github.com/chrishantha/jfr-flame-graph
>>> [2] http://isuru-perera.blogspot.com/2015/05/flame-graphs-with-
>>> java-flight-recordings.html
>>>
>>> On Thu, Apr 9, 2015 at 8:29 AM, Srinath Perera  wrote:
>>>
 Hi Isuru,

 As per our chat yesterday, please look at the $subject. Idea is not to
 intergate those tools, but document and know them so we can use them
 internally for debugging etc.

 Lets start with Flame Graphs http://www.brendangregg.com/blog/2014-06-
 12/java-flame-graphs.html

 Others could be

 http://dtrace.org/blogs/brendan/2013/06/08/linux-
 performance-analysis-and-tools/
 http://www.brendangregg.com/blog/2015-03-03/performance-
 tuning-linux-instances-on-ec2.html

 Keep this thread updated.

 --Srinath

 --
 
 Blog: http://srinathsview.blogspot.com twitter:@srinath_perera
 Site: http://people.apache.org/~hemapani/
 Photos: http://www.flickr.com/photos/hemapani/
 Phone: 0772360902

>>>
>>>
>>>
>>> --
>>> Isuru Perera
>>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
>>> Lean . Enterprise . Middleware
>>>
>>> about.me/chrishantha
>>>
>>
>>
>>
>> --
>> 
>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera
>> Site: http://people.apache.org/~hemapani/
>> Photos: http://www.flickr.com/photos/hemapani/
>> Phone: 0772360902
>>
>
>
>
> --
> Isuru Perera
> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
> Lean . Enterprise . Middleware
>
> about.me/chrishantha
> Contact: +IsuruPereraWSO2 
>



-- 
Isuru Perera
Technical Lead | WSO2, Inc. | http://wso2.com/
Lean . Enterprise . Middleware

about.me/chrishantha
Contact: +IsuruPereraWSO2 
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [Feature] Storing the application certificate in the database.

[Inosh Perera]

Hi Rushmin,

The certificate should be stored in the database so that it is shared and a
restart is not needed.
+1 for the approach. There is a similar implementation[1] in the IoT server
to maintain device identity certificates of the devices for auto enrollment
flows. However, since the client we are using(device agents) are provided
by the IoT server, the in-build CA of the IoT server creates per device
certificates for each device, therefore, we are doing mutual TLS. However,
you may still be able to use some of the logic.

[1]. https://github.com/wso2/carbon-device-mgt/tree/master/
components/certificate-mgt

Regards,
Inosh

On Mon, Jan 8, 2018 at 9:39 PM, Shazni Nazeer  wrote:

> Yes. Seems both options are viable and has it own pros and cons. I'm +1
> for either option. Just that uploading is little more convenient at the
> time of adding it. But having text have its own reason to consider it.
>
> On Sun, Jan 7, 2018 at 11:22 PM, Rushmin Fernando 
> wrote:
>
>>
>>
>> On Mon, Jan 8, 2018 at 3:19 AM, Shazni Nazeer  wrote:
>>
>>> 1) The SP UI will have a new text area to enter the certificate in PEM
>>> format.
>>> Is there any specific reason to use text area here? In IDP UI, we
>>> have an option to upload the idp cert. IMO it is better to have that 
>>> option
>>> in SP UI as well for the UI consistance. '
>>>
>>
>>>
 It is bit easier for users to paste the content staight away rather
>>> than uploading files.
>>> +1 for making both UIs consistent.
>>
>>
>>> +! for certificate upload mechanism for consistency. I think uploading
>>> will be much easier than pasting the text.
>>>
>>
>>
>>
>> Thank you for the feedback Shazni. I agree that uploading might be eaiser
>> when you have the file in hand. But we have following cons as well given
>> that a PEM encoded ceritificate is a text content which is about 10 lines.
>>
>>
>> 1) Uploading a file needs a file name and the filename is an unwated data
>> in our case.
>>
>> 2) If you want to view the uploaded PEM file later, you have to download
>> it. (It is cleaner when you see the content in the UI)
>>
>>
>>
>>
>>>
>>>
>>> Also if the user can still add the certificate of the SP to the keystore
>>> and also from the UI, what would be the behavior when one of the
>>> certificate is invalid or expired and the other one is correct? Will there
>>> be any precedence, such as if it's in DB it will only check in DB or
>>> whether it checks in both DB and keystore?
>>>
>>> On Sat, Jan 6, 2018 at 6:37 AM, Godwin Shrimal  wrote:
>>>
 Thanks for the explanation Rushmin. Yeah, it makes sense. +1


 Thanks
 Godwin

 On Fri, Jan 5, 2018 at 3:30 PM, Rushmin Fernando 
 wrote:

> Hi Godwin,
>
> IMO certificate is a first class member of a service provider. So
> storing it as a field in SP_APP is cleaner.
>
> On the other hand, the datatype of a certificate doesn't really go
> with other metadata.
>
> In the best case, we have to alter the metadata table to hold a
> VARCHAR of like 1000 characters.
>
> Considering all of these facts, I thought of using the SP_APP table
> instead of SP_METADATA.
>
> What do you think?
>
> On Fri, Jan 5, 2018 at 12:56 PM, Godwin Shrimal 
> wrote:
>
>> Hi Rushmin,
>>
>> Any reason to use SP_APP table to persist the certificate? We have a
>> table called SP_METADATA to SP related metadata. I think we can use that
>> table without changing any DB Schema. WDYT?
>>
>>
>> Thanks
>> Godwin
>>
>> On Fri, Jan 5, 2018 at 1:33 PM, Rushmin Fernando 
>> wrote:
>>
>>>
>>>
>>> On Fri, Jan 5, 2018 at 11:55 AM, Isura Karunaratne 
>>> wrote:
>>>
 Hi Rushmin,

 On Fri, Jan 5, 2018 at 11:50 AM, Hasanthi Purnima Dissanayake <
 hasan...@wso2.com> wrote:

> Hi Rushmin,
>
> *How is this done now?*
>>
>> The application certificate should be imported to the keystore file
>> and the alias should be mentioned in the service provider so
>> that the service provider can validate the signature against the
>> certificate identified by that alias.
>>
>
> If we have the current option of  importing the certificate to the
> keystore, in JWT client authentication [1] we have to provide the
> certificate alias as the client id inorder to identify the 
> application. So
> with this implementation we don't need to enforce end users to do the 
> above
> as we can fetch the client_id directly from the db.
>
> +1 for the approach.
>
> [1] [IAM] JWT client authentication for OAuth 2.0 for IS 5.5.0
>
> Thanks,
>
> On Fri, Jan 5, 2018 at 11:31 AM, Rushmin Fernando <
> rush...@wso2.com> wrote:
>
>>

Re: [Architecture] [MB4] Authorization Model for Message Broker

[Hasitha Hiranya]

Hi Waruna,

Thank You. That explains.
Another side question is permission model for hierarchical topics. In a
way, as topic is a concept, we will have to authorise user when the queue
created on behalf of the subscriber binds to the exchange. I would suggest
following.

>> temp queue creating done for a topic subscription need no permission and
system will do that
>> when that queue binds to a topic check if user on subscriber channel has
permission to subscribe.

Thoughts?

Thanks


On Fri, Jan 12, 2018 at 5:22 PM, Waruna Jayaweera  wrote:

> Hi Hasitha,
>
> Please refer my comments inline.
>
> On Fri, Jan 12, 2018 at 9:56 AM, Hasitha Hiranya 
> wrote:
>
>> Hi Waruna,
>>
>> In MB 3.1.0 authentication model, we gave publish/subscribe permission
>> automatically to the user who created the queue.
>> Are we having same concept here as well?
>>
>
> Yes. User will be owner of that resource and he will have all permissions.
> We also give grant permission to queue owner so he will able to grant those
> actions to other users on his queue.
>
>> Topic is a concept and transport will have nothing like "create a topic".
>> It will be an internal queue creation. So it will not be applicable to
>> topic.
>> Are we planning to have a separate action called "create" queue and
>> define a permission other than publish/subscribe?
>>
>> Yes. I need to  finalize the queue actions and reply soon.
>
>> BTW, no:3 is bit confusing to me.
>>
>
> We may need to provide all permissions to given resource type ex. admin
> group can publish any queue. For that will use same permission structure
> with global level permission on resource type.(ex.Queue)
>
>>
>> Thanks
>>
>
> Thanks,
> Waruna
>
>>
>>
>> On Fri, Jan 12, 2018 at 9:10 AM, Pamod Sylvester  wrote:
>>
>>> Hi Waruna,
>>>
>>> In this case how will we handle messages which are in DLC ?
>>>
>>> Are we going to give the option to subscribe to DLC ? (currently we have
>>> restricted it). However, that will be a drawback given the queue owner will
>>> not have access to it's own messages which are in the DLC
>>>
>>> Thanks,
>>> Pamod
>>>
>>> On Fri, Jan 12, 2018 at 7:21 AM, Himasha Guruge 
>>> wrote:
>>>
 Hi Waruna,

 Have we decided which permissions will be allocated for a user  by
 default when creating a queue/topic? Are we going to consider ownership
 concept for this, that was discussed in C5 permission model? [1]


 [1] https://docs.google.com/document/d/1yosWL_kTxUWFukcoU7DtrtZd
 RuiK0ghySs96u4lfUHU/edit#heading=h.81aqdsft1abw

 Thanks,
 Himasha

 On Tue, Jan 9, 2018 at 12:07 AM, Waruna Jayaweera 
 wrote:

> Hi,
> Reattach the missing diagram .
>
> [image: Inline image 1]
>
> Thanks,
> Waruna
>
> On Tue, Jan 9, 2018 at 12:00 AM, Waruna Jayaweera 
> wrote:
>
>> Hi,
>>
>> Message broker requires authorization model to access control of
>> resources like Topics/Queues based on user groups . This is to provide 
>> the
>> initial design for $Subject.
>> Example use case would be as follows. We have three user groups (
>> roles) A , B  and manager and two topics T1 and T2. We need to restrict
>> users in group as below.
>>
>>1. T1 can be subscribed by only A and publish by only B
>>2. T2 can be subscribed by only B and publish by only A
>>3. Manager users can subscribe and publish to any topic but only
>>subscribe queue.
>>
>> Following entities can be identified.
>>
>> *User groups:*  A ,B and manager
>> *Resources *: T1 and T2
>> *Resource Groups *: Topic, Queue
>> *Actions*: subscribe, publish,view  etc.
>> *Permission*: resource+actions
>>
>> We can represent the permissions using binary form  mappings with
>> resource and user group. These permissions can be defined per resource or
>> globally as well.
>>
>> *Per Resource*
>>
>> Resource
>>
>> User Group
>>
>> Actions
>>
>> Permission
>>
>> publish
>>
>> subscribe
>>
>> T1
>>
>> A
>>
>> 0
>>
>> 1
>>
>> 01
>>
>> T2
>>
>> B
>>
>> 1
>>
>> 0
>>
>> 10
>>
>> *Global Permission*
>>
>> Resource Type
>>
>> User Group
>>
>> Actions
>>
>> Permission
>>
>> publish
>>
>> subscribe
>>
>> Topic
>>
>> admin
>>
>> 1
>>
>> 1
>>
>> 11
>>
>> Queue
>>
>> admin
>>
>> 1
>>
>> 0
>>
>> 10
>>
>>
>> Permission will be stored in the database similarly as of  [1].  
>> Following
>> figure shows the proposed implementation for $subject.
>>
>>
>>
>> Connection handler can fetch the mb resource permissions mappings
>> from database and user groups information from underlying user store
>> manager. Authorized users can add