Hi,
As we discussed in the meeting today[1] (19/03/2018), I modified the
summary log as follows.
{"Bulk User Import Operation Performed by":"admin","User
Store":"PRIMARY","Duplicate Users":{"Duplicate User Count":8,"User
Names":[{"Name":"name1"},{"Name":"name2"},{"Name":"name74"},
{"Name":"name3"},{"Name":"name3"},{"Name":"name"},{"
Name":"name83"},{"Name":"name5"}]},"Failed Users":{"Failed User
Count":2,"Failed Users List":[{"Name":"namsdsa","Cause":"Invalid claim uri
has been provided: http://wso2.org/claims/ctry"},
{"Name":"name2ssss","Cause":"Invalid claim uri has been provided:
http://wso2.org/claims/emaiaddress"}]}}
And also, we discussed to log the bulk user import summary to the audit
logs in the following format.
Initiator : admin@carbon.super | Action : Add Role | Target : admin | Data
: {} | Result
The data section will contain the importing user list. As in the
documentation, we support importing a maximum of 500,000 users at a time.
So, considering the worse case scenario, if we log these users as well, it
will eat up the storage very quickly and cause in threat conditions.
So IMO, we do not need to log users that are being imported. Also with the
Megala's feature [2], as the information is also being logged, I think it's
enough if we only log the result of the operation with Initiator, Action
and the Target values.
WDYT?
[1] [IAM] [Discussion] Bulk User Import Improvements
[2] Discussion on Improving Audit logs Related with User Management
Thanks and Regards,
Menaka
On Tue, Mar 13, 2018 at 12:45 PM, Dimuthu Leelarathne <dimut...@wso2.com>
wrote:
>
>
> On Tue, Mar 13, 2018 at 11:47 AM, Menaka Jayawardena <men...@wso2.com>
> wrote:
>
>> Hi,
>>
>> @Denuwanthi: Yes. It can be done. Please find the summery below.
>>
>> SUMMERY :
>> Bulk User Import Operation Performed by: admin
>> User Store : PRIMARY
>> Duplicate user count : 8
>> .
>>
>>
>> *UI Message Modification.*
>>
>> Currently, if an error occurred in the process of performing the Bulk
>> User Import Operation, the following Error message will be shown.
>>
>> *Error occurs while importing usernames. All usernames were not imported.
>> Last error was : Invalid claim uri has been provided:
>> http://wso2.org/claims/emaiaddress <http://wso2.org/claims/emaiaddress>*
>>
>> But there are multiple errors (Duplicate user etc). In this case, I think
>> it's better if we show a more generic error with a brief summery and direct
>> them to view the log file for more information.
>>
>> For an example:
>> Bulk User Import Completed with Errors.
>> Success user count: x Duplicate user count: y Failed user count: z
>> Please check the user import log for more detailed information.
>>
>
> +1
>
> And in the detail log we can log errors and duplicates.
>
> thanks,
> Dimuthu
>
>
>
>>
>> Any ideas, suggestions are highly appreciated.
>>
>> Thanks and Regards,
>> Menaka
>>
>> On Tue, Mar 13, 2018 at 9:24 AM, Denuwanthi De Silva <denuwan...@wso2.com
>> > wrote:
>>
>>>
>>>
>>> On Mon, Mar 12, 2018 at 4:29 PM, Menaka Jayawardena <men...@wso2.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Here is an experimental user import summery.
>>>>
>>>> SUMMERY :
>>>> Bulk User Import Operation Performed by: admin
>>>> Duplicate user count: 8
>>>> Duplicate Users :
>>>> name1, name2, name74, name3, name3, name, name83, name5,
>>>>
>>>> Failed User Count: 2 Failed Users:
>>>> Name : namsdsa
>>>> Cause : Invalid claim uri has been provided:
>>>> http://wso2.org/claims/ctry
>>>> Name : name2ssss
>>>> Cause : Invalid claim uri has been provided:
>>>> http://wso2.org/claims/emaiaddress
>>>>
>>>
>>> Hi Menaka,
>>>
>>> Is it possible to print the user domain in the summary as well? Then the
>>> information of the userstore the users were imported will be available as
>>> well.
>>>
>>> Thanks,
>>>
>>>>
>>>>
>>>> The cause string is the standard error which comes from the exception.
>>>> Do we need to print the stack trace here?
>>>>
>>>> Also, there are two BulkUserImport classes (CSVUserBulkImport[1] and
>>>> ExcelUserBulkImport[2]) and also an unused interface [3] (The classes [1]
>>>> and [2] are concreet classes).
>>>>
>>>> @IAM Team: Is there any particular reason why it kept like this?
>>>>
>>>> IMO in this implementation, we could use it to avoid code and method
>>>> duplication. (By making it an Abstract class)
>>>>
>>>> [1] https://github.com/wso2/carbon-identity-framework/blob/maste
>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java
>>>> /org/wso2/carbon/user/mgt/bulkimport/CSVUserBulkImport.java
>>>> [2] https://github.com/wso2/carbon-identity-framework/blob/maste
>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java
>>>> /org/wso2/carbon/user/mgt/bulkimport/ExcelUserBulkImport.java
>>>> [3] https://github.com/wso2/carbon-identity-framework/blob/maste
>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java
>>>> /org/wso2/carbon/user/mgt/bulkimport/UserBulkImport.java
>>>>
>>>> Thanks and Regards,
>>>> Menaka
>>>>
>>>>
>>>> On Mon, Mar 12, 2018 at 2:14 PM, Menaka Jayawardena <men...@wso2.com>
>>>> wrote:
>>>>
>>>>> [- strategy +Architecture]
>>>>>
>>>>>
>>>>> On Mon, Mar 12, 2018 at 12:21 PM, Menaka Jayawardena <men...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Dimuthu,
>>>>>>
>>>>>> Are you going to add this log appender by default to the
>>>>>>> configuration?
>>>>>>>
>>>>>> We can add the log appender by default and keep it commented. So when
>>>>>> the user enables the Bulk User import, he also can enable the log
>>>>>> appender
>>>>>> as well.
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 12, 2018 at 12:07 PM, Dimuthu Leelarathne <
>>>>>> dimut...@wso2.com> wrote:
>>>>>>
>>>>>>> Hi Menaka,
>>>>>>>
>>>>>>> Are you going to add this log appender by default to the
>>>>>>> configuration?
>>>>>>>
>>>>>>> thanks,
>>>>>>> Dimuthu
>>>>>>>
>>>>>>> On Mon, Mar 12, 2018 at 11:48 AM, Dakshika Jayathilaka <
>>>>>>> daksh...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Ruwan,
>>>>>>>>
>>>>>>>> Do we need to log each success? IMO admin will more interest on
>>>>>>>> failures or duplicates. IMHO we can add detail log on failures and
>>>>>>>> duplicates and then log the summary which includes the success count.
>>>>>>>>
>>>>>>>> WDYT?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> *Dakshika Jayathilaka*
>>>>>>>> PMC Member & Committer of Apache Stratos
>>>>>>>> Associate Technical Lead
>>>>>>>> WSO2, Inc.
>>>>>>>> lean.enterprise.middleware
>>>>>>>> 0771100911 <077%20110%200911>
>>>>>>>>
>>>>>>>> On Mon, Mar 12, 2018 at 11:35 AM, Ruwan Abeykoon <ruw...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Menaka,
>>>>>>>>> This is nice feature.
>>>>>>>>> I would suggest adding one line per each user, before adding, and
>>>>>>>>> one line for each success, failure(with reason).
>>>>>>>>> Also add a line who performs this operation. Any trackable
>>>>>>>>> information of the request for audit purposes.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Ruwan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Mar 12, 2018 at 11:21 AM, Menaka Jayawardena <
>>>>>>>>> men...@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Currently, when performing bulk user import operation in Identity
>>>>>>>>>> Server, users face following issues.
>>>>>>>>>>
>>>>>>>>>> 1. To check import failed users, need to filter the carbon log
>>>>>>>>>> file.
>>>>>>>>>> 2. In UI, it shows only the last error that occurred when
>>>>>>>>>> importing users.
>>>>>>>>>>
>>>>>>>>>> *Requirement Description.*
>>>>>>>>>> There should be a user friendly way to view the import failed
>>>>>>>>>> users.
>>>>>>>>>>
>>>>>>>>>> As a solution for this, we will provide a new log appender which
>>>>>>>>>> will log the messages to a separate log file specific for bulk user
>>>>>>>>>> import.
>>>>>>>>>> This will help users to easily view the status of the imported users
>>>>>>>>>> and
>>>>>>>>>> all the error logs.
>>>>>>>>>>
>>>>>>>>>> Also currently, as the operation summery, we only have
>>>>>>>>>>
>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount +
>>>>>>>>>> ", Duplicate count: " + duplicateCount
>>>>>>>>>>
>>>>>>>>>> Instead, it would be much effective if we could list the failed
>>>>>>>>>> and duplicate user names as well.
>>>>>>>>>>
>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount +
>>>>>>>>>> ", Duplicate count: " + duplicateCount
>>>>>>>>>> "Failed Users : " + [Failed Users List] + "Duplicate Users : " +
>>>>>>>>>> [Duplicate Users List]
>>>>>>>>>>
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>> Thanks and Regards,
>>>>>>>>>> Menaka
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Menaka Jayawardena*
>>>>>>>>>> *Software Engineer - WSO2 Inc*
>>>>>>>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>>>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>>>>>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>>>>>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>>>>>>>> <https://menakamadushanka.wordpress.com/>*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> *Ruwan Abeykoon*
>>>>>>>>> *Associate Director/Architect**,*
>>>>>>>>> *WSO2, Inc. http://wso2.com <https://wso2.com/signature> *
>>>>>>>>> *lean.enterprise.middleware.*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dimuthu Leelarathne
>>>>>>> Director, Rapid Response Team
>>>>>>>
>>>>>>> WSO2, Inc. (http://wso2.com)
>>>>>>> email: dimut...@wso2.com
>>>>>>> Mobile: +94773661935 <+94%2077%20366%201935>
>>>>>>> Blog: http://muthulee.blogspot.com
>>>>>>>
>>>>>>> Lean . Enterprise . Middleware
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Menaka Jayawardena*
>>>>>> *Software Engineer - WSO2 Inc*
>>>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>>>> <https://menakamadushanka.wordpress.com/>*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Menaka Jayawardena*
>>>>> *Software Engineer - WSO2 Inc*
>>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>>> <https://menakamadushanka.wordpress.com/>*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Menaka Jayawardena*
>>>> *Software Engineer - WSO2 Inc*
>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>> <https://menakamadushanka.wordpress.com/>*
>>>>
>>>>
>>>
>>>
>>> --
>>> Denuwanthi De Silva
>>> Senior Software Engineer;
>>> WSO2 Inc.; http://wso2.com,
>>> Email: denuwan...@wso2.com
>>> Blog: https://denuwanthi.wordpress.com/
>>>
>>
>>
>>
>> --
>> *Menaka Jayawardena*
>> *Software Engineer - WSO2 Inc*
>> *Tel : 071 350 5470 <071%20350%205470>*
>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>> <https://lk.linkedin.com/in/menakajayawardena>*
>> *Blog: https://menakamadushanka.wordpress.com/
>> <https://menakamadushanka.wordpress.com/>*
>>
>>
>
>
> --
> Dimuthu Leelarathne
> Director, Rapid Response Team
>
> WSO2, Inc. (http://wso2.com)
> email: dimut...@wso2.com
> Mobile: +94773661935 <+94%2077%20366%201935>
> Blog: http://muthulee.blogspot.com
>
> Lean . Enterprise . Middleware
>
--
*Menaka Jayawardena*
Software Engineer
WSO2 Inc.
Phone : +94 71 350 5470
LinkedIn : https://lk.linkedin.com/in/menakajayawardena
Blog : https://menakamadushanka.wordpress.com/
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
