Hi Denuwanthi, It's just a template of the audit log that is being used currently. For Bulk user import the action would be "*Bulk User Import*".
On Mon, Mar 19, 2018 at 7:02 PM, Denuwanthi De Silva <denuwan...@wso2.com> wrote: > > > On Mon, Mar 19, 2018 at 3:46 PM, Menaka Jayawardena <men...@wso2.com> > wrote: > >> Hi, >> >> As we discussed in the meeting today[1] (19/03/2018), I modified the >> summary log as follows. >> >> {"Bulk User Import Operation Performed by":"admin","User >> Store":"PRIMARY","Duplicate Users":{"Duplicate User Count":8,"User >> Names":[{"Name":"name1"},{"Name":"name2"},{"Name":"name74"}, >> {"Name":"name3"},{"Name":"name3"},{"Name":"name"},{"Name":"n >> ame83"},{"Name":"name5"}]},"Failed Users":{"Failed User Count":2,"Failed >> Users List":[{"Name":"namsdsa","Cause":"Invalid claim uri has been >> provided: http://wso2.org/claims/ctry"},{"Name":"name2ssss","Cause":"Invalid >> claim uri has been provided: http://wso2.org/claims/emaiaddress"}]}} >> >> And also, we discussed to log the bulk user import summary to the audit >> logs in the following format. >> >> Initiator : admin@carbon.super | Action : Add Role | Target : admin | >> Data : {} | Result >> > Does this audit log gives us the message that a bulk user import happened? > Action 'Add Role' does not imply a bulk user import happened IMO. > Is it possible to introduce an action which clearly conveys the actual > operation that occurred? > > >> >> The data section will contain the importing user list. As in the >> documentation, we support importing a maximum of 500,000 users at a time. >> So, considering the worse case scenario, if we log these users as well, it >> will eat up the storage very quickly and cause in threat conditions. >> >> So IMO, we do not need to log users that are being imported. Also with >> the Megala's feature [2], as the information is also being logged, I think >> it's enough if we only log the result of the operation with Initiator, >> Action and the Target values. >> >> WDYT? >> >> [1] [IAM] [Discussion] Bulk User Import Improvements >> [2] Discussion on Improving Audit logs Related with User Management >> >> Thanks and Regards, >> Menaka >> >> On Tue, Mar 13, 2018 at 12:45 PM, Dimuthu Leelarathne <dimut...@wso2.com> >> wrote: >> >>> >>> >>> On Tue, Mar 13, 2018 at 11:47 AM, Menaka Jayawardena <men...@wso2.com> >>> wrote: >>> >>>> Hi, >>>> >>>> @Denuwanthi: Yes. It can be done. Please find the summery below. >>>> >>>> SUMMERY : >>>> Bulk User Import Operation Performed by: admin >>>> User Store : PRIMARY >>>> Duplicate user count : 8 >>>> . >>>> >>>> >>>> *UI Message Modification.* >>>> >>>> Currently, if an error occurred in the process of performing the Bulk >>>> User Import Operation, the following Error message will be shown. >>>> >>>> *Error occurs while importing usernames. All usernames were not >>>> imported. Last error was : Invalid claim uri has been provided: >>>> http://wso2.org/claims/emaiaddress <http://wso2.org/claims/emaiaddress>* >>>> >>>> But there are multiple errors (Duplicate user etc). In this case, I >>>> think it's better if we show a more generic error with a brief summery and >>>> direct them to view the log file for more information. >>>> >>>> For an example: >>>> Bulk User Import Completed with Errors. >>>> Success user count: x Duplicate user count: y Failed user count: z >>>> Please check the user import log for more detailed information. >>>> >>> >>> +1 >>> >>> And in the detail log we can log errors and duplicates. >>> >>> thanks, >>> Dimuthu >>> >>> >>> >>>> >>>> Any ideas, suggestions are highly appreciated. >>>> >>>> Thanks and Regards, >>>> Menaka >>>> >>>> On Tue, Mar 13, 2018 at 9:24 AM, Denuwanthi De Silva < >>>> denuwan...@wso2.com> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Mar 12, 2018 at 4:29 PM, Menaka Jayawardena <men...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Here is an experimental user import summery. >>>>>> >>>>>> SUMMERY : >>>>>> Bulk User Import Operation Performed by: admin >>>>>> Duplicate user count: 8 >>>>>> Duplicate Users : >>>>>> name1, name2, name74, name3, name3, name, name83, name5, >>>>>> >>>>>> Failed User Count: 2 Failed Users: >>>>>> Name : namsdsa >>>>>> Cause : Invalid claim uri has been provided: >>>>>> http://wso2.org/claims/ctry >>>>>> Name : name2ssss >>>>>> Cause : Invalid claim uri has been provided: >>>>>> http://wso2.org/claims/emaiaddress >>>>>> >>>>> >>>>> Hi Menaka, >>>>> >>>>> Is it possible to print the user domain in the summary as well? Then >>>>> the information of the userstore the users were imported will be >>>>> available >>>>> as well. >>>>> >>>>> Thanks, >>>>> >>>>>> >>>>>> >>>>>> The cause string is the standard error which comes from the >>>>>> exception. Do we need to print the stack trace here? >>>>>> >>>>>> Also, there are two BulkUserImport classes (CSVUserBulkImport[1] and >>>>>> ExcelUserBulkImport[2]) and also an unused interface [3] (The classes [1] >>>>>> and [2] are concreet classes). >>>>>> >>>>>> @IAM Team: Is there any particular reason why it kept like this? >>>>>> >>>>>> IMO in this implementation, we could use it to avoid code and method >>>>>> duplication. (By making it an Abstract class) >>>>>> >>>>>> [1] https://github.com/wso2/carbon-identity-framework/blob/maste >>>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java >>>>>> /org/wso2/carbon/user/mgt/bulkimport/CSVUserBulkImport.java >>>>>> [2] https://github.com/wso2/carbon-identity-framework/blob/maste >>>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java >>>>>> /org/wso2/carbon/user/mgt/bulkimport/ExcelUserBulkImport.java >>>>>> [3] https://github.com/wso2/carbon-identity-framework/blob/maste >>>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java >>>>>> /org/wso2/carbon/user/mgt/bulkimport/UserBulkImport.java >>>>>> >>>>>> Thanks and Regards, >>>>>> Menaka >>>>>> >>>>>> >>>>>> On Mon, Mar 12, 2018 at 2:14 PM, Menaka Jayawardena <men...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> [- strategy +Architecture] >>>>>>> >>>>>>> >>>>>>> On Mon, Mar 12, 2018 at 12:21 PM, Menaka Jayawardena < >>>>>>> men...@wso2.com> wrote: >>>>>>> >>>>>>>> Hi Dimuthu, >>>>>>>> >>>>>>>> Are you going to add this log appender by default to the >>>>>>>>> configuration? >>>>>>>>> >>>>>>>> We can add the log appender by default and keep it commented. So >>>>>>>> when the user enables the Bulk User import, he also can enable the log >>>>>>>> appender as well. >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Mar 12, 2018 at 12:07 PM, Dimuthu Leelarathne < >>>>>>>> dimut...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi Menaka, >>>>>>>>> >>>>>>>>> Are you going to add this log appender by default to the >>>>>>>>> configuration? >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Dimuthu >>>>>>>>> >>>>>>>>> On Mon, Mar 12, 2018 at 11:48 AM, Dakshika Jayathilaka < >>>>>>>>> daksh...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Ruwan, >>>>>>>>>> >>>>>>>>>> Do we need to log each success? IMO admin will more interest on >>>>>>>>>> failures or duplicates. IMHO we can add detail log on failures and >>>>>>>>>> duplicates and then log the summary which includes the success count. >>>>>>>>>> >>>>>>>>>> WDYT? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> *Dakshika Jayathilaka* >>>>>>>>>> PMC Member & Committer of Apache Stratos >>>>>>>>>> Associate Technical Lead >>>>>>>>>> WSO2, Inc. >>>>>>>>>> lean.enterprise.middleware >>>>>>>>>> 0771100911 <077%20110%200911> >>>>>>>>>> >>>>>>>>>> On Mon, Mar 12, 2018 at 11:35 AM, Ruwan Abeykoon <ruw...@wso2.com >>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>>>> Hi Menaka, >>>>>>>>>>> This is nice feature. >>>>>>>>>>> I would suggest adding one line per each user, before adding, >>>>>>>>>>> and one line for each success, failure(with reason). >>>>>>>>>>> Also add a line who performs this operation. Any trackable >>>>>>>>>>> information of the request for audit purposes. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Ruwan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Mon, Mar 12, 2018 at 11:21 AM, Menaka Jayawardena < >>>>>>>>>>> men...@wso2.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Currently, when performing bulk user import operation in >>>>>>>>>>>> Identity Server, users face following issues. >>>>>>>>>>>> >>>>>>>>>>>> 1. To check import failed users, need to filter the carbon log >>>>>>>>>>>> file. >>>>>>>>>>>> 2. In UI, it shows only the last error that occurred when >>>>>>>>>>>> importing users. >>>>>>>>>>>> >>>>>>>>>>>> *Requirement Description.* >>>>>>>>>>>> There should be a user friendly way to view the import failed >>>>>>>>>>>> users. >>>>>>>>>>>> >>>>>>>>>>>> As a solution for this, we will provide a new log appender >>>>>>>>>>>> which will log the messages to a separate log file specific for >>>>>>>>>>>> bulk user >>>>>>>>>>>> import. This will help users to easily view the status of the >>>>>>>>>>>> imported >>>>>>>>>>>> users and all the error logs. >>>>>>>>>>>> >>>>>>>>>>>> Also currently, as the operation summery, we only have >>>>>>>>>>>> >>>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount >>>>>>>>>>>> + ", Duplicate count: " + duplicateCount >>>>>>>>>>>> >>>>>>>>>>>> Instead, it would be much effective if we could list the failed >>>>>>>>>>>> and duplicate user names as well. >>>>>>>>>>>> >>>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount >>>>>>>>>>>> + ", Duplicate count: " + duplicateCount >>>>>>>>>>>> "Failed Users : " + [Failed Users List] + "Duplicate Users : " >>>>>>>>>>>> + [Duplicate Users List] >>>>>>>>>>>> >>>>>>>>>>>> WDYT? >>>>>>>>>>>> >>>>>>>>>>>> Thanks and Regards, >>>>>>>>>>>> Menaka >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Menaka Jayawardena* >>>>>>>>>>>> *Software Engineer - WSO2 Inc* >>>>>>>>>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>>>>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>>>>>>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>>>>>>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>>>>>>>>> <https://menakamadushanka.wordpress.com/>* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> *Ruwan Abeykoon* >>>>>>>>>>> *Associate Director/Architect**,* >>>>>>>>>>> *WSO2, Inc. http://wso2.com <https://wso2.com/signature> * >>>>>>>>>>> *lean.enterprise.middleware.* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dimuthu Leelarathne >>>>>>>>> Director, Rapid Response Team >>>>>>>>> >>>>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>>>> email: dimut...@wso2.com >>>>>>>>> Mobile: +94773661935 <+94%2077%20366%201935> >>>>>>>>> Blog: http://muthulee.blogspot.com >>>>>>>>> >>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Menaka Jayawardena* >>>>>>>> *Software Engineer - WSO2 Inc* >>>>>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>>>>> <https://menakamadushanka.wordpress.com/>* >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Menaka Jayawardena* >>>>>>> *Software Engineer - WSO2 Inc* >>>>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>>>> <https://menakamadushanka.wordpress.com/>* >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Menaka Jayawardena* >>>>>> *Software Engineer - WSO2 Inc* >>>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>>> <https://menakamadushanka.wordpress.com/>* >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Denuwanthi De Silva >>>>> Senior Software Engineer; >>>>> WSO2 Inc.; http://wso2.com, >>>>> Email: denuwan...@wso2.com >>>>> Blog: https://denuwanthi.wordpress.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> *Menaka Jayawardena* >>>> *Software Engineer - WSO2 Inc* >>>> *Tel : 071 350 5470 <071%20350%205470>* >>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>> *Blog: https://menakamadushanka.wordpress.com/ >>>> <https://menakamadushanka.wordpress.com/>* >>>> >>>> >>> >>> >>> -- >>> Dimuthu Leelarathne >>> Director, Rapid Response Team >>> >>> WSO2, Inc. (http://wso2.com) >>> email: dimut...@wso2.com >>> Mobile: +94773661935 <+94%2077%20366%201935> >>> Blog: http://muthulee.blogspot.com >>> >>> Lean . Enterprise . Middleware >>> >> >> >> >> -- >> *Menaka Jayawardena* >> Software Engineer >> WSO2 Inc. >> >> Phone : +94 71 350 5470 <+94%2071%20350%205470> >> LinkedIn : https://lk.linkedin.com/in/menakajayawardena >> Blog : https://menakamadushanka.wordpress.com/ >> >> > > > -- > Denuwanthi De Silva > Senior Software Engineer; > WSO2 Inc.; http://wso2.com, > Email: denuwan...@wso2.com > Blog: https://denuwanthi.wordpress.com/ > -- *Menaka Jayawardena* Software Engineer WSO2 Inc. Phone : +94 71 350 5470 LinkedIn : https://lk.linkedin.com/in/menakajayawardena Blog : https://menakamadushanka.wordpress.com/
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture