On Mon, Mar 19, 2018 at 3:46 PM, Menaka Jayawardena <men...@wso2.com> wrote:
> Hi, > > As we discussed in the meeting today[1] (19/03/2018), I modified the > summary log as follows. > > {"Bulk User Import Operation Performed by":"admin","User > Store":"PRIMARY","Duplicate Users":{"Duplicate User Count":8,"User > Names":[{"Name":"name1"},{"Name":"name2"},{"Name":"name74"}, > {"Name":"name3"},{"Name":"name3"},{"Name":"name"},{"Name":" > name83"},{"Name":"name5"}]},"Failed Users":{"Failed User Count":2,"Failed > Users List":[{"Name":"namsdsa","Cause":"Invalid claim uri has been > provided: http://wso2.org/claims/ctry"},{"Name":"name2ssss","Cause":"Invalid > claim uri has been provided: http://wso2.org/claims/emaiaddress"}]}} > > And also, we discussed to log the bulk user import summary to the audit > logs in the following format. > > Initiator : admin@carbon.super | Action : Add Role | Target : admin | > Data : {} | Result > Does this audit log gives us the message that a bulk user import happened? Action 'Add Role' does not imply a bulk user import happened IMO. Is it possible to introduce an action which clearly conveys the actual operation that occurred? > > The data section will contain the importing user list. As in the > documentation, we support importing a maximum of 500,000 users at a time. > So, considering the worse case scenario, if we log these users as well, it > will eat up the storage very quickly and cause in threat conditions. > > So IMO, we do not need to log users that are being imported. Also with the > Megala's feature [2], as the information is also being logged, I think it's > enough if we only log the result of the operation with Initiator, Action > and the Target values. > > WDYT? > > [1] [IAM] [Discussion] Bulk User Import Improvements > [2] Discussion on Improving Audit logs Related with User Management > > Thanks and Regards, > Menaka > > On Tue, Mar 13, 2018 at 12:45 PM, Dimuthu Leelarathne <dimut...@wso2.com> > wrote: > >> >> >> On Tue, Mar 13, 2018 at 11:47 AM, Menaka Jayawardena <men...@wso2.com> >> wrote: >> >>> Hi, >>> >>> @Denuwanthi: Yes. It can be done. Please find the summery below. >>> >>> SUMMERY : >>> Bulk User Import Operation Performed by: admin >>> User Store : PRIMARY >>> Duplicate user count : 8 >>> . >>> >>> >>> *UI Message Modification.* >>> >>> Currently, if an error occurred in the process of performing the Bulk >>> User Import Operation, the following Error message will be shown. >>> >>> *Error occurs while importing usernames. All usernames were not >>> imported. Last error was : Invalid claim uri has been provided: >>> http://wso2.org/claims/emaiaddress <http://wso2.org/claims/emaiaddress>* >>> >>> But there are multiple errors (Duplicate user etc). In this case, I >>> think it's better if we show a more generic error with a brief summery and >>> direct them to view the log file for more information. >>> >>> For an example: >>> Bulk User Import Completed with Errors. >>> Success user count: x Duplicate user count: y Failed user count: z >>> Please check the user import log for more detailed information. >>> >> >> +1 >> >> And in the detail log we can log errors and duplicates. >> >> thanks, >> Dimuthu >> >> >> >>> >>> Any ideas, suggestions are highly appreciated. >>> >>> Thanks and Regards, >>> Menaka >>> >>> On Tue, Mar 13, 2018 at 9:24 AM, Denuwanthi De Silva < >>> denuwan...@wso2.com> wrote: >>> >>>> >>>> >>>> On Mon, Mar 12, 2018 at 4:29 PM, Menaka Jayawardena <men...@wso2.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Here is an experimental user import summery. >>>>> >>>>> SUMMERY : >>>>> Bulk User Import Operation Performed by: admin >>>>> Duplicate user count: 8 >>>>> Duplicate Users : >>>>> name1, name2, name74, name3, name3, name, name83, name5, >>>>> >>>>> Failed User Count: 2 Failed Users: >>>>> Name : namsdsa >>>>> Cause : Invalid claim uri has been provided: >>>>> http://wso2.org/claims/ctry >>>>> Name : name2ssss >>>>> Cause : Invalid claim uri has been provided: >>>>> http://wso2.org/claims/emaiaddress >>>>> >>>> >>>> Hi Menaka, >>>> >>>> Is it possible to print the user domain in the summary as well? Then >>>> the information of the userstore the users were imported will be available >>>> as well. >>>> >>>> Thanks, >>>> >>>>> >>>>> >>>>> The cause string is the standard error which comes from the exception. >>>>> Do we need to print the stack trace here? >>>>> >>>>> Also, there are two BulkUserImport classes (CSVUserBulkImport[1] and >>>>> ExcelUserBulkImport[2]) and also an unused interface [3] (The classes [1] >>>>> and [2] are concreet classes). >>>>> >>>>> @IAM Team: Is there any particular reason why it kept like this? >>>>> >>>>> IMO in this implementation, we could use it to avoid code and method >>>>> duplication. (By making it an Abstract class) >>>>> >>>>> [1] https://github.com/wso2/carbon-identity-framework/blob/maste >>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java >>>>> /org/wso2/carbon/user/mgt/bulkimport/CSVUserBulkImport.java >>>>> [2] https://github.com/wso2/carbon-identity-framework/blob/maste >>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java >>>>> /org/wso2/carbon/user/mgt/bulkimport/ExcelUserBulkImport.java >>>>> [3] https://github.com/wso2/carbon-identity-framework/blob/maste >>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java >>>>> /org/wso2/carbon/user/mgt/bulkimport/UserBulkImport.java >>>>> >>>>> Thanks and Regards, >>>>> Menaka >>>>> >>>>> >>>>> On Mon, Mar 12, 2018 at 2:14 PM, Menaka Jayawardena <men...@wso2.com> >>>>> wrote: >>>>> >>>>>> [- strategy +Architecture] >>>>>> >>>>>> >>>>>> On Mon, Mar 12, 2018 at 12:21 PM, Menaka Jayawardena <men...@wso2.com >>>>>> > wrote: >>>>>> >>>>>>> Hi Dimuthu, >>>>>>> >>>>>>> Are you going to add this log appender by default to the >>>>>>>> configuration? >>>>>>>> >>>>>>> We can add the log appender by default and keep it commented. So >>>>>>> when the user enables the Bulk User import, he also can enable the log >>>>>>> appender as well. >>>>>>> >>>>>>> >>>>>>> On Mon, Mar 12, 2018 at 12:07 PM, Dimuthu Leelarathne < >>>>>>> dimut...@wso2.com> wrote: >>>>>>> >>>>>>>> Hi Menaka, >>>>>>>> >>>>>>>> Are you going to add this log appender by default to the >>>>>>>> configuration? >>>>>>>> >>>>>>>> thanks, >>>>>>>> Dimuthu >>>>>>>> >>>>>>>> On Mon, Mar 12, 2018 at 11:48 AM, Dakshika Jayathilaka < >>>>>>>> daksh...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi Ruwan, >>>>>>>>> >>>>>>>>> Do we need to log each success? IMO admin will more interest on >>>>>>>>> failures or duplicates. IMHO we can add detail log on failures and >>>>>>>>> duplicates and then log the summary which includes the success count. >>>>>>>>> >>>>>>>>> WDYT? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> *Dakshika Jayathilaka* >>>>>>>>> PMC Member & Committer of Apache Stratos >>>>>>>>> Associate Technical Lead >>>>>>>>> WSO2, Inc. >>>>>>>>> lean.enterprise.middleware >>>>>>>>> 0771100911 <077%20110%200911> >>>>>>>>> >>>>>>>>> On Mon, Mar 12, 2018 at 11:35 AM, Ruwan Abeykoon <ruw...@wso2.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Menaka, >>>>>>>>>> This is nice feature. >>>>>>>>>> I would suggest adding one line per each user, before adding, and >>>>>>>>>> one line for each success, failure(with reason). >>>>>>>>>> Also add a line who performs this operation. Any trackable >>>>>>>>>> information of the request for audit purposes. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Ruwan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Mar 12, 2018 at 11:21 AM, Menaka Jayawardena < >>>>>>>>>> men...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Currently, when performing bulk user import operation in >>>>>>>>>>> Identity Server, users face following issues. >>>>>>>>>>> >>>>>>>>>>> 1. To check import failed users, need to filter the carbon log >>>>>>>>>>> file. >>>>>>>>>>> 2. In UI, it shows only the last error that occurred when >>>>>>>>>>> importing users. >>>>>>>>>>> >>>>>>>>>>> *Requirement Description.* >>>>>>>>>>> There should be a user friendly way to view the import failed >>>>>>>>>>> users. >>>>>>>>>>> >>>>>>>>>>> As a solution for this, we will provide a new log appender which >>>>>>>>>>> will log the messages to a separate log file specific for bulk user >>>>>>>>>>> import. >>>>>>>>>>> This will help users to easily view the status of the imported >>>>>>>>>>> users and >>>>>>>>>>> all the error logs. >>>>>>>>>>> >>>>>>>>>>> Also currently, as the operation summery, we only have >>>>>>>>>>> >>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount >>>>>>>>>>> + ", Duplicate count: " + duplicateCount >>>>>>>>>>> >>>>>>>>>>> Instead, it would be much effective if we could list the failed >>>>>>>>>>> and duplicate user names as well. >>>>>>>>>>> >>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount >>>>>>>>>>> + ", Duplicate count: " + duplicateCount >>>>>>>>>>> "Failed Users : " + [Failed Users List] + "Duplicate Users : " + >>>>>>>>>>> [Duplicate Users List] >>>>>>>>>>> >>>>>>>>>>> WDYT? >>>>>>>>>>> >>>>>>>>>>> Thanks and Regards, >>>>>>>>>>> Menaka >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Menaka Jayawardena* >>>>>>>>>>> *Software Engineer - WSO2 Inc* >>>>>>>>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>>>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>>>>>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>>>>>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>>>>>>>> <https://menakamadushanka.wordpress.com/>* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> *Ruwan Abeykoon* >>>>>>>>>> *Associate Director/Architect**,* >>>>>>>>>> *WSO2, Inc. http://wso2.com <https://wso2.com/signature> * >>>>>>>>>> *lean.enterprise.middleware.* >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dimuthu Leelarathne >>>>>>>> Director, Rapid Response Team >>>>>>>> >>>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>>> email: dimut...@wso2.com >>>>>>>> Mobile: +94773661935 <+94%2077%20366%201935> >>>>>>>> Blog: http://muthulee.blogspot.com >>>>>>>> >>>>>>>> Lean . Enterprise . Middleware >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Menaka Jayawardena* >>>>>>> *Software Engineer - WSO2 Inc* >>>>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>>>> <https://menakamadushanka.wordpress.com/>* >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Menaka Jayawardena* >>>>>> *Software Engineer - WSO2 Inc* >>>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>>> <https://menakamadushanka.wordpress.com/>* >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Menaka Jayawardena* >>>>> *Software Engineer - WSO2 Inc* >>>>> *Tel : 071 350 5470 <071%20350%205470>* >>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>>>> <https://lk.linkedin.com/in/menakajayawardena>* >>>>> *Blog: https://menakamadushanka.wordpress.com/ >>>>> <https://menakamadushanka.wordpress.com/>* >>>>> >>>>> >>>> >>>> >>>> -- >>>> Denuwanthi De Silva >>>> Senior Software Engineer; >>>> WSO2 Inc.; http://wso2.com, >>>> Email: denuwan...@wso2.com >>>> Blog: https://denuwanthi.wordpress.com/ >>>> >>> >>> >>> >>> -- >>> *Menaka Jayawardena* >>> *Software Engineer - WSO2 Inc* >>> *Tel : 071 350 5470 <071%20350%205470>* >>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena >>> <https://lk.linkedin.com/in/menakajayawardena>* >>> *Blog: https://menakamadushanka.wordpress.com/ >>> <https://menakamadushanka.wordpress.com/>* >>> >>> >> >> >> -- >> Dimuthu Leelarathne >> Director, Rapid Response Team >> >> WSO2, Inc. (http://wso2.com) >> email: dimut...@wso2.com >> Mobile: +94773661935 <+94%2077%20366%201935> >> Blog: http://muthulee.blogspot.com >> >> Lean . Enterprise . Middleware >> > > > > -- > *Menaka Jayawardena* > Software Engineer > WSO2 Inc. > > Phone : +94 71 350 5470 <+94%2071%20350%205470> > LinkedIn : https://lk.linkedin.com/in/menakajayawardena > Blog : https://menakamadushanka.wordpress.com/ > > -- Denuwanthi De Silva Senior Software Engineer; WSO2 Inc.; http://wso2.com, Email: denuwan...@wso2.com Blog: https://denuwanthi.wordpress.com/
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture