[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-09 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Hi @alexey-tikhonov , I think it is addressed by part of this PR which is commit https://github.com/SSSD/sssd/pull/5552/commits/eae03fbe65488d2379131cf32274eb8cae89564c """ See

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-09 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ Hi @elkoniu, is #5086 fixed by this PR? """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-877065218

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-09 Thread pbrezina
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches pbrezina commented: """ Pushed PR: https://github.com/SSSD/sssd/pull/5552 * `master` * b4ee698ac078e74df51197b5f92432b4ed712d99 - cache_req: do not return cached data if domain is inconsistent *

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-08 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ My questions were addressed. Thank you. Covscan is clean. """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-876569430

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-08 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ My questions were addressed. Thank you. """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-876569430

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-08 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ > I think the latest issues you've found are not strictly related to the > patches. The reason is that if there are many users and groups in the cache > the `sss_cache

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi, the latest version just adds the `:config:` tag for the new option. bye, Sumit """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-872393948

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi, I think the latest issues you've found are not strictly related to the patches. The reason is that if there are many users and groups in the cache the `sss_cache -UG`

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ The [ISSUE 1] and [ISSUE 2] I described above are presented on master branch build too. This way I do not think we are introducing any regression with this PR. """ See the full

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Two more issues exists so far: [ISSUE 1] 1) Add 5000 users without SSSD (raw NSS only, systemctl stop sssd, /etc/nsswitch.conf -> files only) 2) Enable sss in /etc/nsswitch.conf

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Two more issues exists so far: [ISSUE 1] 1) Add 5000 users without SSSD (raw NSS only, systemctl stop sssd, /etc/nsswitch.conf -> files only) 2) Enable sss in /etc/nsswitch.conf

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Two more issues exists so far: [ISSUE 1] 1) Add 5000 users without SSSD (raw NSS only, systemctl stop sssd, /etc/nsswitch.conf -> files only) 2) Enable sss in /etc/nsswitch.conf

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Two more issues exists so far: [ISSUE 1] 1) Add 5000 users without SSSD (raw NSS only, systemctl stop sssd, /etc/nsswitch.conf -> files only) 2) Enable sss in /etc/nsswitch.conf

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Two more issues exists so far: [ISSUE 1] 1) Add 5000 users without SSSD (raw NSS only, systemctl stop sssd, /etc/nsswitch.conf -> files only) 2) Enable sss in /etc/nsswitch.conf

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Two more issues exists so far: [ISSUE 1] 1) Add 5000 users without SSSD (raw NSS only, systemctl stop sssd, /etc/nsswitch.conf -> files only) 2) Enable sss in /etc/nsswitch.conf

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Another observation I have is that with 5000 users `systemctl restart sssd` takes about 40 seconds. Normally it would hit watchdog, with this PR it starts normally. The question

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ > > I have one strange observation but I am not sure if it is related to > > batching process. > > Steeps: > > ``` > > 1. Add 100 users calling useradd in loop > > > > 2. Dell 100

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run test against 5000 users to check marking domain as inconsistent. This PR can be considered as solution for https://github.com/SSSD/sssd/issues/5091 """ See the full

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run test against 5000 users checking how fast we mark domain as inconsistent. In some cases this took up to 3 seconds. It is still much faster than latest master version but

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-07-01 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ > I have one strange observation but I am not sure if it is related to batching > process. > > Steeps: > > 1. Add 100 users calling useradd in loop > > 2. Dell 100

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-30 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi, the latest version contains fixes for @alexey-tikhonov's review comments, I'll try to understand what is happening in @elkoniu latest test now. bye, Sumit """ See the

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-30 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ > There are several minor covscan issues: > https://cov01.lab.eng.brq.redhat.com/covscanhub/task/223840/log/added.err > (sorry for the private link) Thanks, fixed. """ See

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I have one strange observation but I am not sure if it is related to batching process. Steeps: 1) Add 100 users calling useradd in loop 2) Dell 100 users calling userdell in loop

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ There are several minor covscan issues: https://cov01.lab.eng.brq.redhat.com/covscanhub/task/223840/log/added.err (sorry for the private link) """ See the full comment at

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ `files_id_ctx`: ``` const char **passwd_files; const char **group_files; ``` -- do we really need support of multiple passwd/group local files..? This

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Yes, it does. I am running separated test against 5000 users to confirm that. In parallel I am doing normal code review for this PR. Hope to finish both later today. """ See the

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ @elkoniu , did you check if this PR fixes #5091? """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-870661479

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ > Batching version is more than 2 times faster than SSSD master It's actually better according to your results: 73 vs 951 seconds (i.e. ~ x13) for 500 users. And "only"

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ![result](https://user-images.githubusercontent.com/2312108/123786567-229b3800-d8da-11eb-973f-a5b3e49e2371.png) [green] Raw file provider (SSSD disabled) [yellow] SSSD from latest

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ![result](https://user-images.githubusercontent.com/2312108/123786567-229b3800-d8da-11eb-973f-a5b3e49e2371.png) [green] Raw file provider (SSSD disabled) [yellow] SSSD from latest

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-29 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ![result](https://user-images.githubusercontent.com/2312108/123786567-229b3800-d8da-11eb-973f-a5b3e49e2371.png) [green] Raw file provider (SSSD disabled) [yellow] SSSD from latest

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-25 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi, thanks for the logs, they made me realize that my test environment is so old that `useradd` does not call `sss_cache` and this is blocked because

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-24 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ > Hi @elkoniu, > > thank you for the latest tests, what still makes me wonder is that if I run > one of those tests I see: > > ``` > # time for N in `seq 500`; do useradd

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-24 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi @elkoniu, thank you for the latest tests, what still makes me wonder is that if I run one of those tests I see: ``` # time for N in `seq 500`; do useradd test-user-$N;

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-23 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Just run another test run to include `batch_size = 1000`. @sumit-bose I have only one domain in `sssd.conf`: ``` [sssd] config_file_version = 2 services = nss, pam, ifp, sudo

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-20 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi, thanks for testing and reviewing, yes `batch_size` should be larger, I think `1000` would be ok. @elkoniu, do you, by chance, have a second domain in `sssd.conf` in your

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-19 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ > Based on @alexey-tikhonov comments I can run tests again with changes on > `state->batch_size` and `state->delay`. @sumit-bose can you advise what else > we should

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Based on @alexey-tikhonov comments I can run tests again with changes on `state->batch_size` and `state->delay`. @sumit-bose can you advise what else we should experiment with?

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ![files-provider](https://user-images.githubusercontent.com/2312108/122610038-48754100-d07f-11eb-8430-7add79a042cd.png) Test setup: Fedora 32 virtual machine, 1 CPU, 2G RAM I run

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ > What is "raw file provider"? libnss_files.so? This means `systemctl stop sssd` and set `nsswitch.conf` to: ``` ... passwd: files group: files ... ``` This way SSSD

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ > What is "raw file provider"? libnss_files.so? This means `systemctl stop sssd` and set `nsswitch.conf` to: ``` ... passwd: files group: files ... ``` This way SSSD is

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ > What is "raw file provider"? libnss_files.so? This means `systemctl stop sssd` and set `nsswitch.conf` to: ``` ... passwd: files group: files ... ``` """ See the full

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ What is "raw file provider"? libnss_files.so? """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-864258003

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ![files-provider](https://user-images.githubusercontent.com/2312108/122610038-48754100-d07f-11eb-8430-7add79a042cd.png) Test setup: Fedora 32 virtual machine, 1 CPU, 2G RAM I run

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ![files-provider](https://user-images.githubusercontent.com/2312108/122610038-48754100-d07f-11eb-8430-7add79a042cd.png) Test setup: Fedora 32 virtual machine, 1 CPU, 2G RAM I run

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Right now I am running 2nd set of tests to include raw files provider and CPU usage chart. Should have result in about 1h. Side note: if some of you recently used debug_microseconds

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Right now I am running 2nd set of tests to include raw files provider and CPU usage chart. Should have result in about 1h. Side note: if some of you recently used debug_microseconds

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ > * time needed for add / delete users in bash loop has increased by > minimum 25% This is quite unexpected taking into account "files: delay refresh and not run in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Test setup: Fedora 32 virtual machine, 1 CPU, 2G RAM Estimated time needed to mark domain as inconsistent: |Version|30 + 100 users|30 + 250 users|30 + 500 users|

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Test setup: Fedora 32 virtual machine, 1 CPU, 2G RAM Estimated time needed to mark domain as inconsistent: |Version|30 + 100 users|30 + 250 users|30 + 500 users|

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ Test setup: Fedora 32 virtual machine, 1 CPU, 2G RAM Estimated time needed to mark domain as inconsistent: |Version|30 + 100 users|30 + 250 users|30 + 500 users|

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-16 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi, in the latest version a copy-and-paste error which caused one test failure is fixed and the new `fallback_to_nss = False` option is added to more tests which do not have a

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-16 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ Perhaps tests expectations are wrong now and has to be adjusted? """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-862329878

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-16 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ Debian: ``` FAIL test_files_provider.py::test_getgrnam_members_groups_first FAIL test_files_provider.py::test_getgrnam_user_member_and_ghost FAIL

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-16 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ Debian: ``` FAIL test_files_provider.py::test_getgrnam_members_groups_first FAIL test_files_provider.py::test_getgrnam_user_member_and_ghost FAIL

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-06-02 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ Hi, in the latest version I added a patch which implements a fallback to the next nss module in case the files provider backend is busy with updating the data and a patch

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-05-18 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ @sumit-bose Maybe temporary we should convert this PR to [WIP] state? """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-843631952

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-13 Thread pbrezina
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches pbrezina commented: """ Ok, feel free to continue with this PR. """ See the full comment at https://github.com/SSSD/sssd/pull/5552#issuecomment-818665556 ___

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-07 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ > > where do you see the difficulties to add a diff approach? The step the > > remove the cache has to be replaced by the diff step and then the entries > > which changed

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-07 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ > About transition to systemd, I'm all for it. We already rely on systemd at > some places anyway so I think it is time to finally drop the monitor. But > lets not discuss it

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-07 Thread pbrezina
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches pbrezina commented: """ > > The reason why the message is not delivered is because we send it over > > `provider->sbus_conn` which requires accessing the backend's tevent loop > > because the backend

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-07 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I modified this PR by adding initial zero step which only starts database transaction and returns to tevent loop: https://github.com/elkoniu/sssd/tree/file-batching-zero-step End

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-07 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ > > > What is the reason for this patches? > > > > > > IIUC, to avoid single large blocking operation. see #5557 > > IMHO in this case it would be better to do a diff update

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-07 Thread sumit-bose
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches sumit-bose commented: """ > The reason why the message is not delivered is because we send it over > `provider->sbus_conn` which requires accessing the backend's tevent loop > because the backend runs

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-06 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ > > Version 100 useradd 100 userdel 250 useradd 250 userdel > 500 useradd 500 userdel > Vanilla 0m43.464s 0m41.084s 2m46.462s

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-06 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I modified this PR by adding initial zero step which only starts database transaction and returns to tevent loop:

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-06 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ``` == [BATCH WITH ZERO STEP]

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-06 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ > > Version 100 useradd 100 userdel 250 useradd 250 userdel > > 500 useradd 500 userdel > > Vanilla 0m43.464s 0m41.084s 2m46.462s

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-06 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ > > Version 100 useradd 100 userdel 250 useradd 250 userdel > 500 useradd 500 userdel > Vanilla 0m43.464s 0m41.084s 2m46.462s

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ I did some testing comparing this PR with vanilla SSSD. I added extra log in `sf_passwd_cb()` which is `/etc/passwd` file modification callback and in

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-03 Thread elkoniu
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches elkoniu commented: """ ``` == [VANILIA SSSD]

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-01 Thread pbrezina
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches pbrezina commented: """ > > What is the reason for this patches? > > IIUC, to avoid single large blocking operation. see #5557 IMHO in this case it would be better to do a diff update to improve

[SSSD] [sssd PR#5552][comment] files: split update into batches

2021-04-01 Thread alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5552 Title: #5552: files: split update into batches alexey-tikhonov commented: """ > What is the reason for this patches? IIUC, to avoid single large blocking operation. see #5557 > To be able to deliver domain-inconsistent message? If that's the case

  1   2   >