[Owncloud] Heavy LDAP user search

Hi everyone! I work at the University of Costa Rica and we are deploying Owncloud as a service for students and employees. The problem here is that our LDAP directory has about 109120 entries and when we try to share a single file, it takes a long time and does a very heavy process in the getUser

Re: [Owncloud] Heavy LDAP user search

Hi Manuel! Manuel Delgado schrieb: >Hi Jörn, > >On Fri, Sep 21, 2012 at 4:29 AM, Jörn Friedrich Dreyer >wrote: > >> Just brainstorming: lucene by default only starts searching when the >user >> has enterd three characters. Using the same limitation in the share >dialog >> would allow Arthur to

Re: [Owncloud] Heavy LDAP user search

Am 21.09.2012 12:11, schrieb Arthur Schiwon: I think there is a misunderstanding. When you want to share a file and the list of shared users is supposed to be up to date, then every request will need to do a full scan. How else is the software (ownCloud) supposed to know the changes? Except your

Re: [Owncloud] Heavy LDAP user search

On Friday, September 21, 2012 12:29:35 PM Jörn Friedrich Dreyer wrote: > Just brainstorming: lucene by default only starts searching when the user > has enterd three characters. Using the same limitation in the share dialog > would allow Arthur to query the LDAP server with a filter that takes the

Re: [Owncloud] Heavy LDAP user search

On Thursday, September 20, 2012 11:15:54 AM Arthur Schiwon wrote: > On Wednesday, September 19, 2012 04:32:11 PM Manuel Delgado wrote: > > I made the tests with my changes and your patches. It's simply miraculous: > > $ time php /srv/sites/owncloud/cron.php > > > > My changes: > > real235m2.63

Re: [Owncloud] Heavy LDAP user search

Just brainstorming: lucene by default only starts searching when the user has enterd three characters. Using the same limitation in the share dialog would allow Arthur to query the LDAP server with a filter that takes the three digits into account. That should drastically reduce lookup times and

Re: [Owncloud] Heavy LDAP user search

On Friday, September 21, 2012 12:03:09 PM Dirk Kastens wrote: > I have configured many software products with ldap access at work. None > of it scans the whole directory. The problem is, that oc won't recognize > changes in the directory. When I add a new ldap user or when I change > the group memb

Re: [Owncloud] Heavy LDAP user search

On Friday 21 September 2012 12:03:09 Dirk Kastens wrote: > None of it scans the whole directory. Obviously a poor choice of words on my part using 'whole' directory, when I meant the configured section of the directory. But my point was about changing the trigger for populating the cache from th

Re: [Owncloud] Heavy LDAP user search

On Friday, September 21, 2012 11:56:12 AM Diederik de Haas wrote: > On Friday 21 September 2012 11:41:14 Arthur Schiwon wrote: > > > I'm always > > > running into the search time limit of our ldap server. Now, when I try > > > to share something with another ldap user, the owncloud server hangs. I

Re: [Owncloud] Heavy LDAP user search

I have configured many software products with ldap access at work. None of it scans the whole directory. The problem is, that oc won't recognize changes in the directory. When I add a new ldap user or when I change the group membership of a user, I expect that the software immediately recognize

Re: [Owncloud] Heavy LDAP user search

On Friday 21 September 2012 11:41:14 Arthur Schiwon wrote: > > I'm always > > running into the search time limit of our ldap server. Now, when I try > > to share something with another ldap user, the owncloud server hangs. I > > assume, it's because the autocomplete function tries to read in all 50

Re: [Owncloud] Heavy LDAP user search

On Friday, September 21, 2012 11:24:13 AM Dirk Kastens wrote: > Hi, > > I still wonder why owncloud scans the whole directory. It scans according to the User Base DN and User list filter, but given. In ownCloud the username is also the user ID. So we need to create an owncloud username for eac

Re: [Owncloud] Heavy LDAP user search

Hi, I still wonder why owncloud scans the whole directory. I'm always running into the search time limit of our ldap server. Now, when I try to share something with another ldap user, the owncloud server hangs. I assume, it's because the autocomplete function tries to read in all 50k users of

Re: [Owncloud] Heavy LDAP user search

On Wednesday, September 19, 2012 04:32:11 PM Manuel Delgado wrote: > I made the tests with my changes and your patches. It's simply miraculous: > $ time php /srv/sites/owncloud/cron.php > > My changes: > real235m2.634s > user229m51.470s > sys0m14.533s > > Patch A (prepared statement)

Re: [Owncloud] Heavy LDAP user search

Hi Arthur, On 09/19/2012 6:19 PM, Arthur Schiwon wrote: > Hi again, > > On 09/19/2012 02:25 AM, Manuel Delgado wrote: > >* I think the problem is not in the ldap but in the greedy*>* > >(foreach/recursive) search in ldap2ownCloudNames method.* > I have two other approaches which work around the

Re: [Owncloud] Heavy LDAP user search

Hi again, On 09/19/2012 02:25 AM, Manuel Delgado wrote: I think the problem is not in the ldap but in the greedy (foreach/recursive) search in ldap2ownCloudNames method. I have two other approaches which work around the recursive array search. One is simple improving the result array from map

Re: [Owncloud] Heavy LDAP user search

Hi Manuel, On 09/19/2012 02:25 AM, Manuel Delgado wrote: Hello all, it's me again I was working on a kind of optimization for the user_ldap app. As I told you in past emails, we have a ldap with more than 100k users. And the process of creating the cache still not working (takes to much time, m

Re: [Owncloud] Heavy LDAP user search

Hello all, it's me again I was working on a kind of optimization for the user_ldap app. As I told you in past emails, we have a ldap with more than 100k users. And the process of creating the cache still not working (takes to much time, more than 7h). I've cloned the core repo on github and made

Re: [Owncloud] Heavy LDAP user search

Exactly, I'm trying to access Settings/Users page, but it never load. That's why I'm asking how does the cache work, because all the 109.402 user entries and 25 groups are in the database but a heavy work still there. As far as I know the cache resides in a file (not in DB), where is it or how is

Re: [Owncloud] Heavy LDAP user search

What do you mean by trying to make a simple list? The problem areas of sharing and the Settings -> Users page were rewritten to fix the performance issues by loading in batches. I think LDAP may still have to load all the users once to populate the cache. Arthur will be able to answer your ques

Re: [Owncloud] Heavy LDAP user search

Hi! I'm trying again with the Owncloud 4.5 Beta (Well actually the master branch) and I have almost the same issue... the CPU of my server goes to 100% when I try to make a simple list or query over the LDAP users. Actually it loops until time is up... How does the cache work? Another issue, I t

Re: [Owncloud] Heavy LDAP user search

On 08/10/2012 09:02 AM, Dirk Kastens wrote: Hi Manual, When I set the size limit, ldap_serch will only return X entries, so autocompletion will only work on those X entries and I'll be unable to share the file with anyone beyond X. Is this right? or am I missing something? Theoretically yo

Re: [Owncloud] Heavy LDAP user search

Hi Manual, When I set the size limit, ldap_serch will only return X entries, so autocompletion will only work on those X entries and I'll be unable to share the file with anyone beyond X. Is this right? or am I missing something? Theoretically you're right. But with autocompletion, the more l

Re: [Owncloud] Heavy LDAP user search

Hi Dirk, When I set the size limit, ldap_serch will only return X entries, so autocompletion will only work on those X entries and I'll be unable to share the file with anyone beyond X. Is this right? or am I missing something? On Thu, Aug 9, 2012 at 1:07 AM, Dirk Kastens wrote: > Hi, > > ano

Re: [Owncloud] Heavy LDAP user search

Hi, another solution would be to have an ldap search limit implemented. I just configured this for our SOGo server (opengroupware). This has a server option called SOGoLDAPQueryLimit. If you set this to 100, the search will stop after the first found 100 users. Autocompletion works very well

Re: [Owncloud] Heavy LDAP user search

The first alpha release of ownCloud 5 hasn't been tagged yet, but it should be merged in at that time. Michael On Wednesday, August 08, 2012 07:14:39 PM Manuel Delgado wrote: > Are the changes in the alpha versión or somewere on git repositories? > We want to test it with our LDAP and see how i

Re: [Owncloud] Heavy LDAP user search

Are the changes in the alpha versión or somewere on git repositories? We want to test it with our LDAP and see how it works Thanks! Manuel Delgado --- *Usuario Linux* *#520940 * Bach. Computación e Informática Univer

Re: [Owncloud] Heavy LDAP user search

Hi Manuel, We're aware of this problem and have implemented somewhat of a solution for the next release of ownCloud. The first request will still take a while, but after that the users will be cached so searching should not be delayed in the user interface. Michael On Wednesday, August 08, 2

[Owncloud] Heavy LDAP user search

Hi everyone! I work at the University of Costa Rica and we are deploying Owncloud as a service for students and employees. The problem here is that our LDAP directory has about 109120 entries and when we try to share a single file, it takes a long time and does a very heavy process in the getUser