[xwiki-users] [xwiki 3.2] virtual user big performance issues

[Merlin Olivier]

Hello,

On the xwiki  3.2, we have decided to use the virtual xwiki  (multi-tenant) 
feature as it should be better for scalability and performance
http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Virtualization


"Each wiki has its own database schema, thus allowing for a complete data 
separation between the different wikis in the same instance.

The main advantage of multi-tenancy is performance optimization."
Problem:
The main wiki has a relative good level of performance - with a response time 
around 2.5 second to browse a page
So we are in the xwiki database
But when we are in a virtual wiki (another schema ) - the performance are very 
poor.
The average response time is around 5 seconds  per page !!! So at list 2 times 
slower.

We are using MySQL and glassfish 3.1 - and we have done tuning on the MySQL 
database and glassfish.

We have used a profiling tool and see that the request send to the virtual 
environment are raising a big amount of exception ...
The problem is not on the database part (even if they are many SQL calls to 
visualize only one page)
The problem seems really to be on the virtual environment configuration OR in a 
file corruption ...

I have attached the exception list . (exception_list.xls)
>From this is it possible to know if we have an installation issue ??

Does someone have experience of very good performance with xwiki  ? ( response 
time #1second per page)
In that case

Best regards,
Olivier





___
Olivier Merlin
RnD Tuning/profiling community Leader
Tel: + 33.4.42.36.59.87

olivier.mer...@gemalto.com
___

___
users mailing list
users@xwiki.org
http://lists.xwiki.org/mailman/listinfo/users

Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

[Merlin Olivier]

Hello Vincent,

I have recreated the indexes on one multiwiki ... and I get the same 
performance results 
Each page are loaded in 4-5 seconds.
In fact there was an improvement in the DB usage but it was not a lot before.
The API breakdown shows me:
- 1200 ms in the classloading
- 930 ms in the XML processing
- 580 ms for the xwiki API
- 420 ms for JDBC 
- 350 ms for wiki model 
(...)

The problem of performance is really on the application server code response 
time - not in the database.
- the problem could be in the big amount of exception raised (is it normal ?)
- Glassfish 3.1 issues ? We use a 64bits deployment with a sun jdk1.6.0_26 Xms 
= Xmx = 2Gb
- 3.2 version
- the HW not sufficient ? We use a VMware with a Redhat Linux 5.7
- 2 core CPU ( X5650  @ 2.67GHz )
- 4Go RAM

Any ideas ? Good config to test ? 
My next ideas would be :
- migrate to Weblogic
- migrate to xwiki 4.x

I can send you all the list of exception - it consumes a lot of CPU for me ...


Thanks for your help
Olivier




-Original Message-
From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On Behalf Of 
Vincent Massol
Sent: Tuesday, May 15, 2012 5:56 PM
To: XWiki Users
Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

Hi Merlin,

On May 15, 2012, at 5:41 PM, Merlin Olivier wrote:

> Hello,
> 
> On the xwiki  3.2, we have decided to use the virtual xwiki  
> (multi-tenant) feature as it should be better for scalability and 
> performance 
> http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Virtualization
> 
> 
> "Each wiki has its own database schema, thus allowing for a complete data 
> separation between the different wikis in the same instance.
> 
> The main advantage of multi-tenancy is performance optimization."
> Problem:
> The main wiki has a relative good level of performance - with a 
> response time around 2.5 second to browse a page So we are in the 
> xwiki database But when we are in a virtual wiki (another schema ) - the 
> performance are very poor.
> The average response time is around 5 seconds  per page !!! So at list 2 
> times slower.
> 
> We are using MySQL and glassfish 3.1 - and we have done tuning on the MySQL 
> database and glassfish.
> 
> We have used a profiling tool and see that the request send to the virtual 
> environment are raising a big amount of exception ...
> The problem is not on the database part (even if they are many SQL 
> calls to visualize only one page) The problem seems really to be on the 
> virtual environment configuration OR in a file corruption ...
> 
> I have attached the exception list . (exception_list.xls) From this is 
> it possible to know if we have an installation issue ??
> 
> Does someone have experience of very good performance with xwiki  ? ( 
> response time #1second per page) In that case

I can tell you fore sure that there's no performance issue with multiwikis. 
Each wiki has its own DBs so it's really fast.

Maybe you don't have indexes set on the subwikis? I think we started setting 
them automatically in XE 3.2 but it's worth checking, see:
http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Database+Administration

Thanks
-Vincent

___
users mailing list
users@xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
___
users mailing list
users@xwiki.org
http://lists.xwiki.org/mailman/listinfo/users

Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

[Merlin Olivier]

Ok - please follow this link: 
http://dl.dropbox.com/u/19876503/exception_list.xlsx
I suspect we have something wrong in the installation.

Best regards,
Olivier

-Original Message-
From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On Behalf Of 
Thomas Mortagne
Sent: Wednesday, May 16, 2012 8:52 AM
To: XWiki Users
Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

On Tue, May 15, 2012 at 6:55 PM, Merlin Olivier  
wrote:
> Hello Vincent,
>
> I have recreated the indexes on one multiwiki ... and I get the same 
> performance results Each page are loaded in 4-5 seconds.
> In fact there was an improvement in the DB usage but it was not a lot before.
> The API breakdown shows me:
> - 1200 ms in the classloading
> - 930 ms in the XML processing
> - 580 ms for the xwiki API
> - 420 ms for JDBC
> - 350 ms for wiki model
> (...)
>
> The problem of performance is really on the application server code response 
> time - not in the database.
> - the problem could be in the big amount of exception raised (is it 
> normal ?)

No it's not normal and yes it's probably related.

> - Glassfish 3.1 issues ? We use a 64bits deployment with a sun 
> jdk1.6.0_26 Xms = Xmx = 2Gb
> - 3.2 version
> - the HW not sufficient ? We use a VMware with a Redhat Linux 5.7
>        - 2 core CPU ( X5650  @ 2.67GHz )
>        - 4Go RAM
>
> Any ideas ? Good config to test ?
> My next ideas would be :
> - migrate to Weblogic
> - migrate to xwiki 4.x
>
> I can send you all the list of exception - it consumes a lot of CPU for me ...

Yes please or put them somewhere and send the link.

>
>
> Thanks for your help
> Olivier
>
>
>
>
> -Original Message-
> From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On 
> Behalf Of Vincent Massol
> Sent: Tuesday, May 15, 2012 5:56 PM
> To: XWiki Users
> Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance 
> issues
>
> Hi Merlin,
>
> On May 15, 2012, at 5:41 PM, Merlin Olivier wrote:
>
>> Hello,
>>
>> On the xwiki  3.2, we have decided to use the virtual xwiki
>> (multi-tenant) feature as it should be better for scalability and 
>> performance 
>> http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Virtualization
>>
>>
>> "Each wiki has its own database schema, thus allowing for a complete data 
>> separation between the different wikis in the same instance.
>>
>> The main advantage of multi-tenancy is performance optimization."
>> Problem:
>> The main wiki has a relative good level of performance - with a 
>> response time around 2.5 second to browse a page So we are in the 
>> xwiki database But when we are in a virtual wiki (another schema ) - the 
>> performance are very poor.
>> The average response time is around 5 seconds  per page !!! So at list 2 
>> times slower.
>>
>> We are using MySQL and glassfish 3.1 - and we have done tuning on the MySQL 
>> database and glassfish.
>>
>> We have used a profiling tool and see that the request send to the virtual 
>> environment are raising a big amount of exception ...
>> The problem is not on the database part (even if they are many SQL 
>> calls to visualize only one page) The problem seems really to be on the 
>> virtual environment configuration OR in a file corruption ...
>>
>> I have attached the exception list . (exception_list.xls) From this 
>> is it possible to know if we have an installation issue ??
>>
>> Does someone have experience of very good performance with xwiki  ? ( 
>> response time #1second per page) In that case
>
> I can tell you fore sure that there's no performance issue with multiwikis. 
> Each wiki has its own DBs so it's really fast.
>
> Maybe you don't have indexes set on the subwikis? I think we started setting 
> them automatically in XE 3.2 but it's worth checking, see:
> http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Database+Administr
> ation
>
> Thanks
> -Vincent
>
> ___
> users mailing list
> users@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/users
> ___
> users mailing list
> users@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/users



--
Thomas Mortagne
___
users mailing list
users@xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
___
users mailing list
users@xwiki.org
http://lists.xwiki.org/mailman/listinfo/users

Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

[Merlin Olivier]

The only message I have in the logs are:
[#|2012-05-16T09:49:17.787+0200|INFO|glassfish3.1.1|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=17;_ThreadName=Thread-2;|2012-05-16
 09:49:17,786 [http://10.10.166.184:8080/xwiki/bin/view/Main/AllDocs] ERROR 
o.i.ObservationContextListener - Can't find any begin event corresponding to 
[class org.xwiki.bridge.event.ActionExecutedEvent (view)]

I know there is a patch in xwiki 4.x that corrects this ... so we will probably 
correct this.

Anyway - all the exceptions I have sent to you are not neutral at all - they 
are consuming a lot of CPU / response time (even if they are normal) ...
And you are right - I was thinking at the "No row with the given identifier 
exists" exception - to say that something is not good in our installation.


When I have time I will redo some tuning/profiling on the xwiki - but I'm a bit 
skeptical about performance when I see the execution profile ...
I will try to follow the performance tips on the xwiki.org also

http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances

I'm curious to know on which HW you have deployed the xwiki.org and if you are 
using virtualization also ...
Because we are using more and more xwiki ... and 5 seconds for each page seems 
too long for end user for me

So I you have any advice to improve performance - I will try 
Perhaps a good idea is to do a full xar export on the 4.0 version and to 
reinstall everything ??? 

Best regards,
Olivier



-Original Message-
From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On Behalf Of 
Thomas Mortagne
Sent: Wednesday, May 16, 2012 9:41 AM
To: XWiki Users
Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

On Wed, May 16, 2012 at 9:40 AM, Thomas Mortagne  
wrote:
> On Wed, May 16, 2012 at 9:37 AM, Thomas Mortagne 
>  wrote:
>> On Wed, May 16, 2012 at 9:03 AM, Merlin Olivier 
>>  wrote:
>>> Ok - please follow this link: 
>>> http://dl.dropbox.com/u/19876503/exception_list.xlsx
>>> I suspect we have something wrong in the installation.
>>
>> Would be better to have the log. Exception out of context is not always 
>> helpful.
>
> Most of the "Can't find descriptor for the component" are perfectly 
> normal and have nothing to do with multiwiki you should have exactly 
> the same in single wiki.
>
> I tough you got lots of exceptions in the log but those are normal 
> exceptions which are never logged. If you don't get anything in the 
> log then all that is probably normal.
>

The "No row with the given identifier exists" are probably not expected but 
hard to tell out of context.

>>
>>>
>>> Best regards,
>>> Olivier
>>>
>>> -Original Message-
>>> From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On 
>>> Behalf Of Thomas Mortagne
>>> Sent: Wednesday, May 16, 2012 8:52 AM
>>> To: XWiki Users
>>> Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance 
>>> issues
>>>
>>> On Tue, May 15, 2012 at 6:55 PM, Merlin Olivier 
>>>  wrote:
>>>> Hello Vincent,
>>>>
>>>> I have recreated the indexes on one multiwiki ... and I get the 
>>>> same performance results Each page are loaded in 4-5 seconds.
>>>> In fact there was an improvement in the DB usage but it was not a lot 
>>>> before.
>>>> The API breakdown shows me:
>>>> - 1200 ms in the classloading
>>>> - 930 ms in the XML processing
>>>> - 580 ms for the xwiki API
>>>> - 420 ms for JDBC
>>>> - 350 ms for wiki model
>>>> (...)
>>>>
>>>> The problem of performance is really on the application server code 
>>>> response time - not in the database.
>>>> - the problem could be in the big amount of exception raised (is it 
>>>> normal ?)
>>>
>>> No it's not normal and yes it's probably related.
>>>
>>>> - Glassfish 3.1 issues ? We use a 64bits deployment with a sun
>>>> jdk1.6.0_26 Xms = Xmx = 2Gb
>>>> - 3.2 version
>>>> - the HW not sufficient ? We use a VMware with a Redhat Linux 5.7
>>>>        - 2 core CPU ( X5650  @ 2.67GHz )
>>>>        - 4Go RAM
>>>>
>>>> Any ideas ? Good config to test ?
>>>> My next ideas would be :
>>>> - migrate to Weblogic
>>>> - migrate to xwiki 4.x
>>>>
>>>> I can send you all the list of exception - it consumes a lot of CPU for me 
>>>> ...
>>>
>>> Yes please or put them some

Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

[Merlin Olivier]

Ok - we don't use some custom scripts ...
We have followed all the performance tips (rendering caching, cache size 
increase, statistics removing ) ... and it seems better
For statistics we use google analytics - to keep stats 

Thanks for your information that HW doesn't matter that much ...
If you have some preferred deployment (application server) - let us know ... I 
can switch easily ...

Finally the user experience is better - but I cannot give you numbers - I have 
deactivated the profiling tool I was using (because I have to go back to my 
normal activity)

BR
Olivier 

-Original Message-
From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On Behalf Of 
Thomas Mortagne
Sent: Wednesday, May 16, 2012 10:32 AM
To: XWiki Users
Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

Note that sometimes the issue come from some custom script which takes way too 
much resources and slow down everything.

On Wed, May 16, 2012 at 10:30 AM, Thomas Mortagne  
wrote:
> On Wed, May 16, 2012 at 10:23 AM, Merlin Olivier 
>  wrote:
>> The only message I have in the logs are:
>> [#|2012-05-16T09:49:17.787+0200|INFO|glassfish3.1.1|javax.enterprise.
>> system.std.com.sun.enterprise.server.logging|_ThreadID=17;_ThreadName
>> =Thread-2;|2012-05-16 09:49:17,786 
>> [http://10.10.166.184:8080/xwiki/bin/view/Main/AllDocs] ERROR 
>> o.i.ObservationContextListener - Can't find any begin event 
>> corresponding to [class org.xwiki.bridge.event.ActionExecutedEvent 
>> (view)]
>>
>> I know there is a patch in xwiki 4.x that corrects this ... so we will 
>> probably correct this.
>>
>> Anyway - all the exceptions I have sent to you are not neutral at all - they 
>> are consuming a lot of CPU / response time (even if they are normal) ...
>
> You are supposed to get the exact same exception in single wiki too so 
> it's not what make a difference.
>
>> And you are right - I was thinking at the "No row with the given identifier 
>> exists" exception - to say that something is not good in our installation.
>
> If those are not logged they are probably normal. I guess that's when 
> we ask hibernate to load some document which does not exists.
>
>>
>>
>> When I have time I will redo some tuning/profiling on the xwiki - but I'm a 
>> bit skeptical about performance when I see the execution profile ...
>> I will try to follow the performance tips on the xwiki.org also
>>
>> http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances
>>
>> I'm curious to know on which HW you have deployed the xwiki.org and if you 
>> are using virtualization also ...
>
> Yes we are suing virtualization too but I don't know much more about 
> the HW sorry.
>
>> Because we are using more and more xwiki ... and 5 seconds for each 
>> page seems too long for end user for me
>
> That's not normal for sure and AFAIK you don't need to have a speccial 
> HW to have decent performances unless you have a huge load.
>
>>
>> So I you have any advice to improve performance - I will try Perhaps 
>> a good idea is to do a full xar export on the 4.0 version and to reinstall 
>> everything ???
>
> I doubt it would change anything.
>
>>
>> Best regards,
>> Olivier
>>
>>
>>
>> -Original Message-
>> From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On 
>> Behalf Of Thomas Mortagne
>> Sent: Wednesday, May 16, 2012 9:41 AM
>> To: XWiki Users
>> Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance 
>> issues
>>
>> On Wed, May 16, 2012 at 9:40 AM, Thomas Mortagne  
>> wrote:
>>> On Wed, May 16, 2012 at 9:37 AM, Thomas Mortagne 
>>>  wrote:
>>>> On Wed, May 16, 2012 at 9:03 AM, Merlin Olivier 
>>>>  wrote:
>>>>> Ok - please follow this link:
>>>>> http://dl.dropbox.com/u/19876503/exception_list.xlsx
>>>>> I suspect we have something wrong in the installation.
>>>>
>>>> Would be better to have the log. Exception out of context is not always 
>>>> helpful.
>>>
>>> Most of the "Can't find descriptor for the component" are perfectly 
>>> normal and have nothing to do with multiwiki you should have exactly 
>>> the same in single wiki.
>>>
>>> I tough you got lots of exceptions in the log but those are normal 
>>> exceptions which are never logged. If you don't get anything in the 
>>> log then all that is probably normal.
>>>
>>
>> The "No row with the giv

Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

[Merlin Olivier]

Hello,

I will redo some tests with my profiler tool to give you more precision ...

Best regards,
Olivier

-Original Message-
From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On Behalf Of 
Sergiu Dumitriu
Sent: Thursday, May 17, 2012 4:36 AM
To: XWiki Users
Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance issues

On 05/15/2012 12:55 PM, Merlin Olivier wrote:
> Hello Vincent,
>
> I have recreated the indexes on one multiwiki ... and I get the same 
> performance results Each page are loaded in 4-5 seconds.
> In fact there was an improvement in the DB usage but it was not a lot before.
> The API breakdown shows me:
> - 1200 ms in the classloading

Does it spend so much time in classloading for each request, or just for the 
first one?

> - 930 ms in the XML processing
> - 580 ms for the xwiki API
> - 420 ms for JDBC
> - 350 ms for wiki model
> (...)
>
> The problem of performance is really on the application server code response 
> time - not in the database.
> - the problem could be in the big amount of exception raised (is it 
> normal ?)
> - Glassfish 3.1 issues ? We use a 64bits deployment with a sun 
> jdk1.6.0_26 Xms = Xmx = 2Gb
> - 3.2 version
> - the HW not sufficient ? We use a VMware with a Redhat Linux 5.7
>   - 2 core CPU ( X5650  @ 2.67GHz )
>   - 4Go RAM
>
> Any ideas ? Good config to test ?
> My next ideas would be :
> - migrate to Weblogic
> - migrate to xwiki 4.x
>
> I can send you all the list of exception - it consumes a lot of CPU for me ...
>
>
> Thanks for your help
> Olivier
>
>
>
>
> -Original Message-
> From: users-boun...@xwiki.org [mailto:users-boun...@xwiki.org] On 
> Behalf Of Vincent Massol
> Sent: Tuesday, May 15, 2012 5:56 PM
> To: XWiki Users
> Subject: Re: [xwiki-users] [xwiki 3.2] virtual user big performance 
> issues
>
> Hi Merlin,
>
> On May 15, 2012, at 5:41 PM, Merlin Olivier wrote:
>
>> Hello,
>>
>> On the xwiki  3.2, we have decided to use the virtual xwiki
>> (multi-tenant) feature as it should be better for scalability and 
>> performance 
>> http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Virtualization
>>
>>
>> "Each wiki has its own database schema, thus allowing for a complete data 
>> separation between the different wikis in the same instance.
>>
>> The main advantage of multi-tenancy is performance optimization."
>> Problem:
>> The main wiki has a relative good level of performance - with a 
>> response time around 2.5 second to browse a page So we are in the 
>> xwiki database But when we are in a virtual wiki (another schema ) - the 
>> performance are very poor.
>> The average response time is around 5 seconds  per page !!! So at list 2 
>> times slower.
>>
>> We are using MySQL and glassfish 3.1 - and we have done tuning on the MySQL 
>> database and glassfish.
>>
>> We have used a profiling tool and see that the request send to the virtual 
>> environment are raising a big amount of exception ...
>> The problem is not on the database part (even if they are many SQL 
>> calls to visualize only one page) The problem seems really to be on the 
>> virtual environment configuration OR in a file corruption ...
>>
>> I have attached the exception list . (exception_list.xls) From this 
>> is it possible to know if we have an installation issue ??
>>
>> Does someone have experience of very good performance with xwiki  ? ( 
>> response time #1second per page) In that case
>
> I can tell you fore sure that there's no performance issue with multiwikis. 
> Each wiki has its own DBs so it's really fast.
>
> Maybe you don't have indexes set on the subwikis? I think we started setting 
> them automatically in XE 3.2 but it's worth checking, see:
> http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Database+Administr
> ation
>
> Thanks
> -Vincent


--
Sergiu Dumitriu
http://purl.org/net/sergiu/
___
users mailing list
users@xwiki.org
http://lists.xwiki.org/mailman/listinfo/users
___
users mailing list
users@xwiki.org
http://lists.xwiki.org/mailman/listinfo/users