Warwick and Jeff, Thanks for the input. The combination of using the history hack and using a date and time based directory structure has changed the time required to add new files from a linear growth to a constant time. Wouldn't it be a good idea to implement the history hack as the default solution in Slide? Or would that lead to some other negative side effects?
Tweaking the heap size helps with the memory problem, but it only delays the problem. Eventually, no matter what size we use, Slide will run out of memory. As I said before, this is a problem we will solve be a daily restart of Slide. /Pontus -----Original Message----- From: Warwick Burrows [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 04, 2005 3:18 AM To: 'Slide Users Mailing List' Subject: RE: Performance issues Pontus, If you have delta-v versioning enabled then it may help to use the "history collection hack" that basically distributes you're file revision structure in a hierarchical tree in the "/history" path in the file system irrespective of how you choose to layout the files under the "/files" path. Eg. add this to Domain.xml in the <configuration> element: <!-- Speed up access to /history by making hierarchy based on history id eg. fileA.doc was /history/8 stays as /history/8 fileB.doc was /history/18 becomes as /history/1/h8 fileC.doc was /history/137 becomes /history/1/3/h7 --> <parameter name="history-collection-hack">true</parameter> Assuming you have versioning enabled this _may_ save time whenever a file is uploaded as directory searches in the "/history" path are faster. You may also want to check if you have the delta-v auto-version feature on by default. There is an auto-version and auto-version-control parameter in cmdomain.xml that control auto versioning of files. I would assume that it auto-version-control is false then it is disabled so it looks to be disabled in the default Slide distribution. If you have as many files under "/history" as you do in "/files" then versioning is enabled. Warwick > -----Original Message----- > From: Pontus Strand [mailto:[EMAIL PROTECTED] > Sent: Monday, January 03, 2005 5:04 AM > To: Slide Users Mailing List (E-mail) > Subject: Performance issues > > > Hello, > > There has been some talk about performance issues previously > on this list and I would like to return to a couple of them > as they are relevant to my current project. Let me begin by > describing our setup. > > All files will be stored in a file system, most likely > UNIX-based, and all other data will be stored in a database, > MySQL or possibly Oracle. All access to Slide is done via a > RMI-server that uses the Slide WebDAV Client. So far we have > used 2.1rc1 and before that 2.1b2. > > The system that we are developing should be able to handle > tens of thousands of files, if not hundreds of thousands. At > this point, after having run various tests, we're quite > concerned about Slide's performance. > > Let us begin with the famous OutOfMemoryException-problem. > Disabling the cache only delays this issue, leaving us with > the option to increase the memory size for the JVM. Has > anyone looked into it memory leaks related to inserting new > documents? Has any changes been done to that code since > 2.1rc1? We would be willing to look into it, but at the > moment we are hard pressed by a looming deadline so we can't > allocate resources to that problem right now. We are going to > "solve" the problem with an intermediate solution consisting of: > > 1) Increase the JVM memory size > 2) Modify the cache settings > 3) Make sure that Slide is restarted frequently > > The second issue we are having right now is that we see a > linear increase in the time it takes to add files to Slide. I > saw an earlier post where someone who put 2000 files in a > collection ran into to problems and that he/she was > discouraged from having such large collections. And I believe > that our problem is related to this. However, we have tried > to remedy the problem by creating a new collection every 500 > files. This resulted in a small performance benefit but file > 501 still took longer time to add than the first file. We > have the following questions: > > - Is there an optimal maximum size for a collection? > - What is the best way to add large number of files? Does > SLIDE treat collections differently from files in terms of > performance? > - How would one go about adding large number of files to > Slide without having to wait unreasonably long for document inserts? > > Best regards > Pontus Strand > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
