Re: [Harbour] Slow create index under threads

On Wed, 12 May 2010, Maurilio Longo wrote: Hi, > Przemysław Czerpak wrote: > > If you have to keep existing OS then you can try to increase the memory > > used by Harbour RDD driver for buffers during indexing. > > In src/rdd/dbfcd/dbfcdx1.c[9191] I described how DBFCDX allocates memory. > > As y

Re: [Harbour] Slow create index under threads

Przemyslaw, Przemysław Czerpak wrote: > If you have to keep existing OS then you can try to increase the memory > used by Harbour RDD driver for buffers during indexing. > In src/rdd/dbfcd/dbfcdx1.c[9191] I described how DBFCDX allocates memory. > As you can see in the comment it should not alloca

Re: [Harbour] Slow create index under threads

Hi, The time of creating very long index is mostly reduced by disk IO. In systems like MS-Windows it can be performance killer due the very inefficients method of buffer flushing. Creating more threads for simultaneous indexing only increases the problem becaue it increase number of non linear buf

Re: [Harbour] Slow create index under threads

Hi, On 2010.05.11 18:52, rafa wrote: Second; A question What advantages are there in using indexes MEMORY? Rafa, you've completely changed the original idea: >> OrdCreate("mem:file.ext", ...) >> I've sent C code for copying file from/to memory a few days ago. > Ok, thanks! > I add MEMORY a co

Re: [Harbour] Slow create index under threads

Hello All, First; Great! I have noticed is an improvement of about 2 minutes of not using Hb_detach () and remove the threads in the creation of indices. Second; A question What advantages are there in using indexes MEMORY? I do not appreciate any improvement is more, we analyzed under Windows X

Re: [Harbour] Slow create index under threads

Hello, Mindaugas Kavaliauskas escribió: > Hi, > > > On 2010.05.11 12:22, rafa wrote: >>> Of cause. Did you expect your code to behave in a different way?... >> I thought he reindex all at once ;-) >> The reason is because only one index reindex a table, while the others >> wait, I do not know. > >

Re: [Harbour] Slow create index under threads

Hi, On 2010.05.11 12:22, rafa wrote: Of cause. Did you expect your code to behave in a different way?... I thought he reindex all at once ;-) The reason is because only one index reindex a table, while the others wait, I do not know. The reason is hb_dbdetach(), hb_dbattach(). Workarea is at

Re: [Harbour] Slow create index under threads

Mindaugas Kavaliauskas escribió: > Hi, > > >> Please could you tell me how to create the index in memory and then >> flush to disk? > > OrdCreate("mem:file.ext", ...) > I've sent C code for copying file from/to memory a few days ago. Ok, thanks! I add MEMORY a command INDEX , And I'll post later

Re: [Harbour] Slow create index under threads

Hi, Please could you tell me how to create the index in memory and then flush to disk? OrdCreate("mem:file.ext", ...) I've sent C code for copying file from/to memory a few days ago. These crea() threads do not help to do any valuable job. Function aCreateIndexe() starts a separate crea()

Re: [Harbour] Slow create index under threads

Hello Mindaugas, > > It seems disk io is the bottle neck. You can try to use memio driver > to create index in memory, and later write it back to disk. > Please could you tell me how to create the index in memory and then flush to disk? > >> aadd( aThreads, hb_threadStart( @crea(), cAl

Re: [Harbour] Slow create index under threads

CPU. > > Qatan > > > - Original Message - From: "rafa" > To: "Harbour Project Main Developer List." > Sent: Monday, 10 de May de 2010 06:20 > Subject: [Harbour] Slow create index under threads > > >> Hello, >> I'm trying to

Re: [Harbour] Slow create index under threads

Hi, On 2010.05.10 12:20, rafa wrote: Well, times; 1 thread= 29 minutes THE WINNER! 4 threads = 93 minutes WoW! 7 threads = 68 minutes 5 Threads = 64 minutes It seems disk io is the bottle neck. You can try to use memio driver to create index in memory, and later write it back to disk.

Re: [Harbour] Slow create index under threads

, 10 de May de 2010 06:20 Subject: [Harbour] Slow create index under threads Hello, I'm trying to do a re indexing using threads, but according to tests under Windows Vista, Quarda Intel Core, 4 Gig RAM, the tests are not entirely satisfactory; The basic idea was to determine the number of

Re: [Harbour] Slow create index under threads

Indexing is rather disk than CPU intensive, meaning the disk throughput, access times and file I/O performance are the limiting factors. For this reason splitting it to multiple threads may only help if you have very fast disk, like SSD, but even this needs testing to see. I didn't test it,

[Harbour] Slow create index under threads

Hello, I'm trying to do a re indexing using threads, but according to tests under Windows Vista, Quarda Intel Core, 4 Gig RAM, the tests are not entirely satisfactory; The basic idea was to determine the number of threads that want to implement, N_THREADS, which is passed by parameter, for testing