- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: pascal
Subject: Slow indexing... why?
Hi, i have a problem, because my indexing seems real slow.
some data:
/usr/local/dpsearch/sbin/indexer -S
Database statistics
Status Expired Total
-----------------------------
0 242954 243010 Not indexed yet
200 306513 354794 OK
206 84 98 Partial OK
300 43 53 Multiple Choices
301 1242 1451 Moved Permanently
302 7838 10553 Moved Temporarily
303 1 1 See Other
304 2525 4151 Not Modified
400 13 13 Bad Request
401 106 119 Unauthorized
403 55 62 Forbidden
404 356 540 Not found
406 469 531 Not Acceptable
408 0 1 Request Timeout
415 1019 1184 Unsupported Media Type
500 7 13 Internal Server Error
503 2 7 Service Unavailable
504 8 9 Gateway Timeout
-----------------------------
Total 563235 616590
/usr/local/dpsearch/sbin/indexer -n 10
indexer[25296]: {00} indexer from dpsearch-4.30-26042005-mysql started with
'/usr/local/dpsearch/etc/indexer.conf'
indexer[25296]: {01} URL:
http://eshop.mexx.com/is-bin/INTERSHOP.enfinity/eCS/Store/de/
indexer[25296]: {01} ROBOTS: http://eshop.mexx.com/robots.txt
indexer[25296]: {01} robots.txt: 'Disallow /is-bin/'
indexer[25296]: {01} Deleting
http://eshop.mexx.com/is-bin/INTERSHOP.enfinity/eCS/Store/de/
indexer[25296]: {01} URL:
http://www.tchibo.de/is-bin/INTERSHOP.enfinity/eCS/Store/de/-/EUR/TdTchDisplayProductInformation-Start;sid=KQcw0wDvMNUw90KbNUk6twNmnsq7WFdPyrs=?ProductSKU=M1530
indexer[25296]: {01} ROBOTS: http://www.tchibo.de/robots.txt
indexer[25296]: {01} URL:
http://www.koeln.de/branchenfuehrer/pages/front/eintrag.php/2147/Optik-Lenhart-Koeln.html
indexer[25296]: {01} ROBOTS: http://www.koeln.de/robots.txt
indexer[25296]: {01} URL:
http://www.chefkoch.de/rezepte/38871013073464,71/Pilzsalat.html
indexer[25296]: {01} ROBOTS: http://www.chefkoch.de/robots.txt
indexer[25296]: {01} URL:
http://www.w3.org/TR/DOM-Level-3-Core/copyright-notice.html
indexer[25296]: {01} ROBOTS: http://www.w3.org/robots.txt
indexer[25296]: {01} URL:
http://www.w3.org/2004/Talks/0608-StAugustin-IH/15.html
indexer[25296]: {01} Done (8 seconds, 5 documents, 28705 bytes, 3.50
Kbytes/sec.)
indexer[25296]: {00} Total 12 seconds, 5 documents, 28705 bytes, 2.34
Kbytes/sec, 2.40 sec/doc, 5741 bytes/doc.
2,4 sek per doc seeems really slow for me, at start time, I had about 0,2-0,4
sec per doc
i made the DELAY KEY WRITE suggestion for optimizing indexing speed, but now i
ran out of ideas.
Can someboy help me?
thank you,
pascal
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Read the full topic here:
http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;post=
