[ZODB-Dev] Re: Memory Errors reading large ZODB
Gary Poster wrote: you can call cache minimize after a threshold.. maybe every 100 iterations. sounds good, assuming you know you are not writing. Fantastic! My scripts are running without fail now (I used to have to change them to pick up where they left off and re-run about four times.) Thanks! -Jim ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: Understanding conflicts
Dieter Maurer dieter at handshake.de writes: Jim Carroll wrote at 2007-6-12 15:57 +: ... The code that's running essentially does: add message to Persistent list in object get_transaction().commit() send_email() We are using Zodb 3.3a3, and ZEO 2.3a3. I've read that when a transaction retries, it re-runs code on the server side. The ZODB by itself does not rerun anything. Zope (its ZPublisher) retries a request up to three times when it fails due to a ConflictError. I'm not using Zope. I've got a quixote application that's using ZODB and ZEO. Somehow, the code that adds the message to the persistent list is running more than once. I have read that ZEO will re-run python code on a retry, and I think this is what's causing the duplicated data. In my mind, what I need is some sort of assurance that the Persistent List is only being appened to once. For some reason the commit() is causing it to be appended more than once when there's a conflict error. How can I isolate my transactions so that a conflict aborts the current transaction before re-trying? Thanks, -Jim ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Error starting simplest ZEO instance
Hi I'm having trouble getting my web server started. It's starting ZEO, but zeo never completes the startup process. The zdctl.py is wating forever for a runzeo which fails instantly with error code 2. Im running ZODB 3.3.1 on Python 2.4 on Ubuntu. I've tried a stand-alone startup to see what's going on and it tells me: [EMAIL PROTECTED]:~/tmp# zdctl.py -p runzeo.py -a -f Data.fs start Error: invalid value for -m '18': invalid literal for int(): 18 For help, use /usr/bin/zdrun.py -h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (continues forever) I stepped through runzeo (and zdctl) with pdb, and figured out that it's actually running: /usr/bin/python /usr/bin/zdrun.py -S schema.xml -b 10 -s zdsock -u www -x 0,2 runzeo.py -a -f Data.fs -t 10 -m 18 the 18 that it's complaining about really is a valid int. If I remove that parameter I get: [EMAIL PROTECTED]:~/tmp# /usr/bin/python /usr/bin/zdrun.py -S schema.xml -b 10 -s zdsock -u www -x 0,2 runzeo.py -a -f Data.fs -t 10 But I never get the messages that tell me that it's starting up correctly. How can I get zdrun to accept -m 18 happly? Thanks, -Jim ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: Error starting simplest ZEO instance
That leaves me without any clues why zdrun is failing on my new machine. So, what's the best way to troubleshoot why zdrun just waits forever? I'm currently trying to figure out how dulcinea decides what to hand to zeod (zeoctl.) I added the following line to my zeo.conf: runner transcript /www/log/zeo.runner /runner and the transcript told me that it was looking for a script in the wrong directory. Problem solved! ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Memory Errors reading large ZODB
Hi, I'm trying to iterate over all the comments in my discussion .fs and I'm getting different errors each time. The latest is Memory Error (direct fs access), sometimes it's trouble starting a thread (access through ZEO.) I'm running ZODB 3.3 on a FreeBSD system. I've tried doing periodic commit() and cacheGC(), but the memory use just keeps growing until things blow up. My question is: How can I get my zodb client to forget objects very shortly after I read them? (I'm just counting objects, and want to get them out of memory right away.) I don't think I can del object because that would also remove it from the database... isn't that correct? my main loop is below... Thanks, -Jim for groupid, group in db.group_db.root.items(): group_short_name = group.get_user_id() # touch each discussion once for item in group.get_blog().get_items(): for comment in item.get_all_comments(): author_id = comment.author.get_user_id() counts[author_id] = counts.get(author_id, 0) + 1 transaction_commit() db.connection.cacheGC() print counted in , groupid # ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Understanding conflicts
Hi, I've got a problem with a website where occasionally we send multiple e-mail messages when really we only want to send one. The last time it happened, I noticed that there was a Conflict Error at the same time as the e-mail send. The code that's running essentially does: add message to Persistent list in object get_transaction().commit() send_email() We are using Zodb 3.3a3, and ZEO 2.3a3. I've read that when a transaction retries, it re-runs code on the server side. How can I verify that that's what my setup would do? (my maillog on my server side doesn't show the duplicate e-mails, but I might not be looking far enough back in the logs.) Thanks ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
