On 27/01/2011 03:15, Shane Hathaway wrote: >> Okay, so I'll do: >> >> - e2fsck -f
Hmm, how do I e2fsck a mounted filesystem? The MySQL filesystem could be unmounted (after I shut down MySQL!), so I ran e2fsck on it: e2fsck -f /dev/sdb1 e2fsck 1.41.3 (12-Oct-2008) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/sdb1: 166/16777216 files (10.2% non-contiguous), 8244979/67107513 blocks >> - mysqlcheck -c mysqlcheck -c dev_packed -u root -p Enter password: dev_packed.new_oid OK dev_packed.object_ref OK dev_packed.object_refs_added OK dev_packed.object_state OK dev_packed.pack_object OK >> What logs should I hunt through and what kind of things am I looking >> for? > > /var/log/messages and the like. > Look for kernel-level errors such as > block I/O errors, "oops", and system panics. Any such errors have a > chance of corrupting files. We need to rule out errors at the kernel or > below. None of the above, /var/log/messages is actually pretty empty for the last few days. If it helps, both the app server and database server are VMWare virtual machines... > Next, follow the directions in relstorage/tests/README.txt to create the > 4 test databases. Then run "bin/test -p -m relstorage". All tests > should pass. Okay, first problem, the tests only connect to localhost, which means I can't exactly test as the app server is one machine and the database server is another. However, the two machines are identical, so I setup the buildout on the database server with the new test section added. First up, I get the following failures: IOError: [Errno 2] No such file or directory: '/var/buildout-eggs/RelStorage-1.4.0-py2.6.egg/relstorage/tests/blob/blob_connection.txt' OSError: [Errno 2] No such file or directory: '/var/buildout-eggs/RelStorage-1.4.0-py2.6.egg/relstorage/tests/replicas.conf' My guess is that these files aren't included by setuptools? So, I checked out the 1.4.0 tag and added it as a develop egg in the buildout. Now I get: Running .HFMySQLBlobTests tests: Set up .HFMySQLBlobTests in 0.000 seconds. Running: Ran 70 tests with 0 failures and 0 errors in 9.903 seconds. Running .HPMySQLBlobTests tests: Tear down .HFMySQLBlobTests in 0.000 seconds. Set up .HPMySQLBlobTests in 0.000 seconds. Running: Ran 81 tests with 0 failures and 0 errors in 10.511 seconds. Running zope.testing.testrunner.layer.UnitTests tests: Tear down .HPMySQLBlobTests in 0.005 seconds. Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Running: 78/269 (29.0%) Error in test check16MObject (relstorage.tests.testmysql.HPMySQLTests) Traceback (most recent call last): File "/usr/local/lib/python2.6/unittest.py", line 279, in run testMethod() File "/home/zope/relstorage_co/relstorage/tests/reltestbase.py", line 214, in check16MObject self._dostoreNP(oid, data=data) File "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py", line 202, in _dostoreNP return self._dostore(oid, revid, data, 1, user, description) File "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py", line 190, in _dostore r1 = self._storage.store(oid, revid, data, '', t) File "/home/zope/relstorage_co/relstorage/storage.py", line 565, in store cursor, self._batcher, oid_int, prev_tid_int, data) File "/home/zope/relstorage_co/relstorage/adapters/mover.py", line 453, in mysql_store_temp command='REPLACE', File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line 67, in insert_into self.flush() File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line 74, in flush self._do_inserts() File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line 110, in _do_inserts self.cursor.execute(stmt, tuple(params)) File "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 174, in execute self.errorhandler(self, exc, value) File "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue OperationalError: (1153, "Got a packet bigger than 'max_allowed_packet' bytes") 196/269 (72.9%) Error in test check16MObject (relstorage.tests.testmysql.HFMySQLTests) Traceback (most recent call last): File "/usr/local/lib/python2.6/unittest.py", line 279, in run testMethod() File "/home/zope/relstorage_co/relstorage/tests/reltestbase.py", line 214, in check16MObject self._dostoreNP(oid, data=data) File "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py", line 202, in _dostoreNP return self._dostore(oid, revid, data, 1, user, description) File "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py", line 190, in _dostore r1 = self._storage.store(oid, revid, data, '', t) File "/home/zope/relstorage_co/relstorage/storage.py", line 565, in store cursor, self._batcher, oid_int, prev_tid_int, data) File "/home/zope/relstorage_co/relstorage/adapters/mover.py", line 453, in mysql_store_temp command='REPLACE', File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line 67, in insert_into self.flush() File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line 74, in flush self._do_inserts() File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line 110, in _do_inserts self.cursor.execute(stmt, tuple(params)) File "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 174, in execute self.errorhandler(self, exc, value) File "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue OperationalError: (1153, "Got a packet bigger than 'max_allowed_packet' bytes") Ran 269 tests with 0 failures and 2 errors in 38.398 seconds. Tearing down left over layers: Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Total: 420 tests, 0 failures, 2 errors in 58.929 seconds. This is a little weird, as I have max_allowed_packet set to 16M. Should these tests fail? That said, I don't think this has anything to do with the packing bug as I didn't see any exceptions or, in fact, any logging or output at all from zodbpack, and the only other exceptions seen were the POSKeyErrors... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev