On Sat, 29 Nov 2014, Nikolaus Rath wrote:
On 11/29/2014 09:49 AM, Shannon Dealy wrote:
Package: s3ql
Version: 2.11.1+dfsg-1
Severity: critical
Justification: causes serious data loss
Dear Maintainer,
While running rsync to backup data to an s3ql file system mounted from Amazon's
S3 services, the internet connection failed, resulting in the following
[snip]
Chances are good that this can be done, so don't give up hope yet.
Best,
-Nikolaus
Attached are both the mount and fsck log files, stripped of data from
other (irrelevent) sessions and with slight mods to remove references to
my S3 file system info (bucket/prefix).
I have left the original bucket and local cache unmodified except for
whatever changes occured during my fsck attempt in case they can be of
use in debugging this.
Regards,
Shannon C. Dealy | DeaTech Research Inc.
de...@deatech.com | - Custom Software Development -
USA Phone: +1 800-467-5820 | - Natural Building Instruction -
numbers : +1 541-929-4089 | www.deatech.com
2014-11-28 18:08:52.029 31159:MainThread (name)s.determine_threads: Using 8
upload threads.
2014-11-28 18:08:52.032 31159:MainThread (name)s.main: Autodetected 4040 file
descriptors available for cache entries
2014-11-28 18:09:00.435 31159:MainThread (name)s.get_metadata: Using cached
metadata.
2014-11-28 18:09:00.439 31159:MainThread (name)s.main: Mounting filesystem...
2014-11-28 18:09:00.448 31163:MainThread (name)s.detach_process_context:
Daemonizing, new PID is 31167
2014-11-29 02:14:44.787 31167:Thread-9 (name)s.wrapped: Encountered
ConnectionTimedOut exception (send/recv timeout exceeded), retrying call to
ObjectW.close for the 3-th time...
2014-11-29 10:49:36.636 31167:Thread-10 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 509, in create_connection
raise err
File "/usr/lib/python3.4/socket.py", line 500, in create_connection
sock.connect(sa)
OSError: [Errno 113] No route to host
2014-11-29 10:50:11.907 31167:Thread-9 (name)s.exchook: Unhandled top-level
exception during shutdown (will not be re-raised)
2014-11-29 10:50:11.907 31167:Thread-9 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 491, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 530, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
2014-11-29 10:51:02.025 31167:Thread-3 (name)s.exchook: Unhandled top-level
exception during shutdown (will not be re-raised)
2014-11-29 10:51:02.026 31167:Thread-3 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 491, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 530, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
2014-11-29 10:51:48.500 31167:Thread-8 (name)s.exchook: Unhandled top-level
exception during shutdown (will not be re-raised)
2014-11-29 10:51:48.500 31167:Thread-8 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 491, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 530, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
2014-11-29 10:51:59.022 31167:Thread-7 (name)s.exchook: Unhandled top-level
exception during shutdown (will not be re-raised)
2014-11-29 10:51:59.022 31167:Thread-7 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 491, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 530, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
2014-11-29 10:52:04.000 31167:Thread-6 (name)s.exchook: Unhandled top-level
exception during shutdown (will not be re-raised)
2014-11-29 10:52:04.001 31167:Thread-6 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 491, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 530, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
2014-11-29 10:52:11.890 31167:Thread-5 (name)s.exchook: Unhandled top-level
exception during shutdown (will not be re-raised)
2014-11-29 10:52:11.891 31167:Thread-5 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 491, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 530, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
2014-11-29 11:04:07.908 31167:Thread-4 (name)s.exchook: Unhandled top-level
exception during shutdown (will not be re-raised)
2014-11-29 11:04:07.909 31167:Thread-4 (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 413, in connect
self._sock = self.ssl_context.wrap_socket(self._sock,
server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 364, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 577, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 804, in do_handshake
self._sslobj.do_handshake()
OSError: [Errno 113] No route to host
2014-11-29 11:04:10.942 31167:MainThread (name)s.destroy: Unable to flush
cache, no upload threads left alive
2014-11-29 11:04:15.961 31167:MainThread (name)s.unmount: Unmounting file
system...
2014-11-29 11:04:15.970 31167:MainThread (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/mount.py", line 226, in main
raise exc_inst.with_traceback(exc_tb)
File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 404, in _upload_loop
self._do_upload(*tmp)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 431, in _do_upload
% obj_id).get_obj_size()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 258, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 477, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 471, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 636, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 845, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 642, in _send_request
headers=headers, body=BodyFollowing(body_len))
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 477, in
send_request
self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 504, in
co_send_request
self.connect()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 408, in connect
self._sock = socket.create_connection((self.hostname, self.port))
File "/usr/lib/python3.4/socket.py", line 509, in create_connection
raise err
File "/usr/lib/python3.4/socket.py", line 500, in create_connection
sock.connect(sa)
OSError: [Errno 113] No route to host
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/mount.s3ql", line 9, in <module>
load_entry_point('s3ql==2.11.1', 'console_scripts', 'mount.s3ql')()
File "/usr/lib/s3ql/s3ql/mount.py", line 230, in main
unmount_clean = True
File "/usr/lib/python3.4/contextlib.py", line 336, in __exit__
raise exc_details[1]
File "/usr/lib/python3.4/contextlib.py", line 321, in __exit__
if cb(*exc_details):
File "/usr/lib/python3.4/contextlib.py", line 267, in _exit_wrapper
callback(*args, **kwds)
File "/usr/lib/s3ql/s3ql/block_cache.py", line 390, in destroy
os.rmdir(self.path)
OSError: [Errno 39] Directory not empty: '/root/.s3ql/s3:=2F=2F<bucket
name>=2F<file system prefix>-cache'
2014-11-29 19:20:30.411 32100:MainThread (name)s.determine_threads: Using 8
upload threads.
2014-11-29 19:20:30.413 32100:MainThread (name)s.main: Autodetected 4040 file
descriptors available for cache entries
2014-11-29 19:20:41.807 32100:MainThread (name)s.get_metadata: Ignoring locally
cached metadata (outdated).
2014-11-29 19:20:42.798 32100:MainThread (name)s.get_metadata: Downloading and
decompressing metadata...
2014-11-29 19:25:15.137 32100:MainThread (name)s.get_metadata: Reading
metadata...
2014-11-29 19:25:15.144 32100:MainThread (name)s.restore_metadata: ..objects..
2014-11-29 19:25:34.325 32100:MainThread (name)s.restore_metadata: ..blocks..
2014-11-29 19:26:47.814 32100:MainThread (name)s.restore_metadata: ..inodes..
2014-11-29 19:28:39.804 32100:MainThread (name)s.restore_metadata:
..inode_blocks..
2014-11-29 19:29:46.603 32100:MainThread (name)s.restore_metadata:
..symlink_targets..
2014-11-29 19:29:49.307 32100:MainThread (name)s.restore_metadata: ..names..
2014-11-29 19:30:13.694 32100:MainThread (name)s.restore_metadata: ..contents..
2014-11-29 19:32:21.916 32100:MainThread (name)s.restore_metadata:
..ext_attributes..
2014-11-29 19:32:34.055 32100:MainThread (name)s.main: Setting cache size to
58452 MB
2014-11-29 19:32:34.057 32100:MainThread (name)s.main: Mounting filesystem...
2014-11-29 19:32:34.067 32300:MainThread (name)s.detach_process_context:
Daemonizing, new PID is 32304
2014-11-29 11:20:06.840 22883:MainThread (name)s.main: Starting fsck of
s3://<bucket name>/<file system prefix>
2014-11-29 11:20:07.484 22883:MainThread (name)s.main: Using cached metadata.
2014-11-29 11:20:07.551 22883:MainThread (name)s.main: Remote metadata is
outdated.
2014-11-29 11:20:07.553 22883:MainThread (name)s.main: Checking DB integrity...
2014-11-29 11:20:24.678 22883:MainThread (name)s.check: Creating temporary
extra indices...
2014-11-29 11:20:39.794 22883:MainThread (name)s.check_lof: Checking
lost+found...
2014-11-29 11:20:39.795 22883:MainThread (name)s.check_cache: Checking cached
objects...
2014-11-29 11:20:39.797 22883:MainThread (name)s.log_error: Committing block 14
of inode 442809 to backend
2014-11-29 11:21:02.254 22883:MainThread (name)s.log_error: Committing block 16
of inode 442809 to backend
2014-11-29 11:21:24.892 22883:MainThread (name)s.log_error: Committing block 17
of inode 442809 to backend
2014-11-29 11:21:25.524 22883:MainThread (name)s.log_error: Committing block 15
of inode 442809 to backend
2014-11-29 11:21:53.237 22883:MainThread (name)s.log_error: Committing block 19
of inode 442809 to backend
2014-11-29 11:22:02.060 22883:MainThread (name)s.log_error: Committing block 18
of inode 442809 to backend
2014-11-29 11:22:15.130 22883:MainThread (name)s.check_names_refcount: Checking
names (refcounts)...
2014-11-29 11:22:24.769 22883:MainThread (name)s.check_contents_name: Checking
contents (names)...
2014-11-29 11:22:26.220 22883:MainThread (name)s.check_contents_inode: Checking
contents (inodes)...
2014-11-29 11:22:27.933 22883:MainThread (name)s.check_contents_parent_inode:
Checking contents (parent inodes)...
2014-11-29 11:22:28.810 22883:MainThread (name)s.check_objects_refcount:
Checking objects (reference counts)...
2014-11-29 11:22:34.009 22883:MainThread (name)s.check_objects_id: Checking
objects (backend)...
2014-11-29 11:24:38.021 22883:MainThread (name)s.check: Dropping temporary
indices...
2014-11-29 11:24:38.330 22883:MainThread (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/bin/fsck.s3ql", line 9, in <module>
load_entry_point('s3ql==2.11.1', 'console_scripts', 'fsck.s3ql')()
File "/usr/lib/s3ql/s3ql/fsck.py", line 1189, in main
fsck.check()
File "/usr/lib/s3ql/s3ql/fsck.py", line 85, in check
self.check_objects_id()
File "/usr/lib/s3ql/s3ql/fsck.py", line 848, in check_objects_id
self.conn.execute('INSERT INTO obj_ids VALUES(?)', (obj_id,))
File "/usr/lib/s3ql/s3ql/database.py", line 98, in execute
self.conn.cursor().execute(*a, **kw)
File "src/cursor.c", line 231, in resetcursor
apsw.ConstraintError: ConstraintError: PRIMARY KEY must be unique
2014-11-29 16:51:56.251 29276:MainThread (name)s.main: Starting fsck of
s3://<bucket name>/<file system prefix>
2014-11-29 16:51:56.768 29276:MainThread (name)s.excepthook: Uncaught top-level
exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 381, in
_convert_legacy_metadata
meta_new['data'] = meta['data']
KeyError: 'data'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/fsck.s3ql", line 9, in <module>
load_entry_point('s3ql==2.11.1', 'console_scripts', 'fsck.s3ql')()
File "/usr/lib/s3ql/s3ql/fsck.py", line 1111, in main
param = backend.lookup('s3ql_metadata')
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 72, in lookup
meta_raw = self._convert_legacy_metadata(meta_raw)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 383, in
_convert_legacy_metadata
raise CorruptedObjectError('meta key data is missing')
s3ql.backends.common.CorruptedObjectError: meta key data is missing
