How to quote (was: [s3ql] Re: configuring for Swift?)

On Nov 16 2016, Randy Rue wrote: > Bear with me, especially in the format of this post, I've see that you > prefer a specific style of replying but can't find the link you posted on > the style guide right now. Is there a way to put that information here in > any

Re: [s3ql] configuring for Swift?

On Nov 15 2016, Randy Rue wrote: > Hi Again All, more newbie questions! > > For testing I'm trying to configure for our SwiftStack cluster on the > back end, first using the builtin pytest. I thought you wanted to use S3QL with Amazon S3? > > Having trouble finding the right

Re: [s3ql] newbie questions (and Hello!)

On Nov 15 2016, Randy Rue wrote: > Hello All! > > I've been flailing on finding a way to replicate a large POSIX file > system off site and found this project, unless I'm missing some > dealbreaker it looks like S3QL is a great candidate for what we need. > > I've just spent

Re: [s3ql] Re: Install S3QL on Amazon Linux AMI Current 2015 Edition

On Nov 09 2016, Spice Pearl wrote: > Hi, > > I tried to install S3QL and having issue with the below error. > > > 2016-11-09 14:26:53.407 27952:MainThread root.excepthook: Uncaught > top-level exception: > Traceback (most recent call last): > File

[s3ql] [ANNOUNCE] S3QL 2.21 has been released

Dear all, I am pleased to announce a new release of S3QL, version 2.21. >From the changelog: 2016-10-28, S3QL 2.21 * Added workaround for an issue where S3QL crashes with an "ssl.SSLError: [SSL: BAD_WRITE_RETRY]" exception. This problem has to be fixed in Python, but we now work

Re: [s3ql] Creating versioned backups by combining snapshotting and immutable trees.

On Oct 26 2016, Bart Coninckx wrote: > Hi all, > > I plan to use S3QL for creating a versioned backup. I will use rsync to > sync data, because of it' delta block sync capabilities. To preserve disk > space, the snapshot approach sounds the best. I will also make backup

Re: [s3ql] du file size mismatch

On Oct 26 2016, Jonas Lippuner wrote: > I copied some big files (2 GB each) to my S3QL file system which has > compression and encryption enabled. When I run du to see the file sizes in > the S3QL file system and compare this to the original file sizes (also > obtained with

Re: [s3ql] Any way to stabilize S3QL in my convoluted Amazon Cloud Drive setup?

On Oct 26 2016, Jonas Lippuner wrote: > I'm using a similar setup. Mount ACD locally with acd_cli and then use an > s3ql volume inside the mounted ACD path. The reason for using ACD rather > than S3 is very simple: cost. Well, that explains it. You get what you pay for :-).

Re: [s3ql] Is S3QL and snapshotting production-ready

On Oct 26 2016, Bart Coninckx wrote: > Hi, > > I'n contemplating using S3QL to have a versioned backup system of my > fileserver as a replacement for the tape based backups. Can the S3QL > software be considered reliable and production ready as far as using it for > my

Re: [s3ql] About s3ql file truncate

On Oct 21 2016, jingjie li <530841...@qq.com> wrote: > I test the time of block A(size is 2G)transferred is about 3 > minutes. The time is more than 10 seconds. So I think Block B should > be uploaded to backend. But it didn't. Does it has any other reasons > for this? S3QL doesn't upload blocks

Re: [s3ql] About s3ql file truncate

On Oct 21 2016, jingjie li <530841...@qq.com> wrote: >>Most likely your application is not writing the >data into the file as it >>grows, but it first creates a giant empty file with >all zeros, and then >>it overwrites the zeros with the desired content. >> >> S3QL then starts uploading the

Re: [s3ql] About s3ql file truncate

Hi Jingqie, Why do you send all your emails twice? On Oct 21 2016, jingjie li <530841...@qq.com> wrote: > I'm sorry! > My expression is not correct. > This is not a problem with S3QL. > I just can't understand the flow of S3QL put big file(size > max_obj_size) to > backend. > > > Let me

Re: [s3ql] About s3ql file truncate

On Oct 20 2016, jingjie li <530841...@qq.com> wrote: > ＞The only situation where S3QL creates an object that contains only > zeros > ＞is if you created a file (inside the S3QL file system) that contains > ＞that many zeros. > > I don’t create an object that contains only zeros. Maybe you don't,

Re: [s3ql] About s3ql file truncate

On Oct 20 2016, jingjie li <530841...@qq.com> wrote: > My question is that why s3ql create an all 0 object except the actual > data. The only situation where S3QL creates an object that contains only zeros is if you created a file (inside the S3QL file system) that contains that many zeros.

Re: [s3ql] About s3ql file truncate

On Oct 20 2016, jingjie li <530841...@qq.com> wrote: > Hi Nikolaus > > First, thank you for the great software. > > I have a question about the file truncate. Please help me! > > > The question is that when I set the max-obj-size to 200(2G), then put a > 2.5G file(>max-obj-size) from win7 by

Re: [s3ql] After mounting to AWS using s3ql I can only see lost+found directory

On Oct 18 2016, Michael Hehl wrote: > I cannot see any of my media directories and when trying to add the library > in plex, it cannot see the media folders either. S3QL is a file system that can use S3 to store its data. It's not a tool to browse arbitrary data that you

Re: [s3ql] Recovery using local cache and s3ql_data_ only

On Oct 12 2016, Nikhil Choudhary wrote: >> The s3ql_passphrase object can be recovered from the master key, but >> currently there is no code to do that. > > Not so good to hear, if this is the roadblock that it sounds like, I’m > in for a restoration from scratch. Try the

Re: [s3ql] Recovery using local cache and s3ql_data_ only

On Oct 12 2016, Nikhil Choudhary <nikh...@taligentx.com> wrote: >> On Oct 12, 2016, at 3:41 PM, Nikolaus Rath <nikol...@rath.org> wrote: >> >> So the only objects that are lost are s3ql_metadata, s3ql_passphrase >> s3ql_seq_no no? > > Hel

Re: [s3ql] Recovery using local cache and s3ql_data_ only

On Oct 12 2016, Nikhil Choudhary wrote: > Hi folks, > > Is it possible to recover a filesystem with an up-to-date local cache > (.db and .params files), s3ql_data_ files, and the encryption master > key? That depends on what you want to recover from. > I’m using s3ql with

Re: [s3ql] Tests fail on OS X

On Sep 30 2016, Tobias Ammann wrote: > Hello, > > I tried installing S3QL on my OS X 10.11.6 El Capitan with FUSE for > macOS 3.5.2 today. When I ran the tests they failed in various places > (see attached error logs). Another error (no log) mentioned > /usr/bin/sudo, thus I also

Re: [s3ql] I can't use mkfs.m3ql on www.cynnyspace.com

On Sep 29 2016, mayorgarc...@gmail.com wrote: > I'm testing the provider www.cynnyspace.com which is S3 compatible. I run > mkfs,s3ql but i get that error: > > "/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped return > method(*a, *kw) File "/usr/lib/s3ql/s3ql/backends/s3c.py", line

Re: [s3ql] ec2iamprofile support

On Sep 27 2016, Ronald Lee Aughenbaugh wrote: > I did my best reading through the FAQ/Wiki and documentation but I could > not find a definitive yes/no answer. > > Does S3QL support authentication for mounting s3 buckets via ec2iamprofile, > or environmental variables? I

Re: [s3ql] 2.20 fsck fails with "SyntaxError: unexpected EOF while parsing"

On Sep 26 2016, Chris Davies wrote: > I'm really hoping this is a silly, but the code I've been using to call > s3ql.fsck has been pretty stable for the last three or so S3QL versions. > Any thoughts please? I've got a crashed filesystem and short of downgrading > back to

Re: [s3ql] Re: Mount (fuse process) hanging or crashing.

On Sep 22 2016, Roger Gammans wrote: > File > "/usr/lib/python3.4/ssl.py", line 678, in send >v = self._sslobj.write(data) >

Re: [s3ql] Problem in mounting

On September 5, 2016 10:13:07 AM PDT, Shahriyar Nateghi wrote: >Hello everyone > >This is Shahriyar, I have a big problem, somebody hacked our server >and he mounted my Open Stack SWIFT and I can not revert it and I got >following error. >**I wrote this command**: >

Re: [s3ql] Transport endpoint is not connected

On Aug 25 2016, Martin van Es <mrva...@gmail.com> wrote: > On Wed, Aug 24, 2016 at 7:30 PM, Nikolaus Rath <nikol...@rath.org> wrote: >> Other than that, my only suggestion is to check the list archives. The >> optimal intermediate steps were discussed here some time ago

Re: [s3ql] Transport endpoint is not connected

On Aug 24 2016, Martin wrote: > All of a sudden, my s3ql mountpoints have problems. They to have no > connected endpoints. [...] > 2016-08-24 15:14:57.980 [4285] MainThread: [root] Uncaught top-level > exception: > Traceback (most recent call last): > File "./mount.s3ql",

Re: [s3ql] Filesystem crashed during meta dump

Hi Peter, A: Because it confuses the reader. Q: Why? A: No. Q: Should I write my response above the quoted reply? ..so please quote properly, as I'm doing in the rest of this mail: On Aug 15 2016, Peter Auyeung wrote: > Turns out it does happened again a few times. > >

Re: [s3ql] Can s3ql use the boto3 SDK for its backend implementation?

On Aug 13 2016, pisymbol <pisym...@gmail.com> wrote: > On Thursday, August 11, 2016 at 3:40:48 PM UTC-4, Nikolaus Rath wrote: >> >> On Aug 11 2016, pisymbol <pisy...@gmail.com > wrote: >> > I know this is going to sound strange, but is there anything techni

Re: [s3ql] fsck.s3ql fails with 'primary key must be unique'

On Aug 08 2016, Andy Cress wrote: > I encountered this problem on one particular bucket and cannot seem to > recover the s3ql filesystem. This system had been running for some > time with s3ql-1.17 just fine, but now is stuck. I'm hoping that you > can tell me how to use the

Re: [s3ql] limiting Backend.open_read attempts?

On Aug 01 2016, Ian Rico wrote: > Hello, > > When attempting to mount or fsck to a backend that I can't connect to, the > following messages show: > > Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying > Backend.open_read (attempt 3)... [...] > > Obviously

Re: [s3ql] FileSystem unmounting due to timeout

On Jul 12 2016, Kevin Grace wrote: > Filesystem is unmounting after several timeouts. Any > thoughts/suggestions? Options are: * Fix your network connection * Switch to a different storage server * Increase the tcp-timeout value (S3QL backend option) Best, Nikolaus --

Re: [s3ql] Backup and restore metadata of s3ql

On Jul 02 2016, Peter Auyeung wrote: >>> I am trying to reduce the cloud put get traffic during incremental >> >> Then you should be working with two mounted S3QL file systems (one >> using the local backend and one using the Google Storage backend) and do >> the

Re: [s3ql] Backup and restore metadata of s3ql

On Jul 01 2016, Peter Auyeung <pauye...@connexity.com> wrote: >> On Jul 1, 2016, at 1:43 PM, Nikolaus Rath <nikol...@rath.org> wrote: >> >> On Jul 01 2016, Peter Auyeung <pauye...@connexity.com> wrote: >>>>>> On Jun 29 2016, Peter Auyeung <

Re: [s3ql] s3ql local storage snapshot

On Jun 29 2016, Peter Auyeung wrote: > Is there a way I can take a snapshot of s3ql that running on a lvm on local > storage? s3qlcp should do it, no matter what backend. > I tried s3qlctrl flush cache then upload-meta without luck That's not enough information for

Re: [s3ql] Backup and restore metadata of s3ql

AM UTC-7, Nikolaus Rath wrote: >> On Jun 29 2016, Peter Auyeung <pauy...@connexity.com > >> wrote: >>> Is there a way to backup and restore metadata of s3ql? >> >> You could copy/restore the s3ql_metadata object in your storage backend, >>

Re: [s3ql] Backup and restore metadata of s3ql

On Jun 29 2016, Peter Auyeung wrote: > Is there a way to backup and restore metadata of s3ql? You could copy/restore the s3ql_metadata object in your storage backend, or the *.db file in your --cachedir. > I wonder if it is possible to restore metadata of a particular

Re: [s3ql] s3ql.backends.s3c.HTTPError: 404 Not Found

On Jun 24 2016, Fabio Grasso wrote: > Hello Nikolaus, > >> So yes, there is certainly something wrong here. Are you sure you are >> running all three commands with exactly the same storage url, the same >> authinfo file, and the same cache directory? > > Yes, I'm sure and

Re: [s3ql] s3ql.backends.s3c.HTTPError: 404 Not Found

On Jun 23 2016, Fabio Grasso wrote: > Hello Nikolaus, > I was not able to find a simple way to get the file list from Cyberduck and > other clients will crash after a while (there are a lot of files). > > However I am inclined to think that this is a problem related to

Re: [s3ql] Serious issues with S3QL and Amazon S3 storage backend - "Transport Endpoint is not Connected"

On Jun 22 2016, Brandon Orwell wrote: >> No, that's not a coincidence. The problem is with Python's OpenSSL >> integration, so disabling SSL ensures that this problem will not >> happen. >> > > It it know if this happens only in particular versions of OpenSSL or is the >

Re: [s3ql] Serious issues with S3QL and Amazon S3 storage backend - "Transport Endpoint is not Connected"

Hi Brandon, A: Because it confuses the reader. Q: Why? A: No. Q: Should I write my response above the quoted reply? ..so please quote properly, as I'm doing in the rest of this mail: On Jun 21 2016, Brandon Orwell wrote: > Just so everyone knows, since disabling SSL via

Re: [s3ql] Takes a long time to umount s3ql and recommended cache size

On Jun 20 2016, Peter Auyeung wrote: > May I know the recommended cache size of s3ql. There's no recommendation, you should pick whatever best suits your use-case. > I am getting a very slow write and read performance on s3ql Running contrib/benchmark.py should give

Re: [s3ql] Slow rsync from local s3ql to google s3ql

Hi Peter, A: Because it confuses the reader. Q: Why? A: No. Q: Should I write my response above the quoted reply? ..so please use proper quoting as I'm doing below! On Jun 20 2016, Peter Auyeung wrote: >> > s3qlstat on google storage: >> > Preparing test data... >> >

Re: [s3ql] Slow rsync from local s3ql to google s3ql

On Jun 20 2016, Peter Auyeung wrote: > I am trying to rsync between s3ql on a local disk to another s3ql on google > storage > The rsync is SUPER slow with pushing 200GB over a week or so. > > s3qlstat on google storage: > Preparing test data... > Measuring throughput

Re: [s3ql] Re: Check integrity of s3ql data files

On Jun 18 2016, Ilya wrote: > BTW, I've checked all data files with "file" utility (/usr/bin/file on most > linux distribution). > Most files are of type "data" while some of files are of type "8086 > relocatable (Microsoft)". > I've suspect that such files could be

Re: [s3ql] Serious issues with S3QL and Amazon S3 storage backend - "Transport Endpoint is not Connected"

On Jun 19 2016, Brandon wrote: > Got a crash again - this time, after upgrading to 2.18, it was stable > for [...] Thanks for reporting! Unfortunately this is a known problem, see https://bitbucket.org/nikratio/s3ql/issues/87/retry-on-ssl-bad_write_retry. Only known reliable

Re: [s3ql] Serious issues with S3QL and Amazon S3 storage backend - "Transport Endpoint is not Connected"

On Jun 19 2016, Brandon wrote: > Actually, ignore the max object size question - I read your response to my > other question about billing, and see about the (file size)/(object size) > GET/PUT ratio. I will adjust accordingly for the size of files on average I > store to

Re: [s3ql] Serious issues with S3QL and Amazon S3 storage backend - "Transport Endpoint is not Connected"

On Jun 17 2016, Brandon Orwell wrote: > I've been using S3QL for a few days now, and whenever I am copying over > large amounts of data the mount point seems to 'lock up' for a period of > time (as well as anything else trying to access it), and then I start > getting

Re: [s3ql] Experience with actual monthly charges with Amazon S3 and Google Cloud Storage

On Jun 14 2016, Brandon wrote: > 1) How does S3QL work with using Google Cloud's "Nearline" storage > option? I don't know anything about "Nearline", but I'll try to answer your questions: > Does it write all the data once and just forget it - that is, not needing > to

Re: [s3ql] Local metadata is corrupted.

On May 24 2016, Prakash Autade wrote: > Local metadata is corrupted. Remove or repair the following files manually > and re-run fsck: > /var/cache/s3/store/s3:=2F=2Fstore.db (corrupted) > /var/cache/s3/store/s3:=2F=2Fstore.param (intact) > > > For above, I visited

Re: [s3ql] oauth credentials for Amazon Cloud Drive?

On May 23 2016, Justin Refice wrote: > I'm working on a branch of S3QL to add the Amazon Cloud Drive (ACD) support > discussed in " > https://bitbucket.org/nikratio/s3ql/issues/130/add-support-for-amazon-cloud-drive-api;. > > > I wanted to reach out to ask if a specific S3QL

Re: [s3ql] Suggested compression algorithm for archive already compressed files

On May 16 2016, Fabio Grasso wrote: > Hello Nikolaus, > >> How about none? > > Perhaps I missed something, is it possible to set S3QL to not use > compression? How? Perhaps by specifying "none" as the compression method? The users guide and the mount.s3ql --help

Re: [s3ql] Suggested compression algorithm for archive already compressed files

On May 16 2016, Fabio Grasso wrote: > Hello, > I had to use S3QL to archive files that are already compressed with LZ1, so > compressing again with S3QL is pretty much useless. > > Which parameter do you suggested to avoid compression and use CPU/RAM as > little as

Re: [s3ql] would 2.18 build on ubuntu 12.04 launchpad?

On May 11 2016, Peter Auyeung wrote: > Would there be deb pkg build of s3ql 2.18 for ubuntu 12.04? No, IIRC the Python stack is too old. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC

Re: [s3ql] s3ql 2.18 ubuntu 14.04 build_ext failed

On May 05 2016, Peter Auyeung wrote: > Did anyone has luck on build_ext? > I installed s3ql 2.18 on ubuntu 14.04 from deb package and need to run > benchmark.py but missing 's3ql.deltadump' You need to run setup.py build_cython first. But isn't benchmark.py already

Re: [s3ql] Sync s3ql incrementally

On May 05 2016, Peter Auyeung wrote: > What is the recommended way to sync s3ql filesystems? > > I have two s3ql filesystems, one on local disk and the other on gs and > would like to sync the local s3ql to the one on gs. > > Should I use rsync or clone_fs.py? Use rsync.

Re: [s3ql] 1 error and 1 failure in tests

On Apr 28 2016, Andrei wrote: > Hi, > > Trying to do a manual install on Ubuntu 16.04. > > Running the tests I got 1 error and 1 failure: http://pastebin.com/LKz8DC6h > > Anyone can offer some insights? Update your python-llfuse module. Best, -Nikolaus -- GPG encrypted

Re: [s3ql] S3QL Mount limit

On Apr 28 2016, Prakash Autade wrote: > Hello Nikolaus, > > We are using S3QL to mount AWS S3 > > We are using it with almost no challenge. > > Here I would like to clear one query related to mount points > > How many S3 buckets can be mounted on single server using S3QL

Re: [s3ql] Moved s3ql filesystem, not mountable/fskable (Invalid object header)

On Apr 25 2016, chris wrote: > I had an s3ql filesystem on a hubic account and transferred it to Amazon > with rclone (https://github.com/ncw/rclone) - it seemed to work fine. https://bitbucket.org/nikratio/s3ql/wiki/FAQ#!i-would-like-to-use-s3ql-with-hubic-but > > However,

[s3ql] [ANNOUNCE] Issues with Ubuntu PPA

Dear all, Please note that there have been issues with the automatic update of the Ubuntu PPA at https://launchpad.net/~nikratio/+archive/s3ql. I do not have a lot of time to look into this (and I'm not using Ubuntu myself anymore, so my interest is limited as well), but it seems to be that this

Re: [s3ql] Avoid or postpone "committing block" section of fsck.s3ql

On Apr 21 2016, Chris Davies wrote: > Hello, > > I'm using S3QL with OVH Cloud Storage (no, that's not Hubic) and it's > working mostly very nicely, thank you. This is running S3QL from Debian > unstable (version "2.17.1+hg2+dfsg-3"). > > Unfortunately I'm getting around two

Re: [s3ql] Problem after upgrading to s3ql 2.17.1

On Apr 11 2016, Chris Croome wrote: > Hi > > On a server running Debian stretch, after upgrading to 2.17.1 I tried to > upgrade the filesystem and got this error: The only upgrade paths that are guaranteed to work are from one minor S3QL release to the next (e.g. 2.16

Re: [s3ql] Error of FSCK

On Apr 05 2016, Nikolaus Rath <nikol...@rath.org> wrote: > On Apr 04 2016, Kevin Grace <kgr...@cimasg.com> wrote: >> S3QL filesystem dismounted on its own. When trying to remount, got error >> saying to rerun FSCK. Running FSCK results in the following error >

Re: [s3ql] Error of FSCK

On Apr 04 2016, Kevin Grace wrote: > S3QL filesystem dismounted on its own. When trying to remount, got error > saying to rerun FSCK. Running FSCK results in the following error > everytime. We have uploaded quiet a bit of data and wondering if we're > hitting a

Re: [s3ql] s3ql upstart

On Apr 01 2016, Daniel Jagszent wrote: > Hi Alexandre, > > you should not use the |--lazy| switch for the unmount. I don't think this is a problem in this situation. --lazy will cause umount.s3ql to return immediately, but upstart should still wait for the mount.s3ql process.

Re: [s3ql] BackBlaze Backend

On Mar 27 2016, jonathan cooke wrote: > I am looking to use S3QL with BackBlaze, I understand that a new backend > would have to be created, just looking for the best place to start. I am > still kind of new to python. I'd start with the Python tutorial

Re: [s3ql] s3ql 2.17.1 unmount crash

On Mar 18 2016, Ryan Groth wrote: > Just upgraded to 2.17.1 hoping for a bit more stability from one of > our more heavily used s3ql filesystems - What problems did you have before? [...] > return self._sslobj.write(data) > File "/usr/local/lib/python3.5/ssl.py", line

Re: [s3ql] s3qlcp file to file

On Mar 18 2016, Isaac Aymerich wrote: > Hi, > > Exist any way to use s3qlcp for fast copy but only file to file? Not directly, but you could use a simple script along the lines of: #!/bin/sh temp_src=`mktemp -d -p .` temp_dst=`mktemp -u -p .` ln "$1" "${temp_src}/$1"

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

On Mar 09 2016, Nikolaus Rath <nikol...@rath.org> wrote: > On Mar 09 2016, Cliff Stanford <cl...@may.be> wrote: >> On 09/03/16 01:03, Nikolaus Rath wrote: >> >>> I am pleased to announce a new release of S3QL, version 2.17. >> >> Great news, than

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

On Mar 10 2016, Cliff Stanford <cl...@may.be> wrote: > On 10/03/16 00:39, Nikolaus Rath wrote: >> On Mar 09 2016, Cliff Stanford <cl...@may.be> wrote: >>> Just one thing, I'm not sure it's sensible to push the metadata to >>> "infrequent access"

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

On Mar 09 2016, Riku Bister wrote: > torstai 10. maaliskuuta 2016 2.45.02 UTC+2 Riku Bister kirjoitti: >>> On Mar 09 2016, Riku Bister wrote: Backing up old metadata... Uncaught top-level exception: Traceback (most recent call last): >>> [...]

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

On Mar 09 2016, Riku Bister wrote: >> If you did something like that, you can try to get things in order by >> removing the s3ql_metadata_bak* files from the storage backend, as well >> as the local cached copy of the metadata (.db and .param file) and then >> running S3QL

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

On Mar 09 2016, Riku Bister wrote: > Backing up old metadata... > Uncaught top-level exception: > Traceback (most recent call last): > File "/root/s3ql/s3ql-2.17/bin/s3qladm", line 26, in > s3ql.adm.main(sys.argv[1:]) > File "/root/s3ql/s3ql-2.17/src/s3ql/adm.py", line

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

On Mar 09 2016, Cliff Stanford <cl...@may.be> wrote: > On 09/03/16 01:03, Nikolaus Rath wrote: > >> I am pleased to announce a new release of S3QL, version 2.17. > > Great news, thanks. > > Will you be updating ppa:nikratio/s3ql? It seems still to be at > vers

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

On Mar 09 2016, Cliff Stanford wrote: > On 09/03/16 13:46, Cliff Stanford wrote: > >> Will you be updating ppa:nikratio/s3ql? It seems still to be at version >> 2.15. > > In the meanwhile I tried building from the bitbucket version at >

Re: [s3ql] [ANNOUNCE] S3QL 2.17 has been released

Hi Pedro, When replying to emails on this list, please do not put your reply above the quoted text, and do not quote the entire message you're answering to. This makes it unnecessarily hard for other readers to understand the context of your email. Instead, please cut quoted parts that are not

Re: [s3ql] Recovering S3QL filesystem after unsuccessful transfer

On Mar 08 2016, Ilya Yakushin wrote: > Hello. I've transferred the S3QL filesystem from one host to another. > Something gone wrong during the transfer and the filesystem on the target > host is not mount-able or fsck-able at this moment. I don't have access to > the source

Re: [s3ql] How to use s3ql to manage the files directly on the swift storage.

On Mar 06 2016, Robert Aguila wrote: > Hi, I'd like to use s3ql only to manage the files in the swift object > storage. I understand that s3ql creates it's own file system inside the > backend storage, but what I need is to avoid this and allow s3ql to store > the files

Re: [s3ql] s3ql 2.16 - run tests fail

On Mar 02 2016, phantom@gmail.com wrote: > I've just done some more tests : > > - creating a new s3-bucket > - mount, unmount, check, copy file (2Go) both ways, deleting file, etc... > > And everything is running just fine. No problem at all. > > So I will continu testing with more data just

Re: [s3ql] s3ql 2.16 - run tests fail

On Mar 02 2016, phantom@gmail.com wrote: >> >> Also could it be possible that attr cause problem? That's unlikely. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an

Re: [s3ql] s3ql 2.16 - run tests fail

On Mar 02 2016, phantom@gmail.com wrote: >> >> > Sounds like your system doesn't allow setting the "security.selinux" >> extended attribute. Is that possible? Did it work with previous S3QL >> versions? >> > > Well, before I was using s3ql v2.13 (directly installed the rpm with dnf > and

Re: [s3ql] s3ql 2.16 - run tests fail

On Mar 01 2016, phantom@gmail.com wrote: >> >> > That's not good. But are you sure you pasted the complete output? It >> looks as if the 'previous errors' are not actually included. >> > > Please find attached the full log of the python runtests. > [...] > rsync: rsync_xal_set: >

Re: [s3ql] s3ql 2.16 - run tests fail

On Mar 01 2016, phantom@gmail.com wrote: > *Runtests output :* > > Traceback (most recent call last): > File "/home/fedora/s3ql-2.16/tests/t5_cp.py", line 30, in test > self.tst_cp() > File "/home/fedora/s3ql-2.16/tests/t5_cp.py", line 42, in tst_cp > os.path.join(self.mnt_dir,

Re: [s3ql] [s3ql 2.16] Cannot successfully run s3ql.fsck

On Feb 27 2016, Chris wrote: > With the new release of s3ql 2.16, trying to run s3ql.fsck. Every time > without fail, when it gets to the end and tries to upload the metadata, it > times out like such: > > Compressing and uploading metadata... >> Wrote 15.1 MiB of compressed

Re: [s3ql] Assertion error when unmount/remount

On Feb 25 2016, phantom@gmail.com wrote: >> >> >> > Is there anything stored in >> /tmp/ADMIN-cache/s3:=2F=2Fphantom-s3-admin=2F-cache? >> >> > Yes, here are the files : > > -rw-r--r--. 1 root root 1227763 Feb 17 08:59 14049-0 [...] Duh, no idea how that could possibly happen. Are you sure

Re: [s3ql] Assertion error when unmount/remount

On Feb 25 2016, phantom@gmail.com wrote: >> >> > What did fsck.s3ql say? Please post the complete output (including the >> command you used to start it) for both fsck.s3ql and mount.s3ql. >> >> > Here it is! > > *fsck.s3ql :* > > # fsck.s3ql --cachedir /tmp/ADMIN-cache/ --authfile

Re: [s3ql] Assertion error when unmount/remount

On Feb 25 2016, phantom@gmail.com wrote: > Hi, > > We use s3ql since 3 months now and it's just great! > > We actually have 4 x S3 buckets mounted with s3ql and we encounter a > problem. > > When unmounting, 2 buckets got errors and the 2 others are fine. > So we runs fsck.s3ql and you will

[s3ql] [ANNOUNCE] S3QL 2.16 released

Dear all, I am pleased to announce a new release of S3QL, version 2.16. >From the changelog: 2016-02-23, S3QL 2.16 * S3QL now supports (and requires) Python-LLFUSE 0.43 or newer (but not never than 1.0). * S3QL now retries when receiving a "ServiceUnavailable" error from an S3 (or

Re: [s3ql] NoSuchObject: Backend does not have anything stored under key 's3ql_data_8949'

On Feb 19 2016, Александр Чапкин wrote: > In the mount.log there is that error: > > > 2016-02-18 23:13:40.945 [13072] MainThread: [mount] Encountered exception, > trying to clean up... > 2016-02-18 23:13:40.945 [13072] MainThread: [mount] Unmounting file system > ... >

Re: [s3ql] Use https but skip certificate validation on connect?

On Feb 22 2016, pisymbol wrote: [ Use TLS without verifying certificate ] > So I added a backend-option just to see this would work and then noticed > that python-dugong doesn't honor it. Well, how did you implement the option? You probably have to add the feature to Dugong

Re: [s3ql] Use https but skip certificate validation on connect?

On Feb 22 2016, pisymbol wrote: > Also, what about for internal clouds that only allow https traffic > through? Yeah, what about those? Did anyone ever find a situation where that configuration made any sense? SCNR, -Nikolaus (No Cc on replies please, I'm reading the list)

Re: [s3ql] Use https but skip certificate validation on connect?

On Feb 19 2016, pisymbol <pisym...@gmail.com> wrote: > On Friday, February 19, 2016 at 4:58:04 PM UTC-5, Nikolaus Rath wrote: >> Do you know why the verification fails? I assume you're not connecting >> to something like S3 or Google Storage? > > That is correct. I a

Re: [s3ql] Use https but skip certificate validation on connect?

(please quote properly) On Feb 19 2016, Carl Myers wrote: > There needs to be a way to specify a self-signed certificate to trust. You can specify the CA path and use your own CA. That's moderately more work than a self-signed certificate, but shouldn't stop anyone who

Re: [s3ql] Use https but skip certificate validation on connect?

On Feb 19 2016, pisymbol wrote: > I'm trying to use S3QL in https mode but I see this when I try to do a > mkfs.s3ql or mount.s3ql: > > Exception: > > ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed > (_ssl.c:600) > Do you know why the

Re: [s3ql] s3ql 2.15 lost mount

On Feb 18 2016, Ryan Groth wrote: > ssl.SSLError: [SSL: BAD_WRITE_RETRY] bad write retry (_ssl.c:1846) > Feb 14 21:49:04 lsaws-backup journal: mount.s3ql[10286:MainThread] > s3ql.mount.unmount: Unmounting file system... > > Identical failure 3 other times since this log -

Re: [s3ql] Constant crashes, log included

On Feb 10 2016, Chris wrote: >> >> > a) Retry on these errors until the connection responds properly again? >> I'm >> > sure these hiccups are temporary >> >> Yes, that should work and be a relatively simple change. >> > > Could you please tell me how I might do that, or at

Re: [s3ql] Re: Ubuntu 14.04 + hubic2swiftgate + s3ql = CERTIFICATE_VERIFY_FAILED

On Feb 09 2016, Markus Wölfer wrote: > > Am Sonntag, 7. Februar 2016 16:23:35 UTC+1 schrieb Markus Wölfer: >> >> Hi >> >> I have problems getting hubic2swiftgate and s3ql running under ubuntu >> 14.04. > > anyone?

Re: [s3ql] Constant crashes, log included

On Feb 07 2016, Chris <exhu...@gmail.com> wrote: > On Sunday, January 17, 2016 at 4:03:10 PM UTC-5, Nikolaus Rath wrote: >> >> >> I don't understand the question. What does this have to do with your >> Linux distribution? Amazon S3 and Google Storage are probab

Re: [s3ql] s3ql lost mount issue

On Feb 06 2016, Ryan Groth wrote: > S3ql 2.15 on Centos 7 [...] > File > "/usr/local/lib/python3.5/site-packages/dugong-3.5-py3.5.egg/dugong/__init__.py", > line 619, in _co_send > len_ = self._sock.send(buf) > File "/usr/local/lib/python3.5/ssl.py", line 856, in

Re: [s3ql] Re: upgrading to 2.15 raw error

On Feb 04 2016, kurt wrote: >>Then you'll have to go back version by version until you find >>something where the upgrade command works. > > Is there a command like python3 setup.py uninstall that will automatically > uninstall versions as I try to go through this task of

<    1   2   3   4   5   >