Bug#792685: Unable to upgrade from wheezy to jessie

2015-08-01 Thread Nikolaus Rath 
control: tags -1 +pending

I've just uploaded a new package to mentors that should fix the issue
with the unsafe metadata backup.

If this works for you too, I think this is suitable for
stable-updates. I'm not sure what the procedure is to get a package in
there, so please let me know if I can do anything else to facilitate
this.

(While I recently became DM, I still haven't managed to get a DD to
actually give me upload rights for the packages that I maintain :-/).


Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Processed: Re: Bug#792685: Unable to upgrade from wheezy to jessie

2015-08-01 Thread Debian Bug Tracking System 
Processing control commands:

> tags -1 +pending
Bug #792685 [s3ql] Unable to upgrade from  wheezy to  jessie
Added tag(s) pending.

-- 
792685: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792685
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-08-01 Thread Nikolaus Rath 
Hi,

Alright, I figured this one out.

First, the reason you're having an s3ql_metadata_bak_0 object right
after the upgrade is that the upgrade itself backups up the metadata
*after* having added the _pre21 suffix to the existing backups.

Secondly (and contrary to what I said before), this should nevertheless
work fine, because the object metadata (metadata about the storage
object holding the file-system metadata) is upgraded when the object is
renamed (from s3ql_metadata to s3ql_metadata_bak_0).

Thirdly, there is a bug in all versions of S3QL that prevent it to work
properly if the bucket prefix contains a plus. This is what initially
prevented me from reproducing your problem.

Fourth, there is a bug in recent S3QL versions when reading object
metadata created by S3QL 1.x. There is a function that appears to return
a string, but it actually returns a
email.headerregistry._UnstructuredHeader instance that looks and behaves
like a string.

During the upgrade of the metadata, this object (instead of a string) is
then pickled and stored - and when it is then loaded again, S3QL
correctly complains that this is an unsafe pickle that attempts to
instantiate a weird class.

The reason that this does not happen when doing the upgrade in two
separate steps is that the more recent S3QL version never looks at the
metadata object created by the 1.x version - it gets turned into a
backup during the first upgrade, and gets a _pre21 suffix in the second
upgrade.

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-08-01 Thread Nikolaus Rath 
On Jul 30 2015, Sam Hartman  wrote:
> I'll try and reproduce this in the next day or so and give you access to
> a failing example.

No need, I just managed to reproduce it locally with a filesystem
freshly created in wheezy. I'll look into it.


Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-29 Thread Nikolaus Rath 
Note to self:

S3QL versions prior to 2.10 (and all of the 1.x versions) did not read
the object metadata on object copy and object rename. Therefore, the
unsafe pickle could have been there for quite some time. However, this
still does not explain why the object "survived" the upgrade without
being renamed.

The only other explanation would be that this object has actually been
created after the upgrade (e.g. by calling mount.s3ql). But then I don't
understand how it could have been renamed from s3ql_metadata to
s3ql_metadata_bak* without triggering the same error.


-- 
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-29 Thread Nikolaus Rath 
On Jul 29 2015, Sam Hartman  wrote:
> Then run fsck.s3ql.
> It looks like it has trouble with the old metadata:
> Backing up old metadata...
> Uncaught top-level exception:
> Traceback (most recent call last):
[...]

Could you please post the complete traceback? The one you included here
seems to stop in the middle (and the formatting is messed up).

Also, when you deleted the metadata backups manually, can you tell me
the exact names of the objects you deleted? And did you by any chance
try to delete them one-by-one, so that you know which one was causing
problems?


Thanks,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-29 Thread Nikolaus Rath 
On Jul 29 2015, Sam Hartman  wrote:
> Hi.
> I attempted the following:
>
> * Built with the patch applied on a jessie chroot
> * installed s3ql
>
> * On a wheezy system mkfs.s3ql, mount and  add some data.  unmount.
>
> * On my patched jessie system run s3qladm upgrade
>
> So far so good.
>
> Then run fsck.s3ql.
> It looks like it has trouble with the old metadata:
> Backing up old metadata...
> Uncaught top-level exception:
> Traceback (most recent call last):
>   File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 710, in _extractmeta
>   return safe_unpickle(b64decode(buf), encoding='latin1')
> File "/usr/lib/s3ql/s3ql/backends/common.py", line 629, in
> safe_unpickle
> encoding=encoding, errors=errors)
>   File "/usr/lib/s3ql/s3ql/backends/common.py", line 613, in
>   safe_unpickle_fh
>   raise pickle.UnpicklingError('opcode %s is unsafe' %
>   opcode.name)
>   _pickle.UnpicklingError: opcode NEWOBJ is unsafe
>
>   During handling of the above exception, another
>   exception occurred:
>
>   Traceback (most recent call last):
> File "/usr/bin/fsck.s3ql", line 9, in 
> load_entry_point('s3ql==2.11.1',
> 'console_scripts', 'fsck.s3ql')()
>   File "/usr/lib/s3ql/s3ql/fsck.py", line 1224, in
> main
> cycle_metadata(backend)
>   File "/usr/lib/s3ql/s3ql/metadata.py", line
> 121, in cycle_metadata
> cycle_fn("s3ql_metadata_bak_%d" % i,
> "s3ql_metadata_bak_%d" % (i + 1))
>   File "/usr/lib/s3ql/s3ql/backends/comprenc.py",
> line 290, in copy
> self._copy_or_rename(src, dest, rename=False,
> metadata=metadata)
>   File "/usr/lib/s3ql/s3ql/backends/comprenc.py",
> line 299, in _copy_or_rename
> meta_raw = self.backend.lookup(src)
>   File "/usr/lib/s3ql/s3ql/backends/common.py",
> line 46, in wrapped


Ok, this is very suspicious:

1. There should be no s3ql_metadata_bak_* object. "s3qladm upgrade"
   renames them all to prevent a *different* problem than the one you
   have above. Did you run fsck.s3ql right after s3qladm, or did you
   mount or fsck the file system before that? If not, is it possible
   that the changes had not yet propagated?

2. The above problem should not happen in the first place. There should
   never be a NEWOBJ operation in the pickle stream. Actually, if
   someone would try to exploit CVE-2014-0485 by sending you a malicious
   storage object, this is exactly the error you would get.

Do you have any confidential metadata? I may have to look at the actual
pickle to figure out what's happening here.


> It may be as simple as ignoring unsafe pickles in old metadata.

Certaintly not, that would open a remote code execution vulnerability.

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-29 Thread Sam Hartman 
Hi.
I attempted the following:

* Built with the patch applied on a jessie chroot
* installed s3ql

* On a wheezy system mkfs.s3ql, mount and  add some data.  unmount.

* On my patched jessie system run s3qladm upgrade

So far so good.

Then run fsck.s3ql.
It looks like it has trouble with the old metadata:
Backing up old metadata...
Uncaught top-level exception:
Traceback (most recent call last):
  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 710, in _extractmeta
  return safe_unpickle(b64decode(buf), encoding='latin1')
File "/usr/lib/s3ql/s3ql/backends/common.py", line 629, in
safe_unpickle
encoding=encoding, errors=errors)
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 613, in
  safe_unpickle_fh
  raise pickle.UnpicklingError('opcode %s is unsafe' %
  opcode.name)
  _pickle.UnpicklingError: opcode NEWOBJ is unsafe

  During handling of the above exception, another
  exception occurred:

  Traceback (most recent call last):
File "/usr/bin/fsck.s3ql", line 9, in 
load_entry_point('s3ql==2.11.1',
'console_scripts', 'fsck.s3ql')()
  File "/usr/lib/s3ql/s3ql/fsck.py", line 1224, in
main
cycle_metadata(backend)
  File "/usr/lib/s3ql/s3ql/metadata.py", line
121, in cycle_metadata
cycle_fn("s3ql_metadata_bak_%d" % i,
"s3ql_metadata_bak_%d" % (i + 1))
  File "/usr/lib/s3ql/s3ql/backends/comprenc.py",
line 290, in copy
self._copy_or_rename(src, dest, rename=False,
metadata=metadata)
  File "/usr/lib/s3ql/s3ql/backends/comprenc.py",
line 299, in _copy_or_rename
meta_raw = self.backend.lookup(src)
  File "/usr/lib/s3ql/s3ql/backends/common.py",
line 46, in wrapped


It may be as simple as ignoring unsafe pickles in old metadata.


-- 
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-21 Thread Sam Hartman 
Just as an FYI, I'm in Prague at an IETF meeting and that's taking up
most of my time, so it may be this weekend before I get a chance to try
your patch.

--Sam


-- 
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-19 Thread Nikolaus Rath 
Hi,

Could you give the attached patch for Jessie's S3QL a try? It should
allow to upgrade from wheezy file systems.

(Try it with a test file system first)

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
diff --git a/src/s3ql/adm.py b/src/s3ql/adm.py
--- a/src/s3ql/adm.py
+++ b/src/s3ql/adm.py
@@ -305,7 +305,10 @@
 raise QuietError()
 
 # Check revision
-if param['revision'] < CURRENT_FS_REV-1:
+if param['revision'] >= CURRENT_FS_REV:
+print('File system already at most-recent revision')
+return
+elif param['revision'] not in (16,20):
 print(textwrap.dedent('''
 File system revision too old to upgrade!
 
@@ -316,10 +319,6 @@
 print(get_old_rev_msg(param['revision'] + 1, 's3qladm'))
 raise QuietError()
 
-elif param['revision'] >= CURRENT_FS_REV:
-print('File system already at most-recent revision')
-return
-
 print(textwrap.dedent('''
 I am about to update the file system to the newest revision.
 You will not be able to access the file system with any older version
@@ -337,6 +336,22 @@
 if sys.stdin.readline().strip().lower() != 'yes':
 raise QuietError()
 
+if param['revision'] == 16:
+log.info('Upgrading from revision 16 to 20...')
+# For this upgrade, we need to recreate the sqlite database from the
+# metadata dump, because some SQLite types have changed
+log.info('Discarding cached metadata to trigger database rebuild.')
+db = None
+
+# Keep backup of local metadata (just in case...)
+if os.path.exists(cachepath + '.params'):
+assert os.path.exists(cachepath + '.db')
+if (os.path.exists(cachepath + '.db.bak') or
+os.path.exists(cachepath + '.params.bak')):
+raise QuietError('Metadata backup already exists, did something go wrong?')
+os.rename(cachepath + '.db', cachepath + '.db.bak')
+os.rename(cachepath + '.params', cachepath + '.params.bak')
+
 if not db:
 # Need to download metadata
 with tempfile.TemporaryFile() as tmpfh:
@@ -352,7 +367,7 @@
 tmpfh.seek(0)
 db = restore_metadata(tmpfh, cachepath + '.db')
 
-log.info('Upgrading from revision %d to %d...', param['revision'], CURRENT_FS_REV)
+log.info('Upgrading from revision 20 to %d...', CURRENT_FS_REV)
 
 param['revision'] = CURRENT_FS_REV
 param['last-modified'] = time.time()
@@ -402,9 +417,8 @@
 print(textwrap.dedent('''\
 File system upgrade complete.
 
-It is strongly recommended to run the s3ql_verify command with the
---data option as soon as possible. This is necessary to ensure that the
-upgrade to the next (2.11) S3QL release will run smoothly.'''))
+It is strongly recommended to run the new s3ql_verify command with the
+--data option at least once and as soon as possible.'''))
 
 # This should be used on the *next* fs revision update to ensure that all
 # objects conform to newest standards, so we can drop the legacy routines from

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-19 Thread Nikolaus Rath 
On Jul 19 2015, Sam Hartman  wrote:
> Nikolaus> I agree. There shouldn't have been an s3ql package in
> Nikolaus> Wheezy. This was when I was young and inexperienced with
> Nikolaus> Debian packaging. However, the situation is different for
> Nikolaus> Jessie. Now that it is much more mature, there will be
> Nikolaus> issues in keeping the stretch S3QL fully backwards
> Nikolaus> compatible with Jessie S3QL.
>
> I don't understand.
> do you mean perhaps no issues or a managable number of issues or
> something like that?

I meant to write "no issues", sorry.

> >> So, you talked about it being a lot of work to do the upgrade in
> >> one step.  Why does that need to happen?  Why can't the upgrade
> >> be handled in two steps and we just package the necessary parts
> >> to do all the steps?
>
> Nikolaus> That would mean to package ~5 intermediate S3QL
> Nikolaus> versions. Not a big deal in terms of work (since these
> Nikolaus> versions were already in testing at some point), but do
> Nikolaus> you think that would be acceptable for a point release?
>
> >> How much harder are we talking about than getting the s3qladm and
> >> s3ql libraries from the 2.5 version of s3ql into Jessie?
>
> Nikolaus> I don't understand the question.
>
> Ah.
> Looking at the debian news file, the only announced upgrade is  at
> version 2.5 between 1.11 and 2.11.1.

Oh. Obviously my memory has failed me completely here. You are right. I
just checked, all that is needed is one interim version. The best choice
would be to package S3QL 2.9, that is the most-recent version that still
supports upgrading from 1.11.

> It sounds like you believe that many intermediate versions are
> required.  What am I missing?  It looks to me like only one
> intermediate version is needed.  Can we get that into Jessie?  I don't
> know, but I'd be happy to talk to the release team and make my best
> argument for it.

I you can get the release team to agree, I'm happy to get the package in
shape. This would be based on

http://snapshot.debian.org/package/s3ql/2.9%2Bdfsg-2/


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-19 Thread Nikolaus Rath 
On Jul 19 2015, Nikolaus Rath  wrote:
>> Looking at the debian news file, the only announced upgrade is  at
>> version 2.5 between 1.11 and 2.11.1.
>
> Oh. Obviously my memory has failed me completely here. You are right. I
> just checked, all that is needed is one interim version. The best choice
> would be to package S3QL 2.9, that is the most-recent version that still
> supports upgrading from 1.11.
>
>> It sounds like you believe that many intermediate versions are
>> required.  What am I missing?  It looks to me like only one
>> intermediate version is needed.  Can we get that into Jessie?  I don't
>> know, but I'd be happy to talk to the release team and make my best
>> argument for it.
>
> I you can get the release team to agree, I'm happy to get the package in
> shape.

As a matter of fact, if there's just one interim version, it may also be
possible to integrate the necessary upgrade steps into the most recent
version instead.

Let me look into that.


Best,
-Nikolaus, who is still confused why he was so certain that there were
multiple upgrades in the interim.

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-19 Thread Sam Hartman 
> "Nikolaus" == Nikolaus Rath  writes:


Nikolaus> I agree. There shouldn't have been an s3ql package in
Nikolaus> Wheezy. This was when I was young and inexperienced with
Nikolaus> Debian packaging. However, the situation is different for
Nikolaus> Jessie. Now that it is much more mature, there will be
Nikolaus> issues in keeping the stretch S3QL fully backwards
Nikolaus> compatible with Jessie S3QL.

I don't understand.
do you mean perhaps no issues or a managable number of issues or
something like that?

>> So, you talked about it being a lot of work to do the upgrade in
>> one step.  Why does that need to happen?  Why can't the upgrade
>> be handled in two steps and we just package the necessary parts
>> to do all the steps?

Nikolaus> That would mean to package ~5 intermediate S3QL
Nikolaus> versions. Not a big deal in terms of work (since these
Nikolaus> versions were already in testing at some point), but do
Nikolaus> you think that would be acceptable for a point release?

>> How much harder are we talking about than getting the s3qladm and
>> s3ql libraries from the 2.5 version of s3ql into Jessie?

Nikolaus> I don't understand the question.

Ah.
Looking at the debian news file, the only announced upgrade is  at
version 2.5 between 1.11 and 2.11.1.

So, for example, I've installed s3ql 2.7-1 on a jessie system and it
seems happy to upgrade my wheezy filesystem. (Note that some of the
tests failed during the build of 2.7-1 on Jessie I had to build with
DEB_BUILD_OPTIONS=nocheck, so some work is required)

It sounds like you believe that many intermediate versions are required.
What am I missing?  It looks to me like only one intermediate version is
needed.  Can we get that into Jessie?  I don't know, but I'd be happy to
talk to the release team and make my best argument for it.


-- 
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-17 Thread Nikolaus Rath 
On Jul 17 2015, Sam Hartman  wrote:
> If a package is not going to provide an upgrade from one debian
> release to the next, it should not generally be in the Debian release.
> It would be fine for it to continue to be in unstable.
>
> We need to figure out if someone is willing to commit to doing the
> necessary work for stretch and if not probably leave s3ql in unstable
> but pull from stretch and investigate whether it should be pulled from
> Jessie in a point release.

I agree. There shouldn't have been an s3ql package in Wheezy. This was
when I was young and inexperienced with Debian packaging. However, the
situation is different for Jessie. Now that it is much more mature,
there will be issues in keeping the stretch S3QL fully backwards
compatible with Jessie S3QL. 

> Nikolaus> I guess it would have been better to package S3QL 2.x as a
> Nikolaus> new s3ql2 package instead of it replacing the s3ql
> Nikolaus> package. But I don't think there is a way to recticify
> Nikolaus> that now - or is there?
>
> But then people still lose their data from wheezy->jessie.

No, in that case we could have kept the old S3QL 1.x package in jessie.

> Nikolaus> Theoretically, it is possible to do all the upgrades in
> Nikolaus> one step and integrate that into Jessie's s3ql package,
> Nikolaus> but that would be a major coding effort.
>
> So, you talked about it being a lot of work to do the upgrade in one
> step.
> Why does that need to happen?
> Why can't the upgrade be handled in two steps and we just package  the
> necessary parts to do all the steps?

That would mean to package ~5 intermediate S3QL versions. Not a big deal
in terms of work (since these versions were already in testing at some
point), but do you think that would be acceptable for a point release?

> How much harder  are we talking about than getting the s3qladm and s3ql
> libraries from the 2.5 version of s3ql into Jessie?

I don't understand the question. 

Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-17 Thread Sam Hartman 
> "Nikolaus" == Nikolaus Rath  writes:


Nikolaus> I agree that this is unfortunate, but preserving backwards
Nikolaus> compatibility over so many upstream versions just for
Nikolaus> Debian wheezy users did not seem worth the time.

I totally understand that it's a lot of work and totally get that you
might not want to do that work.
However, Debian as a project balances  stability and upgrade tradeoffs
across the project, and this is one of those areas where a project goal
may require more work to be put into a package for that package to be
suitable for inclusion in a Debian release.

No one is obligated to do that work, but Debian's philosophy is
generally that if the work to make upgrades work well can't be done,
then a package should not be included in releases.


If a package is not going to provide an upgrade from one debian release
to the next, it should not generally be in the Debian release.
It would be fine for it to continue to be in unstable.

We need to figure out if someone is willing to commit to doing the
necessary work for stretch   and if not probably leave s3ql in unstable
but pull from stretch and investigate whether it should be pulled from
Jessie in a point release.

s3ql tends to get used for backups and other things where availability
is quite important and saying "oops, you don't get your data when you
upgraded," is kind of a big deal.
If Debian users can't count on maintaining access to their data after
the upgrades, then we probably want to warn them about that up front,
and  the experience may not be good enough to actually recommend that
people store data in s3ql.

I understand that as an upstream author you might well want a more rapid
deployment cycle than Debian's release process.
That's fine.  I'm not judging anything here.
I'm just saying that if software's going to be in a Debian release it
needs to live up to what that means.


Nikolaus> Theoretically, it is possible to do all the upgrades in
Nikolaus> one step and integrate that into Jessie's s3ql package,
Nikolaus> but that would be a major coding effort.


Nikolaus> I guess it would have been better to package S3QL 2.x as a
Nikolaus> new s3ql2 package instead of it replacing the s3ql
Nikolaus> package. But I don't think there is a way to recticify
Nikolaus> that now - or is there?

But then people still lose their data from wheezy->jessie.

So, you talked about it being a lot of work to do the upgrade in one
step.
Why does that need to happen?
Why can't the upgrade be handled in two steps and we just package  the
necessary parts to do all the steps?

How much harder  are we talking about than getting the s3qladm and s3ql
libraries from the 2.5 version of s3ql into Jessie?


-- 
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-17 Thread Nikolaus Rath 
On Jul 17 2015, Sam Hartman  wrote:
> I have a filesystem that I can easily mount and fsck in wheezy, but when
> I try to run the jessie s3qladm upgrade command I get:
> Getting file system parameters..
>
> File system revision too old to upgrade!
[...]
>
> It's critical that there be a documented procedure that works for
> upgrading from the version in wheezy to the version in jessie using
> tools in jessie.

At the moment the only way is to download intermediate S3QL versions and
use them to upgrade in several steps. There is a chance that all
required versions are available on snapshot.debian.org, but I haven't
checked.

I agree that this is unfortunate, but preserving backwards compatibility
over so many upstream versions just for Debian wheezy users did not seem
worth the time.

Theoretically, it is possible to do all the upgrades in one step and
integrate that into Jessie's s3ql package, but that would be a major
coding effort.


I guess it would have been better to package S3QL 2.x as a new s3ql2
package instead of it replacing the s3ql package. But I don't think
there is a way to recticify that now - or is there?


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«


--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#792685: Unable to upgrade from wheezy to jessie

2015-07-17 Thread Sam Hartman 

package: s3ql
severity: grave
version: 2.11.1+dfsg-1
Justification: Renders filesystem unusable and data accessible

Hi.
I'm upgrading a system from wheezy to jessie.
Wheezy ships s3ql 1.11, jessie ships version 2.11.

I have a filesystem that I can easily mount and fsck in wheezy, but when
I try to run the jessie s3qladm upgrade command I get:
Getting file system parameters..

File system revision too old to upgrade!

You need to use an older S3QL version to upgrade to a more recent
revision before you can use this version to upgrade to the newest
revision.

Uncaught top-level exception:
Traceback (most recent call last):
  File "/usr/bin/s3qladm", line 9, in 
  load_entry_point('s3ql==2.11.1', 'console_scripts', 's3qladm')()
File "/usr/lib/s3ql/s3ql/adm.py", line 96, in main
options.cachedir))
  File "/usr/lib/s3ql/s3ql/adm.py", line 316, in upgrade
  print(get_old_rev_msg(param['revision'] + 1, 's3qladm'))
File "/usr/lib/s3ql/s3ql/adm.py", line 224, in 
get_old_rev_msg
''' % { 'version': REV_VER_MAP[rev],
KeyError: 17


It's critical that there be a documented procedure that works for
upgrading from the version in wheezy to the version in jessie using
tools in jessie.


There was another upgrade at version 2.5, which is not in either wheezy
or jessie.  However, it needs to be possible to upgrade from one Debian
release to the next using the software in Debian.


I believe this problem is important enough to fix in a Jessie point
release and would be happy to help with any process issues that come up
in making that happen.


pgpqzly9nonTg.pgp
Description: PGP signature