[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-07 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573674#comment-13573674
 ] 

Stefan Kögl commented on COUCHDB-1654:
--

Here are the logs from the first start of 1.3.x until the first crash: 
https://friendpaste.com/790kHUYRCCnM0ElNS4qCq.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Assignee: Jan Lehnardt
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-07 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573595#comment-13573595
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

@skoegl great find. Do you have more logs from further up? Ideally from 1.3.x 
start?

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Assignee: Jan Lehnardt
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-07 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573340#comment-13573340
 ] 

Stefan Kögl commented on COUCHDB-1654:
--

[~rnewson] asked me to provide the ddoc of the view. It can be found here 
https://friendpaste.com/2iNlKuwxzfNDW3dsYBezmZ and be browsed here 
https://github.com/gpodder/mygpo/tree/4b8b617aa855df020a5d884011471d0f04969e4e/couchdb/_design/podcasts

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573270#comment-13573270
 ] 

Stefan Kögl commented on COUCHDB-1654:
--

When migrating an 1.2.1 instance to 1.3.x for testing purposes, I could query 
most of the views correctly. For one I reproducably receive the following trace 
https://friendpaste.com/2iNlKuwxzfNDW3dsYBnqsv. Wohali in #couchdb recommended 
to add that to this ticket.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572886#comment-13572886
 ] 

ASF subversion and git services commented on COUCHDB-1654:
--

Commit da35ed0f012c265d35a71cc12e3683bdaad9e6d3 in branch refs/heads/master 
from [~janl]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=da35ed0 ]

COUCHDB-1654: Transparently update view signatures from <= 1.2.x.

Updates 1.2.x or earlier view files to 1.3.x or later view files
transparently, the first time the 1.2.x view file is opened by
1.3.x or later.

Here's how it works:

Before opening a view index,
If no matching index file is found in the new location:
 calculate the <= 1.2.x view signature
 if a file with that signature lives in the old location
   copy it to the new location with the new signature in the name.
Then proceed to open the view index as usual.
After opening, read its header.

If the header matches the <= 1.2.x style #index_header record:
  upgrade the header to the new #mrheader record
The next time the view is used, the new header is used.

If we crash after the rename, but before the header upgrade,
  the header upgrade is done on the next view opening.

If we crash between upgrading to the new header and writing
  that header to disk, we start with the old header again,
  do the upgrade and write to disk.

Includes etap tests in 250*.t.


> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572885#comment-13572885
 ] 

ASF subversion and git services commented on COUCHDB-1654:
--

Commit 73195d31fcf192a4d821b0d751a4775f65753632 in branch refs/heads/1.3.x from 
[~janl]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=73195d3 ]

COUCHDB-1654: Transparently update view signatures from <= 1.2.x.

Updates 1.2.x or earlier view files to 1.3.x or later view files
transparently, the first time the 1.2.x view file is opened by
1.3.x or later.

Here's how it works:

Before opening a view index,
If no matching index file is found in the new location:
 calculate the <= 1.2.x view signature
 if a file with that signature lives in the old location
   copy it to the new location with the new signature in the name.
Then proceed to open the view index as usual.
After opening, read its header.

If the header matches the <= 1.2.x style #index_header record:
  upgrade the header to the new #mrheader record
The next time the view is used, the new header is used.

If we crash after the rename, but before the header upgrade,
  the header upgrade is done on the next view opening.

If we crash between upgrading to the new header and writing
  that header to disk, we start with the old header again,
  do the upgrade and write to disk.

Includes etap tests in 250*.t.


> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread Benoit Chesneau (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572874#comment-13572874
 ] 

Benoit Chesneau commented on COUCHDB-1654:
--

+1

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread Robert Newson (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572872#comment-13572872
 ] 

Robert Newson commented on COUCHDB-1654:


+1

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572868#comment-13572868
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

Thanks Randall!

[~rnewson], can I get a +1 to merge this into 1.3.x?

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread Randall Leeds (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572861#comment-13572861
 ] 

Randall Leeds commented on COUCHDB-1654:


+1 -- Ship it!

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572848#comment-13572848
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

Thanks Randall, all your assumptions are correct.

The test case uses an actual 1.2.x-era view file.


> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread Randall Leeds (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572845#comment-13572845
 ] 

Randall Leeds commented on COUCHDB-1654:


Hey, that looks pretty good to me! To make sure I understand, it seems like the 
fields used in signature calculation didn't really change, except that the 
record name changed which makes the first field of the record tuple different. 
Aside from that, it's just a matter of the location changing. The trick there 
is to calculate the index filename as though the module name were empty. Does 
that all sound right?

Code looks fine. Have you tested it on any real views?

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-06 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572817#comment-13572817
 ] 

ASF subversion and git services commented on COUCHDB-1654:
--

Commit 23cc54d54ed20efe01d3b9bfb294cadfca9f950b in branch 
refs/heads/1654-fix-view-signature-change from [~janl]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=23cc54d ]

COUCHDB-1654: Transparently update view signatures from <= 1.2.x.

Updates 1.2.x or earlier view files to 1.3.x or later view files
transparently, the first time the 1.2.x view file is opened by
1.3.x or later.

Here's how it works:

Before opening a view index,
If no matching index file is found in the new location:
 calculate the <= 1.2.x view signature
 if a file with that signature lives in the old location
   copy it to the new location with the new signature in the name.
Then proceed to open the view index as usual.
After opening, read its header.

If the header matches the <= 1.2.x style #index_header record:
  upgrade the header to the new #mrheader record
The next time the view is used, the new header is used.

If we crash after the rename, but before the header upgrade,
  the header upgrade is done on the next view opening.

If we crash between upgrading to the new header and writing
  that header to disk, we start with the old header again,
  do the upgrade and write to disk.

Includes etap tests in 250*.t.


> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-05 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13571453#comment-13571453
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

Now with tests: 
https://github.com/janl/couchdb/compare/1654-fix-view-signature-change

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-05 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13571279#comment-13571279
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

Discard my last message, I must have tested something wrong last night.

I cleaned up the branch: 
https://github.com/janl/couchdb/compare/1654-fix-view-signature-change

It should work now fully. Please review! :)





> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-04 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570614#comment-13570614
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

Update: when adding new docs to the db and requesting an updated view again it 
triggers a reindex, seems a subtlety is missing. This still feels like it is 
close.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-04 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570596#comment-13570596
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

fixed the view calculation, the happy case now seems to work.

it still needs some cleaning up for the non-upgrade path.

Reviews more than welcome now.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-04 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570470#comment-13570470
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

I had a stab at it. Here’s the plan:

in 1.3.x and later:
 - before opening a view index
  - if the index file exists, open that, end of story
  - if the index file doesn’t exist,
- see if the old-style index file exists
- if it does, copy it to the new location 
(.dbname_deign/mrview/newsignature.view)
- if it doesn’t, ignore.
 - when opening a view index
 - when the header can be read
 - if the header matches the 1.2.x and lower #index_header
  - extract its values and splice them into the new #mrheader

 - the next time the view file header is flushed to disk, we have upgraded.
 - if we crash any time before that, the procedure can be repeated.

This branch is a first stab at this: 
https://github.com/janl/couchdb/compare/1654-fix-view-signature-change

It calculates the old view signature wrong, I might be missing something 
subtle, if you find it, let me know.



> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-02-03 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569904#comment-13569904
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

Updating this with dev@ and IRC meeting intelligence:

[~kocolosk] & [~davispw] volunteered to have a stab at this.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-28 Thread Alexander Shorin (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564335#comment-13564335
 ] 

Alexander Shorin commented on COUCHDB-1654:
---

[~rnewson] , I may be wrong about, but roots of the new view engine leads to 
https://issues.apache.org/jira/browse/COUCHDB-1270 - there is benchmark data 
and his results. As for my experience, it's still a bit faster then the old one 
(or may it's because of https://issues.apache.org/jira/browse/COUCHDB-1334 ?), 
but without any proofs or numbers, sorry.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-28 Thread Benoit Chesneau (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564330#comment-13564330
 ] 

Benoit Chesneau commented on COUCHDB-1654:
--

[~janl] That's fine for me. Just wanted to skip another delay..

[~rnewson]wasn't thinking at a shell script for the rlup phase. There is no 
reason you can't do that during this phase though. You can 1. do the migration 
offline then trigger the update. 

About the benchmark can you share the script and data? Will be easier to 
compare the same figure and see if we didn't forget obvious things.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-28 Thread Robert Newson (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564129#comment-13564129
 ] 

Robert Newson commented on COUCHDB-1654:


Also, I'm most keen to hear if the new view engine is slower or not. 1 report 
that it is, and 1 that it isnt', though the latter is from a benchmark, which I 
trust more than an anecdote.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-28 Thread Robert Newson (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564128#comment-13564128
 ] 

Robert Newson commented on COUCHDB-1654:


The new 1.3 view sig is technically better but obviously we can't break 
compatibility. I'll look at tweaking things so that mrview will look for the 
old file first, and then move it to the new file and then use it. It can't be a 
script (in the sense of some external shell script) for the relup case, since 
that must be seamless (we can't stop the server to upgrade).

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-28 Thread Jan Lehnardt (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564126#comment-13564126
 ] 

Jan Lehnardt commented on COUCHDB-1654:
---

@Benoit, I respectfully disagree. With the principle of least surprise, 1.3.0 
should *not* rebuild view files after an upgrade just because we fucked up 
signature generation.

I propose we make this part of the etap test suite and ensure that this is 
fixed now and in the future. Same with db signatures.

If we need to change things in the future for more severe reasons, I’d suggest 
we treat this as a BC break and accompany such a change with a major version 
number increase and copious release notes..

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-28 Thread Benoit Chesneau 
On Jan 28, 2013 9:15 AM, "Stefan Kögl (JIRA)"  wrote:
>
>
> [
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564112#comment-13564112]
>
> Stefan Kögl commented on COUCHDB-1654:
> --
>
> I tried copying a .view file from its 1.2.x path to the 1.3.x path (new
directory and sig) and could access its _info, but the view was rebuild
when I tried to query it. Can somebody else confirm this?

Yeah there is also a change in internal view sig that @rnewson spotted
yesterday so a simple move won't be enough. :/
>
> > couchdb view sig changed
> > 
> >
> > Key: COUCHDB-1654
> > URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> > Project: CouchDB
> >  Issue Type: Bug
> >  Components: Database Core
> >Reporter: Robert Newson
> >Priority: Blocker
> > Fix For: 1.3
> >
> >
> > view sigs and location changed between 1.2.x and 1.3.x, which would
cause full rebuilds.
> > to be clear: upgrading to 1.3 should not cause a view to rebuild from
scratch.
> > ➜  couchdb git:(1.3.x) find tmp
> > tmp
> > tmp/lib
> > tmp/lib/.db1_design
> > tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> > tmp/lib/.delete
> > tmp/lib/_replicator.couch
> > tmp/lib/_users.couch
> > tmp/lib/db1.couch
> > tmp/log
> > tmp/log/couch.log
> > tmp/run
> > tmp/run/couch.uri
> > tmp/run/couchdb
> > ➜  couchdb git:(1.3.x) find tmp
> > tmp
> > tmp/lib
> > tmp/lib/.db1_design
> > tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> > tmp/lib/.db1_design/mrview
> > tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> > tmp/lib/.delete
> > tmp/lib/_replicator.couch
> > tmp/lib/_users.couch
> > tmp/lib/db1.couch
> > tmp/log
> > tmp/log/couch.log
> > tmp/run
> > tmp/run/couch.uri
> > tmp/run/couchdb
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-28 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564112#comment-13564112
 ] 

Stefan Kögl commented on COUCHDB-1654:
--

I tried copying a .view file from its 1.2.x path to the 1.3.x path (new 
directory and sig) and could access its _info, but the view was rebuild when I 
tried to query it. Can somebody else confirm this?

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-27 Thread Benoit Chesneau (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564088#comment-13564088
 ] 

Benoit Chesneau commented on COUCHDB-1654:
--

 I don't see any reason to change the current code. It's here since a while and 
changing it on the last minute is worrying me. (happy to be wrong). 

What about adding a code that "upgrade" the views by moving current index on 
the right places on the initial startup? Can also be a script. For those who do 
hot upgrade, a script can also do the job. Although It would cost some disk 
space though, I'm pretty sure most can already handle it, this is somehow a 
problem that you should already solve during the compaction.

> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1654) couchdb view sig changed

2013-01-27 Thread Benoit Chesneau (JIRA) 

[ 
https://issues.apache.org/jira/browse/COUCHDB-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563783#comment-13563783
 ] 

Benoit Chesneau commented on COUCHDB-1654:
--

Just read the code and it seems that the index file signature is now based on 
the indexer module name:

snippet from couch_index_util (l28):

index_dir(Module, DbName) when is_binary(DbName) ->
DbDir = "." ++ binary_to_list(DbName) ++ "_design",
filename:join([root_dir(), DbDir, Module]);
index_dir(Module, #db{}=Db) ->
index_dir(Module, couch_db:name(Db)).


index_file(Module, DbName, FileName) ->
filename:join(index_dir(Module, DbName), FileName).

Also the path is now based on the module name. So we have two things:

1. signature chane
2. old index file can't be found since the path changed.

Not sure what is the best way to solve that For example how do we manage 
other  index if we don't change the path accordingly?



> couchdb view sig changed
> 
>
> Key: COUCHDB-1654
> URL: https://issues.apache.org/jira/browse/COUCHDB-1654
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Robert Newson
>Priority: Blocker
> Fix For: 1.3
>
>
> view sigs and location changed between 1.2.x and 1.3.x, which would cause 
> full rebuilds.
> to be clear: upgrading to 1.3 should not cause a view to rebuild from scratch.
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb
> ➜  couchdb git:(1.3.x) find tmp
> tmp
> tmp/lib
> tmp/lib/.db1_design
> tmp/lib/.db1_design/0761939631be54bf36d3ed9172bcf689.view
> tmp/lib/.db1_design/mrview
> tmp/lib/.db1_design/mrview/641ce136c0f1dbe93aa94d3a419a0efe.view
> tmp/lib/.delete
> tmp/lib/_replicator.couch
> tmp/lib/_users.couch
> tmp/lib/db1.couch
> tmp/log
> tmp/log/couch.log
> tmp/run
> tmp/run/couch.uri
> tmp/run/couchdb

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira