Stuart Bishop has proposed merging lp:~stub/launchpad/trivial into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
Bug #951401 in Launchpad itself: "parse-ppa-apache-logs failing (missing
files)"
https://bugs.launchpad.net/launchpad/+bug/951401
Bug #1263002 in Launchpad itself: "Twisted feature flag support fails
typecasting when updating"
https://bugs.launchpad.net/launchpad/+bug/1263002
For more details, see:
https://code.launchpad.net/~stub/launchpad/trivial/+merge/239178
Rather manually incrementing the 'start' LibrariFileContent in crontab to avoid
backscans, calculate it by date.
The query is composed to do a backwards index scan, and is as performant as we
can make it without an index on datecreated.
This will be irrelevant once we are removing files from disk, as removed files
don't need to be iterated over and we can stop using the --start argument
entirely.
--
https://code.launchpad.net/~stub/launchpad/trivial/+merge/239178
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~stub/launchpad/trivial into lp:launchpad.
=== modified file 'cronscripts/librarian-feed-swift.py'
--- cronscripts/librarian-feed-swift.py 2014-10-15 13:15:17 +0000
+++ cronscripts/librarian-feed-swift.py 2014-10-22 11:12:35 +0000
@@ -11,8 +11,10 @@
import os
+from lp.services.database.interfaces import ISlaveStore
+from lp.services.librarian.model import LibraryFileContent
+from lp.services.librarianserver import swift
from lp.services.scripts.base import LaunchpadCronScript
-from lp.services.librarianserver import swift
class LibrarianFeedSwift(LaunchpadCronScript):
@@ -28,11 +30,21 @@
dest="start", metavar="CONTENT_ID",
help="Migrate files starting from CONTENT_ID")
self.parser.add_option(
+ "--start-since", action="store", dest='start_since',
+ default=None, metavar="INTERVAL",
+ help="Migrate files starting from INTERVAL (PostgreSQL syntax)")
+ self.parser.add_option(
"-e", "--end", action="store", type=int, default=None,
dest="end", metavar="CONTENT_ID",
help="Migrate files up to and including CONTENT_ID")
def main(self):
+ if self.options.start_since:
+ self.options.start = ISlaveStore(LibraryFileContent).execute("""
+ SELECT MAX(id) FROM LibraryFileContent
+ WHERE datecreated < current_timestamp at time zone 'UTC'
+ - CAST(%s AS INTERVAL)
+ """, (unicode(self.options.start_since),)).get_one()[0]
if self.options.ids and (self.options.start or self.options.end):
self.parser.error(
"Cannot specify both individual file(s) and range")
_______________________________________________
Mailing list: https://launchpad.net/~launchpad-reviewers
Post to : [email protected]
Unsubscribe : https://launchpad.net/~launchpad-reviewers
More help : https://help.launchpad.net/ListHelp
