Title: [233057] trunk/Tools
- Revision
- 233057
- Author
- lforsch...@apple.com
- Date
- 2018-06-21 13:58:52 -0700 (Thu, 21 Jun 2018)
Log Message
Fix for https://bugs.webkit.org/show_bug.cgi?id=185790
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (233056 => 233057)
--- trunk/Tools/ChangeLog 2018-06-21 20:54:32 UTC (rev 233056)
+++ trunk/Tools/ChangeLog 2018-06-21 20:58:52 UTC (rev 233057)
@@ -1,3 +1,22 @@
+2018-06-21 Lucas Forschler <lforsch...@apple.com>
+
+ bisect-builds --list not showing all builds
+ https://bugs.webkit.org/show_bug.cgi?id=185790
+
+ The AWS Gateway API call into dynamoDB is limited to returning 1MB of query data. Our archive set has overgrown this limit,
+ causing us to retrieve a truncated list of results. Fortunately, when this limit is reached, dynamoDB returns a
+ "LastEvaluatedKey", which can be used to continue the query in another request.
+
+ This patch teaches bisect-build to pass along this key when needed. The backend api was updated from v2 to v2_1 to support
+ this new querystring option. Any existing code using v2 will not be affected.
+
+ Reviewed by Aakash Jain.
+
+ * Scripts/bisect-builds:
+ (get_api_url): added optional LastEvaluatedKey argument, which will append the query string to the base api url.
+ (fetch_revision_list): Added to allow a recursive call which will query until we retrieve every item requested.
+ (main): teach how to use fetch_revision_list.
+
2018-06-21 Keith Rollin <krol...@apple.com>
check-webkit-style should warn about exported inline functions
Modified: trunk/Tools/Scripts/bisect-builds (233056 => 233057)
--- trunk/Tools/Scripts/bisect-builds 2018-06-21 20:54:32 UTC (rev 233056)
+++ trunk/Tools/Scripts/bisect-builds 2018-06-21 20:58:52 UTC (rev 233057)
@@ -38,7 +38,7 @@
import urllib2
import urlparse
-REST_API_URL = 'https://q1tzqfy48e.execute-api.us-west-2.amazonaws.com/v2/'
+REST_API_URL = 'https://q1tzqfy48e.execute-api.us-west-2.amazonaws.com/v2_1/'
REST_API_ENDPOINT = 'archives/'
REST_API_MINIFIED_ENDPOINT = 'minified-archives/'
@@ -93,7 +93,7 @@
# ---- end bisect helpers ----
-def get_api_url(options):
+def get_api_url(options, LastEvaluatedKey=None):
if options.full:
base_url = urlparse.urljoin(REST_API_URL, REST_API_ENDPOINT)
else:
@@ -100,6 +100,10 @@
base_url = urlparse.urljoin(REST_API_URL, REST_API_MINIFIED_ENDPOINT)
api_url = urlparse.urljoin(base_url, '-'.join([options.platform, options.architecture, options.configuration]))
+ if LastEvaluatedKey:
+ querystring = urllib2.quote(json.dumps(LastEvaluatedKey))
+ api_url += '?ExclusiveStartKey=' + querystring
+
return api_url
@@ -223,15 +227,22 @@
print(revision_list)
exit(0)
-def main(options):
- validate_options(options)
-
- url = ""
+def fetch_revision_list(options, LastEvaluatedKey=None):
+ url = "" LastEvaluatedKey)
r = urllib2.urlopen(url)
data = ""
-
revision_list = get_sorted_revisions(data)
+ if 'LastEvaluatedKey' in data['revisions']:
+ LastEvaluatedKey = data['revisions']['LastEvaluatedKey']
+ revision_list += fetch_revision_list(options, LastEvaluatedKey)
+
+ return revision_list
+
+def main(options):
+ validate_options(options)
+ revision_list = fetch_revision_list(options)
+
if options.list:
print_list_and_exit(revision_list, options)
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
