Re: [PATCH 3 of 3] lfs: debug print HTTP headers and JSON payload received from the server

2018-03-13 Thread Yuya Nishihara
On Tue, 13 Mar 2018 00:00:08 -0400, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison 
> # Date 1520910527 14400
> #  Mon Mar 12 23:08:47 2018 -0400
> # Node ID d0b66408b224022fd96cb347ff3439e807812b21
> # Parent  f28c282005085edb77f64612be503a7cd43ab994
> lfs: debug print HTTP headers and JSON payload received from the server

Seems fine, queued, thanks.

> +if self.ui.debugflag:
> +self.ui.debug('Status: %d\n' % rsp.status)
> +# lfs-test-server and hg serve return headers in different order
> +self.ui.debug('%s\n'
> +  % '\n'.join(sorted(str(rsp.info()).splitlines(
> +
> +if 'objects' in response:
> +response['objects'] = sorted(response['objects'],
> + key=lambda p: p['oid'])
> +self.ui.debug('%s\n'
> +  % json.dumps(response, indent=2, sort_keys=True))

We'll need some hack for py3 because json is unicode.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH 3 of 3] lfs: debug print HTTP headers and JSON payload received from the server

2018-03-12 Thread Matt Harbison
# HG changeset patch
# User Matt Harbison 
# Date 1520910527 14400
#  Mon Mar 12 23:08:47 2018 -0400
# Node ID d0b66408b224022fd96cb347ff3439e807812b21
# Parent  f28c282005085edb77f64612be503a7cd43ab994
lfs: debug print HTTP headers and JSON payload received from the server

This has been extremely valuable to show divergences between `hg serve` and
`lfs-test-server`.  Once the `hg serve` code lands, there will be a certain
amount of conditionalizing that needs to be done, because `lfs-test-server`
doesn't always follow its spec.

The $ISO_8601_DATE_TIME$ pattern masks the fact that `lfs-test-serve` is sending
out an expires_at value of "0001-01-01T00:00:00Z".  `hg serve` will (probably)
use current time + 10 minutes or similar.  The $HTTP_DATE$ is the current time.

diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
--- a/hgext/lfs/blobstore.py
+++ b/hgext/lfs/blobstore.py
@@ -217,7 +217,8 @@ class _gitlfsremote(object):
 batchreq.add_header('Accept', 'application/vnd.git-lfs+json')
 batchreq.add_header('Content-Type', 'application/vnd.git-lfs+json')
 try:
-rawjson = self.urlopener.open(batchreq).read()
+rsp = self.urlopener.open(batchreq)
+rawjson = rsp.read()
 except util.urlerr.httperror as ex:
 raise LfsRemoteError(_('LFS HTTP error: %s (action=%s)')
  % (ex, action))
@@ -226,6 +227,19 @@ class _gitlfsremote(object):
 except ValueError:
 raise LfsRemoteError(_('LFS server returns invalid JSON: %s')
  % rawjson)
+
+if self.ui.debugflag:
+self.ui.debug('Status: %d\n' % rsp.status)
+# lfs-test-server and hg serve return headers in different order
+self.ui.debug('%s\n'
+  % '\n'.join(sorted(str(rsp.info()).splitlines(
+
+if 'objects' in response:
+response['objects'] = sorted(response['objects'],
+ key=lambda p: p['oid'])
+self.ui.debug('%s\n'
+  % json.dumps(response, indent=2, sort_keys=True))
+
 return response
 
 def _checkforservererror(self, pointers, responses, action):
@@ -301,6 +315,13 @@ class _gitlfsremote(object):
 response = b''
 try:
 req = self.urlopener.open(request)
+
+if self.ui.debugflag:
+self.ui.debug('Status: %d\n' % req.status)
+# lfs-test-server and hg serve return headers in different 
order
+self.ui.debug('%s\n'
+  % 
'\n'.join(sorted(str(req.info()).splitlines(
+
 if action == 'download':
 # If downloading blobs, store downloaded data to local 
blobstore
 localstore.download(oid, req)
diff --git a/tests/test-lfs-test-server.t b/tests/test-lfs-test-server.t
--- a/tests/test-lfs-test-server.t
+++ b/tests/test-lfs-test-server.t
@@ -54,7 +54,32 @@ store.
   checking for updated bookmarks
   listing keys for "bookmarks"
   lfs: computing set of blobs to upload
+  Status: 200
+  Content-Length: 309
+  Content-Type: application/vnd.git-lfs+json
+  Date: $HTTP_DATE$
+  {
+"objects": [
+  {
+"actions": {
+  "upload": {
+"expires_at": "$ISO_8601_DATE_TIME$", 
+"header": {
+  "Accept": "application/vnd.git-lfs"
+}, 
+"href": 
"http://localhost:$HGPORT/objects/31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b";
+  }
+}, 
+"oid": 
"31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b", 
+"size": 12
+  }
+]
+  }
   lfs: uploading 
31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b (12 bytes)
+  Status: 200
+  Content-Length: 0
+  Content-Type: text/plain; charset=utf-8
+  Date: $HTTP_DATE$
   lfs: processed: 
31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b
   lfs: uploaded 1 files (12 bytes)
   1 changesets found
@@ -99,7 +124,32 @@ Clear the cache to force a download
   resolving manifests
branchmerge: False, force: False, partial: False
ancestor: , local: +, remote: 99a7098854a3
+  Status: 200
+  Content-Length: 311
+  Content-Type: application/vnd.git-lfs+json
+  Date: $HTTP_DATE$
+  {
+"objects": [
+  {
+"actions": {
+  "download": {
+"expires_at": "$ISO_8601_DATE_TIME$", 
+"header": {
+  "Accept": "application/vnd.git-lfs"
+}, 
+"href": 
"http://localhost:$HGPORT/objects/31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b";
+  }
+}, 
+"oid": 
"31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b", 
+"size": 12
+  }
+]
+  }
   lfs: downloading 
31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b