Re: How should we include extra modules in a pyoxidized Mercurial?
Yeah, you'll want a pyoxidizer.bzl that installs hg and other stuff. Using PythonDistribution.pip_install() ( https://pyoxidizer.readthedocs.io/en/stable/config_api.html#pythondistribution-pip-install-args-extra-envs) with arguments to process a requirements.txt file is likely how I'd do it. Sadly, PyOxidizer doesn't (yet) support including another config file. That seems like a reasonable feature request. Although I'm unsure if the Rust Starlark implementation can handle it. Can you please file a feature request against PyOxidizer? Another missing feature that would make this easier would be injecting environment variables into pyoxidizer.bzl files. That way we could teach Mercurial's canonical pyoxidizer.bzl to inject other install rules via presence of environment variables. That would make things easier to extend. We could potentially also accept command line arguments to `pyoxidizer build` to inject additional variables into the Starlark context. Again, please file feature requests here! Since you have a bit more resources than the average user and you are probably opinionated about how your binaries are built, you might want to look into bypassing PyOxidizer and doing the resource packaging yourself. See https://pyoxidizer.readthedocs.io/en/latest/oxidized_importer_freezing_applications.html and https://pyoxidizer.readthedocs.io/en/latest/rust.html for more on this topic. On Wed, Jun 17, 2020 at 4:56 PM Augie Fackler wrote: > We're working on a PyOxidizer Mercurial for our users, and we want to bake > some extra stuff beyond just hg (evolve, some in-house stuff) into the > binary. Should we be making our own pyoxidizer.bzl that installs hg and > whatever else we need? Is there a way we could reuse the bits in the > existing configuration? > > Thanks! > Augie ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D8639: py3: fix broken man page generation, it was generating `(default: NUL*)`
spectral created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
`bytes(default)` was producing things like `(default: \x00)` when handed
non-bytes values such as `1`, `10`, or `True`. The man page generation would
apparently ignore these bytes and produce man pages that had the string
`(default: )`.
TEST PLAN
- Ran `cd doc; python3 gendoc.py "hg.1.gendoc"` and grepped for bad output
- Ran `make deb`, extracted the deb, manually inspected `hg.1` file.
REPOSITORY
rHG Mercurial
BRANCH
stable
REVISION DETAIL
https://phab.mercurial-scm.org/D8639
AFFECTED FILES
doc/gendoc.py
CHANGE DETAILS
diff --git a/doc/gendoc.py b/doc/gendoc.py
--- a/doc/gendoc.py
+++ b/doc/gendoc.py
@@ -85,7 +85,12 @@
if b'\n' in desc:
# only remove line breaks and indentation
desc = b' '.join(l.lstrip() for l in desc.split(b'\n'))
-desc += default and _(b" (default: %s)") % bytes(default) or b""
+if default:
+defaulttmpl = _(b" (default: %s)")
+if defaulttmpl:
+if not isinstance(default, bytes):
+default = repr(default).encode('latin1')
+desc += defaulttmpl % default
yield (b", ".join(allopts), desc)
To: spectral, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
How should we include extra modules in a pyoxidized Mercurial?
We're working on a PyOxidizer Mercurial for our users, and we want to bake some extra stuff beyond just hg (evolve, some in-house stuff) into the binary. Should we be making our own pyoxidizer.bzl that installs hg and whatever else we need? Is there a way we could reuse the bits in the existing configuration? Thanks! Augie ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D8374: fuzz: tell manifest fuzzer about longer node hashes
Closed by commit rHG9bedcfb4bb0e: fuzz: tell manifest fuzzer about longer node
hashes (authored by durin42).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs
Revision".
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D8374?vs=20977=21660
CHANGES SINCE LAST ACTION
https://phab.mercurial-scm.org/D8374/new/
REVISION DETAIL
https://phab.mercurial-scm.org/D8374
AFFECTED FILES
contrib/fuzz/manifest.cc
contrib/fuzz/manifest_corpus.py
CHANGE DETAILS
diff --git a/contrib/fuzz/manifest_corpus.py b/contrib/fuzz/manifest_corpus.py
--- a/contrib/fuzz/manifest_corpus.py
+++ b/contrib/fuzz/manifest_corpus.py
@@ -10,7 +10,7 @@
with zipfile.ZipFile(args.out[0], "w", zipfile.ZIP_STORED) as zf:
zf.writestr(
"manifest_zero",
-'''PKG-INFO\09b3ed8f2b81095a13064402e930565f083346e9a
+'''\0PKG-INFO\09b3ed8f2b81095a13064402e930565f083346e9a
README\080b6e76643dcb44d4bc729e932fc464b3e36dbe3
hg\0b6444347c629cc058d478023905cfb83b7f5bb9d
mercurial/__init__.py\0b80de5d138758541c5f05265ad144ab9fa86d1db
@@ -25,9 +25,14 @@
tkmerge\03c922edb43a9c143682f7bc7b00f98b3c756ebe7
''',
)
-zf.writestr("badmanifest_shorthashes", "narf\0aa\nnarf2\0aaa\n")
+zf.writestr("badmanifest_shorthashes", "\0narf\0aa\nnarf2\0aaa\n")
zf.writestr(
"badmanifest_nonull",
-"narf\0\n"
+"\0narf\0\n"
"narf2\n",
)
+
+zf.writestr(
+"manifest_long_nodes",
+
"\1a\0\n",
+)
diff --git a/contrib/fuzz/manifest.cc b/contrib/fuzz/manifest.cc
--- a/contrib/fuzz/manifest.cc
+++ b/contrib/fuzz/manifest.cc
@@ -3,6 +3,7 @@
#include
#include
+#include "FuzzedDataProvider.h"
#include "pyutil.h"
#include
@@ -24,7 +25,7 @@
lm[e]
e in lm
(e + 'nope') in lm
- lm[b'xyzzy'] = (b'\0' * 20, 'x')
+ lm[b'xyzzy'] = (b'\0' * nlen, 'x')
# do an insert, text should change
assert lm.text() != mdata, "insert should change text and didn't: %r %r" %
(lm.text(), mdata)
cloned = lm.filtercopy(lambda x: x != 'xyzzy')
@@ -51,10 +52,14 @@
if (Size > 10) {
return 0;
}
+ FuzzedDataProvider provider(Data, Size);
+ Py_ssize_t nodelength = provider.ConsumeBool() ? 20 : 32;
+ PyObject *nlen = PyLong_FromSsize_t(nodelength);
PyObject *mtext =
PyBytes_FromStringAndSize((const char *)Data, (Py_ssize_t)Size);
PyObject *locals = PyDict_New();
PyDict_SetItemString(locals, "mdata", mtext);
+ PyDict_SetItemString(locals, "nlen", nlen);
PyObject *res = PyEval_EvalCode(code, contrib::pyglobals(), locals);
if (!res) {
PyErr_Print();
To: durin42, #hg-reviewers, Alphare, pulkit
Cc: mercurial-patches, pulkit, Alphare, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 6] py3: pass native string to urlreq.url2pathname()
On 2020-06-17 13:26, Yuya Nishihara wrote: On Wed, 17 Jun 2020 03:51:29 +0200, Manuel Jacob wrote: In the following situation, the behavior is problematic: - We’re on Python 3. - The URL path contains a percent-encoded valid UTF-8 byte sequence. urlreq.url2pathname()’s return value is unicode and will contain the corresponding code point. - pycompat.fsencode() uses a different encoding than UTF-8 (e.g. ISO-8859-1). It will encode the code point to a different byte sequence. - The file will not be found and the warning introduced in this patch is not shown. On Python 2, the percent-decoded bytes are preserved (at least on Linux, I don’t have access to a Windows machine to verify). A proper fix would be to have our own implementation for urlreq.url2pathname() that works with bytes. This is the right thing to do on Unix. On Windows, I think that we should assume that the percent-decoded bytes are UTF-8 (see https://en.wikipedia.org/wiki/File_URI_scheme#Windows_2). But it seems like that would be a change from how it works on Python 2 (again, I don’t have a Windows machine to verify) and therefore should be changed in the default branch. What encoding is expected as a subversion URL? It might be UTF-8 since it is Subversion. Encoding handling in the convert extension is sometimes wrong. It's probably better to fix things rather than copying the Py2 behavior. You’re right. Subversion converts the command line arguments from the local encoding to UTF-8. The percent-encoded bytes are decoded as bytes within this UTF-8-encoded string and thus interpreted as UTF-8. When accessing the FS, the UTF-8 string is converted to the FS encoding. The SVN APIs called by the convert extension also expect UTF-8 bytes. I’ve found a few bugs because of that and will send patches. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D8374: fuzz: tell manifest fuzzer about longer node hashes
durin42 added a comment. This is still valid. REPOSITORY rHG Mercurial CHANGES SINCE LAST ACTION https://phab.mercurial-scm.org/D8374/new/ REVISION DETAIL https://phab.mercurial-scm.org/D8374 To: durin42, #hg-reviewers, Alphare Cc: mercurial-patches, pulkit, Alphare, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D8638: tests: update output in test-chg.t as setprocname missing in pure build
pulkit created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY setprocname is not present when we build a pure version. This leads to output changes in test-chg.t This should make test suite green on Python 2 pure build with chg. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D8638 AFFECTED FILES tests/test-chg.t CHANGE DETAILS diff --git a/tests/test-chg.t b/tests/test-chg.t --- a/tests/test-chg.t +++ b/tests/test-chg.t @@ -229,13 +229,14 @@ server.log.1 print only the last 10 lines, since we aren't sure how many records are -preserved (since setprocname isn't available on py3, the 10th-most-recent line -is different when using py3): +preserved (since setprocname isn't available on py3 and pure version, +the 10th-most-recent line is different when using py3): $ cat log/server.log.1 log/server.log | tail -10 | filterlog /MM/DD HH:MM:SS (PID)> confighash = ... mtimehash = ... (py3 !) + /MM/DD HH:MM:SS (PID)> confighash = ... mtimehash = ... (pure no-py3 !) /MM/DD HH:MM:SS (PID)> forked worker process (pid=...) - /MM/DD HH:MM:SS (PID)> setprocname: ... (no-py3 !) + /MM/DD HH:MM:SS (PID)> setprocname: ... (no-py3 no-pure !) /MM/DD HH:MM:SS (PID)> received fds: ... /MM/DD HH:MM:SS (PID)> chdir to '$TESTTMP/extreload' /MM/DD HH:MM:SS (PID)> setumask 18 To: pulkit, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 3] absorb: preserve branch-changing changesets even if empty
On Tue, 16 Jun 2020 15:58:12 +0200, Manuel Jacob wrote: > # HG changeset patch > # User Manuel Jacob > # Date 1591000380 -7200 > # Mon Jun 01 10:33:00 2020 +0200 > # Node ID 658889a51a83ba348575179ec210bd7a40f79ff0 > # Parent ef8dcee272acbe22af5829e31c55b694f345ceba > # EXP-Topic absorb-empty > absorb: preserve branch-changing changesets even if empty Queued, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 6] py3: pass native string to urlreq.url2pathname()
On Wed, 17 Jun 2020 03:51:29 +0200, Manuel Jacob wrote: > In the following situation, the behavior is problematic: > > - We’re on Python 3. > - The URL path contains a percent-encoded valid UTF-8 byte sequence. > urlreq.url2pathname()’s return value is unicode and will contain the > corresponding code point. > - pycompat.fsencode() uses a different encoding than UTF-8 (e.g. > ISO-8859-1). It will encode the code point to a different byte sequence. > - The file will not be found and the warning introduced in this patch is > not shown. > > On Python 2, the percent-decoded bytes are preserved (at least on Linux, > I don’t have access to a Windows machine to verify). > > A proper fix would be to have our own implementation for > urlreq.url2pathname() that works with bytes. This is the right thing to > do on Unix. On Windows, I think that we should assume that the > percent-decoded bytes are UTF-8 (see > https://en.wikipedia.org/wiki/File_URI_scheme#Windows_2). But it seems > like that would be a change from how it works on Python 2 (again, I > don’t have a Windows machine to verify) and therefore should be changed > in the default branch. What encoding is expected as a subversion URL? It might be UTF-8 since it is Subversion. Encoding handling in the convert extension is sometimes wrong. It's probably better to fix things rather than copying the Py2 behavior. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
