D1581: rust: implementation of `hg`
ruuda added a comment.
Awesome! I have just a few things that could be written more briefly.
INLINE COMMENTS
> build.rs:39
> +let script = "import sysconfig; \
> +c = sysconfig.get_config_vars(); \
> +print('SEPARATOR STRING'.join('%s=%s' % i for i in c.items()))";
You can safely indent those, leading whitespace on the continuation line will
be stripped from the string literal.
> build.rs:107
> +None => false,
> +};
> +
There is no need to match:
let result = config.config.get(*key) == Some("1");
Or using assert as @kevincox recommends:
assert_eq!(config.config.get(*key), Some("1"), "Detected ...");
> main.rs:187
> +Err(255)
> +}
> +Ok(()) => Ok(()),
There exists `Result::map_err` for this:
result = run_py(&env, py).map_err(|err| {
err.print(py);
255
});
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
To: indygreg, #hg-reviewers, yuja, durin42
Cc: ruuda, kevincox, cramertj, yuja, quark, durin42, dlax, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
kevincox added inline comments. INLINE COMMENTS > yuja wrote in main.rs:133 > That's more correct on Unix, but wouldn't work on Windows since > native string is UTF-16 variant (Rust) or ANSI (Python 2.7). Oops, for some reason I thought this was in a unix block. `as_bytes()` isn't actually available otherise. I think that this is fine then. non-UTF-8 paths shouldn't be a major issue.. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja, durin42 Cc: kevincox, cramertj, yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
yuja added inline comments. INLINE COMMENTS > kevincox wrote in main.rs:133 > `CString::new(env.python_exe.as_ref().as_bytes())` That's more correct on Unix, but wouldn't work on Windows since native string is UTF-16 variant (Rust) or ANSI (Python 2.7). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja, durin42 Cc: kevincox, cramertj, yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
kevincox added a comment.
Overall it looks great. I added a bunch of nitpicks. The most common
suggestion was to add some contextual information to error messages. Other then
that mostly minor style improvements. Feel free to push back on anything you
don't agree with :)
INLINE COMMENTS
> cramertj wrote in build.rs:88
> Nit: not sure what version you're targeting, but `'static` is automatic for
> `const` vars, so you could write `[&str; 2]`
I would also recommend using a slice if you don't intend the size of the array
to be part of the type signature.
const REQUIRED_CONFIG_FLAGS: &[&str] = &["Py_USING_UNICODE", "WITH_THREAD"];
> build.rs:33
> +);
> +}
> +
assert!(
Path::new(&python).exists(),
"Python interpreter {} does not exist; this should never happen",
python);
I would also recommend `{:?}` as the quotes highlight the injected variable
nicely and it will make some hard-to-notice things more visible due to escaping.
> build.rs:110
> +if !result {
> +panic!("Detected Python requires feature {}", key);
> +}
Use `assert!`.
> build.rs:116
> +if !have_shared(&config) {
> +panic!("Detected Python lacks a shared library, which is required");
> +}
Use `assert!`
> build.rs:127
> +panic!("Detected Python doesn't support UCS-4 code points");
> +}
> +}
#[cfg(not(target_os = "windows"))]
assert_eq!(
config.config.get("Py_UNICODE_SIZE"), Some("4"),
"Detected Python doesn't support UCS-4 code points");
> main.rs:37
> +fn get_environment() -> Environment {
> +let exe = env::current_exe().unwrap();
> +
Use expect for a better error message. `.expect("Error getting executable
path")`
> main.rs:91
> +.unwrap()
> +.into_raw();
> +unsafe {
This method allows paths that aren't valid UTF-8 by avoiding ever becoming a
`str`.
CString::new(env.python_home.as_ref().as_bytes())
I would also change the unwrap to `.expect("Error setting python home")`.
> main.rs:99
> +// Call sys.setdefaultencoding("undefined") if HGUNICODEPEDANTRY is set.
> +let pedantry = env::var("HGUNICODEPEDANTRY").is_ok();
> +
It appears that HG accepts `HGUNICODEPEDANTRY=` as not enabling unicode
pedantry. Maybe the behavior should be the same here. Untested code below
should work.
let pedantry = !env::var("HGUNICODEPEDANTRY").unwrap_or("").is_empty();
> main.rs:111
> +fn update_modules_path(env: &Environment, py: Python, sys_mod: &PyModule) {
> +let sys_path = sys_mod.get(py, "path").unwrap();
> +sys_path
`.expect("Error accessing sys.path")`
> main.rs:133
> +// not desirable.
> +let program_name = CString::new(env.python_exe.to_str().unwrap())
> +.unwrap()
`CString::new(env.python_exe.as_ref().as_bytes())`
> main.rs:134
> +let program_name = CString::new(env.python_exe.to_str().unwrap())
> +.unwrap()
> +.as_ptr();
`.expect("Error setting program name")`
> main.rs:200
> +fn run_py(env: &Environment, py: Python) -> PyResult<()> {
> +let sys_mod = py.import("sys").unwrap();
> +
Since this method returns a Result why not handle the error?
let sys_mod = py.import("sys")?;
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
To: indygreg, #hg-reviewers, yuja, durin42
Cc: kevincox, cramertj, yuja, quark, durin42, dlax, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
indygreg marked 7 inline comments as done. indygreg added a comment. I'll address this and other review feedback in follow-up patches. Thanks for the review! INLINE COMMENTS > cramertj wrote in main.rs:45 > Nit: you can just write `&str`. Also, I'm not familiar with what you're > trying to do here, but is the PYTHON_INTERPRETER always determined at > compile-time? It seems like something you might want to switch on at runtime. > Is that not the case? This is meant to be dynamic. The gist of this code is we're trying to find the location of the Python install given various search strategies. The search strategy is (currently) defined at compile time. And this `localdev` search strategy defines the path to Python at compile time. Look for my features and documentation for this code later. (I stripped out unused code from this patch to make it smaller.) REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja, durin42 Cc: cramertj, yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG964212780daf: rust: implementation of `hg` (authored by
indygreg, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D1581?vs=4753&id=4762
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
AFFECTED FILES
.hgignore
rust/.cargo/config
rust/Cargo.toml
rust/README.rst
rust/hgcli/Cargo.lock
rust/hgcli/Cargo.toml
rust/hgcli/build.rs
rust/hgcli/src/main.rs
tests/run-tests.py
CHANGE DETAILS
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2435,12 +2435,27 @@
self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
os.makedirs(self._tmpbindir)
-# This looks redundant with how Python initializes sys.path from
-# the location of the script being executed. Needed because the
-# "hg" specified by --with-hg is not the only Python script
-# executed in the test suite that needs to import 'mercurial'
-# ... which means it's not really redundant at all.
-self._pythondir = self._bindir
+normbin = os.path.normpath(os.path.abspath(whg))
+normbin = normbin.replace(os.sep.encode('ascii'), b'/')
+
+# Other Python scripts in the test harness need to
+# `import mercurial`. If `hg` is a Python script, we assume
+# the Mercurial modules are relative to its path and tell the tests
+# to load Python modules from its directory.
+with open(whg, 'rb') as fh:
+initial = fh.read(1024)
+
+if re.match(b'#!.*python', initial):
+self._pythondir = self._bindir
+# If it looks like our in-repo Rust binary, use the source root.
+# This is a bit hacky. But rhg is still not supported outside the
+# source directory. So until it is, do the simple thing.
+elif re.search(b'|/rust/target/[^/]+/hg', normbin):
+self._pythondir = os.path.dirname(self._testdir)
+# Fall back to the legacy behavior.
+else:
+self._pythondir = self._bindir
+
else:
self._installdir = os.path.join(self._hgtmp, b"install")
self._bindir = os.path.join(self._installdir, b"bin")
diff --git a/rust/hgcli/src/main.rs b/rust/hgcli/src/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgcli/src/main.rs
@@ -0,0 +1,222 @@
+// main.rs -- Main routines for `hg` program
+//
+// Copyright 2017 Gregory Szorc
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+extern crate libc;
+extern crate cpython;
+extern crate python27_sys;
+
+use cpython::{NoArgs, ObjectProtocol, PyModule, PyResult, Python};
+use libc::{c_char, c_int};
+
+use std::env;
+use std::path::PathBuf;
+use std::ffi::CString;
+#[cfg(target_family = "unix")]
+use std::os::unix::ffi::OsStringExt;
+
+#[derive(Debug)]
+struct Environment {
+_exe: PathBuf,
+python_exe: PathBuf,
+python_home: PathBuf,
+mercurial_modules: PathBuf,
+}
+
+/// Run Mercurial locally from a source distribution or checkout.
+///
+/// hg is /rust/target//hg
+/// Python interpreter is detected by build script.
+/// Python home is relative to Python interpreter.
+/// Mercurial files are relative to hg binary, which is relative to source
root.
+#[cfg(feature = "localdev")]
+fn get_environment() -> Environment {
+let exe = env::current_exe().unwrap();
+
+let mut mercurial_modules = exe.clone();
+mercurial_modules.pop(); // /rust/target/
+mercurial_modules.pop(); // /rust/target
+mercurial_modules.pop(); // /rust
+mercurial_modules.pop(); // /
+
+let python_exe: &'static str = env!("PYTHON_INTERPRETER");
+let python_exe = PathBuf::from(python_exe);
+
+let mut python_home = python_exe.clone();
+python_home.pop();
+
+// On Windows, python2.7.exe exists at the root directory of the Python
+// install. Everywhere else, the Python install root is one level up.
+if !python_exe.ends_with("python2.7.exe") {
+python_home.pop();
+}
+
+Environment {
+_exe: exe.clone(),
+python_exe: python_exe,
+python_home: python_home,
+mercurial_modules: mercurial_modules.to_path_buf(),
+}
+}
+
+// On UNIX, argv starts as an array of char*. So it is easy to convert
+// to C strings.
+#[cfg(target_family = "unix")]
+fn args_to_cstrings() -> Vec {
+env::args_os()
+.map(|a| CString::new(a.into_vec()).unwrap())
+.collect()
+}
+
+// TODO Windows support is incomplete. We should either use env::args_os()
+// (or call into GetCommandLineW() + CommandLinetoArgvW()), convert these to
+// PyUnicode instances, and pass these into Python/Mercurial outside the
+// standard P
D1581: rust: implementation of `hg`
cramertj added a comment. In https://phab.mercurial-scm.org/D1581#31035, @indygreg wrote: > In https://phab.mercurial-scm.org/D1581#31029, @cramertj wrote: > > > There are a lot of uses of `unwrap`, `expect`, and `panic` that can (and probably should) be replaced with proper error handling using `Result` (and the `failure` crate). > > > I definitely agree. I still consider this just beyond proof-of-concept code. We'll definitely want to shore things up before we ship. Perfect is very much the enemy of good at this stage. > > > There are also a couple of crates that provide safe bindings to Python interpreters-- I'm not sure what your external dependency situation is, but you might consider using something like https://crates.io/crates/pyo3 rather than writing your own `unsafe` calls to the python interpreter. > > pyo3 requires non-stable Rust features last I checked. That makes it a non-starter for us at this time (since downstream packagers will insist on only using stable Rust). > > If other external dependencies provide the interfaces we need, I'm open to taking those dependencies. But this crate is focused on embedding a Python interpreter. Most (all?) of the Rust+Python crates I found seemed to target the "implementing Python extensions with Rust" use case, not embedding Python. As such, their embedding API support is very lacking. I even had to fork rust-cpython because it didn't implement the proper APIs and is forcing its extension-centric religion on consumers. I've upstreamed most of my modifications though. So hopefully the fork doesn't live much longer... SGTM! Thanks for clarifying. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja Cc: cramertj, yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
indygreg added a comment. In https://phab.mercurial-scm.org/D1581#31029, @cramertj wrote: > There are a lot of uses of `unwrap`, `expect`, and `panic` that can (and probably should) be replaced with proper error handling using `Result` (and the `failure` crate). I definitely agree. I still consider this just beyond proof-of-concept code. We'll definitely want to shore things up before we ship. Perfect is very much the enemy of good at this stage. > There are also a couple of crates that provide safe bindings to Python interpreters-- I'm not sure what your external dependency situation is, but you might consider using something like https://crates.io/crates/pyo3 rather than writing your own `unsafe` calls to the python interpreter. pyo3 requires non-stable Rust features last I checked. That makes it a non-starter for us at this time (since downstream packagers will insist on only using stable Rust). If other external dependencies provide the interfaces we need, I'm open to taking those dependencies. But this crate is focused on embedding a Python interpreter. Most (all?) of the Rust+Python crates I found seemed to target the "implementing Python extensions with Rust" use case, not embedding Python. As such, their embedding API support is very lacking. I even had to fork rust-cpython because it didn't implement the proper APIs and is forcing its extension-centric religion on consumers. I've upstreamed most of my modifications though. So hopefully the fork doesn't live much longer... REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja Cc: cramertj, yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
cramertj added a comment.
Looks pretty good to me-- left a couple nits inline.
There are a lot of uses of `unwrap`, `expect`, and `panic` that can (and
probably should) be replaced with proper error handling using `Result` (and the
`failure` crate).
There are also a couple of crates that provide safe bindings to Python
interpreters-- I'm not sure what your external dependency situation is, but you
might consider using something like https://crates.io/crates/pyo3 rather than
writing your own `unsafe` calls to the python interpreter.
INLINE COMMENTS
> build.rs:12
> +#[cfg(target_os = "windows")]
> +use std::path::PathBuf;
> +
Nit: if you move this import into `have_shared`, you'll only need one `cfg` and
it'll be easier to validate that the proper deps are available for the proper
platforms.
> build.rs:88
> +
> +const REQUIRED_CONFIG_FLAGS: [&'static str; 2] = ["Py_USING_UNICODE",
> "WITH_THREAD"];
> +
Nit: not sure what version you're targeting, but `'static` is automatic for
`const` vars, so you could write `[&str; 2]`
> main.rs:45
> +
> +let python_exe: &'static str = env!("PYTHON_INTERPRETER");
> +let python_exe = PathBuf::from(python_exe);
Nit: you can just write `&str`. Also, I'm not familiar with what you're trying
to do here, but is the PYTHON_INTERPRETER always determined at compile-time? It
seems like something you might want to switch on at runtime. Is that not the
case?
> main.rs:125
> +
> +// Set program name. The backing memory needs to live for the duration
> of the
> +// interpreter.
If it needs to live for the whole time, consider storing it in a `static` or
similar. There's a subtle unsafety here: if this program panics (due to
`panic`, `unwrap`, or `expect`, `program_name` will be dropped (free'd) before
the python interpreter is killed (when the process ends, that is-- `Finalize`
won't ever be called in that case). I don't know how much of an issue this will
be in practice, but it's something to think about.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
To: indygreg, #hg-reviewers, yuja
Cc: cramertj, yuja, quark, durin42, dlax, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
indygreg updated this revision to Diff 4753.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D1581?vs=4745&id=4753
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
AFFECTED FILES
.hgignore
rust/.cargo/config
rust/Cargo.toml
rust/README.rst
rust/hgcli/Cargo.lock
rust/hgcli/Cargo.toml
rust/hgcli/build.rs
rust/hgcli/src/main.rs
tests/run-tests.py
CHANGE DETAILS
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2435,12 +2435,27 @@
self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
os.makedirs(self._tmpbindir)
-# This looks redundant with how Python initializes sys.path from
-# the location of the script being executed. Needed because the
-# "hg" specified by --with-hg is not the only Python script
-# executed in the test suite that needs to import 'mercurial'
-# ... which means it's not really redundant at all.
-self._pythondir = self._bindir
+normbin = os.path.normpath(os.path.abspath(whg))
+normbin = normbin.replace(os.sep.encode('ascii'), b'/')
+
+# Other Python scripts in the test harness need to
+# `import mercurial`. If `hg` is a Python script, we assume
+# the Mercurial modules are relative to its path and tell the tests
+# to load Python modules from its directory.
+with open(whg, 'rb') as fh:
+initial = fh.read(1024)
+
+if re.match(b'#!.*python', initial):
+self._pythondir = self._bindir
+# If it looks like our in-repo Rust binary, use the source root.
+# This is a bit hacky. But rhg is still not supported outside the
+# source directory. So until it is, do the simple thing.
+elif re.search(b'|/rust/target/[^/]+/hg', normbin):
+self._pythondir = os.path.dirname(self._testdir)
+# Fall back to the legacy behavior.
+else:
+self._pythondir = self._bindir
+
else:
self._installdir = os.path.join(self._hgtmp, b"install")
self._bindir = os.path.join(self._installdir, b"bin")
diff --git a/rust/hgcli/src/main.rs b/rust/hgcli/src/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgcli/src/main.rs
@@ -0,0 +1,222 @@
+// main.rs -- Main routines for `hg` program
+//
+// Copyright 2017 Gregory Szorc
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+extern crate libc;
+extern crate cpython;
+extern crate python27_sys;
+
+use cpython::{NoArgs, ObjectProtocol, PyModule, PyResult, Python};
+use libc::{c_char, c_int};
+
+use std::env;
+use std::path::PathBuf;
+use std::ffi::CString;
+#[cfg(target_family = "unix")]
+use std::os::unix::ffi::OsStringExt;
+
+#[derive(Debug)]
+struct Environment {
+_exe: PathBuf,
+python_exe: PathBuf,
+python_home: PathBuf,
+mercurial_modules: PathBuf,
+}
+
+/// Run Mercurial locally from a source distribution or checkout.
+///
+/// hg is /rust/target//hg
+/// Python interpreter is detected by build script.
+/// Python home is relative to Python interpreter.
+/// Mercurial files are relative to hg binary, which is relative to source
root.
+#[cfg(feature = "localdev")]
+fn get_environment() -> Environment {
+let exe = env::current_exe().unwrap();
+
+let mut mercurial_modules = exe.clone();
+mercurial_modules.pop(); // /rust/target/
+mercurial_modules.pop(); // /rust/target
+mercurial_modules.pop(); // /rust
+mercurial_modules.pop(); // /
+
+let python_exe: &'static str = env!("PYTHON_INTERPRETER");
+let python_exe = PathBuf::from(python_exe);
+
+let mut python_home = python_exe.clone();
+python_home.pop();
+
+// On Windows, python2.7.exe exists at the root directory of the Python
+// install. Everywhere else, the Python install root is one level up.
+if !python_exe.ends_with("python2.7.exe") {
+python_home.pop();
+}
+
+Environment {
+_exe: exe.clone(),
+python_exe: python_exe,
+python_home: python_home,
+mercurial_modules: mercurial_modules.to_path_buf(),
+}
+}
+
+// On UNIX, argv starts as an array of char*. So it is easy to convert
+// to C strings.
+#[cfg(target_family = "unix")]
+fn args_to_cstrings() -> Vec {
+env::args_os()
+.map(|a| CString::new(a.into_vec()).unwrap())
+.collect()
+}
+
+// TODO Windows support is incomplete. We should either use env::args_os()
+// (or call into GetCommandLineW() + CommandLinetoArgvW()), convert these to
+// PyUnicode instances, and pass these into Python/Mercurial outside the
+// standard PySys_SetArgvEx() mechanism. This will allow us to preserve the
+// raw bytes (since PySys_SetArgvEx() is based on char* and can drop
D1581: rust: implementation of `hg`
yuja added inline comments. INLINE COMMENTS > indygreg wrote in Cargo.lock:1 > According to > https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries > (and other places I found on the Internet), `Cargo.lock` files under version > control are a good practice for binaries - but not libraries. > > Since this `Cargo.lock` encompasses the `hgcli` binary, I think that counts. > > FWIW, Firefox vendors `Cargo.lock` files for libraries. Although those > libraries are attached to C++ binaries, so there isn't an appropriate Rust > binary to attach to. But a main purpose of `Cargo.lock` files is to add > determinism. For that reason, I think we want them vendored. > Cargo.lock files under version control are a good practice for binaries Okay, I didn't know that, thanks. (FWIW, I don't like the idea of version-controlling strict build dependencies, but let's just follow their rule.) Can we move Cargo.lock under hgcli directory, then? Sounds like we'll need a separate Cargo.lock file for "libraries". REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja Cc: yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
indygreg updated this revision to Diff 4745.
indygreg marked an inline comment as done.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D1581?vs=4744&id=4745
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
AFFECTED FILES
.hgignore
rust/.cargo/config
rust/Cargo.lock
rust/Cargo.toml
rust/README.rst
rust/hgcli/Cargo.toml
rust/hgcli/build.rs
rust/hgcli/src/main.rs
tests/run-tests.py
CHANGE DETAILS
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2435,12 +2435,27 @@
self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
os.makedirs(self._tmpbindir)
-# This looks redundant with how Python initializes sys.path from
-# the location of the script being executed. Needed because the
-# "hg" specified by --with-hg is not the only Python script
-# executed in the test suite that needs to import 'mercurial'
-# ... which means it's not really redundant at all.
-self._pythondir = self._bindir
+normbin = os.path.normpath(os.path.abspath(whg))
+normbin = normbin.replace(os.sep.encode('ascii'), b'/')
+
+# Other Python scripts in the test harness need to
+# `import mercurial`. If `hg` is a Python script, we assume
+# the Mercurial modules are relative to its path and tell the tests
+# to load Python modules from its directory.
+with open(whg, 'rb') as fh:
+initial = fh.read(1024)
+
+if re.match(b'#!.*python', initial):
+self._pythondir = self._bindir
+# If it looks like our in-repo Rust binary, use the source root.
+# This is a bit hacky. But rhg is still not supported outside the
+# source directory. So until it is, do the simple thing.
+elif re.search(b'|/rust/target/[^/]+/hg', normbin):
+self._pythondir = os.path.dirname(self._testdir)
+# Fall back to the legacy behavior.
+else:
+self._pythondir = self._bindir
+
else:
self._installdir = os.path.join(self._hgtmp, b"install")
self._bindir = os.path.join(self._installdir, b"bin")
diff --git a/rust/hgcli/src/main.rs b/rust/hgcli/src/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgcli/src/main.rs
@@ -0,0 +1,222 @@
+// main.rs -- Main routines for `hg` program
+//
+// Copyright 2017 Gregory Szorc
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+extern crate libc;
+extern crate cpython;
+extern crate python27_sys;
+
+use cpython::{NoArgs, ObjectProtocol, PyModule, PyResult, Python};
+use libc::{c_char, c_int};
+
+use std::env;
+use std::path::PathBuf;
+use std::ffi::CString;
+#[cfg(target_family = "unix")]
+use std::os::unix::ffi::OsStringExt;
+
+#[derive(Debug)]
+struct Environment {
+_exe: PathBuf,
+python_exe: PathBuf,
+python_home: PathBuf,
+mercurial_modules: PathBuf,
+}
+
+/// Run Mercurial locally from a source distribution or checkout.
+///
+/// hg is /rust/target//hg
+/// Python interpreter is detected by build script.
+/// Python home is relative to Python interpreter.
+/// Mercurial files are relative to hg binary, which is relative to source
root.
+#[cfg(feature = "localdev")]
+fn get_environment() -> Environment {
+let exe = env::current_exe().unwrap();
+
+let mut mercurial_modules = exe.clone();
+mercurial_modules.pop(); // /rust/target/
+mercurial_modules.pop(); // /rust/target
+mercurial_modules.pop(); // /rust
+mercurial_modules.pop(); // /
+
+let python_exe: &'static str = env!("PYTHON_INTERPRETER");
+let python_exe = PathBuf::from(python_exe);
+
+let mut python_home = python_exe.clone();
+python_home.pop();
+
+// On Windows, python2.7.exe exists at the root directory of the Python
+// install. Everywhere else, the Python install root is one level up.
+if !python_exe.ends_with("python2.7.exe") {
+python_home.pop();
+}
+
+Environment {
+_exe: exe.clone(),
+python_exe: python_exe,
+python_home: python_home,
+mercurial_modules: mercurial_modules.to_path_buf(),
+}
+}
+
+// On UNIX, argv starts as an array of char*. So it is easy to convert
+// to C strings.
+#[cfg(target_family = "unix")]
+fn args_to_cstrings() -> Vec {
+env::args_os()
+.map(|a| CString::new(a.into_vec()).unwrap())
+.collect()
+}
+
+// TODO Windows support is incomplete. We should either use env::args_os()
+// (or call into GetCommandLineW() + CommandLinetoArgvW()), convert these to
+// PyUnicode instances, and pass these into Python/Mercurial outside the
+// standard PySys_SetArgvEx() mechanism. This will allow us to preserve the
+// raw bytes (since PySys_SetAr
D1581: rust: implementation of `hg`
indygreg updated this revision to Diff 4744.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D1581?vs=4525&id=4744
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
AFFECTED FILES
.hgignore
rust/.cargo/config
rust/Cargo.lock
rust/Cargo.toml
rust/README.rst
rust/hgcli/Cargo.toml
rust/hgcli/build.rs
rust/hgcli/src/main.rs
tests/run-tests.py
CHANGE DETAILS
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2435,12 +2435,27 @@
self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
os.makedirs(self._tmpbindir)
-# This looks redundant with how Python initializes sys.path from
-# the location of the script being executed. Needed because the
-# "hg" specified by --with-hg is not the only Python script
-# executed in the test suite that needs to import 'mercurial'
-# ... which means it's not really redundant at all.
-self._pythondir = self._bindir
+normbin = os.path.normpath(os.path.abspath(whg))
+normbin = normbin.replace(os.sep.encode('ascii'), b'/')
+
+# Other Python scripts in the test harness need to
+# `import mercurial`. If `hg` is a Python script, we assume
+# the Mercurial modules are relative to its path and tell the tests
+# to load Python modules from its directory.
+with open(whg, 'rb') as fh:
+initial = fh.read(1024)
+
+if re.match(b'#!.*python', initial):
+self._pythondir = self._bindir
+# If it looks like our in-repo Rust binary, use the source root.
+# This is a bit hacky. But rhg is still not supported outside the
+# source directory. So until it is, do the simple thing.
+elif re.search(b'|/rust/target/[^/]+/hg', normbin):
+self._pythondir = os.path.dirname(self._testdir)
+# Fall back to the legacy behavior.
+else:
+self._pythondir = self._bindir
+
else:
self._installdir = os.path.join(self._hgtmp, b"install")
self._bindir = os.path.join(self._installdir, b"bin")
diff --git a/rust/hgcli/src/main.rs b/rust/hgcli/src/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgcli/src/main.rs
@@ -0,0 +1,222 @@
+// main.rs -- Main routines for `hg` program
+//
+// Copyright 2017 Gregory Szorc
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+extern crate libc;
+extern crate cpython;
+extern crate python27_sys;
+
+use cpython::{NoArgs, ObjectProtocol, PyModule, PyResult, Python};
+use libc::{c_char, c_int};
+
+use std::env;
+use std::path::PathBuf;
+use std::ffi::CString;
+#[cfg(target_family = "unix")]
+use std::os::unix::ffi::OsStringExt;
+
+#[derive(Debug)]
+struct Environment {
+_exe: PathBuf,
+python_exe: PathBuf,
+python_home: PathBuf,
+mercurial_modules: PathBuf,
+}
+
+/// Run Mercurial locally from a source distribution or checkout.
+///
+/// hg is /rust/target//hg
+/// Python interpreter is detected by build script.
+/// Python home is relative to Python interpreter.
+/// Mercurial files are relative to hg binary, which is relative to source
root.
+#[cfg(feature = "localdev")]
+fn get_environment() -> Environment {
+let exe = env::current_exe().unwrap();
+
+let mut mercurial_modules = exe.clone();
+mercurial_modules.pop(); // /rust/target/
+mercurial_modules.pop(); // /rust/target
+mercurial_modules.pop(); // /rust
+mercurial_modules.pop(); // /
+
+let python_exe: &'static str = env!("PYTHON_INTERPRETER");
+let python_exe = PathBuf::from(python_exe);
+
+let mut python_home = python_exe.clone();
+python_home.pop();
+
+// On Windows, python2.7.exe exists at the root directory of the Python
+// install. Everywhere else, the Python install root is one level up.
+if !python_exe.ends_with("python2.7.exe") {
+python_home.pop();
+}
+
+Environment {
+_exe: exe.clone(),
+python_exe: python_exe.clone(),
+python_home: python_home.clone(),
+mercurial_modules: mercurial_modules.to_path_buf(),
+}
+}
+
+// On UNIX, argv starts as an array of char*. So it is easy to convert
+// to C strings.
+#[cfg(target_family = "unix")]
+fn args_to_cstrings() -> Vec {
+env::args_os()
+.map(|a| CString::new(a.into_vec()).unwrap())
+.collect()
+}
+
+// TODO Windows support is incomplete. We should either use env::args_os()
+// (or call into GetCommandLineW() + CommandLinetoArgvW()), convert these to
+// PyUnicode instances, and pass these into Python/Mercurial outside the
+// standard PySys_SetArgvEx() mechanism. This will allow us to preserve the
+// raw bytes (since PySys_SetArgvEx() is based on char* an
D1581: rust: implementation of `hg`
indygreg added inline comments. INLINE COMMENTS > yuja wrote in Cargo.lock:1 > Perhaps Cargo.lock should be excluded from the commit. According to https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries (and other places I found on the Internet), `Cargo.lock` files under version control are a good practice for binaries - but not libraries. Since this `Cargo.lock` encompasses the `hgcli` binary, I think that counts. FWIW, Firefox vendors `Cargo.lock` files for libraries. Although those libraries are attached to C++ binaries, so there isn't an appropriate Rust binary to attach to. But a main purpose of `Cargo.lock` files is to add determinism. For that reason, I think we want them vendored. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja Cc: yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
yuja requested changes to this revision. yuja added a comment. This revision now requires changes to proceed. Suppose this is a kind of contrib code, I think it's good enough to accept. Can you drop Cargo.lock file? INLINE COMMENTS > Cargo.lock:1 > +[[package]] > +name = "aho-corasick" Perhaps Cargo.lock should be excluded from the commit. > build.rs:88 > + > +static REQUIRED_CONFIG_FLAGS: [&'static str; 2] = ["Py_USING_UNICODE", > "WITH_THREAD"]; > + Nit: `const` ? > main.rs:62 > +mercurial_modules: mercurial_modules.to_path_buf(), > +} > +} Nit: probably we don't have to `clone()` here, just move these values. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers, yuja Cc: yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
indygreg updated this revision to Diff 4525.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D1581?vs=4244&id=4525
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
AFFECTED FILES
.hgignore
rust/.cargo/config
rust/Cargo.lock
rust/Cargo.toml
rust/README.rst
rust/hgcli/Cargo.toml
rust/hgcli/build.rs
rust/hgcli/src/main.rs
tests/run-tests.py
CHANGE DETAILS
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2438,12 +2438,27 @@
self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
os.makedirs(self._tmpbindir)
-# This looks redundant with how Python initializes sys.path from
-# the location of the script being executed. Needed because the
-# "hg" specified by --with-hg is not the only Python script
-# executed in the test suite that needs to import 'mercurial'
-# ... which means it's not really redundant at all.
-self._pythondir = self._bindir
+normbin = os.path.normpath(os.path.abspath(whg))
+normbin = normbin.replace(os.sep.encode('ascii'), b'/')
+
+# Other Python scripts in the test harness need to
+# `import mercurial`. If `hg` is a Python script, we assume
+# the Mercurial modules are relative to its path and tell the tests
+# to load Python modules from its directory.
+with open(whg, 'rb') as fh:
+initial = fh.read(1024)
+
+if re.match(b'#!.*python', initial):
+self._pythondir = self._bindir
+# If it looks like our in-repo Rust binary, use the source root.
+# This is a bit hacky. But rhg is still not supported outside the
+# source directory. So until it is, do the simple thing.
+elif re.search(b'|/rust/target/[^/]+/hg', normbin):
+self._pythondir = os.path.dirname(self._testdir)
+# Fall back to the legacy behavior.
+else:
+self._pythondir = self._bindir
+
else:
self._installdir = os.path.join(self._hgtmp, b"install")
self._bindir = os.path.join(self._installdir, b"bin")
diff --git a/rust/hgcli/src/main.rs b/rust/hgcli/src/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgcli/src/main.rs
@@ -0,0 +1,222 @@
+// main.rs -- Main routines for `hg` program
+//
+// Copyright 2017 Gregory Szorc
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+extern crate libc;
+extern crate cpython;
+extern crate python27_sys;
+
+use cpython::{NoArgs, ObjectProtocol, PyModule, PyResult, Python};
+use libc::{c_char, c_int};
+
+use std::env;
+use std::path::PathBuf;
+use std::ffi::CString;
+#[cfg(target_family = "unix")]
+use std::os::unix::ffi::OsStringExt;
+
+#[derive(Debug)]
+struct Environment {
+_exe: PathBuf,
+python_exe: PathBuf,
+python_home: PathBuf,
+mercurial_modules: PathBuf,
+}
+
+/// Run Mercurial locally from a source distribution or checkout.
+///
+/// hg is /rust/target//hg
+/// Python interpreter is detected by build script.
+/// Python home is relative to Python interpreter.
+/// Mercurial files are relative to hg binary, which is relative to source
root.
+#[cfg(feature = "localdev")]
+fn get_environment() -> Environment {
+let exe = env::current_exe().unwrap();
+
+let mut mercurial_modules = exe.clone();
+mercurial_modules.pop(); // /rust/target/
+mercurial_modules.pop(); // /rust/target
+mercurial_modules.pop(); // /rust
+mercurial_modules.pop(); // /
+
+let python_exe: &'static str = env!("PYTHON_INTERPRETER");
+let python_exe = PathBuf::from(python_exe);
+
+let mut python_home = python_exe.clone();
+python_home.pop();
+
+// On Windows, python2.7.exe exists at the root directory of the Python
+// install. Everywhere else, the Python install root is one level up.
+if !python_exe.ends_with("python2.7.exe") {
+python_home.pop();
+}
+
+Environment {
+_exe: exe.clone(),
+python_exe: python_exe.clone(),
+python_home: python_home.clone(),
+mercurial_modules: mercurial_modules.to_path_buf(),
+}
+}
+
+// On UNIX, argv starts as an array of char*. So it is easy to convert
+// to C strings.
+#[cfg(target_family = "unix")]
+fn args_to_cstrings() -> Vec {
+env::args_os()
+.map(|a| CString::new(a.into_vec()).unwrap())
+.collect()
+}
+
+// TODO Windows support is incomplete. We should either use env::args_os()
+// (or call into GetCommandLineW() + CommandLinetoArgvW()), convert these to
+// PyUnicode instances, and pass these into Python/Mercurial outside the
+// standard PySys_SetArgvEx() mechanism. This will allow us to preserve the
+// raw bytes (since PySys_SetArgvEx() is based on char* an
D1581: rust: implementation of `hg`
indygreg marked 6 inline comments as done. indygreg added a comment. I removed the standalone distribution code and cleaned things up a bit. Over 97% of the tests pass with the Rust `hg` on Linux. And the failures seem reasonable. There's a bit of work to be done around packaging, Windows support, Rust linting, etc. But I'd like to get something landed so others have something to play with and so we're not reviewing a massive patch. What I'm trying to say is "I think this is ready for a real review." INLINE COMMENTS > quark wrote in main.rs:120 > Probably use `PyResult` since `dispatch.run` returns the exit code. ``dispatch.run()`` actually calls ``sys.exit()``. There is a bug around somewhere in this code around exit code handling. But it only presents in a few tests, which is odd. I documented that in the commit message. > yuja wrote in main.rs:101 > Perhaps we'll need a utility function for platform-specific cstr > conversion. > > On Unix, the story is simple. We can use OsStrExt. On Windows, > maybe we'll have to first convert it to "wide characters" by OsStrExt > and then call `WideCharToMultiByte` to convert it back to ANSI bytes sequence. > > I have no idea if Rust stdlib provides a proper way to convert > OsStr to Windows ANSI bytes. > > https://stackoverflow.com/a/38948854 > https://msdn.microsoft.com/en-us/library/windows/desktop/dd374130(v=vs.85).aspx I documented a potential path forward in the code. I was thinking we can address this is a follow-up because it is non-trivial to implement. And we may want to do something funky, such as inject the raw bytes into `mercurial.argv` or something. Since we can call the Windows API directly, we can actually do that now. This opens up a lot of possibilities for encoding... REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1581 To: indygreg, #hg-reviewers Cc: yuja, quark, durin42, dlax, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1581: rust: implementation of `hg`
indygreg updated this revision to Diff 4244.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D1581?vs=4241&id=4244
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
AFFECTED FILES
.hgignore
rust/.cargo/config
rust/Cargo.lock
rust/Cargo.toml
rust/README.rst
rust/hgcli/Cargo.toml
rust/hgcli/build.rs
rust/hgcli/src/main.rs
tests/run-tests.py
CHANGE DETAILS
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2436,12 +2436,27 @@
self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
os.makedirs(self._tmpbindir)
-# This looks redundant with how Python initializes sys.path from
-# the location of the script being executed. Needed because the
-# "hg" specified by --with-hg is not the only Python script
-# executed in the test suite that needs to import 'mercurial'
-# ... which means it's not really redundant at all.
-self._pythondir = self._bindir
+normbin = os.path.normpath(os.path.abspath(whg))
+normbin = normbin.replace(os.sep.encode('ascii'), b'/')
+
+# Other Python scripts in the test harness need to
+# `import mercurial`. If `hg` is a Python script, we assume
+# the Mercurial modules are relative to its path and tell the tests
+# to load Python modules from its directory.
+with open(whg, 'rb') as fh:
+initial = fh.read(1024)
+
+if re.match(b'#!.*python', initial):
+self._pythondir = self._bindir
+# If it looks like our in-repo Rust binary, use the source root.
+# This is a bit hacky. But rhg is still not supported outside the
+# source directory. So until it is, do the simple thing.
+elif re.search(b'|/rust/target/[^/]+/hg', normbin):
+self._pythondir = os.path.dirname(self._testdir)
+# Fall back to the legacy behavior.
+else:
+self._pythondir = self._bindir
+
else:
self._installdir = os.path.join(self._hgtmp, b"install")
self._bindir = os.path.join(self._installdir, b"bin")
diff --git a/rust/hgcli/src/main.rs b/rust/hgcli/src/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgcli/src/main.rs
@@ -0,0 +1,222 @@
+// main.rs -- Main routines for `hg` program
+//
+// Copyright 2017 Gregory Szorc
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+extern crate libc;
+extern crate cpython;
+extern crate python27_sys;
+
+use cpython::{NoArgs, ObjectProtocol, PyModule, PyResult, Python};
+use libc::{c_char, c_int};
+
+use std::env;
+use std::path::PathBuf;
+use std::ffi::CString;
+#[cfg(target_family = "unix")]
+use std::os::unix::ffi::OsStringExt;
+
+#[derive(Debug)]
+struct Environment {
+_exe: PathBuf,
+python_exe: PathBuf,
+python_home: PathBuf,
+mercurial_modules: PathBuf,
+}
+
+/// Run Mercurial locally from a source distribution or checkout.
+///
+/// hg is /rust/target//hg
+/// Python interpreter is detected by build script.
+/// Python home is relative to Python interpreter.
+/// Mercurial files are relative to hg binary, which is relative to source
root.
+#[cfg(feature = "localdev")]
+fn get_environment() -> Environment {
+let exe = env::current_exe().unwrap();
+
+let mut mercurial_modules = exe.clone();
+mercurial_modules.pop(); // /rust/target/
+mercurial_modules.pop(); // /rust/target
+mercurial_modules.pop(); // /rust
+mercurial_modules.pop(); // /
+
+let python_exe: &'static str = env!("PYTHON_INTERPRETER");
+let python_exe = PathBuf::from(python_exe);
+
+let mut python_home = python_exe.clone();
+python_home.pop();
+
+// On Windows, python2.7.exe exists at the root directory of the Python
+// install. Everywhere else, the Python install root is one level up.
+if !python_exe.ends_with("python2.7.exe") {
+python_home.pop();
+}
+
+Environment {
+_exe: exe.clone(),
+python_exe: python_exe.clone(),
+python_home: python_home.clone(),
+mercurial_modules: mercurial_modules.to_path_buf(),
+}
+}
+
+// On UNIX, argv starts as an array of char*. So it is easy to convert
+// to C strings.
+#[cfg(target_family = "unix")]
+fn args_to_cstrings() -> Vec {
+env::args_os()
+.map(|a| CString::new(a.into_vec()).unwrap())
+.collect()
+}
+
+// TODO Windows support is incomplete. We should either use env::args_os()
+// (or call into GetCommandLineW() + CommandLinetoArgvW()), convert these to
+// PyUnicode instances, and pass these into Python/Mercurial outside the
+// standard PySys_SetArgvEx() mechanism. This will allow us to preserve the
+// raw bytes (since PySys_SetArgvEx() is based on char* an
D1581: rust: implementation of `hg`
indygreg updated this revision to Diff 4241.
indygreg edited the summary of this revision.
indygreg retitled this revision from "[RFC] rust: Rust implementation of `hg`
and standalone packaging" to "rust: implementation of `hg`".
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D1581?vs=4165&id=4241
REVISION DETAIL
https://phab.mercurial-scm.org/D1581
AFFECTED FILES
.hgignore
rust/.cargo/config
rust/Cargo.lock
rust/Cargo.toml
rust/README.rst
rust/hgcli/Cargo.toml
rust/hgcli/build.rs
rust/hgcli/src/main.rs
tests/run-tests.py
CHANGE DETAILS
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2436,12 +2436,27 @@
self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
os.makedirs(self._tmpbindir)
-# This looks redundant with how Python initializes sys.path from
-# the location of the script being executed. Needed because the
-# "hg" specified by --with-hg is not the only Python script
-# executed in the test suite that needs to import 'mercurial'
-# ... which means it's not really redundant at all.
-self._pythondir = self._bindir
+normbin = os.path.normpath(os.path.abspath(whg))
+normbin = normbin.replace(os.sep.encode('ascii'), b'/')
+
+# Other Python scripts in the test harness need to
+# `import mercurial`. If `hg` is a Python script, we assume
+# the Mercurial modules are relative to its path and tell the tests
+# to load Python modules from its directory.
+with open(whg, 'rb') as fh:
+initial = fh.read(1024)
+
+if re.match(b'#!.*python', initial):
+self._pythondir = self._bindir
+# If it looks like our in-repo Rust binary, use the source root.
+# This is a bit hacky. But rhg is still not supported outside the
+# source directory. So until it is, do the simple thing.
+elif re.search(b'|/rust/target/[^/]+/hg', normbin):
+self._pythondir = os.path.dirname(self._testdir)
+# Fall back to the legacy behavior.
+else:
+self._pythondir = self._bindir
+
else:
self._installdir = os.path.join(self._hgtmp, b"install")
self._bindir = os.path.join(self._installdir, b"bin")
diff --git a/rust/hgcli/src/main.rs b/rust/hgcli/src/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgcli/src/main.rs
@@ -0,0 +1,220 @@
+// main.rs -- Main routines for `hg` program
+//
+// Copyright 2017 Gregory Szorc
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+extern crate libc;
+extern crate cpython;
+extern crate python27_sys;
+
+use cpython::{NoArgs, ObjectProtocol, PyModule, PyResult, Python};
+use libc::{c_char, c_int};
+
+use std::env;
+use std::path::PathBuf;
+use std::ffi::CString;
+#[cfg(target_family = "unix")]
+use std::os::unix::ffi::OsStringExt;
+
+#[derive(Debug)]
+struct Environment {
+_exe: PathBuf,
+python_exe: PathBuf,
+python_home: PathBuf,
+mercurial_modules: PathBuf,
+}
+
+/// Run Mercurial locally from a source distribution or checkout.
+///
+/// hg is /rust/target//hg
+/// Python interpreter is detected by build script.
+/// Python home is relative to Python interpreter.
+/// Mercurial files are relative to hg binary, which is relative to source
root.
+#[cfg(feature = "localdev")]
+fn get_environment() -> Environment {
+let exe = env::current_exe().unwrap();
+
+let mut mercurial_modules = exe.clone();
+mercurial_modules.pop(); // /rust/target/
+mercurial_modules.pop(); // /rust/target
+mercurial_modules.pop(); // /rust
+mercurial_modules.pop(); // /
+
+let python_exe: &'static str = env!("PYTHON_INTERPRETER");
+let python_exe = PathBuf::from(python_exe);
+
+let mut python_home = python_exe.clone();
+python_home.pop();
+
+// On Windows, python2.7.exe exists at the root directory of the Python
+// install. Everywhere else, the Python install root is one level up.
+if !python_exe.ends_with("python2.7.exe") {
+python_home.pop();
+}
+
+Environment {
+_exe: exe.clone(),
+python_exe: python_exe.clone(),
+python_home: python_home.clone(),
+mercurial_modules: mercurial_modules.to_path_buf(),
+}
+}
+
+// On UNIX, argv starts as an array of char*. So it is easy to convert
+// to C strings.
+#[cfg(target_family = "unix")]
+fn args_to_cstrings() -> Vec {
+env::args_os().map(|a| CString::new(a.into_vec()).unwrap()).collect()
+}
+
+// TODO Windows support is incomplete. We should either use env::args_os()
+// (or call into GetCommandLineW() + CommandLinetoArgvW()), convert these to
+// PyUnicode instances, and pass these into
