mercurial-devel | Failed pipeline for branch/default | 4eed8466
Pipeline #27753 has failed! Project: mercurial-devel ( https://foss.heptapod.net/mercurial/mercurial-devel ) Branch: branch/default ( https://foss.heptapod.net/mercurial/mercurial-devel/-/commits/branch/default ) Commit: 4eed8466 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/commit/4eed846608ed932549b843ba3c46510023a1f9b6 ) Commit Message: dirstate-v2: Add support when Rust is not enabl... Commit Author: Simon Sapin ( https://foss.heptapod.net/SimonSapin ) Pipeline #27753 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/pipelines/27753 ) triggered by Administrator ( https://foss.heptapod.net/root ) had 8 failed jobs. Job #253369 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253369/raw ) Stage: tests Name: test-py2-chg Job #253362 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253362/raw ) Stage: tests Name: test-py2 Job #253366 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253366/raw ) Stage: tests Name: test-py2-rust Job #253364 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253364/raw ) Stage: tests Name: test-py2-pure Job #253365 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253365/raw ) Stage: tests Name: test-py3-pure Job #253368 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253368/raw ) Stage: tests Name: test-py3-rhg Job #253358 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253358/raw ) Stage: tests Name: checks-py2 Job #253359 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253359/raw ) Stage: tests Name: checks-py3 -- You're receiving this email because of your account on foss.heptapod.net. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial-devel | Failed pipeline for branch/default | aadede8d
Pipeline #27723 has failed! Project: mercurial-devel ( https://foss.heptapod.net/mercurial/mercurial-devel ) Branch: branch/default ( https://foss.heptapod.net/mercurial/mercurial-devel/-/commits/branch/default ) Commit: aadede8d ( https://foss.heptapod.net/mercurial/mercurial-devel/-/commit/aadede8d63eed37c494c3e68516a21e11c0729ad ) Commit Message: rhg: do not fail when the repo is empty Differ... Commit Author: Arseniy Alekseyev Pipeline #27723 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/pipelines/27723 ) triggered by Administrator ( https://foss.heptapod.net/root ) had 7 failed jobs. Job #253219 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253219/raw ) Stage: tests Name: test-py2-chg Job #253216 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253216/raw ) Stage: tests Name: test-py2-rust Job #253214 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253214/raw ) Stage: tests Name: test-py2-pure Job #253212 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253212/raw ) Stage: tests Name: test-py2 Job #253218 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253218/raw ) Stage: tests Name: test-py3-rhg Job #253208 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253208/raw ) Stage: tests Name: checks-py2 Job #253209 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253209/raw ) Stage: tests Name: checks-py3 -- You're receiving this email because of your account on foss.heptapod.net. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D11656: rhg: fix the test
aalekseyev created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D11656 AFFECTED FILES rust/hg-core/src/revlog/revlog.rs CHANGE DETAILS diff --git a/rust/hg-core/src/revlog/revlog.rs b/rust/hg-core/src/revlog/revlog.rs --- a/rust/hg-core/src/revlog/revlog.rs +++ b/rust/hg-core/src/revlog/revlog.rs @@ -424,6 +424,6 @@ .with_version(1) .build(); -assert_eq!(get_version(), 1) +assert_eq!(get_version().map_err(|_err|()), Ok(1)) } } To: aalekseyev, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial-devel | Failed pipeline for branch/default | f6343814
Pipeline #27721 has failed! Project: mercurial-devel ( https://foss.heptapod.net/mercurial/mercurial-devel ) Branch: branch/default ( https://foss.heptapod.net/mercurial/mercurial-devel/-/commits/branch/default ) Commit: f6343814 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/commit/f63438147dd307b672ce5b26ca7fa1a6bc49741e ) Commit Message: rhg: do not fail when the repo is empty Differ... Commit Author: Arseniy Alekseyev Pipeline #27721 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/pipelines/27721 ) triggered by Administrator ( https://foss.heptapod.net/root ) had 9 failed jobs. Job #253200 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253200/raw ) Stage: tests Name: test-py2-chg Job #253193 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253193/raw ) Stage: tests Name: test-py2 Job #253195 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253195/raw ) Stage: tests Name: test-py2-pure Job #253197 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253197/raw ) Stage: tests Name: test-py2-rust Job #253199 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253199/raw ) Stage: tests Name: test-py3-rhg Job #253190 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253190/raw ) Stage: tests Name: checks-py3 Job #253189 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253189/raw ) Stage: tests Name: checks-py2 Job #253192 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253192/raw ) Stage: tests Name: rust-cargo-test-py3 Job #253191 ( https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/253191/raw ) Stage: tests Name: rust-cargo-test-py2 -- You're receiving this email because of your account on foss.heptapod.net. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D11655: dirstate-v2: Add storage space for nanoseconds precision in file mtimes
SimonSapin created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
For now the sub-second component is always set to zero for tracked files and
symlinks. (The mtime of directories for the `readdir`-skipping optimization
is a different code path and already uses the full precision available.)
This extra storage uses the space previously freed by replacing the 32-bit
`mode` field by two bits in the existing `flags` field, so the overall size
of nodes is unchanged. (This space had been left as padding for this purpose.)
Also move things around in the node layout and documentation to have less
duplication. Now that they have the same representation, directory mtime and
file mtime are kept in the same field. (Only either one can exist for a given
node.)
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D11655
AFFECTED FILES
mercurial/helptext/internals/dirstate-v2.txt
rust/hg-core/src/dirstate_tree/on_disk.rs
CHANGE DETAILS
diff --git a/rust/hg-core/src/dirstate_tree/on_disk.rs
b/rust/hg-core/src/dirstate_tree/on_disk.rs
--- a/rust/hg-core/src/dirstate_tree/on_disk.rs
+++ b/rust/hg-core/src/dirstate_tree/on_disk.rs
@@ -97,7 +97,8 @@
pub(super) descendants_with_entry_count: Size,
pub(super) tracked_descendants_count: Size,
flags: Flags,
-data: Entry,
+size: U32Be,
+mtime: PackedTruncatedTimestamp,
}
bitflags! {
@@ -110,23 +111,14 @@
const HAS_MODE_AND_SIZE = 1 << 3;
const HAS_MTIME = 1 << 4;
const MODE_EXEC_PERM = 1 << 5;
-const MODE_IS_SYMLINK = 1 << 7;
+const MODE_IS_SYMLINK = 1 << 6;
}
}
-#[derive(BytesCast, Copy, Clone, Debug)]
-#[repr(C)]
-struct Entry {
-_padding: U32Be,
-size: U32Be,
-mtime: U32Be,
-}
-
/// Duration since the Unix epoch
#[derive(BytesCast, Copy, Clone)]
#[repr(C)]
-struct PackedTimestamp {
-_padding: U32Be,
+struct PackedTruncatedTimestamp {
truncated_seconds: U32Be,
nanoseconds: U32Be,
}
@@ -329,7 +321,7 @@
) -> Result, DirstateV2ParseError> {
Ok(
if self.flags.contains(Flags::HAS_MTIME) && !self.has_entry() {
-Some(self.data.as_timestamp()?)
+Some(self.mtime.try_into()?)
} else {
None
},
@@ -356,12 +348,12 @@
let p1_tracked = self.flags.contains(Flags::P1_TRACKED);
let p2_info = self.flags.contains(Flags::P2_INFO);
let mode_size = if self.flags.contains(Flags::HAS_MODE_AND_SIZE) {
-Some((self.synthesize_unix_mode(), self.data.size.into()))
+Some((self.synthesize_unix_mode(), self.size.into()))
} else {
None
};
let mtime = if self.flags.contains(Flags::HAS_MTIME) {
-Some(self.data.mtime.into())
+Some(self.mtime.truncated_seconds.into())
} else {
None
};
@@ -407,10 +399,10 @@
tracked_descendants_count: self.tracked_descendants_count.get(),
})
}
-}
-impl Entry {
-fn from_dirstate_entry(entry: ) -> (Flags, Self) {
+fn from_dirstate_entry(
+entry: ,
+) -> (Flags, U32Be, PackedTruncatedTimestamp) {
let (wdir_tracked, p1_tracked, p2_info, mode_size_opt, mtime_opt) =
entry.v2_data();
// TODO: convert throug raw flag bits instead?
@@ -418,53 +410,26 @@
flags.set(Flags::WDIR_TRACKED, wdir_tracked);
flags.set(Flags::P1_TRACKED, p1_tracked);
flags.set(Flags::P2_INFO, p2_info);
-let (size, mtime);
-if let Some((m, s)) = mode_size_opt {
+let size = if let Some((m, s)) = mode_size_opt {
let exec_perm = m & libc::S_IXUSR != 0;
let is_symlink = m & libc::S_IFMT == libc::S_IFLNK;
flags.set(Flags::MODE_EXEC_PERM, exec_perm);
flags.set(Flags::MODE_IS_SYMLINK, is_symlink);
-size = s;
-flags.insert(Flags::HAS_MODE_AND_SIZE)
+flags.insert(Flags::HAS_MODE_AND_SIZE);
+s.into()
} else {
-size = 0;
-}
-if let Some(m) = mtime_opt {
-mtime = m;
-flags.insert(Flags::HAS_MTIME);
-} else {
-mtime = 0;
-}
-let raw_entry = Entry {
-_padding: 0.into(),
-size: size.into(),
-mtime: mtime.into(),
+0.into()
};
-(flags, raw_entry)
-}
-
-fn from_timestamp(timestamp: TruncatedTimestamp) -> Self {
-let packed = PackedTimestamp {
-_padding: 0.into(),
-truncated_seconds: timestamp.truncated_seconds().into(),
-nanoseconds: timestamp.nanoseconds().into(),
+let mtime = if let Some(m) = mtime_opt {
+flags.insert(Flags::HAS_MTIME);
+PackedTruncatedTimestamp {
+
D11654: status: Extract TruncatedTimestamp from fs::Metadata without SystemTime
SimonSapin created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
On Unix, the Rust standard library exposes `mtime` and `mtime_nsec` methods
for `std::fs::Metada` whih is exactly what we need to construct a
`TruncatedTimestamp`. This skips the computation in the conversion through
`SystemTime` and `Result`.
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D11654
AFFECTED FILES
rust/hg-core/src/dirstate/entry.rs
rust/hg-core/src/dirstate_tree/status.rs
CHANGE DETAILS
diff --git a/rust/hg-core/src/dirstate_tree/status.rs
b/rust/hg-core/src/dirstate_tree/status.rs
--- a/rust/hg-core/src/dirstate_tree/status.rs
+++ b/rust/hg-core/src/dirstate_tree/status.rs
@@ -199,15 +199,14 @@
// by a previous run of the `status` algorithm which found this
// directory eligible for `read_dir` caching.
if let Some(meta) = directory_metadata {
-if let Ok(current_mtime) = meta.modified() {
-let truncated =
-TruncatedTimestamp::from(current_mtime);
-if truncated.very_likely_equal(_mtime) {
-// The mtime of that directory has not changed
-// since then, which means that the results of
-// `read_dir` should also be unchanged.
-return true;
-}
+if cached_mtime
+.very_likely_equal_to_mtime_of(meta)
+.unwrap_or(false)
+{
+// The mtime of that directory has not changed
+// since then, which means that the results of
+// `read_dir` should also be unchanged.
+return true;
}
}
}
@@ -472,7 +471,7 @@
let is_up_to_date = if let Some(cached) =
dirstate_node.cached_directory_mtime()?
{
-cached.very_likely_equal()
+cached.very_likely_equal(truncated)
} else {
false
};
diff --git a/rust/hg-core/src/dirstate/entry.rs
b/rust/hg-core/src/dirstate/entry.rs
--- a/rust/hg-core/src/dirstate/entry.rs
+++ b/rust/hg-core/src/dirstate/entry.rs
@@ -1,7 +1,9 @@
use crate::dirstate_tree::on_disk::DirstateV2ParseError;
use crate::errors::HgError;
use bitflags::bitflags;
-use std::convert::TryFrom;
+use std::convert::{TryFrom, TryInto};
+use std::fs;
+use std::io;
use std::time::{SystemTime, UNIX_EPOCH};
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
@@ -69,6 +71,21 @@
}
}
+pub fn for_mtime_of(metadata: ::Metadata) -> io::Result {
+#[cfg(unix)]
+{
+use std::os::unix::fs::MetadataExt;
+let seconds = metadata.mtime();
+// i64 -> u32 with value always in the `0 .. NSEC_PER_SEC` range
+let nanoseconds = metadata.mtime_nsec().try_into().unwrap();
+Ok(Self::new_truncate(seconds, nanoseconds))
+}
+#[cfg(not(unix))]
+{
+metadata.modified().map(Self::from)
+}
+}
+
/// The lower 31 bits of the number of seconds since the epoch.
pub fn truncated_seconds() -> u32 {
self.truncated_seconds
@@ -93,10 +110,17 @@
/// If someone is manipulating the modification times of some files to
/// intentionally make `hg status` return incorrect results, not truncating
/// wouldn’t help much since they can set exactly the expected timestamp.
-pub fn very_likely_equal(, other: ) -> bool {
+pub fn very_likely_equal(self, other: Self) -> bool {
self.truncated_seconds == other.truncated_seconds
&& self.nanoseconds == other.nanoseconds
}
+
+pub fn very_likely_equal_to_mtime_of(
+self,
+metadata: ::Metadata,
+) -> io::Result {
+Ok(self.very_likely_equal(Self::for_mtime_of(metadata)?))
+}
}
impl From for TruncatedTimestamp {
To: SimonSapin, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Call for comments on new dirstate format contents
On 28/06/2021 11:49, Raphaël Gomès wrote: Hello all, As you probably know my colleagues at Octobus and I have been working on a new version of the dirstate, and we're coming pretty close to something usable in production, so we need to freeze the format soon. Hello again, Together with the Rust implementation of the new status algorithm, this dirstate-v2 file format enables great performance improvements of `hg status` on large repositories. We Octobus are hoping to stabilize it very soon after a few remaining changes, so that the format will not be experimental anymore in the upcoming Mercurial 6.0 release. It will not yet be enabled by default, but future Mercurial versions will need to be compatible both ways with 6.0 when accessing a given local repository that uses dirstate-v2. A short user guide (how to enable, upgrade, or downgrade) as well as detailed documentation of the file format can be found at: https://www.mercurial-scm.org/repo/hg-committed/file/tip/mercurial/helptext/internals/dirstate-v2.txt … or in a source repository by running `make local` then `./hg help internals.dirstate-v2` The remaining format changes we have planned are: * Add sub-second precision to stored file/symlink mtime, and share its location with that of directory mtime. (This part of the format is a bit of a mess right now since we’re in the middle of this change.) * Maybe add a flag bit to allow marking files as "known modified at this mtime". `hg status` sometimes needs to read the contents of files in case of possible size-preserving changes. If there is indeed a change, currently this read is repeated every time status runs again. The new bit would record that result. * Maybe add some node-specific or dirstate-wide flags or a "mode switch" to make the format and its storage of directory mtimes less tied to details of the current readdir-skipping optimization. (For example, a future version of Mercurial might want to add dirstate nodes for unknown or/and ignored files to skip readdir in more cases.) Non-format changes that we want to have in 6.0: * Merge D11520 and the rest of that stack to have a Python implementation of the format, so that repositories that use it are usable when Rust extensions are not enabled. This is slower, in the order of 0.1 to 0.3 seconds added to `hg status` commands taking 0.4 to 2.5 seconds with dirstate-v1 without Rust on various repositories. * Add configuration to either abort, warn, or silently continue when this slow code path is or would be used. And decide its default. I’m personally inclined at least not to abort by default since the slow path is not *horribly* slow. Please let us know of any question or comment! -- Simon Sapin ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
