Why does "git reset --hard" fail on file/folder conflicts?
Hi, git doesn't want to do a git reset hard when for a certain path the index contains a tree, worktree contains a file and the commit to reset to contains nothing. Is it a bug or is it intended? I would expect git to simply delete that path from index and worktree. > git init Initialized empty Git repository in /tmp/y/.git/ > touch a > git add a > git commit -m addA [master (root-commit) fa08136] addA 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 a > mkdir d > touch d/f > git add d/f > rm -fr d > touch d > git reset --hard HEAD warning: unable to unlink d/f: Not a directory HEAD is now at fa08136 addA Ciao Chris -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Solaris: Fail to build git with CFLAGS=-m64
Hi. Tried to build git-2.6.1 on Solaris (Oracle Solaris 11.2 and OpenIndiana 151.1.9) with CFLAGS="-m64" and got an error during make step: GIT_VERSION = 2.6.1 * new build flags gcc -o credential-store.o -c -MF ./.depend/credential-store.o.d -MQ credential-store.o -MMD -MP --save-temps -O2 -m64 -I. -D__EXTENSIONS__ -D__sun__ -DHAVE_ALLOCA_H -DNO_D_TYPE_IN_DIRENT -DNO_INET_NTOP -DNO_INET_PTON -DHAVE_PATHS_H -DHAVE_STRINGS_H -DHAVE_DEV_TTY -DHAVE_CLOCK_GETTIME -DHAVE_CLOCK_MONOTONIC -DHAVE_GETDELIM -DSHA1_HEADER='' -Icompat/regex -DSHELL_PATH='"/bin/bash"' credential-store.c In file included from cache.h:4:0, from credential-store.c:1: git-compat-util.h:689:13: error: conflicting types for inet_ntop /usr/include/arpa/inet.h:43:20: note: previous declaration of inet_ntop was here gmake: *** [credential-store.o] Error 1 Solaris has the following prototype in the file /usr/include/arpa/inet.h: extern const char *inet_ntop(int, const void *_RESTRICT_KYWD, char *_RESTRICT_KYWD, socklen_t); Git's prototype for inet_ntop is in file git-compat-util.h: #ifdef NO_INET_NTOP const char *inet_ntop(int af, const void *src, char *dst, size_t size); #endif When build with -m64 typedefs for socklen_t typedef unsigned int uint32_t; typedef uint32_t socklen_t; and typedefs for size_t typedef unsigned long ulong_t; typedef ulong_t size_t; With -m32 both socklen_t and size_t are "unsigned int" and there is no any errors. Also Solaris has the functions inet_ntop and inet_pton in libnsl.so so I did the following correction to configure.ac to build git with -m64: diff --git a/configure.ac b/configure.ac index 14012fa..4cf1929 100644 --- a/configure.ac +++ b/configure.ac @@ -637,6 +637,11 @@ AC_CHECK_FUNC([inet_ntop], [NEEDS_RESOLV=YesPlease], [NO_INET_NTOP=YesPlease]) ]) +if test "x$ac_cv_func_inet_ntop" != xyes; then +AC_CHECK_LIB([nsl], [inet_ntop], + [NEEDS_NSL=YesPlease; NO_INET_NTOP=], + []) +fi GIT_CONF_SUBST([NO_INET_NTOP]) # # Define NO_INET_PTON if linking with -lresolv is not enough. @@ -648,6 +653,11 @@ AC_CHECK_FUNC([inet_pton], [NEEDS_RESOLV=YesPlease], [NO_INET_PTON=YesPlease]) ]) +if test "x$ac_cv_func_inet_pton" != xyes; then +AC_CHECK_LIB([nsl], [inet_pton], + [NEEDS_NSL=YesPlease; NO_INET_PTON=], + []) +fi GIT_CONF_SUBST([NO_INET_PTON]) # # Define NO_HSTRERROR if linking with -lresolv is not enough. Is it possible to change prototype for inet_ntop in git-compat-util.h or use Solaris's inet_ntop and inet_pton ? Evgeny -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 18/44] refs: move transaction functions into common code
On 10/12/2015 11:51 PM, David Turner wrote: > The common ref code will build up a ref transaction. Backends will > then commit it. So the transaction creation and update functions should > be in the common code. We also need to move the ref structs into > the common code so that alternate backends can access them. > > Later, we will modify struct ref_update to support alternate backends. This patch leaks internal implementation details into the public refs interface. I want to make sure that code elsewhere in Git treats these structures as opaque. That is why I suggested creating an extra module to hold "protected" code for the reference backend "class hierarchy" [1]. For a sketch of what my suggestion would look like, see the two commits at the tip of branch "refs-be-common" on my GitHub repo [2]. Michael [1] http://article.gmane.org/gmane.comp.version-control.git/279049 [2] https://github.com/mhagger/git -- Michael Haggerty mhag...@alum.mit.edu -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 16/44] refs.c: move ref iterators to the common code
On 10/12/2015 11:51 PM, David Turner wrote:
> From: Ronnie Sahlberg
>
> Signed-off-by: Ronnie Sahlberg
> ---
> refs-be-files.c | 82
> -
> refs.c | 81
> 2 files changed, 81 insertions(+), 82 deletions(-)
>
> diff --git a/refs-be-files.c b/refs-be-files.c
> index ecf2b33..46fa43c 100644
> --- a/refs-be-files.c
> +++ b/refs-be-files.c
> [...]
> @@ -2012,40 +1964,6 @@ int for_each_namespaced_ref(each_ref_fn fn, void
> *cb_data)
> return ret;
> }
>
> -int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
> - const char *prefix, void *cb_data)
> -{
> - struct strbuf real_pattern = STRBUF_INIT;
> - struct ref_filter filter;
> - int ret;
> -
> - if (!prefix && !starts_with(pattern, "refs/"))
> - strbuf_addstr(&real_pattern, "refs/");
> - else if (prefix)
> - strbuf_addstr(&real_pattern, prefix);
> - strbuf_addstr(&real_pattern, pattern);
> -
> - if (!has_glob_specials(pattern)) {
> - /* Append implied '/' '*' if not present. */
> - strbuf_complete(&real_pattern, '/');
> - /* No need to check for '*', there is none. */
> - strbuf_addch(&real_pattern, '*');
> - }
> -
> - filter.pattern = real_pattern.buf;
> - filter.fn = fn;
> - filter.cb_data = cb_data;
> - ret = for_each_ref(filter_refs, &filter);
> -
> - strbuf_release(&real_pattern);
> - return ret;
> -}
> -
> [...]
> diff --git a/refs.c b/refs.c
> index 44ee4f4..7714dad 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -781,3 +781,84 @@ const char *prettify_refname(const char *name)
> starts_with(name, "refs/remotes/") ? 13 :
> 0);
> }
> [...]
> +int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
> + const char *prefix, void *cb_data)
> +{
> + struct strbuf real_pattern = STRBUF_INIT;
> + struct ref_filter filter;
> + int ret;
> +
> + if (!prefix && !starts_with(pattern, "refs/"))
> + strbuf_addstr(&real_pattern, "refs/");
> + else if (prefix)
> + strbuf_addstr(&real_pattern, prefix);
> + strbuf_addstr(&real_pattern, pattern);
> +
> + if (!has_glob_specials(pattern)) {
> + /* Append implied '/' '*' if not present. */
> + if (real_pattern.buf[real_pattern.len - 1] != '/')
> + strbuf_addch(&real_pattern, '/');
The original has
strbuf_complete(&real_pattern, '/');
in place of the above two lines. I think you made a mistake when
rebasing on top of
00b6c178c3 use strbuf_complete to conditionally append slash
> + /* No need to check for '*', there is none. */
> + strbuf_addch(&real_pattern, '*');
> + }
> +
> + filter.pattern = real_pattern.buf;
> + filter.fn = fn;
> + filter.cb_data = cb_data;
> + ret = for_each_ref(filter_refs, &filter);
> +
> + strbuf_release(&real_pattern);
> + return ret;
> +}
> +
> [...]
Michael
--
Michael Haggerty
mhag...@alum.mit.edu
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 06/44] refs.c: move delete_ref and delete_refs to the common code
On 10/12/2015 11:51 PM, David Turner wrote:
> From: Ronnie Sahlberg
>
> Move delete_pseudoref, delete_ref() and delete_refs() to the refs.c
> file since these functions do not contain any backend specific code.
>
> Based on a patch by Ronnie Sahlberg.
>
> Signed-off-by: David Turner
> Signed-off-by: Ronnie Sahlberg
> ---
> refs-be-files.c | 94
> -
> refs.c | 94
> +
> 2 files changed, 94 insertions(+), 94 deletions(-)
>
> diff --git a/refs-be-files.c b/refs-be-files.c
> index 7fe4931..099df75 100644
> --- a/refs-be-files.c
> +++ b/refs-be-files.c
> @@ -2877,100 +2877,6 @@ static int delete_ref_loose(struct ref_lock *lock,
> int flag, struct strbuf *err)
> return 0;
> }
>
> -static int delete_pseudoref(const char *pseudoref, const unsigned char
> *old_sha1)
> -{
> - static struct lock_file lock;
> - const char *filename;
> -
> - filename = git_path("%s", pseudoref);
> -
> - if (old_sha1 && !is_null_sha1(old_sha1)) {
> - int fd;
> - unsigned char actual_old_sha1[20];
> -
> - fd = hold_lock_file_for_update(&lock, filename,
> -LOCK_DIE_ON_ERROR);
> - if (fd < 0)
> - die_errno(_("Could not open '%s' for writing"),
> filename);
> - if (read_ref(pseudoref, actual_old_sha1))
> - die("could not read ref '%s'", pseudoref);
> - if (hashcmp(actual_old_sha1, old_sha1)) {
> - warning("Unexpected sha1 when deleting %s", pseudoref);
> - rollback_lock_file(&lock);
> - return -1;
> - }
> -
> - unlink(filename);
> - rollback_lock_file(&lock);
> - } else {
> - unlink(filename);
> - }
> -
> - return 0;
> -}
> -
> [...]
> diff --git a/refs.c b/refs.c
> index 2d10708..205a899 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -116,3 +116,97 @@ int update_ref(const char *msg, const char *refname,
> ref_transaction_free(t);
> return 0;
> }
> +
> +
> +static int delete_pseudoref(const char *pseudoref, const unsigned char
> *old_sha1)
> +{
> + static struct lock_file lock;
> + const char *filename;
> +
> + filename = git_path("%s", pseudoref);
> +
> + if (old_sha1 && !is_null_sha1(old_sha1)) {
> + int fd;
> + unsigned char actual_old_sha1[20];
> +
> + fd = hold_lock_file_for_update(&lock, filename,
> +LOCK_DIE_ON_ERROR);
> + if (fd < 0)
> + die_errno(_("Could not open '%s' for writing"),
> filename);
> + read_ref(pseudoref, actual_old_sha1);
The same test vanishes here as in the previous patch. The original was
if (read_ref(pseudoref, actual_old_sha1))
die("could not read ref '%s'", pseudoref);
> + if (hashcmp(actual_old_sha1, old_sha1)) {
> + warning("Unexpected sha1 when deleting %s", pseudoref);
> + rollback_lock_file(&lock);
> + return -1;
> + }
> +
> + unlink(filename);
> + rollback_lock_file(&lock);
> + } else {
> + unlink(filename);
> + }
> +
> + return 0;
> +}
> +
> [...]
Michael
--
Michael Haggerty
mhag...@alum.mit.edu
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 05/44] refs.c: move update_ref to refs.c
On 10/12/2015 11:51 PM, David Turner wrote:
> From: Ronnie Sahlberg
>
> Move update_ref() to the refs.c file since this function does not
> contain any backend specific code. Move the ref classifier functions
> as well, since update_ref depends on them.
>
> Based on Ronnie Sahlberg's patch
>
> Signed-off-by: Ronnie Sahlberg
> Signed-off-by: David Turner
> ---
> refs-be-files.c | 117
> +---
> refs.c | 115 +++
> 2 files changed, 116 insertions(+), 116 deletions(-)
>
> diff --git a/refs-be-files.c b/refs-be-files.c
> index d0dfdfc..7fe4931 100644
> --- a/refs-be-files.c
> +++ b/refs-be-files.c
> [...]
> @@ -2879,77 +2877,6 @@ static int delete_ref_loose(struct ref_lock *lock, int
> flag, struct strbuf *err)
> [...]
> -static int write_pseudoref(const char *pseudoref, const unsigned char *sha1,
> -const unsigned char *old_sha1, struct strbuf *err)
> -{
> - const char *filename;
> - int fd;
> - static struct lock_file lock;
> - struct strbuf buf = STRBUF_INIT;
> - int ret = -1;
> -
> - strbuf_addf(&buf, "%s\n", sha1_to_hex(sha1));
> -
> - filename = git_path("%s", pseudoref);
> - fd = hold_lock_file_for_update(&lock, filename, LOCK_DIE_ON_ERROR);
> - if (fd < 0) {
> - strbuf_addf(err, "Could not open '%s' for writing: %s",
> - filename, strerror(errno));
> - return -1;
> - }
> -
> - if (old_sha1) {
> - unsigned char actual_old_sha1[20];
> -
> - if (read_ref(pseudoref, actual_old_sha1))
> - die("could not read ref '%s'", pseudoref);
> - if (hashcmp(actual_old_sha1, old_sha1)) {
> - strbuf_addf(err, "Unexpected sha1 when writing %s",
> pseudoref);
> - rollback_lock_file(&lock);
> - goto done;
> - }
> - }
> -
> - if (write_in_full(fd, buf.buf, buf.len) != buf.len) {
> - strbuf_addf(err, "Could not write to '%s'", filename);
> - rollback_lock_file(&lock);
> - goto done;
> - }
> -
> - commit_lock_file(&lock);
> - ret = 0;
> -done:
> - strbuf_release(&buf);
> - return ret;
> -}
> -
> [...]
> diff --git a/refs.c b/refs.c
> index 77492ff..2d10708 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1,3 +1,118 @@
> [...]
> +static int write_pseudoref(const char *pseudoref, const unsigned char *sha1,
> +const unsigned char *old_sha1, struct strbuf *err)
> +{
> + const char *filename;
> + int fd;
> + static struct lock_file lock;
> + struct strbuf buf = STRBUF_INIT;
> + int ret = -1;
> +
> + strbuf_addf(&buf, "%s\n", sha1_to_hex(sha1));
> +
> + filename = git_path("%s", pseudoref);
> + fd = hold_lock_file_for_update(&lock, filename, LOCK_DIE_ON_ERROR);
> + if (fd < 0) {
> + strbuf_addf(err, "Could not open '%s' for writing: %s",
> + filename, strerror(errno));
> + return -1;
> + }
> +
> + if (old_sha1) {
> + unsigned char actual_old_sha1[20];
> + read_ref(pseudoref, actual_old_sha1);
The original read
if (read_ref(pseudoref, actual_old_sha1))
die("could not read ref '%s'", pseudoref);
This seems like an important test. What happened to it?
If its removal was intentional, it deserves a careful explanation (and
should probably be done as a separate commit). If it was an accident,
please consider how this accident arose and try to think about whether
similar accidents might have happened elsewhere in this series.
> [...]
Michael
--
Michael Haggerty
mhag...@alum.mit.edu
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 02/44] refs: make repack_without_refs and is_branch public
On 10/12/2015 11:51 PM, David Turner wrote: > is_branch was already non-static, but this patch declares it in the > header. > > Signed-off-by: Ronnie Sahlberg > Signed-off-by: David Turner > --- > [...] It seems odd that repack_without_refs() should be made public (and ultimately end up in refs.h) given that it intrinsically only has to do with file-based references. But I will read on... Michael -- Michael Haggerty mhag...@alum.mit.edu -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 01/44] refs.c: create a public version of verify_refname_available
On 10/12/2015 11:51 PM, David Turner wrote:
> From: Ronnie Sahlberg
>
> Create a public version of verify_refname_available that backends can
> provide.
>
> Signed-off-by: Ronnie Sahlberg
> Signed-off-by: David Turner
> ---
> refs.c | 65 ++---
> refs.h | 17 +
> 2 files changed, 47 insertions(+), 35 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 132eff5..fe71ea0 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -279,7 +279,7 @@ struct ref_dir {
> * presence of an empty subdirectory does not block the creation of a
> * similarly-named reference. (The fact that reference names with the
> * same leading components can conflict *with each other* is a
> - * separate issue that is regulated by verify_refname_available().)
> + * separate issue that is regulated by verify_refname_available_dir().)
> *
> * Please note that the name field contains the fully-qualified
> * reference (or subdirectory) name. Space could be saved by only
> @@ -897,25 +897,13 @@ static int nonmatching_ref_fn(struct ref_entry *entry,
> void *vdata)
> /*
> * Return 0 if a reference named refname could be created without
> * conflicting with the name of an existing reference in dir.
> - * Otherwise, return a negative value and write an explanation to err.
> - * If extras is non-NULL, it is a list of additional refnames with
> - * which refname is not allowed to conflict. If skip is non-NULL,
> - * ignore potential conflicts with refs in skip (e.g., because they
> - * are scheduled for deletion in the same operation). Behavior is
> - * undefined if the same name is listed in both extras and skip.
> - *
> - * Two reference names conflict if one of them exactly matches the
> - * leading components of the other; e.g., "refs/foo/bar" conflicts
> - * with both "refs/foo" and with "refs/foo/bar/baz" but not with
> - * "refs/foo/bar" or "refs/foo/barbados".
> - *
> - * extras and skip must be sorted.
> + * See verify_refname_available for details.
> */
> -static int verify_refname_available(const char *refname,
> - const struct string_list *extras,
> - const struct string_list *skip,
> - struct ref_dir *dir,
> - struct strbuf *err)
> +static int verify_refname_available_dir(const char *refname,
> + const struct string_list *extras,
> + const struct string_list *skip,
> + struct ref_dir *dir,
> + struct strbuf *err)
> {
> const char *slash;
> int pos;
> @@ -2464,9 +2452,12 @@ static struct ref_lock *lock_ref_sha1_basic(const char
> *refname,
>*/
> strbuf_git_path(&orig_ref_file, "%s", orig_refname);
> if (remove_empty_directories(&orig_ref_file)) {
> + struct ref_dir *loose_refs;
> + loose_refs = get_loose_refs(&ref_cache);
It's OK to initialize `loose_refs` on the line where it is declared.
> last_errno = errno;
> - if (!verify_refname_available(orig_refname, extras,
> skip,
> -
> get_loose_refs(&ref_cache), err))
> + if (!verify_refname_available_dir(orig_refname, extras,
> + skip, loose_refs,
> + err))
> strbuf_addf(err, "there are still refs under
> '%s'",
> orig_refname);
> goto error_return;
> [...]
> diff --git a/refs.h b/refs.h
> index 6d30c98..79ea220 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -218,6 +218,23 @@ extern void warn_dangling_symrefs(FILE *fp, const char
> *msg_fmt, const struct st
> int pack_refs(unsigned int flags);
>
> /*
> + * Return true iff a reference named refname could be created without
> + * conflicting with the name of an existing reference. If
> + * skip is non-NULL, ignore potential conflicts with refs in skip
> + * (e.g., because they are scheduled for deletion in the same
> + * operation).
> + *
> + * Two reference names conflict if one of them exactly matches the
> + * leading components of the other; e.g., "foo/bar" conflicts with
> + * both "foo" and with "foo/bar/baz" but not with "foo/bar" or
> + * "foo/barbados".
> + *
> + * skip must be sorted.
> + */
> +int verify_refname_available(const char *newname, struct string_list *extra,
> + struct string_list *skip, struct strbuf *err);
This function has an `extra` parameter but the description of this
parameter disappeared as the docstring was moved from refs.c to here.
> +
> +/*
> * Flags controlling ref_transaction_update(), ref_transaction_create(),
[PATCHv2] submodule-config: Shorten logic in parse_config
This makes the parsing more concise by removing the forward goto as well
as unifying the structure of parsing the {ignore, url, path} options.
By unifying we introduce subtle changes in the error cases. We notice
non-boolean variables before noticing duplicate variables now.
CC: Eric Sunshine
CC: Heiko Voigt
Signed-off-by: Stefan Beller
---
submodule-config.c | 69 --
1 file changed, 26 insertions(+), 43 deletions(-)
diff --git a/submodule-config.c b/submodule-config.c
index 393de53..96f1a0b 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -261,74 +261,57 @@ static int parse_config(const char *var, const char
*value, void *data)
name.buf);
if (!strcmp(item.buf, "path")) {
- struct strbuf path = STRBUF_INIT;
- if (!value) {
+ if (!value)
ret = config_error_nonbool(var);
- goto release_return;
- }
- if (!me->overwrite && submodule->path != NULL) {
+ else if (!me->overwrite && submodule->path != NULL)
warn_multiple_config(me->commit_sha1, submodule->name,
"path");
- goto release_return;
+ else {
+ if (submodule->path)
+ cache_remove_path(me->cache, submodule);
+ free((void *) submodule->path);
+ submodule->path = xstrdup(value);
+ cache_put_path(me->cache, submodule);
}
-
- if (submodule->path)
- cache_remove_path(me->cache, submodule);
- free((void *) submodule->path);
- strbuf_addstr(&path, value);
- submodule->path = strbuf_detach(&path, NULL);
- cache_put_path(me->cache, submodule);
} else if (!strcmp(item.buf, "fetchrecursesubmodules")) {
/* when parsing worktree configurations we can die early */
int die_on_error = is_null_sha1(me->gitmodules_sha1);
if (!me->overwrite &&
- submodule->fetch_recurse != RECURSE_SUBMODULES_NONE) {
+ submodule->fetch_recurse != RECURSE_SUBMODULES_NONE)
warn_multiple_config(me->commit_sha1, submodule->name,
"fetchrecursesubmodules");
- goto release_return;
- }
-
- submodule->fetch_recurse = parse_fetch_recurse(var, value,
+ else
+ submodule->fetch_recurse = parse_fetch_recurse(
+ var, value,
die_on_error);
} else if (!strcmp(item.buf, "ignore")) {
- struct strbuf ignore = STRBUF_INIT;
- if (!me->overwrite && submodule->ignore != NULL) {
+ if (!value)
+ ret = config_error_nonbool(var);
+ else if (!me->overwrite && submodule->ignore != NULL)
warn_multiple_config(me->commit_sha1, submodule->name,
"ignore");
- goto release_return;
- }
- if (!value) {
- ret = config_error_nonbool(var);
- goto release_return;
- }
- if (strcmp(value, "untracked") && strcmp(value, "dirty") &&
- strcmp(value, "all") && strcmp(value, "none")) {
+ else if (strcmp(value, "untracked") &&
+strcmp(value, "dirty") &&
+strcmp(value, "all") &&
+strcmp(value, "none"))
warning("Invalid parameter '%s' for config option "
"'submodule.%s.ignore'", value, var);
- goto release_return;
+ else {
+ free((void *) submodule->ignore);
+ submodule->ignore = xstrdup(value);
}
-
- free((void *) submodule->ignore);
- strbuf_addstr(&ignore, value);
- submodule->ignore = strbuf_detach(&ignore, NULL);
} else if (!strcmp(item.buf, "url")) {
- struct strbuf url = STRBUF_INIT;
if (!value) {
ret = config_error_nonbool(var);
- goto release_return;
- }
- if (!me->overwrite && submodule->url != NULL) {
+ } else if (!me->overwrite && submodule->url != NULL) {
warn_multiple_config(me->commit_sha1, submodule->name,
"url");
- goto release_return;
+
Re: [PATCH] Add fetch.recurseSubmoduleParallelism config option
Stefan Beller writes: > There is core.preloadIndex to enable parallel index preload, but > that is boolean and not giving fine control to the user. We want to give > fine control to the user here I'd assume. I'd approach this as "fetching multiple submodules at a time", if I were deciding its name. > ... We could also make it a > submodule specifc thing (submodule.jobs), but that would collide > with submodule.. maybe? I do not think so. You can have area.attr1 area.attr3 area."userthing1".attr1 area."userthing2".attr1 and the parser can differenciate them just fine. So if you want [submodule] fetchParallel = 16 updateParallel = 4 I do not think that would interfere with any [submodule "name"] var = val You can choose to even allow an attribute that is fundamentally per "userthing" (e.g. the branch, the remote, the submodule) defined with area."userthing".attr, but make area.attr to be the fallback value for unspecified area."userthing9".attr (I think http.*.* hierarchy takes that approach), but I do not think the parallelism of fetching is something that should be specified per submodule. >> The parallel_process API could learn a new "verbose" feature that it >> by itself shows some messages like >> >> "processing the 'frotz' job with N tasks" >> "M tasks finished (N still running)" > > I know what to fill in for M and N, 'frotz' is a bit unclear to me. The caller would pass the label to pp_init(); in this codepath perhaps it will say 'submodule fetch' or something. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add fetch.recurseSubmoduleParallelism config option
On Mon, Oct 12, 2015 at 4:14 PM, Junio C Hamano wrote: > Stefan Beller writes: > >> This allows to configure fetching in parallel without having the annoying >> command line option. > > s/annoying//; > > I think this is a sane thing to do, but the name of the variable may > want to be bikeshedded a bit. That's actually what I thought about most for this patch. I expected bikeshedding as well in the workflow (passing around -1 for each unset option). We should not include (threads/processes) as that is implementation detail, and should not be exposed to the user (Looking at pack.threads) There are some options using max_* for configuring parallel stuff such as http.maxRequests. There is core.preloadIndex to enable parallel index preload, but that is boolean and not giving fine control to the user. We want to give fine control to the user here I'd assume. Maybe also the more fundamental question needs to be answered, if we want to stay in the "fetch." prefix. We could also make it a submodule specifc thing (submodule.jobs), but that would collide with submodule.. maybe? (Originally I wanted to postpone this patch until I have parallelized git submodule update, so a "fetch." prefix may not be good, as we want these 2 operations to use the same config option I'd guess) > >> This moved the responsibility to determine how many parallel processes >> to start from builtin/fetch to submodule.c as we need a way to communicate >> "The user did not specify the number of parallel processes in the command >> line options" in the builtin fetch. The submodule code takes care of >> the precedence (CLI > config > default) >> >> Signed-off-by: Stefan Beller >> --- >> Documentation/config.txt | 6 ++ >> builtin/fetch.c | 2 +- >> submodule.c | 14 ++ >> 3 files changed, 21 insertions(+), 1 deletion(-) >> >> I just monkey tested the code and it worked once! The problem with testing >> this parallelizing option is that the expected behavior doesn't change >> except for being faster. And I don't want to add timing tests to the test >> suite because they are unreliable. >> >> Any idea how to test this properly? > > I agree that a test in t/ would catch bugs in the functionality. If > your parallel implementation is somehow broken in the future and > stops functioning correctly, fetching all submodules with a single > task and fetching them with N tasks will produce different results > ;-). > > But it would not help you much in seeing if the parallelism is > really taking place. Adding t/perf/ tests to show how much benefit > you are getting may be of more value. > > The parallel_process API could learn a new "verbose" feature that it > by itself shows some messages like > > "processing the 'frotz' job with N tasks" > "M tasks finished (N still running)" I know what to fill in for M and N, 'frotz' is a bit unclear to me. Would you imagine that to be passed in as a hardcoded string? git fetch --recurse-submodules would pass in "Fetching submodules", but wuld pass in actual "frotz", or would you assume that to be computed from the task data somehow? > > in the output stream from strategic places. For example, the first > message will come at the end of pp_init(), and the second message > will be appended at the end of buffered output of a task that has > just been finished. Once you have something like that, you could > check for them in a test in t/. > > Just a thought. I like that thought. :) > >> >> This applies on top of sb/submodule-parallel-fetch >> >> Thanks, >> Stefan >> >> >> diff --git a/Documentation/config.txt b/Documentation/config.txt >> index 315f271..1172db0 100644 >> --- a/Documentation/config.txt >> +++ b/Documentation/config.txt >> @@ -1140,6 +1140,12 @@ fetch.recurseSubmodules:: >> when its superproject retrieves a commit that updates the submodule's >> reference. >> >> +fetch.recurseSubmoduleParallelism >> + This is used to determine how many submodules can be fetched in >> + parallel. Specifying a positive integer allows up to that number >> + of submodules being fetched in parallel. Specifying 0 the number >> + of cpus will be taken as the maximum number. >> + >> fetch.fsckObjects:: >> If it is set to true, git-fetch-pack will check all fetched >> objects. It will abort in the case of a malformed object or a >> diff --git a/builtin/fetch.c b/builtin/fetch.c >> index f28eac6..b1399dc 100644 >> --- a/builtin/fetch.c >> +++ b/builtin/fetch.c >> @@ -37,7 +37,7 @@ static int prune = -1; /* unspecified */ >> static int all, append, dry_run, force, keep, multiple, update_head_ok, >> verbosity; >> static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT; >> static int tags = TAGS_DEFAULT, unshallow, update_shallow; >> -static int max_children = 1; >> +static int max_children = -1; >> static const char *depth; >> static const char *upload_pack; >> static struct strbuf default_r
Re: [PATCH] Add fetch.recurseSubmoduleParallelism config option
Stefan Beller writes:
> This allows to configure fetching in parallel without having the annoying
> command line option.
s/annoying//;
I think this is a sane thing to do, but the name of the variable may
want to be bikeshedded a bit.
> This moved the responsibility to determine how many parallel processes
> to start from builtin/fetch to submodule.c as we need a way to communicate
> "The user did not specify the number of parallel processes in the command
> line options" in the builtin fetch. The submodule code takes care of
> the precedence (CLI > config > default)
>
> Signed-off-by: Stefan Beller
> ---
> Documentation/config.txt | 6 ++
> builtin/fetch.c | 2 +-
> submodule.c | 14 ++
> 3 files changed, 21 insertions(+), 1 deletion(-)
>
> I just monkey tested the code and it worked once! The problem with testing
> this parallelizing option is that the expected behavior doesn't change
> except for being faster. And I don't want to add timing tests to the test
> suite because they are unreliable.
>
> Any idea how to test this properly?
I agree that a test in t/ would catch bugs in the functionality. If
your parallel implementation is somehow broken in the future and
stops functioning correctly, fetching all submodules with a single
task and fetching them with N tasks will produce different results
;-).
But it would not help you much in seeing if the parallelism is
really taking place. Adding t/perf/ tests to show how much benefit
you are getting may be of more value.
The parallel_process API could learn a new "verbose" feature that it
by itself shows some messages like
"processing the 'frotz' job with N tasks"
"M tasks finished (N still running)"
in the output stream from strategic places. For example, the first
message will come at the end of pp_init(), and the second message
will be appended at the end of buffered output of a task that has
just been finished. Once you have something like that, you could
check for them in a test in t/.
Just a thought.
>
> This applies on top of sb/submodule-parallel-fetch
>
> Thanks,
> Stefan
>
>
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 315f271..1172db0 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1140,6 +1140,12 @@ fetch.recurseSubmodules::
> when its superproject retrieves a commit that updates the submodule's
> reference.
>
> +fetch.recurseSubmoduleParallelism
> + This is used to determine how many submodules can be fetched in
> + parallel. Specifying a positive integer allows up to that number
> + of submodules being fetched in parallel. Specifying 0 the number
> + of cpus will be taken as the maximum number.
> +
> fetch.fsckObjects::
> If it is set to true, git-fetch-pack will check all fetched
> objects. It will abort in the case of a malformed object or a
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index f28eac6..b1399dc 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -37,7 +37,7 @@ static int prune = -1; /* unspecified */
> static int all, append, dry_run, force, keep, multiple, update_head_ok,
> verbosity;
> static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
> static int tags = TAGS_DEFAULT, unshallow, update_shallow;
> -static int max_children = 1;
> +static int max_children = -1;
> static const char *depth;
> static const char *upload_pack;
> static struct strbuf default_rla = STRBUF_INIT;
> diff --git a/submodule.c b/submodule.c
> index c21b265..c85d3ef 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -15,6 +15,7 @@
> #include "thread-utils.h"
>
> static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
> +static int config_fetch_parallel_submodules = -1;
> static struct string_list changed_submodule_paths;
> static int initialized_fetch_ref_tips;
> static struct sha1_array ref_tips_before_fetch;
> @@ -179,6 +180,14 @@ int submodule_config(const char *var, const char *value,
> void *cb)
> else if (!strcmp(var, "fetch.recursesubmodules")) {
> config_fetch_recurse_submodules =
> parse_fetch_recurse_submodules_arg(var, value);
> return 0;
> + } else if (!strcmp(var, "fetch.recursesubmoduleparallelism")) {
> + char *end;
> + int ret;
> + config_fetch_parallel_submodules = strtol(value, &end, 0);
> + ret = (*end == '\0');
> + if (!ret)
> + warning("value for fetch.recurseSubmoduleParallelism
> not recognized");
> + return ret;
> }
> return 0;
> }
> @@ -759,6 +768,11 @@ int fetch_populated_submodules(const struct argv_array
> *options,
> argv_array_push(&spf.args, "--recurse-submodules-default");
> /* default value, "--submodule-prefix" and its value are added later */
>
> + if (max_parallel_jobs < 0)
> + max_parallel_jobs =
[PATCH] Add fetch.recurseSubmoduleParallelism config option
This allows to configure fetching in parallel without having the annoying
command line option.
This moved the responsibility to determine how many parallel processes
to start from builtin/fetch to submodule.c as we need a way to communicate
"The user did not specify the number of parallel processes in the command
line options" in the builtin fetch. The submodule code takes care of
the precedence (CLI > config > default)
Signed-off-by: Stefan Beller
---
Documentation/config.txt | 6 ++
builtin/fetch.c | 2 +-
submodule.c | 14 ++
3 files changed, 21 insertions(+), 1 deletion(-)
I just monkey tested the code and it worked once! The problem with testing
this parallelizing option is that the expected behavior doesn't change
except for being faster. And I don't want to add timing tests to the test
suite because they are unreliable.
Any idea how to test this properly?
This applies on top of sb/submodule-parallel-fetch
Thanks,
Stefan
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 315f271..1172db0 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1140,6 +1140,12 @@ fetch.recurseSubmodules::
when its superproject retrieves a commit that updates the submodule's
reference.
+fetch.recurseSubmoduleParallelism
+ This is used to determine how many submodules can be fetched in
+ parallel. Specifying a positive integer allows up to that number
+ of submodules being fetched in parallel. Specifying 0 the number
+ of cpus will be taken as the maximum number.
+
fetch.fsckObjects::
If it is set to true, git-fetch-pack will check all fetched
objects. It will abort in the case of a malformed object or a
diff --git a/builtin/fetch.c b/builtin/fetch.c
index f28eac6..b1399dc 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -37,7 +37,7 @@ static int prune = -1; /* unspecified */
static int all, append, dry_run, force, keep, multiple, update_head_ok,
verbosity;
static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
static int tags = TAGS_DEFAULT, unshallow, update_shallow;
-static int max_children = 1;
+static int max_children = -1;
static const char *depth;
static const char *upload_pack;
static struct strbuf default_rla = STRBUF_INIT;
diff --git a/submodule.c b/submodule.c
index c21b265..c85d3ef 100644
--- a/submodule.c
+++ b/submodule.c
@@ -15,6 +15,7 @@
#include "thread-utils.h"
static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
+static int config_fetch_parallel_submodules = -1;
static struct string_list changed_submodule_paths;
static int initialized_fetch_ref_tips;
static struct sha1_array ref_tips_before_fetch;
@@ -179,6 +180,14 @@ int submodule_config(const char *var, const char *value,
void *cb)
else if (!strcmp(var, "fetch.recursesubmodules")) {
config_fetch_recurse_submodules =
parse_fetch_recurse_submodules_arg(var, value);
return 0;
+ } else if (!strcmp(var, "fetch.recursesubmoduleparallelism")) {
+ char *end;
+ int ret;
+ config_fetch_parallel_submodules = strtol(value, &end, 0);
+ ret = (*end == '\0');
+ if (!ret)
+ warning("value for fetch.recurseSubmoduleParallelism
not recognized");
+ return ret;
}
return 0;
}
@@ -759,6 +768,11 @@ int fetch_populated_submodules(const struct argv_array
*options,
argv_array_push(&spf.args, "--recurse-submodules-default");
/* default value, "--submodule-prefix" and its value are added later */
+ if (max_parallel_jobs < 0)
+ max_parallel_jobs = config_fetch_parallel_submodules;
+ if (max_parallel_jobs < 0)
+ max_parallel_jobs = 1;
+
calculate_changed_submodule_paths();
run_processes_parallel(max_parallel_jobs,
get_next_submodule,
--
2.5.0.267.g8d6e698.dirty
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] merge: fix cache_entry use-after-free
David Turner writes:
> From: Keith McGuigan
>
> During merges, we would previously free entries that we no longer need
> in the destination index. But those entries might also be stored in
> the dir_entry cache, and when a later call to add_to_index found them,
> they would be used after being freed.
>
> To prevent this, add a ref count for struct cache_entry. Whenever
> a cache entry is added to a data structure, the ref count is incremented;
> when it is removed from the data structure, it is decremented. When
> it hits zero, the cache_entry is freed.
>
> Signed-off-by: Keith McGuigan
> ---
I'll forge your "messenger's sign-off" here ;-)
> diff --git a/unpack-trees.c b/unpack-trees.c
> index f932e80..1a0a637 100644
> --- a/unpack-trees.c
> +++ b/unpack-trees.c
> @@ -606,8 +606,10 @@ static int unpack_nondirectories(int n, unsigned long
> mask,
> o);
> for (i = 0; i < n; i++) {
> struct cache_entry *ce = src[i + o->merge];
> - if (ce != o->df_conflict_entry)
> - free(ce);
> + if (ce != o->df_conflict_entry) {
> + drop_ce_ref(ce);
> + src[i + o->merge] = NULL;
> + }
This one smelled iffy. I think it is safe because the caller does
not look at src[] other than src[0] after this function returns, and
this setting to NULL happens only when o->merge is set to 1, so I do
not think this is buggy, but at the same time I do not think setting
to NULL is necessary.
Other than that, looks nice. Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 19/44] refs-be-files.c: add a backend method structure with transaction functions
Please disregard this one; I rewrorded the commit message and forgot to
delete this patch.
On Mon, 2015-10-12 at 17:51 -0400, David Turner wrote:
> From: Ronnie Sahlberg
>
> Add a ref structure for backend methods. Start by adding a method pointer
> for the transaction commit function.
>
> Add a function set_refs_backend to switch between backends. The files
> based backend is the default.
>
> Signed-off-by: Ronnie Sahlberg
> Signed-off-by: David Turner
> ---
> refs-be-files.c | 10 --
> refs.c | 30 ++
> refs.h | 15 +++
> 3 files changed, 53 insertions(+), 2 deletions(-)
>
> diff --git a/refs-be-files.c b/refs-be-files.c
> index 1308955..3050f1d 100644
> --- a/refs-be-files.c
> +++ b/refs-be-files.c
> @@ -3225,8 +3225,8 @@ static int ref_update_reject_duplicates(struct
> string_list *refnames,
> return 0;
> }
>
> -int ref_transaction_commit(struct ref_transaction *transaction,
> -struct strbuf *err)
> +static int files_transaction_commit(struct ref_transaction *transaction,
> + struct strbuf *err)
> {
> int ret = 0, i;
> int n = transaction->nr;
> @@ -3612,3 +3612,9 @@ int reflog_expire(const char *refname, const unsigned
> char *sha1,
> unlock_ref(lock);
> return -1;
> }
> +
> +struct ref_be refs_be_files = {
> + NULL,
> + "files",
> + files_transaction_commit,
> +};
> diff --git a/refs.c b/refs.c
> index 25ad3b5..f930fe0 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -4,6 +4,29 @@
> #include "cache.h"
> #include "refs.h"
> #include "lockfile.h"
> +/*
> + * We always have a files backend and it is the default.
> + */
> +struct ref_be *the_refs_backend = &refs_be_files;
> +/*
> + * List of all available backends
> + */
> +struct ref_be *refs_backends = &refs_be_files;
> +
> +/*
> + * This function is used to switch to an alternate backend.
> + */
> +int set_refs_backend(const char *name)
> +{
> + struct ref_be *be;
> +
> + for (be = refs_backends; be; be = be->next)
> + if (!strcmp(be->name, name)) {
> + the_refs_backend = be;
> + return 0;
> + }
> + return 1;
> +}
>
> static int is_per_worktree_ref(const char *refname)
> {
> @@ -985,3 +1008,10 @@ int ref_transaction_verify(struct ref_transaction
> *transaction,
> NULL, old_sha1,
> flags, NULL, err);
> }
> +
> +/* backend functions */
> +int ref_transaction_commit(struct ref_transaction *transaction,
> +struct strbuf *err)
> +{
> + return the_refs_backend->transaction_commit(transaction, err);
> +}
> diff --git a/refs.h b/refs.h
> index 4940ae9..419abf4 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -619,4 +619,19 @@ extern int reflog_expire(const char *refname, const
> unsigned char *sha1,
>reflog_expiry_cleanup_fn cleanup_fn,
>void *policy_cb_data);
>
> +/* refs backends */
> +typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
> + struct strbuf *err);
> +typedef void ref_transaction_free_fn(struct ref_transaction *transaction);
> +
> +struct ref_be {
> + struct ref_be *next;
> + const char *name;
> + ref_transaction_commit_fn *transaction_commit;
> +};
> +
> +
> +extern struct ref_be refs_be_files;
> +int set_refs_backend(const char *name);
> +
> #endif /* REFS_H */
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 05/44] refs.c: move update_ref to refs.c
From: Ronnie Sahlberg
Move update_ref() to the refs.c file since this function does not
contain any backend specific code. Move the ref classifier functions
as well, since update_ref depends on them.
Based on Ronnie Sahlberg's patch
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 117 +---
refs.c | 115 +++
2 files changed, 116 insertions(+), 116 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index d0dfdfc..7fe4931 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2675,8 +2675,6 @@ struct pack_refs_cb_data {
struct ref_to_prune *ref_to_prune;
};
-static int is_per_worktree_ref(const char *refname);
-
/*
* An each_ref_entry_fn that is run over loose references only. If
* the loose reference can be packed, add an entry in the packed ref
@@ -2691,7 +2689,7 @@ static int pack_if_possible_fn(struct ref_entry *entry,
void *cb_data)
int is_tag_ref = starts_with(entry->name, "refs/tags/");
/* Do not pack per-worktree refs: */
- if (is_per_worktree_ref(entry->name))
+ if (ref_type(entry->name) == REF_TYPE_PER_WORKTREE)
return 0;
/* ALWAYS pack tags */
@@ -2879,77 +2877,6 @@ static int delete_ref_loose(struct ref_lock *lock, int
flag, struct strbuf *err)
return 0;
}
-static int is_per_worktree_ref(const char *refname)
-{
- return !strcmp(refname, "HEAD") ||
- starts_with(refname, "refs/bisect/");
-}
-
-static int is_pseudoref_syntax(const char *refname)
-{
- const char *c;
-
- for (c = refname; *c; c++) {
- if (!isupper(*c) && *c != '-' && *c != '_')
- return 0;
- }
-
- return 1;
-}
-
-enum ref_type ref_type(const char *refname)
-{
- if (is_per_worktree_ref(refname))
- return REF_TYPE_PER_WORKTREE;
- if (is_pseudoref_syntax(refname))
- return REF_TYPE_PSEUDOREF;
- return REF_TYPE_NORMAL;
-}
-
-static int write_pseudoref(const char *pseudoref, const unsigned char *sha1,
- const unsigned char *old_sha1, struct strbuf *err)
-{
- const char *filename;
- int fd;
- static struct lock_file lock;
- struct strbuf buf = STRBUF_INIT;
- int ret = -1;
-
- strbuf_addf(&buf, "%s\n", sha1_to_hex(sha1));
-
- filename = git_path("%s", pseudoref);
- fd = hold_lock_file_for_update(&lock, filename, LOCK_DIE_ON_ERROR);
- if (fd < 0) {
- strbuf_addf(err, "Could not open '%s' for writing: %s",
- filename, strerror(errno));
- return -1;
- }
-
- if (old_sha1) {
- unsigned char actual_old_sha1[20];
-
- if (read_ref(pseudoref, actual_old_sha1))
- die("could not read ref '%s'", pseudoref);
- if (hashcmp(actual_old_sha1, old_sha1)) {
- strbuf_addf(err, "Unexpected sha1 when writing %s",
pseudoref);
- rollback_lock_file(&lock);
- goto done;
- }
- }
-
- if (write_in_full(fd, buf.buf, buf.len) != buf.len) {
- strbuf_addf(err, "Could not write to '%s'", filename);
- rollback_lock_file(&lock);
- goto done;
- }
-
- commit_lock_file(&lock);
- ret = 0;
-done:
- strbuf_release(&buf);
- return ret;
-}
-
static int delete_pseudoref(const char *pseudoref, const unsigned char
*old_sha1)
{
static struct lock_file lock;
@@ -4098,48 +4025,6 @@ int ref_transaction_verify(struct ref_transaction
*transaction,
flags, NULL, err);
}
-int update_ref(const char *msg, const char *refname,
- const unsigned char *new_sha1, const unsigned char *old_sha1,
- unsigned int flags, enum action_on_err onerr)
-{
- struct ref_transaction *t = NULL;
- struct strbuf err = STRBUF_INIT;
- int ret = 0;
-
- if (ref_type(refname) == REF_TYPE_PSEUDOREF) {
- ret = write_pseudoref(refname, new_sha1, old_sha1, &err);
- } else {
- t = ref_transaction_begin(&err);
- if (!t ||
- ref_transaction_update(t, refname, new_sha1, old_sha1,
- flags, msg, &err) ||
- ref_transaction_commit(t, &err)) {
- ret = 1;
- ref_transaction_free(t);
- }
- }
- if (ret) {
- const char *str = "update_ref failed for ref '%s': %s";
-
- switch (onerr) {
- case UPDATE_REFS_MSG_ON_ERR:
- error(str, refname, err.buf);
- break;
- case UPDATE_REFS_DIE_ON_ERR:
- die(str,
[PATCH v3 01/44] refs.c: create a public version of verify_refname_available
From: Ronnie Sahlberg
Create a public version of verify_refname_available that backends can
provide.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs.c | 65 ++---
refs.h | 17 +
2 files changed, 47 insertions(+), 35 deletions(-)
diff --git a/refs.c b/refs.c
index 132eff5..fe71ea0 100644
--- a/refs.c
+++ b/refs.c
@@ -279,7 +279,7 @@ struct ref_dir {
* presence of an empty subdirectory does not block the creation of a
* similarly-named reference. (The fact that reference names with the
* same leading components can conflict *with each other* is a
- * separate issue that is regulated by verify_refname_available().)
+ * separate issue that is regulated by verify_refname_available_dir().)
*
* Please note that the name field contains the fully-qualified
* reference (or subdirectory) name. Space could be saved by only
@@ -897,25 +897,13 @@ static int nonmatching_ref_fn(struct ref_entry *entry,
void *vdata)
/*
* Return 0 if a reference named refname could be created without
* conflicting with the name of an existing reference in dir.
- * Otherwise, return a negative value and write an explanation to err.
- * If extras is non-NULL, it is a list of additional refnames with
- * which refname is not allowed to conflict. If skip is non-NULL,
- * ignore potential conflicts with refs in skip (e.g., because they
- * are scheduled for deletion in the same operation). Behavior is
- * undefined if the same name is listed in both extras and skip.
- *
- * Two reference names conflict if one of them exactly matches the
- * leading components of the other; e.g., "refs/foo/bar" conflicts
- * with both "refs/foo" and with "refs/foo/bar/baz" but not with
- * "refs/foo/bar" or "refs/foo/barbados".
- *
- * extras and skip must be sorted.
+ * See verify_refname_available for details.
*/
-static int verify_refname_available(const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- struct ref_dir *dir,
- struct strbuf *err)
+static int verify_refname_available_dir(const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ struct ref_dir *dir,
+ struct strbuf *err)
{
const char *slash;
int pos;
@@ -2464,9 +2452,12 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
*/
strbuf_git_path(&orig_ref_file, "%s", orig_refname);
if (remove_empty_directories(&orig_ref_file)) {
+ struct ref_dir *loose_refs;
+ loose_refs = get_loose_refs(&ref_cache);
last_errno = errno;
- if (!verify_refname_available(orig_refname, extras,
skip,
-
get_loose_refs(&ref_cache), err))
+ if (!verify_refname_available_dir(orig_refname, extras,
+ skip, loose_refs,
+ err))
strbuf_addf(err, "there are still refs under
'%s'",
orig_refname);
goto error_return;
@@ -2479,8 +2470,9 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
if (!refname) {
last_errno = errno;
if (last_errno != ENOTDIR ||
- !verify_refname_available(orig_refname, extras, skip,
- get_loose_refs(&ref_cache), err))
+ !verify_refname_available_dir(orig_refname, extras, skip,
+ get_loose_refs(&ref_cache),
+ err))
strbuf_addf(err, "unable to resolve reference %s: %s",
orig_refname, strerror(last_errno));
@@ -2493,8 +2485,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
* our refname.
*/
if (is_null_oid(&lock->old_oid) &&
- verify_refname_available(refname, extras, skip,
-get_packed_refs(&ref_cache), err)) {
+ verify_refname_available_dir(refname, extras, skip,
+get_packed_refs(&ref_cache), err)) {
last_errno = ENOTDIR;
goto error_return;
}
@@ -3127,10 +3119,7 @@ static int rename_ref_available(const char *oldname,
const char *newname)
int ret;
string_list_insert(&skip, oldname);
- ret = !ver
[PATCH v2] merge: fix cache_entry use-after-free
From: Keith McGuigan
During merges, we would previously free entries that we no longer need
in the destination index. But those entries might also be stored in
the dir_entry cache, and when a later call to add_to_index found them,
they would be used after being freed.
To prevent this, add a ref count for struct cache_entry. Whenever
a cache entry is added to a data structure, the ref count is incremented;
when it is removed from the data structure, it is decremented. When
it hits zero, the cache_entry is freed.
Signed-off-by: Keith McGuigan
---
This version addresses Junio's comments on v1. It adds a missing
add_ce_ref, and fixes a formatting nit.
cache.h| 27 +++
name-hash.c| 7 ++-
read-cache.c | 6 +-
split-index.c | 13 -
unpack-trees.c | 6 --
5 files changed, 50 insertions(+), 9 deletions(-)
diff --git a/cache.h b/cache.h
index 752031e..738f76d 100644
--- a/cache.h
+++ b/cache.h
@@ -149,6 +149,7 @@ struct stat_data {
struct cache_entry {
struct hashmap_entry ent;
+ unsigned int ref_count; /* count the number of refs to this in dir_hash
*/
struct stat_data ce_stat_data;
unsigned int ce_mode;
unsigned int ce_flags;
@@ -213,6 +214,32 @@ struct cache_entry {
struct pathspec;
/*
+ * Increment the cache_entry reference count. Should be called
+ * whenever a pointer to a cache_entry is retained in a data structure,
+ * thus marking it as alive.
+ */
+static inline void add_ce_ref(struct cache_entry *ce)
+{
+ assert(ce != NULL && ce->ref_count >= 0);
+ ce->ref_count++;
+}
+
+/*
+ * Decrement the cache_entry reference count. Should be called whenever
+ * a pointer to a cache_entry is dropped. Once the counter drops to 0
+ * the cache_entry memory will be safely freed.
+ */
+static inline void drop_ce_ref(struct cache_entry *ce)
+{
+ if (ce != NULL) {
+ assert(ce->ref_count >= 0);
+ if (--ce->ref_count < 1) {
+ free(ce);
+ }
+ }
+}
+
+/*
* Copy the sha1 and stat state of a cache entry from one to
* another. But we never change the name, or the hash state!
*/
diff --git a/name-hash.c b/name-hash.c
index 702cd05..f12c919 100644
--- a/name-hash.c
+++ b/name-hash.c
@@ -66,6 +66,7 @@ static struct dir_entry *hash_dir_entry(struct index_state
*istate,
dir = xcalloc(1, sizeof(struct dir_entry));
hashmap_entry_init(dir, memihash(ce->name, namelen));
dir->namelen = namelen;
+ add_ce_ref(ce);
dir->ce = ce;
hashmap_add(&istate->dir_hash, dir);
@@ -92,7 +93,9 @@ static void remove_dir_entry(struct index_state *istate,
struct cache_entry *ce)
struct dir_entry *dir = hash_dir_entry(istate, ce, ce_namelen(ce));
while (dir && !(--dir->nr)) {
struct dir_entry *parent = dir->parent;
- hashmap_remove(&istate->dir_hash, dir, NULL);
+ struct dir_entry *removed = hashmap_remove(&istate->dir_hash,
dir, NULL);
+ assert(removed == dir);
+ drop_ce_ref(dir->ce);
free(dir);
dir = parent;
}
@@ -105,6 +108,7 @@ static void hash_index_entry(struct index_state *istate,
struct cache_entry *ce)
ce->ce_flags |= CE_HASHED;
hashmap_entry_init(ce, memihash(ce->name, ce_namelen(ce)));
hashmap_add(&istate->name_hash, ce);
+ add_ce_ref(ce);
if (ignore_case)
add_dir_entry(istate, ce);
@@ -147,6 +151,7 @@ void remove_name_hash(struct index_state *istate, struct
cache_entry *ce)
return;
ce->ce_flags &= ~CE_HASHED;
hashmap_remove(&istate->name_hash, ce, ce);
+ drop_ce_ref(ce);
if (ignore_case)
remove_dir_entry(istate, ce);
diff --git a/read-cache.c b/read-cache.c
index 87204a5..8b685bb 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -52,7 +52,9 @@ static const char *alternate_index_output;
static void set_index_entry(struct index_state *istate, int nr, struct
cache_entry *ce)
{
+ /* istate->cache[nr] is assumed to not hold a live value */
istate->cache[nr] = ce;
+ add_ce_ref(ce);
add_name_hash(istate, ce);
}
@@ -62,7 +64,7 @@ static void replace_index_entry(struct index_state *istate,
int nr, struct cache
replace_index_entry_in_base(istate, old, ce);
remove_name_hash(istate, old);
- free(old);
+ drop_ce_ref(old);
set_index_entry(istate, nr, ce);
ce->ce_flags |= CE_UPDATE_IN_BASE;
istate->cache_changed |= CE_ENTRY_CHANGED;
@@ -75,6 +77,7 @@ void rename_index_entry_at(struct index_state *istate, int
nr, const char *new_n
new = xmalloc(cache_entry_size(namelen));
copy_cache_entry(new, old);
+ new->ref_count = 0;
new->ce_flags &= ~CE_HASHED;
new->ce_namelen = namelen;
Fwd: [PATCH] submodule: Test a shallow branch
I forgot to CC you guys. -- Forwarded message -- From: Stefan Beller Date: Mon, Oct 12, 2015 at 2:36 PM Subject: [PATCH] submodule: Test a shallow branch To: gits...@pobox.com Cc: git@vger.kernel.org, carlosjosep...@gmail.com, Stefan Beller Instead of tracking the submodule at the specified branch, using a shallow clone, we get the following error: fatal: Cannot update paths and switch to branch 'anotherbranch' at the same time. Did you intend to checkout 'origin/anotherbranch' which can not be resolved as commit? Unable to checkout submodule 'submodule' Noticed-by: Carlos Pita Signed-off-by: Stefan Beller --- Hi Carlos, I can reproduce your issue and have a patch which we can add to the testing suite to document this faulty behavior. Thanks, Stefan t/t7400-submodule-basic.sh | 20 +++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index 540771c..cee74cb 100755 --- a/t/t7400-submodule-basic.sh +++ b/t/t7400-submodule-basic.sh @@ -846,7 +846,8 @@ test_expect_success 'submodule add with an existing name fails unless forced' ' test_expect_success 'set up a second submodule' ' git submodule add ./init2 example2 && - git commit -m "submodule example2 added" + git commit -m "submodule example2 added" && + git checkout -b anotherbranch ' test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' ' @@ -987,6 +988,7 @@ test_expect_success 'submodule with UTF-8 name' ' test_expect_success 'submodule add clone shallow submodule' ' mkdir super && + test_when_finished "rm -rf super" && pwd=$(pwd) && ( cd super && @@ -999,5 +1001,21 @@ test_expect_success 'submodule add clone shallow submodule' ' ) ' +test_expect_failure 'submodule add a shallow branch' ' + mkdir super && + test_when_finished "rm -rf super" && + pwd=$(pwd) && + ( + cd super && + git init && + git submodule add --depth=1 --branch anotherbranch file://"$pwd"/example2 submodule && + ( + cd submodule && + test 1 = $(git log --oneline | wc -l) + ) + ) +' + + test_done -- 2.5.0.268.g453a26a -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 11/44] refs.c: move read_ref, read_ref_full and ref_exists to the common code
From: Ronnie Sahlberg
These functions do not depend on the backend implementation so we
move them to the common code.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 18 --
refs.c | 18 ++
2 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 7d82864..bd0dbb5 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1791,24 +1791,6 @@ struct ref_filter {
void *cb_data;
};
-int read_ref_full(const char *refname, int resolve_flags, unsigned char *sha1,
int *flags)
-{
- if (resolve_ref_unsafe(refname, resolve_flags, sha1, flags))
- return 0;
- return -1;
-}
-
-int read_ref(const char *refname, unsigned char *sha1)
-{
- return read_ref_full(refname, RESOLVE_REF_READING, sha1, NULL);
-}
-
-int ref_exists(const char *refname)
-{
- unsigned char sha1[20];
- return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
-}
-
static int filter_refs(const char *refname, const struct object_id *oid,
int flags, void *data)
{
diff --git a/refs.c b/refs.c
index 24d5e28..242f66d 100644
--- a/refs.c
+++ b/refs.c
@@ -633,3 +633,21 @@ void warn_dangling_symrefs(FILE *fp, const char *msg_fmt,
const struct string_li
data.msg_fmt = msg_fmt;
for_each_rawref(warn_if_dangling_symref, &data);
}
+
+int read_ref_full(const char *refname, int resolve_flags, unsigned char *sha1,
int *flags)
+{
+ if (resolve_ref_unsafe(refname, resolve_flags, sha1, flags))
+ return 0;
+ return -1;
+}
+
+int read_ref(const char *refname, unsigned char *sha1)
+{
+ return read_ref_full(refname, RESOLVE_REF_READING, sha1, NULL);
+}
+
+int ref_exists(const char *refname)
+{
+ unsigned char sha1[20];
+ return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
+}
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 14/44] refs.c: move is_branch to the common code
From: Ronnie Sahlberg
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 5 -
refs.c | 5 +
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 9c57dcc..1214d9e 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2957,11 +2957,6 @@ static int log_ref_write(const char *refname, const
unsigned char *old_sha1,
return ret;
}
-int is_branch(const char *refname)
-{
- return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
-}
-
/*
* Write sha1 into the open lockfile, then close the lockfile. On
* errors, rollback the lockfile, fill in *err and
diff --git a/refs.c b/refs.c
index e9cc2d4..bc8750c 100644
--- a/refs.c
+++ b/refs.c
@@ -767,3 +767,8 @@ int check_refname_format(const char *refname, int flags)
return -1; /* Refname has only one component. */
return 0;
}
+
+int is_branch(const char *refname)
+{
+ return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
+}
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 32/44] initdb: move safe_create_dir into common code
In a moment, we'll create initdb functions for ref backends, and code
from initdb that calls this function needs to move into the files
backend. So this function needs to be public.
Signed-off-by: David Turner
---
builtin/init-db.c | 12
cache.h | 5 +
path.c| 12
3 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 313d13f..f6f7259 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -25,18 +25,6 @@ static int init_shared_repository = -1;
static const char *init_db_template_dir;
static const char *git_link;
-static void safe_create_dir(const char *dir, int share)
-{
- if (mkdir(dir, 0777) < 0) {
- if (errno != EEXIST) {
- perror(dir);
- exit(1);
- }
- }
- else if (share && adjust_shared_perm(dir))
- die(_("Could not make %s writable by group"), dir);
-}
-
static void copy_templates_1(struct strbuf *path, struct strbuf *template,
DIR *dir)
{
diff --git a/cache.h b/cache.h
index 1c45b62..cc817dc 100644
--- a/cache.h
+++ b/cache.h
@@ -1749,4 +1749,9 @@ void stat_validity_update(struct stat_validity *sv, int
fd);
int versioncmp(const char *s1, const char *s2);
void sleep_millisec(int millisec);
+/*
+ * Create a directory and (if share is nonzero) adjust its permissions
+ * according to the shared_repository setting.
+ */
+void safe_create_dir(const char *dir, int share);
#endif /* CACHE_H */
diff --git a/path.c b/path.c
index 48bd252..9ab2710 100644
--- a/path.c
+++ b/path.c
@@ -740,6 +740,18 @@ int adjust_shared_perm(const char *path)
return 0;
}
+void safe_create_dir(const char *dir, int share)
+{
+ if (mkdir(dir, 0777) < 0) {
+ if (errno != EEXIST) {
+ perror(dir);
+ exit(1);
+ }
+ }
+ else if (share && adjust_shared_perm(dir))
+ die(_("Could not make %s writable by group"), dir);
+}
+
static int have_same_root(const char *path1, const char *path2)
{
int is_abs1, is_abs2;
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 26/44] refs.c: move copy_msg to the common code
Rename copy_msg to copy_reflog_msg and make it public.
Signed-off-by: David Turner
---
refs-be-files.c | 28 +---
refs.c | 26 ++
refs.h | 2 ++
3 files changed, 29 insertions(+), 27 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 346429e..830b5a1 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2637,32 +2637,6 @@ static int commit_ref(struct ref_lock *lock)
return 0;
}
-/*
- * copy the reflog message msg to buf, which has been allocated sufficiently
- * large, while cleaning up the whitespaces. Especially, convert LF to space,
- * because reflog file is one line per entry.
- */
-static int copy_msg(char *buf, const char *msg)
-{
- char *cp = buf;
- char c;
- int wasspace = 1;
-
- *cp++ = '\t';
- while ((c = *msg++)) {
- if (wasspace && isspace(c))
- continue;
- wasspace = isspace(c);
- if (wasspace)
- c = ' ';
- *cp++ = c;
- }
- while (buf < cp && isspace(cp[-1]))
- cp--;
- *cp++ = '\n';
- return cp - buf;
-}
-
static int should_autocreate_reflog(const char *refname)
{
if (!log_all_ref_updates)
@@ -2765,7 +2739,7 @@ static int log_ref_write_fd(int fd, const unsigned char
*old_sha1,
sha1_to_hex(new_sha1),
committer);
if (msglen)
- len += copy_msg(logrec + len - 1, msg) - 1;
+ len += copy_reflog_msg(logrec + len - 1, msg) - 1;
written = len <= maxlen ? write_in_full(fd, logrec, len) : -1;
free(logrec);
diff --git a/refs.c b/refs.c
index 2515f6e..bd8c71b 100644
--- a/refs.c
+++ b/refs.c
@@ -886,6 +886,32 @@ int for_each_remote_ref_submodule(const char *submodule,
each_ref_fn fn, void *c
return for_each_ref_in_submodule(submodule, "refs/remotes/", fn,
cb_data);
}
+/*
+ * copy the reflog message msg to buf, which has been allocated sufficiently
+ * large, while cleaning up the whitespaces. Especially, convert LF to space,
+ * because reflog file is one line per entry.
+ */
+int copy_reflog_msg(char *buf, const char *msg)
+{
+ char *cp = buf;
+ char c;
+ int wasspace = 1;
+
+ *cp++ = '\t';
+ while ((c = *msg++)) {
+ if (wasspace && isspace(c))
+ continue;
+ wasspace = isspace(c);
+ if (wasspace)
+ c = ' ';
+ *cp++ = c;
+ }
+ while (buf < cp && isspace(cp[-1]))
+ cp--;
+ *cp++ = '\n';
+ return cp - buf;
+}
+
int head_ref_namespaced(each_ref_fn fn, void *cb_data)
{
struct strbuf buf = STRBUF_INIT;
diff --git a/refs.h b/refs.h
index 7a936e2..3da5d09 100644
--- a/refs.h
+++ b/refs.h
@@ -597,6 +597,8 @@ enum ref_type {
enum ref_type ref_type(const char *refname);
+int copy_reflog_msg(char *buf, const char *msg);
+
enum expire_reflog_flags {
EXPIRE_REFLOGS_DRY_RUN = 1 << 0,
EXPIRE_REFLOGS_UPDATE_REF = 1 << 1,
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 06/44] refs.c: move delete_ref and delete_refs to the common code
From: Ronnie Sahlberg
Move delete_pseudoref, delete_ref() and delete_refs() to the refs.c
file since these functions do not contain any backend specific code.
Based on a patch by Ronnie Sahlberg.
Signed-off-by: David Turner
Signed-off-by: Ronnie Sahlberg
---
refs-be-files.c | 94 -
refs.c | 94 +
2 files changed, 94 insertions(+), 94 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 7fe4931..099df75 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2877,100 +2877,6 @@ static int delete_ref_loose(struct ref_lock *lock, int
flag, struct strbuf *err)
return 0;
}
-static int delete_pseudoref(const char *pseudoref, const unsigned char
*old_sha1)
-{
- static struct lock_file lock;
- const char *filename;
-
- filename = git_path("%s", pseudoref);
-
- if (old_sha1 && !is_null_sha1(old_sha1)) {
- int fd;
- unsigned char actual_old_sha1[20];
-
- fd = hold_lock_file_for_update(&lock, filename,
- LOCK_DIE_ON_ERROR);
- if (fd < 0)
- die_errno(_("Could not open '%s' for writing"),
filename);
- if (read_ref(pseudoref, actual_old_sha1))
- die("could not read ref '%s'", pseudoref);
- if (hashcmp(actual_old_sha1, old_sha1)) {
- warning("Unexpected sha1 when deleting %s", pseudoref);
- rollback_lock_file(&lock);
- return -1;
- }
-
- unlink(filename);
- rollback_lock_file(&lock);
- } else {
- unlink(filename);
- }
-
- return 0;
-}
-
-int delete_ref(const char *refname, const unsigned char *old_sha1,
- unsigned int flags)
-{
- struct ref_transaction *transaction;
- struct strbuf err = STRBUF_INIT;
-
- if (ref_type(refname) == REF_TYPE_PSEUDOREF)
- return delete_pseudoref(refname, old_sha1);
-
- transaction = ref_transaction_begin(&err);
- if (!transaction ||
- ref_transaction_delete(transaction, refname, old_sha1,
- flags, NULL, &err) ||
- ref_transaction_commit(transaction, &err)) {
- error("%s", err.buf);
- ref_transaction_free(transaction);
- strbuf_release(&err);
- return 1;
- }
- ref_transaction_free(transaction);
- strbuf_release(&err);
- return 0;
-}
-
-int delete_refs(struct string_list *refnames)
-{
- struct strbuf err = STRBUF_INIT;
- int i, result = 0;
-
- if (!refnames->nr)
- return 0;
-
- result = repack_without_refs(refnames, &err);
- if (result) {
- /*
-* If we failed to rewrite the packed-refs file, then
-* it is unsafe to try to remove loose refs, because
-* doing so might expose an obsolete packed value for
-* a reference that might even point at an object that
-* has been garbage collected.
-*/
- if (refnames->nr == 1)
- error(_("could not delete reference %s: %s"),
- refnames->items[0].string, err.buf);
- else
- error(_("could not delete references: %s"), err.buf);
-
- goto out;
- }
-
- for (i = 0; i < refnames->nr; i++) {
- const char *refname = refnames->items[i].string;
-
- if (delete_ref(refname, NULL, 0))
- result |= error(_("could not remove reference %s"),
refname);
- }
-
-out:
- strbuf_release(&err);
- return result;
-}
-
/*
* People using contrib's git-new-workdir have .git/logs/refs ->
* /some/other/path/.git/logs/refs, and that may live on another device.
diff --git a/refs.c b/refs.c
index 2d10708..205a899 100644
--- a/refs.c
+++ b/refs.c
@@ -116,3 +116,97 @@ int update_ref(const char *msg, const char *refname,
ref_transaction_free(t);
return 0;
}
+
+
+static int delete_pseudoref(const char *pseudoref, const unsigned char
*old_sha1)
+{
+ static struct lock_file lock;
+ const char *filename;
+
+ filename = git_path("%s", pseudoref);
+
+ if (old_sha1 && !is_null_sha1(old_sha1)) {
+ int fd;
+ unsigned char actual_old_sha1[20];
+
+ fd = hold_lock_file_for_update(&lock, filename,
+ LOCK_DIE_ON_ERROR);
+ if (fd < 0)
+ die_errno(_("Could not open '%s' for writing"),
filename);
+ read_ref(pseudoref, actual_old_sha1);
+ if (hashcmp(actual_old_sha1, old_sha1)) {
+
[PATCH v3 21/44] refs-be-files.c: add methods for the ref iterators
From: Ronnie Sahlberg
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 41 --
refs.c | 69 ++---
refs.h | 34 +---
3 files changed, 116 insertions(+), 28 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 116d72d..2e2399e 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1862,32 +1862,36 @@ static int do_head_ref(const char *submodule,
each_ref_fn fn, void *cb_data)
return 0;
}
-int head_ref(each_ref_fn fn, void *cb_data)
+static int files_head_ref(each_ref_fn fn, void *cb_data)
{
return do_head_ref(NULL, fn, cb_data);
}
-int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+static int files_head_ref_submodule(const char *submodule, each_ref_fn fn,
+ void *cb_data)
{
return do_head_ref(submodule, fn, cb_data);
}
-int for_each_ref(each_ref_fn fn, void *cb_data)
+static int files_for_each_ref(each_ref_fn fn, void *cb_data)
{
return do_for_each_ref(&ref_cache, "", fn, 0, 0, cb_data);
}
-int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void
*cb_data)
+static int files_for_each_ref_submodule(const char *submodule, each_ref_fn fn,
+ void *cb_data)
{
return do_for_each_ref(get_ref_cache(submodule), "", fn, 0, 0, cb_data);
}
-int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)
+static int files_for_each_ref_in(const char *prefix, each_ref_fn fn,
+void *cb_data)
{
return do_for_each_ref(&ref_cache, prefix, fn, strlen(prefix), 0,
cb_data);
}
-int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data,
unsigned int broken)
+static int files_for_each_fullref_in(const char *prefix, each_ref_fn fn,
+void *cb_data, unsigned int broken)
{
unsigned int flag = 0;
@@ -1896,18 +1900,21 @@ int for_each_fullref_in(const char *prefix, each_ref_fn
fn, void *cb_data, unsig
return do_for_each_ref(&ref_cache, prefix, fn, 0, flag, cb_data);
}
-int for_each_ref_in_submodule(const char *submodule, const char *prefix,
- each_ref_fn fn, void *cb_data)
+static int files_for_each_ref_in_submodule(const char *submodule,
+ const char *prefix,
+ each_ref_fn fn, void *cb_data)
{
- return do_for_each_ref(get_ref_cache(submodule), prefix, fn,
strlen(prefix), 0, cb_data);
+ return do_for_each_ref(get_ref_cache(submodule), prefix, fn,
+ strlen(prefix), 0, cb_data);
}
-int for_each_replace_ref(each_ref_fn fn, void *cb_data)
+
+static int files_for_each_replace_ref(each_ref_fn fn, void *cb_data)
{
return do_for_each_ref(&ref_cache, git_replace_ref_base, fn,
strlen(git_replace_ref_base), 0, cb_data);
}
-int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
+static int files_for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
{
struct strbuf buf = STRBUF_INIT;
int ret;
@@ -1917,7 +1924,7 @@ int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
return ret;
}
-int for_each_rawref(each_ref_fn fn, void *cb_data)
+static int files_for_each_rawref(each_ref_fn fn, void *cb_data)
{
return do_for_each_ref(&ref_cache, "", fn, 0,
DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
@@ -3635,4 +3642,14 @@ struct ref_be refs_be_files = {
files_peel_ref,
files_create_symref,
files_resolve_gitlink_ref,
+ files_head_ref,
+ files_head_ref_submodule,
+ files_for_each_ref,
+ files_for_each_ref_submodule,
+ files_for_each_ref_in,
+ files_for_each_fullref_in,
+ files_for_each_ref_in_submodule,
+ files_for_each_rawref,
+ files_for_each_namespaced_ref,
+ files_for_each_replace_ref,
};
diff --git a/refs.c b/refs.c
index 61d90f5..657ab73 100644
--- a/refs.c
+++ b/refs.c
@@ -1042,19 +1042,8 @@ int peel_ref(const char *refname, unsigned char *sha1)
int create_symref(const char *ref_target, const char *refs_heads_master,
const char *logmsg)
{
- struct strbuf err = STRBUF_INIT;
- struct ref_transaction *transaction = ref_transaction_begin(&err);
- if (!transaction)
- return -1;
-
- if (the_refs_backend->create_symref(transaction, ref_target,
- refs_heads_master, logmsg))
- return -1;
-
- if (ref_transaction_commit(transaction, &err))
- return -1;
-
- return 0;
+ return the_refs_backend->create_symref(ref_target, refs_heads_master,
+ logmsg);
}
int resolve_gitlink_ref(const
[PATCH v3 34/44] refs-be-files.c: add method to rename refs
We also have to make rename_ref_available public, since alternate
backends for rename_ref will need it.
Signed-off-by: David Turner
---
refs-be-files.c | 20 +++-
refs.c | 21 +
refs.h | 7 ++-
3 files changed, 30 insertions(+), 18 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 5f2602c..4e6f032 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2444,29 +2444,14 @@ out:
return ret;
}
-static int rename_ref_available(const char *oldname, const char *newname)
-{
- struct string_list skip = STRING_LIST_INIT_NODUP;
- struct strbuf err = STRBUF_INIT;
- int ret;
-
- string_list_insert(&skip, oldname);
- ret = !verify_refname_available(newname, NULL, &skip, &err);
- if (!ret)
- error("%s", err.buf);
-
- string_list_clear(&skip, 0);
- strbuf_release(&err);
- return ret;
-}
-
static int write_ref_to_lockfile(struct ref_lock *lock,
const unsigned char *sha1, struct strbuf *err);
static int commit_ref_update(struct ref_lock *lock,
const unsigned char *sha1, const char *logmsg,
int flags, struct strbuf *err);
-int rename_ref(const char *oldrefname, const char *newrefname, const char
*logmsg)
+static int files_rename_ref(const char *oldrefname, const char *newrefname,
+ const char *logmsg)
{
unsigned char sha1[20], orig_sha1[20];
int flag = 0, logmoved = 0;
@@ -3561,6 +3546,7 @@ struct ref_be refs_be_files = {
files_initdb,
files_transaction_commit,
files_initial_transaction_commit,
+ files_rename_ref,
files_for_each_reflog_ent,
files_for_each_reflog_ent_reverse,
files_for_each_reflog,
diff --git a/refs.c b/refs.c
index 5fc8eb7..5a3125d 100644
--- a/refs.c
+++ b/refs.c
@@ -1113,6 +1113,11 @@ int ref_transaction_commit(struct ref_transaction
*transaction,
return the_refs_backend->transaction_commit(transaction, err);
}
+int rename_ref(const char *oldref, const char *newref, const char *logmsg)
+{
+ return the_refs_backend->rename_ref(oldref, newref, logmsg);
+}
+
const char *resolve_ref_unsafe(const char *ref, int resolve_flags,
unsigned char *sha1, int *flags)
{
@@ -1126,6 +1131,22 @@ int verify_refname_available(const char *refname, struct
string_list *extra,
return the_refs_backend->verify_refname_available(refname, extra, skip,
err);
}
+int rename_ref_available(const char *oldname, const char *newname)
+{
+ struct string_list skip = STRING_LIST_INIT_NODUP;
+ struct strbuf err = STRBUF_INIT;
+ int ret;
+
+ string_list_insert(&skip, oldname);
+ ret = !verify_refname_available(newname, NULL, &skip, &err);
+ if (!ret)
+ error("%s", err.buf);
+
+ string_list_clear(&skip, 0);
+ strbuf_release(&err);
+ return ret;
+}
+
int pack_refs(unsigned int flags)
{
return the_refs_backend->pack_refs(flags);
diff --git a/refs.h b/refs.h
index ce71d33..87d5801 100644
--- a/refs.h
+++ b/refs.h
@@ -464,10 +464,13 @@ extern const char *prettify_refname(const char *refname);
extern char *shorten_unambiguous_ref(const char *refname, int strict);
/** rename ref, return 0 on success **/
-extern int rename_ref(const char *oldref, const char *newref, const char
*logmsg);
+extern int rename_ref(const char *oldref, const char *newref,
+ const char *logmsg);
extern int create_symref(const char *ref, const char *refs_heads_master, const
char *logmsg);
+int rename_ref_available(const char *oldname, const char *newname);
+
enum action_on_err {
UPDATE_REFS_MSG_ON_ERR,
UPDATE_REFS_DIE_ON_ERR,
@@ -703,6 +706,7 @@ typedef int reflog_expire_fn(const char *refname, const
unsigned char *sha1,
void *policy_cb_data);
/* resolution functions */
+typedef int rename_ref_fn(const char *oldref, const char *newref, const char
*logmsg);
typedef const char *resolve_ref_unsafe_fn(const char *ref,
int resolve_flags,
unsigned char *sha1, int *flags);
@@ -743,6 +747,7 @@ struct ref_be {
ref_backend_initdb_fn *initdb;
ref_transaction_commit_fn *transaction_commit;
ref_transaction_commit_fn *initial_transaction_commit;
+ rename_ref_fn *rename_ref;
for_each_reflog_ent_fn *for_each_reflog_ent;
for_each_reflog_ent_reverse_fn *for_each_reflog_ent_reverse;
for_each_reflog_fn *for_each_reflog;
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 10/44] refs.c: move warn_if_dangling_symref* to the common code
From: Ronnie Sahlberg
These functions do not use any backend specific code so we move
them to the common code.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 52
refs.c | 52
2 files changed, 52 insertions(+), 52 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 37772a7..7d82864 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1944,58 +1944,6 @@ int peel_ref(const char *refname, unsigned char *sha1)
return peel_object(base, sha1);
}
-struct warn_if_dangling_data {
- FILE *fp;
- const char *refname;
- const struct string_list *refnames;
- const char *msg_fmt;
-};
-
-static int warn_if_dangling_symref(const char *refname, const struct object_id
*oid,
- int flags, void *cb_data)
-{
- struct warn_if_dangling_data *d = cb_data;
- const char *resolves_to;
- struct object_id junk;
-
- if (!(flags & REF_ISSYMREF))
- return 0;
-
- resolves_to = resolve_ref_unsafe(refname, 0, junk.hash, NULL);
- if (!resolves_to
- || (d->refname
- ? strcmp(resolves_to, d->refname)
- : !string_list_has_string(d->refnames, resolves_to))) {
- return 0;
- }
-
- fprintf(d->fp, d->msg_fmt, refname);
- fputc('\n', d->fp);
- return 0;
-}
-
-void warn_dangling_symref(FILE *fp, const char *msg_fmt, const char *refname)
-{
- struct warn_if_dangling_data data;
-
- data.fp = fp;
- data.refname = refname;
- data.refnames = NULL;
- data.msg_fmt = msg_fmt;
- for_each_rawref(warn_if_dangling_symref, &data);
-}
-
-void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct
string_list *refnames)
-{
- struct warn_if_dangling_data data;
-
- data.fp = fp;
- data.refname = NULL;
- data.refnames = refnames;
- data.msg_fmt = msg_fmt;
- for_each_rawref(warn_if_dangling_symref, &data);
-}
-
/*
* Call fn for each reference in the specified ref_cache, omitting
* references not in the containing_dir of base. fn is called for all
diff --git a/refs.c b/refs.c
index 310b7f5..24d5e28 100644
--- a/refs.c
+++ b/refs.c
@@ -581,3 +581,55 @@ char *shorten_unambiguous_ref(const char *refname, int
strict)
free(short_name);
return xstrdup(refname);
}
+
+struct warn_if_dangling_data {
+ FILE *fp;
+ const char *refname;
+ const struct string_list *refnames;
+ const char *msg_fmt;
+};
+
+static int warn_if_dangling_symref(const char *refname, const struct object_id
*oid,
+ int flags, void *cb_data)
+{
+ struct warn_if_dangling_data *d = cb_data;
+ const char *resolves_to;
+ struct object_id junk;
+
+ if (!(flags & REF_ISSYMREF))
+ return 0;
+
+ resolves_to = resolve_ref_unsafe(refname, 0, junk.hash, NULL);
+ if (!resolves_to
+ || (d->refname
+ ? strcmp(resolves_to, d->refname)
+ : !string_list_has_string(d->refnames, resolves_to))) {
+ return 0;
+ }
+
+ fprintf(d->fp, d->msg_fmt, refname);
+ fputc('\n', d->fp);
+ return 0;
+}
+
+void warn_dangling_symref(FILE *fp, const char *msg_fmt, const char *refname)
+{
+ struct warn_if_dangling_data data;
+
+ data.fp = fp;
+ data.refname = refname;
+ data.refnames = NULL;
+ data.msg_fmt = msg_fmt;
+ for_each_rawref(warn_if_dangling_symref, &data);
+}
+
+void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct
string_list *refnames)
+{
+ struct warn_if_dangling_data data;
+
+ data.fp = fp;
+ data.refname = NULL;
+ data.refnames = refnames;
+ data.msg_fmt = msg_fmt;
+ for_each_rawref(warn_if_dangling_symref, &data);
+}
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 19/44] refs-be-files.c: add a backend method structure with transaction functions
From: Ronnie Sahlberg
Add a ref structure for backend methods. Start by adding a method pointer
for the transaction commit function.
Add a function set_refs_backend to switch between backends. The files
based backend is the default.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 10 --
refs.c | 30 ++
refs.h | 15 +++
3 files changed, 53 insertions(+), 2 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 1308955..3050f1d 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3225,8 +3225,8 @@ static int ref_update_reject_duplicates(struct
string_list *refnames,
return 0;
}
-int ref_transaction_commit(struct ref_transaction *transaction,
- struct strbuf *err)
+static int files_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
{
int ret = 0, i;
int n = transaction->nr;
@@ -3612,3 +3612,9 @@ int reflog_expire(const char *refname, const unsigned
char *sha1,
unlock_ref(lock);
return -1;
}
+
+struct ref_be refs_be_files = {
+ NULL,
+ "files",
+ files_transaction_commit,
+};
diff --git a/refs.c b/refs.c
index 25ad3b5..f930fe0 100644
--- a/refs.c
+++ b/refs.c
@@ -4,6 +4,29 @@
#include "cache.h"
#include "refs.h"
#include "lockfile.h"
+/*
+ * We always have a files backend and it is the default.
+ */
+struct ref_be *the_refs_backend = &refs_be_files;
+/*
+ * List of all available backends
+ */
+struct ref_be *refs_backends = &refs_be_files;
+
+/*
+ * This function is used to switch to an alternate backend.
+ */
+int set_refs_backend(const char *name)
+{
+ struct ref_be *be;
+
+ for (be = refs_backends; be; be = be->next)
+ if (!strcmp(be->name, name)) {
+ the_refs_backend = be;
+ return 0;
+ }
+ return 1;
+}
static int is_per_worktree_ref(const char *refname)
{
@@ -985,3 +1008,10 @@ int ref_transaction_verify(struct ref_transaction
*transaction,
NULL, old_sha1,
flags, NULL, err);
}
+
+/* backend functions */
+int ref_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
+{
+ return the_refs_backend->transaction_commit(transaction, err);
+}
diff --git a/refs.h b/refs.h
index 4940ae9..419abf4 100644
--- a/refs.h
+++ b/refs.h
@@ -619,4 +619,19 @@ extern int reflog_expire(const char *refname, const
unsigned char *sha1,
reflog_expiry_cleanup_fn cleanup_fn,
void *policy_cb_data);
+/* refs backends */
+typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
+ struct strbuf *err);
+typedef void ref_transaction_free_fn(struct ref_transaction *transaction);
+
+struct ref_be {
+ struct ref_be *next;
+ const char *name;
+ ref_transaction_commit_fn *transaction_commit;
+};
+
+
+extern struct ref_be refs_be_files;
+int set_refs_backend(const char *name);
+
#endif /* REFS_H */
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 03/44] refs-be-files.c: rename refs to refs-be-files
From: Ronnie Sahlberg Rename refs.c to refs-be-files.c to indicate that this file now holds the implementation for the files based refs backend. A smaller portion of the code in this file is backend agnostic and will be moved to a a new refs.c file that will hold all the common refs code that is shared across all backends. A second reason for first moving all the code to the new file and then move the backend agnostic code back to refs.c instead of the other way around is because the code that will eventually remain in this new refs-be-files.c file is so entangled that it would then be very difficult to break the split up into small independent patches/chunks. Signed-off-by: Ronnie Sahlberg --- Makefile | 2 +- refs.c => refs-be-files.c | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename refs.c => refs-be-files.c (100%) diff --git a/Makefile b/Makefile index 46d0ca7..19036de 100644 --- a/Makefile +++ b/Makefile @@ -766,7 +766,7 @@ LIB_OBJS += quote.o LIB_OBJS += reachable.o LIB_OBJS += read-cache.o LIB_OBJS += reflog-walk.o -LIB_OBJS += refs.o +LIB_OBJS += refs-be-files.o LIB_OBJS += ref-filter.o LIB_OBJS += remote.o LIB_OBJS += replace_object.o diff --git a/refs.c b/refs-be-files.c similarity index 100% rename from refs.c rename to refs-be-files.c -- 2.4.2.644.g97b850b-twtrsrc -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 19/44] refs: add a backend method structure with transaction functions
From: Ronnie Sahlberg
Add a ref structure for backend methods. Start by adding a method pointer
for the transaction commit function.
Add a function set_refs_backend to switch between backends. The files
based backend is the default.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 10 --
refs.c | 30 ++
refs.h | 15 +++
3 files changed, 53 insertions(+), 2 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 1308955..3050f1d 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3225,8 +3225,8 @@ static int ref_update_reject_duplicates(struct
string_list *refnames,
return 0;
}
-int ref_transaction_commit(struct ref_transaction *transaction,
- struct strbuf *err)
+static int files_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
{
int ret = 0, i;
int n = transaction->nr;
@@ -3612,3 +3612,9 @@ int reflog_expire(const char *refname, const unsigned
char *sha1,
unlock_ref(lock);
return -1;
}
+
+struct ref_be refs_be_files = {
+ NULL,
+ "files",
+ files_transaction_commit,
+};
diff --git a/refs.c b/refs.c
index 25ad3b5..f930fe0 100644
--- a/refs.c
+++ b/refs.c
@@ -4,6 +4,29 @@
#include "cache.h"
#include "refs.h"
#include "lockfile.h"
+/*
+ * We always have a files backend and it is the default.
+ */
+struct ref_be *the_refs_backend = &refs_be_files;
+/*
+ * List of all available backends
+ */
+struct ref_be *refs_backends = &refs_be_files;
+
+/*
+ * This function is used to switch to an alternate backend.
+ */
+int set_refs_backend(const char *name)
+{
+ struct ref_be *be;
+
+ for (be = refs_backends; be; be = be->next)
+ if (!strcmp(be->name, name)) {
+ the_refs_backend = be;
+ return 0;
+ }
+ return 1;
+}
static int is_per_worktree_ref(const char *refname)
{
@@ -985,3 +1008,10 @@ int ref_transaction_verify(struct ref_transaction
*transaction,
NULL, old_sha1,
flags, NULL, err);
}
+
+/* backend functions */
+int ref_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
+{
+ return the_refs_backend->transaction_commit(transaction, err);
+}
diff --git a/refs.h b/refs.h
index 4940ae9..419abf4 100644
--- a/refs.h
+++ b/refs.h
@@ -619,4 +619,19 @@ extern int reflog_expire(const char *refname, const
unsigned char *sha1,
reflog_expiry_cleanup_fn cleanup_fn,
void *policy_cb_data);
+/* refs backends */
+typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
+ struct strbuf *err);
+typedef void ref_transaction_free_fn(struct ref_transaction *transaction);
+
+struct ref_be {
+ struct ref_be *next;
+ const char *name;
+ ref_transaction_commit_fn *transaction_commit;
+};
+
+
+extern struct ref_be refs_be_files;
+int set_refs_backend(const char *name);
+
#endif /* REFS_H */
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 07/44] refs.c: move read_ref_at to the common refs file
From: Ronnie Sahlberg
Move read_ref_at() to the refs.c file since this function does not
contain any backend specific code.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 118
refs.c | 118
2 files changed, 118 insertions(+), 118 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 099df75..4e7e05e 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3424,124 +3424,6 @@ int create_symref(const char *ref_target, const char
*refs_heads_master,
return 0;
}
-struct read_ref_at_cb {
- const char *refname;
- unsigned long at_time;
- int cnt;
- int reccnt;
- unsigned char *sha1;
- int found_it;
-
- unsigned char osha1[20];
- unsigned char nsha1[20];
- int tz;
- unsigned long date;
- char **msg;
- unsigned long *cutoff_time;
- int *cutoff_tz;
- int *cutoff_cnt;
-};
-
-static int read_ref_at_ent(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp, int tz,
- const char *message, void *cb_data)
-{
- struct read_ref_at_cb *cb = cb_data;
-
- cb->reccnt++;
- cb->tz = tz;
- cb->date = timestamp;
-
- if (timestamp <= cb->at_time || cb->cnt == 0) {
- if (cb->msg)
- *cb->msg = xstrdup(message);
- if (cb->cutoff_time)
- *cb->cutoff_time = timestamp;
- if (cb->cutoff_tz)
- *cb->cutoff_tz = tz;
- if (cb->cutoff_cnt)
- *cb->cutoff_cnt = cb->reccnt - 1;
- /*
-* we have not yet updated cb->[n|o]sha1 so they still
-* hold the values for the previous record.
-*/
- if (!is_null_sha1(cb->osha1)) {
- hashcpy(cb->sha1, nsha1);
- if (hashcmp(cb->osha1, nsha1))
- warning("Log for ref %s has gap after %s.",
- cb->refname, show_date(cb->date,
cb->tz, DATE_MODE(RFC2822)));
- }
- else if (cb->date == cb->at_time)
- hashcpy(cb->sha1, nsha1);
- else if (hashcmp(nsha1, cb->sha1))
- warning("Log for ref %s unexpectedly ended on %s.",
- cb->refname, show_date(cb->date, cb->tz,
- DATE_MODE(RFC2822)));
- hashcpy(cb->osha1, osha1);
- hashcpy(cb->nsha1, nsha1);
- cb->found_it = 1;
- return 1;
- }
- hashcpy(cb->osha1, osha1);
- hashcpy(cb->nsha1, nsha1);
- if (cb->cnt > 0)
- cb->cnt--;
- return 0;
-}
-
-static int read_ref_at_ent_oldest(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp,
- int tz, const char *message, void *cb_data)
-{
- struct read_ref_at_cb *cb = cb_data;
-
- if (cb->msg)
- *cb->msg = xstrdup(message);
- if (cb->cutoff_time)
- *cb->cutoff_time = timestamp;
- if (cb->cutoff_tz)
- *cb->cutoff_tz = tz;
- if (cb->cutoff_cnt)
- *cb->cutoff_cnt = cb->reccnt;
- hashcpy(cb->sha1, osha1);
- if (is_null_sha1(cb->sha1))
- hashcpy(cb->sha1, nsha1);
- /* We just want the first entry */
- return 1;
-}
-
-int read_ref_at(const char *refname, unsigned int flags, unsigned long
at_time, int cnt,
- unsigned char *sha1, char **msg,
- unsigned long *cutoff_time, int *cutoff_tz, int *cutoff_cnt)
-{
- struct read_ref_at_cb cb;
-
- memset(&cb, 0, sizeof(cb));
- cb.refname = refname;
- cb.at_time = at_time;
- cb.cnt = cnt;
- cb.msg = msg;
- cb.cutoff_time = cutoff_time;
- cb.cutoff_tz = cutoff_tz;
- cb.cutoff_cnt = cutoff_cnt;
- cb.sha1 = sha1;
-
- for_each_reflog_ent_reverse(refname, read_ref_at_ent, &cb);
-
- if (!cb.reccnt) {
- if (flags & GET_SHA1_QUIETLY)
- exit(128);
- else
- die("Log for %s is empty.", refname);
- }
- if (cb.found_it)
- return 0;
-
- for_each_reflog_ent(refname, read_ref_at_ent_oldest, &cb);
-
- return 1;
-}
-
int reflog_exists(const char *refname)
{
struct stat st;
diff --git a/refs.c b/refs.c
index 205a899..5a8ef0c 100644
--- a/refs.c
+++ b/refs.c
@@ -210,3 +210,121 @@ out:
strbuf_release(&err);
return result;
}
+
+struct read_ref_at_cb {
+ const char *refname;
+ unsigned long at_time;
+ int cnt;
+ i
[PATCH v3 18/44] refs: move transaction functions into common code
The common ref code will build up a ref transaction. Backends will
then commit it. So the transaction creation and update functions should
be in the common code. We also need to move the ref structs into
the common code so that alternate backends can access them.
Later, we will modify struct ref_update to support alternate backends.
Signed-off-by: David Turner
---
refs-be-files.c | 198
refs.c | 108 +++
refs.h | 91 +-
3 files changed, 198 insertions(+), 199 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 34ed220..1308955 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -13,41 +13,6 @@ struct ref_lock {
struct object_id old_oid;
};
-/*
- * Flag passed to lock_ref_sha1_basic() telling it to tolerate broken
- * refs (i.e., because the reference is about to be deleted anyway).
- */
-#define REF_DELETING 0x02
-
-/*
- * Used as a flag in ref_update::flags when a loose ref is being
- * pruned.
- */
-#define REF_ISPRUNING 0x04
-
-/*
- * Used as a flag in ref_update::flags when the reference should be
- * updated to new_sha1.
- */
-#define REF_HAVE_NEW 0x08
-
-/*
- * Used as a flag in ref_update::flags when old_sha1 should be
- * checked.
- */
-#define REF_HAVE_OLD 0x10
-
-/*
- * Used as a flag in ref_update::flags when the lockfile needs to be
- * committed.
- */
-#define REF_NEEDS_COMMIT 0x20
-
-/*
- * 0x40 is REF_FORCE_CREATE_REFLOG, so skip it if you're adding a
- * value to ref_update::flags
- */
-
struct ref_entry;
/*
@@ -3243,169 +3208,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
return retval;
}
-/**
- * Information needed for a single ref update. Set new_sha1 to the new
- * value or to null_sha1 to delete the ref. To check the old value
- * while the ref is locked, set (flags & REF_HAVE_OLD) and set
- * old_sha1 to the old value, or to null_sha1 to ensure the ref does
- * not exist before update.
- */
-struct ref_update {
- /*
-* If (flags & REF_HAVE_NEW), set the reference to this value:
-*/
- unsigned char new_sha1[20];
- /*
-* If (flags & REF_HAVE_OLD), check that the reference
-* previously had this value:
-*/
- unsigned char old_sha1[20];
- /*
-* One or more of REF_HAVE_NEW, REF_HAVE_OLD, REF_NODEREF,
-* REF_DELETING, and REF_ISPRUNING:
-*/
- unsigned int flags;
- struct ref_lock *lock;
- int type;
- char *msg;
- const char refname[FLEX_ARRAY];
-};
-
-/*
- * Transaction states.
- * OPEN: The transaction is in a valid state and can accept new updates.
- * An OPEN transaction can be committed.
- * CLOSED: A closed transaction is no longer active and no other operations
- * than free can be used on it in this state.
- * A transaction can either become closed by successfully committing
- * an active transaction or if there is a failure while building
- * the transaction thus rendering it failed/inactive.
- */
-enum ref_transaction_state {
- REF_TRANSACTION_OPEN = 0,
- REF_TRANSACTION_CLOSED = 1
-};
-
-/*
- * Data structure for holding a reference transaction, which can
- * consist of checks and updates to multiple references, carried out
- * as atomically as possible. This structure is opaque to callers.
- */
-struct ref_transaction {
- struct ref_update **updates;
- size_t alloc;
- size_t nr;
- enum ref_transaction_state state;
-};
-
-struct ref_transaction *ref_transaction_begin(struct strbuf *err)
-{
- assert(err);
-
- return xcalloc(1, sizeof(struct ref_transaction));
-}
-
-void ref_transaction_free(struct ref_transaction *transaction)
-{
- int i;
-
- if (!transaction)
- return;
-
- for (i = 0; i < transaction->nr; i++) {
- free(transaction->updates[i]->msg);
- free(transaction->updates[i]);
- }
- free(transaction->updates);
- free(transaction);
-}
-
-static struct ref_update *add_update(struct ref_transaction *transaction,
-const char *refname)
-{
- size_t len = strlen(refname) + 1;
- struct ref_update *update = xcalloc(1, sizeof(*update) + len);
-
- memcpy((char *)update->refname, refname, len); /* includes NUL */
- ALLOC_GROW(transaction->updates, transaction->nr + 1,
transaction->alloc);
- transaction->updates[transaction->nr++] = update;
- return update;
-}
-
-int ref_transaction_update(struct ref_transaction *transaction,
- const char *refname,
- const unsigned char *new_sha1,
- const unsigned char *old_sha1,
- unsigned int flags, const char *msg,
- struct strbuf *err)
-{
- struct ref_update *update;
-
-
[PATCH v3 20/44] refs-be-files.c: add methods for misc ref operations
From: Ronnie Sahlberg
Add ref backend methods for:
resolve_ref_unsafe, verify_refname_available, pack_refs, peel_ref,
create_symref, resolve_gitlink_ref.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
builtin/init-db.c | 1 +
cache.h | 7 +++
refs-be-files.c | 36 +++-
refs.c| 47 +++
refs.h| 18 ++
5 files changed, 100 insertions(+), 9 deletions(-)
diff --git a/builtin/init-db.c b/builtin/init-db.c
index cf6a3c8..313d13f 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -8,6 +8,7 @@
#include "builtin.h"
#include "exec_cmd.h"
#include "parse-options.h"
+#include "refs.h"
#ifndef DEFAULT_GIT_TEMPLATE_DIR
#define DEFAULT_GIT_TEMPLATE_DIR "/usr/share/git-core/templates"
diff --git a/cache.h b/cache.h
index 497d296..1c45b62 100644
--- a/cache.h
+++ b/cache.h
@@ -1104,6 +1104,13 @@ extern char *oid_to_hex(const struct object_id *oid);
/* same static buffer as s
extern int interpret_branch_name(const char *str, int len, struct strbuf *);
extern int get_sha1_mb(const char *str, unsigned char *sha1);
+/*
+ * Return true iff abbrev_name is a possible abbreviation for
+ * full_name according to the rules defined by ref_rev_parse_rules in
+ * refs.c.
+ */
+extern int refname_match(const char *abbrev_name, const char *full_name);
+
extern int validate_headref(const char *ref);
extern int base_name_compare(const char *name1, int len1, int mode1, const
char *name2, int len2, int mode2);
diff --git a/refs-be-files.c b/refs-be-files.c
index 3050f1d..116d72d 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1386,7 +1386,8 @@ static int resolve_gitlink_ref_recursive(struct ref_cache
*refs,
return resolve_gitlink_ref_recursive(refs, p, sha1, recursion+1);
}
-int resolve_gitlink_ref(const char *path, const char *refname, unsigned char
*sha1)
+static int files_resolve_gitlink_ref(const char *path, const char *refname,
+unsigned char *sha1)
{
int len = strlen(path), retval;
char *submodule;
@@ -1618,8 +1619,10 @@ static const char *resolve_ref_1(const char *refname,
}
}
-const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
- unsigned char *sha1, int *flags)
+static const char *files_resolve_ref_unsafe(const char *refname,
+ int resolve_flags,
+ unsigned char *sha1,
+ int *flags)
{
static struct strbuf sb_refname = STRBUF_INIT;
struct strbuf sb_contents = STRBUF_INIT;
@@ -1721,7 +1724,7 @@ static enum peel_status peel_entry(struct ref_entry
*entry, int repeel)
return status;
}
-int peel_ref(const char *refname, unsigned char *sha1)
+static int files_peel_ref(const char *refname, unsigned char *sha1)
{
int flag;
unsigned char base[20];
@@ -2355,7 +2358,7 @@ static void prune_refs(struct ref_to_prune *r)
}
}
-int pack_refs(unsigned int flags)
+static int files_pack_refs(unsigned int flags)
{
struct pack_refs_cb_data cbdata;
@@ -2677,8 +2680,16 @@ static int should_autocreate_reflog(const char *refname)
!strcmp(refname, "HEAD");
}
-int verify_refname_available(const char *newname, struct string_list *extra,
-struct string_list *skip, struct strbuf *err)
+/*
+ * Create a reflog for a ref. If force_create = 0, the reflog will
+ * only be created for certain refs (those for which
+ * should_autocreate_reflog returns non-zero. Otherwise, create it
+ * regardless of the ref name. Fill in *err and return -1 on failure.
+ */
+static int files_verify_refname_available(const char *newname,
+ struct string_list *extra,
+ struct string_list *skip,
+ struct strbuf *err)
{
struct ref_dir *packed_refs = get_packed_refs(&ref_cache);
struct ref_dir *loose_refs = get_loose_refs(&ref_cache);
@@ -2913,8 +2924,9 @@ static int commit_ref_update(struct ref_lock *lock,
return 0;
}
-int create_symref(const char *ref_target, const char *refs_heads_master,
- const char *logmsg)
+static int files_create_symref(const char *ref_target,
+ const char *refs_heads_master,
+ const char *logmsg)
{
char *lockpath = NULL;
char ref[1000];
@@ -3617,4 +3629,10 @@ struct ref_be refs_be_files = {
NULL,
"files",
files_transaction_commit,
+ files_resolve_ref_unsafe,
+ files_verify_refname_available,
+ files_pack_refs,
+ files_peel_ref,
+ files_create_symref,
+ files_resolve_gitlink_ref,
};
diff --git a/refs.c b/refs.c
index f930fe0..
[PATCH v3 23/44] refs-be-files.c: add do_for_each_per_worktree_ref
Alternate refs backends might still use files to store per-worktree
refs. So the files backend's ref-loading infrastructure should be
available to those backends, just for use on per-worktree refs. Add
do_for_each_per_worktree_ref, which iterates over per-worktree refs.
Signed-off-by: David Turner
---
refs-be-files.c | 15 ---
refs.h | 11 +++
2 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index c99319f..9548fb3 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -552,9 +552,6 @@ static void sort_ref_dir(struct ref_dir *dir)
dir->sorted = dir->nr = i;
}
-/* Include broken references in a do_for_each_ref*() iteration: */
-#define DO_FOR_EACH_INCLUDE_BROKEN 0x01
-
/*
* Return true iff the reference described by entry can be resolved to
* an object in the database. Emit a warning if the referred-to
@@ -602,6 +599,10 @@ static int do_one_ref(struct ref_entry *entry, void
*cb_data)
struct ref_entry *old_current_ref;
int retval;
+ if (data->flags & DO_FOR_EACH_PER_WORKTREE_ONLY &&
+ ref_type(entry->name) != REF_TYPE_PER_WORKTREE)
+ return 0;
+
if (!starts_with(entry->name, data->base))
return 0;
@@ -1844,6 +1845,14 @@ static int do_for_each_ref(struct ref_cache *refs, const
char *base,
return do_for_each_entry(refs, base, do_one_ref, &data);
}
+int do_for_each_per_worktree_ref(const char *submodule, const char *base,
+each_ref_fn fn, int trim, int flags,
+void *cb_data)
+{
+ return do_for_each_ref(get_ref_cache(submodule), base, fn, trim,
+ flags | DO_FOR_EACH_PER_WORKTREE_ONLY, cb_data);
+}
+
static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
{
struct object_id oid;
diff --git a/refs.h b/refs.h
index 491aa1d..e2ba725 100644
--- a/refs.h
+++ b/refs.h
@@ -241,6 +241,12 @@ struct ref_transaction {
*/
#define REF_BAD_NAME 0x08
+/* Include broken references in a do_for_each_ref*() iteration */
+#define DO_FOR_EACH_INCLUDE_BROKEN 0x01
+
+/* Only include per-worktree refs in a do_for_each_ref*() iteration */
+#define DO_FOR_EACH_PER_WORKTREE_ONLY 0x02
+
/*
* The signature for the callback function for the for_each_*()
* functions below. The memory pointed to by the refname and sha1
@@ -284,6 +290,11 @@ extern int for_each_namespaced_ref(each_ref_fn fn, void
*cb_data);
/* can be used to learn about broken ref and symref */
extern int for_each_rawref(each_ref_fn fn, void *cb_data);
+
+int do_for_each_per_worktree_ref(const char *submodule, const char *base,
+each_ref_fn fn, int trim, int flags,
+void *cb_data);
+
static inline const char *has_glob_specials(const char *pattern)
{
return strpbrk(pattern, "?*[");
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 28/44] refs.c: move should_autocreate_reflog to common code
Signed-off-by: David Turner
---
refs-be-files.c | 10 --
refs.c | 10 ++
refs.h | 2 ++
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index bf2fd7a..73111e7 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2584,16 +2584,6 @@ static int commit_ref(struct ref_lock *lock)
return 0;
}
-static int should_autocreate_reflog(const char *refname)
-{
- if (!log_all_ref_updates)
- return 0;
- return starts_with(refname, "refs/heads/") ||
- starts_with(refname, "refs/remotes/") ||
- starts_with(refname, "refs/notes/") ||
- !strcmp(refname, "HEAD");
-}
-
/*
* Create a reflog for a ref. If force_create = 0, the reflog will
* only be created for certain refs (those for which
diff --git a/refs.c b/refs.c
index 99b31f6..1de8529 100644
--- a/refs.c
+++ b/refs.c
@@ -685,6 +685,16 @@ char *resolve_refdup(const char *refname, int
resolve_flags,
sha1, flags));
}
+int should_autocreate_reflog(const char *refname)
+{
+ if (!log_all_ref_updates)
+ return 0;
+ return starts_with(refname, "refs/heads/") ||
+ starts_with(refname, "refs/remotes/") ||
+ starts_with(refname, "refs/notes/") ||
+ !strcmp(refname, "HEAD");
+}
+
/*
* How to handle various characters in refnames:
* 0: An acceptable character for refs
diff --git a/refs.h b/refs.h
index 636f959..6d284f5 100644
--- a/refs.h
+++ b/refs.h
@@ -58,6 +58,8 @@ extern const char *resolve_ref_unsafe(const char *refname,
int resolve_flags,
extern char *resolve_refdup(const char *refname, int resolve_flags,
unsigned char *sha1, int *flags);
+extern int should_autocreate_reflog(const char *refname);
+
extern int read_ref_full(const char *refname, int resolve_flags,
unsigned char *sha1, int *flags);
extern int read_ref(const char *refname, unsigned char *sha1);
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 30/44] refs.c: add methods for reflog
In the file-based backend, the reflog piggybacks on the ref lock.
Since other backends won't have the same sort of ref lock, ref backends
must also handle reflogs.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 35 +++
refs.c | 46 ++
refs.h | 28
3 files changed, 97 insertions(+), 12 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index f123459..104e90f 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2650,7 +2650,8 @@ static int log_ref_setup(const char *refname, struct
strbuf *logfile, struct str
}
-int safe_create_reflog(const char *refname, int force_create, struct strbuf
*err)
+static int files_create_reflog(const char *refname, int force_create,
+ struct strbuf *err)
{
int ret;
struct strbuf sb = STRBUF_INIT;
@@ -2898,7 +2899,7 @@ static int files_create_symref(const char *ref_target,
return 0;
}
-int reflog_exists(const char *refname)
+static int files_reflog_exists(const char *refname)
{
struct stat st;
@@ -2906,7 +2907,7 @@ int reflog_exists(const char *refname)
S_ISREG(st.st_mode);
}
-int delete_reflog(const char *refname)
+static int files_delete_reflog(const char *refname)
{
return remove_path(git_path("logs/%s", refname));
}
@@ -2950,7 +2951,9 @@ static char *find_beginning_of_line(char *bob, char *scan)
return scan;
}
-int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn,
void *cb_data)
+static int files_for_each_reflog_ent_reverse(const char *refname,
+each_reflog_ent_fn fn,
+void *cb_data)
{
struct strbuf sb = STRBUF_INIT;
FILE *logfp;
@@ -3052,7 +3055,8 @@ int for_each_reflog_ent_reverse(const char *refname,
each_reflog_ent_fn fn, void
return ret;
}
-int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn, void
*cb_data)
+static int files_for_each_reflog_ent(const char *refname,
+each_reflog_ent_fn fn, void *cb_data)
{
FILE *logfp;
struct strbuf sb = STRBUF_INIT;
@@ -3114,7 +3118,7 @@ static int do_for_each_reflog(struct strbuf *name,
each_ref_fn fn, void *cb_data
return retval;
}
-int for_each_reflog(each_ref_fn fn, void *cb_data)
+static int files_for_each_reflog(each_ref_fn fn, void *cb_data)
{
int retval;
struct strbuf name;
@@ -3429,12 +3433,12 @@ static int expire_reflog_ent(unsigned char *osha1,
unsigned char *nsha1,
return 0;
}
-int reflog_expire(const char *refname, const unsigned char *sha1,
-unsigned int flags,
-reflog_expiry_prepare_fn prepare_fn,
-reflog_expiry_should_prune_fn should_prune_fn,
-reflog_expiry_cleanup_fn cleanup_fn,
-void *policy_cb_data)
+static int files_reflog_expire(const char *refname, const unsigned char *sha1,
+ unsigned int flags,
+ reflog_expiry_prepare_fn prepare_fn,
+ reflog_expiry_should_prune_fn should_prune_fn,
+ reflog_expiry_cleanup_fn cleanup_fn,
+ void *policy_cb_data)
{
static struct lock_file reflog_lock;
struct expire_reflog_cb cb;
@@ -3539,6 +3543,13 @@ struct ref_be refs_be_files = {
"files",
files_init_backend,
files_transaction_commit,
+ files_for_each_reflog_ent,
+ files_for_each_reflog_ent_reverse,
+ files_for_each_reflog,
+ files_reflog_exists,
+ files_create_reflog,
+ files_delete_reflog,
+ files_reflog_expire,
files_resolve_ref_unsafe,
files_verify_refname_available,
files_pack_refs,
diff --git a/refs.c b/refs.c
index 6370ace..304fead 100644
--- a/refs.c
+++ b/refs.c
@@ -1204,3 +1204,49 @@ int for_each_reftype_fullpath(each_ref_fn fn, char
*type, unsigned int broken,
return the_refs_backend->for_each_reftype_fullpath(fn, type, broken,
cb_data);
}
+
+int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn,
+ void *cb_data)
+{
+ return the_refs_backend->for_each_reflog_ent_reverse(refname, fn,
+cb_data);
+}
+
+int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn,
+ void *cb_data)
+{
+ return the_refs_backend->for_each_reflog_ent(refname, fn, cb_data);
+}
+
+int for_each_reflog(each_ref_fn fn, void *cb_data)
+{
+ return the_refs_backend->for_each_reflog(fn, cb_data);
+}
+
+int reflog_exists(const char *refname)
+{
+ return the_refs_backend->refl
[PATCH v3 27/44] refs.c: move peel_object to the common code
This function does not contain any backend specific code so we
move it to the common code.
Signed-off-by: David Turner
---
refs-be-files.c | 53 -
refs.c | 31 +++
refs.h | 27 +++
3 files changed, 58 insertions(+), 53 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 830b5a1..bf2fd7a 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1604,59 +1604,6 @@ static const char *files_resolve_ref_unsafe(const char
*refname,
return ret;
}
-enum peel_status {
- /* object was peeled successfully: */
- PEEL_PEELED = 0,
-
- /*
-* object cannot be peeled because the named object (or an
-* object referred to by a tag in the peel chain), does not
-* exist.
-*/
- PEEL_INVALID = -1,
-
- /* object cannot be peeled because it is not a tag: */
- PEEL_NON_TAG = -2,
-
- /* ref_entry contains no peeled value because it is a symref: */
- PEEL_IS_SYMREF = -3,
-
- /*
-* ref_entry cannot be peeled because it is broken (i.e., the
-* symbolic reference cannot even be resolved to an object
-* name):
-*/
- PEEL_BROKEN = -4
-};
-
-/*
- * Peel the named object; i.e., if the object is a tag, resolve the
- * tag recursively until a non-tag is found. If successful, store the
- * result to sha1 and return PEEL_PEELED. If the object is not a tag
- * or is not valid, return PEEL_NON_TAG or PEEL_INVALID, respectively,
- * and leave sha1 unchanged.
- */
-static enum peel_status peel_object(const unsigned char *name, unsigned char
*sha1)
-{
- struct object *o = lookup_unknown_object(name);
-
- if (o->type == OBJ_NONE) {
- int type = sha1_object_info(name, NULL);
- if (type < 0 || !object_as_type(o, type, 0))
- return PEEL_INVALID;
- }
-
- if (o->type != OBJ_TAG)
- return PEEL_NON_TAG;
-
- o = deref_tag_noverify(o);
- if (!o)
- return PEEL_INVALID;
-
- hashcpy(sha1, o->sha1);
- return PEEL_PEELED;
-}
-
/*
* Peel the entry (if possible) and return its new peel_status. If
* repeel is true, re-peel the entry even if there is an old peeled
diff --git a/refs.c b/refs.c
index bd8c71b..99b31f6 100644
--- a/refs.c
+++ b/refs.c
@@ -4,6 +4,9 @@
#include "cache.h"
#include "refs.h"
#include "lockfile.h"
+#include "object.h"
+#include "tag.h"
+
/*
* We always have a files backend and it is the default.
*/
@@ -1059,6 +1062,34 @@ int refname_is_safe(const char *refname)
return 1;
}
+/*
+ * Peel the named object; i.e., if the object is a tag, resolve the
+ * tag recursively until a non-tag is found. If successful, store the
+ * result to sha1 and return PEEL_PEELED. If the object is not a tag
+ * or is not valid, return PEEL_NON_TAG or PEEL_INVALID, respectively,
+ * and leave sha1 unchanged.
+ */
+enum peel_status peel_object(const unsigned char *name, unsigned char *sha1)
+{
+ struct object *o = lookup_unknown_object(name);
+
+ if (o->type == OBJ_NONE) {
+ int type = sha1_object_info(name, NULL);
+ if (type < 0 || !object_as_type(o, type, 0))
+ return PEEL_INVALID;
+ }
+
+ if (o->type != OBJ_TAG)
+ return PEEL_NON_TAG;
+
+ o = deref_tag_noverify(o);
+ if (!o)
+ return PEEL_INVALID;
+
+ hashcpy(sha1, o->sha1);
+ return PEEL_PEELED;
+}
+
/* backend functions */
int ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
diff --git a/refs.h b/refs.h
index 3da5d09..636f959 100644
--- a/refs.h
+++ b/refs.h
@@ -76,6 +76,33 @@ extern int is_branch(const char *refname);
*/
extern int peel_ref(const char *refname, unsigned char *sha1);
+enum peel_status {
+ /* object was peeled successfully: */
+ PEEL_PEELED = 0,
+
+ /*
+* object cannot be peeled because the named object (or an
+* object referred to by a tag in the peel chain), does not
+* exist.
+*/
+ PEEL_INVALID = -1,
+
+ /* object cannot be peeled because it is not a tag: */
+ PEEL_NON_TAG = -2,
+
+ /* ref_entry contains no peeled value because it is a symref: */
+ PEEL_IS_SYMREF = -3,
+
+ /*
+* ref_entry cannot be peeled because it is broken (i.e., the
+* symbolic reference cannot even be resolved to an object
+* name):
+*/
+ PEEL_BROKEN = -4
+};
+
+enum peel_status peel_object(const unsigned char *name, unsigned char *sha1);
+
/**
* Resolve refname in the nested "gitlink" repository that is located
* at path. If the resolution is successful, return 0 and set sha1 to
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of
[PATCH v3 29/44] refs.c: add ref backend init function
The file backend doesn't need this function, but other backends might.
Signed-off-by: David Turner
Signed-off-by: Ronnie Sahlberg
---
refs-be-files.c | 6 ++
refs.c | 3 ++-
refs.h | 4 +++-
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 73111e7..f123459 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3307,6 +3307,11 @@ static int ref_present(const char *refname,
return string_list_has_string(affected_refnames, refname);
}
+
+void files_init_backend(void *data)
+{
+ /* do nothing */
+}
int initial_ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
@@ -3532,6 +3537,7 @@ int reflog_expire(const char *refname, const unsigned
char *sha1,
struct ref_be refs_be_files = {
NULL,
"files",
+ files_init_backend,
files_transaction_commit,
files_resolve_ref_unsafe,
files_verify_refname_available,
diff --git a/refs.c b/refs.c
index 1de8529..6370ace 100644
--- a/refs.c
+++ b/refs.c
@@ -19,13 +19,14 @@ struct ref_be *refs_backends = &refs_be_files;
/*
* This function is used to switch to an alternate backend.
*/
-int set_refs_backend(const char *name)
+int set_refs_backend(const char *name, void *init_data)
{
struct ref_be *be;
for (be = refs_backends; be; be = be->next)
if (!strcmp(be->name, name)) {
the_refs_backend = be;
+ be->init_backend(init_data);
return 0;
}
return 1;
diff --git a/refs.h b/refs.h
index 6d284f5..e76553a 100644
--- a/refs.h
+++ b/refs.h
@@ -675,6 +675,7 @@ extern int reflog_expire(const char *refname, const
unsigned char *sha1,
void *policy_cb_data);
/* refs backends */
+typedef void ref_backend_init_fn(void *data);
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
struct strbuf *err);
typedef void ref_transaction_free_fn(struct ref_transaction *transaction);
@@ -714,6 +715,7 @@ typedef int for_each_reftype_fullpath_fn(each_ref_fn fn,
char *type,
struct ref_be {
struct ref_be *next;
const char *name;
+ ref_backend_init_fn *init_backend;
ref_transaction_commit_fn *transaction_commit;
resolve_ref_unsafe_fn *resolve_ref_unsafe;
verify_refname_available_fn *verify_refname_available;
@@ -736,6 +738,6 @@ struct ref_be {
extern struct ref_be refs_be_files;
-int set_refs_backend(const char *name);
+int set_refs_backend(const char *name, void *init_data);
#endif /* REFS_H */
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 31/44] refs.c: add method for initial ref transaction commit
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 5 +++--
refs.c | 6 ++
refs.h | 1 +
3 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 104e90f..7560a55 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3317,8 +3317,8 @@ void files_init_backend(void *data)
/* do nothing */
}
-int initial_ref_transaction_commit(struct ref_transaction *transaction,
- struct strbuf *err)
+static int files_initial_transaction_commit(struct ref_transaction
*transaction,
+ struct strbuf *err)
{
int ret = 0, i;
int n = transaction->nr;
@@ -3543,6 +3543,7 @@ struct ref_be refs_be_files = {
"files",
files_init_backend,
files_transaction_commit,
+ files_initial_transaction_commit,
files_for_each_reflog_ent,
files_for_each_reflog_ent_reverse,
files_for_each_reflog,
diff --git a/refs.c b/refs.c
index 304fead..6e41ed8 100644
--- a/refs.c
+++ b/refs.c
@@ -1250,3 +1250,9 @@ int reflog_expire(const char *refname, const unsigned
char *sha1,
prepare_fn, should_prune_fn,
cleanup_fn, policy_cb_data);
}
+
+int initial_ref_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
+{
+ return the_refs_backend->initial_transaction_commit(transaction, err);
+}
diff --git a/refs.h b/refs.h
index 577..e3136ee 100644
--- a/refs.h
+++ b/refs.h
@@ -738,6 +738,7 @@ struct ref_be {
const char *name;
ref_backend_init_fn *init_backend;
ref_transaction_commit_fn *transaction_commit;
+ ref_transaction_commit_fn *initial_transaction_commit;
for_each_reflog_ent_fn *for_each_reflog_ent;
for_each_reflog_ent_reverse_fn *for_each_reflog_ent_reverse;
for_each_reflog_fn *for_each_reflog;
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 40/44] refs: allow ref backend to be set for clone
Add a new option, --refs-backend-type, to allow the ref backend type to
be set on new clones.
Submodules must use the same ref backend as the parent repository, so
we also pass the --refs-backend-type option option when cloning
submodules.
Signed-off-by: David Turner
---
Documentation/git-clone.txt | 4
builtin/clone.c | 27 +--
builtin/submodule--helper.c | 5 -
cache.h | 1 +
refs.c | 2 ++
5 files changed, 36 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index f1f2a3f..d7a4cb0 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,6 +14,7 @@ SYNOPSIS
[-o ] [-b ] [-u ] [--reference ]
[--dissociate] [--separate-git-dir ]
[--depth ] [--[no-]single-branch]
+ [--refs-backend-type=]
[--recursive | --recurse-submodules] [--]
[]
@@ -216,6 +217,9 @@ objects from the source repository into a pack in the
cloned repository.
The result is Git repository can be separated from working
tree.
+--refs-backend-type=::
+ Type of refs backend. Default is to use the original files based
+ backend.
::
The (possibly remote) repository to clone from. See the
diff --git a/builtin/clone.c b/builtin/clone.c
index 3e14491..d489a87 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -92,11 +92,13 @@ static struct option builtin_clone_options[] = {
N_("separate git dir from working tree")),
OPT_STRING_LIST('c', "config", &option_config, N_("key=value"),
N_("set config inside the new repository")),
+ OPT_STRING(0, "refs-backend-type", &refs_backend_type,
+ N_("name"), N_("name of backend type to use")),
OPT_END()
};
static const char *argv_submodule[] = {
- "submodule", "update", "--init", "--recursive", NULL
+ "submodule", "update", "--init", "--recursive", NULL, NULL
};
static const char *get_repo_path_1(struct strbuf *path, int *is_bundle)
@@ -724,8 +726,24 @@ static int checkout(void)
err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
sha1_to_hex(sha1), "1", NULL);
- if (!err && option_recursive)
+ if (!err && option_recursive) {
+ const char **backend_arg = argv_submodule;
+ char *new_backend_arg = NULL;
+ if (refs_backend_type) {
+ while (*backend_arg)
+ ++backend_arg;
+
+ new_backend_arg = xmalloc(21 +
strlen(refs_backend_type));
+ sprintf(new_backend_arg, "--refs-backend-type=%s",
+ refs_backend_type);
+ *backend_arg = new_backend_arg;
+ }
err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+ if (refs_backend_type) {
+ free(new_backend_arg);
+ *backend_arg = NULL;
+ }
+ }
return err;
}
@@ -744,6 +762,11 @@ static void write_config(struct string_list *config)
write_one_config, NULL) < 0)
die("unable to write parameters to config file");
}
+
+ if (refs_backend_type &&
+ write_one_config("core.refs-backend-type",
+refs_backend_type, NULL) < 0)
+ die("unable to write backend parameter to config file");
}
static void write_refspec_config(const char *src_ref_prefix,
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f4c3eff..5c9ca4e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -140,7 +140,10 @@ static int clone_submodule(const char *path, const char
*gitdir, const char *url
argv_array_pushl(&cp.args, "--reference", reference, NULL);
if (gitdir && *gitdir)
argv_array_pushl(&cp.args, "--separate-git-dir", gitdir, NULL);
-
+ if (refs_backend_type && *refs_backend_type) {
+ argv_array_push(&cp.args, "--refs-backend-type");
+ argv_array_push(&cp.args, refs_backend_type);
+ }
argv_array_push(&cp.args, url);
argv_array_push(&cp.args, path);
diff --git a/cache.h b/cache.h
index cc817dc..692cfd3 100644
--- a/cache.h
+++ b/cache.h
@@ -695,6 +695,7 @@ enum object_creation_mode {
extern enum object_creation_mode object_creation_mode;
extern char *notes_ref_name;
+extern const char *refs_backend_type;
extern int grafts_replace_parents;
diff --git a/refs.c b/refs.c
index 025cc08..901558d 100644
--- a/refs.c
+++ b/refs.c
@@ -21,6 +21,8 @@ struct ref_be *the_refs_backend = &refs_be_files;
*/
struct ref_be *refs_backends = &refs_be_files;
+const char *refs_backend_type;
+
/*
* This fun
[PATCH v3 44/44] refs: tests for db backend
Add tests for the database backend.
Signed-off-by: David Turner
Helped-by: Dennis Kaarsemaker
---
t/t1460-refs-be-db.sh| 1109 ++
t/t1470-refs-be-db-reflog.sh | 359 ++
t/test-lib.sh|1 +
3 files changed, 1469 insertions(+)
create mode 100755 t/t1460-refs-be-db.sh
create mode 100755 t/t1470-refs-be-db-reflog.sh
diff --git a/t/t1460-refs-be-db.sh b/t/t1460-refs-be-db.sh
new file mode 100755
index 000..56d2d2b
--- /dev/null
+++ b/t/t1460-refs-be-db.sh
@@ -0,0 +1,1109 @@
+#!/bin/sh
+#
+# Copyright (c) 2015 Twitter, Inc
+# Copyright (c) 2006 Shawn Pearce
+# This test is based on t1400-update-ref.sh
+#
+
+test_description='Test lmdb refs backend'
+TEST_NO_CREATE_REPO=1
+. ./test-lib.sh
+
+if ! test_have_prereq LMDB
+then
+ skip_all="Skipping lmdb refs backend tests, lmdb backend not built"
+ test_done
+fi
+
+raw_ref() {
+ test-refs-be-lmdb "$1"
+}
+
+delete_ref() {
+ test-refs-be-lmdb -d "$1"
+}
+
+write_ref() {
+ test-refs-be-lmdb "$1" "$2"
+}
+
+raw_reflog() {
+ test-refs-be-lmdb -l "$1"
+}
+
+delete_all_reflogs() {
+ test-refs-be-lmdb -c
+}
+
+append_reflog() {
+ test-refs-be-lmdb -a "$1"
+}
+
+Z=$_z40
+
+test_expect_success setup '
+ git init --refs-backend-type=lmdb &&
+ for name in A B C D E F
+ do
+ test_tick &&
+ T=$(git write-tree) &&
+ sha1=$(echo $name | git commit-tree $T) &&
+ eval $name=$sha1
+ done
+'
+
+m=refs/heads/master
+n_dir=refs/heads/gu
+n=$n_dir/fixes
+
+test_expect_success \
+ "create $m" \
+ "git update-ref $m $A &&
+test $A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "create $m" \
+ "git update-ref $m $B $A &&
+test $B"' = $(raw_ref '"$m"')'
+test_expect_success "fail to delete $m with stale ref" '
+ test_must_fail git update-ref -d $m $A &&
+ test $B = "$(raw_ref $m)"
+'
+test_expect_success "delete $m" '
+ git update-ref -d $m $B &&
+ ! raw_ref $m
+'
+delete_ref $m
+
+test_expect_success "delete $m without oldvalue verification" "
+ git update-ref $m $A &&
+ test $A = \$(raw_ref $m) &&
+ git update-ref -d $m &&
+ ! raw_ref $m
+"
+delete_ref $m
+
+test_expect_success \
+ "fail to create $n" \
+ "git update-ref $n_dir $A &&
+test_must_fail git update-ref $n $A >out 2>err"
+
+delete_ref $n_dir
+rm -f out err
+
+test_expect_success \
+ "create $m (by HEAD)" \
+ "git update-ref HEAD $A &&
+test $A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "create $m (by HEAD)" \
+ "git update-ref HEAD $B $A &&
+test $B"' = $(raw_ref '"$m"')'
+test_expect_success "fail to delete $m (by HEAD) with stale ref" '
+ test_must_fail git update-ref -d HEAD $A &&
+ test $B = $(raw_ref '"$m"')
+'
+test_expect_success "delete $m (by HEAD)" '
+ git update-ref -d HEAD $B &&
+ ! raw_ref $m
+'
+delete_ref $m
+
+test_expect_success \
+ "create $m (by HEAD)" \
+ "git update-ref HEAD $A &&
+test $A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "pack refs" \
+ "git pack-refs --all"
+test_expect_success \
+ "move $m (by HEAD)" \
+ "git update-ref HEAD $B $A &&
+test $B"' = $(raw_ref '"$m"')'
+test_expect_success "delete $m (by HEAD) should remove both packed and loose
$m" '
+ git update-ref -d HEAD $B &&
+ ! raw_ref $m
+'
+delete_ref $m
+
+OLD_HEAD=$(raw_ref HEAD)
+test_expect_success "delete symref without dereference" '
+ git update-ref --no-deref -d HEAD &&
+ ! raw_ref HEAD
+'
+write_ref HEAD "$OLD_HEAD"
+
+test_expect_success "delete symref without dereference when the referred ref
is packed" '
+ echo foo >foo.c &&
+ git add foo.c &&
+ git commit -m foo &&
+ git pack-refs --all &&
+ git update-ref --no-deref -d HEAD &&
+ ! raw_ref HEAD
+'
+write_ref HEAD "$OLD_HEAD"
+delete_ref $m
+
+test_expect_success 'update-ref -d is not confused by self-reference' '
+ git symbolic-ref refs/heads/self refs/heads/self &&
+ test_when_finished "delete_ref refs/heads/self" &&
+ test_must_fail git update-ref -d refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete self-reference' '
+ git symbolic-ref refs/heads/self refs/heads/self &&
+ test_when_finished "delete_ref refs/heads/self" &&
+ git update-ref --no-deref -d refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete reference to bad ref'
'
+ test-refs-be-lmdb refs/heads/bad "" &&
+ test_when_finished "delete_ref refs/heads/bad" &&
+ git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
+ test_when_finished "delete_ref refs/heads/ref-to-bad" &&
+ raw_ref refs/heads/ref-to-bad &&
+ git update-ref --no-deref -d refs/heads/ref-to-bad &&
+ ! raw_ref re
[PATCH v3 43/44] refs: add LMDB refs backend
Add a database backend for refs using LMDB. This backend runs git for-each-ref about 30% faster than the files backend with fully-packed refs on a repo with ~120k refs. It's also about 4x faster than using fully-unpacked refs. In addition, and perhaps more importantly , it avoids case-conflict issues on OS X. LMDB has a few features that make it suitable for usage in git: 1. It is relatively lightweight; it requires only one header file, and the library code takes under 64k at runtime. 2. It is well-tested: it's been used in OpenLDAP for years. 3. It's very fast. LMDB's benchmarks show that it is among the fastest key-value stores. 4. It has a relatively simple concurrency story; readers don't block writers and writers don't block readers. Ronnie Sahlberg's original version of this patchset used tdb. The major disadvantage of tdb is that tdb is hard to build on OS X. It's also not in homebrew. So lmdb seemed simpler. To test this backend's correctness, I hacked test-lib.sh and test-lib-functions.sh to run all tests under the refs backend. Dozens of tests use manual ref/reflog reading/writing, or create submodules without passing --refs-backend-type to git init. If those tests are changed to use the update-ref machinery or test-refs-be-db (or, in the case of packed-refs, corrupt refs, and dumb fetch tests, are skipped), the only remaining failing tests are the git-new-workdir tests and the gitweb tests. Signed-off-by: David Turner --- .gitignore |1 + Documentation/config.txt |7 + Documentation/git-clone.txt|2 +- Documentation/git-init-db.txt |2 +- Documentation/git-init.txt |6 + Documentation/technical/refs-be-lmdb.txt | 50 + Documentation/technical/repository-version.txt |5 + Makefile | 12 + builtin/clone.c|2 +- builtin/init-db.c | 20 +- config.c | 34 + configure.ac | 33 + contrib/workdir/git-new-workdir|3 + environment.c |1 + refs-be-lmdb.c | 1997 refs.h | 10 + setup.c| 32 +- test-refs-be-lmdb.c| 68 + 18 files changed, 2275 insertions(+), 10 deletions(-) create mode 100644 Documentation/technical/refs-be-lmdb.txt create mode 100644 refs-be-lmdb.c create mode 100644 test-refs-be-lmdb.c diff --git a/.gitignore b/.gitignore index 1c2f832..7decc2f 100644 --- a/.gitignore +++ b/.gitignore @@ -199,6 +199,7 @@ /test-path-utils /test-prio-queue /test-read-cache +/test-refs-be-lmdb /test-regex /test-revision-walking /test-run-command diff --git a/Documentation/config.txt b/Documentation/config.txt index ab57276..eccfdb6 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -496,6 +496,13 @@ core.repositoryFormatVersion:: Internal variable identifying the repository format and layout version. +core.refsBackendType:: + Type of refs backend. Default is to use the original files + based backend. Set to 'lmdb' to activate the lmdb database + backend. If you use the lmdb backend, + core.repositoryFormatVersion must be set to 1, and + extensions.refBackend must be set to 'lmdb'. + core.sharedRepository:: When 'group' (or 'true'), the repository is made shareable between several users in a group (making sure all the files and objects are diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt index d7a4cb0..6a4cf28 100644 --- a/Documentation/git-clone.txt +++ b/Documentation/git-clone.txt @@ -219,7 +219,7 @@ objects from the source repository into a pack in the cloned repository. --refs-backend-type=:: Type of refs backend. Default is to use the original files based - backend. + backend. Set to "lmdb" to activate the lmdb database backend. :: The (possibly remote) repository to clone from. See the diff --git a/Documentation/git-init-db.txt b/Documentation/git-init-db.txt index 648a6cd..72fbd71 100644 --- a/Documentation/git-init-db.txt +++ b/Documentation/git-init-db.txt @@ -9,7 +9,7 @@ git-init-db - Creates an empty Git repository SYNOPSIS [verse] -'git init-db' [-q | --quiet] [--bare] [--template=] [--separate-git-dir ] [--shared[=]] +'git init-db' [-q | --quiet] [--bare] [--template=] [--separate-git-dir ] [--shared[=]] [--refs-backend-type=] DESCRIPTION diff --git a/Documentation/git-init.txt b/Documentation/git-init.txt index 8174d27..d352788 100644 --- a/Documentation/git-init.txt +++ b/Documentation/git-init.txt @@ -12,6 +12,7 @@ SYNOPSIS 'git init' [-q | --quiet] [--ba
[PATCH v3 22/44] refs-be-files.c: add method for for_each_reftype_...
Add method for for_each_reftype_fullpath.
Signed-off-by: David Turner
---
refs-be-files.c | 11 +++
refs.c | 7 +++
refs.h | 3 +++
3 files changed, 21 insertions(+)
diff --git a/refs-be-files.c b/refs-be-files.c
index 2e2399e..c99319f 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1914,6 +1914,16 @@ static int files_for_each_replace_ref(each_ref_fn fn,
void *cb_data)
strlen(git_replace_ref_base), 0, cb_data);
}
+static int files_for_each_reftype_fullpath(each_ref_fn fn, char *type,
+ unsigned int broken, void *cb_data)
+{
+ unsigned int flag = 0;
+
+ if (broken)
+ flag = DO_FOR_EACH_INCLUDE_BROKEN;
+ return do_for_each_ref(&ref_cache, type, fn, 0, flag, cb_data);
+}
+
static int files_for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
{
struct strbuf buf = STRBUF_INIT;
@@ -3652,4 +3662,5 @@ struct ref_be refs_be_files = {
files_for_each_rawref,
files_for_each_namespaced_ref,
files_for_each_replace_ref,
+ files_for_each_reftype_fullpath,
};
diff --git a/refs.c b/refs.c
index 657ab73..e627d74 100644
--- a/refs.c
+++ b/refs.c
@@ -1105,3 +1105,10 @@ int for_each_replace_ref(each_ref_fn fn, void *cb_data)
{
return the_refs_backend->for_each_replace_ref(fn, cb_data);
}
+
+int for_each_reftype_fullpath(each_ref_fn fn, char *type, unsigned int broken,
+ void *cb_data)
+{
+ return the_refs_backend->for_each_reftype_fullpath(fn, type, broken,
+ cb_data);
+}
diff --git a/refs.h b/refs.h
index 6dec0da..491aa1d 100644
--- a/refs.h
+++ b/refs.h
@@ -653,6 +653,8 @@ typedef int for_each_ref_in_submodule_fn(const char
*submodule,
typedef int for_each_rawref_fn(each_ref_fn fn, void *cb_data);
typedef int for_each_namespaced_ref_fn(each_ref_fn fn, void *cb_data);
typedef int for_each_replace_ref_fn(each_ref_fn fn, void *cb_data);
+typedef int for_each_reftype_fullpath_fn(each_ref_fn fn, char *type,
+unsigned int broken, void *cb_data);
struct ref_be {
struct ref_be *next;
@@ -674,6 +676,7 @@ struct ref_be {
for_each_rawref_fn *for_each_rawref;
for_each_namespaced_ref_fn *for_each_namespaced_ref;
for_each_replace_ref_fn *for_each_replace_ref;
+ for_each_reftype_fullpath_fn *for_each_reftype_fullpath;
};
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 39/44] refs: always handle non-normal refs in files backend
Always handle non-normal (per-worktree or pseudo) refs in the files
backend instead of alternate backends.
Sometimes a ref transaction will update both a per-worktree ref and a
normal ref. For instance, an ordinary commit might update
refs/heads/master and HEAD (or at least HEAD's reflog).
We handle three cases here:
1. updates to normal refs continue to go through the chosen backend
2. updates to non-normal refs with REF_NODEREF or to non-symbolic refs
are moved to a separate files backend transaction.
3. updates to symbolic refs are dereferenced to their base ref. The
update to the base ref then goes through the ordinary backend, while
the files backend is directly called to update the symref's reflog.
Signed-off-by: David Turner
---
refs.c | 141 -
1 file changed, 139 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index b00f910..025cc08 100644
--- a/refs.c
+++ b/refs.c
@@ -7,6 +7,11 @@
#include "object.h"
#include "tag.h"
+const char split_transaction_fail_warning[] =
+ "A ref transaction was split across two refs backends. Part of the "
+ "transaction succeeded, but then the update to the per-worktree refs "
+ "failed. Your repository may be in an inconsistent state.";
+
/*
* We always have a files backend and it is the default.
*/
@@ -963,6 +968,13 @@ void ref_transaction_free(struct ref_transaction
*transaction)
free(transaction);
}
+static void add_update_obj(struct ref_transaction *transaction,
+ struct ref_update *update)
+{
+ ALLOC_GROW(transaction->updates, transaction->nr + 1,
transaction->alloc);
+ transaction->updates[transaction->nr++] = update;
+}
+
static struct ref_update *add_update(struct ref_transaction *transaction,
const char *refname)
{
@@ -970,8 +982,7 @@ static struct ref_update *add_update(struct ref_transaction
*transaction,
struct ref_update *update = xcalloc(1, sizeof(*update) + len);
memcpy((char *)update->refname, refname, len); /* includes NUL */
- ALLOC_GROW(transaction->updates, transaction->nr + 1,
transaction->alloc);
- transaction->updates[transaction->nr++] = update;
+ add_update_obj(transaction, update);
return update;
}
@@ -1160,11 +1171,87 @@ int refs_initdb(struct strbuf *err, int shared)
return the_refs_backend->initdb(err, shared);
}
+/*
+ * Special case for non-normal refs. For symbolic-refs when
+ * REF_NODEREF is not turned on, we dereference them here and replace
+ * updates to the symbolic refs with updates to the underlying ref.
+ * Then we do our own reflogging for the symbolic ref.
+ *
+ * We move other non-normal ref updates with into a specially-created
+ * files-backend transaction
+ */
+static int move_abnormal_ref_updates(struct ref_transaction *transaction,
+struct ref_transaction *files_transaction,
+struct string_list *symrefs)
+{
+ int i;
+
+ for (i = 0; i < transaction->nr; i++) {
+ struct ref_update *update = transaction->updates[i];
+ const char *resolved;
+ int flags = 0;
+ unsigned char sha1[20];
+
+ if (ref_type(update->refname) == REF_TYPE_NORMAL)
+ continue;
+
+ resolved = resolve_ref_unsafe(update->refname, 0, sha1, &flags);
+
+ if (update->flags & REF_NODEREF || !(flags & REF_ISSYMREF)) {
+ int last;
+
+ add_update_obj(files_transaction, update);
+ /*
+* Replace this transaction with the
+* last transaction, removing it from
+* the list of backend transactions
+*/
+ last = --transaction->nr;
+ transaction->updates[i] = transaction->updates[last];
+ continue;
+ }
+
+ if (resolved) {
+ struct ref_update *new_update;
+ struct string_list_item *item;
+
+ if (ref_type(resolved) != REF_TYPE_NORMAL)
+ die("Non-normal symbolic ref `%s` points to
non-normal ref `%s`", update->refname, resolved);
+
+ new_update = xmalloc(sizeof(*new_update) +
+strlen(resolved) + 1);
+ memcpy(new_update, update, sizeof(*update));
+
+ if (update->flags & REF_HAVE_OLD &&
+ hashcmp(sha1, update->old_sha1)) {
+ /* consistency check failed */
+ free(new_update);
+ return -1;
+ } else {
+
[PATCH v3 41/44] refs: add register_refs_backend
Add register_refs_backend, to allow refs backends to be registered.
Signed-off-by: David Turner
---
refs.c | 6 ++
refs.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/refs.c b/refs.c
index 901558d..534d12e 100644
--- a/refs.c
+++ b/refs.c
@@ -23,6 +23,12 @@ struct ref_be *refs_backends = &refs_be_files;
const char *refs_backend_type;
+void register_refs_backend(struct ref_be *be)
+{
+ be->next = refs_backends;
+ refs_backends = be;
+}
+
/*
* This function is used to switch to an alternate backend.
*/
diff --git a/refs.h b/refs.h
index a9fdbf0..45615e8 100644
--- a/refs.h
+++ b/refs.h
@@ -790,5 +790,6 @@ struct ref_be {
extern struct ref_be refs_be_files;
int set_refs_backend(const char *name, void *init_data);
+void register_refs_backend(struct ref_be *be);
#endif /* REFS_H */
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 42/44] introduce "extensions" form of core.repositoryformatversion
From: Jeff King Normally we try to avoid bumps of the whole-repository core.repositoryformatversion field. However, it is unavoidable if we want to safely change certain aspects of git in a backwards-incompatible way (e.g., modifying the set of ref tips that we must traverse to generate a list of unreachable, safe-to-prune objects). If we were to bump the repository version for every such change, then any implementation understanding version `X` would also have to understand `X-1`, `X-2`, and so forth, even though the incompatibilities may be in orthogonal parts of the system, and there is otherwise no reason we cannot implement one without the other (or more importantly, that the user cannot choose to use one feature without the other, weighing the tradeoff in compatibility only for that particular feature). This patch documents the existing repositoryformatversion strategy and introduces a new format, "1", which lets a repository specify that it must run with an arbitrary set of extensions. This can be used, for example: - to inform git that the objects should not be pruned based only on the reachability of the ref tips (e.g, because it has "clone --shared" children) - that the refs are stored in a format besides the usual "refs" and "packed-refs" directories Because we bump to format "1", and because format "1" requires that a running git knows about any extensions mentioned, we know that older versions of the code will not do something dangerous when confronted with these new formats. For example, if the user chooses to use database storage for refs, they may set the "extensions.refbackend" config to "db". Older versions of git will not understand format "1" and bail. Versions of git which understand "1" but do not know about "refbackend", or which know about "refbackend" but not about the "db" backend, will refuse to run. This is annoying, of course, but much better than the alternative of claiming that there are no refs in the repository, or writing to a location that other implementations will not read. Note that we are only defining the rules for format 1 here. We do not ever write format 1 ourselves; it is a tool that is meant to be used by users and future extensions to provide safety with older implementations. Signed-off-by: Jeff King --- Documentation/technical/repository-version.txt | 81 ++ cache.h| 6 ++ setup.c| 37 +++- t/t1302-repo-version.sh| 38 4 files changed, 159 insertions(+), 3 deletions(-) create mode 100644 Documentation/technical/repository-version.txt diff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt new file mode 100644 index 000..3d7106d --- /dev/null +++ b/Documentation/technical/repository-version.txt @@ -0,0 +1,81 @@ +Git Repository Format Versions +== + +Every git repository is marked with a numeric version in the +`core.repositoryformatversion` key of its `config` file. This version +specifies the rules for operating on the on-disk repository data. An +implementation of git which does not understand a particular version +advertised by an on-disk repository MUST NOT operate on that repository; +doing so risks not only producing wrong results, but actually losing +data. + +Because of this rule, version bumps should be kept to an absolute +minimum. Instead, we generally prefer these strategies: + + - bumping format version numbers of individual data files (e.g., +index, packfiles, etc). This restricts the incompatibilities only to +those files. + + - introducing new data that gracefully degrades when used by older +clients (e.g., pack bitmap files are ignored by older clients, which +simply do not take advantage of the optimization they provide). + +A whole-repository format version bump should only be part of a change +that cannot be independently versioned. For instance, if one were to +change the reachability rules for objects, or the rules for locking +refs, that would require a bump of the repository format version. + +Note that this applies only to accessing the repository's disk contents +directly. An older client which understands only format `0` may still +connect via `git://` to a repository using format `1`, as long as the +server process understands format `1`. + +The preferred strategy for rolling out a version bump (whether whole +repository or for a single file) is to teach git to read the new format, +and allow writing the new format with a config switch or command line +option (for experimentation or for those who do not care about backwards +compatibility with older gits). Then after a long period to allow the +reading capability to become common, we may switch to writing the new +format by default. + +The currently defined format versions are: + +Version `0` +--- + +
[PATCH v3 37/44] refs: break out a ref conflict check
Create new function verify_no_descendants, to hold one of the ref
conflict checks used in verify_refname_available. Multiple backends
will need this function, so it goes in the common code.
Signed-off-by: David Turner
---
refs-be-files.c | 33 -
refs.c | 22 ++
refs.h | 7 +++
3 files changed, 37 insertions(+), 25 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 8d1ffce..dd2c42e 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -730,6 +730,7 @@ static int verify_refname_available_dir(const char *refname,
struct strbuf *err)
{
const char *slash;
+ const char *extra_refname;
int pos;
struct strbuf dirname = STRBUF_INIT;
int ret = -1;
@@ -835,33 +836,15 @@ static int verify_refname_available_dir(const char
*refname,
}
}
- if (extras) {
- /*
-* Check for entries in extras that start with
-* "$refname/". We do that by looking for the place
-* where "$refname/" would be inserted in extras. If
-* there is an entry at that position that starts with
-* "$refname/" and is not in skip, then we have a
-* conflict.
-*/
- for (pos = string_list_find_insert_index(extras, dirname.buf,
0);
-pos < extras->nr; pos++) {
- const char *extra_refname = extras->items[pos].string;
-
- if (!starts_with(extra_refname, dirname.buf))
- break;
-
- if (!skip || !string_list_has_string(skip,
extra_refname)) {
- strbuf_addf(err, "cannot process '%s' and '%s'
at the same time",
- refname, extra_refname);
- goto cleanup;
- }
- }
+ extra_refname = find_descendant_ref(dirname.buf, extras, skip);
+ if (extra_refname) {
+ strbuf_addf(err,
+ "cannot process '%s' and '%s' at the same time",
+ refname, extra_refname);
+ } else {
+ ret = 0;
}
- /* No conflicts were found */
- ret = 0;
-
cleanup:
strbuf_release(&dirname);
return ret;
diff --git a/refs.c b/refs.c
index 5a3125d..3ae0274 100644
--- a/refs.c
+++ b/refs.c
@@ -1101,6 +1101,28 @@ enum peel_status peel_object(const unsigned char *name,
unsigned char *sha1)
return PEEL_PEELED;
}
+const char *find_descendant_ref(const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip)
+{
+ int pos;
+ if (!extras)
+ return NULL;
+
+ /* Look for the place where "$refname/" would be inserted in extras. */
+ for (pos = string_list_find_insert_index(extras, refname, 0);
+pos < extras->nr; pos++) {
+ const char *extra_refname = extras->items[pos].string;
+
+ if (!starts_with(extra_refname, refname))
+ break;
+
+ if (!skip || !string_list_has_string(skip, extra_refname))
+ return extra_refname;
+ }
+ return NULL;
+}
+
/* backend functions */
int refs_initdb(struct strbuf *err, int shared)
{
diff --git a/refs.h b/refs.h
index 3aad3b8..f8becea 100644
--- a/refs.h
+++ b/refs.h
@@ -637,6 +637,13 @@ int files_log_ref_write(const char *refname, const
unsigned char *old_sha1,
const unsigned char *new_sha1, const char *msg,
int flags, struct strbuf *err);
+/*
+ * Check for entries in extras that start with "$refname/", ignoring
+ * those in skip. If there is such an entry, then we have a conflict.
+ */
+const char *find_descendant_ref(const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip);
enum expire_reflog_flags {
EXPIRE_REFLOGS_DRY_RUN = 1 << 0,
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 36/44] refs: make files_log_ref_write functions public
Because HEAD and stash are per-worktree, refs.c needs to go through
the files backend to write these refs.
In this patch, we make one files backend internal functions
public. Later, we will use this to handle reflog updates for
per-worktree symbolic refs (HEAD).
Signed-off-by: David Turner
---
refs-be-files.c | 8
refs.h | 5 +
2 files changed, 13 insertions(+)
diff --git a/refs-be-files.c b/refs-be-files.c
index 798a995..8d1ffce 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2710,6 +2710,14 @@ static int log_ref_write(const char *refname, const
unsigned char *old_sha1,
const unsigned char *new_sha1, const char *msg,
int flags, struct strbuf *err)
{
+ return files_log_ref_write(refname, old_sha1, new_sha1, msg, flags,
+ err);
+}
+
+int files_log_ref_write(const char *refname, const unsigned char *old_sha1,
+ const unsigned char *new_sha1, const char *msg,
+ int flags, struct strbuf *err)
+{
struct strbuf sb = STRBUF_INIT;
int ret = log_ref_write_1(refname, old_sha1, new_sha1, msg, &sb, flags,
err);
diff --git a/refs.h b/refs.h
index 4d8b3bb..3aad3b8 100644
--- a/refs.h
+++ b/refs.h
@@ -633,6 +633,11 @@ enum ref_type ref_type(const char *refname);
int copy_reflog_msg(char *buf, const char *msg);
+int files_log_ref_write(const char *refname, const unsigned char *old_sha1,
+ const unsigned char *new_sha1, const char *msg,
+ int flags, struct strbuf *err);
+
+
enum expire_reflog_flags {
EXPIRE_REFLOGS_DRY_RUN = 1 << 0,
EXPIRE_REFLOGS_UPDATE_REF = 1 << 1,
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 25/44] refs.h: document make refname_is_safe and add it to header
This function might be used by other refs backends Signed-off-by: David Turner --- refs.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/refs.h b/refs.h index fc8a748..7a936e2 100644 --- a/refs.h +++ b/refs.h @@ -348,6 +348,17 @@ int verify_refname_available(const char *newname, struct string_list *extra, struct string_list *skip, struct strbuf *err); /* + * Check if a refname is safe. + * For refs that start with "refs/" we consider it safe as long they do + * not try to resolve to outside of refs/. + * + * For all other refs we only consider them safe iff they only contain + * upper case characters and '_' (like "HEAD" AND "MERGE_HEAD", and not like + * "config"). + */ +int refname_is_safe(const char *refname); + +/* * Flags controlling ref_transaction_update(), ref_transaction_create(), etc. * REF_NODEREF: act on the ref directly, instead of dereferencing * symbolic references. -- 2.4.2.644.g97b850b-twtrsrc -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 33/44] refs.c: add method for initializing refs db
Alternate refs backends might not need refs/heads and so on, so we make
ref db initialization part of the backend.
Signed-off-by: David Turner
---
builtin/init-db.c | 14 --
refs-be-files.c | 17 +
refs.c| 5 +
refs.h| 4
4 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/builtin/init-db.c b/builtin/init-db.c
index f6f7259..2705739 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -178,13 +178,7 @@ static int create_default_files(const char *template_path)
char junk[2];
int reinit;
int filemode;
-
- /*
-* Create .git/refs/{heads,tags}
-*/
- safe_create_dir(git_path_buf(&buf, "refs"), 1);
- safe_create_dir(git_path_buf(&buf, "refs/heads"), 1);
- safe_create_dir(git_path_buf(&buf, "refs/tags"), 1);
+ struct strbuf err = STRBUF_INIT;
/* Just look for `init.templatedir` */
git_config(git_init_db_config, NULL);
@@ -208,11 +202,11 @@ static int create_default_files(const char *template_path)
*/
if (shared_repository) {
adjust_shared_perm(get_git_dir());
- adjust_shared_perm(git_path_buf(&buf, "refs"));
- adjust_shared_perm(git_path_buf(&buf, "refs/heads"));
- adjust_shared_perm(git_path_buf(&buf, "refs/tags"));
}
+ if (refs_initdb(&err, shared_repository))
+ die("failed to set up refs db: %s", err.buf);
+
/*
* Create the default symlink from ".git/HEAD" to the "master"
* branch, if it does not exist yet.
diff --git a/refs-be-files.c b/refs-be-files.c
index 7560a55..5f2602c 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3538,10 +3538,27 @@ static int files_reflog_expire(const char *refname,
const unsigned char *sha1,
return -1;
}
+static int files_initdb(struct strbuf *err, int shared)
+{
+ /*
+* Create .git/refs/{heads,tags}
+*/
+ safe_create_dir(git_path("refs"), 1);
+ safe_create_dir(git_path("refs/heads"), 1);
+ safe_create_dir(git_path("refs/tags"), 1);
+ if (shared) {
+ adjust_shared_perm(git_path("refs"));
+ adjust_shared_perm(git_path("refs/heads"));
+ adjust_shared_perm(git_path("refs/tags"));
+ }
+ return 0;
+}
+
struct ref_be refs_be_files = {
NULL,
"files",
files_init_backend,
+ files_initdb,
files_transaction_commit,
files_initial_transaction_commit,
files_for_each_reflog_ent,
diff --git a/refs.c b/refs.c
index 6e41ed8..5fc8eb7 100644
--- a/refs.c
+++ b/refs.c
@@ -1102,6 +1102,11 @@ enum peel_status peel_object(const unsigned char *name,
unsigned char *sha1)
}
/* backend functions */
+int refs_initdb(struct strbuf *err, int shared)
+{
+ return the_refs_backend->initdb(err, shared);
+}
+
int ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
{
diff --git a/refs.h b/refs.h
index e3136ee..ce71d33 100644
--- a/refs.h
+++ b/refs.h
@@ -68,6 +68,8 @@ extern int ref_exists(const char *refname);
extern int is_branch(const char *refname);
+extern int refs_initdb(struct strbuf *err, int shared);
+
/*
* If refname is a non-symbolic reference that refers to a tag object,
* and the tag can be (recursively) dereferenced to a non-tag object,
@@ -676,6 +678,7 @@ extern int reflog_expire(const char *refname, const
unsigned char *sha1,
/* refs backends */
typedef void ref_backend_init_fn(void *data);
+typedef int ref_backend_initdb_fn(struct strbuf *err, int shared);
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
struct strbuf *err);
typedef void ref_transaction_free_fn(struct ref_transaction *transaction);
@@ -737,6 +740,7 @@ struct ref_be {
struct ref_be *next;
const char *name;
ref_backend_init_fn *init_backend;
+ ref_backend_initdb_fn *initdb;
ref_transaction_commit_fn *transaction_commit;
ref_transaction_commit_fn *initial_transaction_commit;
for_each_reflog_ent_fn *for_each_reflog_ent;
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 35/44] refs: make lock generic
Instead of using a files-backend-specific struct ref_lock, the generic
ref_transaction struct should provide a void pointer that backends can use
for their own lock data.
Signed-off-by: David Turner
---
refs-be-files.c | 29 -
refs.h | 2 +-
2 files changed, 17 insertions(+), 14 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 4e6f032..798a995 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3167,11 +3167,12 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
*/
for (i = 0; i < n; i++) {
struct ref_update *update = updates[i];
+ struct ref_lock *lock;
if ((update->flags & REF_HAVE_NEW) &&
is_null_sha1(update->new_sha1))
update->flags |= REF_DELETING;
- update->lock = lock_ref_sha1_basic(
+ lock = lock_ref_sha1_basic(
update->refname,
((update->flags & REF_HAVE_OLD) ?
update->old_sha1 : NULL),
@@ -3179,7 +3180,8 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
update->flags,
&update->type,
err);
- if (!update->lock) {
+ update->backend_data = lock;
+ if (!lock) {
char *reason;
ret = (errno == ENOTDIR)
@@ -3197,12 +3199,12 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
(update->flags &
REF_NODEREF));
if (!overwriting_symref &&
- !hashcmp(update->lock->old_oid.hash,
update->new_sha1)) {
+ !hashcmp(lock->old_oid.hash, update->new_sha1)) {
/*
* The reference already has the desired
* value, so we don't need to write it.
*/
- } else if (write_ref_to_lockfile(update->lock,
+ } else if (write_ref_to_lockfile(lock,
update->new_sha1,
err)) {
char *write_err = strbuf_detach(err, NULL);
@@ -3211,7 +3213,7 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
* The lock was freed upon failure of
* write_ref_to_lockfile():
*/
- update->lock = NULL;
+ update->backend_data = NULL;
strbuf_addf(err,
"cannot update the ref '%s': %s",
update->refname, write_err);
@@ -3227,7 +3229,7 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
* We didn't have to write anything to the lockfile.
* Close it to free up the file descriptor:
*/
- if (close_ref(update->lock)) {
+ if (close_ref(lock)) {
strbuf_addf(err, "Couldn't close %s.lock",
update->refname);
goto cleanup;
@@ -3240,16 +3242,16 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
struct ref_update *update = updates[i];
if (update->flags & REF_NEEDS_COMMIT) {
- if (commit_ref_update(update->lock,
+ if (commit_ref_update(update->backend_data,
update->new_sha1, update->msg,
update->flags, err)) {
/* freed by commit_ref_update(): */
- update->lock = NULL;
+ update->backend_data = NULL;
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
} else {
/* freed by commit_ref_update(): */
- update->lock = NULL;
+ update->backend_data = NULL;
}
}
}
@@ -3257,16 +3259,17 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
/* Perform deletes now that updates are safely completed */
for (i = 0; i < n; i++) {
struct ref_update *update = updates[i];
+ struct ref_lock *lock = upda
[PATCH v3 38/44] refs: move duplicate check to common code
The check for duplicate refnames in a transaction is needed for
all backends, so move it to the common code.
ref_transaction_commit_fn gains a new argument, the sorted
string_list of affected refnames.
Signed-off-by: David Turner
---
refs-be-files.c | 57 -
refs.c | 71 +++--
refs.h | 1 +
3 files changed, 75 insertions(+), 54 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index dd2c42e..b89bd5c 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3104,24 +3104,8 @@ static int files_for_each_reflog(each_ref_fn fn, void
*cb_data)
return retval;
}
-static int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
-{
- int i, n = refnames->nr;
-
- assert(err);
-
- for (i = 1; i < n; i++)
- if (!strcmp(refnames->items[i - 1].string,
refnames->items[i].string)) {
- strbuf_addf(err,
- "Multiple updates for ref '%s' not
allowed.",
- refnames->items[i].string);
- return 1;
- }
- return 0;
-}
-
static int files_transaction_commit(struct ref_transaction *transaction,
+ struct string_list *affected_refnames,
struct strbuf *err)
{
int ret = 0, i;
@@ -3129,26 +3113,6 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
struct ref_update **updates = transaction->updates;
struct string_list refs_to_delete = STRING_LIST_INIT_NODUP;
struct string_list_item *ref_to_delete;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
-
- assert(err);
-
- if (transaction->state != REF_TRANSACTION_OPEN)
- die("BUG: commit called for transaction that is not open");
-
- if (!n) {
- transaction->state = REF_TRANSACTION_CLOSED;
- return 0;
- }
-
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < n; i++)
- string_list_append(&affected_refnames, updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
/*
* Acquire all locks, verify old values if provided, check
@@ -3167,7 +3131,7 @@ static int files_transaction_commit(struct
ref_transaction *transaction,
update->refname,
((update->flags & REF_HAVE_OLD) ?
update->old_sha1 : NULL),
- &affected_refnames, NULL,
+ affected_refnames, NULL,
update->flags,
&update->type,
err);
@@ -3279,7 +3243,6 @@ cleanup:
if (updates[i]->backend_data)
unlock_ref(updates[i]->backend_data);
string_list_clear(&refs_to_delete, 0);
- string_list_clear(&affected_refnames, 0);
return ret;
}
@@ -3297,27 +3260,18 @@ void files_init_backend(void *data)
}
static int files_initial_transaction_commit(struct ref_transaction
*transaction,
+ struct string_list
*affected_refnames,
struct strbuf *err)
{
int ret = 0, i;
int n = transaction->nr;
struct ref_update **updates = transaction->updates;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
assert(err);
if (transaction->state != REF_TRANSACTION_OPEN)
die("BUG: commit called for transaction that is not open");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < n; i++)
- string_list_append(&affected_refnames, updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
@@ -3330,7 +3284,7 @@ static int files_initial_transaction_commit(struct
ref_transaction *transaction,
* so here we really only check that none of the references
* that we are creating already exists.
*/
- if (for_each_rawref(ref_present, &affected_refnames))
+ if (for_each_rawref(ref_present, affected_refnames))
die("BUG: initial ref transaction called with existing refs");
[PATCH v3 24/44] refs.c: move refname_is_safe to the common code
This function does not contain any backend specific code so we
move it to the common code.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 33 -
refs.c | 24
refs.h | 2 ++
3 files changed, 26 insertions(+), 33 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 9548fb3..346429e 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -196,39 +196,6 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
return dir;
}
-/*
- * Check if a refname is safe.
- * For refs that start with "refs/" we consider it safe as long they do
- * not try to resolve to outside of refs/.
- *
- * For all other refs we only consider them safe iff they only contain
- * upper case characters and '_' (like "HEAD" AND "MERGE_HEAD", and not like
- * "config").
- */
-static int refname_is_safe(const char *refname)
-{
- if (starts_with(refname, "refs/")) {
- char *buf;
- int result;
-
- buf = xmalloc(strlen(refname) + 1);
- /*
-* Does the refname try to escape refs/?
-* For example: refs/foo/../bar is safe but refs/foo/../../bar
-* is not.
-*/
- result = !normalize_path_copy(buf, refname + strlen("refs/"));
- free(buf);
- return result;
- }
- while (*refname) {
- if (!isupper(*refname) && *refname != '_')
- return 0;
- refname++;
- }
- return 1;
-}
-
static struct ref_entry *create_ref_entry(const char *refname,
const unsigned char *sha1, int flag,
int check_name)
diff --git a/refs.c b/refs.c
index e627d74..2515f6e 100644
--- a/refs.c
+++ b/refs.c
@@ -1009,6 +1009,30 @@ int ref_transaction_verify(struct ref_transaction
*transaction,
flags, NULL, err);
}
+int refname_is_safe(const char *refname)
+{
+ if (starts_with(refname, "refs/")) {
+ char *buf;
+ int result;
+
+ buf = xmalloc(strlen(refname) + 1);
+ /*
+* Does the refname try to escape refs/?
+* For example: refs/foo/../bar is safe but refs/foo/../../bar
+* is not.
+*/
+ result = !normalize_path_copy(buf, refname + strlen("refs/"));
+ free(buf);
+ return result;
+ }
+ while (*refname) {
+ if (!isupper(*refname) && *refname != '_')
+ return 0;
+ refname++;
+ }
+ return 1;
+}
+
/* backend functions */
int ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
diff --git a/refs.h b/refs.h
index e2ba725..fc8a748 100644
--- a/refs.h
+++ b/refs.h
@@ -415,6 +415,8 @@ extern int for_each_reflog(each_ref_fn, void *);
*/
extern int check_refname_format(const char *refname, int flags);
+extern int refname_is_safe(const char *refname);
+
extern const char *prettify_refname(const char *refname);
extern char *shorten_unambiguous_ref(const char *refname, int strict);
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 13/44] refs.c: move check_refname_format to the common code
From: Ronnie Sahlberg
This function does not contain any backend specific code so we
move it to the common code.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 109
refs.c | 109
2 files changed, 109 insertions(+), 109 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 045bf09..9c57dcc 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -14,27 +14,6 @@ struct ref_lock {
};
/*
- * How to handle various characters in refnames:
- * 0: An acceptable character for refs
- * 1: End-of-component
- * 2: ., look for a preceding . to reject .. in refs
- * 3: {, look for a preceding @ to reject @{ in refs
- * 4: A bad character: ASCII control characters, and
- *":", "?", "[", "\", "^", "~", SP, or TAB
- * 5: *, reject unless REFNAME_REFSPEC_PATTERN is set
- */
-static unsigned char refname_disposition[256] = {
- 1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
- 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
- 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 2, 1,
- 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 4,
- 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
- 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 4, 0,
- 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
- 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4
-};
-
-/*
* Flag passed to lock_ref_sha1_basic() telling it to tolerate broken
* refs (i.e., because the reference is about to be deleted anyway).
*/
@@ -69,94 +48,6 @@ static unsigned char refname_disposition[256] = {
* value to ref_update::flags
*/
-/*
- * Try to read one refname component from the front of refname.
- * Return the length of the component found, or -1 if the component is
- * not legal. It is legal if it is something reasonable to have under
- * ".git/refs/"; We do not like it if:
- *
- * - any path component of it begins with ".", or
- * - it has double dots "..", or
- * - it has ASCII control characters, or
- * - it has ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
- * - it has "*" anywhere unless REFNAME_REFSPEC_PATTERN is set, or
- * - it ends with a "/", or
- * - it ends with ".lock", or
- * - it contains a "@{" portion
- */
-static int check_refname_component(const char *refname, int *flags)
-{
- const char *cp;
- char last = '\0';
-
- for (cp = refname; ; cp++) {
- int ch = *cp & 255;
- unsigned char disp = refname_disposition[ch];
- switch (disp) {
- case 1:
- goto out;
- case 2:
- if (last == '.')
- return -1; /* Refname contains "..". */
- break;
- case 3:
- if (last == '@')
- return -1; /* Refname contains "@{". */
- break;
- case 4:
- return -1;
- case 5:
- if (!(*flags & REFNAME_REFSPEC_PATTERN))
- return -1; /* refspec can't be a pattern */
-
- /*
-* Unset the pattern flag so that we only accept
-* a single asterisk for one side of refspec.
-*/
- *flags &= ~ REFNAME_REFSPEC_PATTERN;
- break;
- }
- last = ch;
- }
-out:
- if (cp == refname)
- return 0; /* Component has zero length. */
- if (refname[0] == '.')
- return -1; /* Component starts with '.'. */
- if (cp - refname >= LOCK_SUFFIX_LEN &&
- !memcmp(cp - LOCK_SUFFIX_LEN, LOCK_SUFFIX, LOCK_SUFFIX_LEN))
- return -1; /* Refname ends with ".lock". */
- return cp - refname;
-}
-
-int check_refname_format(const char *refname, int flags)
-{
- int component_len, component_count = 0;
-
- if (!strcmp(refname, "@"))
- /* Refname is a single character '@'. */
- return -1;
-
- while (1) {
- /* We are at the start of a path component. */
- component_len = check_refname_component(refname, &flags);
- if (component_len <= 0)
- return -1;
-
- component_count++;
- if (refname[component_len] == '\0')
- break;
- /* Skip to next component. */
- refname += component_len + 1;
- }
-
- if (refname[component_len - 1] == '.')
- return -1; /* Refname ends with '.'. */
- if (!(flags & REFNAME_ALLOW_ONELEVEL) && component_count < 2)
- return -1; /* Refname has only one component. */
- return 0;
-}
-
struct ref_entry;
/*
diff --git a/refs.c b/refs.c
index 039e5c0..e9
[PATCH v3 08/44] refs.c: move the hidden refs functions to the common code
From: Ronnie Sahlberg
Move the hidden refs functions to the refs.c file since these
functions do not contain any backend specific code.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 51 ---
refs.c | 50 ++
2 files changed, 50 insertions(+), 51 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 4e7e05e..d110bf8 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -4169,57 +4169,6 @@ char *shorten_unambiguous_ref(const char *refname, int
strict)
return xstrdup(refname);
}
-static struct string_list *hide_refs;
-
-int parse_hide_refs_config(const char *var, const char *value, const char
*section)
-{
- if (!strcmp("transfer.hiderefs", var) ||
- /* NEEDSWORK: use parse_config_key() once both are merged */
- (starts_with(var, section) && var[strlen(section)] == '.' &&
-!strcmp(var + strlen(section), ".hiderefs"))) {
- char *ref;
- int len;
-
- if (!value)
- return config_error_nonbool(var);
- ref = xstrdup(value);
- len = strlen(ref);
- while (len && ref[len - 1] == '/')
- ref[--len] = '\0';
- if (!hide_refs) {
- hide_refs = xcalloc(1, sizeof(*hide_refs));
- hide_refs->strdup_strings = 1;
- }
- string_list_append(hide_refs, ref);
- }
- return 0;
-}
-
-int ref_is_hidden(const char *refname)
-{
- int i;
-
- if (!hide_refs)
- return 0;
- for (i = hide_refs->nr - 1; i >= 0; i--) {
- const char *match = hide_refs->items[i].string;
- int neg = 0;
- int len;
-
- if (*match == '!') {
- neg = 1;
- match++;
- }
-
- if (!starts_with(refname, match))
- continue;
- len = strlen(match);
- if (!refname[len] || refname[len] == '/')
- return !neg;
- }
- return 0;
-}
-
struct expire_reflog_cb {
unsigned int flags;
reflog_expiry_should_prune_fn *should_prune_fn;
diff --git a/refs.c b/refs.c
index 5a8ef0c..6b2fc39 100644
--- a/refs.c
+++ b/refs.c
@@ -328,3 +328,53 @@ int read_ref_at(const char *refname, unsigned int flags,
unsigned long at_time,
return 1;
}
+
+static struct string_list *hide_refs;
+
+int parse_hide_refs_config(const char *var, const char *value, const char
*section)
+{
+ if (!strcmp("transfer.hiderefs", var) ||
+ /* NEEDSWORK: use parse_config_key() once both are merged */
+ (starts_with(var, section) && var[strlen(section)] == '.' &&
+!strcmp(var + strlen(section), ".hiderefs"))) {
+ char *ref;
+ int len;
+
+ if (!value)
+ return config_error_nonbool(var);
+ ref = xstrdup(value);
+ len = strlen(ref);
+ while (len && ref[len - 1] == '/')
+ ref[--len] = '\0';
+ if (!hide_refs) {
+ hide_refs = xcalloc(1, sizeof(*hide_refs));
+ hide_refs->strdup_strings = 1;
+ }
+ string_list_append(hide_refs, ref);
+ }
+ return 0;
+}
+
+int ref_is_hidden(const char *refname)
+{
+ int i;
+
+ if (!hide_refs)
+ return 0;
+ for (i = hide_refs->nr - 1; i >= 0; i--) {
+ const char *match = hide_refs->items[i].string;
+ int neg = 0;
+ int len;
+ if (*match == '!') {
+ neg = 1;
+ match++;
+ }
+
+ if (!starts_with(refname, match))
+ continue;
+ len = strlen(match);
+ if (!refname[len] || refname[len] == '/')
+ return !neg;
+ }
+ return 0;
+}
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 16/44] refs.c: move ref iterators to the common code
From: Ronnie Sahlberg
Signed-off-by: Ronnie Sahlberg
---
refs-be-files.c | 82 -
refs.c | 81
2 files changed, 81 insertions(+), 82 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index ecf2b33..46fa43c 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1668,23 +1668,6 @@ const char *resolve_ref_unsafe(const char *refname, int
resolve_flags,
return ret;
}
-/* The argument to filter_refs */
-struct ref_filter {
- const char *pattern;
- each_ref_fn *fn;
- void *cb_data;
-};
-
-static int filter_refs(const char *refname, const struct object_id *oid,
- int flags, void *data)
-{
- struct ref_filter *filter = (struct ref_filter *)data;
-
- if (wildmatch(filter->pattern, refname, 0, NULL))
- return 0;
- return filter->fn(refname, oid, flags, filter->cb_data);
-}
-
enum peel_status {
/* object was peeled successfully: */
PEEL_PEELED = 0,
@@ -1950,37 +1933,6 @@ int for_each_ref_in_submodule(const char *submodule,
const char *prefix,
{
return do_for_each_ref(get_ref_cache(submodule), prefix, fn,
strlen(prefix), 0, cb_data);
}
-
-int for_each_tag_ref(each_ref_fn fn, void *cb_data)
-{
- return for_each_ref_in("refs/tags/", fn, cb_data);
-}
-
-int for_each_tag_ref_submodule(const char *submodule, each_ref_fn fn, void
*cb_data)
-{
- return for_each_ref_in_submodule(submodule, "refs/tags/", fn, cb_data);
-}
-
-int for_each_branch_ref(each_ref_fn fn, void *cb_data)
-{
- return for_each_ref_in("refs/heads/", fn, cb_data);
-}
-
-int for_each_branch_ref_submodule(const char *submodule, each_ref_fn fn, void
*cb_data)
-{
- return for_each_ref_in_submodule(submodule, "refs/heads/", fn, cb_data);
-}
-
-int for_each_remote_ref(each_ref_fn fn, void *cb_data)
-{
- return for_each_ref_in("refs/remotes/", fn, cb_data);
-}
-
-int for_each_remote_ref_submodule(const char *submodule, each_ref_fn fn, void
*cb_data)
-{
- return for_each_ref_in_submodule(submodule, "refs/remotes/", fn,
cb_data);
-}
-
int for_each_replace_ref(each_ref_fn fn, void *cb_data)
{
return do_for_each_ref(&ref_cache, git_replace_ref_base, fn,
@@ -2012,40 +1964,6 @@ int for_each_namespaced_ref(each_ref_fn fn, void
*cb_data)
return ret;
}
-int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
- const char *prefix, void *cb_data)
-{
- struct strbuf real_pattern = STRBUF_INIT;
- struct ref_filter filter;
- int ret;
-
- if (!prefix && !starts_with(pattern, "refs/"))
- strbuf_addstr(&real_pattern, "refs/");
- else if (prefix)
- strbuf_addstr(&real_pattern, prefix);
- strbuf_addstr(&real_pattern, pattern);
-
- if (!has_glob_specials(pattern)) {
- /* Append implied '/' '*' if not present. */
- strbuf_complete(&real_pattern, '/');
- /* No need to check for '*', there is none. */
- strbuf_addch(&real_pattern, '*');
- }
-
- filter.pattern = real_pattern.buf;
- filter.fn = fn;
- filter.cb_data = cb_data;
- ret = for_each_ref(filter_refs, &filter);
-
- strbuf_release(&real_pattern);
- return ret;
-}
-
-int for_each_glob_ref(each_ref_fn fn, const char *pattern, void *cb_data)
-{
- return for_each_glob_ref_in(fn, pattern, NULL, cb_data);
-}
-
int for_each_rawref(each_ref_fn fn, void *cb_data)
{
return do_for_each_ref(&ref_cache, "", fn, 0,
diff --git a/refs.c b/refs.c
index 44ee4f4..7714dad 100644
--- a/refs.c
+++ b/refs.c
@@ -781,3 +781,84 @@ const char *prettify_refname(const char *name)
starts_with(name, "refs/remotes/") ? 13 :
0);
}
+
+/* The argument to filter_refs */
+struct ref_filter {
+ const char *pattern;
+ each_ref_fn *fn;
+ void *cb_data;
+};
+
+static int filter_refs(const char *refname, const struct object_id *oid, int
flags,
+ void *data)
+{
+ struct ref_filter *filter = (struct ref_filter *)data;
+ if (wildmatch(filter->pattern, refname, 0, NULL))
+ return 0;
+ return filter->fn(refname, oid, flags, filter->cb_data);
+}
+
+int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
+ const char *prefix, void *cb_data)
+{
+ struct strbuf real_pattern = STRBUF_INIT;
+ struct ref_filter filter;
+ int ret;
+
+ if (!prefix && !starts_with(pattern, "refs/"))
+ strbuf_addstr(&real_pattern, "refs/");
+ else if (prefix)
+ strbuf_addstr(&real_pattern, prefix);
+ strbuf_addstr(&real_pattern, pattern);
+
+ if (!has_glob_specials(pattern)) {
+ /* Append implied '/' '*' if not present. */
+ if (real_pattern.buf[real_pattern.len - 1] != '/')
+
[PATCH v3 12/44] refs.c: move resolve_refdup to common
From: Ronnie Sahlberg
This function can be shared across all refs backends so move it
to the common code.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 7 ---
refs.c | 7 +++
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index bd0dbb5..045bf09 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1777,13 +1777,6 @@ const char *resolve_ref_unsafe(const char *refname, int
resolve_flags,
return ret;
}
-char *resolve_refdup(const char *refname, int resolve_flags,
-unsigned char *sha1, int *flags)
-{
- return xstrdup_or_null(resolve_ref_unsafe(refname, resolve_flags,
- sha1, flags));
-}
-
/* The argument to filter_refs */
struct ref_filter {
const char *pattern;
diff --git a/refs.c b/refs.c
index 242f66d..039e5c0 100644
--- a/refs.c
+++ b/refs.c
@@ -651,3 +651,10 @@ int ref_exists(const char *refname)
unsigned char sha1[20];
return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
}
+
+char *resolve_refdup(const char *refname, int resolve_flags,
+unsigned char *sha1, int *flags)
+{
+ return xstrdup_or_null(resolve_ref_unsafe(refname, resolve_flags,
+ sha1, flags));
+}
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 04/44] refs.c: add a new refs.c file to hold all common refs code
From: Ronnie Sahlberg Create a new refs.c file that will be used to hold all the refs code that is backend agnostic and will be shared across all backends. Signed-off-by: Ronnie Sahlberg --- Makefile | 1 + refs.c | 3 +++ 2 files changed, 4 insertions(+) create mode 100644 refs.c diff --git a/Makefile b/Makefile index 19036de..43ceab0 100644 --- a/Makefile +++ b/Makefile @@ -767,6 +767,7 @@ LIB_OBJS += reachable.o LIB_OBJS += read-cache.o LIB_OBJS += reflog-walk.o LIB_OBJS += refs-be-files.o +LIB_OBJS += refs.o LIB_OBJS += ref-filter.o LIB_OBJS += remote.o LIB_OBJS += replace_object.o diff --git a/refs.c b/refs.c new file mode 100644 index 000..77492ff --- /dev/null +++ b/refs.c @@ -0,0 +1,3 @@ +/* + * Common refs code for all backends. + */ -- 2.4.2.644.g97b850b-twtrsrc -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 15/44] refs.c: move prettify_refname to the common code
From: Ronnie Sahlberg
Signed-off-by: Ronnie Sahlberg
---
refs-be-files.c | 9 -
refs.c | 9 +
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 1214d9e..ecf2b33 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2052,15 +2052,6 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)
DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
}
-const char *prettify_refname(const char *name)
-{
- return name + (
- starts_with(name, "refs/heads/") ? 11 :
- starts_with(name, "refs/tags/") ? 10 :
- starts_with(name, "refs/remotes/") ? 13 :
- 0);
-}
-
static void unlock_ref(struct ref_lock *lock)
{
/* Do not free lock->lk -- atexit() still looks at them */
diff --git a/refs.c b/refs.c
index bc8750c..44ee4f4 100644
--- a/refs.c
+++ b/refs.c
@@ -772,3 +772,12 @@ int is_branch(const char *refname)
{
return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
}
+
+const char *prettify_refname(const char *name)
+{
+ return name + (
+ starts_with(name, "refs/heads/") ? 11 :
+ starts_with(name, "refs/tags/") ? 10 :
+ starts_with(name, "refs/remotes/") ? 13 :
+ 0);
+}
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 17/44] refs.c: move head_ref_namespaced to the common code
From: Ronnie Sahlberg
Signed-off-by: Ronnie Sahlberg
---
refs-be-files.c | 15 ---
refs.c | 15 +++
2 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index 46fa43c..34ed220 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1939,21 +1939,6 @@ int for_each_replace_ref(each_ref_fn fn, void *cb_data)
strlen(git_replace_ref_base), 0, cb_data);
}
-int head_ref_namespaced(each_ref_fn fn, void *cb_data)
-{
- struct strbuf buf = STRBUF_INIT;
- int ret = 0;
- struct object_id oid;
- int flag;
-
- strbuf_addf(&buf, "%sHEAD", get_git_namespace());
- if (!read_ref_full(buf.buf, RESOLVE_REF_READING, oid.hash, &flag))
- ret = fn(buf.buf, &oid, flag, cb_data);
- strbuf_release(&buf);
-
- return ret;
-}
-
int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
{
struct strbuf buf = STRBUF_INIT;
diff --git a/refs.c b/refs.c
index 7714dad..0f4e19a 100644
--- a/refs.c
+++ b/refs.c
@@ -862,3 +862,18 @@ int for_each_remote_ref_submodule(const char *submodule,
each_ref_fn fn, void *c
{
return for_each_ref_in_submodule(submodule, "refs/remotes/", fn,
cb_data);
}
+
+int head_ref_namespaced(each_ref_fn fn, void *cb_data)
+{
+ struct strbuf buf = STRBUF_INIT;
+ int ret = 0;
+ struct object_id oid;
+ int flag;
+
+ strbuf_addf(&buf, "%sHEAD", get_git_namespace());
+ if (!read_ref_full(buf.buf, RESOLVE_REF_READING, oid.hash, &flag))
+ ret = fn(buf.buf, &oid, flag, cb_data);
+ strbuf_release(&buf);
+
+ return ret;
+}
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 09/44] refs.c: move dwim and friend functions to the common refs code
From: Ronnie Sahlberg
These functions do not contain any backend specific code so we move
them to the common code and share across all backends.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs-be-files.c | 203
refs.c | 203
2 files changed, 203 insertions(+), 203 deletions(-)
diff --git a/refs-be-files.c b/refs-be-files.c
index d110bf8..37772a7 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2247,30 +2247,6 @@ const char *prettify_refname(const char *name)
0);
}
-static const char *ref_rev_parse_rules[] = {
- "%.*s",
- "refs/%.*s",
- "refs/tags/%.*s",
- "refs/heads/%.*s",
- "refs/remotes/%.*s",
- "refs/remotes/%.*s/HEAD",
- NULL
-};
-
-int refname_match(const char *abbrev_name, const char *full_name)
-{
- const char **p;
- const int abbrev_name_len = strlen(abbrev_name);
-
- for (p = ref_rev_parse_rules; *p; p++) {
- if (!strcmp(full_name, mkpath(*p, abbrev_name_len,
abbrev_name))) {
- return 1;
- }
- }
-
- return 0;
-}
-
static void unlock_ref(struct ref_lock *lock)
{
/* Do not free lock->lk -- atexit() still looks at them */
@@ -2323,92 +2299,6 @@ static int remove_empty_directories(struct strbuf *path)
}
/*
- * *string and *len will only be substituted, and *string returned (for
- * later free()ing) if the string passed in is a magic short-hand form
- * to name a branch.
- */
-static char *substitute_branch_name(const char **string, int *len)
-{
- struct strbuf buf = STRBUF_INIT;
- int ret = interpret_branch_name(*string, *len, &buf);
-
- if (ret == *len) {
- size_t size;
- *string = strbuf_detach(&buf, &size);
- *len = size;
- return (char *)*string;
- }
-
- return NULL;
-}
-
-int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
-{
- char *last_branch = substitute_branch_name(&str, &len);
- const char **p, *r;
- int refs_found = 0;
-
- *ref = NULL;
- for (p = ref_rev_parse_rules; *p; p++) {
- char fullref[PATH_MAX];
- unsigned char sha1_from_ref[20];
- unsigned char *this_result;
- int flag;
-
- this_result = refs_found ? sha1_from_ref : sha1;
- mksnpath(fullref, sizeof(fullref), *p, len, str);
- r = resolve_ref_unsafe(fullref, RESOLVE_REF_READING,
- this_result, &flag);
- if (r) {
- if (!refs_found++)
- *ref = xstrdup(r);
- if (!warn_ambiguous_refs)
- break;
- } else if ((flag & REF_ISSYMREF) && strcmp(fullref, "HEAD")) {
- warning("ignoring dangling symref %s.", fullref);
- } else if ((flag & REF_ISBROKEN) && strchr(fullref, '/')) {
- warning("ignoring broken ref %s.", fullref);
- }
- }
- free(last_branch);
- return refs_found;
-}
-
-int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
-{
- char *last_branch = substitute_branch_name(&str, &len);
- const char **p;
- int logs_found = 0;
-
- *log = NULL;
- for (p = ref_rev_parse_rules; *p; p++) {
- unsigned char hash[20];
- char path[PATH_MAX];
- const char *ref, *it;
-
- mksnpath(path, sizeof(path), *p, len, str);
- ref = resolve_ref_unsafe(path, RESOLVE_REF_READING,
-hash, NULL);
- if (!ref)
- continue;
- if (reflog_exists(path))
- it = path;
- else if (strcmp(ref, path) && reflog_exists(ref))
- it = ref;
- else
- continue;
- if (!logs_found++) {
- *log = xstrdup(it);
- hashcpy(sha1, hash);
- }
- if (!warn_ambiguous_refs)
- break;
- }
- free(last_branch);
- return logs_found;
-}
-
-/*
* Locks a ref returning the lock on success and NULL on failure.
* On failure errno is set to something meaningful.
*/
@@ -4076,99 +3966,6 @@ cleanup:
return ret;
}
-char *shorten_unambiguous_ref(const char *refname, int strict)
-{
- int i;
- static char **scanf_fmts;
- static int nr_rules;
- char *short_name;
-
- if (!nr_rules) {
- /*
-* Pre-generate scanf formats from ref_rev_parse_rules[].
-* Generate a format suitable for scanf from a
-* ref_rev_par
[PATCH v3 00/44] alternate refs backends
This version of the patch set includes the following changes: 1. Numerous formatting and style updates. Thanks for Michael Haggerty to pointing these out; remaining errors are of course my own. 2. Fixed warnings. Thanks to Michael Haggerty, Michael Blume, and Torsten Bögershausen for pointing these out. 3. I reworded some commit messages. 4. Made tests work with LMDB turned off. Thanks to Dennis Kaarsemaker. 5. Added Jeff King's core.repositoryformatversion patch. 6. After discussion with Michael Haggerty and Junio, redesigned the backend structure. Now the generic code builds ref transactions, and backends apply these transactions. The ref transaction backend patch changed as a result of this, and I added three more patches: refs: make lock generic (to allow ref backends to use a generic ref_update struct to handle their own locking) refs: move duplicate check to common code (to simplify the implementation of ref backends, and to simplify the next patch) refs: always handle non-normal refs in files backend (cross-backend transactions) David Turner (24): refs: make repack_without_refs and is_branch public refs: move transaction functions into common code refs-be-files.c: add method for for_each_reftype_... refs-be-files.c: add do_for_each_per_worktree_ref refs.c: move refname_is_safe to the common code refs.h: document make refname_is_safe and add it to header refs.c: move copy_msg to the common code refs.c: move peel_object to the common code refs.c: move should_autocreate_reflog to common code refs.c: add ref backend init function refs.c: add methods for reflog refs.c: add method for initial ref transaction commit initdb: move safe_create_dir into common code refs.c: add method for initializing refs db refs-be-files.c: add method to rename refs refs: make lock generic refs: make files_log_ref_write functions public refs: break out a ref conflict check refs: move duplicate check to common code refs: always handle non-normal refs in files backend refs: allow ref backend to be set for clone refs: add register_refs_backend refs: add LMDB refs backend refs: tests for db backend Jeff King (1): introduce "extensions" form of core.repositoryformatversion Ronnie Sahlberg (19): refs.c: create a public version of verify_refname_available refs-be-files.c: rename refs to refs-be-files refs.c: add a new refs.c file to hold all common refs code refs.c: move update_ref to refs.c refs.c: move delete_ref and delete_refs to the common code refs.c: move read_ref_at to the common refs file refs.c: move the hidden refs functions to the common code refs.c: move dwim and friend functions to the common refs code refs.c: move warn_if_dangling_symref* to the common code refs.c: move read_ref, read_ref_full and ref_exists to the common code refs.c: move resolve_refdup to common refs.c: move check_refname_format to the common code refs.c: move is_branch to the common code refs.c: move prettify_refname to the common code refs.c: move ref iterators to the common code refs.c: move head_ref_namespaced to the common code refs: add a backend method structure with transaction functions refs-be-files.c: add methods for misc ref operations refs-be-files.c: add methods for the ref iterators .gitignore |1 + Documentation/config.txt |7 + Documentation/git-clone.txt|4 + Documentation/git-init-db.txt |2 +- Documentation/git-init.txt |6 + Documentation/technical/refs-be-lmdb.txt | 50 + Documentation/technical/repository-version.txt | 86 + Makefile | 13 + builtin/clone.c| 27 +- builtin/init-db.c | 47 +- builtin/submodule--helper.c|5 +- cache.h| 19 + config.c | 34 + configure.ac | 33 + contrib/workdir/git-new-workdir|3 + environment.c |1 + path.c | 12 + refs-be-files.c| 3521 refs-be-lmdb.c | 1997 + refs.c | 5322 +--- refs.h | 306 +- setup.c| 69 +- t/t1302-repo-version.sh| 38 + t/t1460-refs-be-db.sh | 1109 + t/t1470-refs-be-db-reflog.sh | 359 ++ t/test-lib.sh |1 + test-refs-be-lmdb.c| 68 + 27 files changed, 8849 insertions(+), 4291 deletions(-) create mode 100644 Document
[PATCH v3 02/44] refs: make repack_without_refs and is_branch public
is_branch was already non-static, but this patch declares it in the
header.
Signed-off-by: Ronnie Sahlberg
Signed-off-by: David Turner
---
refs.c | 9 +
refs.h | 13 +
2 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/refs.c b/refs.c
index fe71ea0..d0dfdfc 100644
--- a/refs.c
+++ b/refs.c
@@ -2814,14 +2814,7 @@ int pack_refs(unsigned int flags)
return 0;
}
-/*
- * Rewrite the packed-refs file, omitting any refs listed in
- * 'refnames'. On error, leave packed-refs unchanged, write an error
- * message to 'err', and return a nonzero value.
- *
- * The refs in 'refnames' needn't be sorted. `err` must not be NULL.
- */
-static int repack_without_refs(struct string_list *refnames, struct strbuf
*err)
+int repack_without_refs(struct string_list *refnames, struct strbuf *err)
{
struct ref_dir *packed;
struct string_list_item *refname;
diff --git a/refs.h b/refs.h
index 79ea220..729bc3c 100644
--- a/refs.h
+++ b/refs.h
@@ -218,6 +218,19 @@ extern void warn_dangling_symrefs(FILE *fp, const char
*msg_fmt, const struct st
int pack_refs(unsigned int flags);
/*
+ * Rewrite the packed-refs file, omitting any refs listed in
+ * 'refnames'. On error, packed-refs will be unchanged, the return
+ * value is nonzero, and a message about the error is written to the
+ * 'err' strbuf.
+ *
+ * The refs in 'refnames' needn't be sorted. `err` must not be NULL.
+ */
+extern int repack_without_refs(struct string_list *refnames,
+ struct strbuf *err);
+
+extern int is_branch(const char *refname);
+
+/*
* Return true iff a reference named refname could be created without
* conflicting with the name of an existing reference. If
* skip is non-NULL, ignore potential conflicts with refs in skip
--
2.4.2.644.g97b850b-twtrsrc
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] submodule: Test a shallow branch
Instead of tracking the submodule at the specified branch, using a shallow clone, we get the following error: fatal: Cannot update paths and switch to branch 'anotherbranch' at the same time. Did you intend to checkout 'origin/anotherbranch' which can not be resolved as commit? Unable to checkout submodule 'submodule' Noticed-by: Carlos Pita Signed-off-by: Stefan Beller --- Hi Carlos, I can reproduce your issue and have a patch which we can add to the testing suite to document this faulty behavior. Thanks, Stefan t/t7400-submodule-basic.sh | 20 +++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index 540771c..cee74cb 100755 --- a/t/t7400-submodule-basic.sh +++ b/t/t7400-submodule-basic.sh @@ -846,7 +846,8 @@ test_expect_success 'submodule add with an existing name fails unless forced' ' test_expect_success 'set up a second submodule' ' git submodule add ./init2 example2 && - git commit -m "submodule example2 added" + git commit -m "submodule example2 added" && + git checkout -b anotherbranch ' test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' ' @@ -987,6 +988,7 @@ test_expect_success 'submodule with UTF-8 name' ' test_expect_success 'submodule add clone shallow submodule' ' mkdir super && + test_when_finished "rm -rf super" && pwd=$(pwd) && ( cd super && @@ -999,5 +1001,21 @@ test_expect_success 'submodule add clone shallow submodule' ' ) ' +test_expect_failure 'submodule add a shallow branch' ' + mkdir super && + test_when_finished "rm -rf super" && + pwd=$(pwd) && + ( + cd super && + git init && + git submodule add --depth=1 --branch anotherbranch file://"$pwd"/example2 submodule && + ( + cd submodule && + test 1 = $(git log --oneline | wc -l) + ) + ) +' + + test_done -- 2.5.0.268.g453a26a -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] pull --rebase: reinstate helpful message on abort
Johannes Schindelin writes: > Hi Junio, > > On 2015-10-09 20:36, Junio C Hamano wrote: >> Johannes Schindelin writes: >> >>> When calling `git pull --rebase`, things can go wrong. In such a case, >>> we want to tell the user about the most common ways out of this fix: >>> ... >>> builtin/am.c | 1 + >>> 1 file changed, 1 insertion(+) >> >> It is strange to see a patch to am that does not talk anything about >> it, though. And looking at the codepath, the issue does not have >> much to do with "pull --rebase". It doesn't even have much to do >> with "rebase". This is purely about "am -3" fallback codepath. > > I made it a habit of describing the big picture in commit messages, > including the original motivation for the patch. Naturally, it is > purely an implementation detail that the bug displayed by `git pull > --rebase` is fixed by modifying `am.c`. Yup, but that is "I happened to notice that bug first in a command that uses another command that happens to use this buggy one". That may be interesting in the "peeing in the snow" sense, but not very interesting in the big picture of ensuring the health of the entire codebase. The "common ways out of this" helpful message is not even coming from "pull --rebase" or even "rebase" in the first place. What you are reinstating helpful message on abort is "am -3". "This fixes am -3, hence incidentally fixes rebase and hence fixes pull --rebase, too" would be the most useful way to describe this change. The initial report being about "pull --rebase" is of much lessor importance, I would think. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] Reinstate the helpful message when `git pull --rebase` fails
Johannes Schindelin writes: >> I think the most sensible regression fix as the first step at this >> point is to call it as a separate process, just like the code calls >> "apply" as a separate process for each patch. Optimization can come >> later when it is shown that it matters---we need to regain >> correctness first. > > I fear that you might underestimate the finality of this "first > step". If you reintroduce that separate process, not only is it a > performance regression, but could we really realistically expect any > further steps to happen after that? I do not think so. > ... > For the above reasons, I respectfully remain convinced that > reintroducing the separate process would be a mistake. I am not saying we should forever do run_command() going forward. But I do not want to keep the direct call to merge_recursive() in 'maint'. The topic was supposed to be "rewrite in C". I do not recall (and do not feel the need to read "git log" output to find out) exactly how the series progressed, but a logical progression would have been to run merge-recursive via run_command() like I showed in the quick-fix in an early part of the series, followed by a patch to turn it into a direct call to merge_recursive() as an optimization change. And the latter step turned out to be a regression caused by a premature optimization. If something introduces a regression, it gets reverted. As the scripted version certainly did not make an internal call, we should just run_command(). And that is what we want to have in the stable version people use every day. The only thing I am saying is that the change to make a direct call should come on top of the run_command() version with its own justification as an optimization patch. Going that route may require you to redo your patch, measure performance improvements, ensure there is no unintended fallout in other callers and longer term maintainability of the codebaths involved, write a good log message, etc. And such a fix to merge_recursive() needs to be cooked sufficiently in 'pu' and 'next'. And I view all that as a good thing. I really hate to see that this premature optimization to come back and bite us again---we didn't see it while reviewing because "builtin-am: implement --3way" was done in a single step with premature optimization from the beginning. Now, there are many reasons why the "first step" might turn out to be the permanent optimal solution. I did an unscientific experiment to rebase each of the 25 topics that are cooking in 'next' on top of 'master'. Only 3 of them will fall back to the three-way merge machinery. One possible reason why the "first step" could stay a good-enough solution is that people would not care and/or notice, because it is not like you are paying unnecessary cost to spawn merge-recursive for each and every change. It only kicks in when the patch does not apply. Then I randomly picked one (jc/merge-drop-old-syntax) of the three topics that does fall back, made it into a patch and ran "am -3" on top of 'master' with and without the "first step". The numbers from 5 runs of each look like this: real0m0.109s real0m0.109s user0m0.080s user0m0.079s sys 0m0.034s sys 0m0.035s real0m0.109s real0m0.105s user0m0.095s user0m0.087s sys 0m0.018s sys 0m0.022s real0m0.109s real0m0.110s user0m0.075s user0m0.086s sys 0m0.038s sys 0m0.028s real0m0.107s real0m0.108s user0m0.083s user0m0.075s sys 0m0.029s sys 0m0.038s real0m0.106s real0m0.108s user0m0.086s user0m0.090s sys 0m0.025s sys 0m0.023s I am curious to see a similar number on platforms with slower run_command(). From the above numbers alone, I cannot even see which ones are with run_command(), even though I know the numbers on the right hand side column were taken with run_command() and the numbers on the left hand side column were taken with internal call. Another possible reason why the "first step" could stay a good-enough solution is that merge_recursive() in itself is a heavy-weight operation that the cost of spawning a process is not even felt [*1*]. After all, it's not like we are talking about spawning the cost of "update-ref HEAD" dominating the cost of the actual operation. By the way, in order to make sure that I am running the correct binary, I did "strace -f -e execve" on "am -3". "mailsplit" and "mailinfo", both of which are a lot more likely to be affected by the cost of spawning because they are mostly dumb and straight text-to-text filters, are spawned via run_comm
Re: How to rebase when some commit hashes are in some commit messages
Francois-Xavier Le Bail writes: > Hello, > > [I try some search engines without success, perhaps I have missed something]. > > For example, if I rebase the following commits, I would want that if > the commit hash 222... become 777..., > the message > "Update test output for " > become > "Update test output for 777..." > > Is it possible currently? And if yes how? AFAIK, it's not possible other than by editing the message by hand. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
larsxschnei...@gmail.com writes:
> --- /dev/null
> +++ b/.travis.yml
> @@ -0,0 +1,46 @@
> +language: c
> +
> +os:
> + - linux
> + - osx
> +
> +compiler:
> + - clang
> + - gcc
> +
> +before_install:
> + - >
> +export GIT_TEST_OPTS=" --quiet";
> +case "${TRAVIS_OS_NAME:-linux}" in
> +linux)
> + wget -q https://package.perforce.com/perforce.pubkey -O - \
> +| sudo apt-key add -
> + echo 'deb http://package.perforce.com/apt/ubuntu precise release' \
> +| sudo tee -a /etc/apt/sources.list
> + wget -q https://packagecloud.io/gpg.key -O - | sudo apt-key add -
> + echo 'deb https://packagecloud.io/github/git-lfs/debian/ jessie main' \
> +| sudo tee -a /etc/apt/sources.list
> + sudo apt-get update -qq
> + sudo apt-get install -y apt-transport-https
> + sudo apt-get install perforce-server git-lfs
Sorry if this has been discussed already, but do you really need these
"sudo" calls?
They trigger builds on the legacy Travis CI infrastructure:
http://docs.travis-ci.com/user/migrating-from-legacy/?utm_source=legacy-notice&utm_medium=banner&utm_campaign=legacy-upgrade
No big deal, but getting rid of sudo would be cool, and documenting why
it can't easily be done in commit message and/or comments would be nice
otherwise.
Thanks for working on that!
--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
Lars Schneider writes:
> {Linux | OSX} * {gcc | clang} * {Default, NO_PTHREAD, NO_CURL, NO_OPENSSL,
> NO_MMAP, NO_IPV6, NO_PERL}
Another option would be to have a single "NO_*" build, that would
activate all NO_PTHREAD, NO_CURL, NO_OPENSSL, NO_MMAP, NO_IPV6, NO_PERL
at the same time. We'd miss issues with perl but no pthread or so, but
this should catch most issues.
--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
How to rebase when some commit hashes are in some commit messages
Hello, [I try some search engines without success, perhaps I have missed something]. For example, if I rebase the following commits, I would want that if the commit hash 222... become 777..., the message "Update test output for " become "Update test output for 777..." Is it possible currently? And if yes how? Greetings, Francois-Xavier --- commit Author: First Last Date: Fri Jul 3 17:14:58 2015 -0700 Fix 5 xxx xxx xxx xxx commit Author: First Last Date: Fri Jul 3 16:58:58 2015 -0700 Update test output for commit Author: First Last Date: Fri Jul 3 17:50:27 2015 -0700 Fix 4 commit Author: First Last Date: Fri Jul 3 15:01:36 2015 -0700 Fix 3 xxx xxx xxx xxx commit Author: First Last Date: Fri Jul 3 11:20:28 2015 -0700 Fix 2 xxx xxx xxx xxx commit Author: First Last Date: Fri Jul 3 09:15:59 2015 -0700 Fix 1 xxx xxx xxx xxx --- -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
On Mon, Oct 12, 2015 at 9:43 PM, Lars Schneider wrote: >> Reading through Travis' docs [3] again, "before_script" is documented >> to "return a non-zero exit code, the build is errored and stops >> immediately", while "script" is documented as "returns a non-zero exit >> code, the build is failed, but continues to run before being marked as >> failed". As it does not make much sense to continue the build or even >> start testing if the build failed, maybe it's indeed best to do: >> >> before_script: make configure && ./configure && make >> >> script: make --quiet test > > Ok, then I will make it so :-) Also, this has the added benefit of being quickly able to see how much time building vs testing took. As these two are the big blocks, we'd want to optimize both steps for time, and it's easier to see what we gain e.g. from a possible build-time improvement if these steps are listed individually in the Travis log. -- Sebastian Schuberth -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Add submodule specifying depth and branch at the same time won't work
For example, I can do: git clone --depth 1 --branch devel g...@gitlab.com:memeplex/bash.git But if I try: git submodule add --depth 1 -b devel g...@gitlab.com:memeplex/bash.git I get: fatal: Cannot update paths and switch to branch 'devel' at the same time. Did you intend to checkout 'origin/devel' which can not be resolved as commit? Unable to checkout submodule 'src/aur/bash' I think the devel branch is left outside the update because depth is just 1, but then the same could be said for the clone, which just works. The request should be interpreted as depth 1 inside devel branch. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
On Mon, Oct 12, 2015 at 9:40 PM, Lars Schneider
wrote:
>> This is a slightly related tangent, but we saw a few build issues
>> reported recently on customized configurations like NO_PTHREAD. If
>> we are to start using automated tests, I wonder if we want to build
>> (and optionally test) with various combinations of the customization
>> options (e.g. NO_CURL, NO_OPENSSL, NO_MMAP, NO_IPV6, NO_PERL etc.)
> This easy to do. However, the more we environment settings we define the
> longer the build runs. I created a test matrix that runs the following
> combinations:
> {Linux | OSX} * {gcc | clang} * {Default, NO_PTHREAD, NO_CURL, NO_OPENSSL,
> NO_MMAP, NO_IPV6, NO_PERL}
>
> These result in 28 (= 2*2*7) combinations. I created a build without the
> "Default" environment (=24 combination) here:
> https://travis-ci.org/larsxschneider/git/builds/84978673
>
> Should I add them them to the Travis CI patch?
I'd say it depends on how long such a matrix build would take in
average. Personally, I'd prefer to not wait more than, say, 30 minutes
for testing a PR. From your Travis build history it looks to me as if
we already exceed that limit many fold, so I'd prefer to not use
matrix builds unless we find ways to speed up the build in general,
for example by using ccache [1].
[1] http://docs.travis-ci.com/user/caching/#ccache-cache
--
Sebastian Schuberth
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
On 12 Oct 2015, at 12:37, Sebastian Schuberth wrote: > On Mon, Oct 12, 2015 at 7:12 PM, Lars Schneider > wrote: > +install: make configure && ./configure + +before_script: make + +script: make --quiet test >>> >>> Semantically, it does not seem correct to me that configuarion goes to the >>> install step. As "make test" will build git anyway, I'd instead propose to >>> get rid of "install" and just say: >>> >>> before_script: make configure && ./configure >>> >>> script: make --quiet test >> >> I understand your point. I did this to make the "make" logs easily >> accessible (no option "--quite"). By default Travis CI automatically >> collapses the logs from all stages prior to the "script" stage. You can >> uncollapse these logs by clicking on the little triangle on the left border >> of the log. Therefore the "make" logs are available without noise. > > To make this more clear, I guess what you're referring to is the > visual difference between [1] and [2], correct? correct! > >> Do you see value in "make" logs? >> >> If yes then we could also do: >> before_script: make configure && ./configure && make > > Reading through Travis' docs [3] again, "before_script" is documented > to "return a non-zero exit code, the build is errored and stops > immediately", while "script" is documented as "returns a non-zero exit > code, the build is failed, but continues to run before being marked as > failed". As it does not make much sense to continue the build or even > start testing if the build failed, maybe it's indeed best to do: > > before_script: make configure && ./configure && make > > script: make --quiet test Ok, then I will make it so :-) Thanks, Lars-- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
On 12 Oct 2015, at 09:02, Junio C Hamano wrote:
> Sebastian Schuberth writes:
>
>> Semantically, it does not seem correct to me that configuarion goes to
>> the install step. As "make test" will build git anyway, I'd instead
>> propose to get rid of "install" and just say:
>>
>> before_script: make configure && ./configure
>>
>> script: make --quiet test
>
> Very good point. Do we even need to do anything in the "install"
> target? We aim to be able to testable without any installed Git,
> and not running "make install" at all, ever, would be one way to
> make sure that works.
The Travis CI "install" stage is independent of "make install". AFAIK you can
use the Travis lifecycle stages pretty much as you want. However, I agree we
should not use the "install" stage to avoid confusion.
> This is a slightly related tangent, but we saw a few build issues
> reported recently on customized configurations like NO_PTHREAD. If
> we are to start using automated tests, I wonder if we want to build
> (and optionally test) with various combinations of the customization
> options (e.g. NO_CURL, NO_OPENSSL, NO_MMAP, NO_IPV6, NO_PERL etc.)
This easy to do. However, the more we environment settings we define the longer
the build runs. I created a test matrix that runs the following combinations:
{Linux | OSX} * {gcc | clang} * {Default, NO_PTHREAD, NO_CURL, NO_OPENSSL,
NO_MMAP, NO_IPV6, NO_PERL}
These result in 28 (= 2*2*7) combinations. I created a build without the
"Default" environment (=24 combination) here:
https://travis-ci.org/larsxschneider/git/builds/84978673
Should I add them them to the Travis CI patch?
Thanks,
Lars
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
On Mon, Oct 12, 2015 at 7:12 PM, Lars Schneider wrote: >>> +install: make configure && ./configure >>> + >>> +before_script: make >>> + >>> +script: make --quiet test >> >> Semantically, it does not seem correct to me that configuarion goes to the >> install step. As "make test" will build git anyway, I'd instead propose to >> get rid of "install" and just say: >> >> before_script: make configure && ./configure >> >> script: make --quiet test > > I understand your point. I did this to make the "make" logs easily accessible > (no option "--quite"). By default Travis CI automatically collapses the logs > from all stages prior to the "script" stage. You can uncollapse these logs by > clicking on the little triangle on the left border of the log. Therefore the > "make" logs are available without noise. To make this more clear, I guess what you're referring to is the visual difference between [1] and [2], correct? > Do you see value in "make" logs? > > If yes then we could also do: > before_script: make configure && ./configure && make Reading through Travis' docs [3] again, "before_script" is documented to "return a non-zero exit code, the build is errored and stops immediately", while "script" is documented as "returns a non-zero exit code, the build is failed, but continues to run before being marked as failed". As it does not make much sense to continue the build or even start testing if the build failed, maybe it's indeed best to do: before_script: make configure && ./configure && make script: make --quiet test [1] https://travis-ci.org/larsxschneider/git/jobs/84805733 [2] https://travis-ci.org/larsxschneider/git/jobs/84955658 [3] http://docs.travis-ci.com/user/customizing-the-build/ -- Sebastian Schuberth -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCHv7 0/8] fetch submodules in parallel
Thanks; will replace. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] submodule-config: Untangle logic in parse_config
Stefan Beller writes:
> This improves readability of the parse_config logic by making it more concise.
This does make it shorter, but "improve readability" is highly
subjective and "untangle logic" is content-free without explaining
what aspect of the logic is being untangled into what other shape.
> @@ -257,78 +257,62 @@ static int parse_config(const char *var, const char
> *value, void *data)
> if (!name_and_item_from_var(var, &name, &item))
> return 0;
>
> - submodule = lookup_or_create_by_name(me->cache, me->gitmodules_sha1,
> - name.buf);
> + submodule = lookup_or_create_by_name(me->cache,
> + me->gitmodules_sha1,
> + name.buf);
Just a formatting change, which does make it easier to follow, but
does not untangle the logic.
> ...
> } else if (!strcmp(item.buf, "ignore")) {
> - struct strbuf ignore = STRBUF_INIT;
> - if (!me->overwrite && submodule->ignore != NULL) {
> + if (!value)
> + ret = config_error_nonbool(var);
> + else if (!me->overwrite && submodule->ignore != NULL)
> warn_multiple_config(me->commit_sha1, submodule->name,
> "ignore");
> - goto release_return;
> - }
> - if (!value) {
> - ret = config_error_nonbool(var);
> - goto release_return;
> - }
This is not a faithful conversion, in that we used to complain and
abort when seeing multiple values with or without value but now we
complain about malformed boolean first. I do not think the
difference matters, but it is worth noting in the log, as it is a
clean-up that makes the order of checks consistent between ignore
and url, if I am reading the patch correctly.
> - if (strcmp(value, "untracked") && strcmp(value, "dirty") &&
> - strcmp(value, "all") && strcmp(value, "none")) {
> + else if (strcmp(value, "untracked") &&
> + strcmp(value, "dirty") &&
> + strcmp(value, "all") &&
> + strcmp(value, "none"))
> warning("Invalid parameter '%s' for config option "
> "'submodule.%s.ignore'", value, var);
> - goto release_return;
> + else {
> + free((void *) submodule->ignore);
> + submodule->ignore = xstrdup(value);
> }
> -
> - free((void *) submodule->ignore);
> - strbuf_addstr(&ignore, value);
> - submodule->ignore = strbuf_detach(&ignore, NULL);
> } else if (!strcmp(item.buf, "url")) {
> - struct strbuf url = STRBUF_INIT;
> if (!value) {
> ret = config_error_nonbool(var);
> - goto release_return;
> - }
> - if (!me->overwrite && submodule->url != NULL) {
> + } else if (!me->overwrite && submodule->url != NULL) {
> warn_multiple_config(me->commit_sha1, submodule->name,
> "url");
> - goto release_return;
> + } else {
> + free((void *) submodule->url);
> + submodule->url = xstrdup(value);
> }
> -
> - free((void *) submodule->url);
> - strbuf_addstr(&url, value);
> - submodule->url = strbuf_detach(&url, NULL);
> }
>
> -release_return:
So overall, I think there is not much "untangled", but its primary
effect is that a forward "goto" to the clean-up is removed by making
each component in if/else if/... cascade more independently complete.
Generally, a large piece of code is _easier_ to read with forward
"goto"s that jump to the shared clean-up code, as they serve as
visual cues that tell the reader "you can stop reading here and
ignore the remainder of this if/else if/... cascade". But this
function is no too large and removing them does not make the
result harder to read, so I am not opposed to this change. If each
individual component in if/else if/... cascade grows too large in
the future, it can easily become its own helper function.
So the patch looks OK to me, except for the "hmm, the order of
checks are made uniform without being documented?" comment above.
Thanks.
> strbuf_release(&name);
> strbuf_release(&item);
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/10] port branch.c to use ref-filter's printing options
Junio C Hamano writes: > If you design a new infrastructure to help refactoring early > (i.e. before adding many copies of code that need to be cleaned up > later), it would make the work of reviewing of the design of the > helper and refactoring using that helper smaller, not larger. But most of the code concerned is already reviewed. The first instances of the pattern to refactor is already in next. With a real time machine, we could go back in past, refactor and then have cleaner series, but with Git as our only tool we can't ;-). The current series will just add one more instance of sub-optimal code, it isn't hard to review. Inserting new code before them would make the interdiff far bigger. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/10] port branch.c to use ref-filter's printing options
Matthieu Moy writes: > Karthik Nayak writes: > >> On Mon, Oct 12, 2015 at 6:06 AM, Junio C Hamano wrote: >>> Karthik Nayak writes: >>> On Fri, Oct 9, 2015 at 11:59 PM, Junio C Hamano wrote: ... Also does it make sense to integrate these changes here? Or would you like to have another series on this? >>> >>> I do not think you would want to ask that question, as my answer >>> would most likely be "The most preferable would be a series to clean >>> up the existing codepath that deals with %(align) first, on top of >>> which everything in flight that is not yet in 'next' is rebased." >> >> Ah, but I might take a while to get there, So I'd rather push code which >> is almost ready and work on that slowly, if that's ok? > > That's OK to me. The "most preferable way" above would lead to a cleaner > history, but also more work for you and for me as a reviewer. I do not think the cleanliness of the resulting history is of prime concern. At least, that was not where my preference came from. If you design a new infrastructure to help refactoring early (i.e. before adding many copies of code that need to be cleaned up later), it would make the work of reviewing of the design of the helper and refactoring using that helper smaller, not larger. And the new codepaths that use the helper would become easier to follow (otherwise we wouldn't be doing such a refactoring in the first place), making the reviews easier. That is where my preference comes from. There _is_ a sunk-cost that has already been invested in reviewing the older round that added code that needs to be cleaned up; there are some parts other than the "need to be cleaned-up" parts that we would feel confortable having in the reroll without having to re-review them. I do not know if that is a very high cost, though, especially for those who have already seen the previous rounds. Anyway, I wouldn't insist and we can go either way. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv7 4/8] strbuf: add strbuf_read_once to read without blocking
The new call will read from a file descriptor into a strbuf once. The
underlying call xread_nonblock is meant to execute without blocking if
the file descriptor is set to O_NONBLOCK. It is a bug to call
strbuf_read_once on a file descriptor which would block.
Signed-off-by: Stefan Beller
Signed-off-by: Junio C Hamano
---
strbuf.c | 11 +++
strbuf.h | 9 +
2 files changed, 20 insertions(+)
diff --git a/strbuf.c b/strbuf.c
index cce5eed..35e71b8 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -384,6 +384,17 @@ ssize_t strbuf_read(struct strbuf *sb, int fd, size_t hint)
return sb->len - oldlen;
}
+ssize_t strbuf_read_once(struct strbuf *sb, int fd, size_t hint)
+{
+ ssize_t cnt;
+
+ strbuf_grow(sb, hint ? hint : 8192);
+ cnt = xread_nonblock(fd, sb->buf + sb->len, sb->alloc - sb->len - 1);
+ if (cnt > 0)
+ strbuf_setlen(sb, sb->len + cnt);
+ return cnt;
+}
+
#define STRBUF_MAXLINK (2*PATH_MAX)
int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint)
diff --git a/strbuf.h b/strbuf.h
index aef2794..ea69665 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -367,6 +367,15 @@ extern size_t strbuf_fread(struct strbuf *, size_t, FILE
*);
extern ssize_t strbuf_read(struct strbuf *, int fd, size_t hint);
/**
+ * Read from a file descriptor that is marked as O_NONBLOCK without
+ * blocking. Returns the number of new bytes appended to the sb.
+ * Negative return value signals there was an error returned from
+ * underlying read(2), in which case the caller should check errno.
+ * e.g. errno == EAGAIN when the read may have blocked.
+ */
+extern ssize_t strbuf_read_once(struct strbuf *, int fd, size_t hint);
+
+/**
* Read the contents of a file, specified by its path. The third argument
* can be used to give a hint about the file size, to avoid reallocs.
*/
--
2.5.0.268.g453a26a
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv7 3/8] xread_nonblock: add functionality to read from fds without blocking
Provide a wrapper to read(), similar to xread(), that restarts on
EINTR but not EAGAIN (or EWOULDBLOCK). This enables the caller to
handle polling itself, possibly polling multiple sockets or performing
some other action.
Helped-by: Jacob Keller
Helped-by: Jeff King ,
Helped-by: Junio C Hamano
Signed-off-by: Stefan Beller
Signed-off-by: Junio C Hamano
---
git-compat-util.h | 1 +
wrapper.c | 22 ++
2 files changed, 23 insertions(+)
diff --git a/git-compat-util.h b/git-compat-util.h
index c6d391f..9ccea85 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -718,6 +718,7 @@ extern void *xcalloc(size_t nmemb, size_t size);
extern void *xmmap(void *start, size_t length, int prot, int flags, int fd,
off_t offset);
extern void *xmmap_gently(void *start, size_t length, int prot, int flags, int
fd, off_t offset);
extern ssize_t xread(int fd, void *buf, size_t len);
+extern ssize_t xread_nonblock(int fd, void *buf, size_t len);
extern ssize_t xwrite(int fd, const void *buf, size_t len);
extern ssize_t xpread(int fd, void *buf, size_t len, off_t offset);
extern int xdup(int fd);
diff --git a/wrapper.c b/wrapper.c
index 5517928..41a21e1 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -217,6 +217,28 @@ ssize_t xread(int fd, void *buf, size_t len)
}
/*
+ * xread_nonblock() is the same a read(), but it automatically restarts read()
+ * interrupted operations (EINTR). xread_nonblock() DOES NOT GUARANTEE that
+ * "len" bytes is read. EWOULDBLOCK is turned into EAGAIN.
+ */
+ssize_t xread_nonblock(int fd, void *buf, size_t len)
+{
+ ssize_t nr;
+ if (len > MAX_IO_SIZE)
+ len = MAX_IO_SIZE;
+ while (1) {
+ nr = read(fd, buf, len);
+ if (nr < 0) {
+ if (errno == EINTR)
+ continue;
+ if (errno == EWOULDBLOCK)
+ errno = EAGAIN;
+ }
+ return nr;
+ }
+}
+
+/*
* xwrite() is the same a write(), but it automatically restarts write()
* operations with a recoverable error (EAGAIN and EINTR). xwrite() DOES NOT
* GUARANTEE that "len" bytes is written even if the operation is successful.
--
2.5.0.268.g453a26a
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv7 0/8] fetch submodules in parallel
fixing two smaller nits noticed by Ramsay
* make struct parallel_processes static
* omit the forward declaration of get_next_submodule by reordering.
Jonathan Nieder (1):
submodule.c: write "Fetching submodule " to stderr
Stefan Beller (7):
xread: poll on non blocking fds
xread_nonblock: add functionality to read from fds without blocking
strbuf: add strbuf_read_once to read without blocking
sigchain: add command to pop all common signals
run-command: add an asynchronous parallel child processor
fetch_populated_submodules: use new parallel job processing
submodules: allow parallel fetching, add tests and documentation
Documentation/fetch-options.txt | 7 +
builtin/fetch.c | 6 +-
builtin/pull.c | 6 +
git-compat-util.h | 1 +
run-command.c | 350
run-command.h | 78 +
sigchain.c | 9 ++
sigchain.h | 1 +
strbuf.c| 11 ++
strbuf.h| 9 ++
submodule.c | 144 -
submodule.h | 2 +-
t/t0061-run-command.sh | 20 +++
t/t5526-fetch-submodules.sh | 70 +---
test-run-command.c | 25 +++
wrapper.c | 35 +++-
16 files changed, 701 insertions(+), 73 deletions(-)
$ git diff origin/sb/submodule-parallel-fetch
diff --git a/run-command.c b/run-command.c
index 28048a7..ef3da27 100644
--- a/run-command.c
+++ b/run-command.c
@@ -855,7 +855,7 @@ int capture_command(struct child_process *cmd, struct
strbuf *buf, size_t hint)
return finish_command(cmd);
}
-struct parallel_processes {
+static struct parallel_processes {
void *data;
int max_processes;
diff --git a/submodule.c b/submodule.c
index cf8bf5d..c21b265 100644
--- a/submodule.c
+++ b/submodule.c
@@ -628,68 +628,6 @@ struct submodule_parallel_fetch {
#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0}
static int get_next_submodule(void **task_cb, struct child_process *cp,
- struct strbuf *err, void *data);
-
-static int fetch_start_failure(struct child_process *cp,
- struct strbuf *err,
- void *cb, void *task_cb)
-{
- struct submodule_parallel_fetch *spf = cb;
-
- spf->result = 1;
-
- return 0;
-}
-
-static int fetch_finish(int retvalue, struct child_process *cp,
- struct strbuf *err, void *cb, void *task_cb)
-{
- struct submodule_parallel_fetch *spf = cb;
-
- if (retvalue)
- spf->result = 1;
-
- return 0;
-}
-
-int fetch_populated_submodules(const struct argv_array *options,
- const char *prefix, int command_line_option,
- int quiet, int max_parallel_jobs)
-{
- int i;
- struct submodule_parallel_fetch spf = SPF_INIT;
-
- spf.work_tree = get_git_work_tree();
- spf.command_line_option = command_line_option;
- spf.quiet = quiet;
- spf.prefix = prefix;
-
- if (!spf.work_tree)
- goto out;
-
- if (read_cache() < 0)
- die("index file corrupt");
-
- argv_array_push(&spf.args, "fetch");
- for (i = 0; i < options->argc; i++)
- argv_array_push(&spf.args, options->argv[i]);
- argv_array_push(&spf.args, "--recurse-submodules-default");
- /* default value, "--submodule-prefix" and its value are added later */
-
- calculate_changed_submodule_paths();
- run_processes_parallel(max_parallel_jobs,
- get_next_submodule,
- fetch_start_failure,
- fetch_finish,
- &spf);
-
- argv_array_clear(&spf.args);
-out:
- string_list_clear(&changed_submodule_paths, 1);
- return spf.result;
-}
-
-static int get_next_submodule(void **task_cb, struct child_process *cp,
struct strbuf *err, void *data)
{
int ret = 0;
@@ -775,6 +713,65 @@ static int get_next_submodule(void **task_cb, struct
child_process *cp,
return 0;
}
+static int fetch_start_failure(struct child_process *cp,
+ struct strbuf *err,
+ void *cb, void *task_cb)
+{
+ struct submodule_parallel_fetch *spf = cb;
+
+ spf->result = 1;
+
+ return 0;
+}
+
+static int fetch_finish(int retvalue, struct child_process *cp,
+ struct strbuf *err, void *cb, void *task_cb)
+{
+ struct submodule_parallel_fetch *spf = cb;
+
+ if (retvalue)
+ spf->result = 1;
+
+ return 0;
+}
+
+int fetch_populated_submodules(const struct argv_array *options,
+ const char *prefix, int command_line_opt
[PATCHv7 8/8] submodules: allow parallel fetching, add tests and documentation
This enables the work of the previous patches.
Signed-off-by: Stefan Beller
---
Documentation/fetch-options.txt | 7 +++
builtin/fetch.c | 6 +-
builtin/pull.c | 6 ++
submodule.c | 3 +--
submodule.h | 2 +-
t/t5526-fetch-submodules.sh | 19 +++
6 files changed, 39 insertions(+), 4 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 45583d8..6b109f6 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -100,6 +100,13 @@ ifndef::git-pull[]
reference to a commit that isn't already in the local submodule
clone.
+-j::
+--jobs=::
+ Number of parallel children to be used for fetching submodules.
+ Each will fetch from different submodules, such that fetching many
+ submodules will be faster. By default submodules will be fetched
+ one at a time.
+
--no-recurse-submodules::
Disable recursive fetching of submodules (this has the same effect as
using the '--recurse-submodules=no' option).
diff --git a/builtin/fetch.c b/builtin/fetch.c
index ee1f1a9..f28eac6 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -37,6 +37,7 @@ static int prune = -1; /* unspecified */
static int all, append, dry_run, force, keep, multiple, update_head_ok,
verbosity;
static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
static int tags = TAGS_DEFAULT, unshallow, update_shallow;
+static int max_children = 1;
static const char *depth;
static const char *upload_pack;
static struct strbuf default_rla = STRBUF_INIT;
@@ -99,6 +100,8 @@ static struct option builtin_fetch_options[] = {
N_("fetch all tags and associated objects"), TAGS_SET),
OPT_SET_INT('n', NULL, &tags,
N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
+ OPT_INTEGER('j', "jobs", &max_children,
+ N_("number of submodules fetched in parallel")),
OPT_BOOL('p', "prune", &prune,
N_("prune remote-tracking branches no longer on remote")),
{ OPTION_CALLBACK, 0, "recurse-submodules", NULL, N_("on-demand"),
@@ -1217,7 +1220,8 @@ int cmd_fetch(int argc, const char **argv, const char
*prefix)
result = fetch_populated_submodules(&options,
submodule_prefix,
recurse_submodules,
- verbosity < 0);
+ verbosity < 0,
+ max_children);
argv_array_clear(&options);
}
diff --git a/builtin/pull.c b/builtin/pull.c
index 722a83c..f0af196 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -94,6 +94,7 @@ static int opt_force;
static char *opt_tags;
static char *opt_prune;
static char *opt_recurse_submodules;
+static char *max_children;
static int opt_dry_run;
static char *opt_keep;
static char *opt_depth;
@@ -177,6 +178,9 @@ static struct option pull_options[] = {
N_("on-demand"),
N_("control recursive fetching of submodules"),
PARSE_OPT_OPTARG),
+ OPT_PASSTHRU('j', "jobs", &max_children, N_("n"),
+ N_("number of submodules pulled in parallel"),
+ PARSE_OPT_OPTARG),
OPT_BOOL(0, "dry-run", &opt_dry_run,
N_("dry run")),
OPT_PASSTHRU('k', "keep", &opt_keep, NULL,
@@ -524,6 +528,8 @@ static int run_fetch(const char *repo, const char
**refspecs)
argv_array_push(&args, opt_prune);
if (opt_recurse_submodules)
argv_array_push(&args, opt_recurse_submodules);
+ if (max_children)
+ argv_array_push(&args, max_children);
if (opt_dry_run)
argv_array_push(&args, "--dry-run");
if (opt_keep)
diff --git a/submodule.c b/submodule.c
index 8fdd2cb..c21b265 100644
--- a/submodule.c
+++ b/submodule.c
@@ -737,10 +737,9 @@ static int fetch_finish(int retvalue, struct child_process
*cp,
int fetch_populated_submodules(const struct argv_array *options,
const char *prefix, int command_line_option,
- int quiet)
+ int quiet, int max_parallel_jobs)
{
int i;
- int max_parallel_jobs = 1;
struct submodule_parallel_fetch spf = SPF_INIT;
spf.work_tree = get_git_work_tree();
diff --git a/submodule.h b/submodule.h
index 5507c3d..cbc0003 100644
--- a/submodule.h
+++ b/submodule.h
@@ -31,7 +31,7 @@ void set_config_fetch_recurse_submodules(int value);
void check_for_new_submodule_commits(unsigned char new_sha1[20]);
int fetch_populated_submodules(const struct argv_array *options,
const char *prefix, int comma
[PATCHv7 1/8] submodule.c: write "Fetching submodule " to stderr
From: Jonathan Nieder
The "Pushing submodule " progress output correctly goes to
stderr, but "Fetching submodule " is going to stdout by
mistake. Fix it to write to stderr.
Noticed while trying to implement a parallel submodule fetch. When
this particular output line went to a different file descriptor, it
was buffered separately, resulting in wrongly interleaved output if
we copied it to the terminal naively.
Signed-off-by: Jonathan Nieder
Signed-off-by: Stefan Beller
Signed-off-by: Junio C Hamano
---
submodule.c | 2 +-
t/t5526-fetch-submodules.sh | 51 +++--
2 files changed, 27 insertions(+), 26 deletions(-)
diff --git a/submodule.c b/submodule.c
index 9fcc86f..1d64e57 100644
--- a/submodule.c
+++ b/submodule.c
@@ -694,7 +694,7 @@ int fetch_populated_submodules(const struct argv_array
*options,
git_dir = submodule_git_dir.buf;
if (is_directory(git_dir)) {
if (!quiet)
- printf("Fetching submodule %s%s\n", prefix,
ce->name);
+ fprintf(stderr, "Fetching submodule %s%s\n",
prefix, ce->name);
cp.dir = submodule_path.buf;
argv_array_push(&argv, default_argv);
argv_array_push(&argv, "--submodule-prefix");
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index a4532b0..17759b1 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -16,7 +16,8 @@ add_upstream_commit() {
git add subfile &&
git commit -m new subfile &&
head2=$(git rev-parse --short HEAD) &&
- echo "From $pwd/submodule" > ../expect.err &&
+ echo "Fetching submodule submodule" > ../expect.err &&
+ echo "From $pwd/submodule" >> ../expect.err &&
echo " $head1..$head2 master -> origin/master" >>
../expect.err
) &&
(
@@ -27,6 +28,7 @@ add_upstream_commit() {
git add deepsubfile &&
git commit -m new deepsubfile &&
head2=$(git rev-parse --short HEAD) &&
+ echo "Fetching submodule submodule/subdir/deepsubmodule" >>
../expect.err
echo "From $pwd/deepsubmodule" >> ../expect.err &&
echo " $head1..$head2 master -> origin/master" >>
../expect.err
)
@@ -56,9 +58,7 @@ test_expect_success setup '
(
cd downstream &&
git submodule update --init --recursive
- ) &&
- echo "Fetching submodule submodule" > expect.out &&
- echo "Fetching submodule submodule/subdir/deepsubmodule" >> expect.out
+ )
'
test_expect_success "fetch --recurse-submodules recurses into submodules" '
@@ -67,7 +67,7 @@ test_expect_success "fetch --recurse-submodules recurses into
submodules" '
cd downstream &&
git fetch --recurse-submodules >../actual.out 2>../actual.err
) &&
- test_i18ncmp expect.out actual.out &&
+ test_must_be_empty actual.out &&
test_i18ncmp expect.err actual.err
'
@@ -96,7 +96,7 @@ test_expect_success "using fetchRecurseSubmodules=true in
.gitmodules recurses i
git config -f .gitmodules
submodule.submodule.fetchRecurseSubmodules true &&
git fetch >../actual.out 2>../actual.err
) &&
- test_i18ncmp expect.out actual.out &&
+ test_must_be_empty actual.out &&
test_i18ncmp expect.err actual.err
'
@@ -127,7 +127,7 @@ test_expect_success "--recurse-submodules overrides
fetchRecurseSubmodules setti
git config --unset -f .gitmodules
submodule.submodule.fetchRecurseSubmodules &&
git config --unset submodule.submodule.fetchRecurseSubmodules
) &&
- test_i18ncmp expect.out actual.out &&
+ test_must_be_empty actual.out &&
test_i18ncmp expect.err actual.err
'
@@ -146,7 +146,7 @@ test_expect_success "--dry-run propagates to submodules" '
cd downstream &&
git fetch --recurse-submodules --dry-run >../actual.out
2>../actual.err
) &&
- test_i18ncmp expect.out actual.out &&
+ test_must_be_empty actual.out &&
test_i18ncmp expect.err actual.err
'
@@ -155,7 +155,7 @@ test_expect_success "Without --dry-run propagates to
submodules" '
cd downstream &&
git fetch --recurse-submodules >../actual.out 2>../actual.err
) &&
- test_i18ncmp expect.out actual.out &&
+ test_must_be_empty actual.out &&
test_i18ncmp expect.err actual.err
'
@@ -166,7 +166,7 @@ test_expect_success "recurseSubmodules=true propagates into
submodules" '
git config fetch.recurseSubmodules true
git fetch >../actual.out 2>../actual.err
) &&
- test_i18ncmp expect.out act
[PATCHv7 5/8] sigchain: add command to pop all common signals
The new method removes all common signal handlers that were installed
by sigchain_push.
CC: Jeff King
Signed-off-by: Stefan Beller
Signed-off-by: Junio C Hamano
---
sigchain.c | 9 +
sigchain.h | 1 +
2 files changed, 10 insertions(+)
diff --git a/sigchain.c b/sigchain.c
index faa375d..2ac43bb 100644
--- a/sigchain.c
+++ b/sigchain.c
@@ -50,3 +50,12 @@ void sigchain_push_common(sigchain_fun f)
sigchain_push(SIGQUIT, f);
sigchain_push(SIGPIPE, f);
}
+
+void sigchain_pop_common(void)
+{
+ sigchain_pop(SIGPIPE);
+ sigchain_pop(SIGQUIT);
+ sigchain_pop(SIGTERM);
+ sigchain_pop(SIGHUP);
+ sigchain_pop(SIGINT);
+}
diff --git a/sigchain.h b/sigchain.h
index 618083b..138b20f 100644
--- a/sigchain.h
+++ b/sigchain.h
@@ -7,5 +7,6 @@ int sigchain_push(int sig, sigchain_fun f);
int sigchain_pop(int sig);
void sigchain_push_common(sigchain_fun f);
+void sigchain_pop_common(void);
#endif /* SIGCHAIN_H */
--
2.5.0.268.g453a26a
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv7 2/8] xread: poll on non blocking fds
>From the man page:
EAGAIN The file descriptor fd refers to a file other than a socket
and has been marked nonblocking (O_NONBLOCK), and the read
would block.
EAGAIN or EWOULDBLOCK
The file descriptor fd refers to a socket and has been marked
nonblocking (O_NONBLOCK), and the read would block. POSIX.1-2001
allows either error to be returned for this case, and does not
require these constants to have the same value, so a portable
application should check for both possibilities.
If we get an EAGAIN or EWOULDBLOCK the fd must have set O_NONBLOCK.
As the intent of xread is to read as much as possible either until the
fd is EOF or an actual error occurs, we can ease the feeder of the fd
by not spinning the whole time, but rather wait for it politely by not
busy waiting.
We should not care if the call to poll failed, as we're in an infinite
loop and can only get out with the correct read().
Signed-off-by: Stefan Beller
Signed-off-by: Junio C Hamano
---
wrapper.c | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/wrapper.c b/wrapper.c
index ff49807..5517928 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -201,8 +201,17 @@ ssize_t xread(int fd, void *buf, size_t len)
len = MAX_IO_SIZE;
while (1) {
nr = read(fd, buf, len);
- if ((nr < 0) && (errno == EAGAIN || errno == EINTR))
- continue;
+ if (nr < 0) {
+ if (errno == EINTR)
+ continue;
+ if (errno == EAGAIN || errno == EWOULDBLOCK) {
+ struct pollfd pfd;
+ pfd.events = POLLIN;
+ pfd.fd = fd;
+ /* We deliberately ignore the return value */
+ poll(&pfd, 1, -1);
+ }
+ }
return nr;
}
}
--
2.5.0.268.g453a26a
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv7 6/8] run-command: add an asynchronous parallel child processor
This allows to run external commands in parallel with ordered output
on stderr.
If we run external commands in parallel we cannot pipe the output directly
to the our stdout/err as it would mix up. So each process's output will
flow through a pipe, which we buffer. One subprocess can be directly
piped to out stdout/err for a low latency feedback to the user.
Example:
Let's assume we have 5 submodules A,B,C,D,E and each fetch takes a
different amount of time as the different submodules vary in size, then
the output of fetches in sequential order might look like this:
time -->
output: |---A---| |-B-| |---C---| |-D-| |-E-|
When we schedule these submodules into maximal two parallel processes,
a schedule and sample output over time may look like this:
process 1: |---A---| |-D-| |-E-|
process 2: |-B-| |---C---|
output:|---A---|B|---C---|DE
So A will be perceived as it would run normally in the single child
version. As B has finished by the time A is done, we can dump its whole
progress buffer on stderr, such that it looks like it finished in no
time. Once that is done, C is determined to be the visible child and
its progress will be reported in real time.
So this way of output is really good for human consumption, as it only
changes the timing, not the actual output.
For machine consumption the output needs to be prepared in the tasks,
by either having a prefix per line or per block to indicate whose tasks
output is displayed, because the output order may not follow the
original sequential ordering:
|A| |--B--| |-C-|
will be scheduled to be all parallel:
process 1: |A|
process 2: |--B--|
process 3: |-C-|
output:|A|CB
This happens because C finished before B did, so it will be queued for
output before B.
Signed-off-by: Stefan Beller
Signed-off-by: Junio C Hamano
---
run-command.c | 350 +
run-command.h | 78 +++
t/t0061-run-command.sh | 20 +++
test-run-command.c | 25
4 files changed, 473 insertions(+)
diff --git a/run-command.c b/run-command.c
index 28e1d55..ef3da27 100644
--- a/run-command.c
+++ b/run-command.c
@@ -3,6 +3,8 @@
#include "exec_cmd.h"
#include "sigchain.h"
#include "argv-array.h"
+#include "thread-utils.h"
+#include "strbuf.h"
void child_process_init(struct child_process *child)
{
@@ -852,3 +854,351 @@ int capture_command(struct child_process *cmd, struct
strbuf *buf, size_t hint)
close(cmd->out);
return finish_command(cmd);
}
+
+static struct parallel_processes {
+ void *data;
+
+ int max_processes;
+ int nr_processes;
+
+ get_next_task_fn get_next_task;
+ start_failure_fn start_failure;
+ task_finished_fn task_finished;
+
+ struct {
+ unsigned in_use : 1;
+ struct child_process process;
+ struct strbuf err;
+ void *data;
+ } *children;
+ /*
+* The struct pollfd is logically part of *children,
+* but the system call expects it as its own array.
+*/
+ struct pollfd *pfd;
+
+ unsigned shutdown : 1;
+
+ int output_owner;
+ struct strbuf buffered_output; /* of finished children */
+} parallel_processes_struct;
+
+static int default_start_failure(struct child_process *cp,
+struct strbuf *err,
+void *pp_cb,
+void *pp_task_cb)
+{
+ int i;
+
+ strbuf_addstr(err, "Starting a child failed:");
+ for (i = 0; cp->argv[i]; i++)
+ strbuf_addf(err, " %s", cp->argv[i]);
+
+ return 0;
+}
+
+static int default_task_finished(int result,
+struct child_process *cp,
+struct strbuf *err,
+void *pp_cb,
+void *pp_task_cb)
+{
+ int i;
+
+ if (!result)
+ return 0;
+
+ strbuf_addf(err, "A child failed with return code %d:", result);
+ for (i = 0; cp->argv[i]; i++)
+ strbuf_addf(err, " %s", cp->argv[i]);
+
+ return 0;
+}
+
+static void kill_children(struct parallel_processes *pp, int signo)
+{
+ int i, n = pp->max_processes;
+
+ for (i = 0; i < n; i++)
+ if (pp->children[i].in_use)
+ kill(pp->children[i].process.pid, signo);
+}
+
+static void handle_children_on_signal(int signo)
+{
+ struct parallel_processes *pp = ¶llel_processes_struct;
+
+ kill_children(pp, signo);
+ sigchain_pop(signo);
+ raise(signo);
+}
+
+static struct parallel_processes *pp_init(int n,
+ get_next_task_fn get_next_task,
+ start_failure_fn start_failure,
+ task_finished_fn task_finished,
+
[PATCHv7 7/8] fetch_populated_submodules: use new parallel job processing
In a later patch we enable parallel processing of submodules, this
only adds the possibility for it. So this change should not change
any user facing behavior.
Signed-off-by: Stefan Beller
---
submodule.c | 145 ++--
1 file changed, 101 insertions(+), 44 deletions(-)
diff --git a/submodule.c b/submodule.c
index 1d64e57..8fdd2cb 100644
--- a/submodule.c
+++ b/submodule.c
@@ -12,6 +12,7 @@
#include "sha1-array.h"
#include "argv-array.h"
#include "blob.h"
+#include "thread-utils.h"
static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
static struct string_list changed_submodule_paths;
@@ -615,37 +616,28 @@ static void calculate_changed_submodule_paths(void)
initialized_fetch_ref_tips = 0;
}
-int fetch_populated_submodules(const struct argv_array *options,
- const char *prefix, int command_line_option,
- int quiet)
+struct submodule_parallel_fetch {
+ int count;
+ struct argv_array args;
+ const char *work_tree;
+ const char *prefix;
+ int command_line_option;
+ int quiet;
+ int result;
+};
+#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0}
+
+static int get_next_submodule(void **task_cb, struct child_process *cp,
+ struct strbuf *err, void *data)
{
- int i, result = 0;
- struct child_process cp = CHILD_PROCESS_INIT;
- struct argv_array argv = ARGV_ARRAY_INIT;
- const char *work_tree = get_git_work_tree();
- if (!work_tree)
- goto out;
-
- if (read_cache() < 0)
- die("index file corrupt");
-
- argv_array_push(&argv, "fetch");
- for (i = 0; i < options->argc; i++)
- argv_array_push(&argv, options->argv[i]);
- argv_array_push(&argv, "--recurse-submodules-default");
- /* default value, "--submodule-prefix" and its value are added later */
-
- cp.env = local_repo_env;
- cp.git_cmd = 1;
- cp.no_stdin = 1;
-
- calculate_changed_submodule_paths();
+ int ret = 0;
+ struct submodule_parallel_fetch *spf = data;
- for (i = 0; i < active_nr; i++) {
+ for ( ; spf->count < active_nr; spf->count++) {
struct strbuf submodule_path = STRBUF_INIT;
struct strbuf submodule_git_dir = STRBUF_INIT;
struct strbuf submodule_prefix = STRBUF_INIT;
- const struct cache_entry *ce = active_cache[i];
+ const struct cache_entry *ce = active_cache[spf->count];
const char *git_dir, *default_argv;
const struct submodule *submodule;
@@ -657,7 +649,7 @@ int fetch_populated_submodules(const struct argv_array
*options,
submodule = submodule_from_name(null_sha1, ce->name);
default_argv = "yes";
- if (command_line_option == RECURSE_SUBMODULES_DEFAULT) {
+ if (spf->command_line_option == RECURSE_SUBMODULES_DEFAULT) {
if (submodule &&
submodule->fetch_recurse !=
RECURSE_SUBMODULES_NONE) {
@@ -680,40 +672,105 @@ int fetch_populated_submodules(const struct argv_array
*options,
default_argv = "on-demand";
}
}
- } else if (command_line_option == RECURSE_SUBMODULES_ON_DEMAND)
{
+ } else if (spf->command_line_option ==
RECURSE_SUBMODULES_ON_DEMAND) {
if
(!unsorted_string_list_lookup(&changed_submodule_paths, ce->name))
continue;
default_argv = "on-demand";
}
- strbuf_addf(&submodule_path, "%s/%s", work_tree, ce->name);
+ strbuf_addf(&submodule_path, "%s/%s", spf->work_tree, ce->name);
strbuf_addf(&submodule_git_dir, "%s/.git", submodule_path.buf);
- strbuf_addf(&submodule_prefix, "%s%s/", prefix, ce->name);
+ strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, ce->name);
git_dir = read_gitfile(submodule_git_dir.buf);
if (!git_dir)
git_dir = submodule_git_dir.buf;
if (is_directory(git_dir)) {
- if (!quiet)
- fprintf(stderr, "Fetching submodule %s%s\n",
prefix, ce->name);
- cp.dir = submodule_path.buf;
- argv_array_push(&argv, default_argv);
- argv_array_push(&argv, "--submodule-prefix");
- argv_array_push(&argv, submodule_prefix.buf);
- cp.argv = argv.argv;
- if (run_command(&cp))
- result = 1;
- argv_array
Re: [PATCH v3 1/3] Add Travis CI support
On Mon, Oct 12, 2015 at 6:02 PM, Junio C Hamano wrote: >> Semantically, it does not seem correct to me that configuarion goes to >> the install step. As "make test" will build git anyway, I'd instead >> propose to get rid of "install" and just say: >> >> before_script: make configure && ./configure >> >> script: make --quiet test > > Very good point. Do we even need to do anything in the "install" > target? We aim to be able to testable without any installed Git, > and not running "make install" at all, ever, would be one way to > make sure that works. Note that Travis' "install" step is about installing dependencies for the application to build [1], not for installing the built application (i.e. what "make install" does). In any case, I still think configuring the application to built in this step is wrong. > we are to start using automated tests, I wonder if we want to build > (and optionally test) with various combinations of the customization > options (e.g. NO_CURL, NO_OPENSSL, NO_MMAP, NO_IPV6, NO_PERL etc.) I like that idea, but I think we should save that for a future improvement to .travis.yml. [1] http://docs.travis-ci.com/user/installing-dependencies/ -- Sebastian Schuberth -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/10] port branch.c to use ref-filter's printing options
Karthik Nayak writes: > On Mon, Oct 12, 2015 at 6:06 AM, Junio C Hamano wrote: >> Karthik Nayak writes: >> >>> On Fri, Oct 9, 2015 at 11:59 PM, Junio C Hamano wrote: >>> ... >>> Also does it make sense to integrate these changes here? Or would you like >>> to >>> have another series on this? >> >> I do not think you would want to ask that question, as my answer >> would most likely be "The most preferable would be a series to clean >> up the existing codepath that deals with %(align) first, on top of >> which everything in flight that is not yet in 'next' is rebased." > > Ah, but I might take a while to get there, So I'd rather push code which > is almost ready and work on that slowly, if that's ok? That's OK to me. The "most preferable way" above would lead to a cleaner history, but also more work for you and for me as a reviewer. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] fixup: use xstrfmt instead of fixed-size buf + sprintf + xstrdup
On Mon, Oct 12, 2015 at 11:15 PM, Junio C Hamano wrote: > Matthieu Moy writes: > >> The char buf[40] is safe (at least while the strings are not >> translated), but I'd rather avoid magic numbers like this 40 in the >> code, and use a construct that does not have this size limitation. >> Especially if it makes the code shorter. >> >> Signed-off-by: Matthieu Moy >> --- > > The construct being fixed with this change looks very similar to > Peff's a5e03bf5 (ref-filter: drop sprintf and strcpy calls, > 2015-09-24) on jk/war-on-sprintf topic, but the new code since that > commit cleaned up. > Yes, pretty much the same. > I'd expect that this will be rolled into Karthik's series in the > next reroll? > > Looking good. Thanks. Yes, I've adding this into my series. -- Regards, Karthik Nayak -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] submodule-config: Untangle logic in parse_config
This improves readability of the parse_config logic by making it more concise.
CC: Eric Sunshine
CC: Heiko Voigt
Signed-off-by: Stefan Beller
---
Eric commented on this during the long series for parallelizing
fetching submodules. I realized this can be added as an independant patch
as it doesn't interfere with the series at all, but rather was needed for the
next series I planned to put on top (parallelize git submodule). I need to
rethink the next series anyway, so I extracted this patch as a useful
readability improvement.
submodule-config.c | 74 +-
1 file changed, 29 insertions(+), 45 deletions(-)
diff --git a/submodule-config.c b/submodule-config.c
index 393de53..afe0ea8 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -257,78 +257,62 @@ static int parse_config(const char *var, const char
*value, void *data)
if (!name_and_item_from_var(var, &name, &item))
return 0;
- submodule = lookup_or_create_by_name(me->cache, me->gitmodules_sha1,
- name.buf);
+ submodule = lookup_or_create_by_name(me->cache,
+me->gitmodules_sha1,
+name.buf);
if (!strcmp(item.buf, "path")) {
- struct strbuf path = STRBUF_INIT;
- if (!value) {
+ if (!value)
ret = config_error_nonbool(var);
- goto release_return;
- }
- if (!me->overwrite && submodule->path != NULL) {
+ else if (!me->overwrite && submodule->path != NULL)
warn_multiple_config(me->commit_sha1, submodule->name,
"path");
- goto release_return;
+ else {
+ if (submodule->path)
+ cache_remove_path(me->cache, submodule);
+ free((void *) submodule->path);
+ submodule->path = xstrdup(value);
+ cache_put_path(me->cache, submodule);
}
-
- if (submodule->path)
- cache_remove_path(me->cache, submodule);
- free((void *) submodule->path);
- strbuf_addstr(&path, value);
- submodule->path = strbuf_detach(&path, NULL);
- cache_put_path(me->cache, submodule);
} else if (!strcmp(item.buf, "fetchrecursesubmodules")) {
/* when parsing worktree configurations we can die early */
int die_on_error = is_null_sha1(me->gitmodules_sha1);
if (!me->overwrite &&
- submodule->fetch_recurse != RECURSE_SUBMODULES_NONE) {
+ submodule->fetch_recurse != RECURSE_SUBMODULES_NONE)
warn_multiple_config(me->commit_sha1, submodule->name,
"fetchrecursesubmodules");
- goto release_return;
- }
-
- submodule->fetch_recurse = parse_fetch_recurse(var, value,
+ else
+ submodule->fetch_recurse = parse_fetch_recurse(
+ var, value,
die_on_error);
} else if (!strcmp(item.buf, "ignore")) {
- struct strbuf ignore = STRBUF_INIT;
- if (!me->overwrite && submodule->ignore != NULL) {
+ if (!value)
+ ret = config_error_nonbool(var);
+ else if (!me->overwrite && submodule->ignore != NULL)
warn_multiple_config(me->commit_sha1, submodule->name,
"ignore");
- goto release_return;
- }
- if (!value) {
- ret = config_error_nonbool(var);
- goto release_return;
- }
- if (strcmp(value, "untracked") && strcmp(value, "dirty") &&
- strcmp(value, "all") && strcmp(value, "none")) {
+ else if (strcmp(value, "untracked") &&
+strcmp(value, "dirty") &&
+strcmp(value, "all") &&
+strcmp(value, "none"))
warning("Invalid parameter '%s' for config option "
"'submodule.%s.ignore'", value, var);
- goto release_return;
+ else {
+ free((void *) submodule->ignore);
+ submodule->ignore = xstrdup(value);
}
-
- free((void *) submodule->ignore);
- strbuf_addstr(&ignore, value);
- submodule->ignore = strbuf_detach(&ignore, NULL);
} else if (!strc
Re: thread-utils: build with NO_PTHREADS fails
Junio C Hamano writes:
> Victor Leschuk writes:
>
>> I think that no one tried it for a long time but I needed a
>> single-threaded git version for debug purpose. I tried to build
>> with -DNO_PTHREADS and thread-utils.c failed to compile.
>>
>> In brief the situation is the following:
>>
>> in header file we have something like that:
>>
>>
>> #ifndef NO_PTHREAD
>> extern int online_cpus(void);
>>
>> #else
>> #define online_cpus() 1
>> #endif // NO_PTHREAD
>>
>> and in *.c file:
>>
>>
>> int online_cpus(void)
>> {
>> // ...
>> }
>
> Yeah, that is obviously incorrect.
> ...
Well, no, I spoke too early. I do not see there is much wrong here.
There is this bit in the Makefile:
ifdef NO_PTHREADS
BASIC_CFLAGS += -DNO_PTHREADS
else
BASIC_CFLAGS += $(PTHREAD_CFLAGS)
EXTLIBS += $(PTHREAD_LIBS)
LIB_OBJS += thread-utils.o
endif
The source file thread-utils.c is not compiled to thread-utils.o if
you say NO_PTHREADS, and the resulting libgit.a does not of course
have it.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/10] port branch.c to use ref-filter's printing options
On Mon, Oct 12, 2015 at 6:06 AM, Junio C Hamano wrote: > Karthik Nayak writes: > >> On Fri, Oct 9, 2015 at 11:59 PM, Junio C Hamano wrote: >> ... >> Also does it make sense to integrate these changes here? Or would you like to >> have another series on this? > > I do not think you would want to ask that question, as my answer > would most likely be "The most preferable would be a series to clean > up the existing codepath that deals with %(align) first, on top of > which everything in flight that is not yet in 'next' is rebased." > Ah, but I might take a while to get there, So I'd rather push code which is almost ready and work on that slowly, if that's ok? -- Regards, Karthik Nayak -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] fixup: use xstrfmt instead of fixed-size buf + sprintf + xstrdup
Matthieu Moy writes:
> The char buf[40] is safe (at least while the strings are not
> translated), but I'd rather avoid magic numbers like this 40 in the
> code, and use a construct that does not have this size limitation.
> Especially if it makes the code shorter.
>
> Signed-off-by: Matthieu Moy
> ---
The construct being fixed with this change looks very similar to
Peff's a5e03bf5 (ref-filter: drop sprintf and strcpy calls,
2015-09-24) on jk/war-on-sprintf topic, but the new code since that
commit cleaned up.
I'd expect that this will be rolled into Karthik's series in the
next reroll?
Looking good. Thanks.
> ref-filter.c | 20
> 1 file changed, 8 insertions(+), 12 deletions(-)
>
> diff --git a/ref-filter.c b/ref-filter.c
> index 6044eb0..7932c21 100644
> --- a/ref-filter.c
> +++ b/ref-filter.c
> @@ -1116,7 +1116,6 @@ static void populate_value(struct ref_array_item *ref)
>strcmp(formatp, "trackshort") &&
>(starts_with(name, "upstream") ||
> starts_with(name, "push"))) {
> - char buf[40];
> unsigned int nobracket = 0;
>
> if (!strcmp(valp, ",nobracket"))
> @@ -1135,24 +1134,21 @@ static void populate_value(struct ref_array_item *ref)
> v->s = "";
> else if (!num_ours) {
> if (nobracket)
> - sprintf(buf, "behind %d",
> num_theirs);
> + v->s = xstrfmt("behind %d",
> num_theirs);
> else
> - sprintf(buf, "[behind %d]",
> num_theirs);
> - v->s = xstrdup(buf);
> + v->s = xstrfmt("[behind %d]",
> num_theirs);
> } else if (!num_theirs) {
> if (nobracket)
> - sprintf(buf, "ahead %d",
> num_ours);
> + v->s = xstrfmt("ahead %d",
> num_ours);
> else
> - sprintf(buf, "[ahead %d]",
> num_ours);
> - v->s = xstrdup(buf);
> + v->s = xstrfmt("[ahead %d]",
> num_ours);
> } else {
> if (nobracket)
> - sprintf(buf, "ahead %d, behind
> %d",
> - num_ours, num_theirs);
> + v->s = xstrfmt("ahead %d,
> behind %d",
> +num_ours,
> num_theirs);
> else
> - sprintf(buf, "[ahead %d, behind
> %d]",
> - num_ours, num_theirs);
> - v->s = xstrdup(buf);
> + v->s = xstrfmt("[ahead %d,
> behind %d]",
> +num_ours,
> num_theirs);
> }
> continue;
> } else if (!strcmp(formatp, "trackshort") &&
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/3] Add Travis CI support
On 12 Oct 2015, at 01:05, Sebastian Schuberth wrote: > On 10/11/2015 19:55, larsxschnei...@gmail.com wrote: > >> + sudo apt-get update -qq >> + sudo apt-get install -y apt-transport-https >> + sudo apt-get install perforce-server git-lfs > > Why no "-y" also in this line, or append these to the previous line? > > Or maybe even better, like [1] does, also use "--qq" (which implies "-y") for > "apt-get install"? Agreed! > >> +install: make configure && ./configure >> + >> +before_script: make >> + >> +script: make --quiet test > > Semantically, it does not seem correct to me that configuarion goes to the > install step. As "make test" will build git anyway, I'd instead propose to > get rid of "install" and just say: > > before_script: make configure && ./configure > > script: make --quiet test I understand your point. I did this to make the "make" logs easily accessible (no option "--quite"). By default Travis CI automatically collapses the logs from all stages prior to the "script" stage. You can uncollapse these logs by clicking on the little triangle on the left border of the log. Therefore the "make" logs are available without noise. Do you see value in "make" logs? If yes then we could also do: before_script: make configure && ./configure && make If no then I will take your suggestion. Thanks, Lars-- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v1 2/2] git-p4: Skip t9819 test case on case insensitive file systems
From: Lars Schneider Windows and OS X file systems are case insensitive by default. Consequently the "git-p4-case-folding" test case does not apply to them. Signed-off-by: Lars Schneider Acked-by: Luke Diamand --- t/t9819-git-p4-case-folding.sh | 6 ++ 1 file changed, 6 insertions(+) diff --git a/t/t9819-git-p4-case-folding.sh b/t/t9819-git-p4-case-folding.sh index 78f1d0f..d808c00 100755 --- a/t/t9819-git-p4-case-folding.sh +++ b/t/t9819-git-p4-case-folding.sh @@ -4,6 +4,12 @@ test_description='interaction with P4 case-folding' . ./lib-git-p4.sh +if test_have_prereq CASE_INSENSITIVE_FS +then + skip_all='skipping P4 case-folding tests; case insensitive file system detected' + test_done +fi + test_expect_success 'start p4d with case folding enabled' ' start_p4d -C1 ' -- 2.5.1 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v1 0/2] git-p4: Fix tests on OS X
From: Lars Schneider I extracted this patch series from "[PATCH v3 0/3] Add Travis CI support" as suggested by Junio. Thanks, Lars Lars Schneider (2): git-p4: Improve test case portability for t9815 git-p4-submit-fail git-p4: Skip t9819 test case on case insensitive file systems t/t9815-git-p4-submit-fail.sh | 7 ++- t/t9819-git-p4-case-folding.sh | 6 ++ 2 files changed, 8 insertions(+), 5 deletions(-) -- 2.5.1 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v1 1/2] git-p4: Improve test case portability for t9815 git-p4-submit-fail
From: Lars Schneider Replace the stats command with the ls command to check file mode bits. The stats command is not available on Windows and has different command line options on OS X. Signed-off-by: Lars Schneider --- t/t9815-git-p4-submit-fail.sh | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/t/t9815-git-p4-submit-fail.sh b/t/t9815-git-p4-submit-fail.sh index 4cff6a7..37b42d0 100755 --- a/t/t9815-git-p4-submit-fail.sh +++ b/t/t9815-git-p4-submit-fail.sh @@ -417,11 +417,8 @@ test_expect_success 'cleanup chmod after submit cancel' ' ! p4 fstat -T action text && test_path_is_file text+x && ! p4 fstat -T action text+x && - if test_have_prereq !CYGWIN - then - stat --format=%A text | egrep ^-r-- && - stat --format=%A text+x | egrep ^-r-x - fi + ls -l text | egrep ^-r-- && + ls -l text+x | egrep ^-r-x ) ' -- 2.5.1 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
