Re: Cloned repository has file changes -> bug?

[Torsten Bögershausen]

On Sat, Jan 27, 2018 at 08:59:50PM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> On Sat, Jan 27 2018, Filip Jorissen jotted:
> 
> > I think our git repository is bugged. The reason why I say this is the
> > following. When cloning the repository, the newly cloned repository
> > immediately has file changes[...].
> 
> If you run this:
> 
> git ls-files | tr '[:upper:]' '[:lower:]' | sort | uniq -D | grep '^'
> 
> You'll see that the reason is that you have files that differ only in
> case.
> 
> You are using a Mac, and Macs by default think that files that are
> different binary strings are the same file, since they don't consider
> case to be relevant. The file FOO, foo and FoO and fOo are all the same
> file as far as your Mac is concerned, but would be 4 different files on
> Linux.
> 
> > How can I fix the repository?
> 
> You could check it out on a OS that considers files that differ in case
> to be different files, e.g. on Linux, move them around, push it, and new
> clones should work on your Mac.
> 
> Alternatively I hear that you can create a loopback case-sensitive FS
> image on Macs.

You can even fix the repo locally.
There are 2 files with uppercase/lowercase collisions.
I show you how to fix one off these, the other one goes similar.
After that, do a commit and a push and pull request.



Changes not staged for commit:
  (use "git add ..." to update what will be committed)
  (use "git checkout -- ..." to discard changes in working directory)

modified:   
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
modified:   
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Utilities_Psychrometrics_Functions_Examples_saturationPressure.txt

no changes added to commit (use "git add" and/or "git commit -a")
user@mac:/tmp/IDEAS> git ls-files -s | grep -i 
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
100644 f56cfcf14aa4b53dfc5ecfb488366f721c94c8e2 0   
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
100644 e345e1372111d034b4c5a1c75eb791340b93f55e 0   
IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
user@mac:/tmp/IDEAS> git mv 
IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
 
IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump2.txt
user@mac:/tmp/IDEAS> git checkout  
IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump2.txt
user@mac:/tmp/IDEAS> git checkout 
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
user@mac:/tmp/IDEAS> git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD ..." to unstage)

renamed:
IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
 -> 
IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump2.txt

Changes not staged for commit:
  (use "git add ..." to update what will be committed)
  (use "git checkout -- ..." to discard changes in working directory)

modified:   
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Utilities_Psychrometrics_Functions_Examples_saturationPressure.txt

user@mac:/tmp/IDEAS>

Re: Creating sparse checkout in a new linked git worktree

[Eric Sunshine]

On Wed, Jan 24, 2018 at 11:11 AM, Jessie Hernandez
 wrote:
> I am trying to get a sparse checkout in a linked worktree but cannot get
> it working. I have tried the following
>
> * git worktree add /some/new/path/new-branch --no-checkout
> * git config core.sparseCheckout true
> *  $GIT_DIR/info/sparse-checkout>
> * cd  /some/new/path/new-branch
> * git read-tree -mu sparse-checkout
>
> But I still end up with a fully populated worktree.
> Is there something I am missing or doing wrong?

The sparse-checkout file is specific to each worktree, which allows
you to control "sparsity" on a worktree by worktree basis. Therefore,
you should create $GIT_DIR/worktrees//info/sparse-checkout instead
(where  is "new-branch" in your example).

Re: [PATCH v2] daemon: add --send-log-to=(stderr|syslog|none)

[Eric Sunshine]

On Sat, Jan 27, 2018 at 1:31 PM, Lucas Werkmeister
 wrote:
> This makes it possible to use --inetd while still logging to standard
> error. --syslog is retained as an alias for --send-log-to=syslog. A mode
> to disable logging explicitly is also provided.
>
> The combination of --inetd with --send-log-to=stderr is useful, for
> instance, when running `git daemon` as an instanced systemd service
> (with associated socket unit). In this case, log messages sent via
> syslog are received by the journal daemon, but run the risk of being
> processed at a time when the `git daemon` process has already exited
> (especially if the process was very short-lived, e.g. due to client
> error), so that the journal daemon can no longer read its cgroup and
> attach the message to the correct systemd unit (see systemd/systemd#2913
> [1]). Logging to stderr instead can solve this problem, because systemd
> can connect stderr directly to the journal daemon, which then already
> knows which unit is associated with this stream.

The purpose of this patch would be easier to fathom if the problem was
presented first (systemd race condition), followed by the solution
(ability to log to stderr even when using --inetd), followed finally
by incidental notes ("--syslog is retained as an alias..." and ability
to disable logging).

Not sure, though, if it's worth a re-roll.

> Signed-off-by: Lucas Werkmeister 
> Helped-by: Ævar Arnfjörð Bjarmason 
> Helped-by: Junio C Hamano 
> ---
>
> Notes:
> This was originally “daemon: add --no-syslog to undo implicit
> --syslog”, but Junio pointed out that combining --no-syslog with
> --detach isn’t especially useful and suggested --send-log-to=
> instead. Is Helped-by: the right credit for this or should it be
> something else?

Helped-by: is fine, though typically your Signed-off-by: would be last.

I understand that Junio suggested the name --send-log-to=, but I
wonder if the more concise --log= would be an possibility.

More below...

> diff --git a/Documentation/git-daemon.txt b/Documentation/git-daemon.txt
> @@ -110,8 +111,26 @@ OPTIONS
> +--send-log-to=::
> +   Send log messages to the specified destination.
> +   Note that this option does not imply --verbose,
> +   thus by default only error conditions will be logged.
> +   The  defaults to `stderr`, and must be one of:

Perhaps also update the documentation of --inetd to mention that its
implied --syslog can be overridden by --send-log-to=.

> diff --git a/daemon.c b/daemon.c
> @@ -74,11 +79,14 @@ static const char *get_ip_address(struct hostinfo *hi)
>
>  static void logreport(int priority, const char *err, va_list params)
>  {
> -   if (log_syslog) {
> +   switch (log_destination) {
> +   case LOG_TO_SYSLOG: {
> char buf[1024];
> vsnprintf(buf, sizeof(buf), err, params);
> syslog(priority, "%s", buf);
> -   } else {
> +   break;
> +   }
> +   case LOG_TO_STDERR: {

There aren't many instances of:

case FOO: {

in the code-base, but those that exist don't use braces around cases
which don't need it, so perhaps drop it from the STDERR and NONE
cases. (Probably not worth a re-roll, though.)

> /*
>  * Since stderr is set to buffered mode, the
>  * logging of different processes will not overlap
> @@ -88,6 +96,11 @@ static void logreport(int priority, const char *err, 
> va_list params)
> vfprintf(stderr, err, params);
> fputc('\n', stderr);
> fflush(stderr);
> +   break;
> +   }
> +   case LOG_TO_NONE: {
> +   break;
> +   }
> }

Consecutive lines with braces at the same indentation level is rather
odd (but see previous comment).

>  }
>
> @@ -1289,7 +1302,7 @@ int cmd_main(int argc, const char **argv)
> }
> if (!strcmp(arg, "--inetd")) {
> inetd_mode = 1;
> -   log_syslog = 1;
> +   log_destination = LOG_TO_SYSLOG;

Hmm, so an invocation "--inetd --send-log-to=stderr" works as
expected, but "--send-log-to=stderr --inetd" doesn't; output goes to
syslog despite the explicit request for stderr. Counterintuitive. This
should probably distinguish between 'log_destination' being unset and
set explicitly; if unset, then, and only then, have --inetd imply
syslog. Perhaps something like this:

static enum log_destination {
LOG_TO_UNSET = -1
LOG_TO_NONE,
LOG_TO_STDERR,
LOG_TO_SYSLOG,
} log_destination = LOG_TO_UNSET;

if (!strcmp(arg, "--inetd")) {
inetd_mode = 1;
if (log_destination == LOG_TO_UNSET)
log_destination = LOG_TO_SYSLOG;
...
}
...
if (log_destination == LOG_TO_UNSET)
log_destination = 

[PATCH v2 0/1] setup: recognise extensions.objectFormat

[Patryk Obara]

Compared to v1:

Implemented code suggestions from Duy Nguyễn (string for translation and
strbuf instead of char array). I also added an annotation in
repository-version.txt, clarifying, that this option is useful only for
development purpose for now.

Patryk Obara (1):
  setup: recognise extensions.objectFormat

 Documentation/technical/repository-version.txt | 12 
 setup.c| 27 ++
 t/t1302-repo-version.sh| 15 ++
 3 files changed, 54 insertions(+)


base-commit: 5be1f00a9a701532232f57958efab4be8c959a29
-- 
2.14.3

[PATCH v2 1/1] setup: recognise extensions.objectFormat

[Patryk Obara]

This extension selects which hashing algorithm from vtable should be
used for reading and writing objects in the object store.  At the moment
supports only single value (sha-1).

In case value of objectFormat is an unknown hashing algorithm, Git
command will fail with following message:

  fatal: unknown repository extensions found:
  objectformat = 

To indicate, that this specific objectFormat value is not recognised.

The objectFormat extension is not allowed in repository marked as
version 0 to prevent any possibility of accidentally writing a NewHash
object in the sha-1 object store. This extension behaviour is different
than preciousObjects extension (which is allowed in repo version 0).

Add tests and documentation note about new extension.

Signed-off-by: Patryk Obara 
---
 Documentation/technical/repository-version.txt | 12 
 setup.c| 27 ++
 t/t1302-repo-version.sh| 15 ++
 3 files changed, 54 insertions(+)

diff --git a/Documentation/technical/repository-version.txt 
b/Documentation/technical/repository-version.txt
index 00ad37986e..7e2b832603 100644
--- a/Documentation/technical/repository-version.txt
+++ b/Documentation/technical/repository-version.txt
@@ -86,3 +86,15 @@ for testing format-1 compatibility.
 When the config key `extensions.preciousObjects` is set to `true`,
 objects in the repository MUST NOT be deleted (e.g., by `git-prune` or
 `git repack -d`).
+
+`objectFormat`
+~~
+
+This extension instructs Git to use a specific algorithm for addressing
+and interpreting objects in the object store. Currently, the only
+supported object format is `sha-1`. At the moment, the primary purpose
+of this option is to enable Git developers to experiment with different
+hashing algorithms without re-compilation of git client.
+
+See `hash-function-transition.txt` document for more detailed explanation.
+
diff --git a/setup.c b/setup.c
index 8cc34186ce..9b9993a14e 100644
--- a/setup.c
+++ b/setup.c
@@ -405,6 +405,31 @@ void setup_work_tree(void)
initialized = 1;
 }
 
+static int find_object_format(const char *value)
+{
+   int i;
+   for (i = GIT_HASH_SHA1; i < GIT_HASH_NALGOS; ++i) {
+   if (strcmp(value, hash_algos[i].name) == 0)
+   return i;
+   }
+   return GIT_HASH_UNKNOWN;
+}
+
+static void detect_object_format(struct repository_format *data,
+const char *value)
+{
+   if (data->version == 0)
+   die(_("invalid repository format version '%d'"), data->version);
+
+   data->hash_algo = find_object_format(value);
+   if (data->hash_algo == GIT_HASH_UNKNOWN) {
+   struct strbuf object_format = STRBUF_INIT;
+   strbuf_addf(_format, "objectformat = %s", value);
+   string_list_append(>unknown_extensions, 
object_format.buf);
+   strbuf_release(_format);
+   }
+}
+
 static int check_repo_format(const char *var, const char *value, void *vdata)
 {
struct repository_format *data = vdata;
@@ -422,6 +447,8 @@ static int check_repo_format(const char *var, const char 
*value, void *vdata)
;
else if (!strcmp(ext, "preciousobjects"))
data->precious_objects = git_config_bool(var, value);
+   else if (!strcmp(ext, "objectformat"))
+   detect_object_format(data, value);
else
string_list_append(>unknown_extensions, ext);
} else if (strcmp(var, "core.bare") == 0) {
diff --git a/t/t1302-repo-version.sh b/t/t1302-repo-version.sh
index ce4cff13bb..227b397ff2 100755
--- a/t/t1302-repo-version.sh
+++ b/t/t1302-repo-version.sh
@@ -107,4 +107,19 @@ test_expect_success 'gc runs without complaint' '
git gc
 '
 
+test_expect_success 'object-format not allowed in repo version=0' '
+   mkconfig 0 "objectFormat = sha-1" >.git/config &&
+   check_abort
+'
+
+test_expect_success 'object-format=sha-1 allowed' '
+   mkconfig 1 "objectFormat = sha-1" >.git/config &&
+   check_allow
+'
+
+test_expect_success 'object-format=foo unsupported' '
+   mkconfig 1 "objectFormat = foo" >.git/config &&
+   check_abort
+'
+
 test_done
-- 
2.14.3

[PATCH v4 09/12] sha1_file: convert write_sha1_file to object_id

[Patryk Obara]

Convert the definition and declaration of write_sha1_file to
struct object_id and adjust usage of this function.

This commit also converts static function write_sha1_file_prepare, as it
is closely related.

Rename these functions to write_object_file and
write_object_file_prepare respectively.

Replace sha1_to_hex, hashcpy and hashclr with their oid equivalents
wherever possible.

Signed-off-by: Patryk Obara 
---
 apply.c  |  8 
 builtin/checkout.c   |  3 +--
 builtin/mktag.c  |  6 +++---
 builtin/mktree.c | 10 +-
 builtin/notes.c  |  8 
 builtin/receive-pack.c   | 11 ++-
 builtin/replace.c|  2 +-
 builtin/tag.c|  2 +-
 builtin/unpack-objects.c |  9 ++---
 cache-tree.c |  5 +++--
 cache.h  |  4 +++-
 commit.c |  2 +-
 match-trees.c|  2 +-
 merge-recursive.c|  5 +++--
 notes-cache.c|  2 +-
 notes.c  |  9 -
 read-cache.c |  6 +++---
 sha1_file.c  | 29 +++--
 18 files changed, 65 insertions(+), 58 deletions(-)

diff --git a/apply.c b/apply.c
index 57ab8a8a29..4cd4504008 100644
--- a/apply.c
+++ b/apply.c
@@ -3554,7 +3554,7 @@ static int try_threeway(struct apply_state *state,
 
/* Preimage the patch was prepared for */
if (patch->is_new)
-   write_sha1_file("", 0, blob_type, pre_oid.hash);
+   write_object_file("", 0, blob_type, _oid);
else if (get_oid(patch->old_sha1_prefix, _oid) ||
 read_blob_object(, _oid, patch->old_mode))
return error(_("repository lacks the necessary blob to fall 
back on 3-way merge."));
@@ -3570,7 +3570,7 @@ static int try_threeway(struct apply_state *state,
return -1;
}
/* post_oid is theirs */
-   write_sha1_file(tmp_image.buf, tmp_image.len, blob_type, post_oid.hash);
+   write_object_file(tmp_image.buf, tmp_image.len, blob_type, _oid);
clear_image(_image);
 
/* our_oid is ours */
@@ -3583,7 +3583,7 @@ static int try_threeway(struct apply_state *state,
return error(_("cannot read the current contents of 
'%s'"),
 patch->old_name);
}
-   write_sha1_file(tmp_image.buf, tmp_image.len, blob_type, our_oid.hash);
+   write_object_file(tmp_image.buf, tmp_image.len, blob_type, _oid);
clear_image(_image);
 
/* in-core three-way merge between post and our using pre as base */
@@ -4291,7 +4291,7 @@ static int add_index_file(struct apply_state *state,
}
fill_stat_cache_info(ce, );
}
-   if (write_sha1_file(buf, size, blob_type, ce->oid.hash) < 0) {
+   if (write_object_file(buf, size, blob_type, >oid) < 0) {
free(ce);
return error(_("unable to create backing store "
   "for newly created file %s"), path);
diff --git a/builtin/checkout.c b/builtin/checkout.c
index c54c78df54..191b96c49c 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -227,8 +227,7 @@ static int checkout_merged(int pos, const struct checkout 
*state)
 * (it also writes the merge result to the object database even
 * when it may contain conflicts).
 */
-   if (write_sha1_file(result_buf.ptr, result_buf.size,
-   blob_type, oid.hash))
+   if (write_object_file(result_buf.ptr, result_buf.size, blob_type, ))
die(_("Unable to add merge result for '%s'"), path);
free(result_buf.ptr);
ce = make_cache_entry(mode, oid.hash, path, 2, 0);
diff --git a/builtin/mktag.c b/builtin/mktag.c
index 031b750f06..beb552847b 100644
--- a/builtin/mktag.c
+++ b/builtin/mktag.c
@@ -151,7 +151,7 @@ static int verify_tag(char *buffer, unsigned long size)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
struct strbuf buf = STRBUF_INIT;
-   unsigned char result_sha1[20];
+   struct object_id result;
 
if (argc != 1)
usage("git mktag");
@@ -165,10 +165,10 @@ int cmd_mktag(int argc, const char **argv, const char 
*prefix)
if (verify_tag(buf.buf, buf.len) < 0)
die("invalid tag signature file");
 
-   if (write_sha1_file(buf.buf, buf.len, tag_type, result_sha1) < 0)
+   if (write_object_file(buf.buf, buf.len, tag_type, ) < 0)
die("unable to write tag file");
 
strbuf_release();
-   printf("%s\n", sha1_to_hex(result_sha1));
+   printf("%s\n", oid_to_hex());
return 0;
 }
diff --git a/builtin/mktree.c b/builtin/mktree.c
index da0fd8cd70..8dd9f52f77 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -40,7 +40,7 @@ static int ent_compare(const void *a_, const void *b_)
  

[PATCH v4 02/12] dir: convert struct sha1_stat to use object_id

[Patryk Obara]

Convert the declaration of struct sha1_stat. Adjust all usages of this
struct and replace hash{clr,cmp,cpy} with oid{clr,cmp,cpy} wherever
possible.  Rename it to struct oid_stat.

Rename static function load_sha1_stat to load_oid_stat.

Remove macro EMPTY_BLOB_SHA1_BIN, as it's no longer used.

Signed-off-by: Patryk Obara 
---
 cache.h  |   2 -
 dir.c| 104 +--
 dir.h|  12 ++--
 t/helper/test-dump-untracked-cache.c |   4 +-
 4 files changed, 58 insertions(+), 64 deletions(-)

diff --git a/cache.h b/cache.h
index e4e03ac51d..ed72933ba7 100644
--- a/cache.h
+++ b/cache.h
@@ -1047,8 +1047,6 @@ extern const struct object_id empty_tree_oid;
"\xe6\x9d\xe2\x9b\xb2\xd1\xd6\x43\x4b\x8b" \
"\x29\xae\x77\x5a\xd8\xc2\xe4\x8c\x53\x91"
 extern const struct object_id empty_blob_oid;
-#define EMPTY_BLOB_SHA1_BIN (empty_blob_oid.hash)
-
 
 static inline int is_empty_blob_sha1(const unsigned char *sha1)
 {
diff --git a/dir.c b/dir.c
index 7c4b45e30e..22cadbda9d 100644
--- a/dir.c
+++ b/dir.c
@@ -231,12 +231,10 @@ int within_depth(const char *name, int namelen,
  * 1 along with { data, size } of the (possibly augmented) buffer
  *   when successful.
  *
- * Optionally updates the given sha1_stat with the given OID (when valid).
+ * Optionally updates the given oid_stat with the given OID (when valid).
  */
-static int do_read_blob(const struct object_id *oid,
-   struct sha1_stat *sha1_stat,
-   size_t *size_out,
-   char **data_out)
+static int do_read_blob(const struct object_id *oid, struct oid_stat *oid_stat,
+   size_t *size_out, char **data_out)
 {
enum object_type type;
unsigned long sz;
@@ -251,9 +249,9 @@ static int do_read_blob(const struct object_id *oid,
return -1;
}
 
-   if (sha1_stat) {
-   memset(_stat->stat, 0, sizeof(sha1_stat->stat));
-   hashcpy(sha1_stat->sha1, oid->hash);
+   if (oid_stat) {
+   memset(_stat->stat, 0, sizeof(oid_stat->stat));
+   oidcpy(_stat->oid, oid);
}
 
if (sz == 0) {
@@ -654,9 +652,8 @@ void add_exclude(const char *string, const char *base,
 
 static int read_skip_worktree_file_from_index(const struct index_state *istate,
  const char *path,
- size_t *size_out,
- char **data_out,
- struct sha1_stat *sha1_stat)
+ size_t *size_out, char **data_out,
+ struct oid_stat *oid_stat)
 {
int pos, len;
 
@@ -667,7 +664,7 @@ static int read_skip_worktree_file_from_index(const struct 
index_state *istate,
if (!ce_skip_worktree(istate->cache[pos]))
return -1;
 
-   return do_read_blob(>cache[pos]->oid, sha1_stat, size_out, 
data_out);
+   return do_read_blob(>cache[pos]->oid, oid_stat, size_out, 
data_out);
 }
 
 /*
@@ -795,9 +792,8 @@ static int add_excludes_from_buffer(char *buf, size_t size,
  * ss_valid is non-zero, "ss" must contain good value as input.
  */
 static int add_excludes(const char *fname, const char *base, int baselen,
-   struct exclude_list *el,
-   struct index_state *istate,
-   struct sha1_stat *sha1_stat)
+   struct exclude_list *el, struct index_state *istate,
+   struct oid_stat *oid_stat)
 {
struct stat st;
int r;
@@ -815,16 +811,16 @@ static int add_excludes(const char *fname, const char 
*base, int baselen,
return -1;
r = read_skip_worktree_file_from_index(istate, fname,
   , ,
-  sha1_stat);
+  oid_stat);
if (r != 1)
return r;
} else {
size = xsize_t(st.st_size);
if (size == 0) {
-   if (sha1_stat) {
-   fill_stat_data(_stat->stat, );
-   hashcpy(sha1_stat->sha1, EMPTY_BLOB_SHA1_BIN);
-   sha1_stat->valid = 1;
+   if (oid_stat) {
+   fill_stat_data(_stat->stat, );
+   oidcpy(_stat->oid, _blob_oid);
+   oid_stat->valid = 1;
}
close(fd);
return 0;
@@ -837,22 +833,23 @@ static int add_excludes(const char *fname, const char 
*base, int baselen,
  

[PATCH v4 04/12] cache: clear whole hash buffer with oidclr

[Patryk Obara]

As long as GIT_SHA1_RAWSZ is equal to GIT_MAX_RAWSZ there's no problem,
but when new hashing algorithm will be in place this memset will clear
only 20-byte prefix of hash buffer.

Alternatively, hashclr implementation could be adjusted, but this
function is almost removed from codebase already.  Separate
implementation of oidclr prevents potential buffer overrun in case
someone incorrectly used hashclr on object_id in future.

Signed-off-by: Patryk Obara 
---
 cache.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cache.h b/cache.h
index 08f2b81e1b..d5d78d6a51 100644
--- a/cache.h
+++ b/cache.h
@@ -1029,7 +1029,7 @@ static inline void hashclr(unsigned char *hash)
 
 static inline void oidclr(struct object_id *oid)
 {
-   hashclr(oid->hash);
+   memset(oid->hash, 0, GIT_MAX_RAWSZ);
 }
 
 
-- 
2.14.3

[PATCH v4 11/12] sha1_file: convert write_loose_object to object_id

[Patryk Obara]

Convert the definition and declaration of static write_loose_object
function to struct object_id.

Signed-off-by: Patryk Obara 
---
 sha1_file.c | 28 
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index d9ee966d74..59238f5bea 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1548,16 +1548,17 @@ static int create_tmpfile(struct strbuf *tmp, const 
char *filename)
return fd;
 }
 
-static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
- const void *buf, unsigned long len, time_t mtime)
+static int write_loose_object(const struct object_id *oid, char *hdr,
+ int hdrlen, const void *buf, unsigned long len,
+ time_t mtime)
 {
int fd, ret;
unsigned char compressed[4096];
git_zstream stream;
git_SHA_CTX c;
-   unsigned char parano_sha1[20];
+   struct object_id parano_oid;
static struct strbuf tmp_file = STRBUF_INIT;
-   const char *filename = sha1_file_name(sha1);
+   const char *filename = sha1_file_name(oid->hash);
 
fd = create_tmpfile(_file, filename);
if (fd < 0) {
@@ -1594,13 +1595,16 @@ static int write_loose_object(const unsigned char 
*sha1, char *hdr, int hdrlen,
} while (ret == Z_OK);
 
if (ret != Z_STREAM_END)
-   die("unable to deflate new object %s (%d)", sha1_to_hex(sha1), 
ret);
+   die("unable to deflate new object %s (%d)", oid_to_hex(oid),
+   ret);
ret = git_deflate_end_gently();
if (ret != Z_OK)
-   die("deflateEnd on object %s failed (%d)", sha1_to_hex(sha1), 
ret);
-   git_SHA1_Final(parano_sha1, );
-   if (hashcmp(sha1, parano_sha1) != 0)
-   die("confused by unstable object source data for %s", 
sha1_to_hex(sha1));
+   die("deflateEnd on object %s failed (%d)", oid_to_hex(oid),
+   ret);
+   git_SHA1_Final(parano_oid.hash, );
+   if (oidcmp(oid, _oid) != 0)
+   die("confused by unstable object source data for %s",
+   oid_to_hex(oid));
 
close_sha1_file(fd);
 
@@ -1645,7 +1649,7 @@ int write_object_file(const void *buf, unsigned long len, 
const char *type,
write_object_file_prepare(buf, len, type, oid, hdr, );
if (freshen_packed_object(oid->hash) || freshen_loose_object(oid->hash))
return 0;
-   return write_loose_object(oid->hash, hdr, hdrlen, buf, len, 0);
+   return write_loose_object(oid, hdr, hdrlen, buf, len, 0);
 }
 
 int hash_sha1_file_literally(const void *buf, unsigned long len, const char 
*type,
@@ -1663,7 +1667,7 @@ int hash_sha1_file_literally(const void *buf, unsigned 
long len, const char *typ
goto cleanup;
if (freshen_packed_object(oid->hash) || freshen_loose_object(oid->hash))
goto cleanup;
-   status = write_loose_object(oid->hash, header, hdrlen, buf, len, 0);
+   status = write_loose_object(oid, header, hdrlen, buf, len, 0);
 
 cleanup:
free(header);
@@ -1685,7 +1689,7 @@ int force_object_loose(const struct object_id *oid, 
time_t mtime)
if (!buf)
return error("cannot read sha1_file for %s", oid_to_hex(oid));
hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %lu", typename(type), len) + 1;
-   ret = write_loose_object(oid->hash, hdr, hdrlen, buf, len, mtime);
+   ret = write_loose_object(oid, hdr, hdrlen, buf, len, mtime);
free(buf);
 
return ret;
-- 
2.14.3

[PATCH v4 10/12] sha1_file: convert force_object_loose to object_id

[Patryk Obara]

Convert the definition and declaration of force_object_loose to
struct object_id and adjust usage of this function.

Signed-off-by: Patryk Obara 
---
 builtin/pack-objects.c |  2 +-
 cache.h|  3 ++-
 sha1_file.c| 10 +-
 3 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 6b9cfc289d..f38197543d 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2768,7 +2768,7 @@ static void loosen_unused_packed_objects(struct rev_info 
*revs)
if (!packlist_find(_pack, oid.hash, NULL) &&
!has_sha1_pack_kept_or_nonlocal() &&
!loosened_object_can_be_discarded(, p->mtime))
-   if (force_object_loose(oid.hash, p->mtime))
+   if (force_object_loose(, p->mtime))
die("unable to force loose object");
}
}
diff --git a/cache.h b/cache.h
index d80141eb64..0a8be9c87f 100644
--- a/cache.h
+++ b/cache.h
@@ -1248,7 +1248,8 @@ extern int hash_sha1_file_literally(const void *buf, 
unsigned long len, const ch
 extern int pretend_object_file(void *, unsigned long, enum object_type,
   struct object_id *oid);
 
-extern int force_object_loose(const unsigned char *sha1, time_t mtime);
+extern int force_object_loose(const struct object_id *oid, time_t mtime);
+
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
 extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
diff --git a/sha1_file.c b/sha1_file.c
index d1569b1b96..d9ee966d74 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1670,7 +1670,7 @@ int hash_sha1_file_literally(const void *buf, unsigned 
long len, const char *typ
return status;
 }
 
-int force_object_loose(const unsigned char *sha1, time_t mtime)
+int force_object_loose(const struct object_id *oid, time_t mtime)
 {
void *buf;
unsigned long len;
@@ -1679,13 +1679,13 @@ int force_object_loose(const unsigned char *sha1, 
time_t mtime)
int hdrlen;
int ret;
 
-   if (has_loose_object(sha1))
+   if (has_loose_object(oid->hash))
return 0;
-   buf = read_object(sha1, , );
+   buf = read_object(oid->hash, , );
if (!buf)
-   return error("cannot read sha1_file for %s", sha1_to_hex(sha1));
+   return error("cannot read sha1_file for %s", oid_to_hex(oid));
hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %lu", typename(type), len) + 1;
-   ret = write_loose_object(sha1, hdr, hdrlen, buf, len, mtime);
+   ret = write_loose_object(oid->hash, hdr, hdrlen, buf, len, mtime);
free(buf);
 
return ret;
-- 
2.14.3

[PATCH v4 07/12] notes: convert combine_notes_* to object_id

[Patryk Obara]

Convert the definition and declarations of combine_notes_* functions
to struct object_id and adjust usage of these functions.

Signed-off-by: Patryk Obara 
---
 notes.c | 46 +++---
 notes.h | 25 +++--
 2 files changed, 38 insertions(+), 33 deletions(-)

diff --git a/notes.c b/notes.c
index c7f21fae44..3f4f94507a 100644
--- a/notes.c
+++ b/notes.c
@@ -270,8 +270,8 @@ static int note_tree_insert(struct notes_tree *t, struct 
int_node *tree,
if (!oidcmp(>val_oid, >val_oid))
return 0;
 
-   ret = combine_notes(l->val_oid.hash,
-   entry->val_oid.hash);
+   ret = combine_notes(>val_oid,
+   >val_oid);
if (!ret && is_null_oid(>val_oid))
note_tree_remove(t, tree, n, entry);
free(entry);
@@ -786,8 +786,8 @@ static int prune_notes_helper(const struct object_id 
*object_oid,
return 0;
 }
 
-int combine_notes_concatenate(unsigned char *cur_sha1,
-   const unsigned char *new_sha1)
+int combine_notes_concatenate(struct object_id *cur_oid,
+ const struct object_id *new_oid)
 {
char *cur_msg = NULL, *new_msg = NULL, *buf;
unsigned long cur_len, new_len, buf_len;
@@ -795,18 +795,18 @@ int combine_notes_concatenate(unsigned char *cur_sha1,
int ret;
 
/* read in both note blob objects */
-   if (!is_null_sha1(new_sha1))
-   new_msg = read_sha1_file(new_sha1, _type, _len);
+   if (!is_null_oid(new_oid))
+   new_msg = read_sha1_file(new_oid->hash, _type, _len);
if (!new_msg || !new_len || new_type != OBJ_BLOB) {
free(new_msg);
return 0;
}
-   if (!is_null_sha1(cur_sha1))
-   cur_msg = read_sha1_file(cur_sha1, _type, _len);
+   if (!is_null_oid(cur_oid))
+   cur_msg = read_sha1_file(cur_oid->hash, _type, _len);
if (!cur_msg || !cur_len || cur_type != OBJ_BLOB) {
free(cur_msg);
free(new_msg);
-   hashcpy(cur_sha1, new_sha1);
+   oidcpy(cur_oid, new_oid);
return 0;
}
 
@@ -825,20 +825,20 @@ int combine_notes_concatenate(unsigned char *cur_sha1,
free(new_msg);
 
/* create a new blob object from buf */
-   ret = write_sha1_file(buf, buf_len, blob_type, cur_sha1);
+   ret = write_sha1_file(buf, buf_len, blob_type, cur_oid->hash);
free(buf);
return ret;
 }
 
-int combine_notes_overwrite(unsigned char *cur_sha1,
-   const unsigned char *new_sha1)
+int combine_notes_overwrite(struct object_id *cur_oid,
+   const struct object_id *new_oid)
 {
-   hashcpy(cur_sha1, new_sha1);
+   oidcpy(cur_oid, new_oid);
return 0;
 }
 
-int combine_notes_ignore(unsigned char *cur_sha1,
-   const unsigned char *new_sha1)
+int combine_notes_ignore(struct object_id *cur_oid,
+const struct object_id *new_oid)
 {
return 0;
 }
@@ -848,17 +848,17 @@ int combine_notes_ignore(unsigned char *cur_sha1,
  * newlines removed.
  */
 static int string_list_add_note_lines(struct string_list *list,
- const unsigned char *sha1)
+ const struct object_id *oid)
 {
char *data;
unsigned long len;
enum object_type t;
 
-   if (is_null_sha1(sha1))
+   if (is_null_oid(oid))
return 0;
 
/* read_sha1_file NUL-terminates */
-   data = read_sha1_file(sha1, , );
+   data = read_sha1_file(oid->hash, , );
if (t != OBJ_BLOB || !data || !len) {
free(data);
return t != OBJ_BLOB || !data;
@@ -884,17 +884,17 @@ static int string_list_join_lines_helper(struct 
string_list_item *item,
return 0;
 }
 
-int combine_notes_cat_sort_uniq(unsigned char *cur_sha1,
-   const unsigned char *new_sha1)
+int combine_notes_cat_sort_uniq(struct object_id *cur_oid,
+   const struct object_id *new_oid)
 {
struct string_list sort_uniq_list = STRING_LIST_INIT_DUP;
struct strbuf buf = STRBUF_INIT;
int ret = 1;
 
/* read both note blob objects into unique_lines */
-   if (string_list_add_note_lines(_uniq_list, cur_sha1))
+   if (string_list_add_note_lines(_uniq_list, cur_oid))
goto out;
-   if (string_list_add_note_lines(_uniq_list, new_sha1))
+   if (string_list_add_note_lines(_uniq_list, new_oid))
goto out;
string_list_remove_empty_items(_uniq_list, 0);
string_list_sort(_uniq_list);

[PATCH v4 06/12] commit: convert commit_tree* to object_id

[Patryk Obara]

Convert the definitions and declarations of commit_tree and
commit_tree_extended to use struct object_id and adjust all usages of
these functions.

Signed-off-by: Patryk Obara 
---
 builtin/am.c  |  4 ++--
 builtin/commit-tree.c |  4 ++--
 builtin/commit.c  |  5 +++--
 builtin/merge.c   |  8 
 commit.c  | 15 +++
 commit.h  | 11 ++-
 notes-cache.c |  4 ++--
 notes-merge.c |  9 -
 notes-utils.c |  7 ---
 notes-utils.h |  3 ++-
 10 files changed, 36 insertions(+), 34 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index acfe9d3c8c..6e6abb05cd 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1641,8 +1641,8 @@ static void do_commit(const struct am_state *state)
setenv("GIT_COMMITTER_DATE",
state->ignore_date ? "" : state->author_date, 1);
 
-   if (commit_tree(state->msg, state->msg_len, tree.hash, parents, 
commit.hash,
-   author, state->sign_commit))
+   if (commit_tree(state->msg, state->msg_len, , parents, ,
+   author, state->sign_commit))
die(_("failed to write commit object"));
 
reflog_msg = getenv("GIT_REFLOG_ACTION");
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index 2177251e24..e5bdf57b1e 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -117,8 +117,8 @@ int cmd_commit_tree(int argc, const char **argv, const char 
*prefix)
die_errno("git commit-tree: failed to read");
}
 
-   if (commit_tree(buffer.buf, buffer.len, tree_oid.hash, parents,
-   commit_oid.hash, NULL, sign_commit)) {
+   if (commit_tree(buffer.buf, buffer.len, _oid, parents, _oid,
+   NULL, sign_commit)) {
strbuf_release();
return 1;
}
diff --git a/builtin/commit.c b/builtin/commit.c
index 4610e3d8e3..e5974a5999 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1794,8 +1794,9 @@ int cmd_commit(int argc, const char **argv, const char 
*prefix)
append_merge_tag_headers(parents, );
}
 
-   if (commit_tree_extended(sb.buf, sb.len, active_cache_tree->oid.hash,
-parents, oid.hash, author_ident.buf, sign_commit, 
extra)) {
+   if (commit_tree_extended(sb.buf, sb.len, _cache_tree->oid,
+parents, , author_ident.buf, sign_commit,
+extra)) {
rollback_index_files();
die(_("failed to write commit object"));
}
diff --git a/builtin/merge.c b/builtin/merge.c
index 30264cfd7c..92ba99a1a5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -820,8 +820,8 @@ static int merge_trivial(struct commit *head, struct 
commit_list *remoteheads)
pptr = commit_list_append(head, pptr);
pptr = commit_list_append(remoteheads->item, pptr);
prepare_to_commit(remoteheads);
-   if (commit_tree(merge_msg.buf, merge_msg.len, result_tree.hash, parents,
-   result_commit.hash, NULL, sign_commit))
+   if (commit_tree(merge_msg.buf, merge_msg.len, _tree, parents,
+   _commit, NULL, sign_commit))
die(_("failed to write commit object"));
finish(head, remoteheads, _commit, "In-index merge");
drop_save();
@@ -845,8 +845,8 @@ static int finish_automerge(struct commit *head,
commit_list_insert(head, );
strbuf_addch(_msg, '\n');
prepare_to_commit(remoteheads);
-   if (commit_tree(merge_msg.buf, merge_msg.len, result_tree->hash, 
parents,
-   result_commit.hash, NULL, sign_commit))
+   if (commit_tree(merge_msg.buf, merge_msg.len, result_tree, parents,
+   _commit, NULL, sign_commit))
die(_("failed to write commit object"));
strbuf_addf(, "Merge made by the '%s' strategy.", wt_strategy);
finish(head, remoteheads, _commit, buf.buf);
diff --git a/commit.c b/commit.c
index ff51c9f34a..643f3daec3 100644
--- a/commit.c
+++ b/commit.c
@@ -1380,9 +1380,8 @@ void free_commit_extra_headers(struct commit_extra_header 
*extra)
}
 }
 
-int commit_tree(const char *msg, size_t msg_len,
-   const unsigned char *tree,
-   struct commit_list *parents, unsigned char *ret,
+int commit_tree(const char *msg, size_t msg_len, const struct object_id *tree,
+   struct commit_list *parents, struct object_id *ret,
const char *author, const char *sign_commit)
 {
struct commit_extra_header *extra = NULL, **tail = 
@@ -1511,8 +1510,8 @@ N_("Warning: commit message did not conform to UTF-8.\n"
"variable i18n.commitencoding to the encoding your project uses.\n");
 
 int commit_tree_extended(const char *msg, size_t msg_len,
-const unsigned 

[PATCH v4 00/12] A bunch of object_id conversions

[Patryk Obara]

Thank you, everyone, for review and thanks, Junio for queueing two
first patches - I removed them from v4, as there were no more comments
to them anyway.

Compared to v3:

Patch 2 (formerly 4)
- renamed all parameter names sha1_stat to oid_stat
- renamed static function load sha1_stat to load_oid_stat
- adjusted a comment, that included "sha1_stat" in text

Patch 11 (formerly 13)
- fixed a typo in the commit message

Hopefully, this will be the last iteration on this batch of object_id
conversions; I already have next batch prepared (I just need to clean
it up before sending).

Patryk Obara (12):
  sha1_file: convert pretend_sha1_file to object_id
  dir: convert struct sha1_stat to use object_id
  sha1_file: convert hash_sha1_file to object_id
  cache: clear whole hash buffer with oidclr
  match-trees: convert splice_tree to object_id
  commit: convert commit_tree* to object_id
  notes: convert combine_notes_* to object_id
  notes: convert write_notes_tree to object_id
  sha1_file: convert write_sha1_file to object_id
  sha1_file: convert force_object_loose to object_id
  sha1_file: convert write_loose_object to object_id
  sha1_file: rename hash_sha1_file_literally

 Documentation/technical/api-object-access.txt |   2 +-
 apply.c   |  12 +--
 blame.c   |   2 +-
 builtin/am.c  |   4 +-
 builtin/checkout.c|   3 +-
 builtin/commit-tree.c |   4 +-
 builtin/commit.c  |   5 +-
 builtin/hash-object.c |   3 +-
 builtin/index-pack.c  |   5 +-
 builtin/merge.c   |   8 +-
 builtin/mktag.c   |   6 +-
 builtin/mktree.c  |  10 +--
 builtin/notes.c   |   8 +-
 builtin/pack-objects.c|   2 +-
 builtin/receive-pack.c|  11 +--
 builtin/replace.c |   4 +-
 builtin/tag.c |   2 +-
 builtin/unpack-objects.c  |  11 ++-
 cache-tree.c  |  16 ++--
 cache.h   |  25 +--
 commit.c  |  15 ++--
 commit.h  |  11 +--
 convert.c |   6 +-
 diffcore-rename.c |   4 +-
 dir.c | 104 +-
 dir.h |  12 +--
 log-tree.c|   2 +-
 match-trees.c |  46 ++--
 merge-recursive.c |   5 +-
 notes-cache.c |   8 +-
 notes-merge.c |   9 +--
 notes-utils.c |   9 ++-
 notes-utils.h |   3 +-
 notes.c   |  63 
 notes.h   |  29 ---
 read-cache.c  |   6 +-
 sha1_file.c   | 100 +
 t/helper/test-dump-untracked-cache.c  |   4 +-
 38 files changed, 300 insertions(+), 279 deletions(-)


base-commit: 5be1f00a9a701532232f57958efab4be8c959a29
-- 
2.14.3

[PATCH v4 03/12] sha1_file: convert hash_sha1_file to object_id

[Patryk Obara]

Convert the declaration and definition of hash_sha1_file to use
struct object_id and adjust all function calls.

Rename this function to hash_object_file.

Signed-off-by: Patryk Obara 
---
 apply.c  |  4 ++--
 builtin/index-pack.c |  5 ++---
 builtin/replace.c|  2 +-
 builtin/unpack-objects.c |  2 +-
 cache-tree.c | 11 +--
 cache.h  |  5 -
 convert.c|  6 +++---
 diffcore-rename.c|  4 ++--
 dir.c|  4 ++--
 log-tree.c   |  2 +-
 sha1_file.c  | 26 +-
 11 files changed, 36 insertions(+), 35 deletions(-)

diff --git a/apply.c b/apply.c
index 321a9fa68d..57ab8a8a29 100644
--- a/apply.c
+++ b/apply.c
@@ -3154,7 +3154,7 @@ static int apply_binary(struct apply_state *state,
 * See if the old one matches what the patch
 * applies to.
 */
-   hash_sha1_file(img->buf, img->len, blob_type, oid.hash);
+   hash_object_file(img->buf, img->len, blob_type, );
if (strcmp(oid_to_hex(), patch->old_sha1_prefix))
return error(_("the patch applies to '%s' (%s), "
   "which does not match the "
@@ -3199,7 +3199,7 @@ static int apply_binary(struct apply_state *state,
 name);
 
/* verify that the result matches */
-   hash_sha1_file(img->buf, img->len, blob_type, oid.hash);
+   hash_object_file(img->buf, img->len, blob_type, );
if (strcmp(oid_to_hex(), patch->new_sha1_prefix))
return error(_("binary patch to '%s' creates incorrect 
result (expecting %s, got %s)"),
name, patch->new_sha1_prefix, oid_to_hex());
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 4c51aec81f..7f5a95e6ff 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -958,9 +958,8 @@ static void resolve_delta(struct object_entry *delta_obj,
free(delta_data);
if (!result->data)
bad_object(delta_obj->idx.offset, _("failed to apply delta"));
-   hash_sha1_file(result->data, result->size,
-  typename(delta_obj->real_type),
-  delta_obj->idx.oid.hash);
+   hash_object_file(result->data, result->size,
+typename(delta_obj->real_type), _obj->idx.oid);
sha1_object(result->data, NULL, result->size, delta_obj->real_type,
_obj->idx.oid);
counter_lock();
diff --git a/builtin/replace.c b/builtin/replace.c
index 10078ae371..814bf6bfde 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -355,7 +355,7 @@ static void check_one_mergetag(struct commit *commit,
struct tag *tag;
int i;
 
-   hash_sha1_file(extra->value, extra->len, typename(OBJ_TAG), 
tag_oid.hash);
+   hash_object_file(extra->value, extra->len, typename(OBJ_TAG), _oid);
tag = lookup_tag(_oid);
if (!tag)
die(_("bad mergetag in commit '%s'"), ref);
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 62ea264c46..85a40d1af7 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -258,7 +258,7 @@ static void write_object(unsigned nr, enum object_type type,
} else {
struct object *obj;
int eaten;
-   hash_sha1_file(buf, size, typename(type), 
obj_list[nr].oid.hash);
+   hash_object_file(buf, size, typename(type), _list[nr].oid);
added_object(nr, type, buf, size);
obj = parse_object_buffer(_list[nr].oid, type, size, buf,
  );
diff --git a/cache-tree.c b/cache-tree.c
index e03e72c34a..6574eeb80d 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -400,15 +400,14 @@ static int update_one(struct cache_tree *it,
}
 
if (repair) {
-   unsigned char sha1[20];
-   hash_sha1_file(buffer.buf, buffer.len, tree_type, sha1);
-   if (has_sha1_file(sha1))
-   hashcpy(it->oid.hash, sha1);
+   struct object_id oid;
+   hash_object_file(buffer.buf, buffer.len, tree_type, );
+   if (has_sha1_file(oid.hash))
+   oidcpy(>oid, );
else
to_invalidate = 1;
} else if (dryrun)
-   hash_sha1_file(buffer.buf, buffer.len, tree_type,
-  it->oid.hash);
+   hash_object_file(buffer.buf, buffer.len, tree_type, >oid);
else if (write_sha1_file(buffer.buf, buffer.len, tree_type, 
it->oid.hash)) {
strbuf_release();
return -1;
diff --git a/cache.h b/cache.h
index ed72933ba7..08f2b81e1b 100644
--- a/cache.h
+++ b/cache.h
@@ -1236,7 +1236,10 @@ 

[PATCH v4 12/12] sha1_file: rename hash_sha1_file_literally

[Patryk Obara]

This function was already converted to use struct object_id earlier.

Signed-off-by: Patryk Obara 
---
 builtin/hash-object.c | 3 ++-
 cache.h   | 4 +++-
 sha1_file.c   | 5 +++--
 3 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index c532ff9320..526da5c185 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -24,7 +24,8 @@ static int hash_literally(struct object_id *oid, int fd, 
const char *type, unsig
if (strbuf_read(, fd, 4096) < 0)
ret = -1;
else
-   ret = hash_sha1_file_literally(buf.buf, buf.len, type, oid, 
flags);
+   ret = hash_object_file_literally(buf.buf, buf.len, type, oid,
+flags);
strbuf_release();
return ret;
 }
diff --git a/cache.h b/cache.h
index 0a8be9c87f..6ef4248931 100644
--- a/cache.h
+++ b/cache.h
@@ -1243,7 +1243,9 @@ extern int hash_object_file(const void *buf, unsigned 
long len,
 extern int write_object_file(const void *buf, unsigned long len,
 const char *type, struct object_id *oid);
 
-extern int hash_sha1_file_literally(const void *buf, unsigned long len, const 
char *type, struct object_id *oid, unsigned flags);
+extern int hash_object_file_literally(const void *buf, unsigned long len,
+ const char *type, struct object_id *oid,
+ unsigned flags);
 
 extern int pretend_object_file(void *, unsigned long, enum object_type,
   struct object_id *oid);
diff --git a/sha1_file.c b/sha1_file.c
index 59238f5bea..34c041e8cd 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1652,8 +1652,9 @@ int write_object_file(const void *buf, unsigned long len, 
const char *type,
return write_loose_object(oid, hdr, hdrlen, buf, len, 0);
 }
 
-int hash_sha1_file_literally(const void *buf, unsigned long len, const char 
*type,
-struct object_id *oid, unsigned flags)
+int hash_object_file_literally(const void *buf, unsigned long len,
+  const char *type, struct object_id *oid,
+  unsigned flags)
 {
char *header;
int hdrlen, status = 0;
-- 
2.14.3

[PATCH v4 05/12] match-trees: convert splice_tree to object_id

[Patryk Obara]

Convert the definition of static recursive splice_tree function to use
struct object_id and adjust single caller.

Signed-off-by: Patryk Obara 
---
 match-trees.c | 46 ++
 1 file changed, 22 insertions(+), 24 deletions(-)

diff --git a/match-trees.c b/match-trees.c
index 396b7338df..afb771c4f5 100644
--- a/match-trees.c
+++ b/match-trees.c
@@ -158,22 +158,20 @@ static void match_trees(const struct object_id *hash1,
 }
 
 /*
- * A tree "hash1" has a subdirectory at "prefix".  Come up with a
- * tree object by replacing it with another tree "hash2".
+ * A tree "oid1" has a subdirectory at "prefix".  Come up with a tree object by
+ * replacing it with another tree "oid2".
  */
-static int splice_tree(const unsigned char *hash1,
-  const char *prefix,
-  const unsigned char *hash2,
-  unsigned char *result)
+static int splice_tree(const struct object_id *oid1, const char *prefix,
+  const struct object_id *oid2, struct object_id *result)
 {
char *subpath;
int toplen;
char *buf;
unsigned long sz;
struct tree_desc desc;
-   unsigned char *rewrite_here;
-   const unsigned char *rewrite_with;
-   unsigned char subtree[20];
+   struct object_id *rewrite_here;
+   const struct object_id *rewrite_with;
+   struct object_id subtree;
enum object_type type;
int status;
 
@@ -182,9 +180,9 @@ static int splice_tree(const unsigned char *hash1,
if (*subpath)
subpath++;
 
-   buf = read_sha1_file(hash1, , );
+   buf = read_sha1_file(oid1->hash, , );
if (!buf)
-   die("cannot read tree %s", sha1_to_hex(hash1));
+   die("cannot read tree %s", oid_to_hex(oid1));
init_tree_desc(, buf, sz);
 
rewrite_here = NULL;
@@ -197,26 +195,26 @@ static int splice_tree(const unsigned char *hash1,
if (strlen(name) == toplen &&
!memcmp(name, prefix, toplen)) {
if (!S_ISDIR(mode))
-   die("entry %s in tree %s is not a tree",
-   name, sha1_to_hex(hash1));
-   rewrite_here = (unsigned char *) oid->hash;
+   die("entry %s in tree %s is not a tree", name,
+   oid_to_hex(oid1));
+   rewrite_here = (struct object_id *)oid;
break;
}
update_tree_entry();
}
if (!rewrite_here)
-   die("entry %.*s not found in tree %s",
-   toplen, prefix, sha1_to_hex(hash1));
+   die("entry %.*s not found in tree %s", toplen, prefix,
+   oid_to_hex(oid1));
if (*subpath) {
-   status = splice_tree(rewrite_here, subpath, hash2, subtree);
+   status = splice_tree(rewrite_here, subpath, oid2, );
if (status)
return status;
-   rewrite_with = subtree;
+   rewrite_with = 
+   } else {
+   rewrite_with = oid2;
}
-   else
-   rewrite_with = hash2;
-   hashcpy(rewrite_here, rewrite_with);
-   status = write_sha1_file(buf, sz, tree_type, result);
+   oidcpy(rewrite_here, rewrite_with);
+   status = write_sha1_file(buf, sz, tree_type, result->hash);
free(buf);
return status;
 }
@@ -280,7 +278,7 @@ void shift_tree(const struct object_id *hash1,
if (!*add_prefix)
return;
 
-   splice_tree(hash1->hash, add_prefix, hash2->hash, shifted->hash);
+   splice_tree(hash1, add_prefix, hash2, shifted);
 }
 
 /*
@@ -334,7 +332,7 @@ void shift_tree_by(const struct object_id *hash1,
 * shift tree2 down by adding shift_prefix above it
 * to match tree1.
 */
-   splice_tree(hash1->hash, shift_prefix, hash2->hash, 
shifted->hash);
+   splice_tree(hash1, shift_prefix, hash2, shifted);
else
/*
 * shift tree2 up by removing shift_prefix from it
-- 
2.14.3

[PATCH v4 01/12] sha1_file: convert pretend_sha1_file to object_id

[Patryk Obara]

Convert the declaration and definition of pretend_sha1_file to use
struct object_id and adjust all usages of this function.  Rename it to
pretend_object_file.

Signed-off-by: Patryk Obara 
---
 Documentation/technical/api-object-access.txt |  2 +-
 blame.c   |  2 +-
 cache.h   |  5 -
 sha1_file.c   | 10 +-
 4 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/Documentation/technical/api-object-access.txt 
b/Documentation/technical/api-object-access.txt
index 03bb0e950d..a1162e5bcd 100644
--- a/Documentation/technical/api-object-access.txt
+++ b/Documentation/technical/api-object-access.txt
@@ -7,7 +7,7 @@ Talk about  and  family, things like
 * read_object_with_reference()
 * has_sha1_file()
 * write_sha1_file()
-* pretend_sha1_file()
+* pretend_object_file()
 * lookup_{object,commit,tag,blob,tree}
 * parse_{object,commit,tag,blob,tree}
 * Use of object flags
diff --git a/blame.c b/blame.c
index 2893f3c103..1fc22b304b 100644
--- a/blame.c
+++ b/blame.c
@@ -232,7 +232,7 @@ static struct commit *fake_working_tree_commit(struct 
diff_options *opt,
convert_to_git(_index, path, buf.buf, buf.len, , 0);
origin->file.ptr = buf.buf;
origin->file.size = buf.len;
-   pretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_oid.hash);
+   pretend_object_file(buf.buf, buf.len, OBJ_BLOB, >blob_oid);
 
/*
 * Read the current index, replace the path entry with
diff --git a/cache.h b/cache.h
index d8b975a571..e4e03ac51d 100644
--- a/cache.h
+++ b/cache.h
@@ -1241,7 +1241,10 @@ extern int sha1_object_info(const unsigned char *, 
unsigned long *);
 extern int hash_sha1_file(const void *buf, unsigned long len, const char 
*type, unsigned char *sha1);
 extern int write_sha1_file(const void *buf, unsigned long len, const char 
*type, unsigned char *return_sha1);
 extern int hash_sha1_file_literally(const void *buf, unsigned long len, const 
char *type, struct object_id *oid, unsigned flags);
-extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned 
char *);
+
+extern int pretend_object_file(void *, unsigned long, enum object_type,
+  struct object_id *oid);
+
 extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
diff --git a/sha1_file.c b/sha1_file.c
index 3da70ac650..830b93b428 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1312,13 +1312,13 @@ static void *read_object(const unsigned char *sha1, 
enum object_type *type,
return content;
 }
 
-int pretend_sha1_file(void *buf, unsigned long len, enum object_type type,
- unsigned char *sha1)
+int pretend_object_file(void *buf, unsigned long len, enum object_type type,
+   struct object_id *oid)
 {
struct cached_object *co;
 
-   hash_sha1_file(buf, len, typename(type), sha1);
-   if (has_sha1_file(sha1) || find_cached_object(sha1))
+   hash_sha1_file(buf, len, typename(type), oid->hash);
+   if (has_sha1_file(oid->hash) || find_cached_object(oid->hash))
return 0;
ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
co = _objects[cached_object_nr++];
@@ -1326,7 +1326,7 @@ int pretend_sha1_file(void *buf, unsigned long len, enum 
object_type type,
co->type = type;
co->buf = xmalloc(len);
memcpy(co->buf, buf, len);
-   hashcpy(co->sha1, sha1);
+   hashcpy(co->sha1, oid->hash);
return 0;
 }
 
-- 
2.14.3

[PATCH v4 08/12] notes: convert write_notes_tree to object_id

[Patryk Obara]

Convert the definition and declaration of write_notes_tree to
struct object_id and adjust usage of this function.

Additionally, improve style of small part of this function, as old
formatting made it hard to understand at glance what this part of
code is doing.

Signed-off-by: Patryk Obara 
---
 notes-cache.c |  2 +-
 notes-utils.c |  2 +-
 notes.c   | 16 +---
 notes.h   |  4 ++--
 4 files changed, 13 insertions(+), 11 deletions(-)

diff --git a/notes-cache.c b/notes-cache.c
index d2f87147cc..010ad236cb 100644
--- a/notes-cache.c
+++ b/notes-cache.c
@@ -54,7 +54,7 @@ int notes_cache_write(struct notes_cache *c)
if (!c->tree.dirty)
return 0;
 
-   if (write_notes_tree(>tree, tree_oid.hash))
+   if (write_notes_tree(>tree, _oid))
return -1;
if (commit_tree(c->validity, strlen(c->validity), _oid, NULL,
_oid, NULL, NULL) < 0)
diff --git a/notes-utils.c b/notes-utils.c
index 058c642dac..02407fe2a7 100644
--- a/notes-utils.c
+++ b/notes-utils.c
@@ -12,7 +12,7 @@ void create_notes_commit(struct notes_tree *t, struct 
commit_list *parents,
 
assert(t->initialized);
 
-   if (write_notes_tree(t, tree_oid.hash))
+   if (write_notes_tree(t, _oid))
die("Failed to write notes tree to database");
 
if (!parents) {
diff --git a/notes.c b/notes.c
index 3f4f94507a..09ef1ce33a 100644
--- a/notes.c
+++ b/notes.c
@@ -1123,11 +1123,12 @@ int for_each_note(struct notes_tree *t, int flags, 
each_note_fn fn,
return for_each_note_helper(t, t->root, 0, 0, flags, fn, cb_data);
 }
 
-int write_notes_tree(struct notes_tree *t, unsigned char *result)
+int write_notes_tree(struct notes_tree *t, struct object_id *result)
 {
struct tree_write_stack root;
struct write_each_note_data cb_data;
int ret;
+   int flags;
 
if (!t)
t = _notes_tree;
@@ -1141,12 +1142,13 @@ int write_notes_tree(struct notes_tree *t, unsigned 
char *result)
cb_data.next_non_note = t->first_non_note;
 
/* Write tree objects representing current notes tree */
-   ret = for_each_note(t, FOR_EACH_NOTE_DONT_UNPACK_SUBTREES |
-   FOR_EACH_NOTE_YIELD_SUBTREES,
-   write_each_note, _data) ||
-   write_each_non_note_until(NULL, _data) ||
-   tree_write_stack_finish_subtree() ||
-   write_sha1_file(root.buf.buf, root.buf.len, tree_type, result);
+   flags = FOR_EACH_NOTE_DONT_UNPACK_SUBTREES |
+   FOR_EACH_NOTE_YIELD_SUBTREES;
+   ret = for_each_note(t, flags, write_each_note, _data) ||
+ write_each_non_note_until(NULL, _data) ||
+ tree_write_stack_finish_subtree() ||
+ write_sha1_file(root.buf.buf, root.buf.len, tree_type,
+ result->hash);
strbuf_release();
return ret;
 }
diff --git a/notes.h b/notes.h
index 88da38b5f4..0433f45db5 100644
--- a/notes.h
+++ b/notes.h
@@ -217,7 +217,7 @@ int for_each_note(struct notes_tree *t, int flags, 
each_note_fn fn,
  * Write the given notes_tree structure to the object database
  *
  * Creates a new tree object encapsulating the current state of the given
- * notes_tree, and stores its SHA1 into the 'result' argument.
+ * notes_tree, and stores its object id into the 'result' argument.
  *
  * Returns zero on success, non-zero on failure.
  *
@@ -225,7 +225,7 @@ int for_each_note(struct notes_tree *t, int flags, 
each_note_fn fn,
  * this function has returned zero. Please also remember to create a
  * corresponding commit object, and update the appropriate notes ref.
  */
-int write_notes_tree(struct notes_tree *t, unsigned char *result);
+int write_notes_tree(struct notes_tree *t, struct object_id *result);
 
 /* Flags controlling the operation of prune */
 #define NOTES_PRUNE_VERBOSE 1
-- 
2.14.3

git send-email sets date

[Michal Suchánek]

Hello,

git send-email sets the message date to author date.

This is wrong because the message will most likely not get delivered
when the author date differs from current time. It might give slightly
better results with commit date instead of author date but can't is
just skip that header and leave it to the mailer?

It does not even seem to have an option to suppress adding the date
header.

Thanks

Michal

Re: Cloned repository has file changes -> bug?

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 27 2018, Filip Jorissen jotted:

> I think our git repository is bugged. The reason why I say this is the
> following. When cloning the repository, the newly cloned repository
> immediately has file changes[...].

If you run this:

git ls-files | tr '[:upper:]' '[:lower:]' | sort | uniq -D | grep '^'

You'll see that the reason is that you have files that differ only in
case.

You are using a Mac, and Macs by default think that files that are
different binary strings are the same file, since they don't consider
case to be relevant. The file FOO, foo and FoO and fOo are all the same
file as far as your Mac is concerned, but would be 4 different files on
Linux.

> How can I fix the repository?

You could check it out on a OS that considers files that differ in case
to be different files, e.g. on Linux, move them around, push it, and new
clones should work on your Mac.

Alternatively I hear that you can create a loopback case-sensitive FS
image on Macs.

Cloned repository has file changes -> bug?

[Filip Jorissen]

Dear all,

I think our git repository is bugged. The reason why I say this is the 
following. When cloning the repository, the newly cloned repository immediately 
has file changes. Steps to reproduce and illustration is at the end of this 
email. Git checkout does not work to remove the file changes. This behavior 
seems to be reproducible across multiple computers. Is this a bug? How can I 
fix the repository?

Thanks in advance for the support!

Filip


MacBook-Pro-van-Filip:git filip$ git clone g...@github.com:open-ideas/IDEAS.git
Cloning into 'IDEAS'...
remote: Counting objects: 48419, done.
remote: Compressing objects: 100% (198/198), done.
remote: Total 48419 (delta 134), reused 217 (delta 97), pack-reused 48108
Receiving objects: 100% (48419/48419), 42.32 MiB | 1.81 MiB/s, done.
Resolving deltas: 100% (32847/32847), done.
MacBook-Pro-van-Filip:git filip$ cd IDEAS
MacBook-Pro-van-Filip:IDEAS filip$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add ..." to update what will be committed)
  (use "git checkout -- ..." to discard changes in working directory)

modified:   
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
modified:   
IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Utilities_Psychrometrics_Functions_Examples_SaturationPressure.txt

no changes added to commit (use "git add" and/or "git commit -a")

Re: Some rough edges of core.fsmonitor

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 27 2018, Duy Nguyen jotted:

> On Sat, Jan 27, 2018 at 07:39:27PM +0700, Duy Nguyen wrote:
>> On Sat, Jan 27, 2018 at 6:43 PM, Ævar Arnfjörð Bjarmason
>>  wrote:
>> > a) no fsmonitor
>> >
>> > $ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
>> > 12:32:44.947651 read-cache.c:1890   performance: 0.053153609 s: 
>> > read cache .git/index
>> > 12:32:44.967943 preload-index.c:112 performance: 0.020161093 s: 
>> > preload index
>> > 12:32:44.974217 read-cache.c:1446   performance: 0.006230611 s: 
>> > refresh index
>> >
>> > ...
>> >
>> > b) with fsmonitor
>> >
>> > $ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
>> > 12:34:23.833625 read-cache.c:1890   performance: 0.049485685 s: 
>> > read cache .git/index
>> > 12:34:23.838622 preload-index.c:112 performance: 0.001221197 s: 
>> > preload index
>> > 12:34:23.858723 fsmonitor.c:170 performance: 0.020059647 s: 
>> > fsmonitor process '.git/hooks/fsmonitor-watchman'
>> > 12:34:23.871532 read-cache.c:1446   performance: 0.032870818 s: 
>> > refresh index
>>
>> Hmm.. why does refresh take longer with fsmonitor/watchman? With the
>> help from watchman, we know what files are modified. We don't need
>> manual stat()'ing and this line should be lower than the "no
>> fsmonitor" case, which is 0.006230611s.
>
> Ahh.. my patch probably does not see that fsmonitor could be activated
> lazily inside refresh_index() call. The patch below should fix it.

Will have to get those numbers to you later, or alternatively clone
https://github.com/avar/2015-04-03-1M-git (or some other test repo) and
test it yourself, sorry. Don't have time to follow-up much this weekend.

> But between your normal refresh time (0.020 preload + 0.006 actual
> refresh) and fsmonitor taking 0.020 just to talk to watchman, this
> repo seems "too small" for fsmonitor/watchman to shine.

Surely that's an implementation limitation and not something inherent,
given that watchman itself returns in 5ms?

I.e. status could work like this, no?:

 1. At start, record the timestamp & find out canonical state via some
expansive method.
 2. Print out xyz changed, abc added etc.
 3. Record *just* what status would report about xyz, abc etc.
 4. On subsequent git status, just amend that information, e.g. if
watchman says nothing changed $(cat .git/last-status-output).

We shouldn't need to be reading the entire index in the common case
where just a few things change.

There's also a lot of things that use status to just check "are we
clean?", those would only need to record the last known timestamp when
the tree was clean, and then ask watchman if there were any changes, if
not we're done.

> I'm still a bit curious that refresh index time, after excluding 0.020
> for fsmonitor, is stil 0.012s. What does it do? It should really be
> doing nothing. Either way, read index time seems to be the elephant in
> the room now.
>
> -- 8< --
> diff --git a/read-cache.c b/read-cache.c
> index eac74bc9f1..d60e0a8480 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -1367,12 +1367,21 @@ int refresh_index(struct index_state *istate, 
> unsigned int flags,
>   unsigned int options = (CE_MATCH_REFRESH |
>   (really ? CE_MATCH_IGNORE_VALID : 0) |
>   (not_new ? CE_MATCH_IGNORE_MISSING : 0));
> + int ignore_fsmonitor = options & CE_MATCH_IGNORE_FSMONITOR;
>   const char *modified_fmt;
>   const char *deleted_fmt;
>   const char *typechange_fmt;
>   const char *added_fmt;
>   const char *unmerged_fmt;
> - uint64_t start = getnanotime();
> + uint64_t start;
> +
> + /*
> +  * If fsmonitor is used, force its communication early to
> +  * accurately measure how long this function takes without it.
> +  */
> + if (!ignore_fsmonitor)
> + refresh_fsmonitor(istate);
> + start = getnanotime();
>
>   modified_fmt = (in_porcelain ? "M\t%s\n" : "%s: needs update\n");
>   deleted_fmt = (in_porcelain ? "D\t%s\n" : "%s: needs update\n");
> -- 8< --

[PATCH v2] daemon: add --send-log-to=(stderr|syslog|none)

[Lucas Werkmeister]

This makes it possible to use --inetd while still logging to standard
error. --syslog is retained as an alias for --send-log-to=syslog. A mode
to disable logging explicitly is also provided.

The combination of --inetd with --send-log-to=stderr is useful, for
instance, when running `git daemon` as an instanced systemd service
(with associated socket unit). In this case, log messages sent via
syslog are received by the journal daemon, but run the risk of being
processed at a time when the `git daemon` process has already exited
(especially if the process was very short-lived, e.g. due to client
error), so that the journal daemon can no longer read its cgroup and
attach the message to the correct systemd unit (see systemd/systemd#2913
[1]). Logging to stderr instead can solve this problem, because systemd
can connect stderr directly to the journal daemon, which then already
knows which unit is associated with this stream.

[1]: https://github.com/systemd/systemd/issues/2913

Signed-off-by: Lucas Werkmeister 
Helped-by: Ævar Arnfjörð Bjarmason 
Helped-by: Junio C Hamano 
---

Notes:
This was originally “daemon: add --no-syslog to undo implicit
--syslog”, but Junio pointed out that combining --no-syslog with
--detach isn’t especially useful and suggested --send-log-to=
instead. Is Helped-by: the right credit for this or should it be
something else?

I’m also not quite sure if the systemd part of the commit message is
accurate – see my comment on the linked issue [2]. TL;DR: this might
no longer be necessary on systemd v235. (I’m experiencing the
problem on Debian Stretch, systemd v232.) As in the last patch, feel
free to remove that part of the commit message.

[2]: https://github.com/systemd/systemd/issues/2913#issuecomment-361002589

 Documentation/git-daemon.txt | 23 +--
 daemon.c | 43 ---
 2 files changed, 57 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-daemon.txt b/Documentation/git-daemon.txt
index 3c91db7be..e973f4390 100644
--- a/Documentation/git-daemon.txt
+++ b/Documentation/git-daemon.txt
@@ -20,6 +20,7 @@ SYNOPSIS
 [--inetd |
  [--listen=] [--port=]
  [--user= [--group=]]]
+[--send-log-to=(stderr|syslog|none)]
 [...]
 
 DESCRIPTION
@@ -110,8 +111,26 @@ OPTIONS
zero for no limit.
 
 --syslog::
-   Log to syslog instead of stderr. Note that this option does not imply
-   --verbose, thus by default only error conditions will be logged.
+   Short for `--send-log-to=syslog`.
+
+--send-log-to=::
+   Send log messages to the specified destination.
+   Note that this option does not imply --verbose,
+   thus by default only error conditions will be logged.
+   The  defaults to `stderr`, and must be one of:
++
+--
+stderr::
+   Write to standard error.
+   Note that if `--detach` is specified,
+   the process disconnects from the real standard error,
+   making this destination effectively equivalent to `none`.
+syslog::
+   Write to syslog, using the `git-daemon` identifier.
+none::
+   Disable all logging.
+--
++
 
 --user-path::
 --user-path=::
diff --git a/daemon.c b/daemon.c
index e37e343d0..3d8e16ede 100644
--- a/daemon.c
+++ b/daemon.c
@@ -9,7 +9,11 @@
 #define initgroups(x, y) (0) /* nothing */
 #endif
 
-static int log_syslog;
+static enum log_destination {
+   LOG_TO_NONE = -1,
+   LOG_TO_STDERR = 0,
+   LOG_TO_SYSLOG = 1,
+} log_destination;
 static int verbose;
 static int reuseaddr;
 static int informative_errors;
@@ -25,6 +29,7 @@ static const char daemon_usage[] =
 "   [--access-hook=]\n"
 "   [--inetd | [--listen=] [--port=]\n"
 "  [--detach] [--user= [--group=]]\n"
+"   [--send-log-to=(stderr|syslog|none)]\n"
 "   [...]";
 
 /* List of acceptable pathname prefixes */
@@ -74,11 +79,14 @@ static const char *get_ip_address(struct hostinfo *hi)
 
 static void logreport(int priority, const char *err, va_list params)
 {
-   if (log_syslog) {
+   switch (log_destination) {
+   case LOG_TO_SYSLOG: {
char buf[1024];
vsnprintf(buf, sizeof(buf), err, params);
syslog(priority, "%s", buf);
-   } else {
+   break;
+   }
+   case LOG_TO_STDERR: {
/*
 * Since stderr is set to buffered mode, the
 * logging of different processes will not overlap
@@ -88,6 +96,11 @@ static void logreport(int priority, const char *err, va_list 
params)
vfprintf(stderr, err, params);
fputc('\n', stderr);
fflush(stderr);
+   break;
+   }
+   case LOG_TO_NONE: {
+   break;
+   }
}
 }
 
@@ -1289,7 +1302,7 @@ int 

lucky winner

[Mr Johnson Paul]

Wow!!! Congratulation

It is a great pleasure to inform you that you emerge winner to the sum
of 1,850,000.00 Euro

For more inquiries contact: Mr Johnson Paul trough his Email 
address:johnsonop...@gmail.com

Yours faithfully,
Online Coordinator's

Re: [PATCH v2 0/2] wrap format-patch diffstats around 72 columns

[Jeff King]

On Thu, Jan 25, 2018 at 06:59:25PM +0700, Nguyễn Thái Ngọc Duy wrote:

> Like v1, these changes keep diffstat generated by format-patch in 72
> columns. This constant is already used in the code, so it's a bit
> better than my random "70 or 75" value.
> 
> Granted these hard coded values (both 80 and 72) are not really nice.
> But I would wait for somebody to say "I need or want this" before I
> add code to make the default configurable.
> 
> Nguyễn Thái Ngọc Duy (2):
>   format-patch: keep cover-letter diffstat wrapped in 72 columns
>   format-patch: reduce patch diffstat width to 72

This looks OK to me. There was one head-scratcher in the second patch
which I mentioned, but I think it's probably OK.

I also notice that we have no tests for diffstat on a cover letter,
which might be worth addressing.

-Peff

Re: [PATCH v2 2/2] format-patch: reduce patch diffstat width to 72

[Jeff King]

On Thu, Jan 25, 2018 at 06:59:27PM +0700, Nguyễn Thái Ngọc Duy wrote:

> diff --git a/t/t4052-stat-output.sh b/t/t4052-stat-output.sh
> index 9f563db20a..1e62333b46 100755
> --- a/t/t4052-stat-output.sh
> +++ b/t/t4052-stat-output.sh
> @@ -60,7 +60,7 @@ do
>   test_cmp expect actual
>   '
>  done <<\EOF
> -format-patch -1 --stdout
> +format-patch --stat=80 -1 --stdout
>  diff HEAD^ HEAD --stat
>  show --stat
>  log -1 --stat

This hunk confused me. I think what is going on is this:

  - we have a loop that runs the same test on several commands

  - that loop expects format-patch, diff, etc, to have the same output

  - now that format-patch differs from the other commands in its default
length, we need to use a manual --stat-width to get identical output

It seems like that kind of nullifies the point of some of the tests in
the loop, though, since they are meant to check the behavior without
--stat.

OTOH, I think that case is tested later (in the other tests you
adjusted). So I guess these tests are just covering the "name vs bar
length" part?

-Peff

Re: [ANNOUNCE] Git Merge Contributor's Summit Mar 7, 2018, Barcelona

[Lars Schneider]

Hi Peff,

I would like to register to the contributor summit :-)

---

As I am writing you, I thought I could ask you a question:

"git verify-pack" tells me the "size-in-packfile" which is
kind of the "real" size of a file in a Git repo. Are you 
aware of a way to get this number via the GitHub API?

We have written a GitHub bot (soon to be open sourced!) that
warns about large files. We query the file size via Repo Content
API [1]. Largish text files usually compress well and therefore
our bot generates false positive warnings.

I assume it is not possible to query "size-in-packfile" via GitHub
API as this is kind of an internal and not necessarily stable
number. But I thought maybe you happen to know some way!

Thanks,
Lars



[1] https://developer.github.com/v3/repos/contents/#get-content

PS: In my last email I asked you about AsciiDoc rendering via *.adoc
extension on GitHub. Your argument that Git has custom AsciiDoc 
configs and attributes convinced me to not propose that idea on 
the list.


> On 19 Jan 2018, at 01:10, Jeff King  wrote:
> 
> Git Merge 2018 is happening on March 8th; there will be a Contributor's
> Summit the day before. Here are the details:
> 
>  When: Wednesday, March 7, 2018. 10am-5pm.
>  Where: Convent Dels Àngels[1], Barcelona, Spain
>  What: Round-table discussion about Git
>  Who: All contributors to Git or related projects in the Git ecosystem
>   are invited; if you're not sure if you qualify, just ask!
> 
> In order to attend, you'll need to register ahead of time. There's a
> super-secret link to do so; email me and I will provide it.
> Registration is free, and comes with a ticket to the main conference on
> the 8th (which I encourage you to attend, but you don't have to).
> 
> As with past years, the agenda is whatever we choose. We'll have room
> for about 25 people with "boardroom-style seating" and a projector.
> Come prepared with topics to present or discuss.
> 
> If you're interested in financial aid for traveling to the conference,
> please send an email to g...@sfconservancy.org.  And please do so soon
> (let's say by the end of next week, Jan 26th), so that we have an idea
> of the number and size of requests before making any grants.
> 
> -Peff
> 
> [1] 
> https://www.google.com/maps/place/Convent+Dels+Angels/@41.3827189,2.1652982,17z/data=!3m1!4b1!4m5!3m4!1s0x12a4a310123f3dc1:0x4588a81b66dce9dc!8m2!3d41.3827189!4d2.1674869

Re: Some rough edges of core.fsmonitor

[Duy Nguyen]

On Sat, Jan 27, 2018 at 07:39:27PM +0700, Duy Nguyen wrote:
> On Sat, Jan 27, 2018 at 6:43 PM, Ævar Arnfjörð Bjarmason
>  wrote:
> > a) no fsmonitor
> >
> > $ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
> > 12:32:44.947651 read-cache.c:1890   performance: 0.053153609 s: 
> > read cache .git/index
> > 12:32:44.967943 preload-index.c:112 performance: 0.020161093 s: 
> > preload index
> > 12:32:44.974217 read-cache.c:1446   performance: 0.006230611 s: 
> > refresh index
> >
> > ...
> >
> > b) with fsmonitor
> >
> > $ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
> > 12:34:23.833625 read-cache.c:1890   performance: 0.049485685 s: 
> > read cache .git/index
> > 12:34:23.838622 preload-index.c:112 performance: 0.001221197 s: 
> > preload index
> > 12:34:23.858723 fsmonitor.c:170 performance: 0.020059647 s: 
> > fsmonitor process '.git/hooks/fsmonitor-watchman'
> > 12:34:23.871532 read-cache.c:1446   performance: 0.032870818 s: 
> > refresh index
> 
> Hmm.. why does refresh take longer with fsmonitor/watchman? With the
> help from watchman, we know what files are modified. We don't need
> manual stat()'ing and this line should be lower than the "no
> fsmonitor" case, which is 0.006230611s.

Ahh.. my patch probably does not see that fsmonitor could be activated
lazily inside refresh_index() call. The patch below should fix it.

But between your normal refresh time (0.020 preload + 0.006 actual
refresh) and fsmonitor taking 0.020 just to talk to watchman, this
repo seems "too small" for fsmonitor/watchman to shine.

I'm still a bit curious that refresh index time, after excluding 0.020
for fsmonitor, is stil 0.012s. What does it do? It should really be
doing nothing. Either way, read index time seems to be the elephant in
the room now.

-- 8< --
diff --git a/read-cache.c b/read-cache.c
index eac74bc9f1..d60e0a8480 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1367,12 +1367,21 @@ int refresh_index(struct index_state *istate, unsigned 
int flags,
unsigned int options = (CE_MATCH_REFRESH |
(really ? CE_MATCH_IGNORE_VALID : 0) |
(not_new ? CE_MATCH_IGNORE_MISSING : 0));
+   int ignore_fsmonitor = options & CE_MATCH_IGNORE_FSMONITOR;
const char *modified_fmt;
const char *deleted_fmt;
const char *typechange_fmt;
const char *added_fmt;
const char *unmerged_fmt;
-   uint64_t start = getnanotime();
+   uint64_t start;
+
+   /*
+* If fsmonitor is used, force its communication early to
+* accurately measure how long this function takes without it.
+*/
+   if (!ignore_fsmonitor)
+   refresh_fsmonitor(istate);
+   start = getnanotime();
 
modified_fmt = (in_porcelain ? "M\t%s\n" : "%s: needs update\n");
deleted_fmt = (in_porcelain ? "D\t%s\n" : "%s: needs update\n");
-- 8< --

Re: Feature request: Improve diff algorithm

[KES]

One yet more:

@@ -43,22 +44,25 @@ sub tariff_title {
 1;
 
 __DATA__
-@@ control/tariff.css
-* {
-margin: 0;
-padding: 0;
-border: 0;
--webkit-box-sizing: border-box;
-box-sizing: border-box; }
-html {
-background-color: #121212;
-color: white;
-font-family: 'Roboto', 'Arial',  sans-serif;
-font-size: 16px; }
-a {
-cursor: pointer; }
 
 
+@@ control/tariff_about_old.html.ep
+
+  
+
+<%= $title %>
+  
+  
+<%== $option1 %><%= stash->{ comment1 }? " (" .stash->{ comment1 
}.")": '' %>
+  
+  
+
+<%== $option2 %><%= stash->{ comment2 }? " (" .stash->{ comment2 
}.")": '' %>
+
+
+@@ control/tariff.css
 /*  BASE BUTTON FOR TARIFF CARD  */
 .button {
   display: -webkit-box;

But it would be better if `@@ control/tariff.css` were untouched:

@@ -43,22 +44,25 @@ sub tariff_title {
 1;
 
 __DATA__
+
+
+@@ control/tariff_about_old.html.ep
+
+  
+
+<%= $title %>
+  
+  
+<%== $option1 %><%= stash->{ comment1 }? " (" .stash->{ comment1 
}.")": '' %>
+  
+  
+
+<%== $option2 %><%= stash->{ comment2 }? " (" .stash->{ comment2 
}.")": '' %>
+
+
 @@ control/tariff.css
-* {
-margin: 0;
-padding: 0;
-border: 0;
--webkit-box-sizing: border-box;
-box-sizing: border-box; }
-html {
-background-color: #121212;
-color: white;
-font-family: 'Roboto', 'Arial',  sans-serif;
-font-size: 16px; }
-a {
-cursor: pointer; }
-
-
 /*  BASE BUTTON FOR TARIFF CARD  */
 .button {
   display: -webkit-box;

Re: Some rough edges of core.fsmonitor

[Duy Nguyen]

On Sat, Jan 27, 2018 at 6:43 PM, Ævar Arnfjörð Bjarmason
 wrote:
> a) no fsmonitor
>
> $ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
> 12:32:44.947651 read-cache.c:1890   performance: 0.053153609 s: read 
> cache .git/index
> 12:32:44.967943 preload-index.c:112 performance: 0.020161093 s: 
> preload index
> 12:32:44.974217 read-cache.c:1446   performance: 0.006230611 s: 
> refresh index
>
> ...
>
> b) with fsmonitor
>
> $ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
> 12:34:23.833625 read-cache.c:1890   performance: 0.049485685 s: read 
> cache .git/index
> 12:34:23.838622 preload-index.c:112 performance: 0.001221197 s: 
> preload index
> 12:34:23.858723 fsmonitor.c:170 performance: 0.020059647 s: 
> fsmonitor process '.git/hooks/fsmonitor-watchman'
> 12:34:23.871532 read-cache.c:1446   performance: 0.032870818 s: 
> refresh index

Hmm.. why does refresh take longer with fsmonitor/watchman? With the
help from watchman, we know what files are modified. We don't need
manual stat()'ing and this line should be lower than the "no
fsmonitor" case, which is 0.006230611s.

> 12:34:23.876427 diff-lib.c:250  performance: 0.004731427 s: 
> diff-files
> 12:34:23.880669 diff-lib.c:527  performance: 0.003944422 s: 
> diff-index
> 12:34:23.899225 dir.c:2290  performance: 0.018509066 s: read 
> directory
> 12:34:23.901914 trace.c:417 performance: 0.118250995 s: git 
> command: '/home/aearnfjord/g/git/git-status'

I don't see any "write index" line here, which is interesting since
your case c) is about "don't write index".

> c) with fsmonitor + don't write index
-- 
Duy

[PATCH v2] trace: measure where the time is spent in the index-heavy operations

[Nguyễn Thái Ngọc Duy]

All the known heavy code blocks are measured (except object database
access). This should help identify if an optimization is effective or
not. An unoptimized git-status would give something like below:

0.001791141 s: read cache ...
0.004011363 s: preload index
0.000516161 s: refresh index
0.003139257 s: git command: ... 'status' '--porcelain=2'
0.006788129 s: diff-files
0.002090267 s: diff-index
0.001885735 s: initialize name hash
0.032013138 s: read directory
0.051781209 s: git command: './git' 'status'

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 > Would it be worth doing this on top of tg/split-index-fixes?  OTOH
 > this will only give a wrong output when tracing performance is on, and
 > it should be easy enough to figure out where the sharedindex actually
 > is.  So it might be better to keep this separate, and then just add a
 > patch on top for fixing the path later, which might be less work for
 > Junio.

 I updated the patch a bit to avoid git_path(). A merge on 'pu' still
 conflicts, but it's much easier to resolve by making sure free() is
 called after the trace_performance_since() line in read_index_from().

 It's technically dangerous to re-use base_path again this way, too
 far away from its assignment since 4 other git_path() calls may have
 been done and changed base_path value. But since tg/split-index-fixes
 should enter 'master' eventually and make it safe to re-use
 base_path, I think it's ok.

 diff-lib.c  | 4 
 dir.c   | 2 ++
 name-hash.c | 3 +++
 preload-index.c | 2 ++
 read-cache.c| 7 +++
 5 files changed, 18 insertions(+)

diff --git a/diff-lib.c b/diff-lib.c
index 8104603a3b..a228e1a219 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -92,6 +92,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)
int diff_unmerged_stage = revs->max_count;
unsigned ce_option = ((option & DIFF_RACY_IS_MODIFIED)
  ? CE_MATCH_RACY_IS_DIRTY : 0);
+   uint64_t start = getnanotime();
 
diff_set_mnemonic_prefix(>diffopt, "i/", "w/");
 
@@ -246,6 +247,7 @@ int run_diff_files(struct rev_info *revs, unsigned int 
option)
}
diffcore_std(>diffopt);
diff_flush(>diffopt);
+   trace_performance_since(start, "diff-files");
return 0;
 }
 
@@ -512,6 +514,7 @@ static int diff_cache(struct rev_info *revs,
 int run_diff_index(struct rev_info *revs, int cached)
 {
struct object_array_entry *ent;
+   uint64_t start = getnanotime();
 
ent = revs->pending.objects;
if (diff_cache(revs, >item->oid, ent->name, cached))
@@ -521,6 +524,7 @@ int run_diff_index(struct rev_info *revs, int cached)
diffcore_fix_diff_index(>diffopt);
diffcore_std(>diffopt);
diff_flush(>diffopt);
+   trace_performance_since(start, "diff-index");
return 0;
 }
 
diff --git a/dir.c b/dir.c
index 7c4b45e30e..4479a02a49 100644
--- a/dir.c
+++ b/dir.c
@@ -2248,6 +2248,7 @@ int read_directory(struct dir_struct *dir, struct 
index_state *istate,
   const char *path, int len, const struct pathspec *pathspec)
 {
struct untracked_cache_dir *untracked;
+   uint64_t start = getnanotime();
 
if (has_symlink_leading_path(path, len))
return dir->nr;
@@ -2286,6 +2287,7 @@ int read_directory(struct dir_struct *dir, struct 
index_state *istate,
dir->nr = i;
}
 
+   trace_performance_since(start, "read directory %.*s", len, path);
if (dir->untracked) {
static struct trace_key trace_untracked_stats = 
TRACE_KEY_INIT(UNTRACKED_STATS);
trace_printf_key(_untracked_stats,
diff --git a/name-hash.c b/name-hash.c
index 45c98db0a0..ada66f066a 100644
--- a/name-hash.c
+++ b/name-hash.c
@@ -578,6 +578,8 @@ static void threaded_lazy_init_name_hash(
 
 static void lazy_init_name_hash(struct index_state *istate)
 {
+   uint64_t start = getnanotime();
+
if (istate->name_hash_initialized)
return;
hashmap_init(>name_hash, cache_entry_cmp, NULL, 
istate->cache_nr);
@@ -600,6 +602,7 @@ static void lazy_init_name_hash(struct index_state *istate)
}
 
istate->name_hash_initialized = 1;
+   trace_performance_since(start, "initialize name hash");
 }
 
 /*
diff --git a/preload-index.c b/preload-index.c
index 2a83255e4e..4d08d44874 100644
--- a/preload-index.c
+++ b/preload-index.c
@@ -78,6 +78,7 @@ static void preload_index(struct index_state *index,
 {
int threads, i, work, offset;
struct thread_data data[MAX_PARALLEL];
+   uint64_t start = getnanotime();
 
if (!core_preload_index)
return;
@@ -108,6 +109,7 @@ static void preload_index(struct index_state *index,
if (pthread_join(p->pthread, NULL))
die("unable to join threaded lstat");
}
+   trace_performance_since(start, "preload 

Re: [PATCH v3 1/3] read-cache: fix reading the shared index for other repos

[Thomas Gummerer]

On 01/21, Junio C Hamano wrote:
> Thomas Gummerer  writes:
> 
> > On 01/19, Junio C Hamano wrote:
> >> Thomas Gummerer  writes:
> >> 
> >> > read_cache_from() defaults to using the gitdir of the_repository.  As it
> >> > is mostly a convenience macro, having to pass get_git_dir() for every
> >> > call seems overkill, and if necessary users can have more control by
> >> > using read_index_from().
> >> 
> >> This was a bit painful change, given that some changes in flight do
> >> add new callsites to read_index_from() and they got the function
> >> changed under their feet.
> >
> > Sorry about that.  Is there any way to make such a change less painful
> > in the future?
> 
> One way is to do for read_index_from() what you did for the existing
> users of read_cache_from().  Introduce a _new_ helper that will not
> be known for any existing topics in flight, and use that to make the
> existing API a thin wrapper around it.

I'll do that next time.  My worries were just that the
'read_index_from()' API is broken with split index, which is what this
series fixes.  It would be easy to introduce new breakages if we're
not careful.

> I _think_ I got it right with evil merge, so unless this fix needs
> to be delayed for extended period of time for whatever reason while
> any more new callers of the function appears (which is unlikely), we
> should be OK ;-)

Thanks!  As mentioned there's one call that was added that the evil
merge didn't get quite right, for which I sent the diff below in my
previous email.  But I'm happy to fix that on top once this series
goes to master or next if that's preferred.

diff --git a/builtin/worktree.c b/builtin/worktree.c
index 6a49f9e628..4d86a3574f 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -612,7 +612,8 @@ static void validate_no_submodules(const struct worktree 
*wt)
struct index_state istate = {0};
int i, found_submodules = 0;
 
-   if (read_index_from(, worktree_git_path(wt, "index"), 
get_git_dir()) > 0) {
+   if (read_index_from(, worktree_git_path(wt, "index"),
+   get_worktree_git_dir(wt)) > 0) {
for (i = 0; i < istate.cache_nr; i++) {
struct cache_entry *ce = istate.cache[i];
 
diff --git a/t/t2028-worktree-move.sh b/t/t2028-worktree-move.sh
index b3105eaaed..8faf61bbf5 100755
--- a/t/t2028-worktree-move.sh
+++ b/t/t2028-worktree-move.sh
@@ -90,6 +90,16 @@ test_expect_success 'move main worktree' '
test_must_fail git worktree move . def
 '
 
+test_expect_success 'move worktree with split index' '
+   git worktree add test &&
+   (
+   cd test &&
+   test_commit file &&
+   git update-index --split-index
+   ) &&
+   git worktree move test test-destination
+'
+
 test_expect_success 'remove main worktree' '
test_must_fail git worktree remove .
 '

> Thanks.

Re: [PATCH] trace: measure where the time is spent in the index-heavy operations

[Thomas Gummerer]

On 01/27, Nguyễn Thái Ngọc Duy wrote:
> All the known heavy code blocks are measured (except object database
> access). This should help identify if an optimization is effective or
> not. An unoptimized git-status would give something like below (92% of
> time is accounted).
> 
> Signed-off-by: Nguyễn Thái Ngọc Duy 
> ---
>  This was in my old index-helper series. The series was replaced by
>  fsmonitor but perhaps some measurements like this still helps.
> 
>  In my old version I measured packed-refs read time too. But
>  packed-refs is mmap'd now, no need to worry about it (or at least its
>  initial cost).
> 
>  diff-lib.c  |  4 
>  dir.c   |  2 ++
>  name-hash.c |  3 +++
>  preload-index.c |  2 ++
>  read-cache.c| 11 +++
>  5 files changed, 22 insertions(+)
>
> [...]
>  
> diff --git a/read-cache.c b/read-cache.c
> index 2eb81a66b9..1f00aee6a2 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -1372,6 +1372,7 @@ int refresh_index(struct index_state *istate, unsigned 
> int flags,
>   const char *typechange_fmt;
>   const char *added_fmt;
>   const char *unmerged_fmt;
> + uint64_t start = getnanotime();
>  
>   modified_fmt = (in_porcelain ? "M\t%s\n" : "%s: needs update\n");
>   deleted_fmt = (in_porcelain ? "D\t%s\n" : "%s: needs update\n");
> @@ -1442,6 +1443,7 @@ int refresh_index(struct index_state *istate, unsigned 
> int flags,
>  
>   replace_index_entry(istate, i, new);
>   }
> + trace_performance_since(start, "refresh index");
>   return has_errors;
>  }
>  
> @@ -1877,12 +1879,15 @@ int read_index_from(struct index_state *istate, const 
> char *path)
>   int ret;
>   char *base_sha1_hex;
>   const char *base_path;
> + uint64_t start;
>  
>   /* istate->initialized covers both .git/index and .git/sharedindex.xxx 
> */
>   if (istate->initialized)
>   return istate->cache_nr;
>  
> + start = getnanotime();
>   ret = do_read_index(istate, path, 0);
> + trace_performance_since(start, "read cache %s", path);
>  
>   split_index = istate->split_index;
>   if (!split_index || is_null_sha1(split_index->base_sha1)) {
> @@ -1897,6 +1902,7 @@ int read_index_from(struct index_state *istate, const 
> char *path)
>  
>   base_sha1_hex = sha1_to_hex(split_index->base_sha1);
>   base_path = git_path("sharedindex.%s", base_sha1_hex);
> + start = getnanotime();
>   ret = do_read_index(split_index->base, base_path, 1);
>   if (hashcmp(split_index->base_sha1, split_index->base->sha1))
>   die("broken index, expect %s in %s, got %s",
> @@ -1906,6 +1912,9 @@ int read_index_from(struct index_state *istate, const 
> char *path)
>   freshen_shared_index(base_sha1_hex, 0);
>   merge_base_index(istate);
>   post_read_index_from(istate);
> + trace_performance_since(start, "read cache %s",
> + git_path("sharedindex.%s",
> +  sha1_to_hex(split_index->base_sha1)));

Would it be worth doing this on top of tg/split-index-fixes?  OTOH
this will only give a wrong output when tracing performance is on, and
it should be easy enough to figure out where the sharedindex actually
is.  So it might be better to keep this separate, and then just add a
patch on top for fixing the path later, which might be less work for
Junio.

So dunno what the best way is, just wanted to mention it.

>   return ret;
>  }
>  
> @@ -2244,6 +2253,7 @@ static int do_write_index(struct index_state *istate, 
> struct tempfile *tempfile,
>   struct ondisk_cache_entry_extended ondisk;
>   struct strbuf previous_name_buf = STRBUF_INIT, *previous_name;
>   int drop_cache_tree = 0;
> + uint64_t start = getnanotime();
>  
>   for (i = removed = extended = 0; i < entries; i++) {
>   if (cache[i]->ce_flags & CE_REMOVE)
> @@ -2374,6 +2384,7 @@ static int do_write_index(struct index_state *istate, 
> struct tempfile *tempfile,
>   return -1;
>   istate->timestamp.sec = (unsigned int)st.st_mtime;
>   istate->timestamp.nsec = ST_MTIME_NSEC(st);
> + trace_performance_since(start, "write index, changed mask = %x", 
> istate->cache_changed);
>   return 0;
>  }
>  
> -- 
> 2.16.1.205.g271f633410
> 

Re: Some rough edges of core.fsmonitor

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 27 2018, Duy Nguyen jotted:

> On Sat, Jan 27, 2018 at 7:28 AM, Ævar Arnfjörð Bjarmason
>  wrote:
>> 3) A lot of time spend reading the index (or something..)
>
> I'm resending a patch from my old index-helper series. It should
> measure all big time consuming blocks in git. Maybe we should get it
> merged...
>
>> While the hook itself takes ~20ms (and watchman itself 1/4 of that)
>> status as a whole takes much longer. gprof reveals:
>>
>> Each sample counts as 0.01 seconds.
>>   %   cumulative   self  self total
>>  time   seconds   secondscalls  ms/call  ms/call  name
>>  15.38  0.02 0.02   221690 0.00 0.00  memihash
>
> This sounds like name-hash to me.
>
>>  15.38  0.04 0.02   221689 0.00 0.00  create_from_disk
>>   7.69  0.05 0.01  2216897 0.00 0.00  git_bswap32
>>   7.69  0.06 0.01   222661 0.00 0.00  ce_path_match
>>   7.69  0.07 0.01   221769 0.00 0.00  hashmap_add
>>   7.69  0.08 0.0139941 0.00 0.00  prep_exclude
>>   7.69  0.09 0.0139940 0.00 0.00  strbuf_addch
>>   7.69  0.10 0.01110.0010.00  read_one
>>   7.69  0.11 0.01110.0010.00  refresh_index
>>   7.69  0.12 0.01110.0010.00  tweak_fsmonitor
>>   7.69  0.13 0.01 preload_thread
>>
>> The index is 24M in this case, I guess it's unpacking it, but I wonder
>> if this couldn't be much faster if we saved away the result of the last
>> "status" in something that's quick to access, and then if nothing
>
> No we could do better, we could cache parsed index content so
> everybody benefits. I demonstrated it with my "index v254" patch a
> while back:
>
> https://public-inbox.org/git/1399980019-8706-1-git-send-email-pclo...@gmail.com/
>
> With the patch I'm sending soon, we can see how much time reading an
> index take out of that ~140-150ms (and we probably can cut down index
> read time to like 10-20% when cached).
>
>> changed we just report that, and no need to re-write the index (or just
>> write the "it was clean at this time" part).
>
> Hmm.. does an index write increase that much time?

Your patch is very useful. Here's (with gcc -03) some runtimes. This
also includes my .git exclusion patch.

These are all best out of 5, and with the top (until <0.5% time) of
strace -c output (run as another invocation, timing not done with
strace)::

a) no fsmonitor

$ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
12:32:44.947651 read-cache.c:1890   performance: 0.053153609 s: read 
cache .git/index
12:32:44.967943 preload-index.c:112 performance: 0.020161093 s: preload 
index
12:32:44.974217 read-cache.c:1446   performance: 0.006230611 s: refresh 
index
12:32:44.979083 diff-lib.c:250  performance: 0.004649994 s: 
diff-files
12:32:44.982511 diff-lib.c:527  performance: 0.002918416 s: 
diff-index
12:32:45.037880 dir.c:2290  performance: 0.055331063 s: read 
directory
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
12:32:45.040666 trace.c:417 performance: 0.146724289 s: git 
command: '/home/aearnfjord/g/git/git-status'

real0m0.153s
user0m0.110s
sys 0m0.354s
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
 59.930.031924   1 39978 9 stat
 35.860.0191046368 3   futex
  0.840.000446  1236   mprotect
  0.730.000389  1329   munmap
  0.660.000349   662   mmap
  0.530.000285  1420   clone

b) with fsmonitor

$ time GIT_TRACE_PERFORMANCE=1 ~/g/git/git-status
12:34:23.833625 read-cache.c:1890   performance: 0.049485685 s: read 
cache .git/index
12:34:23.838622 preload-index.c:112 performance: 0.001221197 s: preload 
index
12:34:23.858723 fsmonitor.c:170 performance: 0.020059647 s: 
fsmonitor process '.git/hooks/fsmonitor-watchman'
12:34:23.871532 read-cache.c:1446   performance: 0.032870818 s: refresh 
index
12:34:23.876427 diff-lib.c:250  performance: 0.004731427 s: 
diff-files
12:34:23.880669 diff-lib.c:527  performance: 0.003944422 s: 
diff-index
12:34:23.899225 dir.c:2290  performance: 0.018509066 s: read 
directory
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
12:34:23.901914 trace.c:417 performance: 0.118250995 s: git 
command: '/home/aearnfjord/g/git/git-status'

real0m0.125s
user0m0.086s
sys 0m0.043s
  

Re: [PATCH 0/2] Add "git rebase --show-patch"

[Ævar Arnfjörð Bjarmason]

On Fri, Jan 26 2018, Nguyễn Thái Ngọc Duy jotted:

> When a conflict happens during a rebase, you often need to look at the
> original patch to see what the changes are. This requires opening your
> favourite pager with some random path inside $GIT_DIR.
>
> This series makes that experience a bit better, by providing a command
> to read the patch. This is along the line of --edit-todo and --quit
> where you can just tell git what to do and not bother with details.
>
> My main focus is "git rebase", but because rebase uses "git am" behind
> the scene, "git am" gains --show-patch option too.
>
> There was something more I wanted to do, like coloring to the patch.
> But that probably will come later. I'll try to merge these two
> 21-months-old patches first.

This is only tangentially related to what you're doing, but I've long
wanted to add a commit.verbose config option to emulate `git commit
--verbose`, and furthermore to show the patch in rebase under "reword",
"squash" etc.

There's been so many times when I start editing the todo list, and
reword this or that, only to forget (because I don't have good commit
messages yet) what the patch is even about, and then switch to a
terminal, "git show" etc.

I'm just mentioning that here because if and when we have such a
feature, I think the --show-patch option is going to be very confusing,
people might want to enable this thing I'm talking about, but find that
--show-patch is something else entirely.

I don't know a good solution to that, just putting that out there.

Hello Dear Friend,

[ATIKOLA AWA]

Hello Dear Friend,
Greetings and how are you doing?
I want to know if you are kind to be my
partner in claiming the fund of  $13.6
MillionUSD left by a late client. If you're
interested please Revert for more details.
You can visit the web for more details.
http://newswww.bbc.net.uk/2/hi/uk_news/england/oxfordshire/4537663.stm
Dr.Atikola Awa,
aitburk...@gmail.com