[PATCH v4/wip 02/12] generate-cmds.sh: export all commands to command-list.h

2018-04-25 Thread Nguyễn Thái Ngọc Duy
The current generate-cmds.sh generates just enough to print "git help"
output. That is, it only extracts help text for common commands.

The script is now updated to extract help text for all commands and
keep command classification a new file, command-list.h. This will be
useful later:

- "git help -a" could print a short summary of all commands instead of
  just the common ones.

- "git" could produce a list of commands of one or more category. One
  of its use is to reduce another command classification embedded in
  git-completion.bash.

The new file can be generated but is not used anywhere yet. The plan
is we migrate away from common-cmds.h. Then we can kill off
common-cmds.h build rules and generation code (and also delete
duplicate content in command-list.h which we keep for now to not mess
generate-cmds.sh up too much).

PS. The new fixed column requirement on command-list.txt is
technically not needed. But it helps simplify the code a bit at this
stage. We could lift this restriction later if we want to.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 .gitignore  |  1 +
 Makefile| 13 ++---
 command-list.txt|  4 +--
 generate-cmdlist.sh | 67 ++---
 4 files changed, 75 insertions(+), 10 deletions(-)

diff --git a/.gitignore b/.gitignore
index 833ef3b0b7..d4c3914167 100644
--- a/.gitignore
+++ b/.gitignore
@@ -180,6 +180,7 @@
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
 /common-cmds.h
+/command-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index f181687250..2a8913ea21 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h
+GENERATED_H += common-cmds.h command-list.h
 
 LIB_H = $(shell $(FIND) . \
-name .git -prune -o \
@@ -1938,6 +1938,11 @@ $(BUILT_INS): git$X
 common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
+   $(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON 
>$@+ && mv $@+ $@
+
+command-list.h: generate-cmdlist.sh command-list.txt
+
+command-list.h: $(wildcard Documentation/git-*.txt)
$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ 
&& mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
@@ -2148,7 +2153,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h
+# Dependencies on automatically generated headers such as common-cmds.h or 
command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2527,7 +2532,7 @@ sparse: $(SP_OBJ)
 style:
git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h
+check: common-cmds.h command-list.h
@if sparse; \
then \
echo >&2 "Use 'make sparse' instead"; \
@@ -2775,7 +2780,7 @@ clean: profile-clean coverage-clean
$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
$(RM) -r bin-wrappers $(dep_dirs)
$(RM) -r po/build/
-   $(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h $(ETAGS_TARGET) tags 
cscope*
+   $(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h 
$(ETAGS_TARGET) tags cscope*
$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/command-list.txt b/command-list.txt
index a1fad28fd8..786536aba0 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -8,8 +8,8 @@ info examine the history and state (see also: git help 
revisions)
 history  grow, mark and tweak your common history
 remote   collaborate (see also: git help workflows)
 
-### command list (do not change this line)
-# command name  category [deprecated] [common]
+### command list (do not change this line, also do not change alignment)
+# command name  category [category] [category]
 git-add mainporcelain   worktree
 git-am  mainporcelain
 git-annotateancillaryinterrogators
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 17d6809ef5..9ba9911f09 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,27 @@
 #!/bin/sh
 
+die () {
+   echo "$@" >&2
+   exit 1
+}
+
+command_list () {
+   sed '1,/^### command list/d;/^#/d' "$1"
+}
+
+get_categories() {
+   tr ' ' '\n'|
+   grep -v '^$' |
+   sort |
+   uniq
+}
+
+category_list () {
+   command_list "$1" |
+   cut -c 40- |
+   get_categories
+}
+
 get_synopsis () {
local cmd="$1"
 
@@ -12,14 +34,51 @@ get_synopsis () {
}' "Documentation/$cmd.txt"
 }
 
+define_categories()

Re: [PATCH v4/wip 02/12] generate-cmds.sh: export all commands to command-list.h

2018-04-25 Thread Eric Sunshine
On Wed, Apr 25, 2018 at 12:30 PM, Nguyễn Thái Ngọc Duy
 wrote:
> The current generate-cmds.sh generates just enough to print "git help"
> output. That is, it only extracts help text for common commands.
>
> The script is now updated to extract help text for all commands and
> keep command classification a new file, command-list.h. This will be
> useful later:
> [...]
> Signed-off-by: Nguyễn Thái Ngọc Duy 
> ---
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> @@ -12,14 +34,51 @@ get_synopsis () {
> +define_categories() {
> +   echo
> +   echo "/* Command categories */"
> +   bit=0
> +   category_list "$1" |
> +   while read cat
> +   do
> +   echo "#define CAT_$cat (1UL << $bit)"
> +   bit=$(($bit+1))
> +   done
> +   test "$bit" -gt 32 && die "Urgh.. too many categories?"

Should this be '-ge' rather than '-gt'?

> +}


Re: [PATCH v4/wip 02/12] generate-cmds.sh: export all commands to command-list.h

2018-04-29 Thread Duy Nguyen
On Wed, Apr 25, 2018 at 8:07 PM, Eric Sunshine  wrote:
> On Wed, Apr 25, 2018 at 12:30 PM, Nguyễn Thái Ngọc Duy
>  wrote:
>> The current generate-cmds.sh generates just enough to print "git help"
>> output. That is, it only extracts help text for common commands.
>>
>> The script is now updated to extract help text for all commands and
>> keep command classification a new file, command-list.h. This will be
>> useful later:
>> [...]
>> Signed-off-by: Nguyễn Thái Ngọc Duy 
>> ---
>> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
>> @@ -12,14 +34,51 @@ get_synopsis () {
>> +define_categories() {
>> +   echo
>> +   echo "/* Command categories */"
>> +   bit=0
>> +   category_list "$1" |
>> +   while read cat
>> +   do
>> +   echo "#define CAT_$cat (1UL << $bit)"
>> +   bit=$(($bit+1))
>> +   done
>> +   test "$bit" -gt 32 && die "Urgh.. too many categories?"
>
> Should this be '-ge' rather than '-gt'?

After we print "1UL << 31" we increment it to 32 and exit the loop,
then it's still within limits, -ge would incorrectly complain
-- 
Duy