Merge branch 'add-i-in-c-status-and-help'

The first part of the long journey to a fully built-in `git add -i`.

It reflects the part that was submitted a couple of times (see
https://github.com/gitgitgadget/git/pull/103) during the Outreachy
project by Slavica Đukić that continued the journey based on an initial
patch series by Daniel Ferreira.

This part only implements the `status` and the `help` part, like
Slavica's last iteration did, in the interest of making the review
remotely more reviewable. I fear that this attempt of making it a bit
more reviewable is pretty futile, as so many things changed. So I will
ask the reviewers for forgiveness: please be kind, and give this sort of
a fresh review.

I threw in a couple of major changes on top of that iteration, though:

- The original plan was to add a helper (`git add--helper`) that takes
  over more and more responsibility from the Perl script over the course
  of the conversion.

  This plan is no longer in effect, as I encountered a serious problem
  with that: the MSYS2 runtime used by the Perl interpreter which Git
  for Windows employs to run `git add -i` has a curious bug (that is
  safely outside the purview of this here patch series) where it fails
  to read from standard input after it spawned a non-MSYS2 program that
  reads from standard input. To keep my `git add -i` in a working state,
  I therefore adopted a different strategy:

  Just like `git difftool` was converted by starting with a built-in
  that did nothing but handing off to the scripted version, guarded by
  the (opt-in) `difftool.useBuiltin` config setting, I start this patch
  series by a built-in `add -i` that does nothing else but state that it
  is not implemented yet, guarded by the (opt-in)
  `add.interactive.useBuiltin` config setting.

  In contrast to the `git difftool` situation, it is quite a bit easier
  here, as we do not even have to rename the script to
  `git-legacy-add--interactive.perl`: the `add--interactive` command is
  an implementation detail that users are not even supposed to know
  about. Therefore, we can implement that road fork between the built-in
  and the scripted version in `builtin/add.c`, i.e. in the user-facing
  `git add` command.

  This will also naturally help with the transition to a fully built-in
  `git add -i`/`git add -p`, as we saw with the built-in `git rebase`
  how important it is for end users to have an escape hatch (and for
  that reason, tried our best to provide the same with the built-in `git
  stash`).

- The `help` command was actually not hooked up in `git add -i`, but was
  only available as a special option of the `git add--helper` command.
  As that command no longer exists, I kind of *had* to implement
  some way to let the built-in `git add -i` show the help text.

- The main loop of `git add -i` (i.e. the thing that lets you choose
  `status` or `help`) is now implemented (but only lists `status` and
  `help`, of course), as it makes use of that feature that took the main
  chunk of the Outreachy project: the function to determine unique
  prefixes of a list of strings.

- Speaking of the unique prefixes: the functionality to determine those
  is now encapsulated in the `prefix-map.c` file, and I also added a
  regression test.

- Speaking of the tests: I also implemented support for the environment
  variable `GIT_TEST_ADD_I_USE_BUILTIN`: by setting it, the test suite
  can be forced to use the built-in, or the Perl script, version of `git
  add -i`. Needless to say: by the end of this patch series, running the
  test suite with `GIT_TEST_ADD_I_USE_BUILTIN=true` will still result in
  a ton of test failures due to not-yet-implemented commands, but it
  will also demonstrate what *already* works.

- Since the main loop starts not only by showing the status, but
  refreshes the index before that, I added that, and I actually
  refactored that code into a new function
  (`repo_refresh_and_write_index()`), as it will be used a couple of
  times by the end of the complete conversion of `git add -i` into a
  built-in command.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Documentation/config/add.txt

@@ -5,3 +5,8 @@ add.ignore-errors (deprecated)::
 	option of linkgit:git-add[1].  `add.ignore-errors` is deprecated,
 	as it does not follow the usual naming convention for configuration
 	variables.
+
+add.interactive.useBuiltin::
+	[EXPERIMENTAL] Set to `true` to use the experimental built-in
+	implementation of the interactive version of linkgit:git-add[1]
+	instead of the Perl script version. Is `false` by default.

Makefile

@@ -754,6 +754,7 @@ TEST_BUILTINS_OBJS += test-online-cpus.o
 TEST_BUILTINS_OBJS += test-parse-options.o
 TEST_BUILTINS_OBJS += test-path-utils.o
 TEST_BUILTINS_OBJS += test-pkt-line.o
+TEST_BUILTINS_OBJS += test-prefix-map.o
 TEST_BUILTINS_OBJS += test-prio-queue.o
 TEST_BUILTINS_OBJS += test-reach.o
 TEST_BUILTINS_OBJS += test-read-cache.o
@@ -848,6 +849,7 @@ LIB_H = $(shell $(FIND) . \
 	-name '*.h' -print)
 
 LIB_OBJS += abspath.o
+LIB_OBJS += add-interactive.o
 LIB_OBJS += advice.o
 LIB_OBJS += alias.o
 LIB_OBJS += alloc.o
@@ -966,6 +968,7 @@ LIB_OBJS += patch-ids.o
 LIB_OBJS += path.o
 LIB_OBJS += pathspec.o
 LIB_OBJS += pkt-line.o
+LIB_OBJS += prefix-map.o
 LIB_OBJS += preload-index.o
 LIB_OBJS += pretty.o
 LIB_OBJS += prio-queue.o

add-interactive.c

@@ -0,0 +1,552 @@
+#include "cache.h"
+#include "add-interactive.h"
+#include "color.h"
+#include "config.h"
+#include "diffcore.h"
+#include "revision.h"
+#include "refs.h"
+#include "prefix-map.h"
+
+struct add_i_state {
+	struct repository *r;
+	int use_color;
+	char header_color[COLOR_MAXLEN];
+	char help_color[COLOR_MAXLEN];
+	char prompt_color[COLOR_MAXLEN];
+	char error_color[COLOR_MAXLEN];
+	char reset_color[COLOR_MAXLEN];
+};
+
+static void init_color(struct repository *r, struct add_i_state *s,
+		       const char *slot_name, char *dst,
+		       const char *default_color)
+{
+	char *key = xstrfmt("color.interactive.%s", slot_name);
+	const char *value;
+
+	if (!s->use_color)
+		dst[0] = '\0';
+	else if (repo_config_get_value(r, key, &value) ||
+		 color_parse(value, dst))
+		strlcpy(dst, default_color, COLOR_MAXLEN);
+
+	free(key);
+}
+
+static int init_add_i_state(struct repository *r, struct add_i_state *s)
+{
+	const char *value;
+
+	s->r = r;
+
+	if (repo_config_get_value(r, "color.interactive", &value))
+		s->use_color = -1;
+	else
+		s->use_color =
+			git_config_colorbool("color.interactive", value);
+	s->use_color = want_color(s->use_color);
+
+	init_color(r, s, "header", s->header_color, GIT_COLOR_BOLD);
+	init_color(r, s, "help", s->help_color, GIT_COLOR_BOLD_RED);
+	init_color(r, s, "prompt", s->prompt_color, GIT_COLOR_BOLD_BLUE);
+	init_color(r, s, "error", s->error_color, GIT_COLOR_BOLD_RED);
+	init_color(r, s, "reset", s->reset_color, GIT_COLOR_RESET);
+
+	return 0;
+}
+
+static ssize_t find_unique(const char *string,
+			   struct prefix_item **list, size_t nr)
+{
+	ssize_t found = -1, i;
+
+	for (i = 0; i < nr; i++) {
+		struct prefix_item *item = list[i];
+		if (!starts_with(item->name, string))
+			continue;
+		if (found >= 0)
+			return -1;
+		found = i;
+	}
+
+	return found;
+}
+
+struct list_options {
+	int columns;
+	const char *header;
+	void (*print_item)(int i, struct prefix_item *item,
+			   void *print_item_data);
+	void *print_item_data;
+};
+
+static void list(struct prefix_item **list, size_t nr,
+		 struct add_i_state *s, struct list_options *opts)
+{
+	int i, last_lf = 0;
+
+	if (!nr)
+		return;
+
+	if (opts->header)
+		color_fprintf_ln(stdout, s->header_color,
+				 "%s", opts->header);
+
+	for (i = 0; i < nr; i++) {
+		opts->print_item(i, list[i], opts->print_item_data);
+
+		if ((opts->columns) && ((i + 1) % (opts->columns))) {
+			putchar('\t');
+			last_lf = 0;
+		}
+		else {
+			putchar('\n');
+			last_lf = 1;
+		}
+	}
+
+	if (!last_lf)
+		putchar('\n');
+}
+struct list_and_choose_options {
+	struct list_options list_opts;
+
+	const char *prompt;
+	void (*print_help)(struct add_i_state *s);
+};
+
+/*
+ * Returns the selected index.
+ */
+static ssize_t list_and_choose(struct prefix_item **items, size_t nr,
+			       struct add_i_state *s,
+			       struct list_and_choose_options *opts)
+{
+	struct strbuf input = STRBUF_INIT;
+	ssize_t res = -1;
+
+	find_unique_prefixes(items, nr, 1, 4);
+
+	for (;;) {
+		char *p, *endp;
+
+		strbuf_reset(&input);
+
+		list(items, nr, s, &opts->list_opts);
+
+		color_fprintf(stdout, s->prompt_color, "%s", opts->prompt);
+		fputs("> ", stdout);
+		fflush(stdout);
+
+		if (strbuf_getline(&input, stdin) == EOF) {
+			putchar('\n');
+			res = -2;
+			break;
+		}
+		strbuf_trim(&input);
+
+		if (!input.len)
+			break;
+
+		if (!strcmp(input.buf, "?")) {
+			opts->print_help(s);
+			continue;
+		}
+
+		p = input.buf;
+		for (;;) {
+			size_t sep = strcspn(p, " \t\r\n,");
+			ssize_t index = -1;
+
+			if (!sep) {
+				if (!*p)
+					break;
+				p++;
+				continue;
+			}
+
+			if (isdigit(*p)) {
+				index = strtoul(p, &endp, 10) - 1;
+				if (endp != p + sep)
+					index = -1;
+			}
+
+			p[sep] = '\0';
+			if (index < 0)
+				index = find_unique(p, items, nr);
+
+			if (index < 0 || index >= nr)
+				color_fprintf_ln(stdout, s->error_color,
+						 _("Huh (%s)?"), p);
+			else {
+				res = index;
+				break;
+			}
+
+			p += sep + 1;
+		}
+
+		if (res >= 0)
+			break;
+	}
+
+	strbuf_release(&input);
+	return res;
+}
+
+struct adddel {
+	uintmax_t add, del;
+	unsigned seen:1, binary:1;
+};
+
+struct file_list {
+	struct file_item {
+		struct prefix_item item;
+		struct adddel index, worktree;
+	} **file;
+	size_t nr, alloc;
+};
+
+static void add_file_item(struct file_list *list, const char *name)
+{
+	struct file_item *item;
+
+	FLEXPTR_ALLOC_STR(item, item.name, name);
+
+	ALLOC_GROW(list->file, list->nr + 1, list->alloc);
+	list->file[list->nr++] = item;
+}
+
+static void reset_file_list(struct file_list *list)
+{
+	size_t i;
+
+	for (i = 0; i < list->nr; i++)
+		free(list->file[i]);
+	list->nr = 0;
+}
+
+static void release_file_list(struct file_list *list)
+{
+	reset_file_list(list);
+	FREE_AND_NULL(list->file);
+	list->alloc = 0;
+}
+
+static int file_item_cmp(const void *a, const void *b)
+{
+	const struct file_item * const *f1 = a;
+	const struct file_item * const *f2 = b;
+
+	return strcmp((*f1)->item.name, (*f2)->item.name);
+}
+
+struct pathname_entry {
+	struct hashmap_entry ent;
+	size_t index;
+	char pathname[FLEX_ARRAY];
+};
+
+static int pathname_entry_cmp(const void *unused_cmp_data,
+			      const void *entry, const void *entry_or_key,
+			      const void *pathname)
+{
+	const struct pathname_entry *e1 = entry, *e2 = entry_or_key;
+
+	return strcmp(e1->pathname,
+		      pathname ? (const char *)pathname : e2->pathname);
+}
+
+struct collection_status {
+	enum { FROM_WORKTREE = 0, FROM_INDEX = 1 } phase;
+
+	const char *reference;
+
+	struct file_list *list;
+	struct hashmap file_map;
+};
+
+static void collect_changes_cb(struct diff_queue_struct *q,
+			       struct diff_options *options,
+			       void *data)
+{
+	struct collection_status *s = data;
+	struct diffstat_t stat = { 0 };
+	int i;
+
+	if (!q->nr)
+		return;
+
+	compute_diffstat(options, &stat, q);
+
+	for (i = 0; i < stat.nr; i++) {
+		const char *name = stat.files[i]->name;
+		int hash = strhash(name);
+		struct pathname_entry *entry;
+		size_t file_index;
+		struct file_item *file;
+		struct adddel *adddel;
+
+		entry = hashmap_get_from_hash(&s->file_map, hash, name);
+		if (entry)
+			file_index = entry->index;
+		else {
+			FLEX_ALLOC_STR(entry, pathname, name);
+			hashmap_entry_init(entry, hash);
+			entry->index = file_index = s->list->nr;
+			hashmap_add(&s->file_map, entry);
+
+			add_file_item(s->list, name);
+		}
+		file = s->list->file[file_index];
+
+		adddel = s->phase == FROM_INDEX ? &file->index : &file->worktree;
+		adddel->seen = 1;
+		adddel->add = stat.files[i]->added;
+		adddel->del = stat.files[i]->deleted;
+		if (stat.files[i]->is_binary)
+			adddel->binary = 1;
+	}
+}
+
+static int get_modified_files(struct repository *r, struct file_list *list,
+			      const struct pathspec *ps)
+{
+	struct object_id head_oid;
+	int is_initial = !resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+					     &head_oid, NULL);
+	struct collection_status s = { FROM_WORKTREE };
+
+	if (repo_read_index_preload(r, ps, 0) < 0)
+		return error(_("could not read index"));
+
+	s.list = list;
+	hashmap_init(&s.file_map, pathname_entry_cmp, NULL, 0);
+
+	for (s.phase = FROM_WORKTREE; s.phase <= FROM_INDEX; s.phase++) {
+		struct rev_info rev;
+		struct setup_revision_opt opt = { 0 };
+
+		opt.def = is_initial ?
+			empty_tree_oid_hex() : oid_to_hex(&head_oid);
+
+		init_revisions(&rev, NULL);
+		setup_revisions(0, NULL, &rev, &opt);
+
+		rev.diffopt.output_format = DIFF_FORMAT_CALLBACK;
+		rev.diffopt.format_callback = collect_changes_cb;
+		rev.diffopt.format_callback_data = &s;
+
+		if (ps)
+			copy_pathspec(&rev.prune_data, ps);
+
+		if (s.phase == FROM_INDEX)
+			run_diff_index(&rev, 1);
+		else {
+			rev.diffopt.flags.ignore_dirty_submodules = 1;
+			run_diff_files(&rev, 0);
+		}
+	}
+	hashmap_free(&s.file_map, 1);
+
+	/* While the diffs are ordered already, we ran *two* diffs... */
+	QSORT(list->file, list->nr, file_item_cmp);
+
+	return 0;
+}
+
+static void populate_wi_changes(struct strbuf *buf,
+				struct adddel *ad, const char *no_changes)
+{
+	if (ad->binary)
+		strbuf_addstr(buf, _("binary"));
+	else if (ad->seen)
+		strbuf_addf(buf, "+%"PRIuMAX"/-%"PRIuMAX,
+			    (uintmax_t)ad->add, (uintmax_t)ad->del);
+	else
+		strbuf_addstr(buf, no_changes);
+}
+
+/* filters out prefixes which have special meaning to list_and_choose() */
+static int is_valid_prefix(const char *prefix, size_t prefix_len)
+{
+	return prefix_len && prefix &&
+		/*
+		 * We expect `prefix` to be NUL terminated, therefore this
+		 * `strcspn()` call is okay, even if it might do much more
+		 * work than strictly necessary.
+		 */
+		strcspn(prefix, " \t\r\n,") >= prefix_len &&	/* separators */
+		*prefix != '-' &&				/* deselection */
+		!isdigit(*prefix) &&				/* selection */
+		(prefix_len != 1 ||
+		 (*prefix != '*' &&				/* "all" wildcard */
+		  *prefix != '?'));				/* prompt help */
+}
+
+struct print_file_item_data {
+	const char *modified_fmt;
+	struct strbuf buf, index, worktree;
+};
+
+static void print_file_item(int i, struct prefix_item *item,
+			    void *print_file_item_data)
+{
+	struct file_item *c = (struct file_item *)item;
+	struct print_file_item_data *d = print_file_item_data;
+
+	strbuf_reset(&d->index);
+	strbuf_reset(&d->worktree);
+	strbuf_reset(&d->buf);
+
+	populate_wi_changes(&d->worktree, &c->worktree, _("nothing"));
+	populate_wi_changes(&d->index, &c->index, _("unchanged"));
+	strbuf_addf(&d->buf, d->modified_fmt,
+		    d->index.buf, d->worktree.buf, item->name);
+
+	printf(" %2d: %s", i + 1, d->buf.buf);
+}
+
+static int run_status(struct add_i_state *s, const struct pathspec *ps,
+		      struct file_list *files, struct list_options *opts)
+{
+	reset_file_list(files);
+
+	if (get_modified_files(s->r, files, ps) < 0)
+		return -1;
+
+	if (files->nr)
+		list((struct prefix_item **)files->file, files->nr, s, opts);
+	putchar('\n');
+
+	return 0;
+}
+
+static int run_help(struct add_i_state *s, const struct pathspec *ps,
+		    struct file_list *files, struct list_options *opts)
+{
+	const char *help_color = s->help_color;
+
+	color_fprintf_ln(stdout, help_color, "status        - %s",
+			 _("show paths with changes"));
+	color_fprintf_ln(stdout, help_color, "update        - %s",
+			 _("add working tree state to the staged set of changes"));
+	color_fprintf_ln(stdout, help_color, "revert        - %s",
+			 _("revert staged set of changes back to the HEAD version"));
+	color_fprintf_ln(stdout, help_color, "patch         - %s",
+			 _("pick hunks and update selectively"));
+	color_fprintf_ln(stdout, help_color, "diff          - %s",
+			 _("view diff between HEAD and index"));
+	color_fprintf_ln(stdout, help_color, "add untracked - %s",
+			 _("add contents of untracked files to the staged set of changes"));
+
+	return 0;
+}
+
+struct print_command_item_data {
+	const char *color, *reset;
+};
+
+static void print_command_item(int i, struct prefix_item *item,
+			       void *print_command_item_data)
+{
+	struct print_command_item_data *d = print_command_item_data;
+
+	if (!item->prefix_length ||
+	    !is_valid_prefix(item->name, item->prefix_length))
+		printf(" %2d: %s", i + 1, item->name);
+	else
+		printf(" %2d: %s%.*s%s%s", i + 1,
+		       d->color, (int)item->prefix_length, item->name, d->reset,
+		       item->name + item->prefix_length);
+}
+
+struct command_item {
+	struct prefix_item item;
+	int (*command)(struct add_i_state *s, const struct pathspec *ps,
+		       struct file_list *files, struct list_options *opts);
+};
+
+static void command_prompt_help(struct add_i_state *s)
+{
+	const char *help_color = s->help_color;
+	color_fprintf_ln(stdout, help_color, "%s", _("Prompt help:"));
+	color_fprintf_ln(stdout, help_color, "1          - %s",
+			 _("select a numbered item"));
+	color_fprintf_ln(stdout, help_color, "foo        - %s",
+			 _("select item based on unique prefix"));
+	color_fprintf_ln(stdout, help_color, "           - %s",
+			 _("(empty) select nothing"));
+}
+
+int run_add_i(struct repository *r, const struct pathspec *ps)
+{
+	struct add_i_state s = { NULL };
+	struct print_command_item_data data;
+	struct list_and_choose_options main_loop_opts = {
+		{ 4, N_("*** Commands ***"), print_command_item, &data },
+		N_("What now"), command_prompt_help
+	};
+	struct command_item
+		status = { { "status" }, run_status },
+		help = { { "help" }, run_help };
+	struct command_item *commands[] = {
+		&status,
+		&help
+	};
+
+	struct print_file_item_data print_file_item_data = {
+		"%12s %12s %s", STRBUF_INIT, STRBUF_INIT, STRBUF_INIT
+	};
+	struct list_options opts = {
+		0, NULL, print_file_item, &print_file_item_data
+	};
+	struct strbuf header = STRBUF_INIT;
+	struct file_list files = { NULL };
+	ssize_t i;
+	int res = 0;
+
+	if (init_add_i_state(r, &s))
+		return error("could not parse `add -i` config");
+
+	/*
+	 * When color was asked for, use the prompt color for
+	 * highlighting, otherwise use square brackets.
+	 */
+	if (s.use_color) {
+		data.color = s.prompt_color;
+		data.reset = s.reset_color;
+	} else {
+		data.color = "[";
+		data.reset = "]";
+	}
+
+	strbuf_addstr(&header, "      ");
+	strbuf_addf(&header, print_file_item_data.modified_fmt,
+		    _("staged"), _("unstaged"), _("path"));
+	opts.header = header.buf;
+
+	repo_refresh_and_write_index(r, REFRESH_QUIET, 1);
+	if (run_status(&s, ps, &files, &opts) < 0)
+		res = -1;
+
+	for (;;) {
+		i = list_and_choose((struct prefix_item **)commands,
+				    ARRAY_SIZE(commands), &s, &main_loop_opts);
+		if (i < -1) {
+			printf(_("Bye.\n"));
+			res = 0;
+			break;
+		}
+		if (i >= 0)
+			res = commands[i]->command(&s, ps, &files, &opts);
+	}
+
+	release_file_list(&files);
+	strbuf_release(&print_file_item_data.buf);
+	strbuf_release(&print_file_item_data.index);
+	strbuf_release(&print_file_item_data.worktree);
+	strbuf_release(&header);
+
+	return res;
+}

add-interactive.h

@@ -0,0 +1,8 @@
+#ifndef ADD_INTERACTIVE_H
+#define ADD_INTERACTIVE_H
+
+struct repository;
+struct pathspec;
+int run_add_i(struct repository *r, const struct pathspec *ps);
+
+#endif

builtin/add.c

@@ -20,6 +20,7 @@
 #include "bulk-checkin.h"
 #include "argv-array.h"
 #include "submodule.h"
+#include "add-interactive.h"
 
 static const char * const builtin_add_usage[] = {
 	N_("git add [<options>] [--] <pathspec>..."),
@@ -185,6 +186,14 @@ int run_add_interactive(const char *revision, const char *patch_mode,
 {
 	int status, i;
 	struct argv_array argv = ARGV_ARRAY_INIT;
+	int use_builtin_add_i =
+		git_env_bool("GIT_TEST_ADD_I_USE_BUILTIN", -1);
+	if (use_builtin_add_i < 0)
+		git_config_get_bool("add.interactive.usebuiltin",
+				    &use_builtin_add_i);
+
+	if (use_builtin_add_i == 1 && !patch_mode)
+		return !!run_add_i(the_repository, pathspec);
 
 	argv_array_push(&argv, "add--interactive");
 	if (patch_mode)
@@ -319,6 +328,7 @@ static int add_config(const char *var, const char *value, void *cb)
 		ignore_add_errors = git_config_bool(var, value);
 		return 0;
 	}
+
 	return git_default_config(var, value, cb);
 }
 

diff.c

@@ -2489,22 +2489,6 @@ static void pprint_rename(struct strbuf *name, const char *a, const char *b)
 	}
 }
 
-struct diffstat_t {
-	int nr;
-	int alloc;
-	struct diffstat_file {
-		char *from_name;
-		char *name;
-		char *print_name;
-		const char *comments;
-		unsigned is_unmerged:1;
-		unsigned is_binary:1;
-		unsigned is_renamed:1;
-		unsigned is_interesting:1;
-		uintmax_t added, deleted;
-	} **files;
-};
-
 static struct diffstat_file *diffstat_add(struct diffstat_t *diffstat,
 					  const char *name_a,
 					  const char *name_b)
@@ -6001,12 +5985,7 @@ void diff_flush(struct diff_options *options)
 	    dirstat_by_line) {
 		struct diffstat_t diffstat;
 
-		memset(&diffstat, 0, sizeof(struct diffstat_t));
-		for (i = 0; i < q->nr; i++) {
-			struct diff_filepair *p = q->queue[i];
-			if (check_pair_status(p))
-				diff_flush_stat(p, options, &diffstat);
-		}
+		compute_diffstat(options, &diffstat, q);
 		if (output_format & DIFF_FORMAT_NUMSTAT)
 			show_numstat(&diffstat, options);
 		if (output_format & DIFF_FORMAT_DIFFSTAT)
@@ -6306,6 +6285,20 @@ static int is_submodule_ignored(const char *path, struct diff_options *options)
 	return ignored;
 }
 
+void compute_diffstat(struct diff_options *options,
+		      struct diffstat_t *diffstat,
+		      struct diff_queue_struct *q)
+{
+	int i;
+
+	memset(diffstat, 0, sizeof(struct diffstat_t));
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		if (check_pair_status(p))
+			diff_flush_stat(p, options, diffstat);
+	}
+}
+
 void diff_addremove(struct diff_options *options,
 		    int addremove, unsigned mode,
 		    const struct object_id *oid,

diff.h

@@ -240,6 +240,22 @@ void diff_emit_submodule_error(struct diff_options *o, const char *err);
 void diff_emit_submodule_pipethrough(struct diff_options *o,
 				     const char *line, int len);
 
+struct diffstat_t {
+	int nr;
+	int alloc;
+	struct diffstat_file {
+		char *from_name;
+		char *name;
+		char *print_name;
+		const char *comments;
+		unsigned is_unmerged:1;
+		unsigned is_binary:1;
+		unsigned is_renamed:1;
+		unsigned is_interesting:1;
+		uintmax_t added, deleted;
+	} **files;
+};
+
 enum color_diff {
 	DIFF_RESET = 0,
 	DIFF_CONTEXT = 1,
@@ -328,6 +344,9 @@ void diff_change(struct diff_options *,
 
 struct diff_filepair *diff_unmerge(struct diff_options *, const char *path);
 
+void compute_diffstat(struct diff_options *options, struct diffstat_t *diffstat,
+		      struct diff_queue_struct *q);
+
 #define DIFF_SETUP_REVERSE      	1
 #define DIFF_SETUP_USE_SIZE_CACHE	4
 

prefix-map.c

@@ -0,0 +1,111 @@
+#include "cache.h"
+#include "prefix-map.h"
+
+static int map_cmp(const void *unused_cmp_data,
+		   const void *entry,
+		   const void *entry_or_key,
+		   const void *unused_keydata)
+{
+	const struct prefix_map_entry *a = entry;
+	const struct prefix_map_entry *b = entry_or_key;
+
+	return a->prefix_length != b->prefix_length ||
+		strncmp(a->name, b->name, a->prefix_length);
+}
+
+static void add_prefix_entry(struct hashmap *map, const char *name,
+			     size_t prefix_length, struct prefix_item *item)
+{
+	struct prefix_map_entry *result = xmalloc(sizeof(*result));
+	result->name = name;
+	result->prefix_length = prefix_length;
+	result->item = item;
+	hashmap_entry_init(result, memhash(name, prefix_length));
+	hashmap_add(map, result);
+}
+
+static void init_prefix_map(struct prefix_map *prefix_map,
+			    int min_prefix_length, int max_prefix_length)
+{
+	hashmap_init(&prefix_map->map, map_cmp, NULL, 0);
+	prefix_map->min_length = min_prefix_length;
+	prefix_map->max_length = max_prefix_length;
+}
+
+static void add_prefix_item(struct prefix_map *prefix_map,
+			    struct prefix_item *item)
+{
+	struct prefix_map_entry *e = xmalloc(sizeof(*e)), *e2;
+	int j;
+
+	e->item = item;
+	e->name = e->item->name;
+
+	for (j = prefix_map->min_length; j <= prefix_map->max_length; j++) {
+		if (!isascii(e->name[j])) {
+			free(e);
+			break;
+		}
+
+		e->prefix_length = j;
+		hashmap_entry_init(e, memhash(e->name, j));
+		e2 = hashmap_get(&prefix_map->map, e, NULL);
+		if (!e2) {
+			/* prefix is unique so far */
+			e->item->prefix_length = j;
+			hashmap_add(&prefix_map->map, e);
+			break;
+		}
+
+		if (!e2->item)
+			continue; /* non-unique prefix */
+
+		if (j != e2->item->prefix_length)
+			BUG("unexpected prefix length: %d != %d",
+			    (int)j, (int)e2->item->prefix_length);
+
+		/* skip common prefix */
+		for (; j < prefix_map->max_length && e->name[j]; j++) {
+			if (e->item->name[j] != e2->item->name[j])
+				break;
+			add_prefix_entry(&prefix_map->map, e->name, j + 1,
+					 NULL);
+		}
+
+		/* e2 no longer refers to a unique prefix */
+		if (j < prefix_map->max_length && e2->name[j]) {
+			/* found a new unique prefix for e2's item */
+			e2->item->prefix_length = j + 1;
+			add_prefix_entry(&prefix_map->map, e2->name, j + 1,
+					 e2->item);
+		}
+		else
+			e2->item->prefix_length = 0;
+		e2->item = NULL;
+
+		if (j < prefix_map->max_length && e->name[j]) {
+			/* found a unique prefix for the item */
+			e->item->prefix_length = j + 1;
+			add_prefix_entry(&prefix_map->map, e->name, j + 1,
+					 e->item);
+		} else {
+			/* item has no (short enough) unique prefix */
+			e->item->prefix_length = 0;
+			free(e);
+		}
+
+		break;
+	}
+}
+
+void find_unique_prefixes(struct prefix_item **list, size_t nr,
+			  int min_length, int max_length)
+{
+	int i;
+	struct prefix_map prefix_map;
+
+	init_prefix_map(&prefix_map, min_length, max_length);
+	for (i = 0; i < nr; i++)
+		add_prefix_item(&prefix_map, list[i]);
+	hashmap_free(&prefix_map.map, 1);
+}

prefix-map.h

@@ -0,0 +1,40 @@
+#ifndef PREFIX_MAP_H
+#define PREFIX_MAP_H
+
+#include "hashmap.h"
+
+struct prefix_item {
+	const char *name;
+	size_t prefix_length;
+};
+
+struct prefix_map_entry {
+	struct hashmap_entry e;
+	const char *name;
+	size_t prefix_length;
+	/* if item is NULL, the prefix is not unique */
+	struct prefix_item *item;
+};
+
+struct prefix_map {
+	struct hashmap map;
+	int min_length, max_length;
+};
+
+/*
+ * Find unique prefixes in a given list of strings.
+ *
+ * Typically, the `struct prefix_item` information will be but a field in the
+ * actual item struct; For this reason, the `list` parameter is specified as a
+ * list of pointers to the items.
+ *
+ * The `min_length`/`max_length` parameters define what length the unique
+ * prefixes should have.
+ *
+ * If no unique prefix could be found for a given item, its `prefix_length`
+ * will be set to 0.
+ */
+void find_unique_prefixes(struct prefix_item **list, size_t nr,
+			  int min_length, int max_length);
+
+#endif

repository.c

@@ -272,3 +272,22 @@ int repo_hold_locked_index(struct repository *repo,
 		BUG("the repo hasn't been setup");
 	return hold_lock_file_for_update(lf, repo->index_file, flags);
 }
+
+int repo_refresh_and_write_index(struct repository *r,
+				 unsigned int flags, int gentle)
+{
+	struct lock_file lock_file = LOCK_INIT;
+	int fd;
+
+	if (repo_read_index_preload(r, NULL, 0) < 0)
+		return error(_("could not read index"));
+	fd = repo_hold_locked_index(r, &lock_file, 0);
+	if (!gentle && fd < 0)
+		return error(_("could not lock index for writing"));
+	refresh_index(r->index, flags, NULL, NULL, NULL);
+	if (0 <= fd)
+		repo_update_index_if_able(r, &lock_file);
+	rollback_lock_file(&lock_file);
+
+	return 0;
+}

repository.h

@@ -154,5 +154,12 @@ int repo_read_index_unmerged(struct repository *);
  */
 void repo_update_index_if_able(struct repository *, struct lock_file *);
 
+/*
+ * Refresh the index and write it out. If the index file could not be
+ * locked, error out, except in gentle mode. The flags will be passed
+ * through to refresh_index().
+ */
+int repo_refresh_and_write_index(struct repository *r,
+				 unsigned int flags, int gentle);
 
 #endif /* REPOSITORY_H */

t/README

@@ -383,6 +383,10 @@ GIT_TEST_REBASE_USE_BUILTIN=<boolean>, when false, disables the
 builtin version of git-rebase. See 'rebase.useBuiltin' in
 git-config(1).
 
+GIT_TEST_ADD_I_USE_BUILTIN=<boolean>, when true, enables the
+builtin version of git add -i. See 'add.interactive.useBuiltin' in
+git-config(1).
+
 GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading
 of the index for the whole test suite by bypassing the default number of
 cache entries and thread minimums. Setting this to 1 will make the

t/helper/test-prefix-map.c

@@ -0,0 +1,58 @@
+#include "test-tool.h"
+#include "cache.h"
+#include "prefix-map.h"
+
+static size_t test_count, failed_count;
+
+static void check(int succeeded, const char *file, size_t line_no,
+		  const char *fmt, ...)
+{
+	va_list ap;
+
+	test_count++;
+	if (succeeded)
+		return;
+
+	va_start(ap, fmt);
+	fprintf(stderr, "%s:%d: ", file, (int)line_no);
+	vfprintf(stderr, fmt, ap);
+	fputc('\n', stderr);
+	va_end(ap);
+
+	failed_count++;
+}
+
+#define EXPECT_SIZE_T_EQUALS(expect, actual, hint) \
+	check(expect == actual, __FILE__, __LINE__, \
+	      "size_t's do not match: %" \
+	      PRIdMAX " != %" PRIdMAX " (%s) (%s)", \
+	      (intmax_t)expect, (intmax_t)actual, #actual, hint)
+
+int cmd__prefix_map(int argc, const char **argv)
+{
+#define NR 5
+	struct prefix_item items[NR] = {
+		{ "unique" },
+		{ "hell" },
+		{ "hello" },
+		{ "wok" },
+		{ "world" },
+	};
+	struct prefix_item *list[NR] = {
+		items, items + 1, items + 2, items + 3, items + 4
+	};
+
+	find_unique_prefixes(list, NR, 1, 3);
+
+#define EXPECT_PREFIX_LENGTH_EQUALS(expect, index) \
+	EXPECT_SIZE_T_EQUALS(expect, list[index]->prefix_length, \
+			     list[index]->name)
+
+	EXPECT_PREFIX_LENGTH_EQUALS(1, 0);
+	EXPECT_PREFIX_LENGTH_EQUALS(0, 1);
+	EXPECT_PREFIX_LENGTH_EQUALS(0, 2);
+	EXPECT_PREFIX_LENGTH_EQUALS(3, 3);
+	EXPECT_PREFIX_LENGTH_EQUALS(3, 4);
+
+	return !!failed_count;
+}

t/helper/test-tool.c

@@ -32,6 +32,7 @@ static struct test_cmd cmds[] = {
 	{ "parse-options", cmd__parse_options },
 	{ "path-utils", cmd__path_utils },
 	{ "pkt-line", cmd__pkt_line },
+	{ "prefix-map", cmd__prefix_map },
 	{ "prio-queue", cmd__prio_queue },
 	{ "reach", cmd__reach },
 	{ "read-cache", cmd__read_cache },

t/helper/test-tool.h

@@ -29,6 +29,7 @@ int cmd__online_cpus(int argc, const char **argv);
 int cmd__parse_options(int argc, const char **argv);
 int cmd__path_utils(int argc, const char **argv);
 int cmd__pkt_line(int argc, const char **argv);
+int cmd__prefix_map(int argc, const char **argv);
 int cmd__prio_queue(int argc, const char **argv);
 int cmd__reach(int argc, const char **argv);
 int cmd__read_cache(int argc, const char **argv);

t/t0016-prefix-map.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+
+test_description='basic tests for prefix map'
+. ./test-lib.sh
+
+test_expect_success 'prefix map' '
+	test-tool prefix-map
+'
+
+test_done

t/t3701-add-interactive.sh

@@ -639,4 +639,28 @@ test_expect_success 'add -p patch editing works with pathological context lines'
 	test_cmp expected-2 actual
 '
 
+test_expect_success 'show help from add--helper' '
+	git reset --hard &&
+	cat >expect <<-EOF &&
+
+	<BOLD>*** Commands ***<RESET>
+	  1: <BOLD;BLUE>s<RESET>tatus	  2: <BOLD;BLUE>u<RESET>pdate	  3: <BOLD;BLUE>r<RESET>evert	  4: <BOLD;BLUE>a<RESET>dd untracked
+	  5: <BOLD;BLUE>p<RESET>atch	  6: <BOLD;BLUE>d<RESET>iff	  7: <BOLD;BLUE>q<RESET>uit	  8: <BOLD;BLUE>h<RESET>elp
+	<BOLD;BLUE>What now<RESET>> <BOLD;RED>status        - show paths with changes<RESET>
+	<BOLD;RED>update        - add working tree state to the staged set of changes<RESET>
+	<BOLD;RED>revert        - revert staged set of changes back to the HEAD version<RESET>
+	<BOLD;RED>patch         - pick hunks and update selectively<RESET>
+	<BOLD;RED>diff          - view diff between HEAD and index<RESET>
+	<BOLD;RED>add untracked - add contents of untracked files to the staged set of changes<RESET>
+	<BOLD>*** Commands ***<RESET>
+	  1: <BOLD;BLUE>s<RESET>tatus	  2: <BOLD;BLUE>u<RESET>pdate	  3: <BOLD;BLUE>r<RESET>evert	  4: <BOLD;BLUE>a<RESET>dd untracked
+	  5: <BOLD;BLUE>p<RESET>atch	  6: <BOLD;BLUE>d<RESET>iff	  7: <BOLD;BLUE>q<RESET>uit	  8: <BOLD;BLUE>h<RESET>elp
+	<BOLD;BLUE>What now<RESET>>$SP
+	Bye.
+	EOF
+	test_write_lines h | GIT_PAGER_IN_USE=true TERM=vt100 git add -i >actual.colored &&
+	test_decode_color <actual.colored >actual &&
+	test_i18ncmp expect actual
+'
+
 test_done