diff --git a/Documentation/config/hook.adoc b/Documentation/config/hook.adoc index 9e78f26439..a9dc0063c1 100644 --- a/Documentation/config/hook.adoc +++ b/Documentation/config/hook.adoc @@ -15,6 +15,12 @@ hook..event:: events, specify the key more than once. An empty value resets the list of events, clearing any previously defined events for `hook.`. See linkgit:git-hook[1]. ++ +The `` must not be the same as a known hook event name +(e.g. do not use `hook.pre-commit.event`). Using a known event name as +a friendly-name is a fatal error because it creates an ambiguity with +`hook..enabled` and `hook..jobs`. For unknown event names, +a warning is issued when `` matches the event value. hook..enabled:: Whether the hook `hook.` is enabled. Defaults to `true`. @@ -22,3 +28,73 @@ hook..enabled:: configuration. This is particularly useful when a hook is defined in a system or global config file and needs to be disabled for a specific repository. See linkgit:git-hook[1]. + +hook..parallel:: + Whether the hook `hook.` may run in parallel with other hooks + for the same event. Defaults to `false`. Set to `true` only when the + hook script is safe to run concurrently with other hooks for the same + event. If any hook for an event does not have this set to `true`, + all hooks for that event run sequentially regardless of `hook.jobs`. + Only configured (named) hooks need to declare this. Traditional hooks + found in the hooks directory do not need to, and run in parallel when + the effective job count is greater than 1. See linkgit:git-hook[1]. + +hook..enabled:: + Switch to enable or disable all hooks for the `` hook event. + When set to `false`, no hooks fire for that event, regardless of any + per-hook `hook..enabled` settings. Defaults to `true`. + See linkgit:git-hook[1]. ++ +Note on naming: `` must be the event name (e.g. `pre-commit`), +not a hook friendly-name. Since using a known event name as a +friendly-name is disallowed (see `hook..event` above), +there is no ambiguity between event-level and per-hook `.enabled` +settings for known events. For unknown events, if a friendly-name +matches the event name despite the warning, `.enabled` is treated +as per-hook only. + +hook..jobs:: + Specifies how many hooks can be run simultaneously for the `` + hook event (e.g. `hook.post-receive.jobs = 4`). Overrides `hook.jobs` + for this specific event. The same parallelism restrictions apply: this + setting has no effect unless all configured hooks for the event have + `hook..parallel` set to `true`. Set to `-1` to use the + number of available CPU cores. Must be a positive integer or `-1`; + zero is rejected with a warning. See linkgit:git-hook[1]. ++ +Note on naming: although this key resembles `hook..*` +(a per-hook setting), `` must be the event name, not a hook +friendly name. The key component is stored literally and looked up by +event name at runtime with no translation between the two namespaces. +A key like `hook.my-hook.jobs` is stored under `"my-hook"` but the +lookup at runtime uses the event name (e.g. `"post-receive"`), so +`hook.my-hook.jobs` is silently ignored even when `my-hook` is +registered for that event. Use `hook.post-receive.jobs` or any other +valid event name when setting `hook..jobs`. + +hook.jobs:: + Specifies how many hooks can be run simultaneously during parallelized + hook execution. If unspecified, defaults to 1 (serial execution). + Set to `-1` to use the number of available CPU cores. + Can be overridden on a per-event basis with `hook..jobs`. + Some hooks always run sequentially regardless of this setting because + they operate on shared data and cannot safely be parallelized: ++ +-- +`applypatch-msg`;; +`prepare-commit-msg`;; +`commit-msg`;; + Receive a commit message file and may rewrite it in place. +`pre-commit`;; +`post-checkout`;; +`push-to-checkout`;; +`post-commit`;; + Access the working tree, index, or repository state. +-- ++ +This setting has no effect unless all configured hooks for the event have +`hook..parallel` set to `true`. ++ +For `pre-push` hooks, which normally keep stdout and stderr separate, +setting this to a value greater than 1 (or passing `-j`) will merge stdout +into stderr to allow correct de-interleaving of parallel output. diff --git a/Documentation/git-hook.adoc b/Documentation/git-hook.adoc index 318c637bd8..46ea52db55 100644 --- a/Documentation/git-hook.adoc +++ b/Documentation/git-hook.adoc @@ -8,7 +8,8 @@ git-hook - Run git hooks SYNOPSIS -------- [verse] -'git hook' run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [-- ] +'git hook' run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [(-j|--jobs) ] + [-- ] 'git hook' list [--allow-unknown-hook-name] [-z] [--show-scope] DESCRIPTION @@ -147,6 +148,23 @@ OPTIONS mirroring the output style of `git config --show-scope`. Traditional hooks from the hookdir are unaffected. +-j:: +--jobs:: + Only valid for `run`. ++ +Specify how many hooks to run simultaneously. If this flag is not specified, +the value of the `hook.jobs` config is used, see linkgit:git-config[1]. If +neither is specified, defaults to 1 (serial execution). ++ +When greater than 1, it overrides the per-hook `hook..parallel` +setting, allowing all hooks for the event to run concurrently, even if they +are not individually marked as parallel. ++ +Some hooks always run sequentially regardless of this flag or the +`hook.jobs` config, because git knows they cannot safely run in parallel: +`applypatch-msg`, `pre-commit`, `prepare-commit-msg`, `commit-msg`, +`post-commit`, `post-checkout`, and `push-to-checkout`. + WRAPPERS -------- @@ -169,7 +187,8 @@ running: git hook run --allow-unknown-hook-name mywrapper-start-tests \ # providing something to stdin --stdin some-tempfile-123 \ - # execute hooks in serial + # execute multiple hooks in parallel + --jobs 3 \ # plus some arguments of your own... -- \ --testname bar \ diff --git a/Makefile b/Makefile index cedc234173..bd24dd0409 100644 --- a/Makefile +++ b/Makefile @@ -2671,7 +2671,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h builtin/bugreport.sp builtin/bugreport.s builtin/bugreport.o: hook-list.h -builtin/hook.sp builtin/hook.s builtin/hook.o: hook-list.h +hook.sp hook.s hook.o: hook-list.h builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ diff --git a/builtin/am.c b/builtin/am.c index fe6e087eee..e9623b8307 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -490,9 +490,11 @@ static int run_applypatch_msg_hook(struct am_state *state) assert(state->msg); - if (!state->no_verify) - ret = run_hooks_l(the_repository, "applypatch-msg", - am_path(state, "final-commit"), NULL); + if (!state->no_verify) { + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; + strvec_push(&opt.args, am_path(state, "final-commit")); + ret = run_hooks_opt(the_repository, "applypatch-msg", &opt); + } if (!ret) { FREE_AND_NULL(state->msg); diff --git a/builtin/checkout.c b/builtin/checkout.c index e031e61886..ac0186a33e 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -31,6 +31,7 @@ #include "resolve-undo.h" #include "revision.h" #include "setup.h" +#include "strvec.h" #include "submodule.h" #include "symlinks.h" #include "trace2.h" @@ -123,13 +124,19 @@ static void branch_info_release(struct branch_info *info) static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit, int changed) { - return run_hooks_l(the_repository, "post-checkout", - oid_to_hex(old_commit ? &old_commit->object.oid : null_oid(the_hash_algo)), - oid_to_hex(new_commit ? &new_commit->object.oid : null_oid(the_hash_algo)), - changed ? "1" : "0", NULL); - /* "new_commit" can be NULL when checking out from the index before - a commit exists. */ + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; + /* + * "new_commit" can be NULL when checking out from the index before + * a commit exists. + */ + strvec_pushl(&opt.args, + oid_to_hex(old_commit ? &old_commit->object.oid : null_oid(the_hash_algo)), + oid_to_hex(new_commit ? &new_commit->object.oid : null_oid(the_hash_algo)), + changed ? "1" : "0", + NULL); + + return run_hooks_opt(the_repository, "post-checkout", &opt); } static int update_some(const struct object_id *oid, struct strbuf *base, diff --git a/builtin/clone.c b/builtin/clone.c index fba3c9c508..d23b0cafcf 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -647,6 +647,7 @@ static int checkout(int submodule_progress, struct tree *tree; struct tree_desc t; int err = 0; + struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; if (option_no_checkout) return 0; @@ -697,8 +698,9 @@ static int checkout(int submodule_progress, if (write_locked_index(the_repository->index, &lock_file, COMMIT_LOCK)) die(_("unable to write new index file")); - err |= run_hooks_l(the_repository, "post-checkout", oid_to_hex(null_oid(the_hash_algo)), - oid_to_hex(&oid), "1", NULL); + strvec_pushl(&hook_opt.args, oid_to_hex(null_oid(the_hash_algo)), + oid_to_hex(&oid), "1", NULL); + err |= run_hooks_opt(the_repository, "post-checkout", &hook_opt); if (!err && (option_recurse_submodules.nr > 0)) { struct child_process cmd = CHILD_PROCESS_INIT; diff --git a/builtin/hook.c b/builtin/hook.c index c0585587e5..cceeb3586e 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -4,23 +4,15 @@ #include "environment.h" #include "gettext.h" #include "hook.h" -#include "hook-list.h" #include "parse-options.h" +#include "thread-utils.h" #define BUILTIN_HOOK_RUN_USAGE \ - N_("git hook run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [-- ]") + N_("git hook run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [(-j|--jobs) ]\n" \ + " [-- ]") #define BUILTIN_HOOK_LIST_USAGE \ N_("git hook list [--allow-unknown-hook-name] [-z] [--show-scope] ") -static int is_known_hook(const char *name) -{ - const char **p; - for (p = hook_name_list; *p; p++) - if (!strcmp(*p, name)) - return 1; - return 0; -} - static const char * const builtin_hook_usage[] = { BUILTIN_HOOK_RUN_USAGE, BUILTIN_HOOK_LIST_USAGE, @@ -96,14 +88,22 @@ static int list(int argc, const char **argv, const char *prefix, const char *name = h->u.configured.friendly_name; const char *scope = show_scope ? config_scope_name(h->u.configured.scope) : NULL; + /* + * Show the most relevant disable reason. Event-level + * takes precedence: if the whole event is off, that + * is what the user needs to know. The per-hook + * "disabled" surfaces once the event is re-enabled. + */ + const char *disability = + h->u.configured.event_disabled ? "event-disabled\t" : + h->u.configured.disabled ? "disabled\t" : + ""; if (scope) - printf("%s\t%s%s%c", scope, - h->u.configured.disabled ? "disabled\t" : "", - name, line_terminator); + printf("%s\t%s%s%c", scope, disability, name, + line_terminator); else - printf("%s%s%c", - h->u.configured.disabled ? "disabled\t" : "", - name, line_terminator); + printf("%s%s%c", disability, name, + line_terminator); break; } default: @@ -124,6 +124,7 @@ static int run(int argc, const char **argv, const char *prefix, struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; int ignore_missing = 0; int allow_unknown = 0; + int jobs = 0; const char *hook_name; struct option run_options[] = { OPT_BOOL(0, "allow-unknown-hook-name", &allow_unknown, @@ -132,6 +133,8 @@ static int run(int argc, const char **argv, const char *prefix, N_("silently ignore missing requested ")), OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"), N_("file to read into hooks' stdin")), + OPT_INTEGER('j', "jobs", &jobs, + N_("run up to hooks simultaneously (-1 for CPU count)")), OPT_END(), }; int ret; @@ -140,6 +143,15 @@ static int run(int argc, const char **argv, const char *prefix, builtin_hook_run_usage, PARSE_OPT_KEEP_DASHDASH); + if (jobs == -1) + opt.jobs = online_cpus(); + else if (jobs < 0) + die(_("invalid value for -j: %d" + " (use -1 for CPU count or a" + " positive integer)"), jobs); + else + opt.jobs = jobs; + if (!argc) goto usage; diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 878aa7f0ed..f0771590a7 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1456,7 +1456,8 @@ static const char *push_to_checkout(unsigned char *hash, struct strvec *env, const char *work_tree) { - struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; + opt.invoked_hook = invoked_hook; strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree)); diff --git a/builtin/worktree.c b/builtin/worktree.c index 4fd6f7575f..d21c43fde3 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -609,7 +609,7 @@ done: * is_junk is cleared, but do return appropriate code when hook fails. */ if (!ret && opts->checkout && !opts->orphan) { - struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL); strvec_pushl(&opt.args, diff --git a/commit.c b/commit.c index 80d8d07875..4385ae4329 100644 --- a/commit.c +++ b/commit.c @@ -1970,7 +1970,7 @@ size_t ignored_log_message_bytes(const char *buf, size_t len) int run_commit_hook(int editor_is_used, const char *index_file, int *invoked_hook, const char *name, ...) { - struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; va_list args; const char *arg; diff --git a/config.c b/config.c index 156f2a24fa..a1b92fe083 100644 --- a/config.c +++ b/config.c @@ -1212,6 +1212,15 @@ int git_config_int(const char *name, const char *value, return ret; } +unsigned int git_config_uint(const char *name, const char *value, + const struct key_value_info *kvi) +{ + unsigned int ret; + if (!git_parse_uint(value, &ret)) + die_bad_number(name, value, kvi); + return ret; +} + int64_t git_config_int64(const char *name, const char *value, const struct key_value_info *kvi) { @@ -1907,6 +1916,18 @@ int git_configset_get_int(struct config_set *set, const char *key, int *dest) return 1; } +int git_configset_get_uint(struct config_set *set, const char *key, unsigned int *dest) +{ + const char *value; + struct key_value_info kvi; + + if (!git_configset_get_value(set, key, &value, &kvi)) { + *dest = git_config_uint(key, value, &kvi); + return 0; + } else + return 1; +} + int git_configset_get_ulong(struct config_set *set, const char *key, unsigned long *dest) { const char *value; @@ -2356,6 +2377,13 @@ int repo_config_get_int(struct repository *repo, return git_configset_get_int(repo->config, key, dest); } +int repo_config_get_uint(struct repository *repo, + const char *key, unsigned int *dest) +{ + git_config_check_init(repo); + return git_configset_get_uint(repo->config, key, dest); +} + int repo_config_get_ulong(struct repository *repo, const char *key, unsigned long *dest) { diff --git a/config.h b/config.h index ba426a960a..bf47fb3afc 100644 --- a/config.h +++ b/config.h @@ -267,6 +267,12 @@ int git_config_int(const char *, const char *, const struct key_value_info *); int64_t git_config_int64(const char *, const char *, const struct key_value_info *); +/** + * Identical to `git_config_int`, but for unsigned ints. + */ +unsigned int git_config_uint(const char *, const char *, + const struct key_value_info *); + /** * Identical to `git_config_int`, but for unsigned longs. */ @@ -560,6 +566,7 @@ int git_configset_get_value(struct config_set *cs, const char *key, int git_configset_get_string(struct config_set *cs, const char *key, char **dest); int git_configset_get_int(struct config_set *cs, const char *key, int *dest); +int git_configset_get_uint(struct config_set *cs, const char *key, unsigned int *dest); int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest); int git_configset_get_bool(struct config_set *cs, const char *key, int *dest); int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest); @@ -650,6 +657,12 @@ int repo_config_get_string_tmp(struct repository *r, */ int repo_config_get_int(struct repository *r, const char *key, int *dest); +/** + * Similar to `repo_config_get_int` but for unsigned ints. + */ +int repo_config_get_uint(struct repository *r, + const char *key, unsigned int *dest); + /** * Similar to `repo_config_get_int` but for unsigned longs. */ diff --git a/hook.c b/hook.c index cc23276d27..d10eef4763 100644 --- a/hook.c +++ b/hook.c @@ -5,12 +5,23 @@ #include "environment.h" #include "gettext.h" #include "hook.h" +#include "hook-list.h" #include "parse.h" #include "path.h" #include "run-command.h" #include "setup.h" #include "strbuf.h" #include "strmap.h" +#include "thread-utils.h" + +bool is_known_hook(const char *name) +{ + const char **h; + for (h = hook_name_list; *h; h++) + if (!strcmp(*h, name)) + return true; + return false; +} const char *find_hook(struct repository *r, const char *name) { @@ -116,18 +127,27 @@ struct hook_config_cache_entry { char *command; enum config_scope scope; bool disabled; + bool parallel; }; /* * Callback struct to collect all hook.* keys in a single config pass. * commands: friendly-name to command map. * event_hooks: event-name to list of friendly-names map. - * disabled_hooks: set of friendly-names with hook..enabled = false. + * disabled_hooks: set of all names with hook..enabled = false; after + * parsing, names that are not friendly-names become event-level + * disables stored in r->disabled_events. This collects all. + * parallel_hooks: friendly-name to parallel flag. + * event_jobs: event-name to per-event jobs count (stored as uintptr_t, NULL == unset). + * jobs: value of the global hook.jobs key. Defaults to 0 if unset (stored in r->hook_jobs). */ struct hook_all_config_cb { struct strmap commands; struct strmap event_hooks; struct string_list disabled_hooks; + struct strmap parallel_hooks; + struct strmap event_jobs; + unsigned int jobs; }; /* repo_config() callback that collects all hook.* configuration in one pass. */ @@ -143,6 +163,24 @@ static int hook_config_lookup_all(const char *key, const char *value, if (parse_config_key(key, "hook", &name, &name_len, &subkey)) return 0; + /* Handle plain hook. entries that have no hook name component. */ + if (!name) { + if (!strcmp(subkey, "jobs") && value) { + int v; + if (!git_parse_int(value, &v)) + warning(_("hook.jobs must be an integer, ignoring: '%s'"), value); + else if (v == -1) + data->jobs = online_cpus(); + else if (v > 0) + data->jobs = v; + else + warning(_("hook.jobs must be a positive integer" + " or -1, ignoring: '%s'"), + value); + } + return 0; + } + if (!value) return config_error_nonbool(key); @@ -158,8 +196,21 @@ static int hook_config_lookup_all(const char *key, const char *value, strmap_for_each_entry(&data->event_hooks, &iter, e) unsorted_string_list_remove(e->value, hook_name, 0); } else { - struct string_list *hooks = - strmap_get(&data->event_hooks, value); + struct string_list *hooks; + + if (is_known_hook(hook_name)) + die(_("hook friendly-name '%s' collides with " + "a known event name; please choose a " + "different friendly-name"), + hook_name); + + if (!strcmp(hook_name, value)) + warning(_("hook friendly-name '%s' is the " + "same as its event; this may cause " + "ambiguity with hook.%s.enabled"), + hook_name, hook_name); + + hooks = strmap_get(&data->event_hooks, value); if (!hooks) { CALLOC_ARRAY(hooks, 1); @@ -203,6 +254,31 @@ static int hook_config_lookup_all(const char *key, const char *value, default: break; /* ignore unrecognised values */ } + } else if (!strcmp(subkey, "parallel")) { + int v = git_parse_maybe_bool(value); + if (v >= 0) + strmap_put(&data->parallel_hooks, hook_name, + (void *)(uintptr_t)v); + else + warning(_("hook.%s.parallel must be a boolean," + " ignoring: '%s'"), + hook_name, value); + } else if (!strcmp(subkey, "jobs")) { + int v; + if (!git_parse_int(value, &v)) + warning(_("hook.%s.jobs must be an integer," + " ignoring: '%s'"), + hook_name, value); + else if (v == -1) + strmap_put(&data->event_jobs, hook_name, + (void *)(uintptr_t)online_cpus()); + else if (v > 0) + strmap_put(&data->event_jobs, hook_name, + (void *)(uintptr_t)v); + else + warning(_("hook.%s.jobs must be a positive" + " integer or -1, ignoring: '%s'"), + hook_name, value); } free(hook_name); @@ -237,20 +313,78 @@ void hook_cache_clear(struct strmap *cache) strmap_clear(cache, 0); } +/* + * Return true if `name` is a hook friendly-name, i.e. it has at least one of + * .command, .event, or .parallel configured. These are the reliable clues + * that distinguish a friendly-name from an event name. Note: .enabled is + * deliberately excluded because it can appear under both namespaces. + */ +static int is_friendly_name(struct hook_all_config_cb *cb, const char *name) +{ + struct hashmap_iter iter; + struct strmap_entry *e; + + if (strmap_get(&cb->commands, name) || strmap_get(&cb->parallel_hooks, name)) + return 1; + + strmap_for_each_entry(&cb->event_hooks, &iter, e) { + if (unsorted_string_list_lookup(e->value, name)) + return 1; + } + + return 0; +} + +/* Warn if any name in event_jobs is also a hook friendly-name. */ +static void warn_jobs_on_friendly_names(struct hook_all_config_cb *cb_data) +{ + struct hashmap_iter iter; + struct strmap_entry *e; + + strmap_for_each_entry(&cb_data->event_jobs, &iter, e) { + if (is_friendly_name(cb_data, e->key)) + warning(_("hook.%s.jobs is set but '%s' looks like a " + "hook friendly-name, not an event name; " + "hook..jobs uses the event name " + "(e.g. hook.post-receive.jobs), so this " + "setting will be ignored"), e->key, e->key); + } +} + /* Populate `cache` with the complete hook configuration */ static void build_hook_config_map(struct repository *r, struct strmap *cache) { - struct hook_all_config_cb cb_data; + struct hook_all_config_cb cb_data = { 0 }; struct hashmap_iter iter; struct strmap_entry *e; strmap_init(&cb_data.commands); strmap_init(&cb_data.event_hooks); string_list_init_dup(&cb_data.disabled_hooks); + strmap_init(&cb_data.parallel_hooks); + strmap_init(&cb_data.event_jobs); - /* Parse all configs in one run. */ + /* Parse all configs in one run, capturing hook.* including hook.jobs. */ repo_config(r, hook_config_lookup_all, &cb_data); + warn_jobs_on_friendly_names(&cb_data); + + /* + * Populate disabled_events: names in disabled_hooks that are not + * friendly-names are event-level switches (hook..enabled = false). + * Names that are friendly-names are already handled per-hook via the + * hook_config_cache_entry.disabled flag below. + */ + if (r) { + string_list_clear(&r->disabled_events, 0); + string_list_init_dup(&r->disabled_events); + for (size_t i = 0; i < cb_data.disabled_hooks.nr; i++) { + const char *n = cb_data.disabled_hooks.items[i].string; + if (!is_friendly_name(&cb_data, n)) + string_list_append(&r->disabled_events, n); + } + } + /* Construct the cache from parsed configs. */ strmap_for_each_entry(&cb_data.event_hooks, &iter, e) { struct string_list *hook_names = e->value; @@ -266,6 +400,7 @@ static void build_hook_config_map(struct repository *r, struct strmap *cache) struct hook_config_cache_entry *entry; char *command; + bool is_par = !!strmap_get(&cb_data.parallel_hooks, hname); bool is_disabled = !!unsorted_string_list_lookup( &cb_data.disabled_hooks, hname); @@ -286,13 +421,20 @@ static void build_hook_config_map(struct repository *r, struct strmap *cache) entry->command = xstrdup_or_null(command); entry->scope = scope; entry->disabled = is_disabled; + entry->parallel = is_par; string_list_append(hooks, hname)->util = entry; } strmap_put(cache, e->key, hooks); } + if (r) { + r->hook_jobs = cb_data.jobs; + r->event_jobs = cb_data.event_jobs; + } + strmap_clear(&cb_data.commands, 1); + strmap_clear(&cb_data.parallel_hooks, 0); /* values are uintptr_t, not heap ptrs */ string_list_clear(&cb_data.disabled_hooks, 0); strmap_for_each_entry(&cb_data.event_hooks, &iter, e) { string_list_clear(e->value, 0); @@ -344,6 +486,8 @@ static void list_hooks_add_configured(struct repository *r, { struct strmap *cache = get_hook_config_cache(r); struct string_list *configured_hooks = strmap_get(cache, hookname); + bool event_is_disabled = r ? !!unsorted_string_list_lookup(&r->disabled_events, + hookname) : 0; /* Iterate through configured hooks and initialize internal states */ for (size_t i = 0; configured_hooks && i < configured_hooks->nr; i++) { @@ -370,6 +514,8 @@ static void list_hooks_add_configured(struct repository *r, entry->command ? xstrdup(entry->command) : NULL; hook->u.configured.scope = entry->scope; hook->u.configured.disabled = entry->disabled; + hook->u.configured.event_disabled = event_is_disabled; + hook->parallel = entry->parallel; string_list_append(list, friendly_name)->util = hook; } @@ -381,6 +527,8 @@ static void list_hooks_add_configured(struct repository *r, if (!r || !r->gitdir) { hook_cache_clear(cache); free(cache); + if (r) + string_list_clear(&r->disabled_events, 0); } } @@ -412,7 +560,7 @@ int hook_exists(struct repository *r, const char *name) for (size_t i = 0; i < hooks->nr; i++) { struct hook *h = hooks->items[i].util; if (h->kind == HOOK_TRADITIONAL || - !h->u.configured.disabled) { + (!h->u.configured.disabled && !h->u.configured.event_disabled)) { exists = 1; break; } @@ -435,7 +583,8 @@ static int pick_next_hook(struct child_process *cp, if (hook_cb->hook_to_run_index >= hook_list->nr) return 0; h = hook_list->items[hook_cb->hook_to_run_index++].util; - } while (h->kind == HOOK_CONFIGURED && h->u.configured.disabled); + } while (h->kind == HOOK_CONFIGURED && + (h->u.configured.disabled || h->u.configured.event_disabled)); cp->no_stdin = 1; strvec_pushv(&cp->env, hook_cb->options->env.v); @@ -519,21 +668,136 @@ static void run_hooks_opt_clear(struct run_hooks_opt *options) strvec_clear(&options->args); } +/* + * When running in parallel, stdout must be merged into stderr so + * run-command can buffer and de-interleave outputs correctly. This + * applies even to hooks like pre-push that normally keep stdout and + * stderr separate: the user has opted into parallelism, so the output + * stream behavior changes accordingly. + */ +static void merge_output_if_parallel(struct run_hooks_opt *options) +{ + if (options->jobs > 1) + options->stdout_to_stderr = 1; +} + +static void warn_non_parallel_hooks_override(unsigned int jobs, + struct string_list *hook_list) +{ + /* Don't warn for hooks running sequentially. */ + if (jobs == 1) + return; + + for (size_t i = 0; i < hook_list->nr; i++) { + struct hook *h = hook_list->items[i].util; + if (h->kind == HOOK_CONFIGURED && !h->parallel) + warning(_("hook '%s' is not marked as parallel=true, " + "running in parallel anyway due to -j%u"), + h->u.configured.friendly_name, jobs); + } +} + +/* Resolve a hook.jobs config key, handling -1 as online_cpus(). */ +static void resolve_hook_config_jobs(struct repository *r, + const char *key, + unsigned int *jobs) +{ + int v; + + if (repo_config_get_int(r, key, &v)) + return; + + if (v == -1) + *jobs = online_cpus(); + else if (v > 0) + *jobs = v; + else + warning(_("%s must be a positive integer or -1," + " ignoring: %d"), key, v); +} + +/* Determine how many jobs to use for hook execution. */ +static unsigned int get_hook_jobs(struct repository *r, + struct run_hooks_opt *options, + const char *hook_name, + struct string_list *hook_list) +{ + /* + * An explicit job count overrides everything else: this covers both + * FORCE_SERIAL callers (for hooks that must never run in parallel) + * and the -j flag from the CLI. The CLI override is intentional: users + * may want to serialize hooks declared parallel or to parallelize more + * aggressively than the default. + */ + if (options->jobs) + goto cleanup; + + /* + * Use hook.jobs from the already-parsed config cache (in-repo), or + * fallback to a direct config lookup (out-of-repo). + * Default to 1 (serial execution) on failure. + */ + options->jobs = 1; + if (r) { + if (r->gitdir && r->hook_config_cache) { + void *event_jobs; + + if (r->hook_jobs) + options->jobs = r->hook_jobs; + + event_jobs = strmap_get(&r->event_jobs, hook_name); + if (event_jobs) + options->jobs = (unsigned int)(uintptr_t)event_jobs; + } else { + char *key; + + resolve_hook_config_jobs(r, "hook.jobs", &options->jobs); + + key = xstrfmt("hook.%s.jobs", hook_name); + resolve_hook_config_jobs(r, key, &options->jobs); + free(key); + } + } + + /* + * Cap to serial any configured hook not marked as parallel = true. + * This enforces the parallel = false default, even for "traditional" + * hooks from the hookdir which cannot be marked parallel = true. + * The same restriction applies whether jobs came from hook.jobs or + * hook..jobs. + */ + for (size_t i = 0; i < hook_list->nr; i++) { + struct hook *h = hook_list->items[i].util; + if (h->kind == HOOK_CONFIGURED && !h->parallel) { + options->jobs = 1; + break; + } + } + +cleanup: + merge_output_if_parallel(options); + warn_non_parallel_hooks_override(options->jobs, hook_list); + return options->jobs; +} + int run_hooks_opt(struct repository *r, const char *hook_name, struct run_hooks_opt *options) { + struct string_list *hook_list = list_hooks(r, hook_name, options); struct hook_cb_data cb_data = { .rc = 0, .hook_name = hook_name, + .hook_command_list = hook_list, .options = options, }; int ret = 0; + unsigned int jobs = get_hook_jobs(r, options, hook_name, hook_list); const struct run_process_parallel_opts opts = { .tr2_category = "hook", .tr2_label = hook_name, - .processes = options->jobs, - .ungroup = options->jobs == 1, + .processes = jobs, + .ungroup = jobs == 1, .get_next_task = pick_next_hook, .start_failure = notify_start_failure, @@ -549,9 +813,6 @@ int run_hooks_opt(struct repository *r, const char *hook_name, if (options->path_to_stdin && options->feed_pipe) BUG("options path_to_stdin and feed_pipe are mutually exclusive"); - if (!options->jobs) - BUG("run_hooks_opt must be called with options.jobs >= 1"); - /* * Ensure cb_data copy and free functions are either provided together, * or neither one is provided. @@ -562,7 +823,6 @@ int run_hooks_opt(struct repository *r, const char *hook_name, if (options->invoked_hook) *options->invoked_hook = 0; - cb_data.hook_command_list = list_hooks(r, hook_name, options); if (!cb_data.hook_command_list->nr) { if (options->error_if_missing) ret = error("cannot find a hook named %s", hook_name); diff --git a/hook.h b/hook.h index 5c5628dd1f..b4372b636f 100644 --- a/hook.h +++ b/hook.h @@ -32,9 +32,17 @@ struct hook { const char *command; enum config_scope scope; bool disabled; + bool event_disabled; } configured; } u; + /** + * Whether this hook may run in parallel with other hooks for the same + * event. Only useful for configured (named) hooks. Traditional hooks + * always default to 0 (serial). Set via `hook..parallel = true`. + */ + bool parallel; + /** * Opaque data pointer used to keep internal state across callback calls. * @@ -72,6 +80,8 @@ struct run_hooks_opt { * * If > 1, output will be buffered and de-interleaved (ungroup=0). * If == 1, output will be real-time (ungroup=1). + * If == 0, the 'hook.jobs' config is used or, if the config is unset, + * defaults to 1 (serial execution). */ unsigned int jobs; @@ -97,8 +107,10 @@ struct run_hooks_opt { * Send the hook's stdout to stderr. * * This is the default behavior for all hooks except pre-push, - * which has separate stdout and stderr streams for backwards - * compatibility reasons. + * which keeps stdout and stderr separate for backwards compatibility. + * When parallel execution is requested (jobs > 1), get_hook_jobs() + * overrides this to 1 for all hooks so run-command can de-interleave + * their outputs correctly. */ unsigned int stdout_to_stderr:1; @@ -152,7 +164,23 @@ struct run_hooks_opt { hook_data_free_fn feed_pipe_cb_data_free; }; +/** + * Default initializer for hooks. Parallelism is opt-in: .jobs = 0 defers to + * the 'hook.jobs' config, falling back to serial (1) if unset. + */ #define RUN_HOOKS_OPT_INIT { \ + .env = STRVEC_INIT, \ + .args = STRVEC_INIT, \ + .stdout_to_stderr = 1, \ + .jobs = 0, \ +} + +/** + * Initializer for hooks that must always run sequentially regardless of + * 'hook.jobs'. Use this when git knows the hook cannot safely be parallelized + * .jobs = 1 is non-overridable. + */ +#define RUN_HOOKS_OPT_INIT_FORCE_SERIAL { \ .env = STRVEC_INIT, \ .args = STRVEC_INIT, \ .stdout_to_stderr = 1, \ @@ -207,6 +235,12 @@ void hook_free(void *p, const char *str); */ void hook_cache_clear(struct strmap *cache); +/** + * Returns true if `name` is a recognized hook event name + * (e.g. "pre-commit", "post-receive"). + */ +bool is_known_hook(const char *name); + /** * Returns the path to the hook file, or NULL if the hook is missing * or disabled. Note that this points to static storage that will be diff --git a/meson.build b/meson.build index 11488623bf..b28571e08e 100644 --- a/meson.build +++ b/meson.build @@ -563,6 +563,18 @@ libgit_sources += custom_target( env: script_environment, ) +libgit_sources += custom_target( + input: 'Documentation/githooks.adoc', + output: 'hook-list.h', + command: [ + shell, + meson.current_source_dir() + '/tools/generate-hooklist.sh', + meson.current_source_dir(), + '@OUTPUT@', + ], + env: script_environment, +) + builtin_sources = [ 'builtin/add.c', 'builtin/am.c', @@ -739,18 +751,6 @@ builtin_sources += custom_target( env: script_environment, ) -builtin_sources += custom_target( - input: 'Documentation/githooks.adoc', - output: 'hook-list.h', - command: [ - shell, - meson.current_source_dir() + '/tools/generate-hooklist.sh', - meson.current_source_dir(), - '@OUTPUT@', - ], - env: script_environment, -) - # This contains the variables for GIT-BUILD-OPTIONS, which we use to propagate # build options to our tests. build_options_config = configuration_data() diff --git a/parse.c b/parse.c index 48313571aa..d77f28046a 100644 --- a/parse.c +++ b/parse.c @@ -107,6 +107,15 @@ int git_parse_int64(const char *value, int64_t *ret) return 1; } +int git_parse_uint(const char *value, unsigned int *ret) +{ + uintmax_t tmp; + if (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(unsigned int))) + return 0; + *ret = tmp; + return 1; +} + int git_parse_ulong(const char *value, unsigned long *ret) { uintmax_t tmp; diff --git a/parse.h b/parse.h index ea32de9a91..a6dd37c4cb 100644 --- a/parse.h +++ b/parse.h @@ -5,6 +5,7 @@ int git_parse_signed(const char *value, intmax_t *ret, intmax_t max); int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max); int git_parse_ssize_t(const char *, ssize_t *); int git_parse_ulong(const char *, unsigned long *); +int git_parse_uint(const char *value, unsigned int *ret); int git_parse_int(const char *value, int *ret); int git_parse_int64(const char *value, int64_t *ret); int git_parse_double(const char *value, double *ret); diff --git a/repository.c b/repository.c index 9e5537f539..db57b8308b 100644 --- a/repository.c +++ b/repository.c @@ -323,6 +323,7 @@ int repo_init(struct repository *repo, return 0; error: + clear_repository_format(&format); repo_clear(repo); return -1; } @@ -425,6 +426,8 @@ void repo_clear(struct repository *repo) hook_cache_clear(repo->hook_config_cache); FREE_AND_NULL(repo->hook_config_cache); } + strmap_clear(&repo->event_jobs, 0); /* values are uintptr_t, not heap ptrs */ + string_list_clear(&repo->disabled_events, 0); if (repo->promisor_remote_config) { promisor_remote_clear(repo->promisor_remote_config); diff --git a/repository.h b/repository.h index 078059a6e0..4969d8b8eb 100644 --- a/repository.h +++ b/repository.h @@ -2,6 +2,7 @@ #define REPOSITORY_H #include "strmap.h" +#include "string-list.h" #include "repo-settings.h" #include "environment.h" @@ -172,6 +173,15 @@ struct repository { */ struct strmap *hook_config_cache; + /* Cached value of hook.jobs config (0 if unset, defaults to serial). */ + unsigned int hook_jobs; + + /* Cached map of event-name -> jobs count (as uintptr_t) from hook..jobs. */ + struct strmap event_jobs; + + /* Cached list of event names with hook..enabled = false. */ + struct string_list disabled_events; + /* Configurations related to promisor remotes. */ char *repository_format_partial_clone; struct promisor_remote_config *promisor_remote_config; diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh index 33decc66c0..0132e772e4 100755 --- a/t/t1800-hook.sh +++ b/t/t1800-hook.sh @@ -21,6 +21,57 @@ setup_hookdir () { test_when_finished rm -rf .git/hooks } +# write_sentinel_hook [sentinel] +# +# Writes a hook that marks itself as started, sleeps for a few seconds, then +# marks itself done. The sleep must be long enough that sentinel_detector can +# observe .started before .done appears when both hooks +# run concurrently in parallel mode. +write_sentinel_hook () { + sentinel="${2:-sentinel}" + write_script "$1" <<-EOF + touch ${sentinel}.started && + sleep 2 && + touch ${sentinel}.done + EOF +} + +# sentinel_detector +# +# Returns a shell command string suitable for use as hook..command. +# The detector must be registered after the sentinel: +# 1. In serial mode, the sentinel has completed (and .done exists) +# before the detector starts. +# 2. In parallel mode, both run concurrently so .done has not appeared +# yet and the detector just sees .started. +# +# At start, poll until .started exists to absorb startup jitter, then +# write to : +# 1. 'serial' if .done exists (sentinel finished before we started), +# 2. 'parallel' if only .started exists (sentinel still running), +# 3. 'timeout' if .started never appeared. +# +# The command ends with ':' so when git appends "$@" for hooks that receive +# positional arguments (e.g. pre-push), the result ': "$@"' is valid shell +# rather than a syntax error 'fi "$@"'. +sentinel_detector () { + cat <<-EOF + i=0 + while ! test -f ${1}.started && test \$i -lt 10; do + sleep 1 + i=\$((i+1)) + done + if test -f ${1}.done; then + echo serial >${2} + elif test -f ${1}.started; then + echo parallel >${2} + else + echo timeout >${2} + fi + : + EOF +} + test_expect_success 'git hook usage' ' test_expect_code 129 git hook && test_expect_code 129 git hook run && @@ -217,10 +268,20 @@ test_expect_success 'git -c core.hooksPath= hook run' ' ' test_hook_tty () { - cat >expect <<-\EOF - STDOUT TTY - STDERR TTY - EOF + expect_tty=$1 + shift + + if test "$expect_tty" != "no_tty"; then + cat >expect <<-\EOF + STDOUT TTY + STDERR TTY + EOF + else + cat >expect <<-\EOF + STDOUT NO TTY + STDERR NO TTY + EOF + fi test_when_finished "rm -rf repo" && git init repo && @@ -238,12 +299,21 @@ test_hook_tty () { test_cmp expect repo/actual } -test_expect_success TTY 'git hook run: stdout and stderr are connected to a TTY' ' - test_hook_tty hook run pre-commit +test_expect_success TTY 'git hook run -j1: stdout and stderr are connected to a TTY' ' + # hooks running sequentially (-j1) are always connected to the tty for + # optimum real-time performance. + test_hook_tty tty hook run -j1 pre-commit +' + +test_expect_success TTY 'git hook run -jN: stdout and stderr are not connected to a TTY' ' + # Hooks are not connected to the tty when run in parallel, instead they + # output to a pipe through which run-command collects and de-interlaces + # their outputs, which then gets passed either to the tty or a sideband. + test_hook_tty no_tty hook run -j2 pre-commit ' test_expect_success TTY 'git commit: stdout and stderr are connected to a TTY' ' - test_hook_tty commit -m"B.new" + test_hook_tty tty commit -m"B.new" ' test_expect_success 'git hook list orders by config order' ' @@ -658,4 +728,504 @@ test_expect_success 'server push-to-checkout hook expects stdout redirected to s check_stdout_merged_to_stderr push-to-checkout ' +test_expect_success 'parallel hook output is not interleaved' ' + test_when_finished "rm -rf .git/hooks" && + + write_script .git/hooks/test-hook <<-EOF && + echo "Hook 1 Start" + sleep 1 + echo "Hook 1 End" + EOF + + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "echo \"Hook 2 Start\"; sleep 2; echo \"Hook 2 End\"" && + test_config hook.hook-2.parallel true && + test_config hook.hook-3.event test-hook && + test_config hook.hook-3.command \ + "echo \"Hook 3 Start\"; sleep 3; echo \"Hook 3 End\"" && + test_config hook.hook-3.parallel true && + + git hook run --allow-unknown-hook-name -j3 test-hook >out 2>err.parallel && + + # Verify Hook 1 output is grouped + sed -n "/Hook 1 Start/,/Hook 1 End/p" err.parallel >hook1_out && + test_line_count = 2 hook1_out && + + # Verify Hook 2 output is grouped + sed -n "/Hook 2 Start/,/Hook 2 End/p" err.parallel >hook2_out && + test_line_count = 2 hook2_out && + + # Verify Hook 3 output is grouped + sed -n "/Hook 3 Start/,/Hook 3 End/p" err.parallel >hook3_out && + test_line_count = 2 hook3_out +' + +test_expect_success 'git hook run -j1 runs hooks in series' ' + test_when_finished "rm -rf .git/hooks" && + + test_config hook.series-1.event "test-hook" && + test_config hook.series-1.command "echo 1" --add && + test_config hook.series-2.event "test-hook" && + test_config hook.series-2.command "echo 2" --add && + + mkdir -p .git/hooks && + write_script .git/hooks/test-hook <<-EOF && + echo 3 + EOF + + cat >expected <<-\EOF && + 1 + 2 + 3 + EOF + + git hook run --allow-unknown-hook-name -j1 test-hook 2>actual && + test_cmp expected actual +' + +test_expect_success 'git hook run -j2 runs hooks in parallel' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_when_finished "rm -rf .git/hooks" && + + mkdir -p .git/hooks && + write_sentinel_hook .git/hooks/test-hook && + + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + git hook run --allow-unknown-hook-name -j2 test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'git hook run -j2 overrides parallel=false' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + # hook-1 intentionally has no parallel=true + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + # hook-2 also has no parallel=true + + # -j2 overrides parallel=false; hooks run in parallel with a warning. + git hook run --allow-unknown-hook-name -j2 test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'git hook run -j2 warns for hooks not marked parallel=true' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command "true" && + # neither hook has parallel=true + + git hook run --allow-unknown-hook-name -j2 test-hook >out 2>err && + grep "hook .hook-1. is not marked as parallel=true" err && + grep "hook .hook-2. is not marked as parallel=true" err +' + +test_expect_success 'hook.jobs=1 config runs hooks in series' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + + # Use two configured hooks so the execution order is deterministic: + # hook-1 (sentinel) is listed before hook-2 (detector), so hook-1 + # always runs first even in serial mode. + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + + test_config hook.jobs 1 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook.jobs=2 config runs hooks in parallel' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_when_finished "rm -rf .git/hooks" && + + mkdir -p .git/hooks && + write_sentinel_hook .git/hooks/test-hook && + + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..parallel=true enables parallel execution' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..parallel=false (default) forces serial execution' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'one non-parallel hook forces the whole event to run serially' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + # hook-2 has no parallel=true: should force serial for all + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'client hooks: pre-push parallel execution merges stdout to stderr' ' + test_when_finished "rm -rf remote-par stdout.actual stderr.actual" && + git init --bare remote-par && + git remote add origin-par remote-par && + test_commit par-commit && + mkdir -p .git/hooks && + setup_hooks pre-push && + test_config hook.jobs 2 && + git push origin-par HEAD:main >stdout.actual 2>stderr.actual && + check_stdout_merged_to_stderr pre-push +' + +test_expect_success 'client hooks: pre-push runs in parallel when hook.jobs > 1' ' + test_when_finished "rm -rf repo-parallel remote-parallel" && + git init --bare remote-parallel && + git init repo-parallel && + git -C repo-parallel remote add origin ../remote-parallel && + test_commit -C repo-parallel A && + + write_sentinel_hook repo-parallel/.git/hooks/pre-push && + git -C repo-parallel config hook.hook-2.event pre-push && + git -C repo-parallel config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + git -C repo-parallel config hook.hook-2.parallel true && + + git -C repo-parallel config hook.jobs 2 && + + git -C repo-parallel push origin HEAD >out 2>err && + echo parallel >expect && + test_cmp expect repo-parallel/hook.order +' + +test_expect_success 'hook.jobs=2 is ignored for force-serial hooks (pre-commit)' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event pre-commit && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event pre-commit && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + test_config hook.jobs 2 && + git commit --allow-empty -m "test: verify force-serial on pre-commit" && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs overrides hook.jobs for that event' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + # Global hook.jobs=1 (serial), but per-event override allows parallel. + test_config hook.jobs 1 && + test_config hook.test-hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs=1 forces serial even when hook.jobs>1' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + # Global hook.jobs=4 allows parallel, but per-event override forces serial. + test_config hook.jobs 4 && + test_config hook.test-hook.jobs 1 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs still requires hook..parallel=true' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + # hook-1 intentionally has no parallel=true + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + # hook-2 also has no parallel=true + + # Per-event jobs=2 but no hook has parallel=true: must still run serially. + test_config hook.test-hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs warns when name has .command' ' + test_config hook.my-hook.command "true" && + test_config hook.my-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.my-hook.jobs.*friendly-name" err +' + +test_expect_success 'hook..jobs warns when name has .event' ' + test_config hook.my-hook.event test-hook && + test_config hook.my-hook.command "true" && + test_config hook.my-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.my-hook.jobs.*friendly-name" err +' + +test_expect_success 'hook..jobs warns when name has .parallel' ' + test_config hook.my-hook.event test-hook && + test_config hook.my-hook.command "true" && + test_config hook.my-hook.parallel true && + test_config hook.my-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.my-hook.jobs.*friendly-name" err +' + +test_expect_success 'hook..jobs does not warn for a real event name' ' + test_config hook.test-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep ! "friendly-name" err +' + +test_expect_success 'hook.jobs=-1 resolves to online_cpus()' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-1.parallel true && + + test_config hook.jobs -1 && + + cpus=$(test-tool online-cpus) && + GIT_TRACE2_EVENT="$(pwd)/trace.txt" \ + git hook run --allow-unknown-hook-name test-hook >out 2>err && + grep "\"region_enter\".*\"hook\".*\"test-hook\".*\"max:$cpus\"" trace.txt +' + +test_expect_success 'hook..jobs=-1 resolves to online_cpus()' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-1.parallel true && + + test_config hook.test-hook.jobs -1 && + + cpus=$(test-tool online-cpus) && + GIT_TRACE2_EVENT="$(pwd)/trace.txt" \ + git hook run --allow-unknown-hook-name test-hook >out 2>err && + grep "\"region_enter\".*\"hook\".*\"test-hook\".*\"max:$cpus\"" trace.txt +' + +test_expect_success 'git hook run -j-1 resolves to online_cpus()' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-1.parallel true && + + cpus=$(test-tool online-cpus) && + GIT_TRACE2_EVENT="$(pwd)/trace.txt" \ + git hook run --allow-unknown-hook-name -j-1 test-hook >out 2>err && + grep "\"region_enter\".*\"hook\".*\"test-hook\".*\"max:$cpus\"" trace.txt +' + +test_expect_success 'hook.jobs rejects values less than -1' ' + test_config hook.jobs -2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.jobs must be a positive integer or -1" err +' + +test_expect_success 'hook..jobs rejects values less than -1' ' + test_config hook.test-hook.jobs -5 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.test-hook.jobs must be a positive integer or -1" err +' + +test_expect_success 'hook..enabled=false skips all hooks for event' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_must_be_empty out +' + +test_expect_success 'hook..enabled=true does not suppress hooks' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled true && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep "ran" err +' + +test_expect_success 'hook..enabled=false does not affect other events' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.other-event.enabled false && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep "ran" err +' + +test_expect_success 'hook..enabled=false still disables that hook' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo hook-1" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command "echo hook-2" && + test_config hook.hook-1.enabled false && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep ! "hook-1" err && + test_grep "hook-2" err +' + +test_expect_success 'git hook list shows event-disabled hooks as event-disabled' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook list --allow-unknown-hook-name test-hook >actual && + test_grep "^event-disabled hook-1$" actual && + test_grep "^event-disabled hook-2$" actual +' + +test_expect_success 'git hook list shows scope with event-disabled' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook list --allow-unknown-hook-name --show-scope test-hook >actual && + test_grep "^local event-disabled hook-1$" actual +' + +test_expect_success 'git hook list still shows hooks when event is disabled' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook list --allow-unknown-hook-name test-hook >actual && + test_grep "event-disabled" actual +' + +test_expect_success 'friendly-name matching known event name is rejected' ' + test_config hook.pre-commit.event pre-commit && + test_config hook.pre-commit.command "echo oops" && + test_must_fail git hook run pre-commit 2>err && + test_grep "collides with a known event name" err +' + +test_expect_success 'friendly-name matching known event name is rejected even for different event' ' + test_config hook.pre-commit.event post-commit && + test_config hook.pre-commit.command "echo oops" && + test_must_fail git hook run post-commit 2>err && + test_grep "collides with a known event name" err +' + +test_expect_success 'friendly-name matching unknown event warns' ' + test_config hook.test-hook.event test-hook && + test_config hook.test-hook.command "echo ran" && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep "same as its event" err +' + +test_expect_success 'hooks in parallel that do not read input' ' + # Add this to our $PATH to avoid having to write the whole trash + # directory into our config options, which would require quoting. + mkdir bin && + PATH=$PWD/bin:$PATH && + + write_script bin/hook-fast <<-\EOF && + # This hook does not read its input, so the parent process + # may see SIGPIPE if it is not ignored. It should happen + # relatively quickly. + exit 0 + EOF + + write_script bin/hook-slow <<-\EOF && + # This hook is slow, so we expect it to still be running + # when the other hook has exited (and the parent has a pipe error + # writing to it). + # + # So we want to be slow enough that we expect this to happen, but not + # so slow that the test takes forever. 1 second is probably enough + # in practice (and if it is occasionally not on a loaded system, we + # will err on the side of having the test pass). + sleep 1 + exit 0 + EOF + + git init --bare parallel.git && + git -C parallel.git config hook.fast.command "hook-fast" && + git -C parallel.git config hook.fast.event pre-receive && + git -C parallel.git config hook.fast.parallel true && + git -C parallel.git config hook.slow.command "hook-slow" && + git -C parallel.git config hook.slow.event pre-receive && + git -C parallel.git config hook.slow.parallel true && + git -C parallel.git config hook.jobs 2 && + + git push ./parallel.git "+refs/heads/*:refs/heads/*" +' + test_done diff --git a/transport.c b/transport.c index e53936d87b..9406ec4f2d 100644 --- a/transport.c +++ b/transport.c @@ -1391,8 +1391,10 @@ static int run_pre_push_hook(struct transport *transport, opt.feed_pipe_cb_data_free = pre_push_hook_data_free; /* - * pre-push hooks expect stdout & stderr to be separate, so don't merge - * them to keep backwards compatibility with existing hooks. + * pre-push hooks keep stdout and stderr separate by default for + * backwards compatibility. When the user opts into parallel execution + * via hook.jobs > 1 or -j, get_hook_jobs() will set stdout_to_stderr=1 + * automatically so run-command can de-interleave the outputs. */ opt.stdout_to_stderr = 0;