From 67fa6aac5af508b307c0d6968a28d50d14e7c026 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?SZEDER=20G=C3=A1bor?= Date: Sat, 7 Sep 2019 01:01:33 -0400 Subject: [PATCH 1/2] commit-graph: don't show progress percentages while expanding reachable commits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Commit 49bbc57a57 (commit-graph write: emit a percentage for all progress, 2019-01-19) was a bit overeager when it added progress percentages to the "Expanding reachable commits in commit graph" phase as well, because most of the time the number of commits that phase has to iterate over is not known in advance and grows significantly, and, consequently, we end up with nonsensical numbers: $ git commit-graph write --reachable Expanding reachable commits in commit graph: 138606% (824706/595), done. [...] $ git rev-parse v5.0 | git commit-graph write --stdin-commits Expanding reachable commits in commit graph: 81264400% (812644/1), done. [...] Even worse, because the percentage grows so quickly, the progress code outputs much more often than it should (because it ticks every second, or every 1%), slowing the whole process down. My time for "git commit-graph write --reachable" on linux.git went from 13.463s to 12.521s with this patch, ~7% savings. Therefore, don't show progress percentages in the "Expanding reachable commits in commit graph" phase. Note that the current code does sometimes do the right thing, if we picked up all commits initially (e.g., omitting "--reachable" in a fully-packed repository would get the correct count without any parent traversal). So it may be possible to come up with a way to tell when we could use a percentage here. But in the meantime, let's make sure we robustly avoid printing nonsense. Signed-off-by: SZEDER Gábor Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- commit-graph.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index fe954ab5f8..6e3b9afa74 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1049,7 +1049,7 @@ static void close_reachable(struct write_commit_graph_context *ctx) if (ctx->report_progress) ctx->progress = start_delayed_progress( _("Expanding reachable commits in commit graph"), - ctx->oids.nr); + 0); for (i = 0; i < ctx->oids.nr; i++) { display_progress(ctx->progress, i + 1); commit = lookup_commit(ctx->r, &ctx->oids.list[i]); From dd2e50a84ea431a6cec69f37251f29bf3cfcbb68 Mon Sep 17 00:00:00 2001 From: Jeff King Date: Sat, 7 Sep 2019 01:04:40 -0400 Subject: [PATCH 2/2] commit-graph: turn off save_commit_buffer The commit-graph tool may read a lot of commits, but it only cares about parsing their metadata (parents, trees, etc) and doesn't ever show the messages to the user. And so it should not need save_commit_buffer, which is meant for holding onto the object data of parsed commits so that we can show them later. In fact, it's quite harmful to do so. According to massif, the max heap of "git commit-graph write --reachable" in linux.git before/after this patch (removing the commit graph file in between) goes from ~1.1GB to ~270MB. Which isn't surprising, since the difference is about the sum of the uncompressed sizes of all commits in the repository, and this was equivalent to leaking them. This obviously helps if you're under memory pressure, but even without it, things go faster. My before/after times for that command (without massif) went from 12.521s to 11.874s, a speedup of ~5%. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- builtin/commit-graph.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 38027b83d9..3b5d58a5fa 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -249,6 +249,8 @@ int cmd_commit_graph(int argc, const char **argv, const char *prefix) builtin_commit_graph_usage, PARSE_OPT_STOP_AT_NON_OPTION); + save_commit_buffer = 0; + if (argc > 0) { if (!strcmp(argv[0], "read")) return graph_read(argc, argv);