Windows' equivalent to "bind mounts", NTFS junction points, can be
unlinked without affecting the mount target. This is clearly what users
expect to happen when they call `git clean -dfx` in a worktree that
contains NTFS junction points: the junction should be removed, and the
target directory of said junction should be left alone (unless it is
inside the worktree).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It seems to be not exactly rare on Windows to install NTFS junction
points (the equivalent of "bind mounts" on Linux/Unix) in worktrees,
e.g. to map some development tools into a subdirectory.
In such a scenario, it is pretty horrible if `git clean -dfx` traverses
into the mapped directory and starts to "clean up".
Let's just not do that. Let's make sure before we traverse into a
directory that it is not a mount point (or junction).
This addresses https://github.com/git-for-windows/git/issues/607
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
"git repo info -h" and "git repo structure -h" limit their help output
to the part that is specific to the subcommand.
* mk/repo-help-strings:
repo: show subcommand-specific help text
repo: factor repo usage strings into shared macros
Code paths that loop over another array to push each element into a
strvec have been rewritten to use strvec_pushv() instead.
* rs/use-strvec-pushv:
use strvec_pushv() to add another strvec
The HTTP transport learned to react to "429 Too Many Requests".
* vp/http-rate-limit-retries:
http: add support for HTTP 429 rate limit retries
strbuf_attach: fix call sites to pass correct alloc
strbuf: pass correct alloc to strbuf_attach() in strbuf_reencode()
The unsigned integer that is used as an bitset to specify the kind
of branches interpret_branch_name() function has been changed to
use a dedicated enum type.
* jw/object-name-bitset-to-enum:
object-name: turn INTERPRET_BRANCH_* constants into enum values
Doc updates.
* kh/doc-interpret-trailers-1:
interpret-trailers: use placeholder instead of *
doc: config: convert trailers section to synopsis style
doc: interpret-trailers: normalize and fill out options
doc: interpret-trailers: convert to synopsis style
merge-file --object-id used to trigger a BUG when run in a linked
worktree, which has been fixed.
* mr/merge-file-object-id-worktree-fix:
merge-file: fix BUG when --object-id is used in a worktree
Uses of prio_queue as a LIFO stack of commits have been written
with commit_stack.
* rs/prio-queue-to-commit-stack:
use commit_stack instead of prio_queue in LIFO mode
The logic to count objects has been cleaned up.
* ps/object-counting:
odb: introduce generic object counting
odb/source: introduce generic object counting
object-file: generalize counting objects
object-file: extract logic to approximate object count
packfile: extract logic to count number of objects
odb: stop including "odb/source.h"
Further work on incremental repacking using MIDX/bitmap
* tb/incremental-midx-part-3.2:
midx: enable reachability bitmaps during MIDX compaction
midx: implement MIDX compaction
t/helper/test-read-midx.c: plug memory leak when selecting layer
midx-write.c: factor fanout layering from `compute_sorted_entries()`
midx-write.c: enumerate `pack_int_id` values directly
midx-write.c: extract `fill_pack_from_midx()`
midx-write.c: introduce `midx_pack_perm()` helper
midx: do not require packs to be sorted in lexicographic order
midx-write.c: introduce `struct write_midx_opts`
midx-write.c: don't use `pack_perm` when assigning `bitmap_pos`
t/t5319-multi-pack-index.sh: fix copy-and-paste error in t5319.39
git-multi-pack-index(1): align SYNOPSIS with 'git multi-pack-index -h'
git-multi-pack-index(1): remove non-existent incompatibility
builtin/multi-pack-index.c: make '--progress' a common option
midx: introduce `midx_get_checksum_hex()`
midx: rename `get_midx_checksum()` to `midx_get_checksum_hash()`
midx: mark `get_midx_checksum()` arguments as const
Use subcommand-specific usage arrays for "git repo info" and
"git repo structure" so that each command shows only its own
synopsis in help output.
Add tests to cover the subcommand help behavior.
Signed-off-by: Mahi Kassa <mahlet.takassa@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Factor the "git repo info" and "git repo structure" usage
strings into shared macros so they can be reused in multiple
usage arrays.
This is a preparatory refactoring for subsequent changes to
subcommand-specific help output.
Signed-off-by: Mahi Kassa <mahlet.takassa@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a coccinelle rule to break the build when "struct strbuf" gets
passed by value.
* dd/cocci-do-not-pass-strbuf-by-value:
stash: do not pass strbuf by value
coccinelle: detect struct strbuf passed by value
Reduce system overhead "git upload-pack" spends on relaying "git
pack-objects" output to the "git fetch" running on the other end of
the connection.
* ps/upload-pack-buffer-more-writes:
builtin/pack-objects: reduce lock contention when writing packfile data
csum-file: drop `hashfd_throughput()`
csum-file: introduce `hashfd_ext()`
sideband: use writev(3p) to send pktlines
wrapper: introduce writev(3p) wrappers
compat/posix: introduce writev(3p) wrapper
upload-pack: reduce lock contention when writing packfile data
upload-pack: prefer flushing data over sending keepalive
upload-pack: adapt keepalives based on buffering
upload-pack: fix debug statement when flushing packfile data
"git history" learned the "split" subcommand.
* ps/history-split:
builtin/history: implement "split" subcommand
builtin/history: split out extended function to create commits
cache-tree: allow writing in-memory index as tree
add-patch: allow disabling editing of hunks
add-patch: add support for in-memory index patching
add-patch: remove dependency on "add-interactive" subsystem
add-patch: split out `struct interactive_options`
add-patch: split out header from "add-interactive.h"
"git fast-import" learned to optionally replace signature on
commits whose signatures get invalidated due to replaying by
signing afresh.
* jt/fast-import-sign-again:
fast-import: add mode to sign commits with invalid signatures
gpg-interface: allow sign_buffer() to use default signing key
commit: remove unused forward declaration
Add and apply a semantic patch that simplifies the code by letting
strvec_pushv() append the items of a second strvec instead of pushing
them one by one.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code in "git help" that shows configuration items in sorted
order was awkwardly organized and prone to bugs.
* ac/help-sort-correctly:
help: cleanup the contruction of keys_uniq
Instead of hardcoded 'origin', use the configured default remote
when fetching from submodules.
* ng/submodule-default-remote:
submodule: fetch missing objects from default remote
The run_command() API lost its implicit dependencyon the singleton
`the_repository` instance.
* bk/run-command-wo-the-repository:
run-command: wean auto_maintenance() functions off the_repository
run-command: wean start_command() off the_repository
The way combined list-object filter options are parsed has been
revamped.
* dd/list-objects-filter-options-wo-strbuf-split:
list-objects-filter-options: avoid strbuf_split_str()
worktree: do not pass strbuf by value
Replace the INTERPRET_BRANCH_* preprocessor constants with enum
values and use that type where these flags are stored or passed
around.
These flags describe which kinds of branches may be considered during
branch-name interpretation, so represent them as an enum describing
branch kinds while keeping the existing bitmask semantics and
INTERPRET_BRANCH_* element names.
Signed-off-by: Jialong Wang <jerrywang183@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `--object-id` option was added in commit e1068f0ad4
(merge-file: add an option to process object IDs, 2023-11-01)
together with a call to setup_git_directory() to avoid crashing
when run outside a repository.
However, the call to setup_git_directory() is redundant when run inside
a repository, as merge-file runs with RUN_SETUP_GENTLY, so the
repository has already been set up. The redundant call is harmless
when linked worktrees are not used, but in a linked worktree,
the repo_set_gitdir() function ends up being called twice.
Calling repo_set_gitdir() used to be silently accepted, but commit
2816b748e5 (odb: handle changing a repository's commondir, 2025-11-19)
changed this to a BUG in repository.c with the error message:
"cannot reinitialize an already-initialized object directory".
Guard the redundant call to setup_git_directory() behind a repo pointer
check, to ensure that we continue to give the correct "not a git repo"
error whilst avoiding the BUG when running in a linked worktree.
Signed-off-by: Mathias Rav <m@git.strova.dk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A prio_queue with a NULL compare function acts as a stack -- the last
element in is the first one out (LIFO). Use an actual commit_stack
instead where possible, as it documents the behavior better, provides
type safety and saves some memory because prio_queue stores an
additional tie-breaking counter per element.
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
strbuf_attach(sb, buf, len, alloc) requires alloc > len (the buffer
must have at least len+1 bytes to hold the NUL). Several call sites
passed alloc == len, relying on strbuf_grow(sb, 0) inside strbuf_attach
to reallocate. Fix these in mailinfo, am, refs/files-backend,
fast-import, and trailer by passing len+1 when the buffer is a
NUL-terminated string (or from strbuf_detach).
Signed-off-by: Vaidas Pilkauskas <vaidas.pilkauskas@shopify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use `<key-alias>` instead of `*` in order to be consistent with
the documentation.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git repo structure" command learns to report maximum values on
various aspects of objects it inspects.
* jt/repo-structure-extrema:
builtin/repo: find tree with most entries
builtin/repo: find commit with most parents
builtin/repo: add OID annotations to table output
builtin/repo: collect largest inflated objects
builtin/repo: add helper for printing keyvalue output
builtin/repo: update stats for each object
save_untracked_files() takes its 'files' parameter as struct strbuf
by value. Passing a strbuf by value copies the struct but shares
the underlying buffer between caller and callee, risking a dangling
pointer and double-free if the callee reallocates.
The function needs both the buffer and its length for
pipe_command(), so a plain const char * is not sufficient here.
Switch the parameter to struct strbuf * and update the caller to
pass a pointer.
Signed-off-by: Deveshi Dwivedi <deveshigurgaon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running `git pack-objects --stdout` we feed the data through
`hashfd_ext()` with a progress meter and a smaller-than-usual buffer
length of 8kB so that we can track throughput more granularly. But as
packfiles tend to be on the larger side, this small buffer size may
cause a ton of write(3p) syscalls.
Originally, the buffer we used in `hashfd()` was 8kB for all use cases.
This was changed though in 2ca245f8be (csum-file.h: increase hashfile
buffer size, 2021-05-18) because we noticed that the number of writes
can have an impact on performance. So the buffer size was increased to
128kB, which improved performance a bit for some use cases.
But the commit didn't touch the buffer size for `hashd_throughput()`.
The reasoning here was that callers expect the progress indicator to
update frequently, and a larger buffer size would of course reduce the
update frequency especially on slow networks.
While that is of course true, there was (and still is, even though it's
now a call to `hashfd_ext()`) only a single caller of this function in
git-pack-objects(1). This command is responsible for writing packfiles,
and those packfiles are often on the bigger side. So arguably:
- The user won't care about increments of 8kB when packfiles tend to
be megabytes or even gigabytes in size.
- Reducing the number of syscalls would be even more valuable here
than it would be for multi-pack indices, which was the benchmark
done in the mentioned commit, as MIDXs are typically significantly
smaller than packfiles.
- Nowadays, many internet connections should be able to transfer data
at a rate significantly higher than 8kB per second.
Update the buffer to instead have a size of `LARGE_PACKET_DATA_MAX - 1`,
which translates to ~64kB. This limit was chosen because `git
pack-objects --stdout` is most often used when sending packfiles via
git-upload-pack(1), where packfile data is chunked into pktlines when
using the sideband. Furthermore, most internet connections should have a
bandwidth signifcantly higher than 64kB/s, so we'd still be able to
observe progress updates at a rate of at least once per second.
This change significantly reduces the number of write(3p) syscalls from
355,000 to 44,000 when packing the Linux repository. While this results
in a small performance improvement on an otherwise-unused system, this
improvement is mostly negligible. More importantly though, it will
reduce lock contention in the kernel on an extremely busy system where
we have many processes writing data at once.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `hashfd_throughput()` function is used by a single callsite in
git-pack-objects(1). In contrast to `hashfd()`, this function uses a
progress meter to measure throughput and a smaller buffer length so that
the progress meter can provide more granular metrics.
We're going to change that caller in the next commit to be a bit more
specific to packing objects. As such, `hashfd_throughput()` will be a
somewhat unfitting mechanism for any potential new callers.
Drop the function and replace it with a call to `hashfd_ext()`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With git-fast-import(1), handling of signed commits is controlled via
the `--signed-commits=<mode>` option. When an invalid signature is
encountered, a user may want the option to sign the commit again as
opposed to just stripping the signature. To facilitate this, introduce a
"sign-if-invalid" mode for the `--signed-commits` option. Optionally, a
key ID may be explicitly provided in the form
`sign-if-invalid[=<keyid>]` to specify which signing key should be used
when signing invalid commit signatures.
Note that to properly support interoperability mode when signing commit
signatures, the commit buffer must be created in both the repository and
compatability object formats to generate the appropriate signatures
accordingly. As currently implemented, the commit buffer for the
compatability object format is not reconstructed and thus signing
commits in interoperability mode is not yet supported. Support may be
added in the future.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `sign_commit_to_strbuf()` helper in "commit.c" provides fallback
logic to get the default configured signing key when a key is not
provided and handles generating the commit signature accordingly. This
signing operation is not really specific to commits as any arbitrary
buffer can be signed. Also, in a subsequent commit, this same logic is
reused by git-fast-import(1) when signing commits with invalid
signatures.
Remove the `sign_commit_to_strbuf()` helper from "commit.c" and extend
`sign_buffer()` in "gpg-interface.c" to support using the default key as
a fallback when the `SIGN_BUFFER_USE_DEFAULT_KEY` flag is provided. Call
sites are updated accordingly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The object source API is getting restructured to allow plugging new
backends.
* ps/odb-sources:
odb/source: make `begin_transaction()` function pluggable
odb/source: make `write_alternate()` function pluggable
odb/source: make `read_alternates()` function pluggable
odb/source: make `write_object_stream()` function pluggable
odb/source: make `write_object()` function pluggable
odb/source: make `freshen_object()` function pluggable
odb/source: make `for_each_object()` function pluggable
odb/source: make `read_object_stream()` function pluggable
odb/source: make `read_object_info()` function pluggable
odb/source: make `close()` function pluggable
odb/source: make `reprepare()` function pluggable
odb/source: make `free()` function pluggable
odb/source: introduce source type for robustness
odb: move reparenting logic into respective subsystems
odb: embed base source in the "files" backend
odb: introduce "files" source
odb: split `struct odb_source` into separate header
"git for-each-repo" started from a secondary worktree did not work
as expected, which has been corrected.
* ds/for-each-repo-w-worktree:
for-each-repo: simplify passing of parameters
for-each-repo: work correctly in a worktree
run-command: extract sanitize_repo_env helper
for-each-repo: test outside of repo context
A bit of OIDmap API enhancement and cleanup.
* sk/oidmap-clear-with-custom-free-func:
builtin/rev-list: migrate missing_objects cleanup to oidmap_clear_with_free()
oidmap: make entry cleanup explicit in oidmap_clear
The 'cmd_mktree()' function already receives a 'struct repository *repo'
pointer, but it was previously marked as UNUSED.
Pass the 'repo' pointer down to 'mktree_line()' and 'write_tree()'.
Consequently, remove the 'USE_THE_REPOSITORY_VARIABLE' macro, replace
usages of 'the_repository', and swap 'parse_oid_hex()' with its context-aware
version 'parse_oid_hex_algop()'.
This refactoring is safe because 'cmd_mktree()' is registered with the
'RUN_SETUP' flag in 'git.c', which guarantees that the command is
executed within a initialized repository, ensuring that the passed 'repo'
pointer is never 'NULL'.
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar to the preceding commit, introduce counting of objects on the
object database level, replacing the logic that we have in
`repo_approximate_object_count()`.
Note that the function knows to cache the object count. It's unclear
whether this cache is really required as we shouldn't have that many
cases where we count objects repeatedly. But to be on the safe side the
caching mechanism is retained, with the only excepting being that we
also have to use the passed flags as caching key.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Generalize the function introduced in the preceding commit to not only
be able to approximate the number of loose objects, but to also provide
an accurate count. The behaviour can be toggled via a new flag.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In "builtin/gc.c" we have some logic that checks whether we need to
repack objects. This is done by counting the number of objects that we
have and checking whether it exceeds a certain threshold. We don't
really need an accurate object count though, which is why we only
open a single object directory shard and then extrapolate from there.
Extract this logic into a new function that is owned by the loose object
database source. This is done to prepare for a subsequent change, where
we'll introduce object counting on the object database source level.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "odb.h" header currently includes the "odb/source.h" file. This is
somewhat roundabout though: most callers shouldn't have to care about
the `struct odb_source`, but should rather use the ODB-level functions.
Furthermore, it means that a couple of definitions have to live on the
source level even though they should be part of the generic interface.
Reverse the relation between "odb/source.h" and "odb.h" and move the
enums and typedefs that relate to the generic interfaces back into
"odb.h". Add the necessary includes to all files that rely on the
transitive include.
Suggested-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The prepare_auto_maintenance() relies on the_repository to read
configurations. Since run_auto_maintenance() calls
prepare_auto_maintenance(), it also implicitly depends the_repository.
Add 'struct repository *' as a parameter to both functions and update
all callers to pass the_repository.
With no global repository dependencies left in this file, remove the
USE_THE_REPOSITORY_VARIABLE macro.
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Burak Kaan Karaçay <bkkaracay@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The start_command() relies on the_repository due to the
close_object_store flag in 'struct child_process'. When this flag is
set, start_command() closes the object store associated with
the_repository before spawning a child process.
To eliminate this dependency, replace the 'close_object_store' with the
new 'struct object_database *odb_to_close' field. This allows callers to
specify the object store that needs to be closed.
Suggested-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Burak Kaan Karaçay <bkkaracay@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
construction of keys_uniq depends on sort operation
executed on keys before processing, which does not
gurantee that keys_uniq will be sorted.
refactor the code to shift the sort operation after
the processing to remove dependency on key's sort operation
and strictly maintain the sorted order of keys_uniq.
move strbuf init and release out of loop to reuse same buffer.
dedent sort -u and sed in tests and replace grep with sed, to
avoid piping grep's output to sed.
Suggested-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Signed-off-by: Amisha Chhajed <amishhhaaaa@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The submodule_summary_callback() function currently uses a raw malloc()
which could lead to a NULL pointer dereference.
Standardize this by replacing malloc() with xmalloc() for error handling.
To improve maintainability, use sizeof(*temp) instead of the struct name,
and drop the typecast of void pointer assignment.
Signed-off-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
write_worktree_linking_files() takes two struct strbuf parameters by
value, even though it only reads path strings from them.
Passing a strbuf by value is misleading and dangerous. The structure
carries a pointer to its underlying character array; caller and callee
end up sharing that storage. If the callee ever causes the strbuf to
be reallocated, the caller's copy becomes a dangling pointer, which
results in a double-free when the caller does strbuf_release().
The function only needs the string values, not the strbuf machinery.
Switch it to take const char * and update all callers to pass .buf.
Signed-off-by: Deveshi Dwivedi <deveshigurgaon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
