Commit Graph

3263 Commits

Author SHA1 Message Date
Johannes Schindelin
3c576157bc Lazy load libcurl, allowing for an SSL/TLS backend-specific libcurl (#4410)
As per
https://github.com/git-for-windows/git/issues/4350#issuecomment-1485041503,
the major block for upgrading Git for Windows' OpenSSL from v1.1 to v3
is the tricky part where such an upgrade would break `git fetch`/`git
clone` and `git push` because the libcurl depends on the OpenSSL DLL,
and the major version bump will _change_ the file name of said DLL.

To overcome that, the plan is to build libcurl flavors for each
supported SSL/TLS backend, aligning with the way MSYS2 builds libcurl,
then switch Git for Windows' SDK to the Secure Channel-flavored libcurl,
and teach Git to look for the specific flavor of libcurl corresponding
to the `http.sslBackend` setting (if that was configured).

Here is the PR to teach Git that trick.
2026-06-11 17:32:29 +02:00
Johannes Schindelin
83d7f0b856 http: support lazy-loading libcurl also on Windows
This implements the Windows-specific support code, because everything is
slightly different on Windows, even loading shared libraries.

Note: I specifically do _not_ use the code from
`compat/win32/lazyload.h` here because that code is optimized for
loading individual functions from various system DLLs, while we
specifically want to load _many_ functions from _one_ DLL here, and
distinctly not a system DLL (we expect libcurl to be located outside
`C:\Windows\system32`, something `INIT_PROC_ADDR` refuses to work with).
Also, the `curl_easy_getinfo()`/`curl_easy_setopt()` functions are
declared as vararg functions, which `lazyload.h` cannot handle. Finally,
we are about to optionally override the exact file name that is to be
loaded, which is a goal contrary to `lazyload.h`'s design.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2026-06-11 17:32:25 +02:00
Johannes Schindelin
0270dbddfb http: optionally load libcurl lazily
This compile-time option allows to ask Git to load libcurl dynamically
at runtime.

Together with a follow-up patch that optionally overrides the file name
depending on the `http.sslBackend` setting, this kicks open the door for
installing multiple libcurl flavors side by side, and load the one
corresponding to the (runtime-)configured SSL/TLS backend.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2026-06-11 17:32:25 +02:00
Jeff Hostetler
515a11de0b Makefile: clean up .ilk files when MSVC=1
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2026-06-11 17:32:25 +02:00
Junio C Hamano
06f63df846 Merge branch 'ps/odb-source-loose'
The loose object source has been refactored into a proper `struct
odb_source`.

* ps/odb-source-loose:
  odb/source-loose: drop pointer to the "files" source
  odb/source-loose: stub out remaining callbacks
  odb/source-loose: wire up `write_object_stream()` callback
  object-file: refactor writing objects to use loose source
  odb/source-loose: wire up `write_object()` callback
  loose: refactor object map to operate on `struct odb_source_loose`
  odb/source-loose: wire up `freshen_object()` callback
  odb/source-loose: drop `odb_source_loose_has_object()`
  odb/source-loose: wire up `count_objects()` callback
  odb/source-loose: wire up `find_abbrev_len()` callback
  odb/source-loose: wire up `for_each_object()` callback
  odb/source-loose: wire up `read_object_stream()` callback
  odb/source-loose: wire up `read_object_info()` callback
  odb/source-loose: wire up `close()` callback
  odb/source-loose: wire up `reprepare()` callback
  odb/source-loose: start converting to a proper `struct odb_source`
  odb/source-loose: store pointer to "files" instead of generic source
  odb/source-loose: move loose source into "odb/" subsystem
2026-06-11 04:31:18 -07:00
Patrick Steinhardt
514f039c90 odb/source-loose: move loose source into "odb/" subsystem
In subsequent patches we'll be turning `struct odb_source_loose` into a
proper `struct odb_source`. As a first step towards this goal, move its
struct out of "object-file.c" and into "odb/source-loose.c".

This detaches the implementation of the loose object source from the
generic object file code, following the same convention already used by
the "files" and "in-memory" sources.

No functional changes are intended.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01 18:47:17 +09:00
Junio C Hamano
4d11b9c218 Merge branch 'pt/fsmonitor-linux'
The fsmonitor daemon has been implemented for Linux.

* pt/fsmonitor-linux:
  fsmonitor: convert shown khash to strset in do_handle_client
  fsmonitor: add tests for Linux
  fsmonitor: add timeout to daemon stop command
  fsmonitor: close inherited file descriptors and detach in daemon
  run-command: add close_fd_above_stderr option
  fsmonitor: implement filesystem change listener for Linux
  fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c
  fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c
  fsmonitor: use pthread_cond_timedwait for cookie wait
  compat/win32: add pthread_cond_timedwait
  fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon
  fsmonitor: fix khash memory leak in do_handle_client
  t9210, t9211: disable GIT_TEST_SPLIT_INDEX for scalar clone tests
2026-05-31 10:00:38 +09:00
Junio C Hamano
0839e56957 Merge branch 'ps/odb-in-memory'
Add a new odb "in-memory" source that is meant to only hold
tentative objects (like the virtual blob object that represents the
working tree file used by "git blame").

* ps/odb-in-memory:
  t/unit-tests: add tests for the in-memory object source
  odb: generic in-memory source
  odb/source-inmemory: stub out remaining functions
  odb/source-inmemory: implement `freshen_object()` callback
  odb/source-inmemory: implement `count_objects()` callback
  odb/source-inmemory: implement `find_abbrev_len()` callback
  odb/source-inmemory: implement `for_each_object()` callback
  odb/source-inmemory: convert to use oidtree
  oidtree: add ability to store data
  cbtree: allow using arbitrary wrapper structures for nodes
  odb/source-inmemory: implement `write_object_stream()` callback
  odb/source-inmemory: implement `write_object()` callback
  odb/source-inmemory: implement `read_object_stream()` callback
  odb/source-inmemory: implement `read_object_info()` callback
  odb: fix unnecessary call to `find_cached_object()`
  odb/source-inmemory: implement `free()` callback
  odb: introduce "in-memory" source
2026-05-27 14:15:46 +09:00
Junio C Hamano
2f952b81ed Merge branch 'jt/odb-transaction-write'
ODB transaction interface is being reworked to explicitly handle
object writes.

* jt/odb-transaction-write:
  odb/transaction: make `write_object_stream()` pluggable
  object-file: generalize packfile writes to use odb_write_stream
  object-file: avoid fd seekback by checking object size upfront
  object-file: remove flags from transaction packfile writes
  odb: update `struct odb_write_stream` read() callback
  odb/transaction: use pluggable `begin_transaction()`
  odb: split `struct odb_transaction` into separate header
2026-05-27 14:15:45 +09:00
Junio C Hamano
9ebc19b760 Merge branch 'ps/odb-in-memory' into ps/odb-source-loose
* ps/odb-in-memory: (24 commits)
  t/unit-tests: add tests for the in-memory object source
  odb: generic in-memory source
  odb/source-inmemory: stub out remaining functions
  odb/source-inmemory: implement `freshen_object()` callback
  odb/source-inmemory: implement `count_objects()` callback
  odb/source-inmemory: implement `find_abbrev_len()` callback
  odb/source-inmemory: implement `for_each_object()` callback
  odb/source-inmemory: convert to use oidtree
  oidtree: add ability to store data
  cbtree: allow using arbitrary wrapper structures for nodes
  odb/source-inmemory: implement `write_object_stream()` callback
  odb/source-inmemory: implement `write_object()` callback
  odb/source-inmemory: implement `read_object_stream()` callback
  odb/source-inmemory: implement `read_object_info()` callback
  odb: fix unnecessary call to `find_cached_object()`
  odb/source-inmemory: implement `free()` callback
  odb: introduce "in-memory" source
  odb/transaction: make `write_object_stream()` pluggable
  object-file: generalize packfile writes to use odb_write_stream
  object-file: avoid fd seekback by checking object size upfront
  ...
2026-05-21 22:34:55 +09:00
Junio C Hamano
686213114e Merge branch 'mm/git-url-parse'
The internal URL parsing logic has been made accessible via a new
subcommand "git url-parse".

* mm/git-url-parse:
  t9904: add tests for the new url-parse builtin
  doc: describe the url-parse builtin
  builtin: create url-parse command
  urlmatch: define url_parse function
  url: return URL_SCHEME_UNKNOWN instead of dying
  url: move scheme detection to URL header/source
  url: move url_is_local_not_ssh to url.h
  connect: rename enum protocol to url_scheme
2026-05-21 12:06:48 +09:00
Junio C Hamano
91ddfe3d5c Merge branch 'js/mingw-no-nedmalloc'
Stop using unmaintained custom allocator in Windows build which was
the last user of the code.

* js/mingw-no-nedmalloc:
  mingw: remove the vendored compat/nedmalloc/ subtree
  mingw: drop the build-system plumbing for nedmalloc
  mingw: stop using nedmalloc
2026-05-20 10:30:56 +09:00
Junio C Hamano
a6876b2068 Merge branch 'js/objects-larger-than-4gb-on-windows'
Update code paths that assumed "unsigned long" was long enough for
"size_t".

* js/objects-larger-than-4gb-on-windows:
  ci: run expensive tests on push builds to integration branches
  t5608: mark >4GB tests as EXPENSIVE
  test-tool synthesize: add precomputed SHA-256 pack for 4 GiB + 1
  test-tool synthesize: precompute pack for 4 GiB + 1
  test-tool synthesize: use the unsafe hash for speed
  t5608: add regression test for >4GB object clone
  test-tool: add a helper to synthesize large packfiles
  delta, packfile: use size_t for delta header sizes
  odb, packfile: use size_t for streaming object sizes
  git-zlib: handle data streams larger than 4GB
  index-pack, unpack-objects: use size_t for object size
2026-05-20 10:30:56 +09:00
Junio C Hamano
056472b82d Merge branch 'kh/name-rev-custom-format'
A new builtin "git format-rev" is introduced for pretty formatting
one revision expression per line or commit object names found in
running text.

* kh/name-rev-custom-format:
  format-rev: introduce builtin for on-demand pretty formatting
  name-rev: make dedicated --annotate-stdin --name-only test
  name-rev: factor code for sharing with a new command
  name-rev: run clang-format before factoring code
  name-rev: wrap both blocks in braces
2026-05-19 09:57:44 +09:00
Patrick Steinhardt
d2902a4549 t/unit-tests: add tests for the in-memory object source
While the in-memory object source is a full-fledged source, our code
base only exercises parts of its functionality because we only use it in
git-blame(1). Implement unit tests to verify that the yet-unused
functionality of the backend works as expected.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-15 04:50:45 +09:00
Patrick Steinhardt
822d403651 odb: introduce "in-memory" source
Next to our typical object database sources, each object database also
has an implicit source of "cached" objects. These cached objects only
exist in memory and some use cases:

  - They contain evergreen objects that we expect to always exist, like
    for example the empty tree.

  - They can be used to store temporary objects that we don't want to
    persist to disk, which is used by git-blame(1) to create a fake
    worktree commit.

Overall, their use is somewhat restricted though. For example, we don't
provide the ability to use it as a temporary object database source that
allows the user to write objects, but discard them after Git exists. So
while these cached objects behave almost like a source, they aren't used
as one.

This is about to change over the following commits, where we will turn
cached objects into a new "in-memory" source. This will allow us to use
it exactly the same as any other source by providing the same common
interface as the "files" source.

For now, the in-memory source only hosts the cached objects and doesn't
provide any logic yet. This will change with subsequent commits, where
we move respective functionality into the source.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-15 04:50:44 +09:00
Junio C Hamano
2f124686e8 Merge branch 'jt/odb-transaction-write' into ps/odb-in-memory
* jt/odb-transaction-write:
  odb/transaction: make `write_object_stream()` pluggable
  object-file: generalize packfile writes to use odb_write_stream
  object-file: avoid fd seekback by checking object size upfront
  object-file: remove flags from transaction packfile writes
  odb: update `struct odb_write_stream` read() callback
  odb/transaction: use pluggable `begin_transaction()`
  odb: split `struct odb_transaction` into separate header
2026-05-15 04:50:31 +09:00
Justin Tobler
5f6744d3eb odb: split struct odb_transaction into separate header
The current ODB transaction interface is colocated with other ODB
interfaces in "odb.{c,h}". Subsequent commits will expand `struct
odb_transaction` to support write operations on the transaction
directly. To keep things organized and prevent "odb.{c,h}" from becoming
more unwieldy, split out `struct odb_transaction` into a separate
header.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-15 04:44:39 +09:00
Kristoffer Haugsbakk
19e3106c45 format-rev: introduce builtin for on-demand pretty formatting
Introduce a new builtin for pretty formatting one revision expression
per line or commit object names found in running text.

Sometimes you want to format commits. Most of the time you’re
walking the graph, e.g. getting a range of commits like
`master..topic`. That’s a job for git-log(1).

But there are times when you want to format commits that you encounter
on demand:

• Full hashes in running text that you might want to pretty-print
• git-last-modified(1) outputs full hashes that you can do the same
  with
• git-cherry(1) has `-v` for commit subject, but maybe you want
  something else?

But now you can’t use git-log(1), git-show(1), or git-rev-list(1):

• You can’t feed commits piecemeal to these commands, one input
  for one output; they block until standard in is closed
• You can’t feed a list of possibly duplicate commits, like the output
  of git-last-modified(1); they effectively deduplicate the output

Beyond these two points there’s also the input massage problem: you
cannot feed mixed input (revisions mixed with arbitrary text).

One might hope that git-cat-file(1) can save us. But it doesn’t
support pretty formats.

But there is one command that already both handles revisions as
arguments, revisions on standard input, and even revisions mixed in
with arbitrary text. Namely git-name-rev(1): the command for outputting
symbolic names for commits.

We made some room in `builtin/name-rev.c` two commits ago. Let’s
now add this new git-format-rev(1) command. Taking inspiration from
git-name-rev(1), there are two modes:

• revs: like git-name-rev(1) in argv mode, but one revision per line
  on standard in
• text: like git-name-rev(1) with `--annotate-stdin`

***

We need to add this command to the exception list in
`t/t1517-outside-repo.sh` because it uses “EXPERIMENTAL!”
in the usage line.

Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-12 09:09:51 +09:00
Junio C Hamano
bd5c2827b2 Merge branch 'bc/rust-by-default'
Rust support is enabled by default (but still allows opting out) in
some future version of Git.

* bc/rust-by-default:
  Enable Rust by default
  Linux: link against libdl
  ci: install cargo on Alpine
  docs: update version with default Rust support
2026-05-11 10:05:54 +09:00
Junio C Hamano
718db095c2 Merge branch 'ar/parallel-hooks'
Hook scripts defined via the configuration system can now be
configured to run in parallel.

* ar/parallel-hooks:
  t1800: test SIGPIPE with parallel hooks
  hook: allow hook.jobs=-1 to use all available CPU cores
  hook: add hook.<event>.enabled switch
  hook: move is_known_hook() to hook.c for wider use
  hook: warn when hook.<friendly-name>.jobs is set
  hook: add per-event jobs config
  hook: add -j/--jobs option to git hook run
  hook: mark non-parallelizable hooks
  hook: allow pre-push parallel execution
  hook: allow parallel hook execution
  hook: parse the hook.jobs config
  config: add a repo_config_get_uint() helper
  repository: fix repo_init() memleak due to missing _clear()
2026-05-11 10:05:53 +09:00
Johannes Schindelin
438886aecb test-tool: add a helper to synthesize large packfiles
To test Git's behavior with very large pack files, we need a way to
generate such files quickly.

A naive approach using only readily-available Git commands would take
over 10 hours for a 4GB pack file, which is prohibitive.

Side-stepping Git's machinery and actual zlib compression by writing
uncompressed content with the appropriate zlib header makes things
much faster. The fastest method using this approach generates many
small, unreachable blob objects and takes about 1.5 minutes for 4GB.
However, this cannot be used because we need to test git clone, which
requires a reachable commit history.

Generating many reachable commits with small, uncompressed blobs takes
about 4 minutes for 4GB. But this approach 1) does not reproduce the
issues we want to fix (which require individual objects larger than
4GB) and 2) is comparatively slow because of the many SHA-1
calculations.

The approach taken here generates a single large blob (filled with NUL
bytes), along with the trees and commits needed to make it reachable.
This takes about 2.5 minutes for 4.5GB, which is the fastest option
that produces a valid, clonable repository with an object large enough
to trigger the bugs we want to test.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-09 11:25:32 +09:00
Johannes Schindelin
cefcada1d3 mingw: drop the build-system plumbing for nedmalloc
With the previous commit removing every opt-in, the build-system
plumbing for nedmalloc has nothing left to switch on. Remove it so
that the upcoming deletion of the compat/nedmalloc/ tree is a pure
file removal.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-09 11:19:23 +09:00
Matheus Afonso Martins Moreira
533eb14798 builtin: create url-parse command
Git commands can accept a rather wide variety of URLs syntaxes.
The range of accepted inputs might expand even more in the future.
This makes the parsing of URL components difficult since standard URL
parsers cannot be used. Extracting the components of a git URL would
require implementing all the schemes that git itself supports, not to
mention tracking its development continuously in case new URL schemes
are added.

The url-parse builtin command is designed to solve this problem
by exposing git's native URL parsing facilities as a plumbing command.
Other programs can then call upon git itself to parse the git URLs
and extract their components. This should be quite useful for scripts.

Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-06 09:48:28 +09:00
Paul Tarjan
ff384ebfad fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c
The fsmonitor IPC path logic in fsm-ipc-darwin.c is not
Darwin-specific and will be reused by the upcoming Linux
implementation.  Rename it to fsm-ipc-unix.c to reflect that it
is shared by all Unix platforms.

Introduce FSMONITOR_OS_SETTINGS (set to "unix" for non-Windows, "win32"
for Windows) as a separate variable from FSMONITOR_DAEMON_BACKEND so
that the build files can distinguish between platform-specific files
(listen, health, path-utils) and shared Unix files (ipc, settings).

Move fsm-ipc to the FSMONITOR_OS_SETTINGS section in the Makefile, and
switch fsm-path-utils to use FSMONITOR_DAEMON_BACKEND since path-utils
is platform-specific (there will be separate darwin and linux versions).

Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com>
Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com>
Signed-off-by: Paul Tarjan <github@paulisageek.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-15 08:44:33 -07:00
Adrian Ratiu
2eb541e8f2 hook: move is_known_hook() to hook.c for wider use
Move is_known_hook() from builtin/hook.c (static) into hook.c and
export it via hook.h so it can be reused.

Make it return bool and the iterator `h` for clarity (iterate hooks).

Both meson.build and the Makefile are updated to reflect that the
header is now used by libgit, not the builtin sources.

The next commit will use this to reject hook friendly-names that
collide with known event names.

Co-authored-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-10 07:58:54 -07:00
brian m. carlson
32d5b90590 Enable Rust by default
Our breaking changes document says that we'll enable Rust by default in
Git 2.54.  Adjust the Makefile to switch the option from WITH_RUST to
NO_RUST to enable it by default and update the help text accordingly.
Similarly, for Meson, enable the option by default and do not
automatically disable it if Cargo is missing, since the goal is to help
users find where they are likely to have problems in the future.

Update our CI tests to swap out the single Linux job with Rust to a
single job without, both for Makefile and Meson.  Similarly, update the
Windows Makefile job to not use Rust, while the Meson job (which does
not build with ci/lib.sh) will default to having it enabled.

Move the check for Cargo in the Meson build because it is no longer
needed in the main script.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-09 17:25:36 -07:00
Junio C Hamano
7798034171 Revert "compat/posix: introduce writev(3p) wrapper"
This reverts commit 3b9b2c2a29a1d529ca9884fa0a6529f6e2496abe; let's
not use writev() for now.
2026-04-09 14:48:24 -07:00
Junio C Hamano
0cd4fb9f46 Merge branch 'ar/config-hook-cleanups'
Code clean-up around the recent "hooks defined in config" topic.

* ar/config-hook-cleanups:
  hook: reject unknown hook names in git-hook(1)
  hook: show disabled hooks in "git hook list"
  hook: show config scope in git hook list
  hook: introduce hook_config_cache_entry for per-hook data
  t1800: add test to verify hook execution ordering
  hook: make consistent use of friendly-name in docs
  hook: replace hook_list_clear() -> string_list_clear_func()
  hook: detect & emit two more bugs
  hook: rename cb_data_free/alloc -> hook_data_free/alloc
  hook: fix minor style issues
  builtin/receive-pack: properly init receive_hook strbuf
  hook: move unsorted_string_list_remove() to string-list.[ch]
2026-04-03 13:01:09 -07:00
Junio C Hamano
d1f07dd500 Merge branch 'ps/build-tweaks'
Tweak the build infrastructure by moving tools around.

* ps/build-tweaks:
  meson: precompile "git-compat-util.h"
  meson: compile compatibility sources separately
  git-compat-util.h: move warning infra to prepare for PCHs
  builds: move build scripts into "tools/"
  contrib: move "update-unicode.sh" script into "tools/"
  contrib: move "coverage-diff.sh" script into "tools/"
  contrib: move "coccinelle/" directory into "tools/"
  Introduce new "tools/" directory
2026-03-27 11:00:01 -07:00
Adrian Ratiu
5c58dbc887 hook: reject unknown hook names in git-hook(1)
Teach "git hook run" and "git hook list" to reject hook event names
that are not recognized by Git. This helps catch typos such as
"prereceive" when "pre-receive" was intended, since in 99% of the
cases users want known (already-existing) hook names.

The list of known hooks is derived from the generated hook-list.h
(built from Documentation/githooks.adoc). This is why the Makefile
is updated, so builtin/hook.c depends on hook-list.h. In meson the
header is already a dependency for all builtins, no change required.

The "--allow-unknown-hook-name" flag can be used to bypass this check.

Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-25 14:00:48 -07:00
Junio C Hamano
8023abc632 Merge branch 'ps/upload-pack-buffer-more-writes'
Reduce system overhead "git upload-pack" spends on relaying "git
pack-objects" output to the "git fetch" running on the other end of
the connection.

* ps/upload-pack-buffer-more-writes:
  builtin/pack-objects: reduce lock contention when writing packfile data
  csum-file: drop `hashfd_throughput()`
  csum-file: introduce `hashfd_ext()`
  sideband: use writev(3p) to send pktlines
  wrapper: introduce writev(3p) wrappers
  compat/posix: introduce writev(3p) wrapper
  upload-pack: reduce lock contention when writing packfile data
  upload-pack: prefer flushing data over sending keepalive
  upload-pack: adapt keepalives based on buffering
  upload-pack: fix debug statement when flushing packfile data
2026-03-24 12:31:34 -07:00
Patrick Steinhardt
a767f2fd6c builds: move build scripts into "tools/"
We have a bunch of scripts used by our different build systems that are
all located in the top-level directory. Now that we have introduced the
new "tools/" directory though we have a better home for them.

Move the scripts into the "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-19 06:40:09 -07:00
Patrick Steinhardt
8ca1b4472c contrib: move "coccinelle/" directory into "tools/"
The Coccinelle tool is an ingrained part of our build infrastructure. It
is executed by our CI to detect antipatterns and is used to detect
misuses of certain interfaces. It's presence in "contrib/" is thus
rather misleading.

Promote the configuration into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-19 06:40:08 -07:00
Patrick Steinhardt
8872941fd2 Introduce new "tools/" directory
According to its readme, the "contrib/" directory's main intent is to
collect stuff that is not an official part of Git, either because it is
too specialized or because it is still considered experimental. The
reality tells a bit of a different story though: while it _does_ contain
such things, it also contains other things:

  - Our credential helpers, which are being distributed by many
    packagers nowadays and which can be considered "stable".

  - A bunch of tooling that relates to our build and test
    infrastructure.

Especially the second category is somewhat of a sore spot. You really
wouldn't expect build-related tooling to be considered an optional part
of Git. Quite the opposite.

Create a new top-level "tools/" directory to fix this discrepancy. This
directory will contain all kind of tools that are related to our build
infrastructure and that Git developers are likely to use day to day.

For now, this directory doesn't contain anything yet except for a
readme and a Meson skeleton. This will change in subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-19 06:40:08 -07:00
Junio C Hamano
2eec0f5115 Merge branch 'jk/unleak-mmap'
Plug a few leaks where mmap'ed memory regions are not unmapped.

* jk/unleak-mmap:
  meson: turn on NO_MMAP when building with LSan
  Makefile: turn on NO_MMAP when building with LSan
  object-file: fix mmap() leak in odb_source_loose_read_object_stream()
  pack-revindex: avoid double-loading .rev files
  check_connected(): fix leak of pack-index mmap
  check_connected(): delay opening new_pack
2026-03-16 10:48:15 -07:00
Patrick Steinhardt
3b9b2c2a29 compat/posix: introduce writev(3p) wrapper
In a subsequent commit we're going to add the first caller to
writev(3p). Introduce a compatibility wrapper for this syscall that we
can use on systems that don't have this syscall.

The syscall exists on modern Unixes like Linux and macOS, and seemingly
even for NonStop according to [1]. It doesn't seem to exist on Windows
though.

[1]: http://nonstoptools.com/manuals/OSS-SystemCalls.pdf
[2]: https://www.gnu.org/software/gnulib/manual/html_node/writev.html

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-13 08:54:14 -07:00
Junio C Hamano
c89a495ce4 Merge branch 'ps/odb-sources'
The object source API is getting restructured to allow plugging new
backends.

* ps/odb-sources:
  odb/source: make `begin_transaction()` function pluggable
  odb/source: make `write_alternate()` function pluggable
  odb/source: make `read_alternates()` function pluggable
  odb/source: make `write_object_stream()` function pluggable
  odb/source: make `write_object()` function pluggable
  odb/source: make `freshen_object()` function pluggable
  odb/source: make `for_each_object()` function pluggable
  odb/source: make `read_object_stream()` function pluggable
  odb/source: make `read_object_info()` function pluggable
  odb/source: make `close()` function pluggable
  odb/source: make `reprepare()` function pluggable
  odb/source: make `free()` function pluggable
  odb/source: introduce source type for robustness
  odb: move reparenting logic into respective subsystems
  odb: embed base source in the "files" backend
  odb: introduce "files" source
  odb: split `struct odb_source` into separate header
2026-03-12 14:09:07 -07:00
Junio C Hamano
8194f1795b Merge branch 'bc/sha1-256-interop-02'
The code to maintain mapping between object names in multiple hash
functions is being added, written in Rust.

* bc/sha1-256-interop-02:
  object-file-convert: always make sure object ID algo is valid
  rust: add a small wrapper around the hashfile code
  rust: add a new binary object map format
  rust: add functionality to hash an object
  rust: add a build.rs script for tests
  rust: fix linking binaries with cargo
  hash: expose hash context functions to Rust
  write-or-die: add an fsync component for the object map
  csum-file: define hashwrite's count as a uint32_t
  rust: add additional helpers for ObjectID
  hash: add a function to look up hash algo structs
  rust: add a hash algorithm abstraction
  rust: add a ObjectID struct
  hash: use uint32_t for object_id algorithm
  conversion: don't crash when no destination algo
  repository: require Rust support for interoperability
2026-03-12 10:56:02 -07:00
Jeff King
00611d86c6 Makefile: turn on NO_MMAP when building with LSan
The past few commits fixed some cases where we leak memory allocated by
mmap(). Building with SANITIZE=leak doesn't detect these because it
covers only heap buffers allocated by malloc().

But if we build with NO_MMAP, our compat mmap() implementation will
allocate a heap buffer and pread() into it. And thus Lsan will detect
these leaks for free.

Using NO_MMAP is less performant, of course, since we have to use extra
memory and read in the whole file, rather than faulting in pages from
disk. But LSan builds are already slow, and this doesn't make them
measurably worse. Getting extra coverage for our leak-checking is worth
it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-06 21:12:10 -08:00
Patrick Steinhardt
cb506a8a69 odb: introduce "files" source
Introduce a new "files" object database source. This source encapsulates
access to both loose object files and the packfile store, similar to how
the "files" backend for refs encapsulates access to loose refs and the
packed-refs file.

Note that for now the "files" source is still a direct member of a
`struct odb_source`. This architecture will be reversed in the next
commit so that the files source contains a `struct odb_source`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-05 11:45:14 -08:00
Patrick Steinhardt
ba1c21d343 odb: split struct odb_source into separate header
Subsequent commits will expand the `struct odb_source` to become a
generic interface for accessing an object database source. As part of
these refactorings we'll add a set of function pointers that will
significantly expand the structure overall.

Prepare for this by splitting out the `struct odb_source` into a
separate header. This keeps the high-level object database interface
detached from the low-level object database sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-05 11:45:14 -08:00
Junio C Hamano
efd5fdbcf9 Merge branch 'dk/meson-regen-config-list'
Fix dependency screw-up in meson-based builds.

* dk/meson-regen-config-list:
  build: regenerate config-list.h when Documentation changes
2026-03-04 10:53:00 -08:00
Junio C Hamano
22c9b6bd93 Merge branch 'kn/osxkeychain-buildfix'
Simplify build procedure for oxskeychain (in contrib/).

* kn/osxkeychain-buildfix:
  osxkeychain: define build targets in the top-level Makefile.
2026-03-04 10:52:59 -08:00
Junio C Hamano
6b5ad01886 Merge branch 'cc/lop-filter-auto'
"auto filter" logic for large-object promisor remote.

* cc/lop-filter-auto:
  fetch-pack: wire up and enable auto filter logic
  promisor-remote: change promisor_remote_reply()'s signature
  promisor-remote: keep advertised filters in memory
  list-objects-filter-options: support 'auto' mode for --filter
  doc: fetch: document `--filter=<filter-spec>` option
  fetch: make filter_options local to cmd_fetch()
  clone: make filter_options local to cmd_clone()
  promisor-remote: allow a client to store fields
  promisor-remote: refactor initialising field lists
2026-02-25 11:54:17 -08:00
D. Ben Knoble
ebeea3c471 build: regenerate config-list.h when Documentation changes
The Meson-based build doesn't know when to rebuild config-list.h, so the
header is sometimes stale.

For example, an old build directory might have config-list.h from before
4173df5187 (submodule: introduce extensions.submodulePathConfig,
2026-01-12), which added submodule.<name>.gitdir to the list. Without
it, t9902-completion.sh fails. Regenerating the config-list.h artifact
from sources fixes the artifact and the test.

Since Meson does not have (or want) builtin support for globbing like
Make, teach generate-configlist.sh to also generate a list of
Documentation files its output depends on, and incorporate that into the
Meson build. We honor the undocumented GCC/Clang contract of outputting
empty targets for all the dependencies (like they do with -MP). That is,
generate lines like

    build/config-list.h: $SOURCE_DIR/Documentation/config.adoc
    $SOURCE_DIR/Documentation/config.adoc:

We assume that if a user adds a new file under
Documentation/config then they will also edit one of the existing files
to include that new file, and that will trigger a rebuild. Also mark the
generator script as a dependency.

While we're at it, teach the Makefile to use the same "the script knows
it's dependencies" logic.

For Meson, combining the following commands helps debug dependencies:

    ninja -C <builddir> -t deps config-list.h
    ninja -C <builddir> -t browse config-list.h

The former lists all the dependencies discovered from our output ".d"
file (the config documentation) and the latter shows the dependency on
the script itself, among other useful edges in the dependency graph.

Helped-by: Patrick Steinhardt <ps@pks.im>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: D. Ben Knoble <ben.knoble+github@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-24 10:51:54 -08:00
Koji Nakamaru
3e9cc24e68 osxkeychain: define build targets in the top-level Makefile.
The fix for git-credential-osxkeychain in 4580bcd235 (osxkeychain: avoid
incorrectly skipping store operation, 2025-11-14) introduced linkage
with libgit.a, and its Makefile was adjusted accordingly. However, the
build fails as of 864f55e190 because several macOS-specific refinements
were applied to the top-level Makefile and config.mak.uname, such as:

  - 363837afe7 (macOS: make Homebrew use configurable, 2025-12-24)
  - cee341e9dd (macOS: use iconv from Homebrew if needed and present,
    2025-12-24)
  - d281241518 (utf8.c: enable workaround for iconv under macOS 14/15,
    2026-01-12)

Since libgit.a and its corresponding header files depend on many flags
defined in the top-level Makefile, these flags must be consistently
defined when building git-credential-osxkeychain. Continuing to manually
adjust the git-credential-osxkeychain Makefile is cumbersome and
fragile.

Define the build targets for git-credential-osxkeychain in the top-level
Makefile and modify its local Makefile to simply rely on those targets.

Helped-by: Junio C Hamano <gitster@pobox.com>
Reported-by: D. Ben Knoble <ben.knoble@gmail.com>
Helped-by: Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com>
Signed-off-by: Koji Nakamaru <koji.nakamaru@gree.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-20 08:10:59 -08:00
Christian Couder
cd1a89838a list-objects-filter-options: support 'auto' mode for --filter
In a following commit, we are going to allow passing "auto" as a
<filterspec> to the `--filter=<filterspec>` option, but only for some
commands. Other commands that support the `--filter=<filterspec>`
option should still die() when 'auto' is passed.

Let's set up the "list-objects-filter-options.{c,h}" infrastructure to
support that:

- Add a new `unsigned int allow_auto_filter : 1;` flag to
  `struct list_objects_filter_options` which specifies if "auto" is
  accepted or not by the current command.
- Change gently_parse_list_objects_filter() to parse "auto" if it's
  accepted.
- Make sure we die() if "auto" is combined with another filter.
- Update list_objects_filter_release() to preserve the
  allow_auto_filter flag, as this function is often called (via
  opt_parse_list_objects_filter) to reset the struct before parsing a
  new value.

Let's also update `list-objects-filter.c` to recognize the new
`LOFC_AUTO` choice. Since "auto" must be resolved to a concrete filter
before filtering actually begins, initializing a filter with
`LOFC_AUTO` is invalid and will trigger a BUG().

Note that ideally combining "auto" with "auto" could be allowed, but in
practice, it's probably not worth the added code complexity. And if we
really want it, nothing prevents us to allow it in future work.

If we ever want to give a meaning to combining "auto" with a different
filter too, nothing prevents us to do that in future work either.

Also note that the new `allow_auto_filter` flag depends on the command,
not user choices, so it should be reset to the command default when
`struct list_objects_filter_options` instances are reset.

While at it, let's add a new "u-list-objects-filter-options.c" file for
`struct list_objects_filter_options` related unit tests. For now it
only tests gently_parse_list_objects_filter() though.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-17 11:46:40 -08:00
Junio C Hamano
e10d5fcad0 Merge branch 'jc/ci-test-contrib-too'
Test contrib/ things in CI to catch breakages before they enter the
"next" branch.

* jc/ci-test-contrib-too:
  : Some of our downstream folks run more tests than we do and catch
  : breakages in them, namely, where contrib/*/Makefile has "test" target.
  : Let's make sure we fail upon accepting a new topic that break them in
  : 'seen'.
  ci: ubuntu: use GNU coreutils for dirname
  test: optionally test contrib in CI
2026-02-13 13:39:26 -08:00
Junio C Hamano
7bf3785d09 Merge branch 'ps/history'
"git history" history rewriting UI.

* ps/history:
  builtin/history: implement "reword" subcommand
  builtin: add new "history" command
  wt-status: provide function to expose status for trees
  replay: support updating detached HEAD
  replay: support empty commit ranges
  replay: small set of cleanups
  builtin/replay: move core logic into "libgit.a"
  builtin/replay: extract core logic to replay revisions
2026-02-09 12:09:09 -08:00