Commit Graph

148027 Commits

Author SHA1 Message Date
Karsten Blees
44ddce3322 mingw: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow
`lstat()` emulation (git calls `lstat()` once for each file in the
index). Windows operating system APIs seem to be much better at scanning
the status of entire directories than checking single files.

Add an `lstat()` implementation that uses a cache for lstat data. Cache
misses read the entire parent directory and add it to the cache.
Subsequent `lstat()` calls for the same directory are served directly
from the cache.

Also implement `opendir()`/`readdir()`/`closedir()` so that they create
and use directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:46 +02:00
Karsten Blees
aac792fed0 mingw: add infrastructure for read-only file system level caches
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.

This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.

Enable read-only sections for 'git status' and preload_index.

Signed-off-by: Karsten Blees <blees@dcon.de>
2023-08-10 18:11:46 +02:00
Karsten Blees
81e5c339f6 Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:46 +02:00
Karsten Blees
532beb7055 mingw: make the dirent implementation pluggable
Emulating the POSIX `dirent` API on Windows via
`FindFirstFile()`/`FindNextFile()` is pretty staightforward, however,
most of the information provided in the `WIN32_FIND_DATA` structure is
thrown away in the process. A more sophisticated implementation may
cache this data, e.g. for later reuse in calls to `lstat()`.

Make the `dirent` implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to `readdir()`/`closedir()`
that match the `opendir()` implementation (similar to vtable pointers in
Object-Oriented Programming). Define `readdir()`/`closedir()` so that
they call the function pointers in the `DIR` structure. This allows to
choose the `opendir()` implementation on a call-by-call basis.

Make the fixed-size `dirent.d_name` buffer a flex array, as `d_name` may
be implementation specific (e.g. a caching implementation may allocate a
`struct dirent` with _just_ the size needed to hold the `d_name` in
question).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:46 +02:00
Karsten Blees
2e6bd3d3ec Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2023-08-10 18:11:46 +02:00
Karsten Blees
e8b510eb04 Win32: make FILETIME conversion functions public
We will use them in the upcoming "FSCache" patches (to accelerate
sequential lstat() calls).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:46 +02:00
Johannes Schindelin
a71e43d748 Merge branch 'ready-for-upstream'
This is the branch thicket of patches in Git for Windows that are
considered ready for upstream. To keep them in a ready-to-submit shape,
they are kept as close to the beginning of the branch thicket as
possible.
2023-08-10 18:11:45 +02:00
Johannes Schindelin
a820fb24c3 Merge branch 'suppress-t0040-whitespace-error'
A recently-introduced commit fails Git's own whitespace check. Fix this
by exempting the affected file from that particular whitespace check
because it does not apply here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:45 +02:00
Johannes Schindelin
e496c71656 Additional error checks for issuing the windows.appendAtomically warning (#4528)
Another (hopefully clean) PR for showing the error warning about atomic
append on windows after failure on APFS, which returns EBADF not EINVAL.

Signed-off-by: David Lomas <dl3@pale-eds.co.uk>
2023-08-10 18:11:44 +02:00
Johannes Schindelin
62c94376c8 Merge branch 'nano-server'
This patch adds a GitHub workflow (to be triggered manually) to allow
for conveniently verifying that Git and Scalar still work as intended in
Windows Nano Server (a relatively small container base image that is
frequently used where a "small Windows" is needed, e.g. in automation
;-))

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:44 +02:00
Johannes Schindelin
91ebd92786 Lazy load libcurl, allowing for an SSL/TLS backend-specific libcurl (#4410)
As per
https://github.com/git-for-windows/git/issues/4350#issuecomment-1485041503,
the major block for upgrading Git for Windows' OpenSSL from v1.1 to v3
is the tricky part where such an upgrade would break `git fetch`/`git
clone` and `git push` because the libcurl depends on the OpenSSL DLL,
and the major version bump will _change_ the file name of said DLL.

To overcome that, the plan is to build libcurl flavors for each
supported SSL/TLS backend, aligning with the way MSYS2 builds libcurl,
then switch Git for Windows' SDK to the Secure Channel-flavored libcurl,
and teach Git to look for the specific flavor of libcurl corresponding
to the `http.sslBackend` setting (if that was configured).

Here is the PR to teach Git that trick.
2023-08-10 18:11:44 +02:00
Johannes Schindelin
62a7815f6a Git GUI: fix Repository>Explore Working Copy (#4357)
This is a companion PR of https://github.com/prati0100/git-gui/pull/95

Since Git v2.39.1, we are a bit more stringent in searching the PATH. In
particular, we specifically require the `.exe` suffix.

However, the `Repository>Explore Working Copy` command asks for
`explorer.exe` to be found on the `PATH`, which _already_ has that
suffix.

Let's unstartle the PATH-finding logic about this scenario.

This fixes https://github.com/git-for-windows/git/issues/4356
2023-08-10 18:11:43 +02:00
Johannes Schindelin
4722150a8f Skip linking the "dashed" git-<command>s for built-ins (#4252)
It is merely a historical wart that, say, `git-commit` exists in the
`libexec/git-core/` directory, a tribute to the original idea to let Git
be essentially a bunch of Unix shell scripts revolving around very few
"plumbing" (AKA low-level) commands.

Git has evolved a lot from there. These days, most of Git's
functionality is contained within the `git` executable, in the form of
"built-in" commands.

To accommodate for scripts that use the "dashed" form of Git commands,
even today, Git provides hard-links that make the `git` executable
available as, say, `git-commit`, just in case that an old script has not
been updated to invoke `git commit`.

Those hard-links do not come cheap: they take about half a minute for
every build of Git on Windows, they are mistaken for taking up huge
amounts of space by some Windows Explorer versions that do not
understand hard-links, and therefore many a "bug" report had to be
addressed.

The "dashed form" has been officially deprecated in Git version 1.5.4,
which was released on February 2nd, 2008, i.e. a very long time ago.
This deprecation was never finalized by skipping these hard-links, but
we can start the process now, in Git for Windows.

This addresses the concern raised in
https://github.com/git-for-windows/git/pull/4185#discussion_r1051661894
2023-08-10 18:11:43 +02:00
Johannes Schindelin
c3aea846c9 Fix global repository field not being cleared (#4083)
It is checked for w.r.t. global repository struct down in the callstack
in compatibility layer for MinGW before being assigned in the function
that `free()`'d it.
2023-08-10 18:11:43 +02:00
Johannes Schindelin
8747a67d96 Add support for CLANGARM64 target (#3916)
**This requires an ARM64-machine with Windows 11 installed (which
supports x64 emulation for MSYS2)**

### The main idea

- Use the main MSYS2/git-sdk-64 setup, which works on Windows 11 on ARM
thanks to x64-emulation
- Configure the official `clangarm64` MSYS2 repo
- Install `mingw-w64-clang-aarch64-toolchain` which contains the
ARM64-native Clang compiler
2023-08-10 18:11:42 +02:00
Johannes Schindelin
343c63a1b9 Merge branch 'builtin-swap-functions'
Do prefer GCC's built-in bit-swap functions.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:42 +02:00
Johannes Schindelin
35c871eb50 Fix Windows version resources (#4092)
Add `FileVersion`, which is a required string ([Microsoft
documentation](https://learn.microsoft.com/en-us/windows/win32/menurc/versioninfo-resource))
in the `StringFileInfo` block.
As not all required strings were present in the block, none were being
included.
Fixes #4090

After including the `FileVersion` string, all other defined strings are
now being included on executables.

File version information for `git.exe` has changed from:
```
PS C:\Program Files\Git\bin> [System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Data\git-sdk-64\usr\src\git\git.exe") | Select-Object *

FileVersionRaw     : 2.38.1.1
ProductVersionRaw  : 2.38.1.1
Comments           :
CompanyName        :
FileBuildPart      : 1
FileDescription    :
FileMajorPart      : 2
FileMinorPart      : 38
FileName           : C:\Data\git-sdk-64\usr\src\git\git.exe
FilePrivatePart    : 1
FileVersion        :
InternalName       :
IsDebug            : False
IsPatched          : False
IsPrivateBuild     : False
IsPreRelease       : False
IsSpecialBuild     : False
Language           : English (United States)
LegalCopyright     :
LegalTrademarks    :
OriginalFilename   :
PrivateBuild       :
ProductBuildPart   : 1
ProductMajorPart   : 2
ProductMinorPart   : 38
ProductName        :
ProductPrivatePart : 1
ProductVersion     :
SpecialBuild       :
```

To the following:
```
PS C:\Program Files\Git\bin> [System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Data\git-sdk-64\usr\src\git\git.exe") | Select-Object *

FileVersionRaw     : 2.38.1.1
ProductVersionRaw  : 2.38.1.1
Comments           :
CompanyName        : The Git Development Community
FileBuildPart      : 1
FileDescription    : Git for Windows
FileMajorPart      : 2
FileMinorPart      : 38
FileName           : C:\Data\git-sdk-64\usr\src\git\git.exe
FilePrivatePart    : 1
FileVersion        : 2.38.1.windows.1.10.g6ed65a6fab
InternalName       : git
IsDebug            : False
IsPatched          : False
IsPrivateBuild     : False
IsPreRelease       : False
IsSpecialBuild     : False
Language           : English (United States)
LegalCopyright     :
LegalTrademarks    :
OriginalFilename   : git.exe
PrivateBuild       :
ProductBuildPart   : 1
ProductMajorPart   : 2
ProductMinorPart   : 38
ProductName        : Git
ProductPrivatePart : 1
ProductVersion     : 2.38.1.windows.1.10.g6ed65a6fab
SpecialBuild       :
```

I wasn't really expecting `GIT_VERSION` to contain the Git commit, I was
hoping for just `2.38.1` or `2.38.1.1`, at least for the `FileVersion`
string.

Anybody know if it's possible to concatenate the `MAJOR`, `MINOR`,
`MICRO`, and `PATCHLEVEL` fields with dots, or if there's another
variable that can be used (with or without `PATCHLEVEL`)?
Alternatively, use the complete `GIT_VERSION` for both `FileVersion` and
`ProductVersion`.
2023-08-10 18:11:42 +02:00
Johannes Schindelin
80d91e7dd8 Merge pull request #3942 from rimrul/mingw-tsaware
MinGW: link as terminal server aware
2023-08-10 18:11:42 +02:00
Johannes Schindelin
a44487bfd0 Merge branch 'ci-fixes'
Backport a couple fixes to make the CI build run again (so much for
reproducible builds...).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:41 +02:00
Johannes Schindelin
e1be12d4ed Merge branch 'fsync-object-files-always'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:41 +02:00
Johannes Schindelin
336942cb21 Merge branch 'optionally-dont-append-atomically-on-windows'
Fix append failure issue under remote directories #2753

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:41 +02:00
Johannes Schindelin
652e47ad82 Merge pull request #3875 from 1480c1/wine/detect_msys_tty
winansi: check result before using Name for pty
2023-08-10 18:11:40 +02:00
Johannes Schindelin
b7434cbcc1 Merge pull request #3751 from rkitover/native-term
mingw: set $env:TERM=xterm-256color for newer OSes
2023-08-10 18:11:40 +02:00
Derrick Stolee
1e0801905f Merge pull request #3791: Various fixes around safe.directory
The first three commits are rebased versions of those in gitgitgadget/git#1215. These allow the following:

1. Fix `git config --global foo.bar <path>` from allowing the `<path>`. As a bonus, users with a config value starting with `/` will not get a warning about "old-style" paths needing a "`%(prefix)/`".

2. When in WSL, the path starts with `/` so it needs to be interpolated properly. Update the warning to include `%(prefix)/` to get the right value for WSL users. (This is specifically for using Git for Windows from Git Bash, but in a WSL directory.)

3. When using WSL, the ownership check fails and reports an error message. This is noisy, and happens even if the user has marked the path with `safe.directory`. Remove that error message.
2023-08-10 18:11:40 +02:00
Johannes Schindelin
5d0824a766 Merge pull request #3533 from PhilipOakley/hashliteral_t
Begin `unsigned long`->`size_t` conversion to support large files on Windows
2023-08-10 18:11:39 +02:00
Jeff Hostetler
161676dc1c Merge branch 'mark-v4-fsmonitor-experimental' into try-v4-fsmonitor 2023-08-10 18:11:39 +02:00
Johannes Schindelin
e1e35c51f9 Merge pull request #3417 from dscho/initialize-core.symlinks-earlier
init: respect core.symlinks before copying the templates
2023-08-10 18:11:39 +02:00
Johannes Schindelin
70411c51d5 Merge pull request #3306 from PhilipOakley/vs-sln
Make Git for Windows start builds in modern Visual Studio
2023-08-10 18:11:38 +02:00
Johannes Schindelin
67c5df5408 Merge pull request #3349 from vdye/feature/ci-subtree-tests
Add `contrib/subtree` test execution to CI builds
2023-08-10 18:11:38 +02:00
Johannes Schindelin
c522651c01 Merge pull request #3293 from pascalmuller/http-support-automatically-sending-client-certificate
http: Add support for enabling automatic sending of SSL client certificate
2023-08-10 18:11:38 +02:00
Johannes Schindelin
5303d3eb27 Merge pull request #3220 from dscho/there-is-no-vs/master-anymore
Let the documentation reflect that there is no vs/master anymore
2023-08-10 18:11:37 +02:00
Johannes Schindelin
00b7b9771a Merge pull request #3165 from dscho/increase-allowed-length-of-interpreter-path
mingw: allow for longer paths in `parse_interpreter()`
2023-08-10 18:11:37 +02:00
Johannes Schindelin
837c879408 Merge pull request #3150 from dscho/ci-cache-vcpkg-artifacts-g4w
ci: cache vcpkg artifacts
2023-08-10 18:11:37 +02:00
Johannes Schindelin
e9e1538a9d Merge pull request #3327 from dennisameling/fix-host-cpu
cmake(): allow setting HOST_CPU for cross-compilation
2023-08-10 18:11:36 +02:00
Johannes Schindelin
0de5d38c79 Merge pull request #2915 from dennisameling/windows-arm64-support
Windows arm64 support
2023-08-10 18:11:36 +02:00
Johannes Schindelin
baf3805465 Merge pull request #2351 from PhilipOakley/vcpkg-tip
Vcpkg Install: detect lack of working Git, and note possible vcpkg time outs
2023-08-10 18:11:36 +02:00
Johannes Schindelin
db4c4d833a cmake: install headless-git. (#4338)
Even if CMake is not the canonical way to build Git for Windows, but
CMake support merely exists in Git to support building Git for Windows
using Visual Studio, we should include `headless-git` in such a scenario
when installing the binaries to a given location.
2023-08-10 18:11:36 +02:00
Johannes Schindelin
a482d71a6a Merge pull request #2974 from derrickstolee/maintenance-and-headless
Include Windows-specific maintenance and headless-git
2023-08-10 18:11:35 +02:00
Johannes Schindelin
3983f44e2b Merge 'git-gui/js/intent-to-add'
This merges the current version of the patch that tries to address Git
GUI's problems with intent-to-add files.

This patch will likely be improved substantially before it is merged
into Git GUI's main branch, but we want to have _something_ resembling a
fix already in Git for Windows v2.29.0.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:35 +02:00
Johannes Schindelin
68286e50c8 Merge pull request #2655 from jglathe/jg/t0014_trace_extra_info
t/t0014: fix: eliminate additional lines from trace
2023-08-10 18:11:35 +02:00
Johannes Schindelin
870494fef7 Merge pull request #2714 from lbonanomi/crlf-scissors
Rationalize line endings for scissors-cleanup
2023-08-10 18:11:34 +02:00
Johannes Schindelin
21981be875 Merge pull request #2730 from dscho/crlf-aware-git-add-i
git add -i: handle CR/LF line endings in the interactive input
2023-08-10 18:11:34 +02:00
Johannes Schindelin
673ab9b694 Merge 'add-p-many-files'
This topic branch allows `add -p` and `add -i` with a large number of
files. It is kind of a hack that was never really meant to be
upstreamed. Let's see if we can do better in the built-in `add -p`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-08-10 18:11:34 +02:00
Johannes Schindelin
f958ca7e1c Merge pull request #2618 from dscho/avoid-d/f-conflict-in-vs/master
ci: avoid d/f conflict in vs/master
2023-08-10 18:11:33 +02:00
Johannes Schindelin
d0d91ca6aa Merge pull request #2535 from dscho/schannel-revoke-best-effort
Introduce and use the new "best effort" strategy for Secure Channel revoke checking
2023-08-10 18:11:33 +02:00
Johannes Schindelin
6199d83337 Merge pull request #2506 from dscho/issue-2283
Allow running Git directly from `C:\Program Files\Git\mingw64\bin\git.exe`
2023-08-10 18:11:33 +02:00
Johannes Schindelin
5148bc7d95 Merge pull request #2504 from dscho/access-repo-via-junction
Handle `git add <file>` where <file> traverses an NTFS junction
2023-08-10 18:11:32 +02:00
Johannes Schindelin
7f30b81ec5 Merge pull request #2501 from jeffhostetler/clink-debug-curl
clink.pl: fix MSVC compile script to handle libcurl-d.lib
2023-08-10 18:11:32 +02:00
Johannes Schindelin
83bc5d192e Merge pull request #2488 from bmueller84/master
mingw: fix fatal error working on mapped network drives on Windows
2023-08-10 18:11:32 +02:00
Johannes Schindelin
8fa0f98983 Merge pull request #2449 from dscho/mingw-getcwd-and-symlinks
Do resolve symlinks in `getcwd()`
2023-08-10 18:11:31 +02:00