Commit Graph

989 Commits

Author SHA1 Message Date
Karsten Blees
e2c5f7282a add infrastructure for read-only file system level caches
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.

This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.

Enable read-only sections for 'git status' and preload_index.

Signed-off-by: Karsten Blees <blees@dcon.de>
2022-08-11 15:44:24 -07:00
Karsten Blees
14f083ad8e Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:24 -07:00
Karsten Blees
d5d8220fc8 mingw: make the dirent implementation pluggable
Emulating the POSIX `dirent` API on Windows via
`FindFirstFile()`/`FindNextFile()` is pretty staightforward, however,
most of the information provided in the `WIN32_FIND_DATA` structure is
thrown away in the process. A more sophisticated implementation may
cache this data, e.g. for later reuse in calls to `lstat()`.

Make the `dirent` implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to `readdir()`/`closedir()`
that match the `opendir()` implementation (similar to vtable pointers in
Object-Oriented Programming). Define `readdir()`/`closedir()` so that
they call the function pointers in the `DIR` structure. This allows to
choose the `opendir()` implementation on a call-by-call basis.

Make the fixed-size `dirent.d_name` buffer a flex array, as `d_name` may
be implementation specific (e.g. a caching implementation may allocate a
`struct dirent` with _just_ the size needed to hold the `d_name` in
question).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:24 -07:00
Karsten Blees
acdf2a9386 Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2022-08-11 15:44:24 -07:00
Karsten Blees
0020768ece Win32: make FILETIME conversion functions public
We will use them in the upcoming "FSCache" patches (to accelerate
sequential lstat() calls).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:24 -07:00
Johannes Schindelin
467bc7592b Merge branch 'optionally-dont-append-atomically-on-windows'
Fix append failure issue under remote directories #2753

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:23 -07:00
Johannes Schindelin
8a5e55bcf8 Merge pull request #3875 from 1480c1/wine/detect_msys_tty
winansi: check result before using Name for pty
2022-08-11 15:44:23 -07:00
Johannes Schindelin
5db0dddede Merge pull request #3751 from rkitover/native-term
mingw: set $env:TERM=xterm-256color for newer OSes
2022-08-11 15:44:23 -07:00
Derrick Stolee
f6ab3d2bc2 Merge pull request #3791: Various fixes around safe.directory
The first three commits are rebased versions of those in gitgitgadget/git#1215. These allow the following:

1. Fix `git config --global foo.bar <path>` from allowing the `<path>`. As a bonus, users with a config value starting with `/` will not get a warning about "old-style" paths needing a "`%(prefix)/`".

2. When in WSL, the path starts with `/` so it needs to be interpolated properly. Update the warning to include `%(prefix)/` to get the right value for WSL users. (This is specifically for using Git for Windows from Git Bash, but in a WSL directory.)

3. When using WSL, the ownership check fails and reports an error message. This is noisy, and happens even if the user has marked the path with `safe.directory`. Remove that error message.
2022-08-11 15:44:23 -07:00
Victoria Dye
74614e163b Merge branch 'safe.directory-and-windows'
These two patches made it into Git for Windows v2.35.2, but not into Git
v2.35.2.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:23 -07:00
Johannes Schindelin
66adfa531d Merge pull request #3220 from dscho/there-is-no-vs/master-anymore
Let the documentation reflect that there is no vs/master anymore
2022-08-11 15:44:22 -07:00
Johannes Schindelin
4ab8091c46 Merge pull request #3165 from dscho/increase-allowed-length-of-interpreter-path
mingw: allow for longer paths in `parse_interpreter()`
2022-08-11 15:44:22 -07:00
Johannes Schindelin
a917385f63 Merge pull request #2915 from dennisameling/windows-arm64-support
Windows arm64 support
2022-08-11 15:44:22 -07:00
Johannes Schindelin
37c4024ffe Merge pull request #2351 from PhilipOakley/vcpkg-tip
Vcpkg Install: detect lack of working Git, and note possible vcpkg time outs
2022-08-11 15:44:22 -07:00
Johannes Schindelin
14b7ead5e5 Merge pull request #2974 from derrickstolee/maintenance-and-headless
Include Windows-specific maintenance and headless-git
2022-08-11 15:44:22 -07:00
Johannes Schindelin
db095eb3f4 Merge pull request #2506 from dscho/issue-2283
Allow running Git directly from `C:\Program Files\Git\mingw64\bin\git.exe`
2022-08-11 15:44:21 -07:00
Johannes Schindelin
9534411497 Merge pull request #2504 from dscho/access-repo-via-junction
Handle `git add <file>` where <file> traverses an NTFS junction
2022-08-11 15:44:21 -07:00
Johannes Schindelin
5b5c6244e9 Merge pull request #2501 from jeffhostetler/clink-debug-curl
clink.pl: fix MSVC compile script to handle libcurl-d.lib
2022-08-11 15:44:21 -07:00
Johannes Schindelin
dad893060f Merge pull request #2488 from bmueller84/master
mingw: fix fatal error working on mapped network drives on Windows
2022-08-11 15:44:20 -07:00
Johannes Schindelin
8b8eb20072 Merge pull request #2449 from dscho/mingw-getcwd-and-symlinks
Do resolve symlinks in `getcwd()`
2022-08-11 15:44:20 -07:00
Johannes Schindelin
720f7283e8 Merge pull request #2405 from dscho/mingw-setsockopt
Make sure `errno` is set when socket operations fail
2022-08-11 15:44:20 -07:00
Johannes Schindelin
d652896f1c Merge branch 'dont-clean-junctions'
This topic branch teaches `git clean` to respect NTFS junctions and Unix
bind mounts: it will now stop at those boundaries.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:20 -07:00
Johannes Schindelin
3919c7a47d Merge branch 'fsync-object-files-always'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:20 -07:00
孙卓识
c011f24925 Add config option windows.appendAtomically
Atomic append on windows is only supported on local disk files, and it may
cause errors in other situations, e.g. network file system. If that is the
case, this config option should be used to turn atomic append off.

Co-Authored-By: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: 孙卓识 <sunzhuoshi@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:19 -07:00
Christopher Degawa
123a7894a5 winansi: check result and Buffer before using Name
NtQueryObject under Wine can return a success but fill out no name.
In those situations, Wine will set Buffer to NULL, and set result to
the sizeof(OBJECT_NAME_INFORMATION).

Running a command such as

echo "$(git.exe --version 2>/dev/null)"

will crash due to a NULL pointer dereference when the code attempts to
null terminate the buffer, although, weirdly, removing the subshell or
redirecting stdout to a file will not trigger the crash.

Code has been added to also check Buffer and Length to ensure the check
is as robust as possible due to the current behavior being fragile at
best, and could potentially change in the future

This code is based on the behavior of NtQueryObject under wine and
reactos.

Signed-off-by: Christopher Degawa <ccom@randomderp.com>
2022-08-11 15:44:19 -07:00
Rafael Kitover
4a58acbb2e mingw: $env:TERM="xterm-256color" for newer OSes
For Windows builds >= 15063 set $env:TERM to "xterm-256color" instead of
"cygwin" because they have a more capable console system that supports
this. Also set $env:COLORTERM="truecolor" if unset.

$env:TERM is initialized so that ANSI colors in color.c work, see
29a3963484 (Win32: patch Windows environment on startup, 2012-01-15).

See git-for-windows/git#3629 regarding problems caused by always setting
$env:TERM="cygwin".

This is the same heuristic used by the Cygwin runtime.

Signed-off-by: Rafael Kitover <rkitover@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:19 -07:00
Derrick Stolee
28131b9e06 compat/mingw.c: do not warn when failing to get owner
In the case of Git for Windows (say, in a Git Bash window) running in a
Windows Subsystem for Linux (WSL) directory, the GetNamedSecurityInfoW()
call in is_path_owned_By_current_side() returns an error code other than
ERROR_SUCCESS. This is consistent behavior across this boundary.

In these cases, the owner would always be different because the WSL
owner is a different entity than the Windows user.

The change here is to suppress the error message that looks like this:

  error: failed to get owner for '//wsl.localhost/...' (1)

Before this change, this warning happens for every Git command,
regardless of whether the directory is marked with safe.directory.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
2022-08-11 15:44:19 -07:00
Johannes Schindelin
024af91364 mingw: be more informative when ownership check fails on FAT32
The FAT file system has no concept of ACLs. Therefore, it cannot store
any ownership information anyway, and the `GetNamedSecurityInfoW()` call
pretends that everything is owned "by the world".

Let's special-case that scenario and tell the user what's going on, at
least when they set `GIT_TEST_DEBUG_UNSAFE_DIRECTORIES`.

This addresses https://github.com/git-for-windows/git/issues/3886

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:19 -07:00
Johannes Schindelin
b7890b8b3a compat/vcbuild: document preferred way to build in Visual Studio
We used to have that `make vcxproj` hack, but a hack it is. In the
meantime, we have a much cleaner solution: using CMake, either
explicitly, or even more conveniently via Visual Studio's built-in CMake
support (simply open Git's top-level directory via File>Open>Folder...).

Let's let the `README` reflect this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:19 -07:00
Johannes Schindelin
e8b37371b3 mingw: allow for longer paths in parse_interpreter()
As reported in https://github.com/newren/git-filter-repo/pull/225, it
looks like 99 bytes is not really sufficient to represent e.g. the full
path to Python when installed via Windows Store (and this path is used
in the hasb bang line when installing scripts via `pip`).

Let's increase it to what is probably the maximum sensible path size:
MAX_PATH. This makes `parse_interpreter()` in line with what
`lookup_prog()` handles.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Vilius Šumskas <vilius@sumskas.eu>
2022-08-11 15:44:19 -07:00
Johannes Schindelin
06534c4d4f mingw: handle a file owned by the Administrators group correctly
When an Administrator creates a file or directory, the created
file/directory is owned not by the Administrator SID, but by the
_Administrators Group_ SID. The reason is that users with administrator
privileges usually run in unprivileged ("non-elevated") mode, and their
user SID does not change when running in elevated mode.

This is is relevant e.g. when running a GitHub workflow on a build
agent, which runs in elevated mode: cloning a Git repository in a script
step will cause the worktree to be owned by the Administrators Group
SID, for example.

Let's handle this case as following: if the current user is an
administrator, Git should consider a worktree owned by the
Administrators Group as if it were owned by said user.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:19 -07:00
Johannes Schindelin
4a593a01ca Allow debugging unsafe directories' ownership
When Git refuses to use an existing repository because it is owned by
someone else than the current user, it can be a bit tricky on Windows to
figure out what is going on.

Let's help with that by offering some more information via the
environment variable `GIT_TEST_DEBUG_UNSAFE_DIRECTORIES`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:19 -07:00
Dennis Ameling
8a87991560 Add schannel to curl installation
Signed-off-by: Dennis Ameling <dennis@dennisameling.com>
2022-08-11 15:44:18 -07:00
Philip Oakley
df6b4a047d vcpkg_install: add comment regarding slow network connections
The vcpkg downloads may not succeed. Warn careful readers of the time out.

A simple retry will usually resolve the issue.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:18 -07:00
Philip Oakley
b1c462ea4a vcpkg_install: detect lack of Git
The vcpkg_install batch file depends on the availability of a
working Git on the CMD path. This may not be present if the user
has selected the 'bash only' option during Git-for-Windows install.

Detect and tell the user about their lack of a working Git in the CMD
window.

Fixes #2348.
A separate PR https://github.com/git-for-windows/build-extra/pull/258
now highlights the recommended path setting during install.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
2022-08-11 15:44:18 -07:00
Johannes Schindelin
bb6352ba57 win32: add a helper to run git.exe without a foreground window
On Windows, there are two kinds of executables, console ones and
non-console ones. Git's executables are all console ones.

When launching the former e.g. in a scheduled task, a CMD window pops
up. This is not what we want for the tasks installed via the `git
maintenance` command.

To work around this, let's introduce `headless-git.exe`, which is a
non-console program that does _not_ pop up any window. All it does is to
re-launch `git.exe`, suppressing that console window, passing through
all command-line arguments as-are.

Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Helped-by: Yuyi Wang <Strawberry_Str@hotmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
2022-08-11 15:44:18 -07:00
Ian Bearman
18bdadfcd7 vcbuild: add an option to install individual 'features'
In this context, a "feature" is a dependency combined with its own
dependencies.

Signed-off-by: Ian Bearman <ianb@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:18 -07:00
Ian Bearman
20f57cb5a3 vcbuild: install ARM64 dependencies when building ARM64 binaries
Co-authored-by: Dennis Ameling <dennis@dennisameling.com>
Signed-off-by: Ian Bearman <ianb@microsoft.com>
Signed-off-by: Dennis Ameling <dennis@dennisameling.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:18 -07:00
Jeff Hostetler
8f7dc4c0f8 clink.pl: move default linker options for MSVC=1 builds
Move the default `-ENTRY` and `-SUBSYSTEM` arguments for
MSVC=1 builds from `config.mak.uname` into `clink.pl`.
These args are constant for console-mode executables.

Add support to `clink.pl` for generating a Win32 GUI application
using the `-mwindows` argument (to match how GCC does it).  This
changes the `-ENTRY` and `-SUBSYSTEM` arguments accordingly.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2022-08-11 15:44:18 -07:00
Jeff Hostetler
be0746c399 clink.pl: ignore no-stack-protector arg on MSVC=1 builds
Ignore the `-fno-stack-protector` compiler argument when building
with MSVC.  This will be used in a later commit that needs to build
a Win32 GUI app.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2022-08-11 15:44:18 -07:00
Jeff Hostetler
9bfaefae24 vcbuild: add support for compiling Windows resource files
Create a wrapper for the Windows Resource Compiler (RC.EXE)
for use by the MSVC=1 builds. This is similar to the CL.EXE
and LIB.EXE wrappers used for the MSVC=1 builds.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2022-08-11 15:44:18 -07:00
Jeff Hostetler
e94e702b5f clink.pl: fix libexpatd.lib link error when using MSVC
When building with `make MSVC=1 DEBUG=1`, link to `libexpatd.lib`
rather than `libexpat.lib`.

It appears that the `vcpkg` package for "libexpat" has changed and now
creates `libexpatd.lib` for debug mode builds.  Previously, both debug
and release builds created a ".lib" with the same basename.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2022-08-11 15:44:18 -07:00
Johannes Schindelin
02d5a1a84c mingw: ignore HOMEDRIVE/HOMEPATH if it points to Windows' system directory
Internally, Git expects the environment variable `HOME` to be set, and
to point to the current user's home directory.

This environment variable is not set by default on Windows, and
therefore Git tries its best to construct one if it finds `HOME` unset.

There are actually two different approaches Git tries: first, it looks
at `HOMEDRIVE`/`HOMEPATH` because this is widely used in corporate
environments with roaming profiles, and a user generally wants their
global Git settings to be in a roaming profile.

Only when `HOMEDRIVE`/`HOMEPATH` is either unset or does not point to a
valid location, Git will fall back to using `USERPROFILE` instead.

However, starting with Windows Vista, for secondary logons and services,
the environment variables `HOMEDRIVE`/`HOMEPATH` point to Windows'
system directory (usually `C:\Windows\system32`).

That is undesirable, and that location is usually write-protected anyway.

So let's verify that the `HOMEDRIVE`/`HOMEPATH` combo does not point to
Windows' system directory before using it, falling back to `USERPROFILE`
if it does.

This fixes git-for-windows#2709

Initial-Path-by: Ivan Pozdeev <vano@mail.mipt.ru>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:17 -07:00
Johannes Schindelin
51c6069ce8 mingw: implement a platform-specific strbuf_realpath()
There is a Win32 API function to resolve symbolic links, and we can use
that instead of resolving them manually. Even better, this function also
resolves NTFS junction points (which are somewhat similar to bind
mounts).

This fixes https://github.com/git-for-windows/git/issues/2481.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:17 -07:00
Jeff Hostetler
cf6155049e clink.pl: fix MSVC compile script to handle libcurl-d.lib
Update clink.pl to link with either libcurl.lib or libcurl-d.lib
depending on whether DEBUG=1 is set.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:17 -07:00
Bjoern Mueller
3bc0767740 mingw: fix fatal error working on mapped network drives on Windows
In 1e64d18 (mingw: do resolve symlinks in `getcwd()`) a problem was
introduced that causes git for Windows to stop working with certain
mapped network drives (in particular, drives that are mapped to
locations with long path names). Error message was "fatal: Unable to
read current working directory: No such file or directory". Present
change fixes this issue as discussed in
https://github.com/git-for-windows/git/issues/2480

Signed-off-by: Bjoern Mueller <bjoernm@gmx.de>
2022-08-11 15:44:17 -07:00
Johannes Schindelin
b9671e2cc0 mingw: do resolve symlinks in getcwd()
As pointed out in https://github.com/git-for-windows/git/issues/1676,
the `git rev-parse --is-inside-work-tree` command currently fails when
the current directory's path contains symbolic links.

The underlying reason for this bug is that `getcwd()` is supposed to
resolve symbolic links, but our `mingw_getcwd()` implementation did not.

We do have all the building blocks for that, though: the
`GetFinalPathByHandleW()` function will resolve symbolic links. However,
we only called that function if `GetLongPathNameW()` failed, for
historical reasons: the latter function was supported for a long time,
but the former API function was introduced only with Windows Vista, and
we used to support also Windows XP. With that support having been
dropped, we are free to call the symbolic link-resolving function right
away.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:17 -07:00
Johannes Schindelin
12bedeac8c mingw: make sure errno is set correctly when socket operations fail
The winsock2 library provides functions that work on different data
types than file descriptors, therefore we wrap them.

But that is not the only difference: they also do not set `errno` but
expect the callers to enquire about errors via `WSAGetLastError()`.

Let's translate that into appropriate `errno` values whenever the socket
operations fail so that Git's code base does not have to change its
expectations.

This closes https://github.com/git-for-windows/git/issues/2404

Helped-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:17 -07:00
Johannes Schindelin
c9a2ff2c0d clean: remove mount points when possible
Windows' equivalent to "bind mounts", NTFS junction points, can be
unlinked without affecting the mount target. This is clearly what users
expect to happen when they call `git clean -dfx` in a worktree that
contains NTFS junction points: the junction should be removed, and the
target directory of said junction should be left alone (unless it is
inside the worktree).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:17 -07:00
Johannes Schindelin
09f49eb33f mingw: change core.fsyncObjectFiles = 1 by default
From the documentation of said setting:

	This boolean will enable fsync() when writing object files.

	This is a total waste of time and effort on a filesystem that
	orders data writes properly, but can be useful for filesystems
	that do not use journalling (traditional UNIX filesystems) or
	that only journal metadata and not file contents (OS X’s HFS+,
	or Linux ext3 with "data=writeback").

The most common file system on Windows (NTFS) does not guarantee that
order, therefore a sudden loss of power (or any other event causing an
unclean shutdown) would cause corrupt files (i.e. files filled with
NULs). Therefore we need to change the default.

Note that the documentation makes it sound as if this causes really bad
performance. In reality, writing loose objects is something that is done
only rarely, and only a handful of files at a time.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-08-11 15:44:17 -07:00