Commit Graph

918 Commits

Author SHA1 Message Date
Jeff Hostetler
6e01541f8b Merge branch 'try-v4-fsmonitor-part4' into try-v4-fsmonitor 2021-11-03 20:20:03 -04:00
Jeff Hostetler
d504baee33 Merge branch 'try-v4-fsmonitor-part3' into try-v4-fsmonitor 2021-11-03 20:20:03 -04:00
Johannes Schindelin
f67ed3e7da Merge pull request #3398 from carenas/pthread-unistd
mingw: avoid fallback for {local,gm}time_r()
2021-11-03 20:20:02 -04:00
Johannes Schindelin
2f1a577ca4 Merge pull request #3220 from dscho/there-is-no-vs/master-anymore
Let the documentation reflect that there is no vs/master anymore
2021-11-03 20:20:02 -04:00
Johannes Schindelin
0e67089b0b Merge pull request #3165 from dscho/increase-allowed-length-of-interpreter-path
mingw: allow for longer paths in `parse_interpreter()`
2021-11-03 20:20:02 -04:00
Johannes Schindelin
14e2680df5 Merge pull request #2915 from dennisameling/windows-arm64-support
Windows arm64 support
2021-11-03 20:20:01 -04:00
Johannes Schindelin
58efccf5b3 Merge pull request #2351 from PhilipOakley/vcpkg-tip
Vcpkg Install: detect lack of working Git, and note possible vcpkg time outs
2021-11-03 20:20:01 -04:00
Johannes Schindelin
2f1e772b5e Merge pull request #2974 from derrickstolee/maintenance-and-headless
Include Windows-specific maintenance and headless-git
2021-11-03 20:20:01 -04:00
Johannes Schindelin
8a49738453 Merge pull request #2506 from dscho/issue-2283
Allow running Git directly from `C:\Program Files\Git\mingw64\bin\git.exe`
2021-11-03 20:20:00 -04:00
Johannes Schindelin
30c6860290 Merge pull request #2504 from dscho/access-repo-via-junction
Handle `git add <file>` where <file> traverses an NTFS junction
2021-11-03 20:20:00 -04:00
Johannes Schindelin
5bd9e46fa6 Merge pull request #2501 from jeffhostetler/clink-debug-curl
clink.pl: fix MSVC compile script to handle libcurl-d.lib
2021-11-03 20:20:00 -04:00
Johannes Schindelin
01f5498fb4 Merge pull request #2488 from bmueller84/master
mingw: fix fatal error working on mapped network drives on Windows
2021-11-03 20:19:59 -04:00
Johannes Schindelin
58c1a05ff4 Merge pull request #2449 from dscho/mingw-getcwd-and-symlinks
Do resolve symlinks in `getcwd()`
2021-11-03 20:19:59 -04:00
Johannes Schindelin
454a6a3374 Merge pull request #2405 from dscho/mingw-setsockopt
Make sure `errno` is set when socket operations fail
2021-11-03 20:19:59 -04:00
Johannes Schindelin
c9f752d6a9 Merge branch 'dont-clean-junctions'
This topic branch teaches `git clean` to respect NTFS junctions and Unix
bind mounts: it will now stop at those boundaries.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:59 -04:00
Johannes Schindelin
0c2b027de1 Merge branch 'fsync-object-files-always'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:59 -04:00
Jeff Hostetler
564eb98b64 compat/fsmonitor/fsm-listen-darwin: shutdown daemon if worktree root is moved/renamed
Teach the listener thread to shutdown the daemon if the spelling of the
worktree root directory changes.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
a7a7111610 compat/fsmonitor/fsm-health-win32: force shutdown daemon if worktree root moves
Force shutdown fsmonitor daemon if the worktree root directory
is moved, renamed, or deleted.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
5a7ad4adfd compat/fsmonitor/fsm-health-win32: add framework for periodically monitoring
Create framework in Win32 version of the "health" thread to
periodically inspect the system and shutdown if warranted.

This version just include the setup for the timeout in
WaitForMultipleObjects() and calls (currently empty) table
of functions.

A later commit will add functions to the table to actually
inspect the system.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
5e74e6111a fsmonitor--daemon: stub in health thread
Create another thread to watch over the daemon process and
automatically shut it down if necessary.

This commit creates the basic framework for a "health" thread
to monitor the daemon and/or the file system.  Later commits
will add platform-specific code to do the actual work.

The "health" thread is intended to monitor conditions that
would be difficult to track inside the IPC thread pool and/or
the file system listener threads.  For example, when there are
file system events outside of the watched worktree root or if
we want to have an idle-timeout auto-shutdown feature.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
a17cef4789 fsmonitor--daemon: rename listener thread related variables
Rename platform-specific listener thread related variables
and data types as we prepare to add another backend thread
type.

[] `struct fsmonitor_daemon_backend_data` becomes `struct fsm_listen_data`
[] `state->backend_data` becomes `state->listen_data`
[] `state->error_code` becomes `state->listen_error_code`

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
fe97311460 fsmonitor--daemon: cd out of worktree root
Teach the fsmonitor--daemon to CD outside of the worktree
before starting up.

The common Git startup mechanism causes the CWD of the daemon process
to be in the root of the worktree.  On Windows, this causes the daemon
process to hold a locked handle on the CWD and prevents other
processes from moving or deleting the worktree while the daemon is
running.

CD to HOME before entering main event loops.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
bfb0976bfa compat/fsmonitor/fsm-listen-darwin: ignore FSEvents caused by xattr changes on MacOS
Ignore FSEvents resulting from `xattr` changes.  Git does not care about
xattr's or changes to xattr's, so don't waste time collecting these
events in the daemon nor transmitting them to clients.

Various security tools add xattrs to files and/or directories, such as
to mark them as having been downloaded.  We should ignore these events
since it doesn't affect the content of the file/directory or the normal
meta-data that Git cares about.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
d825514b4b fsmonitor-settings: remote repos on Windows are incompatible with FSMonitor
Teach Git to detect remote working directories on Windows and mark them as
incompatible with FSMonitor.

With this `git fsmonitor--daemon run` will error out with a message like it
does for bare repos.

Client commands, such as `git status`, will not attempt to start the daemon.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
3f84ce337b fsmonitor-settings: remote repos on MacOS are incompatible with FSMonitor
Teach Git to detect remote working directories on MacOS and mark them as
incompatible with FSMonitor.

With this, `git fsmonitor--daemon run` will error out with a message
like it does for bare repos.

Client commands, like `git status`, will not attempt to start the daemon.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
8faa6bc48e fsmonitor-settings: stub in platform-specific incompatibility checking on MacOS
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
eff4c11f0f fsmonitor-settings: virtual repos are incompatible with FSMonitor
Virtual repos, such as GVFS (aka VFS for Git), are incompatible
with FSMonitor.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:58 -04:00
Jeff Hostetler
9a1ae86a50 fsmonitor-settings: stub in platform-specific incompatibility checking
Extend generic incompatibility checkout with platform-specific
mechanism.  Stub in Win32 version.

In the existing fsmonitor-settings code we have a way to mark
types of repos as incompatible with fsmonitor (whether via the
hook and ipc APIs).  For example, we do this for bare repos,
since there are no files to watch.

Extend this exclusion mechanism for platfor-specific reasons.
This commit just creates the framework and adds a stub for Win32.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:57 -04:00
Jeff Hostetler
64c0cef838 compat/fsmonitor/fsm-listen-win32: handle shortnames
Teach FSMonitor daemon on Windows to recognize shortname paths as
aliases of normal longname paths.  FSMonitor clients, such as `git
status`, should receive the longname spelling of changed files (when
possible).

Sometimes we receive FS events using the shortname, such as when a CMD
shell runs "RENAME GIT~1 FOO" or "RMDIR GIT~1".  The FS notification
arrives using whatever combination of long and shortnames were used by
the other process.  (Shortnames do seem to be case normalized,
however.)

Use Windows GetLongPathNameW() to try to map the pathname spelling in
the notification event into the normalized longname spelling.  (This
can fail if the file/directory is deleted, moved, or renamed, because
we are asking the FS for the mapping in response to the event and
after it has already happened, but we try.)

Special case the shortname spelling of ".git" to avoid under-reporting
these events.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:57 -04:00
Jeff Hostetler
4a3bcae79b compat/fsmonitor/fsm-listen-darwin: implement FSEvent listener on MacOS
Implement file system event listener on MacOS using FSEvent,
CoreFoundation, and CoreServices.

Co-authored-by: Kevin Willford <Kevin.Willford@microsoft.com>
Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:57 -04:00
Jeff Hostetler
bfd7ffbc4f compat/fsmonitor/fsm-listen-darwin: add macos header files for FSEvent
Include MacOS system declarations to allow us to use FSEvent and
CoreFoundation APIs.  We need GCC and clang versions because of
compiler and header file conflicts.

While it is quite possible to #include Apple's CoreServices.h when
compiling C source code with clang, trying to build it with GCC
currently fails with this error:

In file included
   from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/AuthSession.h:32,
   from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/Security.h:42,
   from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/OSServices.framework/Headers/CSIdentity.h:43,
   from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/OSServices.framework/Headers/OSServices.h:29,
   from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Headers/IconsCore.h:23,
   from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Headers/LaunchServices.h:23,
   from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:45,
     /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/Authorization.h:193:7: error: variably modified 'bytes' at file scope
       193 | char bytes[kAuthorizationExternalFormLength];
           |      ^~~~~

The underlying reason is that GCC (rightfully) objects that an `enum`
value such as `kAuthorizationExternalFormLength` is not a constant
(because it is not, the preprocessor has no knowledge of it, only the
actual C compiler does) and can therefore not be used to define the size
of a C array.

This is a known problem and tracked in GCC's bug tracker:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93082

In the meantime, let's not block things and go the slightly ugly route
of declaring/defining the FSEvents constants, data structures and
functions that we need, so that we can avoid above-mentioned issue.

Let's do this _only_ for GCC, though, so that the CI/PR builds (which
build both with clang and with GCC) can guarantee that we _are_ using
the correct data types.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:57 -04:00
Jeff Hostetler
39696b1c2f compat/fsmonitor/fsm-listen-win32: implement FSMonitor backend on Windows
Teach the win32 backend to register a watch on the working tree
root directory (recursively).  Also watch the <gitdir> if it is
not inside the working tree.  And to collect path change notifications
into batches and publish.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:57 -04:00
Jeff Hostetler
644337e073 compat/fsmonitor/fsm-listen-darwin: stub in backend for Darwin
Stub in empty implementation of fsmonitor--daemon
backend for Darwin (aka MacOS).

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:57 -04:00
Jeff Hostetler
0c7286d44c compat/fsmonitor/fsm-listen-win32: stub in backend for Windows
Stub in empty filesystem listener backend for fsmonitor--daemon on Windows.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:57 -04:00
Carlo Marcelo Arenas Belón
c97bbecbf0 mingw: avoid fallback for {local,gm}time_r()
mingw-w64's pthread_unistd.h had a bug that mistakenly (because there is
no support for the *lockfile() functions required[1]) defined
_POSIX_THREAD_SAFE_FUNCTIONS and that was being worked around since
3ecd153a3b (compat/mingw: support MSys2-based MinGW build, 2016-01-14).

the bug was fixed in winphtreads, but as a sideeffect, leaves the
reentrant functions from time.h not longer visible and therefore breaks
the build.

since the intention all along was to avoid using the fallback functions,
formalize the use of POSIX by setting the corresponding feature flag and
to make the intention clearer compile out the fallback functions.

[1] https://unix.org/whitepapers/reentrant.html

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
2021-11-03 20:19:56 -04:00
Johannes Schindelin
3c4dcc5d26 compat/vcbuild: document preferred way to build in Visual Studio
We used to have that `make vcxproj` hack, but a hack it is. In the
meantime, we have a much cleaner solution: using CMake, either
explicitly, or even more conveniently via Visual Studio's built-in CMake
support (simply open Git's top-level directory via File>Open>Folder...).

Let's let the `README` reflect this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:56 -04:00
Johannes Schindelin
380985273b mingw: allow for longer paths in parse_interpreter()
As reported in https://github.com/newren/git-filter-repo/pull/225, it
looks like 99 bytes is not really sufficient to represent e.g. the full
path to Python when installed via Windows Store (and this path is used
in the hasb bang line when installing scripts via `pip`).

Let's increase it to what is probably the maximum sensible path size:
MAX_PATH. This makes `parse_interpreter()` in line with what
`lookup_prog()` handles.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Vilius Šumskas <vilius@sumskas.eu>
2021-11-03 20:19:56 -04:00
Dennis Ameling
c3b96f59f3 Add schannel to curl installation
Signed-off-by: Dennis Ameling <dennis@dennisameling.com>
2021-11-03 20:19:55 -04:00
Philip Oakley
60c77d4a37 vcpkg_install: add comment regarding slow network connections
The vcpkg downloads may not succeed. Warn careful readers of the time out.

A simple retry will usually resolve the issue.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:55 -04:00
Philip Oakley
8c8d756158 vcpkg_install: detect lack of Git
The vcpkg_install batch file depends on the availability of a
working Git on the CMD path. This may not be present if the user
has selected the 'bash only' option during Git-for-Windows install.

Detect and tell the user about their lack of a working Git in the CMD
window.

Fixes #2348.
A separate PR https://github.com/git-for-windows/build-extra/pull/258
now highlights the recommended path setting during install.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
2021-11-03 20:19:55 -04:00
Johannes Schindelin
d716a43ddd win32: add a helper to run git.exe without a foreground window
On Windows, there are two kinds of executables, console ones and
non-console ones. Git's executables are all console ones.

When launching the former e.g. in a scheduled task, a CMD window pops
up. This is not what we want for the tasks installed via the `git
maintenance` command.

To work around this, let's introduce `headless-git.exe`, which is a
non-console program that does _not_ pop up any window. All it does is to
re-launch `git.exe`, suppressing that console window, passing through
all command-line arguments as-are.

Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Helped-by: Yuyi Wang <Strawberry_Str@hotmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
2021-11-03 20:19:55 -04:00
Ian Bearman
f4a1c27a31 vcbuild: add an option to install individual 'features'
In this context, a "feature" is a dependency combined with its own
dependencies.

Signed-off-by: Dennis Ameling <dennis@dennisameling.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:55 -04:00
Ian Bearman
e1e1ae5922 vcbuild: install ARM64 dependencies when building ARM64 binaries
Co-authored-by: Dennis Ameling <dennis@dennisameling.com>
Signed-off-by: Ian Bearman <ianb@microsoft.com>
Signed-off-by: Dennis Ameling <dennis@dennisameling.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:55 -04:00
Jeff Hostetler
9472b3532b clink.pl: move default linker options for MSVC=1 builds
Move the default `-ENTRY` and `-SUBSYSTEM` arguments for
MSVC=1 builds from `config.mak.uname` into `clink.pl`.
These args are constant for console-mode executables.

Add support to `clink.pl` for generating a Win32 GUI application
using the `-mwindows` argument (to match how GCC does it).  This
changes the `-ENTRY` and `-SUBSYSTEM` arguments accordingly.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:55 -04:00
Jeff Hostetler
37b1d84fac clink.pl: ignore no-stack-protector arg on MSVC=1 builds
Ignore the `-fno-stack-protector` compiler argument when building
with MSVC.  This will be used in a later commit that needs to build
a Win32 GUI app.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:55 -04:00
Jeff Hostetler
dd4feabfd2 vcbuild: add support for compiling Windows resource files
Create a wrapper for the Windows Resource Compiler (RC.EXE)
for use by the MSVC=1 builds. This is similar to the CL.EXE
and LIB.EXE wrappers used for the MSVC=1 builds.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:55 -04:00
Jeff Hostetler
3937729754 clink.pl: fix libexpatd.lib link error when using MSVC
When building with `make MSVC=1 DEBUG=1`, link to `libexpatd.lib`
rather than `libexpat.lib`.

It appears that the `vcpkg` package for "libexpat" has changed and now
creates `libexpatd.lib` for debug mode builds.  Previously, both debug
and release builds created a ".lib" with the same basename.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-11-03 20:19:55 -04:00
Johannes Schindelin
8c3dcf1f33 mingw: ignore HOMEDRIVE/HOMEPATH if it points to Windows' system directory
Internally, Git expects the environment variable `HOME` to be set, and
to point to the current user's home directory.

This environment variable is not set by default on Windows, and
therefore Git tries its best to construct one if it finds `HOME` unset.

There are actually two different approaches Git tries: first, it looks
at `HOMEDRIVE`/`HOMEPATH` because this is widely used in corporate
environments with roaming profiles, and a user generally wants their
global Git settings to be in a roaming profile.

Only when `HOMEDRIVE`/`HOMEPATH` is either unset or does not point to a
valid location, Git will fall back to using `USERPROFILE` instead.

However, starting with Windows Vista, for secondary logons and services,
the environment variables `HOMEDRIVE`/`HOMEPATH` point to Windows'
system directory (usually `C:\Windows\system32`).

That is undesirable, and that location is usually write-protected anyway.

So let's verify that the `HOMEDRIVE`/`HOMEPATH` combo does not point to
Windows' system directory before using it, falling back to `USERPROFILE`
if it does.

This fixes git-for-windows#2709

Initial-Path-by: Ivan Pozdeev <vano@mail.mipt.ru>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:54 -04:00
Johannes Schindelin
47958acdb0 mingw: implement a platform-specific strbuf_realpath()
There is a Win32 API function to resolve symbolic links, and we can use
that instead of resolving them manually. Even better, this function also
resolves NTFS junction points (which are somewhat similar to bind
mounts).

This fixes https://github.com/git-for-windows/git/issues/2481.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:54 -04:00
Jeff Hostetler
1c5f4c028a clink.pl: fix MSVC compile script to handle libcurl-d.lib
Update clink.pl to link with either libcurl.lib or libcurl-d.lib
depending on whether DEBUG=1 is set.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-11-03 20:19:54 -04:00