Commit Graph

120829 Commits

Author SHA1 Message Date
Karsten Blees
b452e96d1c mingw: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow
`lstat()` emulation (git calls `lstat()` once for each file in the
index). Windows operating system APIs seem to be much better at scanning
the status of entire directories than checking single files.

Add an `lstat()` implementation that uses a cache for lstat data. Cache
misses read the entire parent directory and add it to the cache.
Subsequent `lstat()` calls for the same directory are served directly
from the cache.

Also implement `opendir()`/`readdir()`/`closedir()` so that they create
and use directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:01:55 +02:00
Karsten Blees
26c1ed5e08 add infrastructure for read-only file system level caches
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.

This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.

Enable read-only sections for 'git status' and preload_index.

Signed-off-by: Karsten Blees <blees@dcon.de>
2021-05-28 10:01:55 +02:00
Karsten Blees
75c6ec2692 Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:01:55 +02:00
Karsten Blees
97684de098 mingw: make the dirent implementation pluggable
Emulating the POSIX `dirent` API on Windows via
`FindFirstFile()`/`FindNextFile()` is pretty staightforward, however,
most of the information provided in the `WIN32_FIND_DATA` structure is
thrown away in the process. A more sophisticated implementation may
cache this data, e.g. for later reuse in calls to `lstat()`.

Make the `dirent` implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to `readdir()`/`closedir()`
that match the `opendir()` implementation (similar to vtable pointers in
Object-Oriented Programming). Define `readdir()`/`closedir()` so that
they call the function pointers in the `DIR` structure. This allows to
choose the `opendir()` implementation on a call-by-call basis.

Make the fixed-size `dirent.d_name` buffer a flex array, as `d_name` may
be implementation specific (e.g. a caching implementation may allocate a
`struct dirent` with _just_ the size needed to hold the `d_name` in
question).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:01:55 +02:00
Karsten Blees
547a429de4 Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2021-05-28 10:01:55 +02:00
Karsten Blees
5e7f638bc3 Win32: make FILETIME conversion functions public
We will use them in the upcoming "FSCache" patches (to accelerate
sequential lstat() calls).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:01:54 +02:00
Johannes Schindelin
d7f251dae9 Merge branch 'ready-for-upstream'
This is the branch thicket of patches in Git for Windows that are
considered ready for upstream. To keep them in a ready-to-submit shape,
they are kept as close to the beginning of the branch thicket as
possible.
2021-05-28 10:01:54 +02:00
Johannes Schindelin
f096c8903e Merge pull request #3220 from dscho/there-is-no-vs/master-anymore
Let the documentation reflect that there is no vs/master anymore
2021-05-28 10:01:54 +02:00
Johannes Schindelin
d415a84c29 Merge pull request #3165 from dscho/increase-allowed-length-of-interpreter-path
mingw: allow for longer paths in `parse_interpreter()`
2021-05-28 10:01:53 +02:00
Johannes Schindelin
ef31b99fe1 Merge pull request #3150 from dscho/ci-cache-vcpkg-artifacts-g4w
ci: cache vcpkg artifacts
2021-05-28 10:01:53 +02:00
Johannes Schindelin
271ee963f8 Merge pull request #3236 from dscho/no-fsmonitor-in-bare-repos
fsmonitor: only enable it in non-bare repositories
2021-05-28 10:01:53 +02:00
Johannes Schindelin
0ef58afed2 Merge pull request #3234 from dscho/fsmonitor-v2
This branch thicket adds a built-in, Git-aware FSMonitor.
2021-05-28 10:00:48 +02:00
Johannes Schindelin
267ec61d5a Merge pull request #3076 from dennisameling/git-credential-manager-core-arm64
git-artifacts: add workaround for GCM Core on ARM64
2021-05-28 10:00:48 +02:00
Johannes Schindelin
2840de370a Merge pull request #3053 from dscho/assorted-git-artifacts-fixes
Assorted fixes/enhancements for the `git-artifacts` workflow
2021-05-28 10:00:48 +02:00
Johannes Schindelin
1d363ddf9c Merge pull request #3056 from dscho/cmake-and-skip-dashed-built-ins
Teach our CMake configuration to optionally skip hard-linking the dashed built-ins
2021-05-28 10:00:47 +02:00
Johannes Schindelin
bfe746d39c Merge pull request #3017 from dennisameling/add-arm64-artifacts
git-artifacts: add ARM64 artifacts
2021-05-28 10:00:47 +02:00
Johannes Schindelin
bb890a97bd Merge pull request #2586 from dscho/build-git-artifacts
Add a GitHub workflow to generate Git for Windows' installer, portable Git, MinGit, etc
2021-05-28 10:00:47 +02:00
Johannes Schindelin
66b5733b63 Merge pull request #2915 from dennisameling/windows-arm64-support
Windows arm64 support
2021-05-28 10:00:46 +02:00
Johannes Schindelin
ab7ded8390 Merge pull request #2351 from PhilipOakley/vcpkg-tip
Vcpkg Install: detect lack of working Git, and note possible vcpkg time outs
2021-05-28 10:00:46 +02:00
Johannes Schindelin
0638096ac0 Merge pull request #3040 from dennisameling/vs-build-no-gettext
.github: build vs-build with NO_GETTEXT
2021-05-28 10:00:46 +02:00
Johannes Schindelin
1124394ac2 Merge pull request #2974 from derrickstolee/maintenance-and-headless
Include Windows-specific maintenance and headless-git
2021-05-28 10:00:45 +02:00
Johannes Schindelin
f4710481d9 Merge 'git-gui/js/intent-to-add'
This merges the current version of the patch that tries to address Git
GUI's problems with intent-to-add files.

This patch will likely be improved substantially before it is merged
into Git GUI's main branch, but we want to have _something_ resembling a
fix already in Git for Windows v2.29.0.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:45 +02:00
Johannes Schindelin
51b6103d0a Merge pull request #2655 from jglathe/jg/t0014_trace_extra_info
t/t0014: fix: eliminate additional lines from trace
2021-05-28 10:00:44 +02:00
Johannes Schindelin
b1f6c194ae Merge pull request #2714 from lbonanomi/main
Rationalize line endings for scissors-cleanup
2021-05-28 10:00:44 +02:00
Johannes Schindelin
771467523c Merge pull request #2730 from dscho/crlf-aware-git-add-i
git add -i: handle CR/LF line endings in the interactive input
2021-05-28 10:00:44 +02:00
Johannes Schindelin
d6856b4238 Merge 'add-p-many-files'
This topic branch allows `add -p` and `add -i` with a large number of
files. It is kind of a hack that was never really meant to be
upstreamed. Let's see if we can do better in the built-in `add -p`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:43 +02:00
Johannes Schindelin
d635677268 Merge pull request #2618 from dscho/avoid-d/f-conflict-in-vs/master
ci: avoid d/f conflict in vs/master
2021-05-28 10:00:43 +02:00
Johannes Schindelin
60438828ca Merge pull request #2535 from dscho/schannel-revoke-best-effort
Introduce and use the new "best effort" strategy for Secure Channel revoke checking
2021-05-28 10:00:43 +02:00
Johannes Schindelin
d9c09ed180 Merge pull request #2506 from dscho/issue-2283
Allow running Git directly from `C:\Program Files\Git\mingw64\bin\git.exe`
2021-05-28 10:00:42 +02:00
Johannes Schindelin
11b42042a0 Merge pull request #2504 from dscho/access-repo-via-junction
Handle `git add <file>` where <file> traverses an NTFS junction
2021-05-28 10:00:42 +02:00
Johannes Schindelin
0cb906088c Merge pull request #2501 from jeffhostetler/clink-debug-curl
clink.pl: fix MSVC compile script to handle libcurl-d.lib
2021-05-28 10:00:42 +02:00
Johannes Schindelin
a173d41852 Merge pull request #2488 from bmueller84/master
mingw: fix fatal error working on mapped network drives on Windows
2021-05-28 10:00:41 +02:00
Johannes Schindelin
09d0d8d54c Merge pull request #2449 from dscho/mingw-getcwd-and-symlinks
Do resolve symlinks in `getcwd()`
2021-05-28 10:00:41 +02:00
Johannes Schindelin
cd461c7701 Merge pull request #2405 from dscho/mingw-setsockopt
Make sure `errno` is set when socket operations fail
2021-05-28 10:00:41 +02:00
Johannes Schindelin
ac92f06e1f Merge pull request #2375 from assarbad/reintroduce-sideband-config
Config option to disable side-band-64k for transport
2021-05-28 10:00:40 +02:00
Johannes Schindelin
3400ec8294 Merge branch 'msys2-python'
In MSYS2, we have two Python interpreters at our disposal, so we can
include the Python stuff in the build.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:40 +02:00
Johannes Schindelin
32dddd9481 Merge branch 'move-gfw-system-config-to-top-level'
Make `git config --system` work like you think it should on Windows: it
should edit the file that is located in `<Git>\etc\gitconfig`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:40 +02:00
Johannes Schindelin
a8a95eb67e Merge pull request #2316 from carenas/win-pcre1-cleanup
config.mak.uname: PCRE1 cleanup
2021-05-28 10:00:39 +02:00
Johannes Schindelin
8cbdaf9dbc Merge branch 'dont-clean-junctions'
This topic branch teaches `git clean` to respect NTFS junctions and Unix
bind mounts: it will now stop at those boundaries.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:39 +02:00
Johannes Schindelin
de2fc44f02 Merge pull request #2170 from dscho/gitk-long-cmdline
Fix gitk (long cmdline)
2021-05-28 10:00:39 +02:00
Johannes Schindelin
47caccd6ef Merge branch 'fsync-object-files-always'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:38 +02:00
Johannes Schindelin
1d77d98b50 Merge branch 'dont-spawn-gzip-in-archive'
This topic branch avoids spawning `gzip` when asking `git archive` to
create `.tar.gz` files.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:38 +02:00
Johannes Schindelin
c53b7ae5e5 Merge branch 'mingw-expand-absolute-user-path'
When compiling Git with a runtime prefix (so that it can be installed
into any location, finding its libexec/ directory relative to the
location of the `git` executable), it is convenient to provide
"absolute" Unix-y paths e.g. for http.sslCAInfo, and have those absolute
paths be resolved relative to the runtime prefix.

This patch makes it so for Windows. It is up for discussion whether we
want this for other platforms, too, as long as building with
RUNTIME_PREFIX.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:38 +02:00
Johannes Schindelin
d94714c152 compat/vcbuild: document preferred way to build in Visual Studio
We used to have that `make vcxproj` hack, but a hack it is. In the
meantime, we have a much cleaner solution: using CMake, either
explicitly, or even more conveniently via Visual Studio's built-in CMake
support (simply open Git's top-level directory via File>Open>Folder...).

Let's let the `README` reflect this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:37 +02:00
Johannes Schindelin
fe79801e10 mingw: allow for longer paths in parse_interpreter()
As reported in https://github.com/newren/git-filter-repo/pull/225, it
looks like 99 bytes is not really sufficient to represent e.g. the full
path to Python when installed via Windows Store (and this path is used
in the hasb bang line when installing scripts via `pip`).

Let's increase it to what is probably the maximum sensible path size:
MAX_PATH. This makes `parse_interpreter()` in line with what
`lookup_prog()` handles.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Vilius Šumskas <vilius@sumskas.eu>
2021-05-28 10:00:37 +02:00
Johannes Schindelin
9474385faa Merge branch 'drive-prefix'
This topic branch allows us to specify absolute paths without the drive
prefix e.g. when cloning.

Example:

	C:\Users\me> git clone https://github.com/git/git \upstream-git

This will clone into a new directory C:\upstream-git, in line with how
Windows interprets absolute paths.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:37 +02:00
Johannes Schindelin
817375c1ca Merge 'remote-hg-prerequisites' into HEAD
These fixes were necessary for Sverre Rabbelier's remote-hg to work,
but for some magic reason they are not necessary for the current
remote-hg. Makes you wonder how that one gets away with it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:37 +02:00
Johannes Schindelin
3a8ad7b10b ci(vs-build): download the vcpkg artifacts using a dedicated Action
We now have a GitHub Action to download and cache Azure Pipelines
artifacts (such as the `vcpkg` artifacts), hiding gnarly internals, and
also providing some robustness against network glitches. Let's use it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:36 +02:00
Johannes Schindelin
06755c32b5 fsmonitor: only enable it in non-bare repositories
The entire point of the FSMonitor is to monitor the worktree changes in
a more efficient manner than `lstat()`ing all worktree files every time
we refresh the index.

But if there is no worktree, FSMonitor has nothing to monitor.

So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
and we're running in a repository without worktree.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:36 +02:00
Johannes Schindelin
004f7e68b2 Enable the built-in FSMonitor as an experimental feature
If `feature.experimental` and `feature.manyFiles` are set, we now start
the built-in FSMonitor by default.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-05-28 10:00:36 +02:00