Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.
This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.
Enable read-only sections for 'git status' and preload_index.
Signed-off-by: Karsten Blees <blees@dcon.de>
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.
Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Emulating the POSIX `dirent` API on Windows via
`FindFirstFile()`/`FindNextFile()` is pretty staightforward, however,
most of the information provided in the `WIN32_FIND_DATA` structure is
thrown away in the process. A more sophisticated implementation may
cache this data, e.g. for later reuse in calls to `lstat()`.
Make the `dirent` implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.
Define a base DIR structure with pointers to `readdir()`/`closedir()`
that match the `opendir()` implementation (similar to vtable pointers in
Object-Oriented Programming). Define `readdir()`/`closedir()` so that
they call the function pointers in the `DIR` structure. This allows to
choose the `opendir()` implementation on a call-by-call basis.
Make the fixed-size `dirent.d_name` buffer a flex array, as `d_name` may
be implementation specific (e.g. a caching implementation may allocate a
`struct dirent` with _just_ the size needed to hold the `d_name` in
question).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We will use them in the upcoming "FSCache" patches (to accelerate
sequential lstat() calls).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is the branch thicket of patches in Git for Windows that are
considered ready for upstream. To keep them in a ready-to-submit shape,
they are kept as close to the beginning of the branch thicket as
possible.
This merges the current version of the patch that tries to address Git
GUI's problems with intent-to-add files.
This patch will likely be improved substantially before it is merged
into Git GUI's main branch, but we want to have _something_ resembling a
fix already in Git for Windows v2.29.0.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch allows `add -p` and `add -i` with a large number of
files. It is kind of a hack that was never really meant to be
upstreamed. Let's see if we can do better in the built-in `add -p`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In MSYS2, we have two Python interpreters at our disposal, so we can
include the Python stuff in the build.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch teaches `git clean` to respect NTFS junctions and Unix
bind mounts: it will now stop at those boundaries.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch avoids spawning `gzip` when asking `git archive` to
create `.tar.gz` files.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When compiling Git with a runtime prefix (so that it can be installed
into any location, finding its libexec/ directory relative to the
location of the `git` executable), it is convenient to provide
"absolute" Unix-y paths e.g. for http.sslCAInfo, and have those absolute
paths be resolved relative to the runtime prefix.
This patch makes it so for Windows. It is up for discussion whether we
want this for other platforms, too, as long as building with
RUNTIME_PREFIX.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch allows us to specify absolute paths without the drive
prefix e.g. when cloning.
Example:
C:\Users\me> git clone https://github.com/git/git \upstream-git
This will clone into a new directory C:\upstream-git, in line with how
Windows interprets absolute paths.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We used to have that `make vcxproj` hack, but a hack it is. In the
meantime, we have a much cleaner solution: using CMake, either
explicitly, or even more conveniently via Visual Studio's built-in CMake
support (simply open Git's top-level directory via File>Open>Folder...).
Let's let the `README` reflect this.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
These fixes were necessary for Sverre Rabbelier's remote-hg to work,
but for some magic reason they are not necessary for the current
remote-hg. Makes you wonder how that one gets away with it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
As reported in https://github.com/newren/git-filter-repo/pull/225, it
looks like 99 bytes is not really sufficient to represent e.g. the full
path to Python when installed via Windows Store (and this path is used
in the hasb bang line when installing scripts via `pip`).
Let's increase it to what is probably the maximum sensible path size:
MAX_PATH. This makes `parse_interpreter()` in line with what
`lookup_prog()` handles.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Vilius Šumskas <vilius@sumskas.eu>
We now have a GitHub Action to download and cache Azure Pipelines
artifacts (such as the `vcpkg` artifacts), hiding gnarly internals, and
also providing some robustness against network glitches. Let's use it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The entire point of the FSMonitor is to monitor the worktree changes in
a more efficient manner than `lstat()`ing all worktree files every time
we refresh the index.
But if there is no worktree, FSMonitor has nothing to monitor.
So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
and we're running in a repository without worktree.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
If `feature.experimental` and `feature.manyFiles` are set, we now start
the built-in FSMonitor by default.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Builtin FSMonitor Feature
Here is V2 of my patch series to add a builtin FSMonitor daemon to Git.
V2 includes addresses most of the review comments from the RFC and many of
the issues seen during out beta-testing with selected GVFS users. However,
there are still a few items that I need to address:
[ ] Revisit the how the client handles the IPC_STATE__NOT_LISTENING state
(where a daemon appears to be running, but is non-responsive) [ ] Revisit
use of global core_fsmonitor as both a pathname and a boolean. The existing
fsmonitor code uses it as the pathname to the fsmonitor hook and as a flag
to indicate that a hook is configured. [ ] Consider having daemon chdir()
out of the working directory to avoid directory handle issues on Windows. [
] Some documentation recommendations. [ ] Split up the commit containing the
tests and move some earlier in patch series. [ ] Move my FSMonitor PREREQ to
test-lib.sh instead of having it in my test scripts. [ ] Document
performance gains. [ ] On Windows, If the daemon is started as an elevated
process, then client commands might not have access to communicate with it.
[ ] Review if/how we decide to shutdown the FSMonitor daemon after and a
significant idle period. [ ] Investigate ways to temporarily shutdown
FSMonitor daemons processes so that the Git for Windows installer can
install an upgrade.
In this version, the first commit updates the Simple IPC API to make it
easier to pass binary data using {char *, size_t} rather than assuming that
the message is a null-terminated string. FSMonitor does not use binary
messages and doesn't really need this API change, but I thought it best to
fix the API now before we have other callers of IPC.
This V2 version will hopefully be previewed as an experimental feature in
Git for Windows v2.32.0.windows.*.
Jeff Hostetler (26):
simple-ipc: preparations for supporting binary messages.
fsmonitor--daemon: man page
fsmonitor--daemon: update fsmonitor documentation
fsmonitor-ipc: create client routines for git-fsmonitor--daemon
help: include fsmonitor--daemon feature flag in version info
fsmonitor--daemon: add a built-in fsmonitor daemon
fsmonitor--daemon: implement client command options
t/helper/fsmonitor-client: create IPC client to talk to FSMonitor
Daemon
fsmonitor-fs-listen-win32: stub in backend for Windows
fsmonitor-fs-listen-macos: stub in backend for MacOS
fsmonitor--daemon: implement daemon command options
fsmonitor--daemon: add pathname classification
fsmonitor--daemon: define token-ids
fsmonitor--daemon: create token-based changed path cache
fsmonitor-fs-listen-win32: implement FSMonitor backend on Windows
fsmonitor-fs-listen-macos: add macos header files for FSEvent
fsmonitor-fs-listen-macos: implement FSEvent listener on MacOS
fsmonitor--daemon: implement handle_client callback
fsmonitor--daemon: periodically truncate list of modified files
fsmonitor--daemon: use a cookie file to sync with file system
fsmonitor: enhance existing comments
fsmonitor: force update index after large responses
t7527: create test for fsmonitor--daemon
p7519: add fsmonitor--daemon
t7527: test status with untracked-cache and fsmonitor--daemon
t/perf: avoid copying builtin fsmonitor files into test repo
Johannes Schindelin (2):
config: FSMonitor is repository-specific
fsmonitor: introduce `core.useBuiltinFSMonitor` to call the daemon via
IPC
.gitignore | 1 +
Documentation/config/core.txt | 56 +-
Documentation/git-fsmonitor--daemon.txt | 75 +
Documentation/git-update-index.txt | 27 +-
Documentation/githooks.txt | 3 +-
Makefile | 16 +
builtin.h | 1 +
builtin/fsmonitor--daemon.c | 1511 ++++++++++++++++++
builtin/update-index.c | 4 +-
compat/fsmonitor/fsmonitor-fs-listen-macos.c | 497 ++++++
compat/fsmonitor/fsmonitor-fs-listen-win32.c | 553 +++++++
compat/fsmonitor/fsmonitor-fs-listen.h | 49 +
compat/simple-ipc/ipc-unix-socket.c | 14 +-
compat/simple-ipc/ipc-win32.c | 14 +-
config.c | 9 +-
config.h | 2 +-
config.mak.uname | 4 +
contrib/buildsystems/CMakeLists.txt | 8 +
fsmonitor--daemon.h | 140 ++
fsmonitor-ipc.c | 179 +++
fsmonitor-ipc.h | 48 +
fsmonitor.c | 132 +-
git.c | 1 +
help.c | 4 +
repo-settings.c | 3 +
repository.h | 2 +
simple-ipc.h | 7 +-
t/helper/test-fsmonitor-client.c | 125 ++
t/helper/test-simple-ipc.c | 34 +-
t/helper/test-tool.c | 1 +
t/helper/test-tool.h | 1 +
t/perf/p7519-fsmonitor.sh | 42 +-
t/perf/perf-lib.sh | 2 +-
t/t7527-builtin-fsmonitor.sh | 572 +++++++
34 files changed, 4069 insertions(+), 68 deletions(-)
create mode 100644 Documentation/git-fsmonitor--daemon.txt
create mode 100644 builtin/fsmonitor--daemon.c
create mode 100644 compat/fsmonitor/fsmonitor-fs-listen-macos.c
create mode 100644 compat/fsmonitor/fsmonitor-fs-listen-win32.c
create mode 100644 compat/fsmonitor/fsmonitor-fs-listen.h
create mode 100644 fsmonitor--daemon.h
create mode 100644 fsmonitor-ipc.c
create mode 100644 fsmonitor-ipc.h
create mode 100644 t/helper/test-fsmonitor-client.c
create mode 100755 t/t7527-builtin-fsmonitor.sh
base-commit: b0c09ab879
Submitted-As: https://lore.kernel.org/git/pull.923.v2.git.1621691828.gitgitgadget@gmail.com
In-Reply-To: https://lore.kernel.org/git/pull.923.git.1617291666.gitgitgadget@gmail.com
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Do not try to copy a fsmonitor--daemon socket from the current
development directory into the test trash directory.
When we run the perf suite without an explicit source repo set,
we copy of the current $GIT_DIR into the test trash directory.
Unix domain sockets cannot be copied in that manner, so the test
setup fails.
Additionally, omit any other fsmonitor--daemon temp files inside
the $GIT_DIR directory.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Create 2x2 test matrix with the untracked-cache and fsmonitor--daemon
features and a series of edits and verify that status output is
identical.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
