git-for-windows/git
1
0
Fork 0
mirror of https://github.com/git-for-windows/git.git synced 2026-03-26 22:24:27 -05:00
Code Issues Packages Projects Releases 335 Wiki Activity

Merge branch 'builtin-fsmonitor' (preview of V4)

Browse Source 
Left side is alternate version of v2.33.0-rc0.windows.1 with
the previous V2 version of FSMonitor removed.
This commit is contained in:
Jeff Hostetler
2021-08-05 13:15:15 -04:00
committed by Victoria Dye
parent 822bebd773 73067d5df3
commit ad72b8d18b
45 changed files with 5148 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@@ -72,6 +72,7 @@
/git-format-patch
/git-fsck
/git-fsck-objects
/git-fsmonitor--daemon
/git-gc
/git-get-tar-commit-id
/git-grep

56
Documentation/config/core.txt
View File

@@ -62,22 +62,50 @@ core.protectNTFS::
Defaults to `true` on Windows, and `false` elsewhere.
core.fsmonitor::
If set, the value of this variable is used as a command which
will identify all files that may have changed since the
requested date/time. This information is used to speed up git by
avoiding unnecessary processing of files that have not changed.
See the "fsmonitor-watchman" section of linkgit:githooks[5].
If set, this variable contains the pathname of the "fsmonitor"
hook command.
+
This hook command is used to identify all files that may have changed
since the requested date/time. This information is used to speed up
git by avoiding unnecessary scanning of files that have not changed.
+
See the "fsmonitor-watchman" section of linkgit:githooks[5].
+
Note: The value of this config setting is ignored if the
built-in file system monitor is enabled (see `core.useBuiltinFSMonitor`).
core.fsmonitorHookVersion::
Sets the version of hook that is to be used when calling fsmonitor.
There are currently versions 1 and 2. When this is not set,
version 2 will be tried first and if it fails then version 1
will be tried. Version 1 uses a timestamp as input to determine
which files have changes since that time but some monitors
like watchman have race conditions when used with a timestamp.
Version 2 uses an opaque string so that the monitor can return
something that can be used to determine what files have changed
without race conditions.
Sets the protocol version to be used when invoking the
"fsmonitor" hook.
+
There are currently versions 1 and 2. When this is not set,
version 2 will be tried first and if it fails then version 1
will be tried. Version 1 uses a timestamp as input to determine
which files have changes since that time but some monitors
like Watchman have race conditions when used with a timestamp.
Version 2 uses an opaque string so that the monitor can return
something that can be used to determine what files have changed
without race conditions.
+
Note: The value of this config setting is ignored if the
built-in file system monitor is enabled (see `core.useBuiltinFSMonitor`).
core.useBuiltinFSMonitor::
If set to true, enable the built-in file system monitor
daemon for this working directory (linkgit:git-fsmonitor--daemon[1]).
+
Like hook-based file system monitors, the built-in file system monitor
can speed up Git commands that need to refresh the Git index
(e.g. `git status`) in a working directory with many files. The
built-in monitor eliminates the need to install and maintain an
external third-party tool.
+
The built-in file system monitor is currently available only on a
limited set of supported platforms. Currently, this includes Windows
and MacOS.
+
Note: if this config setting is set to `true`, the values of
`core.fsmonitor` and `core.fsmonitorHookVersion` are ignored.
core.trustctime::
If false, the ctime differences between the index and the

75
Documentation/git-fsmonitor--daemon.txt Normal file
View File

@@ -0,0 +1,75 @@
git-fsmonitor--daemon(1)
========================
NAME
----
git-fsmonitor--daemon - A Built-in File System Monitor
SYNOPSIS
--------
[verse]
'git fsmonitor--daemon' start
'git fsmonitor--daemon' run
'git fsmonitor--daemon' stop
'git fsmonitor--daemon' status
DESCRIPTION
-----------
A daemon to watch the working directory for file and directory
changes using platform-specific file system notification facilities.
This daemon communicates directly with commands like `git status`
using the link:technical/api-simple-ipc.html[simple IPC] interface
instead of the slower linkgit:githooks[5] interface.
This daemon is built into Git so that no third-party tools are
required.
OPTIONS
-------
start::
Starts a daemon in the background.
run::
Runs a daemon in the foreground.
stop::
Stops the daemon running in the current working
directory, if present.
status::
Exits with zero status if a daemon is watching the
current working directory.
REMARKS
-------
This daemon is a long running process used to watch a single working
directory and maintain a list of the recently changed files and
directories. Performance of commands such as `git status` can be
increased if they just ask for a summary of changes to the working
directory and can avoid scanning the disk.
When `core.useBuiltinFSMonitor` is set to `true` (see
linkgit:git-config[1]) commands, such as `git status`, will ask the
daemon for changes and automatically start it (if necessary).
For more information see the "File System Monitor" section in
linkgit:git-update-index[1].
CAVEATS
-------
The fsmonitor daemon does not currently know about submodules and does
not know to filter out file system events that happen within a
submodule. If fsmonitor daemon is watching a super repo and a file is
modified within the working directory of a submodule, it will report
the change (as happening against the super repo). However, the client
will properly ignore these extra events, so performance may be affected
but it will not cause an incorrect result.
GIT
---
Part of the linkgit:git[1] suite

25
Documentation/git-update-index.txt
View File

@@ -498,7 +498,9 @@ FILE SYSTEM MONITOR
This feature is intended to speed up git operations for repos that have
large working directories.
It enables git to work together with a file system monitor (see the
It enables git to work together with a file system monitor (see
linkgit:git-fsmonitor--daemon[1]
and the
"fsmonitor-watchman" section of linkgit:githooks[5]) that can
inform it as to what files have been modified. This enables git to avoid
having to lstat() every file to find modified files.
@@ -508,17 +510,18 @@ performance by avoiding the cost of scanning the entire working directory
looking for new files.
If you want to enable (or disable) this feature, it is easier to use
the `core.fsmonitor` configuration variable (see
linkgit:git-config[1]) than using the `--fsmonitor` option to
`git update-index` in each repository, especially if you want to do so
across all repositories you use, because you can set the configuration
variable in your `$HOME/.gitconfig` just once and have it affect all
repositories you touch.
the `core.fsmonitor` or `core.useBuiltinFSMonitor` configuration
variable (see linkgit:git-config[1]) than using the `--fsmonitor`
option to `git update-index` in each repository, especially if you
want to do so across all repositories you use, because you can set the
configuration variable in your `$HOME/.gitconfig` just once and have
it affect all repositories you touch.
When the `core.fsmonitor` configuration variable is changed, the
file system monitor is added to or removed from the index the next time
a command reads the index. When `--[no-]fsmonitor` are used, the file
system monitor is immediately added to or removed from the index.
When the `core.fsmonitor` or `core.useBuiltinFSMonitor` configuration
variable is changed, the file system monitor is added to or removed
from the index the next time a command reads the index. When
`--[no-]fsmonitor` are used, the file system monitor is immediately
added to or removed from the index.
CONFIGURATION
-------------

3
Documentation/githooks.txt
View File

@@ -593,7 +593,8 @@ fsmonitor-watchman
This hook is invoked when the configuration option `core.fsmonitor` is
set to `.git/hooks/fsmonitor-watchman` or `.git/hooks/fsmonitor-watchmanv2`
depending on the version of the hook to use.
depending on the version of the hook to use, unless overridden via
`core.useBuiltinFSMonitor` (see linkgit:git-config[1]).
Version 1 takes two arguments, a version (1) and the time in elapsed
nanoseconds since midnight, January 1, 1970.

30
Makefile
View File

@@ -471,6 +471,16 @@ all::
# directory, and the JSON compilation database 'compile_commands.json' will be
# created at the root of the repository.
#
# If your platform supports a built-in fsmonitor backend, set
# FSMONITOR_DAEMON_BACKEND to the "<name>" of the corresponding
# `compat/fsmonitor/fsm-listen-<name>.c` that implements the
# `fsm_listen__*()` routines.
#
# If your platform has os-specific ways to tell if a repo is incompatible with
# fsmonitor (whether the hook or ipc daemon version), set FSMONITOR_OS_SETTINGS
# to the "<name>" of the corresponding `compat/fsmonitor/fsm-settings-<name>.c`
# that implements the `fsm_os_settings__*()` routines.
#
# Define DEVELOPER to enable more compiler warnings. Compiler version
# and family are auto detected, but could be overridden by defining
# COMPILER_FEATURES (see config.mak.dev). You can still set
@@ -713,6 +723,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o
TEST_BUILTINS_OBJS += test-dump-untracked-cache.o
TEST_BUILTINS_OBJS += test-example-decorate.o
TEST_BUILTINS_OBJS += test-fast-rebase.o
TEST_BUILTINS_OBJS += test-fsmonitor-client.o
TEST_BUILTINS_OBJS += test-genrandom.o
TEST_BUILTINS_OBJS += test-genzeros.o
TEST_BUILTINS_OBJS += test-getcwd.o
@@ -901,6 +912,8 @@ LIB_OBJS += fetch-pack.o
LIB_OBJS += fmt-merge-msg.o
LIB_OBJS += fsck.o
LIB_OBJS += fsmonitor.o
LIB_OBJS += fsmonitor-ipc.o
LIB_OBJS += fsmonitor-settings.o
LIB_OBJS += gettext.o
LIB_OBJS += gpg-interface.o
LIB_OBJS += graph.o
@@ -1105,6 +1118,7 @@ BUILTIN_OBJS += builtin/fmt-merge-msg.o
BUILTIN_OBJS += builtin/for-each-ref.o
BUILTIN_OBJS += builtin/for-each-repo.o
BUILTIN_OBJS += builtin/fsck.o
BUILTIN_OBJS += builtin/fsmonitor--daemon.o
BUILTIN_OBJS += builtin/gc.o
BUILTIN_OBJS += builtin/get-tar-commit-id.o
BUILTIN_OBJS += builtin/grep.o
@@ -1935,6 +1949,16 @@ ifdef NEED_ACCESS_ROOT_HANDLER
COMPAT_OBJS += compat/access.o
endif
ifdef FSMONITOR_DAEMON_BACKEND
COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
endif
ifdef FSMONITOR_OS_SETTINGS
COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS
COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o
endif
ifeq ($(TCLTK_PATH),)
NO_TCLTK = NoThanks
endif
@@ -2816,6 +2840,12 @@ GIT-BUILD-OPTIONS: FORCE
@echo PAGER_ENV=\''$(subst ','\'',$(subst ','\'',$(PAGER_ENV)))'\' >>$@+
@echo DC_SHA1=\''$(subst ','\'',$(subst ','\'',$(DC_SHA1)))'\' >>$@+
@echo X=\'$(X)\' >>$@+
ifdef FSMONITOR_DAEMON_BACKEND
@echo FSMONITOR_DAEMON_BACKEND=\''$(subst ','\'',$(subst ','\'',$(FSMONITOR_DAEMON_BACKEND)))'\' >>$@+
endif
ifdef FSMONITOR_OS_SETTINGS
@echo FSMONITOR_OS_SETTINGS=\''$(subst ','\'',$(subst ','\'',$(FSMONITOR_OS_SETTINGS)))'\' >>$@+
endif
ifdef TEST_OUTPUT_DIRECTORY
@echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst ','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+
endif

1
builtin.h
View File

@@ -159,6 +159,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix);
int cmd_for_each_repo(int argc, const char **argv, const char *prefix);
int cmd_format_patch(int argc, const char **argv, const char *prefix);
int cmd_fsck(int argc, const char **argv, const char *prefix);
int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix);
int cmd_gc(int argc, const char **argv, const char *prefix);
int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
int cmd_grep(int argc, const char **argv, const char *prefix);

1593
builtin/fsmonitor--daemon.c Normal file
View File

File diff suppressed because it is too large Load Diff

27
builtin/update-index.c
View File

@@ -1216,14 +1216,33 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
}
if (fsmonitor > 0) {
if (git_config_get_fsmonitor() == 0)
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
if (fsm_mode == FSMONITOR_MODE_INCOMPATIBLE) {
struct strbuf buf_reason = STRBUF_INIT;
fsm_settings__get_reason(r, &buf_reason);
error("%s", buf_reason.buf);
strbuf_release(&buf_reason);
return -1;
}
if (fsm_mode == FSMONITOR_MODE_DISABLED) {
warning(_("core.useBuiltinFSMonitor is unset; "
"set it if you really want to enable the "
"builtin fsmonitor"));
warning(_("core.fsmonitor is unset; "
"set it if you really want to "
"enable fsmonitor"));
"set it if you really want to enable the "
"hook-based fsmonitor"));
}
add_fsmonitor(&the_index);
report(_("fsmonitor enabled"));
} else if (!fsmonitor) {
if (git_config_get_fsmonitor() == 1)
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
if (fsm_mode == FSMONITOR_MODE_IPC)
warning(_("core.useBuiltinFSMonitor is set; "
"remove it if you really want to "
"disable fsmonitor"));
if (fsm_mode == FSMONITOR_MODE_HOOK)
warning(_("core.fsmonitor is set; "
"remove it if you really want to "
"disable fsmonitor"));

1
cache.h
View File

@@ -990,7 +990,6 @@ extern int core_preload_index;
extern int precomposed_unicode;
extern int protect_hfs;
extern int protect_ntfs;
extern const char *core_fsmonitor;
extern int core_apply_sparse_checkout;
extern int core_sparse_checkout_cone;

497
compat/fsmonitor/fsm-listen-darwin.c Normal file
View File

@@ -0,0 +1,497 @@
#if defined(__GNUC__)
/*
* It is possible to #include CoreFoundation/CoreFoundation.h when compiling
* with clang, but not with GCC as of time of writing.
*
* See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93082 for details.
*/
typedef unsigned int FSEventStreamCreateFlags;
#define kFSEventStreamEventFlagNone 0x00000000
#define kFSEventStreamEventFlagMustScanSubDirs 0x00000001
#define kFSEventStreamEventFlagUserDropped 0x00000002
#define kFSEventStreamEventFlagKernelDropped 0x00000004
#define kFSEventStreamEventFlagEventIdsWrapped 0x00000008
#define kFSEventStreamEventFlagHistoryDone 0x00000010
#define kFSEventStreamEventFlagRootChanged 0x00000020
#define kFSEventStreamEventFlagMount 0x00000040
#define kFSEventStreamEventFlagUnmount 0x00000080
#define kFSEventStreamEventFlagItemCreated 0x00000100
#define kFSEventStreamEventFlagItemRemoved 0x00000200
#define kFSEventStreamEventFlagItemInodeMetaMod 0x00000400
#define kFSEventStreamEventFlagItemRenamed 0x00000800
#define kFSEventStreamEventFlagItemModified 0x00001000
#define kFSEventStreamEventFlagItemFinderInfoMod 0x00002000
#define kFSEventStreamEventFlagItemChangeOwner 0x00004000
#define kFSEventStreamEventFlagItemXattrMod 0x00008000
#define kFSEventStreamEventFlagItemIsFile 0x00010000
#define kFSEventStreamEventFlagItemIsDir 0x00020000
#define kFSEventStreamEventFlagItemIsSymlink 0x00040000
#define kFSEventStreamEventFlagOwnEvent 0x00080000
#define kFSEventStreamEventFlagItemIsHardlink 0x00100000
#define kFSEventStreamEventFlagItemIsLastHardlink 0x00200000
#define kFSEventStreamEventFlagItemCloned 0x00400000
typedef struct __FSEventStream *FSEventStreamRef;
typedef const FSEventStreamRef ConstFSEventStreamRef;
typedef unsigned int CFStringEncoding;
#define kCFStringEncodingUTF8 0x08000100
typedef const struct __CFString *CFStringRef;
typedef const struct __CFArray *CFArrayRef;
typedef const struct __CFRunLoop *CFRunLoopRef;
struct FSEventStreamContext {
long long version;
void *cb_data, *retain, *release, *copy_description;
};
typedef struct FSEventStreamContext FSEventStreamContext;
typedef unsigned int FSEventStreamEventFlags;
#define kFSEventStreamCreateFlagNoDefer 0x02
#define kFSEventStreamCreateFlagWatchRoot 0x04
#define kFSEventStreamCreateFlagFileEvents 0x10
typedef unsigned long long FSEventStreamEventId;
#define kFSEventStreamEventIdSinceNow 0xFFFFFFFFFFFFFFFFULL
typedef void (*FSEventStreamCallback)(ConstFSEventStreamRef streamRef,
void *context,
__SIZE_TYPE__ num_of_events,
void *event_paths,
const FSEventStreamEventFlags event_flags[],
const FSEventStreamEventId event_ids[]);
typedef double CFTimeInterval;
FSEventStreamRef FSEventStreamCreate(void *allocator,
FSEventStreamCallback callback,
FSEventStreamContext *context,
CFArrayRef paths_to_watch,
FSEventStreamEventId since_when,
CFTimeInterval latency,
FSEventStreamCreateFlags flags);
CFStringRef CFStringCreateWithCString(void *allocator, const char *string,
CFStringEncoding encoding);
CFArrayRef CFArrayCreate(void *allocator, const void **items, long long count,
void *callbacks);
void CFRunLoopRun(void);
void CFRunLoopStop(CFRunLoopRef run_loop);
CFRunLoopRef CFRunLoopGetCurrent(void);
extern CFStringRef kCFRunLoopDefaultMode;
void FSEventStreamScheduleWithRunLoop(FSEventStreamRef stream,
CFRunLoopRef run_loop,
CFStringRef run_loop_mode);
unsigned char FSEventStreamStart(FSEventStreamRef stream);
void FSEventStreamStop(FSEventStreamRef stream);
void FSEventStreamInvalidate(FSEventStreamRef stream);
void FSEventStreamRelease(FSEventStreamRef stream);
#else
/*
* Let Apple's headers declare `isalnum()` first, before
* Git's headers override it via a constant
*/
#include <string.h>
#include <CoreFoundation/CoreFoundation.h>
#include <CoreServices/CoreServices.h>
#endif
#include "cache.h"
#include "fsmonitor.h"
#include "fsm-listen.h"
#include "fsmonitor--daemon.h"
struct fsmonitor_daemon_backend_data
{
CFStringRef cfsr_worktree_path;
CFStringRef cfsr_gitdir_path;
CFArrayRef cfar_paths_to_watch;
int nr_paths_watching;
FSEventStreamRef stream;
CFRunLoopRef rl;
enum shutdown_style {
SHUTDOWN_EVENT = 0,
FORCE_SHUTDOWN,
FORCE_ERROR_STOP,
} shutdown_style;
unsigned int stream_scheduled:1;
unsigned int stream_started:1;
};
static void log_flags_set(const char *path, const FSEventStreamEventFlags flag)
{
struct strbuf msg = STRBUF_INIT;
if (flag & kFSEventStreamEventFlagMustScanSubDirs)
strbuf_addstr(&msg, "MustScanSubDirs|");
if (flag & kFSEventStreamEventFlagUserDropped)
strbuf_addstr(&msg, "UserDropped|");
if (flag & kFSEventStreamEventFlagKernelDropped)
strbuf_addstr(&msg, "KernelDropped|");
if (flag & kFSEventStreamEventFlagEventIdsWrapped)
strbuf_addstr(&msg, "EventIdsWrapped|");
if (flag & kFSEventStreamEventFlagHistoryDone)
strbuf_addstr(&msg, "HistoryDone|");
if (flag & kFSEventStreamEventFlagRootChanged)
strbuf_addstr(&msg, "RootChanged|");
if (flag & kFSEventStreamEventFlagMount)
strbuf_addstr(&msg, "Mount|");
if (flag & kFSEventStreamEventFlagUnmount)
strbuf_addstr(&msg, "Unmount|");
if (flag & kFSEventStreamEventFlagItemChangeOwner)
strbuf_addstr(&msg, "ItemChangeOwner|");
if (flag & kFSEventStreamEventFlagItemCreated)
strbuf_addstr(&msg, "ItemCreated|");
if (flag & kFSEventStreamEventFlagItemFinderInfoMod)
strbuf_addstr(&msg, "ItemFinderInfoMod|");
if (flag & kFSEventStreamEventFlagItemInodeMetaMod)
strbuf_addstr(&msg, "ItemInodeMetaMod|");
if (flag & kFSEventStreamEventFlagItemIsDir)
strbuf_addstr(&msg, "ItemIsDir|");
if (flag & kFSEventStreamEventFlagItemIsFile)
strbuf_addstr(&msg, "ItemIsFile|");
if (flag & kFSEventStreamEventFlagItemIsHardlink)
strbuf_addstr(&msg, "ItemIsHardlink|");
if (flag & kFSEventStreamEventFlagItemIsLastHardlink)
strbuf_addstr(&msg, "ItemIsLastHardlink|");
if (flag & kFSEventStreamEventFlagItemIsSymlink)
strbuf_addstr(&msg, "ItemIsSymlink|");
if (flag & kFSEventStreamEventFlagItemModified)
strbuf_addstr(&msg, "ItemModified|");
if (flag & kFSEventStreamEventFlagItemRemoved)
strbuf_addstr(&msg, "ItemRemoved|");
if (flag & kFSEventStreamEventFlagItemRenamed)
strbuf_addstr(&msg, "ItemRenamed|");
if (flag & kFSEventStreamEventFlagItemXattrMod)
strbuf_addstr(&msg, "ItemXattrMod|");
if (flag & kFSEventStreamEventFlagOwnEvent)
strbuf_addstr(&msg, "OwnEvent|");
if (flag & kFSEventStreamEventFlagItemCloned)
strbuf_addstr(&msg, "ItemCloned|");
trace_printf_key(&trace_fsmonitor, "fsevent: '%s', flags=%u %s",
path, flag, msg.buf);
strbuf_release(&msg);
}
static int ef_is_root_delete(const FSEventStreamEventFlags ef)
{
return (ef & kFSEventStreamEventFlagItemIsDir &&
ef & kFSEventStreamEventFlagItemRemoved);
}
static int ef_is_root_renamed(const FSEventStreamEventFlags ef)
{
return (ef & kFSEventStreamEventFlagItemIsDir &&
ef & kFSEventStreamEventFlagItemRenamed);
}
static int ef_is_dropped(const FSEventStreamEventFlags ef)
{
return (ef & kFSEventStreamEventFlagKernelDropped ||
ef & kFSEventStreamEventFlagUserDropped);
}
static void fsevent_callback(ConstFSEventStreamRef streamRef,
void *ctx,
size_t num_of_events,
void *event_paths,
const FSEventStreamEventFlags event_flags[],
const FSEventStreamEventId event_ids[])
{
struct fsmonitor_daemon_state *state = ctx;
struct fsmonitor_daemon_backend_data *data = state->backend_data;
char **paths = (char **)event_paths;
struct fsmonitor_batch *batch = NULL;
struct string_list cookie_list = STRING_LIST_INIT_DUP;
const char *path_k;
const char *slash;
int k;
struct strbuf tmp = STRBUF_INIT;
/*
* Build a list of all filesystem changes into a private/local
* list and without holding any locks.
*/
for (k = 0; k < num_of_events; k++) {
/*
* On Mac, we receive an array of absolute paths.
*/
path_k = paths[k];
/*
* If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
* Please don't log them to Trace2.
*
* trace_printf_key(&trace_fsmonitor, "Path: '%s'", path_k);
*/
/*
* If event[k] is marked as dropped, we assume that we have
* lost sync with the filesystem and should flush our cached
* data. We need to:
*
* [1] Abort/wake any client threads waiting for a cookie and
* flush the cached state data (the current token), and
* create a new token.
*
* [2] Discard the batch that we were locally building (since
* they are conceptually relative to the just flushed
* token).
*/
if (ef_is_dropped(event_flags[k])) {
/*
* see also kFSEventStreamEventFlagMustScanSubDirs
*/
trace_printf_key(&trace_fsmonitor, "event: dropped");
fsmonitor_force_resync(state);
fsmonitor_batch__pop(batch);
string_list_clear(&cookie_list, 0);
/*
* We assume that any events that we received
* in this callback after this dropped event
* may still be valid, so we continue rather
* than break. (And just in case there is a
* delete of ".git" hiding in there.)
*/
continue;
}
switch (fsmonitor_classify_path_absolute(state, path_k)) {
case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX:
case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
/* special case cookie files within .git or gitdir */
/* Use just the filename of the cookie file. */
slash = find_last_dir_sep(path_k);
string_list_append(&cookie_list,
slash ? slash + 1 : path_k);
break;
case IS_INSIDE_DOT_GIT:
case IS_INSIDE_GITDIR:
/* ignore all other paths inside of .git or gitdir */
break;
case IS_DOT_GIT:
case IS_GITDIR:
/*
* If .git directory is deleted or renamed away,
* we have to quit.
*/
if (ef_is_root_delete(event_flags[k])) {
trace_printf_key(&trace_fsmonitor,
"event: gitdir removed");
goto force_shutdown;
}
if (ef_is_root_renamed(event_flags[k])) {
trace_printf_key(&trace_fsmonitor,
"event: gitdir renamed");
goto force_shutdown;
}
break;
case IS_WORKDIR_PATH:
/* try to queue normal pathnames */
if (trace_pass_fl(&trace_fsmonitor))
log_flags_set(path_k, event_flags[k]);
/*
* Because of the implicit "binning" (the
* kernel calls us at a given frequency) and
* de-duping (the kernel is free to combine
* multiple events for a given pathname), an
* individual fsevent could be marked as both
* a file and directory. Add it to the queue
* with both spellings so that the client will
* know how much to invalidate/refresh.
*/
if (event_flags[k] & kFSEventStreamEventFlagItemIsFile) {
const char *rel = path_k +
state->path_worktree_watch.len + 1;
if (!batch)
batch = fsmonitor_batch__new();
fsmonitor_batch__add_path(batch, rel);
}
if (event_flags[k] & kFSEventStreamEventFlagItemIsDir) {
const char *rel = path_k +
state->path_worktree_watch.len + 1;
strbuf_reset(&tmp);
strbuf_addstr(&tmp, rel);
strbuf_addch(&tmp, '/');
if (!batch)
batch = fsmonitor_batch__new();
fsmonitor_batch__add_path(batch, tmp.buf);
}
break;
case IS_OUTSIDE_CONE:
default:
trace_printf_key(&trace_fsmonitor,
"ignoring '%s'", path_k);
break;
}
}
fsmonitor_publish(state, batch, &cookie_list);
string_list_clear(&cookie_list, 0);
strbuf_release(&tmp);
return;
force_shutdown:
fsmonitor_batch__pop(batch);
string_list_clear(&cookie_list, 0);
data->shutdown_style = FORCE_SHUTDOWN;
CFRunLoopStop(data->rl);
strbuf_release(&tmp);
return;
}
/*
* NEEDSWORK: Investigate the proper value for the `latency` argument
* in the call to `FSEventStreamCreate()`. I'm not sure that this
* needs to be a config setting or just something that we tune after
* some testing.
*
* With a latency of 0.1, I was seeing lots of dropped events during
* the "touch 100000" files test within t/perf/p7519, but with a
* latency of 0.001 I did not see any dropped events. So the
* "correct" value may be somewhere in between.
*
* https://developer.apple.com/documentation/coreservices/1443980-fseventstreamcreate
*/
int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
{
FSEventStreamCreateFlags flags = kFSEventStreamCreateFlagNoDefer |
kFSEventStreamCreateFlagWatchRoot |
kFSEventStreamCreateFlagFileEvents;
FSEventStreamContext ctx = {
0,
state,
NULL,
NULL,
NULL
};
struct fsmonitor_daemon_backend_data *data;
const void *dir_array[2];
CALLOC_ARRAY(data, 1);
state->backend_data = data;
data->cfsr_worktree_path = CFStringCreateWithCString(
NULL, state->path_worktree_watch.buf, kCFStringEncodingUTF8);
dir_array[data->nr_paths_watching++] = data->cfsr_worktree_path;
if (state->nr_paths_watching > 1) {
data->cfsr_gitdir_path = CFStringCreateWithCString(
NULL, state->path_gitdir_watch.buf,
kCFStringEncodingUTF8);
dir_array[data->nr_paths_watching++] = data->cfsr_gitdir_path;
}
data->cfar_paths_to_watch = CFArrayCreate(NULL, dir_array,
data->nr_paths_watching,
NULL);
data->stream = FSEventStreamCreate(NULL, fsevent_callback, &ctx,
data->cfar_paths_to_watch,
kFSEventStreamEventIdSinceNow,
0.001, flags);
if (data->stream == NULL)
goto failed;
/*
* `data->rl` needs to be set inside the listener thread.
*/
return 0;
failed:
error("Unable to create FSEventStream.");
FREE_AND_NULL(state->backend_data);
return -1;
}
void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data;
if (!state || !state->backend_data)
return;
data = state->backend_data;
if (data->stream) {
if (data->stream_started)
FSEventStreamStop(data->stream);
if (data->stream_scheduled)
FSEventStreamInvalidate(data->stream);
FSEventStreamRelease(data->stream);
}
FREE_AND_NULL(state->backend_data);
}
void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data;
data = state->backend_data;
data->shutdown_style = SHUTDOWN_EVENT;
CFRunLoopStop(data->rl);
}
void fsm_listen__loop(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data;
data = state->backend_data;
data->rl = CFRunLoopGetCurrent();
FSEventStreamScheduleWithRunLoop(data->stream, data->rl, kCFRunLoopDefaultMode);
data->stream_scheduled = 1;
if (!FSEventStreamStart(data->stream)) {
error("Failed to start the FSEventStream");
goto force_error_stop_without_loop;
}
data->stream_started = 1;
CFRunLoopRun();
switch (data->shutdown_style) {
case FORCE_ERROR_STOP:
state->error_code = -1;
/* fall thru */
case FORCE_SHUTDOWN:
ipc_server_stop_async(state->ipc_server_data);
/* fall thru */
case SHUTDOWN_EVENT:
default:
break;
}
return;
force_error_stop_without_loop:
state->error_code = -1;
ipc_server_stop_async(state->ipc_server_data);
return;
}

694
compat/fsmonitor/fsm-listen-win32.c Normal file
View File

@@ -0,0 +1,694 @@
#include "cache.h"
#include "config.h"
#include "fsmonitor.h"
#include "fsm-listen.h"
#include "fsmonitor--daemon.h"
/*
* The documentation of ReadDirectoryChangesW() states that the maximum
* buffer size is 64K when the monitored directory is remote.
*
* Larger buffers may be used when the monitored directory is local and
* will help us receive events faster from the kernel and avoid dropped
* events.
*
* So we try to use a very large buffer and silently fallback to 64K if
* we get an error.
*/
#define MAX_RDCW_BUF_FALLBACK (65536)
#define MAX_RDCW_BUF (65536 * 8)
struct one_watch
{
char buffer[MAX_RDCW_BUF];
DWORD buf_len;
DWORD count;
struct strbuf path;
HANDLE hDir;
HANDLE hEvent;
OVERLAPPED overlapped;
/*
* Is there an active ReadDirectoryChangesW() call pending. If so, we
* need to later call GetOverlappedResult() and possibly CancelIoEx().
*/
BOOL is_active;
};
struct fsmonitor_daemon_backend_data
{
struct one_watch *watch_worktree;
struct one_watch *watch_gitdir;
HANDLE hEventShutdown;
HANDLE hListener[3]; /* we don't own these handles */
#define LISTENER_SHUTDOWN 0
#define LISTENER_HAVE_DATA_WORKTREE 1
#define LISTENER_HAVE_DATA_GITDIR 2
int nr_listener_handles;
struct strbuf dot_git_shortname;
};
/*
* Convert the WCHAR path from the notification into UTF8 and
* then normalize it.
*/
static int normalize_path_in_utf8(FILE_NOTIFY_INFORMATION *info,
struct strbuf *normalized_path)
{
int reserve;
int len = 0;
strbuf_reset(normalized_path);
if (!info->FileNameLength)
goto normalize;
/*
* Pre-reserve enough space in the UTF8 buffer for
* each Unicode WCHAR character to be mapped into a
* sequence of 2 UTF8 characters. That should let us
* avoid ERROR_INSUFFICIENT_BUFFER 99.9+% of the time.
*/
reserve = info->FileNameLength + 1;
strbuf_grow(normalized_path, reserve);
for (;;) {
len = WideCharToMultiByte(CP_UTF8, 0, info->FileName,
info->FileNameLength / sizeof(WCHAR),
normalized_path->buf,
strbuf_avail(normalized_path) - 1,
NULL, NULL);
if (len > 0)
goto normalize;
if (GetLastError() != ERROR_INSUFFICIENT_BUFFER) {
error("[GLE %ld] could not convert path to UTF-8: '%.*ls'",
GetLastError(),
(int)(info->FileNameLength / sizeof(WCHAR)),
info->FileName);
return -1;
}
strbuf_grow(normalized_path,
strbuf_avail(normalized_path) + reserve);
}
normalize:
strbuf_setlen(normalized_path, len);
return strbuf_normalize_path(normalized_path);
}
void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
{
SetEvent(state->backend_data->hListener[LISTENER_SHUTDOWN]);
}
static struct one_watch *create_watch(struct fsmonitor_daemon_state *state,
const char *path)
{
struct one_watch *watch = NULL;
DWORD desired_access = FILE_LIST_DIRECTORY;
DWORD share_mode =
FILE_SHARE_WRITE | FILE_SHARE_READ | FILE_SHARE_DELETE;
HANDLE hDir;
wchar_t wpath[MAX_PATH];
if (xutftowcs_path(wpath, path) < 0) {
error(_("could not convert to wide characters: '%s'"), path);
return NULL;
}
hDir = CreateFileW(wpath,
desired_access, share_mode, NULL, OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED,
NULL);
if (hDir == INVALID_HANDLE_VALUE) {
error(_("[GLE %ld] could not watch '%s'"),
GetLastError(), path);
return NULL;
}
CALLOC_ARRAY(watch, 1);
watch->buf_len = sizeof(watch->buffer); /* assume full MAX_RDCW_BUF */
strbuf_init(&watch->path, 0);
strbuf_addstr(&watch->path, path);
watch->hDir = hDir;
watch->hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
return watch;
}
static void destroy_watch(struct one_watch *watch)
{
if (!watch)
return;
strbuf_release(&watch->path);
if (watch->hDir != INVALID_HANDLE_VALUE)
CloseHandle(watch->hDir);
if (watch->hEvent != INVALID_HANDLE_VALUE)
CloseHandle(watch->hEvent);
free(watch);
}
static int start_rdcw_watch(struct fsmonitor_daemon_backend_data *data,
struct one_watch *watch)
{
DWORD dwNotifyFilter =
FILE_NOTIFY_CHANGE_FILE_NAME |
FILE_NOTIFY_CHANGE_DIR_NAME |
FILE_NOTIFY_CHANGE_ATTRIBUTES |
FILE_NOTIFY_CHANGE_SIZE |
FILE_NOTIFY_CHANGE_LAST_WRITE |
FILE_NOTIFY_CHANGE_CREATION;
ResetEvent(watch->hEvent);
memset(&watch->overlapped, 0, sizeof(watch->overlapped));
watch->overlapped.hEvent = watch->hEvent;
/*
* Queue an async call using Overlapped IO. This returns immediately.
* Our event handle will be signalled when the real result is available.
*
* The return value here just means that we successfully queued it.
* We won't know if the Read...() actually produces data until later.
*/
watch->is_active = ReadDirectoryChangesW(
watch->hDir, watch->buffer, watch->buf_len, TRUE,
dwNotifyFilter, &watch->count, &watch->overlapped, NULL);
if (watch->is_active)
return 0;
error("ReadDirectoryChangedW failed on '%s' [GLE %ld]",
watch->path.buf, GetLastError());
return -1;
}
static int recv_rdcw_watch(struct one_watch *watch)
{
DWORD gle;
watch->is_active = FALSE;
/*
* The overlapped result is ready. If the Read...() was successful
* we finally receive the actual result into our buffer.
*/
if (GetOverlappedResult(watch->hDir, &watch->overlapped, &watch->count,
TRUE))
return 0;
gle = GetLastError();
if (gle == ERROR_INVALID_PARAMETER &&
/*
* The kernel throws an invalid parameter error when our
* buffer is too big and we are pointed at a remote
* directory (and possibly for other reasons). Quietly
* set it down and try again.
*
* See note about MAX_RDCW_BUF at the top.
*/
watch->buf_len > MAX_RDCW_BUF_FALLBACK) {
watch->buf_len = MAX_RDCW_BUF_FALLBACK;
return -2;
}
/*
* NEEDSWORK: If an external <gitdir> is deleted, the above
* returns an error. I'm not sure that there's anything that
* we can do here other than failing -- the <worktree>/.git
* link file would be broken anyway. We might try to check
* for that and return a better error message, but I'm not
* sure it is worth it.
*/
error("GetOverlappedResult failed on '%s' [GLE %ld]",
watch->path.buf, gle);
return -1;
}
static void cancel_rdcw_watch(struct one_watch *watch)
{
DWORD count;
if (!watch || !watch->is_active)
return;
/*
* The calls to ReadDirectoryChangesW() and GetOverlappedResult()
* form a "pair" (my term) where we queue an IO and promise to
* hang around and wait for the kernel to give us the result.
*
* If for some reason after we queue the IO, we have to quit
* or otherwise not stick around for the second half, we must
* tell the kernel to abort the IO. This prevents the kernel
* from writing to our buffer and/or signalling our event
* after we free them.
*
* (Ask me how much fun it was to track that one down).
*/
CancelIoEx(watch->hDir, &watch->overlapped);
GetOverlappedResult(watch->hDir, &watch->overlapped, &count, TRUE);
watch->is_active = FALSE;
}
/*
* Process a single relative pathname event.
* Return 1 if we should shutdown.
*/
static int process_1_worktree_event(
FILE_NOTIFY_INFORMATION *info,
struct string_list *cookie_list,
struct fsmonitor_batch **batch,
const struct strbuf *path,
enum fsmonitor_path_type t)
{
const char *slash;
switch (t) {
case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX:
/* special case cookie files within .git */
/* Use just the filename of the cookie file. */
slash = find_last_dir_sep(path->buf);
string_list_append(cookie_list,
slash ? slash + 1 : path->buf);
break;
case IS_INSIDE_DOT_GIT:
/* ignore everything inside of "<worktree>/.git/" */
break;
case IS_DOT_GIT:
/* "<worktree>/.git" was deleted (or renamed away) */
if ((info->Action == FILE_ACTION_REMOVED) ||
(info->Action == FILE_ACTION_RENAMED_OLD_NAME)) {
trace2_data_string("fsmonitor", NULL,
"fsm-listen/dotgit",
"removed");
return 1;
}
break;
case IS_WORKDIR_PATH:
/* queue normal pathname */
if (!*batch)
*batch = fsmonitor_batch__new();
fsmonitor_batch__add_path(*batch, path->buf);
break;
case IS_GITDIR:
case IS_INSIDE_GITDIR:
case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
default:
BUG("unexpected path classification '%d' for '%s'",
t, path->buf);
}
return 0;
}
/*
* Process filesystem events that happen anywhere (recursively) under the
* <worktree> root directory. For a normal working directory, this includes
* both version controlled files and the contents of the .git/ directory.
*
* If <worktree>/.git is a file, then we only see events for the file
* itself.
*/
static int process_worktree_events(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data = state->backend_data;
struct one_watch *watch = data->watch_worktree;
struct strbuf path = STRBUF_INIT;
struct string_list cookie_list = STRING_LIST_INIT_DUP;
struct fsmonitor_batch *batch = NULL;
const char *p = watch->buffer;
/*
* If the kernel gets more events than will fit in the kernel
* buffer associated with our RDCW handle, it drops them and
* returns a count of zero.
*
* Yes, the call returns WITHOUT error and with length zero.
*
* (The "overflow" case is not ambiguous with the "no data" case
* because we did an INFINITE wait.)
*
* This means we have a gap in coverage. Tell the daemon layer
* to resync.
*/
if (!watch->count) {
trace2_data_string("fsmonitor", NULL, "fsm-listen/kernel",
"overflow");
fsmonitor_force_resync(state);
return LISTENER_HAVE_DATA_WORKTREE;
}
/*
* On Windows, `info` contains an "array" of paths that are
* relative to the root of whichever directory handle received
* the event.
*/
for (;;) {
FILE_NOTIFY_INFORMATION *info = (void *)p;
enum fsmonitor_path_type t;
strbuf_reset(&path);
if (normalize_path_in_utf8(info, &path) == -1)
goto skip_this_path;
t = fsmonitor_classify_path_workdir_relative(path.buf);
if (process_1_worktree_event(info, &cookie_list, &batch,
&path, t))
goto force_shutdown;
/*
* NEEDSWORK: If `path` contains a shortname (that is,
* if any component within it is a shortname), we
* should expand it to a longname (See
* `GetLongPathNameW()`) and re-normalize, classify,
* and process it because our client is probably
* expecting "normal" paths.
*
* HOWEVER, if our process has called `chdir()` to get
* us out of the root of the worktree (so that the
* root directory is not busy), then we have to be
* careful to convert the paths in the INFO array
* (which are relative to the directory of the RDCW
* watch and not the CWD) into absolute paths before
* calling GetLongPathNameW() and then convert the
* computed value back to a RDCW-relative pathname
* (which is what we and the client expect).
*
* FOR NOW, just handle case (1) exactly so that we
* shutdown properly when ".git" is deleted via the
* shortname alias.
*
* We might see case (2) events for cookie files, but
* we can ignore them.
*
* FOR LATER, handle case (3) where the worktree
* events contain shortnames. We should convert
* them to longnames to avoid confusing the client.
*/
if (data->dot_git_shortname.len &&
!strcmp(path.buf, data->dot_git_shortname.buf) &&
process_1_worktree_event(info, &cookie_list, &batch,
&data->dot_git_shortname,
IS_DOT_GIT))
goto force_shutdown;
skip_this_path:
if (!info->NextEntryOffset)
break;
p += info->NextEntryOffset;
}
fsmonitor_publish(state, batch, &cookie_list);
batch = NULL;
string_list_clear(&cookie_list, 0);
strbuf_release(&path);
return LISTENER_HAVE_DATA_WORKTREE;
force_shutdown:
fsmonitor_batch__pop(batch);
string_list_clear(&cookie_list, 0);
strbuf_release(&path);
return LISTENER_SHUTDOWN;
}
/*
* Process filesystem events that happened anywhere (recursively) under the
* external <gitdir> (such as non-primary worktrees or submodules).
* We only care about cookie files that our client threads created here.
*
* Note that we DO NOT get filesystem events on the external <gitdir>
* itself (it is not inside something that we are watching). In particular,
* we do not get an event if the external <gitdir> is deleted.
*/
static int process_gitdir_events(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data = state->backend_data;
struct one_watch *watch = data->watch_gitdir;
struct strbuf path = STRBUF_INIT;
struct string_list cookie_list = STRING_LIST_INIT_DUP;
const char *p = watch->buffer;
if (!watch->count) {
trace2_data_string("fsmonitor", NULL, "fsm-listen/kernel",
"overflow");
fsmonitor_force_resync(state);
return LISTENER_HAVE_DATA_GITDIR;
}
for (;;) {
FILE_NOTIFY_INFORMATION *info = (void *)p;
const char *slash;
enum fsmonitor_path_type t;
strbuf_reset(&path);
if (normalize_path_in_utf8(info, &path) == -1)
goto skip_this_path;
t = fsmonitor_classify_path_gitdir_relative(path.buf);
switch (t) {
case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
/* special case cookie files within gitdir */
/* Use just the filename of the cookie file. */
slash = find_last_dir_sep(path.buf);
string_list_append(&cookie_list,
slash ? slash + 1 : path.buf);
break;
case IS_INSIDE_GITDIR:
goto skip_this_path;
default:
BUG("unexpected path classification '%d' for '%s'",
t, path.buf);
}
/*
* WRT shortnames, this external gitdir will not see
* case (1) nor case (3) events.
*
* We might see case (2) events for cookie files, but
* we can ignore them.
*/
skip_this_path:
if (!info->NextEntryOffset)
break;
p += info->NextEntryOffset;
}
fsmonitor_publish(state, NULL, &cookie_list);
string_list_clear(&cookie_list, 0);
strbuf_release(&path);
return LISTENER_HAVE_DATA_GITDIR;
}
void fsm_listen__loop(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data = state->backend_data;
DWORD dwWait;
int result;
state->error_code = 0;
if (start_rdcw_watch(data, data->watch_worktree) == -1)
goto force_error_stop;
if (data->watch_gitdir &&
start_rdcw_watch(data, data->watch_gitdir) == -1)
goto force_error_stop;
for (;;) {
dwWait = WaitForMultipleObjects(data->nr_listener_handles,
data->hListener,
FALSE, INFINITE);
if (dwWait == WAIT_OBJECT_0 + LISTENER_HAVE_DATA_WORKTREE) {
result = recv_rdcw_watch(data->watch_worktree);
if (result == -1) {
/* hard error */
goto force_error_stop;
}
if (result == -2) {
/* retryable error */
if (start_rdcw_watch(data, data->watch_worktree) == -1)
goto force_error_stop;
continue;
}
/* have data */
if (process_worktree_events(state) == LISTENER_SHUTDOWN)
goto force_shutdown;
if (start_rdcw_watch(data, data->watch_worktree) == -1)
goto force_error_stop;
continue;
}
if (dwWait == WAIT_OBJECT_0 + LISTENER_HAVE_DATA_GITDIR) {
result = recv_rdcw_watch(data->watch_gitdir);
if (result == -1) {
/* hard error */
goto force_error_stop;
}
if (result == -2) {
/* retryable error */
if (start_rdcw_watch(data, data->watch_gitdir) == -1)
goto force_error_stop;
continue;
}
/* have data */
if (process_gitdir_events(state) == LISTENER_SHUTDOWN)
goto force_shutdown;
if (start_rdcw_watch(data, data->watch_gitdir) == -1)
goto force_error_stop;
continue;
}
if (dwWait == WAIT_OBJECT_0 + LISTENER_SHUTDOWN)
goto clean_shutdown;
error(_("could not read directory changes [GLE %ld]"),
GetLastError());
goto force_error_stop;
}
force_error_stop:
state->error_code = -1;
force_shutdown:
/*
* Tell the IPC thead pool to stop (which completes the await
* in the main thread (which will also signal this thread (if
* we are still alive))).
*/
ipc_server_stop_async(state->ipc_server_data);
clean_shutdown:
cancel_rdcw_watch(data->watch_worktree);
cancel_rdcw_watch(data->watch_gitdir);
}
int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data;
char shortname[16]; /* a padded 8.3 buffer */
CALLOC_ARRAY(data, 1);
data->hEventShutdown = CreateEvent(NULL, TRUE, FALSE, NULL);
data->watch_worktree = create_watch(state,
state->path_worktree_watch.buf);
if (!data->watch_worktree)
goto failed;
if (state->nr_paths_watching > 1) {
data->watch_gitdir = create_watch(state,
state->path_gitdir_watch.buf);
if (!data->watch_gitdir)
goto failed;
}
data->hListener[LISTENER_SHUTDOWN] = data->hEventShutdown;
data->nr_listener_handles++;
data->hListener[LISTENER_HAVE_DATA_WORKTREE] =
data->watch_worktree->hEvent;
data->nr_listener_handles++;
if (data->watch_gitdir) {
data->hListener[LISTENER_HAVE_DATA_GITDIR] =
data->watch_gitdir->hEvent;
data->nr_listener_handles++;
}
/*
* NEEDSWORK: Properly handle 8.3 shortnames. RDCW events can
* contain a shortname (if another application uses a
* shortname in a system call). We care about aliasing and
* the use of shortnames for:
*
* (1) ".git",
* -- if an external process deletes ".git" using "GIT~1",
* we need to catch that and shutdown.
*
* (2) our cookie files,
* -- if an external process deletes one of our cookie
* files using a shortname, we will get a shortname
* event for it. However, we should have already
* gotten a longname event for it when we created the
* cookie, so we can safely discard the shortname
* events for cookie files.
*
* (3) the spelling of modified files that we report to clients.
* -- we need to report the longname to the client because
* that is what they are expecting. Presumably, the
* client is going to lookup the paths that we report
* in their index and untracked-cache, so we should
* normalize the data for them. (Technically, they
* could adapt, so we could relax this maybe.)
*
* FOR NOW, while our CWD is at the root of the worktree we
* can easily get the spelling of the shortname of ".git" (if
* the volume has shortnames enabled). For most worktrees
* this value will be "GIT~1", but we don't want to assume
* that.
*
* Capture this so that we can handle (1).
*
* We leave (3) for a future effort.
*/
strbuf_init(&data->dot_git_shortname, 0);
GetShortPathNameA(".git", shortname, sizeof(shortname));
if (!strcmp(".git", shortname))
trace_printf_key(&trace_fsmonitor, "No shortname for '.git'");
else {
trace_printf_key(&trace_fsmonitor,
"Shortname of '.git' is '%s'", shortname);
strbuf_addstr(&data->dot_git_shortname, shortname);
}
state->backend_data = data;
return 0;
failed:
CloseHandle(data->hEventShutdown);
destroy_watch(data->watch_worktree);
destroy_watch(data->watch_gitdir);
return -1;
}
void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
{
struct fsmonitor_daemon_backend_data *data;
if (!state || !state->backend_data)
return;
data = state->backend_data;
CloseHandle(data->hEventShutdown);
destroy_watch(data->watch_worktree);
destroy_watch(data->watch_gitdir);
FREE_AND_NULL(state->backend_data);
}

49
compat/fsmonitor/fsm-listen.h Normal file
View File

@@ -0,0 +1,49 @@
#ifndef FSM_LISTEN_H
#define FSM_LISTEN_H
/* This needs to be implemented by each backend */
#ifdef HAVE_FSMONITOR_DAEMON_BACKEND
struct fsmonitor_daemon_state;
/*
* Initialize platform-specific data for the fsmonitor listener thread.
* This will be called from the main thread PRIOR to staring the
* fsmonitor_fs_listener thread.
*
* Returns 0 if successful.
* Returns -1 otherwise.
*/
int fsm_listen__ctor(struct fsmonitor_daemon_state *state);
/*
* Cleanup platform-specific data for the fsmonitor listener thread.
* This will be called from the main thread AFTER joining the listener.
*/
void fsm_listen__dtor(struct fsmonitor_daemon_state *state);
/*
* The main body of the platform-specific event loop to watch for
* filesystem events. This will run in the fsmonitor_fs_listen thread.
*
* It should call `ipc_server_stop_async()` if the listener thread
* prematurely terminates (because of a filesystem error or if it
* detects that the .git directory has been deleted). (It should NOT
* do so if the listener thread receives a normal shutdown signal from
* the IPC layer.)
*
* It should set `state->error_code` to -1 if the daemon should exit
* with an error.
*/
void fsm_listen__loop(struct fsmonitor_daemon_state *state);
/*
* Gently request that the fsmonitor listener thread shutdown.
* It does not wait for it to stop. The caller should do a JOIN
* to wait for it.
*/
void fsm_listen__stop_async(struct fsmonitor_daemon_state *state);
#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
#endif /* FSM_LISTEN_H */

75
compat/fsmonitor/fsm-settings-darwin.c Normal file
View File

@@ -0,0 +1,75 @@
#include "cache.h"
#include "config.h"
#include "repository.h"
#include "fsmonitor-settings.h"
#include "fsmonitor.h"
#include <sys/param.h>
#include <sys/mount.h>
/*
* Remote working directories are problematic for FSMonitor.
*
* The underlying file system on the server machine and/or the remote
* mount type (NFS, SAMBA, etc.) dictates whether notification events
* are available at all to remote client machines.
*
* Kernel differences between the server and client machines also
* dictate the how (buffering, frequency, de-dup) the events are
* delivered to client machine processes.
*
* A client machine (such as a laptop) may choose to suspend/resume
* and it is unclear (without lots of testing) whether the watcher can
* resync after a resume. We might be able to treat this as a normal
* "events were dropped by the kernel" event and do our normal "flush
* and resync" --or-- we might need to close the existing (zombie?)
* notification fd and create a new one.
*
* In theory, the above issues need to be addressed whether we are
* using the Hook or IPC API.
*
* For the builtin FSMonitor, we create the Unix domain socket for the
* IPC in the .git directory. If the working directory is remote,
* then the socket will be created on the remote file system. This
* can fail if the remote file system does not support UDS file types
* (e.g. smbfs to a Windows server) or if the remote kernel does not
* allow a non-local process to bind() the socket. (These problems
* could be fixed by moving the UDS out of the .git directory and to a
* well-known local directory on the client machine, but care should
* be taken to ensure that $HOME is actually local and not a managed
* file share.)
*
* So (for now at least), mark remote working directories as
* incompatible.
*/
static enum fsmonitor_reason is_remote(struct repository *r)
{
struct statfs fs;
if (statfs(r->worktree, &fs) == -1) {
int saved_errno = errno;
trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
r->worktree, strerror(saved_errno));
errno = saved_errno;
return FSMONITOR_REASON_ZERO;
}
trace_printf_key(&trace_fsmonitor,
"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
if (!(fs.f_flags & MNT_LOCAL))
return FSMONITOR_REASON_REMOTE;
return FSMONITOR_REASON_ZERO;
}
enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
{
enum fsmonitor_reason reason;
reason = is_remote(r);
if (reason)
return reason;
return FSMONITOR_REASON_ZERO;
}

137
compat/fsmonitor/fsm-settings-win32.c Normal file
View File

@@ -0,0 +1,137 @@
#include "cache.h"
#include "config.h"
#include "repository.h"
#include "fsmonitor-settings.h"
#include "fsmonitor.h"
/*
* GVFS (aka VFS for Git) is incompatible with FSMonitor.
*
* Granted, core Git does not know anything about GVFS and we
* shouldn't make assumptions about a downstream feature, but users
* can install both versions. And this can lead to incorrect results
* from core Git commands. So, without bringing in any of the GVFS
* code, do a simple config test for a published config setting. (We
* do not look at the various *_TEST_* environment variables.)
*/
static enum fsmonitor_reason is_virtual(struct repository *r)
{
const char *const_str;
if (!repo_config_get_value(r, "core.virtualfilesystem", &const_str))
return FSMONITOR_REASON_VIRTUAL;
return FSMONITOR_REASON_ZERO;
}
/*
* Remote working directories are problematic for FSMonitor.
*
* The underlying file system on the server machine and/or the remote
* mount type dictates whether notification events are available at
* all to remote client machines.
*
* Kernel differences between the server and client machines also
* dictate the how (buffering, frequency, de-dup) the events are
* delivered to client machine processes.
*
* A client machine (such as a laptop) may choose to suspend/resume
* and it is unclear (without lots of testing) whether the watcher can
* resync after a resume. We might be able to treat this as a normal
* "events were dropped by the kernel" event and do our normal "flush
* and resync" --or-- we might need to close the existing (zombie?)
* notification fd and create a new one.
*
* In theory, the above issues need to be addressed whether we are
* using the Hook or IPC API.
*
* So (for now at least), mark remote working directories as
* incompatible.
*
* Notes for testing:
*
* (a) Windows allows a network share to be mapped to a drive letter.
* (This is the normal method to access it.)
*
* $ NET USE Z: \\server\share
* $ git -C Z:/repo status
*
* (b) Windows allows a network share to be referenced WITHOUT mapping
* it to drive letter.
*
* $ NET USE \\server\share\dir
* $ git -C //server/share/repo status
*
* (c) Windows allows "SUBST" to create a fake drive mapping to an
* arbitrary path (which may be remote)
*
* $ SUBST Q: Z:\repo
* $ git -C Q:/ status
*
* (d) Windows allows a directory symlink to be created on a local
* file system that points to a remote repo.
*
* $ mklink /d ./link //server/share/repo
* $ git -C ./link status
*/
static enum fsmonitor_reason is_remote(struct repository *r)
{
wchar_t wpath[MAX_PATH];
wchar_t wfullpath[MAX_PATH];
size_t wlen;
UINT driveType;
/*
* Do everything in wide chars because the drive letter might be
* a multi-byte sequence. See win32_has_dos_drive_prefix().
*/
if (xutftowcs_path(wpath, r->worktree) < 0)
return FSMONITOR_REASON_ZERO;
/*
* GetDriveTypeW() requires a final slash. We assume that the
* worktree pathname points to an actual directory.
*/
wlen = wcslen(wpath);
if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
wpath[wlen++] = L'\\';
wpath[wlen] = 0;
}
/*
* Normalize the path. If nothing else, this converts forward
* slashes to backslashes. This is essential to get GetDriveTypeW()
* correctly handle some UNC "\\server\share\..." paths.
*/
if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL))
return FSMONITOR_REASON_ZERO;
driveType = GetDriveTypeW(wfullpath);
trace_printf_key(&trace_fsmonitor,
"DriveType '%s' L'%S' (%u)",
r->worktree, wfullpath, driveType);
if (driveType == DRIVE_REMOTE) {
trace_printf_key(&trace_fsmonitor,
"is_remote('%s') true",
r->worktree);
return FSMONITOR_REASON_REMOTE;
}
return FSMONITOR_REASON_ZERO;
}
enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
{
enum fsmonitor_reason reason;
reason = is_virtual(r);
if (reason)
return reason;
reason = is_remote(r);
if (reason)
return reason;
return FSMONITOR_REASON_ZERO;
}

14
compat/simple-ipc/ipc-unix-socket.c
View File

@@ -168,7 +168,8 @@ void ipc_client_close_connection(struct ipc_client_connection *connection)
int ipc_client_send_command_to_connection(
struct ipc_client_connection *connection,
const char *message, struct strbuf *answer)
const char *message, size_t message_len,
struct strbuf *answer)
{
int ret = 0;
@@ -176,7 +177,7 @@ int ipc_client_send_command_to_connection(
trace2_region_enter("ipc-client", "send-command", NULL);
if (write_packetized_from_buf_no_flush(message, strlen(message),
if (write_packetized_from_buf_no_flush(message, message_len,
connection->fd) < 0 ||
packet_flush_gently(connection->fd) < 0) {
ret = error(_("could not send IPC command"));
@@ -197,7 +198,8 @@ done:
int ipc_client_send_command(const char *path,
const struct ipc_client_connect_options *options,
const char *message, struct strbuf *answer)
const char *message, size_t message_len,
struct strbuf *answer)
{
int ret = -1;
enum ipc_active_state state;
@@ -208,7 +210,9 @@ int ipc_client_send_command(const char *path,
if (state != IPC_STATE__LISTENING)
return ret;
ret = ipc_client_send_command_to_connection(connection, message, answer);
ret = ipc_client_send_command_to_connection(connection,
message, message_len,
answer);
ipc_client_close_connection(connection);
@@ -503,7 +507,7 @@ static int worker_thread__do_io(
if (ret >= 0) {
ret = worker_thread_data->server_data->application_cb(
worker_thread_data->server_data->application_data,
buf.buf, do_io_reply_callback, &reply_data);
buf.buf, buf.len, do_io_reply_callback, &reply_data);
packet_flush_gently(reply_data.fd);
}

36
compat/simple-ipc/ipc-win32.c
View File

@@ -49,6 +49,9 @@ static enum ipc_active_state get_active_state(wchar_t *pipe_path)
if (GetLastError() == ERROR_FILE_NOT_FOUND)
return IPC_STATE__PATH_NOT_FOUND;
trace2_data_intmax("ipc-debug", NULL, "getstate/waitpipe/gle",
(intmax_t)GetLastError());
return IPC_STATE__OTHER_ERROR;
}
@@ -112,6 +115,11 @@ static enum ipc_active_state connect_to_server(
if (GetLastError() == ERROR_SEM_TIMEOUT)
return IPC_STATE__NOT_LISTENING;
gle = GetLastError();
trace2_data_intmax("ipc-debug", NULL,
"connect/waitpipe/gle",
(intmax_t)gle);
return IPC_STATE__OTHER_ERROR;
}
@@ -133,17 +141,31 @@ static enum ipc_active_state connect_to_server(
break; /* try again */
default:
trace2_data_intmax("ipc-debug", NULL,
"connect/createfile/gle",
(intmax_t)gle);
return IPC_STATE__OTHER_ERROR;
}
}
if (!SetNamedPipeHandleState(hPipe, &mode, NULL, NULL)) {
gle = GetLastError();
trace2_data_intmax("ipc-debug", NULL,
"connect/setpipestate/gle",
(intmax_t)gle);
CloseHandle(hPipe);
return IPC_STATE__OTHER_ERROR;
}
*pfd = _open_osfhandle((intptr_t)hPipe, O_RDWR|O_BINARY);
if (*pfd < 0) {
gle = GetLastError();
trace2_data_intmax("ipc-debug", NULL,
"connect/openosfhandle/gle",
(intmax_t)gle);
CloseHandle(hPipe);
return IPC_STATE__OTHER_ERROR;
}
@@ -208,7 +230,8 @@ void ipc_client_close_connection(struct ipc_client_connection *connection)
int ipc_client_send_command_to_connection(
struct ipc_client_connection *connection,
const char *message, struct strbuf *answer)
const char *message, size_t message_len,
struct strbuf *answer)
{
int ret = 0;
@@ -216,7 +239,7 @@ int ipc_client_send_command_to_connection(
trace2_region_enter("ipc-client", "send-command", NULL);
if (write_packetized_from_buf_no_flush(message, strlen(message),
if (write_packetized_from_buf_no_flush(message, message_len,
connection->fd) < 0 ||
packet_flush_gently(connection->fd) < 0) {
ret = error(_("could not send IPC command"));
@@ -239,7 +262,8 @@ done:
int ipc_client_send_command(const char *path,
const struct ipc_client_connect_options *options,
const char *message, struct strbuf *response)
const char *message, size_t message_len,
struct strbuf *response)
{
int ret = -1;
enum ipc_active_state state;
@@ -250,7 +274,9 @@ int ipc_client_send_command(const char *path,
if (state != IPC_STATE__LISTENING)
return ret;
ret = ipc_client_send_command_to_connection(connection, message, response);
ret = ipc_client_send_command_to_connection(connection,
message, message_len,
response);
ipc_client_close_connection(connection);
@@ -458,7 +484,7 @@ static int do_io(struct ipc_server_thread_data *server_thread_data)
if (ret >= 0) {
ret = server_thread_data->server_data->application_cb(
server_thread_data->server_data->application_data,
buf.buf, do_io_reply_callback, &reply_data);
buf.buf, buf.len, do_io_reply_callback, &reply_data);
packet_flush_gently(reply_data.fd);

14
config.c
View File

@@ -2517,20 +2517,6 @@ int git_config_get_max_percent_split_change(void)
return -1; /* default value */
}
int git_config_get_fsmonitor(void)
{
if (git_config_get_pathname("core.fsmonitor", &core_fsmonitor))
core_fsmonitor = getenv("GIT_TEST_FSMONITOR");
if (core_fsmonitor && !*core_fsmonitor)
core_fsmonitor = NULL;
if (core_fsmonitor)
return 1;
return 0;
}
int git_config_get_index_threads(int *dest)
{
int is_bool, val;

1
config.h
View File

@@ -609,7 +609,6 @@ int git_config_get_index_threads(int *dest);
int git_config_get_untracked_cache(void);
int git_config_get_split_index(void);
int git_config_get_max_percent_split_change(void);
int git_config_get_fsmonitor(void);
/* This dies if the configured or default date is in the future */
int git_config_get_expiry(const char *key, const char **output);

7
config.mak.uname
View File

@@ -147,6 +147,9 @@ ifeq ($(uname_S),Darwin)
MSGFMT = /usr/local/opt/gettext/bin/msgfmt
endif
endif
FSMONITOR_DAEMON_BACKEND = darwin
FSMONITOR_OS_SETTINGS = darwin
BASIC_LDFLAGS += -framework CoreServices
endif
ifeq ($(uname_S),SunOS)
NEEDS_SOCKET = YesPlease
@@ -420,6 +423,8 @@ ifeq ($(uname_S),Windows)
# so we don't need this:
#
# SNPRINTF_RETURNS_BOGUS = YesPlease
FSMONITOR_DAEMON_BACKEND = win32
FSMONITOR_OS_SETTINGS = win32
NO_SVN_TESTS = YesPlease
RUNTIME_PREFIX = YesPlease
HAVE_WPGMPTR = YesWeDo
@@ -607,6 +612,8 @@ ifneq (,$(findstring MINGW,$(uname_S)))
NO_STRTOUMAX = YesPlease
NO_MKDTEMP = YesPlease
NO_SVN_TESTS = YesPlease
FSMONITOR_DAEMON_BACKEND = win32
FSMONITOR_OS_SETTINGS = win32
RUNTIME_PREFIX = YesPlease
HAVE_WPGMPTR = YesWeDo
NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease

16
contrib/buildsystems/CMakeLists.txt
View File

@@ -292,6 +292,22 @@ else()
endif()
endif()
if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin")
add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
endif()
if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin")
add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
endif()
set(EXE_EXTENSION ${CMAKE_EXECUTABLE_SUFFIX})
#header checks

1
environment.c
View File

@@ -84,7 +84,6 @@ int protect_hfs = PROTECT_HFS_DEFAULT;
#define PROTECT_NTFS_DEFAULT 1
#endif
int protect_ntfs = PROTECT_NTFS_DEFAULT;
const char *core_fsmonitor;
/*
* The character that begins a commented line in user-editable file

140
fsmonitor--daemon.h Normal file
View File

@@ -0,0 +1,140 @@
#ifndef FSMONITOR_DAEMON_H
#define FSMONITOR_DAEMON_H
#ifdef HAVE_FSMONITOR_DAEMON_BACKEND
#include "cache.h"
#include "dir.h"
#include "run-command.h"
#include "simple-ipc.h"
#include "thread-utils.h"
struct fsmonitor_batch;
struct fsmonitor_token_data;
/*
* Create a new batch of path(s). The returned batch is considered
* private and not linked into the fsmonitor daemon state. The caller
* should fill this batch with one or more paths and then publish it.
*/
struct fsmonitor_batch *fsmonitor_batch__new(void);
/*
* Free this batch and return the value of the batch->next field.
*/
struct fsmonitor_batch *fsmonitor_batch__pop(struct fsmonitor_batch *batch);
/*
* Add this path to this batch of modified files.
*
* The batch should be private and NOT (yet) linked into the fsmonitor
* daemon state and therefore not yet visible to worker threads and so
* no locking is required.
*/
void fsmonitor_batch__add_path(struct fsmonitor_batch *batch, const char *path);
struct fsmonitor_daemon_backend_data; /* opaque platform-specific data */
struct fsmonitor_daemon_state {
pthread_t listener_thread;
pthread_mutex_t main_lock;
struct strbuf path_worktree_watch;
struct strbuf path_gitdir_watch;
int nr_paths_watching;
struct fsmonitor_token_data *current_token_data;
struct strbuf path_cookie_prefix;
pthread_cond_t cookies_cond;
int cookie_seq;
struct hashmap cookies;
int error_code;
struct fsmonitor_daemon_backend_data *backend_data;
struct ipc_server_data *ipc_server_data;
};
/*
* Pathname classifications.
*
* The daemon classifies the pathnames that it receives from file
* system notification events into the following categories and uses
* that to decide whether clients are told about them. (And to watch
* for file system synchronization events.)
*
* The client should only care about paths within the working
* directory proper (inside the working directory and not ".git" nor
* inside of ".git/"). That is, the client has read the index and is
* asking for a list of any paths in the working directory that have
* been modified since the last token. The client does not care about
* file system changes within the .git directory (such as new loose
* objects or packfiles). So the client will only receive paths that
* are classified as IS_WORKDIR_PATH.
*
* The daemon uses the IS_DOT_GIT and IS_GITDIR internally to mean the
* exact ".git" directory or GITDIR. If the daemon receives a delete
* event for either of these directories, it will automatically
* shutdown, for example.
*
* Note that the daemon DOES NOT explicitly watch nor special case the
* ".git/index" file. The daemon does not read the index and does not
* have any internal index-relative state. The daemon only collects
* the set of modified paths within the working directory.
*/
enum fsmonitor_path_type {
IS_WORKDIR_PATH = 0,
IS_DOT_GIT,
IS_INSIDE_DOT_GIT,
IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX,
IS_GITDIR,
IS_INSIDE_GITDIR,
IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX,
IS_OUTSIDE_CONE,
};
/*
* Classify a pathname relative to the root of the working directory.
*/
enum fsmonitor_path_type fsmonitor_classify_path_workdir_relative(
const char *relative_path);
/*
* Classify a pathname relative to a <gitdir> that is external to the
* worktree directory.
*/
enum fsmonitor_path_type fsmonitor_classify_path_gitdir_relative(
const char *relative_path);
/*
* Classify an absolute pathname received from a filesystem event.
*/
enum fsmonitor_path_type fsmonitor_classify_path_absolute(
struct fsmonitor_daemon_state *state,
const char *path);
/*
* Prepend the this batch of path(s) onto the list of batches associated
* with the current token. This makes the batch visible to worker threads.
*
* The caller no longer owns the batch and must not free it.
*
* Wake up the client threads waiting on these cookies.
*/
void fsmonitor_publish(struct fsmonitor_daemon_state *state,
struct fsmonitor_batch *batch,
const struct string_list *cookie_names);
/*
* If the platform-specific layer loses sync with the filesystem,
* it should call this to invalidate cached data and abort waiting
* threads.
*/
void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
#endif /* FSMONITOR_DAEMON_H */

179
fsmonitor-ipc.c Normal file
View File

@@ -0,0 +1,179 @@
#include "cache.h"
#include "fsmonitor.h"
#include "simple-ipc.h"
#include "fsmonitor-ipc.h"
#include "run-command.h"
#include "strbuf.h"
#include "trace2.h"
#ifdef HAVE_FSMONITOR_DAEMON_BACKEND
int fsmonitor_ipc__is_supported(void)
{
return 1;
}
GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
enum ipc_active_state fsmonitor_ipc__get_state(void)
{
return ipc_get_active_state(fsmonitor_ipc__get_path());
}
static int spawn_daemon(void)
{
const char *args[] = { "fsmonitor--daemon", "start", NULL };
return run_command_v_opt_tr2(args, RUN_COMMAND_NO_STDIN | RUN_GIT_CMD,
"fsmonitor");
}
int fsmonitor_ipc__send_query(const char *since_token,
struct strbuf *answer)
{
int ret = -1;
int tried_to_spawn = 0;
enum ipc_active_state state = IPC_STATE__OTHER_ERROR;
struct ipc_client_connection *connection = NULL;
struct ipc_client_connect_options options
= IPC_CLIENT_CONNECT_OPTIONS_INIT;
const char *tok = since_token ? since_token : "";
size_t tok_len = since_token ? strlen(since_token) : 0;
options.wait_if_busy = 1;
options.wait_if_not_found = 0;
trace2_region_enter("fsm_client", "query", NULL);
trace2_data_string("fsm_client", NULL, "query/command", tok);
try_again:
state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
&connection);
switch (state) {
case IPC_STATE__LISTENING:
ret = ipc_client_send_command_to_connection(
connection, tok, tok_len, answer);
ipc_client_close_connection(connection);
trace2_data_intmax("fsm_client", NULL,
"query/response-length", answer->len);
if (fsmonitor_is_trivial_response(answer))
trace2_data_intmax("fsm_client", NULL,
"query/trivial-response", 1);
goto done;
case IPC_STATE__NOT_LISTENING:
ret = error(_("fsmonitor_ipc__send_query: daemon not available"));
goto done;
case IPC_STATE__PATH_NOT_FOUND:
if (tried_to_spawn)
goto done;
tried_to_spawn++;
if (spawn_daemon())
goto done;
/*
* Try again, but this time give the daemon a chance to
* actually create the pipe/socket.
*
* Granted, the daemon just started so it can't possibly have
* any FS cached yet, so we'll always get a trivial answer.
* BUT the answer should include a new token that can serve
* as the basis for subsequent requests.
*/
options.wait_if_not_found = 1;
goto try_again;
case IPC_STATE__INVALID_PATH:
ret = error(_("fsmonitor_ipc__send_query: invalid path '%s'"),
fsmonitor_ipc__get_path());
goto done;
case IPC_STATE__OTHER_ERROR:
default:
ret = error(_("fsmonitor_ipc__send_query: unspecified error on '%s'"),
fsmonitor_ipc__get_path());
goto done;
}
done:
trace2_region_leave("fsm_client", "query", NULL);
return ret;
}
int fsmonitor_ipc__send_command(const char *command,
struct strbuf *answer)
{
struct ipc_client_connection *connection = NULL;
struct ipc_client_connect_options options
= IPC_CLIENT_CONNECT_OPTIONS_INIT;
int ret;
enum ipc_active_state state;
const char *c = command ? command : "";
size_t c_len = command ? strlen(command) : 0;
strbuf_reset(answer);
options.wait_if_busy = 1;
options.wait_if_not_found = 0;
state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
&connection);
if (state != IPC_STATE__LISTENING) {
die("fsmonitor--daemon is not running");
return -1;
}
ret = ipc_client_send_command_to_connection(connection, c, c_len,
answer);
ipc_client_close_connection(connection);
if (ret == -1) {
die("could not send '%s' command to fsmonitor--daemon", c);
return -1;
}
return 0;
}
#else
/*
* A trivial implementation of the fsmonitor_ipc__ API for unsupported
* platforms.
*/
int fsmonitor_ipc__is_supported(void)
{
return 0;
}
const char *fsmonitor_ipc__get_path(void)
{
return NULL;
}
enum ipc_active_state fsmonitor_ipc__get_state(void)
{
return IPC_STATE__OTHER_ERROR;
}
int fsmonitor_ipc__send_query(const char *since_token,
struct strbuf *answer)
{
return -1;
}
int fsmonitor_ipc__send_command(const char *command,
struct strbuf *answer)
{
return -1;
}
#endif

48
fsmonitor-ipc.h Normal file
View File

@@ -0,0 +1,48 @@
#ifndef FSMONITOR_IPC_H
#define FSMONITOR_IPC_H
/*
* Returns true if built-in file system monitor daemon is defined
* for this platform.
*/
int fsmonitor_ipc__is_supported(void);
/*
* Returns the pathname to the IPC named pipe or Unix domain socket
* where a `git-fsmonitor--daemon` process will listen. This is a
* per-worktree value.
*
* Returns NULL if the daemon is not supported on this platform.
*/
const char *fsmonitor_ipc__get_path(void);
/*
* Try to determine whether there is a `git-fsmonitor--daemon` process
* listening on the IPC pipe/socket.
*/
enum ipc_active_state fsmonitor_ipc__get_state(void);
/*
* Connect to a `git-fsmonitor--daemon` process via simple-ipc
* and ask for the set of changed files since the given token.
*
* This DOES NOT use the hook interface.
*
* Spawn a daemon process in the background if necessary.
*
* Returns -1 on error; 0 on success.
*/
int fsmonitor_ipc__send_query(const char *since_token,
struct strbuf *answer);
/*
* Connect to a `git-fsmonitor--daemon` process via simple-ipc and
* send a command verb. If no daemon is available, we DO NOT try to
* start one.
*
* Returns -1 on error; 0 on success.
*/
int fsmonitor_ipc__send_command(const char *command,
struct strbuf *answer);
#endif /* FSMONITOR_IPC_H */

184
fsmonitor-settings.c Normal file
View File

@@ -0,0 +1,184 @@
#include "cache.h"
#include "config.h"
#include "repository.h"
#include "fsmonitor-settings.h"
/*
* We keep this structure defintion private and have getters
* for all fields so that we can lazy load it as needed.
*/
struct fsmonitor_settings {
enum fsmonitor_mode mode;
enum fsmonitor_reason reason;
char *hook_path;
};
static void set_incompatible(struct repository *r,
enum fsmonitor_reason reason)
{
struct fsmonitor_settings *s = r->settings.fsmonitor;
s->mode = FSMONITOR_MODE_INCOMPATIBLE;
s->reason = reason;
}
static int check_for_incompatible(struct repository *r)
{
if (!r->worktree) {
/*
* Bare repositories don't have a working directory and
* therefore have nothing to watch.
*/
set_incompatible(r, FSMONITOR_REASON_BARE);
return 1;
}
#ifdef HAVE_FSMONITOR_OS_SETTINGS
{
enum fsmonitor_reason reason;
reason = fsm_os__incompatible(r);
if (reason != FSMONITOR_REASON_ZERO) {
set_incompatible(r, reason);
return 1;
}
}
#endif
return 0;
}
static struct fsmonitor_settings *s_init(struct repository *r)
{
if (!r->settings.fsmonitor)
CALLOC_ARRAY(r->settings.fsmonitor, 1);
return r->settings.fsmonitor;
}
void fsm_settings__set_ipc(struct repository *r)
{
struct fsmonitor_settings *s = s_init(r);
if (check_for_incompatible(r))
return;
s->mode = FSMONITOR_MODE_IPC;
}
void fsm_settings__set_hook(struct repository *r, const char *path)
{
struct fsmonitor_settings *s = s_init(r);
if (check_for_incompatible(r))
return;
s->mode = FSMONITOR_MODE_HOOK;
s->hook_path = strdup(path);
}
void fsm_settings__set_disabled(struct repository *r)
{
struct fsmonitor_settings *s = s_init(r);
s->mode = FSMONITOR_MODE_DISABLED;
s->reason = FSMONITOR_REASON_ZERO;
FREE_AND_NULL(s->hook_path);
}
static int check_for_ipc(struct repository *r)
{
int value;
if (!repo_config_get_bool(r, "core.usebuiltinfsmonitor", &value) &&
value) {
fsm_settings__set_ipc(r);
return 1;
}
return 0;
}
static int check_for_hook(struct repository *r)
{
const char *const_str;
if (repo_config_get_pathname(r, "core.fsmonitor", &const_str))
const_str = getenv("GIT_TEST_FSMONITOR");
if (const_str && *const_str) {
fsm_settings__set_hook(r, const_str);
return 1;
}
return 0;
}
static void lookup_fsmonitor_settings(struct repository *r)
{
if (check_for_ipc(r))
return;
if (check_for_hook(r))
return;
fsm_settings__set_disabled(r);
}
enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
{
if (!r->settings.fsmonitor)
lookup_fsmonitor_settings(r);
return r->settings.fsmonitor->mode;
}
const char *fsm_settings__get_hook_path(struct repository *r)
{
if (!r->settings.fsmonitor)
lookup_fsmonitor_settings(r);
return r->settings.fsmonitor->hook_path;
}
static void create_reason_message(struct repository *r,
struct strbuf *buf_reason)
{
struct fsmonitor_settings *s = r->settings.fsmonitor;
switch (s->reason) {
case FSMONITOR_REASON_ZERO:
return;
case FSMONITOR_REASON_BARE:
strbuf_addstr(buf_reason,
_("bare repos are incompatible with fsmonitor"));
return;
case FSMONITOR_REASON_VIRTUAL:
strbuf_addstr(buf_reason,
_("virtual repos are incompatible with fsmonitor"));
return;
case FSMONITOR_REASON_REMOTE:
strbuf_addstr(buf_reason,
_("remote repos are incompatible with fsmonitor"));
return;
default:
BUG("Unhandled case in create_reason_message '%d'", s->reason);
}
}
enum fsmonitor_reason fsm_settings__get_reason(struct repository *r,
struct strbuf *buf_reason)
{
strbuf_reset(buf_reason);
if (!r->settings.fsmonitor)
lookup_fsmonitor_settings(r);
if (r->settings.fsmonitor->mode == FSMONITOR_MODE_INCOMPATIBLE)
create_reason_message(r, buf_reason);
return r->settings.fsmonitor->reason;
}

47
fsmonitor-settings.h Normal file
View File

@@ -0,0 +1,47 @@
#ifndef FSMONITOR_SETTINGS_H
#define FSMONITOR_SETTINGS_H
struct repository;
enum fsmonitor_mode {
FSMONITOR_MODE_INCOMPATIBLE = -1, /* see _reason */
FSMONITOR_MODE_DISABLED = 0,
FSMONITOR_MODE_HOOK = 1, /* core.fsmonitor */
FSMONITOR_MODE_IPC = 2, /* core.useBuiltinFSMonitor */
};
/*
* Incompatibility reasons.
*/
enum fsmonitor_reason {
FSMONITOR_REASON_ZERO = 0,
FSMONITOR_REASON_BARE = 1,
FSMONITOR_REASON_VIRTUAL = 2,
FSMONITOR_REASON_REMOTE = 3,
};
void fsm_settings__set_ipc(struct repository *r);
void fsm_settings__set_hook(struct repository *r, const char *path);
void fsm_settings__set_disabled(struct repository *r);
enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
const char *fsm_settings__get_hook_path(struct repository *r);
enum fsmonitor_reason fsm_settings__get_reason(struct repository *r,
struct strbuf *buf_reason);
struct fsmonitor_settings;
#ifdef HAVE_FSMONITOR_OS_SETTINGS
/*
* Ask platform-specific code whether the repository is incompatible
* with fsmonitor (both hook and ipc modes). For example, if the working
* directory is on a remote volume and mounted via a technology that does
* not support notification events.
*
* fsm_os__* routines should considered private to fsm_settings__
* routines.
*/
enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
#endif /* HAVE_FSMONITOR_OS_SETTINGS */
#endif /* FSMONITOR_SETTINGS_H */

180
fsmonitor.c
View File

@@ -3,6 +3,7 @@
#include "dir.h"
#include "ewah/ewok.h"
#include "fsmonitor.h"
#include "fsmonitor-ipc.h"
#include "run-command.h"
#include "strbuf.h"
@@ -148,15 +149,18 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate)
/*
* Call the query-fsmonitor hook passing the last update token of the saved results.
*/
static int query_fsmonitor(int version, const char *last_update, struct strbuf *query_result)
static int query_fsmonitor_hook(struct repository *r,
int version,
const char *last_update,
struct strbuf *query_result)
{
struct child_process cp = CHILD_PROCESS_INIT;
int result;
if (!core_fsmonitor)
if (fsm_settings__get_mode(r) != FSMONITOR_MODE_HOOK)
return -1;
strvec_push(&cp.args, core_fsmonitor);
strvec_push(&cp.args, fsm_settings__get_hook_path(r));
strvec_pushf(&cp.args, "%d", version);
strvec_pushf(&cp.args, "%s", last_update);
cp.use_shell = 1;
@@ -229,6 +233,45 @@ static void fsmonitor_refresh_callback(struct index_state *istate, char *name)
untracked_cache_invalidate_path(istate, name, 0);
}
/*
* The number of pathnames that we need to receive from FSMonitor
* before we force the index to be updated.
*
* Note that any pathname within the set of received paths MAY cause
* cache-entry or istate flag bits to be updated and thus cause the
* index to be updated on disk.
*
* However, the response may contain many paths (such as ignored
* paths) that will not update any flag bits. And thus not force the
* index to be updated. (This is fine and normal.) It also means
* that the token will not be updated in the FSMonitor index
* extension. So the next Git command will find the same token in the
* index, make the same token-relative request, and receive the same
* response (plus any newly changed paths). If this response is large
* (and continues to grow), performance could be impacted.
*
* For example, if the user runs a build and it writes 100K object
* files but doesn't modify any source files, the index would not need
* to be updated. The FSMonitor response (after the build and
* relative to a pre-build token) might be 5MB. Each subsequent Git
* command will receive that same 100K/5MB response until something
* causes the index to be updated. And `refresh_fsmonitor()` will
* have to iterate over those 100K paths each time.
*
* Performance could be improved if we optionally force update the
* index after a very large response and get an updated token into
* the FSMonitor index extension. This should allow subsequent
* commands to get smaller and more current responses.
*
* The value chosen here does not need to be precise. The index
* will be updated automatically the first time the user touches
* a tracked file and causes a command like `git status` to
* update an mtime to be updated and/or set a flag bit.
*
* NEEDSWORK: Does this need to be a config value?
*/
static int fsmonitor_force_update_threshold = 100;
void refresh_fsmonitor(struct index_state *istate)
{
struct strbuf query_result = STRBUF_INIT;
@@ -238,17 +281,57 @@ void refresh_fsmonitor(struct index_state *istate)
struct strbuf last_update_token = STRBUF_INIT;
char *buf;
unsigned int i;
struct repository *r = istate->repo ? istate->repo : the_repository;
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
if (!core_fsmonitor || istate->fsmonitor_has_run_once)
if (fsm_mode <= FSMONITOR_MODE_DISABLED ||
istate->fsmonitor_has_run_once)
return;
hook_version = fsmonitor_hook_version();
istate->fsmonitor_has_run_once = 1;
trace_printf_key(&trace_fsmonitor, "refresh fsmonitor");
if (fsm_mode == FSMONITOR_MODE_IPC) {
query_success = !fsmonitor_ipc__send_query(
istate->fsmonitor_last_update ?
istate->fsmonitor_last_update : "builtin:fake",
&query_result);
if (query_success) {
/*
* The response contains a series of nul terminated
* strings. The first is the new token.
*
* Use `char *buf` as an interlude to trick the CI
* static analysis to let us use `strbuf_addstr()`
* here (and only copy the token) rather than
* `strbuf_addbuf()`.
*/
buf = query_result.buf;
strbuf_addstr(&last_update_token, buf);
bol = last_update_token.len + 1;
} else {
/*
* The builtin daemon is not available on this
* platform -OR- we failed to get a response.
*
* Generate a fake token (rather than a V1
* timestamp) for the index extension. (If
* they switch back to the hook API, we don't
* want ambiguous state.)
*/
strbuf_addstr(&last_update_token, "builtin:fake");
}
goto apply_results;
}
assert(fsm_mode == FSMONITOR_MODE_HOOK);
hook_version = fsmonitor_hook_version();
/*
* This could be racy so save the date/time now and query_fsmonitor
* This could be racy so save the date/time now and query_fsmonitor_hook
* should be inclusive to ensure we don't miss potential changes.
*/
last_update = getnanotime();
@@ -256,13 +339,14 @@ void refresh_fsmonitor(struct index_state *istate)
strbuf_addf(&last_update_token, "%"PRIu64"", last_update);
/*
* If we have a last update token, call query_fsmonitor for the set of
* If we have a last update token, call query_fsmonitor_hook for the set of
* changes since that token, else assume everything is possibly dirty
* and check it all.
*/
if (istate->fsmonitor_last_update) {
if (hook_version == -1 || hook_version == HOOK_INTERFACE_VERSION2) {
query_success = !query_fsmonitor(HOOK_INTERFACE_VERSION2,
query_success = !query_fsmonitor_hook(
r, HOOK_INTERFACE_VERSION2,
istate->fsmonitor_last_update, &query_result);
if (query_success) {
@@ -292,37 +376,71 @@ void refresh_fsmonitor(struct index_state *istate)
}
if (hook_version == HOOK_INTERFACE_VERSION1) {
query_success = !query_fsmonitor(HOOK_INTERFACE_VERSION1,
query_success = !query_fsmonitor_hook(
r, HOOK_INTERFACE_VERSION1,
istate->fsmonitor_last_update, &query_result);
}
trace_performance_since(last_update, "fsmonitor process '%s'", core_fsmonitor);
trace_printf_key(&trace_fsmonitor, "fsmonitor process '%s' returned %s",
core_fsmonitor, query_success ? "success" : "failure");
trace_performance_since(last_update, "fsmonitor process '%s'",
fsm_settings__get_hook_path(r));
trace_printf_key(&trace_fsmonitor,
"fsmonitor process '%s' returned %s",
fsm_settings__get_hook_path(r),
query_success ? "success" : "failure");
}
/* a fsmonitor process can return '/' to indicate all entries are invalid */
apply_results:
/*
* The response from FSMonitor (excluding the header token) is
* either:
*
* [a] a (possibly empty) list of NUL delimited relative
* pathnames of changed paths. This list can contain
* files and directories. Directories have a trailing
* slash.
*
* [b] a single '/' to indicate the provider had no
* information and that we should consider everything
* invalid. We call this a trivial response.
*/
if (query_success && query_result.buf[bol] != '/') {
/* Mark all entries returned by the monitor as dirty */
/*
* Mark all pathnames returned by the monitor as dirty.
*
* This updates both the cache-entries and the untracked-cache.
*/
int count = 0;
buf = query_result.buf;
for (i = bol; i < query_result.len; i++) {
if (buf[i] != '\0')
continue;
fsmonitor_refresh_callback(istate, buf + bol);
bol = i + 1;
count++;
}
if (bol < query_result.len)
if (bol < query_result.len) {
fsmonitor_refresh_callback(istate, buf + bol);
count++;
}
/* Now mark the untracked cache for fsmonitor usage */
if (istate->untracked)
istate->untracked->use_fsmonitor = 1;
} else {
/* We only want to run the post index changed hook if we've actually changed entries, so keep track
* if we actually changed entries or not */
if (count > fsmonitor_force_update_threshold)
istate->cache_changed |= FSMONITOR_CHANGED;
} else {
/*
* We received a trivial response, so invalidate everything.
*
* We only want to run the post index changed hook if
* we've actually changed entries, so keep track if we
* actually changed entries or not.
*/
int is_cache_changed = 0;
/* Mark all entries invalid */
for (i = 0; i < istate->cache_nr; i++) {
if (istate->cache[i]->ce_flags & CE_FSMONITOR_VALID) {
is_cache_changed = 1;
@@ -330,7 +448,10 @@ void refresh_fsmonitor(struct index_state *istate)
}
}
/* If we're going to check every file, ensure we save the results */
/*
* If we're going to check every file, ensure we save
* the results.
*/
if (is_cache_changed)
istate->cache_changed |= FSMONITOR_CHANGED;
@@ -411,7 +532,8 @@ void remove_fsmonitor(struct index_state *istate)
void tweak_fsmonitor(struct index_state *istate)
{
unsigned int i;
int fsmonitor_enabled = git_config_get_fsmonitor();
struct repository *r = istate->repo ? istate->repo : the_repository;
int fsmonitor_enabled = (fsm_settings__get_mode(r) > FSMONITOR_MODE_DISABLED);
if (istate->fsmonitor_dirty) {
if (fsmonitor_enabled) {
@@ -431,16 +553,8 @@ void tweak_fsmonitor(struct index_state *istate)
istate->fsmonitor_dirty = NULL;
}
switch (fsmonitor_enabled) {
case -1: /* keep: do nothing */
break;
case 0: /* false */
remove_fsmonitor(istate);
break;
case 1: /* true */
if (fsmonitor_enabled)
add_fsmonitor(istate);
break;
default: /* unknown value: do nothing */
break;
}
else
remove_fsmonitor(istate);
}

18
fsmonitor.h
View File

@@ -3,6 +3,7 @@
#include "cache.h"
#include "dir.h"
#include "fsmonitor-settings.h"
extern struct trace_key trace_fsmonitor;
@@ -57,7 +58,11 @@ int fsmonitor_is_trivial_response(const struct strbuf *query_result);
*/
static inline int is_fsmonitor_refreshed(const struct index_state *istate)
{
return !core_fsmonitor || istate->fsmonitor_has_run_once;
struct repository *r = istate->repo ? istate->repo : the_repository;
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
return fsm_mode <= FSMONITOR_MODE_DISABLED ||
istate->fsmonitor_has_run_once;
}
/*
@@ -67,7 +72,11 @@ static inline int is_fsmonitor_refreshed(const struct index_state *istate)
*/
static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache_entry *ce)
{
if (core_fsmonitor && !(ce->ce_flags & CE_FSMONITOR_VALID)) {
struct repository *r = istate->repo ? istate->repo : the_repository;
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
if (fsm_mode > FSMONITOR_MODE_DISABLED &&
!(ce->ce_flags & CE_FSMONITOR_VALID)) {
istate->cache_changed = 1;
ce->ce_flags |= CE_FSMONITOR_VALID;
trace_printf_key(&trace_fsmonitor, "mark_fsmonitor_clean '%s'", ce->name);
@@ -83,7 +92,10 @@ static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache
*/
static inline void mark_fsmonitor_invalid(struct index_state *istate, struct cache_entry *ce)
{
if (core_fsmonitor) {
struct repository *r = istate->repo ? istate->repo : the_repository;
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
if (fsm_mode > FSMONITOR_MODE_DISABLED) {
ce->ce_flags &= ~CE_FSMONITOR_VALID;
untracked_cache_invalidate_path(istate, ce->name, 1);
trace_printf_key(&trace_fsmonitor, "mark_fsmonitor_invalid '%s'", ce->name);

1
git.c
View File

@@ -533,6 +533,7 @@ static struct cmd_struct commands[] = {
{ "format-patch", cmd_format_patch, RUN_SETUP },
{ "fsck", cmd_fsck, RUN_SETUP },
{ "fsck-objects", cmd_fsck, RUN_SETUP },
{ "fsmonitor--daemon", cmd_fsmonitor__daemon, RUN_SETUP },
{ "gc", cmd_gc, RUN_SETUP },
{ "get-tar-commit-id", cmd_get_tar_commit_id, NO_PARSEOPT },
{ "grep", cmd_grep, RUN_SETUP_GENTLY },

4
help.c
View File

@@ -11,6 +11,7 @@
#include "version.h"
#include "refs.h"
#include "parse-options.h"
#include "fsmonitor-ipc.h"
struct category_description {
uint32_t category;
@@ -664,6 +665,9 @@ void get_version_info(struct strbuf *buf, int show_build_options)
strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
if (fsmonitor_ipc__is_supported())
strbuf_addstr(buf, "feature: fsmonitor--daemon\n");
}
}

3
repo-settings.c
View File

@@ -2,6 +2,7 @@
#include "config.h"
#include "repository.h"
#include "midx.h"
#include "compat/fsmonitor/fsm-listen.h"
#define UPDATE_DEFAULT_BOOL(s,v) do { if (s == -1) { s = v; } } while(0)
@@ -26,6 +27,8 @@ void prepare_repo_settings(struct repository *r)
UPDATE_DEFAULT_BOOL(r->settings.commit_graph_read_changed_paths, 1);
UPDATE_DEFAULT_BOOL(r->settings.gc_write_commit_graph, 1);
r->settings.fsmonitor = NULL; /* lazy loaded */
if (!repo_config_get_int(r, "index.version", &value))
r->settings.index_version = value;
if (!repo_config_get_maybe_bool(r, "core.untrackedcache", &value)) {

3
repository.h
View File

@@ -4,6 +4,7 @@
#include "path.h"
struct config_set;
struct fsmonitor_settings;
struct git_hash_algo;
struct index_state;
struct lock_file;
@@ -35,6 +36,8 @@ struct repo_settings {
int gc_write_commit_graph;
int fetch_write_commit_graph;
struct fsmonitor_settings *fsmonitor; /* lazy loaded */
int index_version;
enum untracked_cache_setting core_untracked_cache;

7
simple-ipc.h
View File

@@ -107,7 +107,8 @@ void ipc_client_close_connection(struct ipc_client_connection *connection);
*/
int ipc_client_send_command_to_connection(
struct ipc_client_connection *connection,
const char *message, struct strbuf *answer);
const char *message, size_t message_len,
struct strbuf *answer);
/*
* Used by the client to synchronously connect and send and receive a
@@ -119,7 +120,8 @@ int ipc_client_send_command_to_connection(
*/
int ipc_client_send_command(const char *path,
const struct ipc_client_connect_options *options,
const char *message, struct strbuf *answer);
const char *message, size_t message_len,
struct strbuf *answer);
/*
* Simple IPC Server Side API.
@@ -144,6 +146,7 @@ typedef int (ipc_server_reply_cb)(struct ipc_server_reply_data *,
*/
typedef int (ipc_server_application_cb)(void *application_data,
const char *request,
size_t request_len,
ipc_server_reply_cb *reply_cb,
struct ipc_server_reply_data *reply_data);

4
t/README
View File

@@ -398,8 +398,8 @@ every 'git commit-graph write', as if the `--changed-paths` option was
passed in.
GIT_TEST_FSMONITOR=$PWD/t7519/fsmonitor-all exercises the fsmonitor
code path for utilizing a file system monitor to speed up detecting
new or changed files.
code path for utilizing a (hook based) file system monitor to speed up
detecting new or changed files.
GIT_TEST_INDEX_VERSION=<n> exercises the index read/write code path
for the index version specified. Can be set to any valid version

15
t/helper/test-chmtime.c
View File

@@ -134,6 +134,21 @@ int cmd__chmtime(int argc, const char **argv)
}
if (utb.modtime != sb.st_mtime && utime(argv[i], &utb) < 0) {
#ifdef GIT_WINDOWS_NATIVE
if (S_ISDIR(sb.st_mode)) {
/*
* NEEDSWORK: The Windows version of `utime()`
* (aka `mingw_utime()`) does not correctly
* handle directory arguments, since it uses
* `_wopen()`. Ignore it for now since this
* is just a test.
*/
fprintf(stderr,
("Failed to modify time on directory %s. "
"Skipping\n"), argv[i]);
continue;
}
#endif
fprintf(stderr, "Failed to modify time on %s: %s\n",
argv[i], strerror(errno));
return 1;

226
t/helper/test-fsmonitor-client.c Normal file
View File

@@ -0,0 +1,226 @@
/*
* test-fsmonitor-client.c: client code to send commands/requests to
* a `git fsmonitor--daemon` daemon.
*/
#include "test-tool.h"
#include "cache.h"
#include "parse-options.h"
#include "fsmonitor-ipc.h"
#include "thread-utils.h"
#include "trace2.h"
#ifndef HAVE_FSMONITOR_DAEMON_BACKEND
int cmd__fsmonitor_client(int argc, const char **argv)
{
die("fsmonitor--daemon not available on this platform");
}
#else
/*
* Read the `.git/index` to get the last token written to the
* FSMonitor Index Extension.
*/
static const char *get_token_from_index(void)
{
struct index_state *istate = the_repository->index;
if (do_read_index(istate, the_repository->index_file, 0) < 0)
die("unable to read index file");
if (!istate->fsmonitor_last_update)
die("index file does not have fsmonitor extension");
return istate->fsmonitor_last_update;
}
/*
* Send an IPC query to a `git-fsmonitor--daemon` daemon and
* ask for the changes since the given token or from the last
* token in the index extension.
*
* This will implicitly start a daemon process if necessary. The
* daemon process will persist after we exit.
*/
static int do_send_query(const char *token)
{
struct strbuf answer = STRBUF_INIT;
int ret;
if (!token || !*token)
token = get_token_from_index();
ret = fsmonitor_ipc__send_query(token, &answer);
if (ret < 0)
die(_("could not query fsmonitor--daemon"));
write_in_full(1, answer.buf, answer.len);
strbuf_release(&answer);
return 0;
}
/*
* Send a "flush" command to the `git-fsmonitor--daemon` (if running)
* and tell it to flush its cache.
*
* This feature is primarily used by the test suite to simulate a loss of
* sync with the filesystem where we miss kernel events.
*/
static int do_send_flush(void)
{
struct strbuf answer = STRBUF_INIT;
int ret;
ret = fsmonitor_ipc__send_command("flush", &answer);
if (ret)
return ret;
write_in_full(1, answer.buf, answer.len);
strbuf_release(&answer);
return 0;
}
struct hammer_thread_data
{
pthread_t pthread_id;
int thread_nr;
int nr_requests;
const char *token;
int sum_successful;
int sum_errors;
};
static void *hammer_thread_proc(void *_hammer_thread_data)
{
struct hammer_thread_data *data = _hammer_thread_data;
struct strbuf answer = STRBUF_INIT;
int k;
int ret;
trace2_thread_start("hammer");
for (k = 0; k < data->nr_requests; k++) {
strbuf_reset(&answer);
ret = fsmonitor_ipc__send_query(data->token, &answer);
if (ret < 0)
data->sum_errors++;
else
data->sum_successful++;
}
strbuf_release(&answer);
trace2_thread_exit();
return NULL;
}
/*
* Start a pool of client threads that will each send a series of
* commands to the daemon.
*
* The goal is to overload the daemon with a sustained series of
* concurrent requests.
*/
static int do_hammer(const char *token, int nr_threads, int nr_requests)
{
struct hammer_thread_data *data = NULL;
int k;
int sum_join_errors = 0;
int sum_commands = 0;
int sum_errors = 0;
if (!token || !*token)
token = get_token_from_index();
if (nr_threads < 1)
nr_threads = 1;
if (nr_requests < 1)
nr_requests = 1;
CALLOC_ARRAY(data, nr_threads);
for (k = 0; k < nr_threads; k++) {
struct hammer_thread_data *p = &data[k];
p->thread_nr = k;
p->nr_requests = nr_requests;
p->token = token;
if (pthread_create(&p->pthread_id, NULL, hammer_thread_proc, p)) {
warning("failed to create thread[%d] skipping remainder", k);
nr_threads = k;
break;
}
}
for (k = 0; k < nr_threads; k++) {
struct hammer_thread_data *p = &data[k];
if (pthread_join(p->pthread_id, NULL))
sum_join_errors++;
sum_commands += p->sum_successful;
sum_errors += p->sum_errors;
}
fprintf(stderr, "HAMMER: [threads %d][requests %d] [ok %d][err %d][join %d]\n",
nr_threads, nr_requests, sum_commands, sum_errors, sum_join_errors);
free(data);
/*
* TODO Decide if/when to return an error or call die().
*/
return 0;
}
int cmd__fsmonitor_client(int argc, const char **argv)
{
const char *subcmd;
const char *token = NULL;
int nr_threads = 1;
int nr_requests = 1;
const char * const fsmonitor_client_usage[] = {
N_("test-helper fsmonitor-client query [<token>]"),
N_("test-helper fsmonitor-client flush"),
N_("test-helper fsmonitor-client hammer [<token>] [<threads>] [<requests>]"),
NULL,
};
struct option options[] = {
OPT_STRING(0, "token", &token, N_("token"),
N_("command token to send to the server")),
OPT_INTEGER(0, "threads", &nr_threads, N_("number of client threads")),
OPT_INTEGER(0, "requests", &nr_requests, N_("number of requests per thread")),
OPT_END()
};
if (argc < 2)
usage_with_options(fsmonitor_client_usage, options);
if (argc == 2 && !strcmp(argv[1], "-h"))
usage_with_options(fsmonitor_client_usage, options);
subcmd = argv[1];
argv--;
argc++;
argc = parse_options(argc, argv, NULL, options, fsmonitor_client_usage, 0);
setup_git_directory();
if (!strcmp(subcmd, "query"))
return !!do_send_query(token);
if (!strcmp(subcmd, "flush"))
return !!do_send_flush();
if (!strcmp(subcmd, "hammer"))
return !!do_hammer(token, nr_threads, nr_requests);
die("Unhandled subcommand: '%s'", subcmd);
}
#endif

34
t/helper/test-simple-ipc.c
View File

@@ -112,7 +112,7 @@ static int app__slow_command(ipc_server_reply_cb *reply_cb,
/*
* The client sent a command followed by a (possibly very) large buffer.
*/
static int app__sendbytes_command(const char *received,
static int app__sendbytes_command(const char *received, size_t received_len,
ipc_server_reply_cb *reply_cb,
struct ipc_server_reply_data *reply_data)
{
@@ -123,6 +123,13 @@ static int app__sendbytes_command(const char *received,
int errs = 0;
int ret;
/*
* The test is setup to send:
* "sendbytes" SP <n * char>
*/
if (received_len < strlen("sendbytes "))
BUG("received_len is short in app__sendbytes_command");
if (skip_prefix(received, "sendbytes ", &p))
len_ballast = strlen(p);
@@ -160,7 +167,7 @@ static ipc_server_application_cb test_app_cb;
* by this application.
*/
static int test_app_cb(void *application_data,
const char *command,
const char *command, size_t command_len,
ipc_server_reply_cb *reply_cb,
struct ipc_server_reply_data *reply_data)
{
@@ -173,7 +180,7 @@ static int test_app_cb(void *application_data,
if (application_data != (void*)&my_app_data)
BUG("application_cb: application_data pointer wrong");
if (!strcmp(command, "quit")) {
if (command_len == 4 && !strncmp(command, "quit", 4)) {
/*
* The client sent a "quit" command. This is an async
* request for the server to shutdown.
@@ -193,22 +200,23 @@ static int test_app_cb(void *application_data,
return SIMPLE_IPC_QUIT;
}
if (!strcmp(command, "ping")) {
if (command_len == 4 && !strncmp(command, "ping", 4)) {
const char *answer = "pong";
return reply_cb(reply_data, answer, strlen(answer));
}
if (!strcmp(command, "big"))
if (command_len == 3 && !strncmp(command, "big", 3))
return app__big_command(reply_cb, reply_data);
if (!strcmp(command, "chunk"))
if (command_len == 5 && !strncmp(command, "chunk", 5))
return app__chunk_command(reply_cb, reply_data);
if (!strcmp(command, "slow"))
if (command_len == 4 && !strncmp(command, "slow", 4))
return app__slow_command(reply_cb, reply_data);
if (starts_with(command, "sendbytes "))
return app__sendbytes_command(command, reply_cb, reply_data);
if (command_len >= 10 && starts_with(command, "sendbytes "))
return app__sendbytes_command(command, command_len,
reply_cb, reply_data);
return app__unhandled_command(command, reply_cb, reply_data);
}
@@ -488,7 +496,9 @@ static int client__send_ipc(void)
options.wait_if_busy = 1;
options.wait_if_not_found = 0;
if (!ipc_client_send_command(cl_args.path, &options, command, &buf)) {
if (!ipc_client_send_command(cl_args.path, &options,
command, strlen(command),
&buf)) {
if (buf.len) {
printf("%s\n", buf.buf);
fflush(stdout);
@@ -556,7 +566,9 @@ static int do_sendbytes(int bytecount, char byte, const char *path,
strbuf_addstr(&buf_send, "sendbytes ");
strbuf_addchars(&buf_send, byte, bytecount);
if (!ipc_client_send_command(path, options, buf_send.buf, &buf_resp)) {
if (!ipc_client_send_command(path, options,
buf_send.buf, buf_send.len,
&buf_resp)) {
strbuf_rtrim(&buf_resp);
printf("sent:%c%08d %s\n", byte, bytecount, buf_resp.buf);
fflush(stdout);

1
t/helper/test-tool.c
View File

@@ -31,6 +31,7 @@ static struct test_cmd cmds[] = {
{ "dump-untracked-cache", cmd__dump_untracked_cache },
{ "example-decorate", cmd__example_decorate },
{ "fast-rebase", cmd__fast_rebase },
{ "fsmonitor-client", cmd__fsmonitor_client },
{ "genrandom", cmd__genrandom },
{ "genzeros", cmd__genzeros },
{ "getcwd", cmd__getcwd },

1
t/helper/test-tool.h
View File

@@ -21,6 +21,7 @@ int cmd__dump_split_index(int argc, const char **argv);
int cmd__dump_untracked_cache(int argc, const char **argv);
int cmd__example_decorate(int argc, const char **argv);
int cmd__fast_rebase(int argc, const char **argv);
int cmd__fsmonitor_client(int argc, const char **argv);
int cmd__genrandom(int argc, const char **argv);
int cmd__genzeros(int argc, const char **argv);
int cmd__getcwd(int argc, const char **argv);

61
t/perf/p7519-fsmonitor.sh
View File

@@ -24,7 +24,8 @@ test_description="Test core.fsmonitor"
# GIT_PERF_7519_SPLIT_INDEX: used to configure core.splitIndex
# GIT_PERF_7519_FSMONITOR: used to configure core.fsMonitor. May be an
# absolute path to an integration. May be a space delimited list of
# absolute paths to integrations.
# absolute paths to integrations. (This hook or list of hooks does not
# include the built-in fsmonitor--daemon.)
#
# The big win for using fsmonitor is the elimination of the need to scan the
# working directory looking for changed and untracked files. If the file
@@ -98,6 +99,13 @@ trace_stop() {
fi
}
touch_files() {
n=$1
d="$n"_files
(cd $d ; test_seq 1 $n | xargs touch )
}
test_expect_success "one time repo setup" '
# set untrackedCache depending on the environment
if test -n "$GIT_PERF_7519_UNTRACKED_CACHE"
@@ -119,10 +127,11 @@ test_expect_success "one time repo setup" '
fi &&
mkdir 1_file 10_files 100_files 1000_files 10000_files &&
for i in $(test_seq 1 10); do touch 10_files/$i; done &&
for i in $(test_seq 1 100); do touch 100_files/$i; done &&
for i in $(test_seq 1 1000); do touch 1000_files/$i; done &&
for i in $(test_seq 1 10000); do touch 10000_files/$i; done &&
touch_files 1 &&
touch_files 10 &&
touch_files 100 &&
touch_files 1000 &&
touch_files 10000 &&
git add 1_file 10_files 100_files 1000_files 10000_files &&
git commit -qm "Add files" &&
@@ -135,10 +144,16 @@ test_expect_success "one time repo setup" '
setup_for_fsmonitor() {
# set INTEGRATION_SCRIPT depending on the environment
if test -n "$INTEGRATION_PATH"
if test -n "$USE_FSMONITOR_DAEMON"
then
git config core.useBuiltinFSMonitor true &&
INTEGRATION_SCRIPT=false
elif test -n "$INTEGRATION_PATH"
then
git config core.useBuiltinFSMonitor false &&
INTEGRATION_SCRIPT="$INTEGRATION_PATH"
else
git config core.useBuiltinFSMonitor false &&
#
# Choose integration script based on existence of Watchman.
# Fall back to an empty integration script.
@@ -174,7 +189,10 @@ test_perf_w_drop_caches () {
}
test_fsmonitor_suite() {
if test -n "$INTEGRATION_SCRIPT"; then
if test -n "$USE_FSMONITOR_DAEMON"
then
DESC="builtin fsmonitor--daemon"
elif test -n "$INTEGRATION_SCRIPT"; then
DESC="fsmonitor=$(basename $INTEGRATION_SCRIPT)"
else
DESC="fsmonitor=disabled"
@@ -199,15 +217,15 @@ test_fsmonitor_suite() {
# Update the mtimes on upto 100k files to make status think
# that they are dirty. For simplicity, omit any files with
# LFs (i.e. anything that ls-files thinks it needs to dquote).
# Then fully backslash-quote the paths to capture any
# whitespace so that they pass thru xargs properly.
# LFs (i.e. anything that ls-files thinks it needs to dquote)
# and any files with whitespace so that they pass thru xargs
# properly.
#
test_perf_w_drop_caches "status (dirty) ($DESC)" '
git ls-files | \
head -100000 | \
grep -v \" | \
sed '\''s/\(.\)/\\\1/g'\'' | \
egrep -v " ." | \
xargs test-tool chmtime -300 &&
git status
'
@@ -285,4 +303,25 @@ test_expect_success "setup without fsmonitor" '
test_fsmonitor_suite
trace_stop
#
# Run a full set of perf tests using the built-in fsmonitor--daemon.
# It does not use the Hook API, so it has a different setup.
# Explicitly start the daemon here and before we start client commands
# so that we can later add custom tracing.
#
if test_have_prereq FSMONITOR_DAEMON
then
USE_FSMONITOR_DAEMON=t
trace_start fsmonitor--daemon--server
git fsmonitor--daemon start
trace_start fsmonitor--daemon--client
test_expect_success "setup for fsmonitor--daemon" 'setup_for_fsmonitor'
test_fsmonitor_suite
git fsmonitor--daemon stop
trace_stop
fi
test_done

2
t/perf/perf-lib.sh
View File

@@ -74,7 +74,7 @@ test_perf_copy_repo_contents () {
for stuff in "$1"/*
do
case "$stuff" in
*/objects|*/hooks|*/config|*/commondir|*/gitdir|*/worktrees)
*/objects|*/hooks|*/config|*/commondir|*/gitdir|*/worktrees|*/fsmonitor--daemon*)
;;
*)
cp -R "$stuff" "$repo/.git/" || exit 1

35
t/t7519-status-fsmonitor.sh
View File

@@ -55,6 +55,41 @@ test_lazy_prereq UNTRACKED_CACHE '
test $ret -ne 1
'
# Test that we detect and disallow repos that are incompatible with FSMonitor.
test_expect_success 'incompatible bare repo' '
test_when_finished "rm -rf ./bare-clone actual expect" &&
git init --bare bare-clone &&
cat >expect <<-\EOF &&
error: bare repos are incompatible with fsmonitor
EOF
test_must_fail \
git -C ./bare-clone -c core.fsmonitor=foo \
update-index --fsmonitor 2>actual &&
test_cmp expect actual &&
test_must_fail \
git -C ./bare-clone -c core.usebuiltinfsmonitor=true \
update-index --fsmonitor 2>actual &&
test_cmp expect actual
'
test_expect_success FSMONITOR_DAEMON 'run fsmonitor-daemon in bare repo' '
test_when_finished "rm -rf ./bare-clone actual" &&
git init --bare bare-clone &&
test_must_fail git -C ./bare-clone fsmonitor--daemon run 2>actual &&
grep "bare repos are incompatible with fsmonitor" actual
'
test_expect_success MINGW,FSMONITOR_DAEMON 'run fsmonitor-daemon in virtual repo' '
test_when_finished "rm -rf ./fake-virtual-clone actual" &&
git init fake-virtual-clone &&
test_must_fail git -C ./fake-virtual-clone \
-c core.virtualfilesystem=true \
fsmonitor--daemon run 2>actual &&
grep "virtual repos are incompatible with fsmonitor" actual
'
test_expect_success 'setup' '
mkdir -p .git/hooks &&
: >tracked &&

716
t/t7527-builtin-fsmonitor.sh Executable file
View File

@@ -0,0 +1,716 @@
#!/bin/sh
test_description='built-in file system watcher'
. ./test-lib.sh
if ! test_have_prereq FSMONITOR_DAEMON
then
skip_all="fsmonitor--daemon is not supported on this platform"
test_done
fi
stop_daemon_delete_repo () {
r=$1
git -C $r fsmonitor--daemon stop >/dev/null 2>/dev/null
rm -rf $1
return 0
}
start_daemon () {
case "$#" in
1) r="-C $1";;
*) r="";
esac
git $r fsmonitor--daemon start || return $?
git $r fsmonitor--daemon status || return $?
return 0
}
# Is a Trace2 data event present with the given catetory and key?
# We do not care what the value is.
#
have_t2_data_event () {
c=$1
k=$2
grep -e '"event":"data".*"category":"'"$c"'".*"key":"'"$k"'"'
}
test_expect_success 'explicit daemon start and stop' '
test_when_finished "stop_daemon_delete_repo test_explicit" &&
git init test_explicit &&
start_daemon test_explicit &&
git -C test_explicit fsmonitor--daemon stop &&
test_must_fail git -C test_explicit fsmonitor--daemon status
'
test_expect_success 'implicit daemon start' '
test_when_finished "stop_daemon_delete_repo test_implicit" &&
git init test_implicit &&
test_must_fail git -C test_implicit fsmonitor--daemon status &&
# query will implicitly start the daemon.
#
# for test-script simplicity, we send a V1 timestamp rather than
# a V2 token. either way, the daemon response to any query contains
# a new V2 token. (the daemon may complain that we sent a V1 request,
# but this test case is only concerned with whether the daemon was
# implicitly started.)
GIT_TRACE2_EVENT="$(pwd)/.git/trace" \
test-tool -C test_implicit fsmonitor-client query --token 0 >actual &&
nul_to_q <actual >actual.filtered &&
grep "builtin:" actual.filtered &&
# confirm that a daemon was started in the background.
#
# since the mechanism for starting the background daemon is platform
# dependent, just confirm that the foreground command received a
# response from the daemon.
have_t2_data_event fsm_client query/response-length <.git/trace &&
git -C test_implicit fsmonitor--daemon status &&
git -C test_implicit fsmonitor--daemon stop &&
test_must_fail git -C test_implicit fsmonitor--daemon status
'
test_expect_success 'implicit daemon stop (delete .git)' '
test_when_finished "stop_daemon_delete_repo test_implicit_1" &&
git init test_implicit_1 &&
start_daemon test_implicit_1 &&
# deleting the .git directory will implicitly stop the daemon.
rm -rf test_implicit_1/.git &&
# [1] Create an empty .git directory so that the following Git
# command will stay relative to the `-C` directory.
#
# Without this, the Git command will override the requested
# -C argument and crawl out to the containing Git source tree.
# This would make the test result dependent upon whether we
# were using fsmonitor on our development worktree.
#
sleep 1 &&
mkdir test_implicit_1/.git &&
test_must_fail git -C test_implicit_1 fsmonitor--daemon status
'
test_expect_success 'implicit daemon stop (rename .git)' '
test_when_finished "stop_daemon_delete_repo test_implicit_2" &&
git init test_implicit_2 &&
start_daemon test_implicit_2 &&
# renaming the .git directory will implicitly stop the daemon.
mv test_implicit_2/.git test_implicit_2/.xxx &&
# See [1] above.
#
sleep 1 &&
mkdir test_implicit_2/.git &&
test_must_fail git -C test_implicit_2 fsmonitor--daemon status
'
# File systems on Windows may or may not have shortnames.
# This is a volume-specific setting on modern systems.
# "C:/" drives are required to have them enabled. Other
# hard drives default to disabled.
#
# This is a crude test to see if shortnames are enabled
# on the volume containing the test directory. It is
# crude, but it does not require elevation like `fsutil`.
#
test_lazy_prereq SHORTNAMES '
mkdir .foo &&
test -d "FOO~1"
'
# Here we assume that the shortname of ".git" is "GIT~1".
test_expect_success MINGW,SHORTNAMES 'implicit daemon stop (rename GIT~1)' '
test_when_finished "stop_daemon_delete_repo test_implicit_1s" &&
git init test_implicit_1s &&
start_daemon test_implicit_1s &&
# renaming the .git directory will implicitly stop the daemon.
# this moves {.git, GIT~1} to {.gitxyz, GITXYZ~1}.
# the rename-from FS Event will contain the shortname.
#
mv test_implicit_1s/GIT~1 test_implicit_1s/.gitxyz &&
sleep 1 &&
# put it back so that our status will not crawl out to our
# parent directory.
# this moves {.gitxyz, GITXYZ~1} to {.git, GIT~1}.
mv test_implicit_1s/.gitxyz test_implicit_1s/.git &&
test_must_fail git -C test_implicit_1s fsmonitor--daemon status
'
# Here we first create a file with LONGNAME of "GIT~1" before
# we create the repo. This will cause the shortname of ".git"
# to be "GIT~2".
test_expect_success MINGW,SHORTNAMES 'implicit daemon stop (rename GIT~2)' '
test_when_finished "stop_daemon_delete_repo test_implicit_1s2" &&
mkdir test_implicit_1s2 &&
echo HELLO >test_implicit_1s2/GIT~1 &&
git init test_implicit_1s2 &&
test_path_is_file test_implicit_1s2/GIT~1 &&
test_path_is_dir test_implicit_1s2/GIT~2 &&
start_daemon test_implicit_1s2 &&
# renaming the .git directory will implicitly stop the daemon.
# the rename-from FS Event will contain the shortname.
#
mv test_implicit_1s2/GIT~2 test_implicit_1s2/.gitxyz &&
sleep 1 &&
# put it back so that our status will not crawl out to our
# parent directory.
mv test_implicit_1s2/.gitxyz test_implicit_1s2/.git &&
test_must_fail git -C test_implicit_1s2 fsmonitor--daemon status
'
# Confirm that MacOS hides all of the Unicode normalization and/or
# case folding from the FS events. That is, are the pathnames in the
# FS events reported using the spelling on the disk or in the spelling
# used by the other process.
#
# Note that we assume that the filesystem is set to case insensitive.
#
# NEEDSWORK: APFS handles Unicode and Unicode normalization
# differently than HFS+. I only have an APFS partition, so
# more testing here would be helpful.
#
# Rename .git using alternate spelling and confirm that the daemon
# sees the event using the correct spelling and shutdown.
test_expect_success UTF8_NFD_TO_NFC 'MacOS event spelling (rename .GIT)' '
test_when_finished "stop_daemon_delete_repo test_apfs" &&
git init test_apfs &&
start_daemon test_apfs &&
test_path_is_dir test_apfs/.git &&
test_path_is_dir test_apfs/.GIT &&
mv test_apfs/.GIT test_apfs/.FOO &&
sleep 1 &&
mv test_apfs/.FOO test_apfs/.git &&
test_must_fail git -C test_apfs fsmonitor--daemon status
'
test_expect_success 'cannot start multiple daemons' '
test_when_finished "stop_daemon_delete_repo test_multiple" &&
git init test_multiple &&
start_daemon test_multiple &&
test_must_fail git -C test_multiple fsmonitor--daemon start 2>actual &&
grep "fsmonitor--daemon is already running" actual &&
git -C test_multiple fsmonitor--daemon stop &&
test_must_fail git -C test_multiple fsmonitor--daemon status
'
# These tests use the main repo in the trash directory
test_expect_success 'setup' '
>tracked &&
>modified &&
>delete &&
>rename &&
mkdir dir1 &&
>dir1/tracked &&
>dir1/modified &&
>dir1/delete &&
>dir1/rename &&
mkdir dir2 &&
>dir2/tracked &&
>dir2/modified &&
>dir2/delete &&
>dir2/rename &&
mkdir dirtorename &&
>dirtorename/a &&
>dirtorename/b &&
cat >.gitignore <<-\EOF &&
.gitignore
expect*
actual*
flush*
trace*
EOF
git -c core.useBuiltinFSMonitor= add . &&
test_tick &&
git -c core.useBuiltinFSMonitor= commit -m initial &&
git config core.useBuiltinFSMonitor true
'
# The test already explicitly stopped (or tried to stop) the daemon.
# This is here in case something else fails first.
#
redundant_stop_daemon () {
git fsmonitor--daemon stop
return 0
}
test_expect_success 'update-index implicitly starts daemon' '
test_when_finished redundant_stop_daemon &&
test_must_fail git fsmonitor--daemon status &&
GIT_TRACE2_EVENT="$(pwd)/.git/trace_implicit_1" \
git update-index --fsmonitor &&
git fsmonitor--daemon status &&
test_might_fail git fsmonitor--daemon stop &&
# Confirm that the trace2 log contains a record of the
# daemon starting.
test_subcommand git fsmonitor--daemon start <.git/trace_implicit_1
'
test_expect_success 'status implicitly starts daemon' '
test_when_finished redundant_stop_daemon &&
test_must_fail git fsmonitor--daemon status &&
GIT_TRACE2_EVENT="$(pwd)/.git/trace_implicit_2" \
git status >actual &&
git fsmonitor--daemon status &&
test_might_fail git fsmonitor--daemon stop &&
# Confirm that the trace2 log contains a record of the
# daemon starting.
test_subcommand git fsmonitor--daemon start <.git/trace_implicit_2
'
edit_files() {
echo 1 >modified
echo 2 >dir1/modified
echo 3 >dir2/modified
>dir1/untracked
}
delete_files() {
rm -f delete
rm -f dir1/delete
rm -f dir2/delete
}
create_files() {
echo 1 >new
echo 2 >dir1/new
echo 3 >dir2/new
}
rename_files() {
mv rename renamed
mv dir1/rename dir1/renamed
mv dir2/rename dir2/renamed
}
file_to_directory() {
rm -f delete
mkdir delete
echo 1 >delete/new
}
directory_to_file() {
rm -rf dir1
echo 1 >dir1
}
verify_status() {
git status >actual &&
GIT_INDEX_FILE=.git/fresh-index git read-tree master &&
GIT_INDEX_FILE=.git/fresh-index git -c core.useBuiltinFSMonitor= status >expect &&
test_cmp expect actual &&
echo HELLO AFTER &&
cat .git/trace &&
echo HELLO AFTER
}
# The next few test cases confirm that our fsmonitor daemon sees each type
# of OS filesystem notification that we care about. At this layer we just
# ensure we are getting the OS notifications and do not try to confirm what
# is reported by `git status`.
#
# We run a simple query after modifying the filesystem just to introduce
# a bit of a delay so that the trace logging from the daemon has time to
# get flushed to disk.
#
# We `reset` and `clean` at the bottom of each test (and before stopping the
# daemon) because these commands might implicitly restart the daemon.
clean_up_repo_and_stop_daemon () {
git reset --hard HEAD
git clean -fd
git fsmonitor--daemon stop
rm -f .git/trace
}
test_expect_success 'edit some files' '
test_when_finished clean_up_repo_and_stop_daemon &&
(
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
export GIT_TRACE_FSMONITOR &&
start_daemon
) &&
edit_files &&
test-tool fsmonitor-client query --token 0 >/dev/null 2>&1 &&
grep "^event: dir1/modified$" .git/trace &&
grep "^event: dir2/modified$" .git/trace &&
grep "^event: modified$" .git/trace &&
grep "^event: dir1/untracked$" .git/trace
'
test_expect_success 'create some files' '
test_when_finished clean_up_repo_and_stop_daemon &&
(
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
export GIT_TRACE_FSMONITOR &&
start_daemon
) &&
create_files &&
test-tool fsmonitor-client query --token 0 >/dev/null 2>&1 &&
grep "^event: dir1/new$" .git/trace &&
grep "^event: dir2/new$" .git/trace &&
grep "^event: new$" .git/trace
'
test_expect_success 'delete some files' '
test_when_finished clean_up_repo_and_stop_daemon &&
(
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
export GIT_TRACE_FSMONITOR &&
start_daemon
) &&
delete_files &&
test-tool fsmonitor-client query --token 0 >/dev/null 2>&1 &&
grep "^event: dir1/delete$" .git/trace &&
grep "^event: dir2/delete$" .git/trace &&
grep "^event: delete$" .git/trace
'
test_expect_success 'rename some files' '
test_when_finished clean_up_repo_and_stop_daemon &&
(
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
export GIT_TRACE_FSMONITOR &&
start_daemon
) &&
rename_files &&
test-tool fsmonitor-client query --token 0 >/dev/null 2>&1 &&
grep "^event: dir1/rename$" .git/trace &&
grep "^event: dir2/rename$" .git/trace &&
grep "^event: rename$" .git/trace &&
grep "^event: dir1/renamed$" .git/trace &&
grep "^event: dir2/renamed$" .git/trace &&
grep "^event: renamed$" .git/trace
'
test_expect_success 'rename directory' '
test_when_finished clean_up_repo_and_stop_daemon &&
(
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
export GIT_TRACE_FSMONITOR &&
start_daemon
) &&
mv dirtorename dirrenamed &&
test-tool fsmonitor-client query --token 0 >/dev/null 2>&1 &&
grep "^event: dirtorename/*$" .git/trace &&
grep "^event: dirrenamed/*$" .git/trace
'
test_expect_success 'file changes to directory' '
test_when_finished clean_up_repo_and_stop_daemon &&
(
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
export GIT_TRACE_FSMONITOR &&
start_daemon
) &&
file_to_directory &&
test-tool fsmonitor-client query --token 0 >/dev/null 2>&1 &&
grep "^event: delete$" .git/trace &&
grep "^event: delete/new$" .git/trace
'
test_expect_success 'directory changes to a file' '
test_when_finished clean_up_repo_and_stop_daemon &&
(
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
export GIT_TRACE_FSMONITOR &&
start_daemon
) &&
directory_to_file &&
test-tool fsmonitor-client query --token 0 >/dev/null 2>&1 &&
grep "^event: dir1$" .git/trace
'
# The next few test cases exercise the token-resync code. When filesystem
# drops events (because of filesystem velocity or because the daemon isn't
# polling fast enough), we need to discard the cached data (relative to the
# current token) and start collecting events under a new token.
#
# the 'test-tool fsmonitor-client flush' command can be used to send a
# "flush" message to a running daemon and ask it to do a flush/resync.
test_expect_success 'flush cached data' '
test_when_finished "stop_daemon_delete_repo test_flush" &&
git init test_flush &&
(
GIT_TEST_FSMONITOR_TOKEN=true &&
export GIT_TEST_FSMONITOR_TOKEN &&
GIT_TRACE_FSMONITOR="$(pwd)/.git/trace_daemon" &&
export GIT_TRACE_FSMONITOR &&
start_daemon test_flush
) &&
# The daemon should have an initial token with no events in _0 and
# then a few (probably platform-specific number of) events in _1.
# These should both have the same <token_id>.
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000001:0" >actual_0 &&
nul_to_q <actual_0 >actual_q0 &&
touch test_flush/file_1 &&
touch test_flush/file_2 &&
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000001:0" >actual_1 &&
nul_to_q <actual_1 >actual_q1 &&
grep "file_1" actual_q1 &&
# Force a flush. This will change the <token_id>, reset the <seq_nr>, and
# flush the file data. Then create some events and ensure that the file
# again appears in the cache. It should have the new <token_id>.
test-tool -C test_flush fsmonitor-client flush >flush_0 &&
nul_to_q <flush_0 >flush_q0 &&
grep "^builtin:test_00000002:0Q/Q$" flush_q0 &&
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000002:0" >actual_2 &&
nul_to_q <actual_2 >actual_q2 &&
grep "^builtin:test_00000002:0Q$" actual_q2 &&
touch test_flush/file_3 &&
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000002:0" >actual_3 &&
nul_to_q <actual_3 >actual_q3 &&
grep "file_3" actual_q3
'
# The next few test cases create repos where the .git directory is NOT
# inside the one of the working directory. That is, where .git is a file
# that points to a directory elsewhere. This happens for submodules and
# non-primary worktrees.
test_expect_success 'setup worktree base' '
git init wt-base &&
echo 1 >wt-base/file1 &&
git -C wt-base add file1 &&
git -C wt-base commit -m "c1"
'
test_expect_success 'worktree with .git file' '
git -C wt-base worktree add ../wt-secondary &&
(
GIT_TRACE2_PERF="$(pwd)/trace2_wt_secondary" &&
export GIT_TRACE2_PERF &&
GIT_TRACE_FSMONITOR="$(pwd)/trace_wt_secondary" &&
export GIT_TRACE_FSMONITOR &&
start_daemon wt-secondary
) &&
git -C wt-secondary fsmonitor--daemon stop &&
test_must_fail git -C wt-secondary fsmonitor--daemon status
'
# NEEDSWORK: Repeat one of the "edit" tests on wt-secondary and
# confirm that we get the same events and behavior -- that is, that
# fsmonitor--daemon correctly watches BOTH the working directory and
# the external GITDIR directory and behaves the same as when ".git"
# is a directory inside the working directory.
test_expect_success 'cleanup worktrees' '
stop_daemon_delete_repo wt-secondary &&
stop_daemon_delete_repo wt-base
'
# The next few tests perform arbitrary/contrived file operations and
# confirm that status is correct. That is, that the data (or lack of
# data) from fsmonitor doesn't cause incorrect results. And doesn't
# cause incorrect results when the untracked-cache is enabled.
test_lazy_prereq UNTRACKED_CACHE '
{ git update-index --test-untracked-cache; ret=$?; } &&
test $ret -ne 1
'
test_expect_success 'Matrix: setup for untracked-cache,fsmonitor matrix' '
test_might_fail git config --unset core.useBuiltinFSMonitor &&
git update-index --no-fsmonitor &&
test_might_fail git fsmonitor--daemon stop
'
matrix_clean_up_repo () {
git reset --hard HEAD
git clean -fd
}
matrix_try () {
uc=$1
fsm=$2
fn=$3
test_expect_success "Matrix[uc:$uc][fsm:$fsm] $fn" '
matrix_clean_up_repo &&
$fn &&
if test $uc = false -a $fsm = false
then
git status --porcelain=v1 >.git/expect.$fn
else
git status --porcelain=v1 >.git/actual.$fn
test_cmp .git/expect.$fn .git/actual.$fn
fi
'
return $?
}
uc_values="false"
test_have_prereq UNTRACKED_CACHE && uc_values="false true"
for uc_val in $uc_values
do
if test $uc_val = false
then
test_expect_success "Matrix[uc:$uc_val] disable untracked cache" '
git config core.untrackedcache false &&
git update-index --no-untracked-cache
'
else
test_expect_success "Matrix[uc:$uc_val] enable untracked cache" '
git config core.untrackedcache true &&
git update-index --untracked-cache
'
fi
fsm_values="false true"
for fsm_val in $fsm_values
do
if test $fsm_val = false
then
test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] disable fsmonitor" '
test_might_fail git config --unset core.useBuiltinFSMonitor &&
git update-index --no-fsmonitor &&
test_might_fail git fsmonitor--daemon stop 2>/dev/null
'
else
test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] enable fsmonitor" '
git config core.useBuiltinFSMonitor true &&
git fsmonitor--daemon start &&
git update-index --fsmonitor
'
fi
matrix_try $uc_val $fsm_val edit_files
matrix_try $uc_val $fsm_val delete_files
matrix_try $uc_val $fsm_val create_files
matrix_try $uc_val $fsm_val rename_files
matrix_try $uc_val $fsm_val file_to_directory
matrix_try $uc_val $fsm_val directory_to_file
done
done
# Test Unicode UTF-8 characters in the pathname of the working
# directory. Use of "*A()" routines rather than "*W()" routines
# on Windows can sometimes lead to odd failures.
#
u1=$(printf "u_c3_a6__\xC3\xA6")
u2=$(printf "u_e2_99_ab__\xE2\x99\xAB")
u_values="$u1 $u2"
for u in $u_values
do
test_expect_success "Unicode path: $u" '
test_when_finished "stop_daemon_delete_repo $u" &&
git init "$u" &&
echo 1 >"$u"/file1 &&
git -C "$u" add file1 &&
git -C "$u" config core.useBuiltinFSMonitor true &&
start_daemon "$u" &&
git -C "$u" status >actual &&
grep "new file: file1" actual
'
done
test_done

6
t/test-lib.sh
View File

@@ -1718,3 +1718,9 @@ test_lazy_prereq REBASE_P '
# Tests that verify the scheduler integration must set this locally
# to avoid errors.
GIT_TEST_MAINT_SCHEDULER="none:exit 1"
# Does this platform support `git fsmonitor--daemon`
#
test_lazy_prereq FSMONITOR_DAEMON '
git version --build-options | grep "feature:" | grep "fsmonitor--daemon"
'