Commit Graph

123620 Commits

Author SHA1 Message Date
Jeff Hostetler
64971b4d86 fsmonitor--daemon: add pathname classification
Teach fsmonitor--daemon to classify relative and absolute
pathnames and decide how they should be handled.  This will
be used by the platform-specific backend to respond to each
filesystem event.

When we register for filesystem notifications on a directory,
we get events for everything (recursively) in the directory.
We want to report to clients changes to tracked and untracked
paths within the working directory.  We do not want to report
changes within the .git directory, for example.

This classification will be used in a later commit by the
different backends to classify paths as events are received.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:08 +02:00
Jeff Hostetler
34d228cf96 fsmonitor--daemon: implement 'start' command
Implement 'git fsmonitor--daemon start' command.  This command starts
an instance of 'git fsmonitor--daemon run' in the background using
the new 'start_bg_command()' function.

We avoid the fork-and-call technique on Unix systems in favor of a
fork-and-exec technique.  This gives us more uniform Trace2 child-*
events.  It also makes our usage more consistent with Windows usage.

On Windows, teach 'git fsmonitor--daemon run' to optionally call
'FreeConsole()' to release handles to the inherited Win32 console
(despite being passed invalid handles for stdin/out/err).  Without
this, command prompts and powershell terminal windows could hang
in "exit" until the last background child process exited or released
their Win32 console handle.  (This was not seen with git-bash shells
because they don't have a Win32 console attached to them.)

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Jeff Hostetler
2685c53a7c fsmonitor--daemon: implement 'run' command
Implement `run` command to try to begin listening for file system events.

This version defines the thread structure with a single fsmonitor_fs_listen
thread to watch for file system events and a simple IPC thread pool to
watch for connection from Git clients over a well-known named pipe or
Unix domain socket.

This commit does not actually do anything yet because the platform
backends are still just stubs.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Jeff Hostetler
c569886157 compat/fsmonitor/fsm-listen-darwin: stub in backend for Darwin
Stub in empty implementation of fsmonitor--daemon
backend for Darwin (aka MacOS).

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Jeff Hostetler
3aac461ddf compat/fsmonitor/fsm-listen-win32: stub in backend for Windows
Stub in empty filesystem listener backend for fsmonitor--daemon on Windows.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Jeff Hostetler
94ec0e57e3 fsmonitor--daemon: implement 'stop' and 'status' commands
Implement `stop` and `status` client commands to control and query the
status of a `fsmonitor--daemon` server process (and implicitly start a
server process if necessary).

Later commits will implement the actual server and monitor the file
system.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Carlo Marcelo Arenas Belón
0e97154174 builtin/fsmonitor--daemon: use parse-options API fully
--help and -h are already handled internally so just parse_options()
do the parsing and extract the command from the remaining options.

as a side effect, avoid setting a variable argc to a value that was
never used.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Jeff Hostetler
19e42f8f8e fsmonitor--daemon: add a built-in fsmonitor daemon
Create a built-in file system monitoring daemon that can be used by
the existing `fsmonitor` feature (protocol API and index extension)
to improve the performance of various Git commands, such as `status`.

The `fsmonitor--daemon` feature builds upon the `Simple IPC` API and
provides an alternative to hook access to existing fsmonitors such
as `watchman`.

This commit merely adds the new command without any functionality.

Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Jeff Hostetler
0f795e40f6 fsmonitor--daemon: man page
Create a manual page describing the `git fsmonitor--daemon` feature.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:07 +02:00
Jeff Hostetler
8744392ff1 Merge branch 'try-v4-fsmonitor-part2' into try-v4-fsmonitor 2021-10-02 22:40:07 +02:00
Jeff Hostetler
585b1555fe Merge branch 'try-v4-fsmonitor-part1' into try-v4-fsmonitor 2021-10-02 22:40:06 +02:00
Jeff Hostetler
8d563f0824 fsmonitor: update fsmonitor config documentation
Update references to `core.fsmonitor` and `core.fsmonitorHookVersion` and
pointers to `Watchman` to mention the new `core.useBuiltinFSMonitor`
value.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:06 +02:00
Jeff Hostetler
64e68477ef t/helper/simple-ipc: convert test-simple-ipc to use start_bg_command
Convert test helper to use `start_bg_command()` when spawning a server
daemon in the background rather than blocks of platform-specific code.

Also, while here, remove _() translation around error messages since
this is a test helper and not Git code.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:06 +02:00
Jeff Hostetler
3587437d0c fsmonitor: use IPC to query the builtin FSMonitor daemon
Use simple IPC to directly communicate with the new builtin file
system monitor daemon when `core.useBuiltinFSMonitor` is set.

The `core.fsmonitor` setting has already been defined as a HOOK
pathname.  Historically, this has been set to a HOOK script that will
talk with Watchman.  For compatibility reasons, we do not want to
overload that definition (and cause problems if users have multiple
versions of Git installed).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:06 +02:00
Jeff Hostetler
c008c3bdd3 fsmonitor: config settings are repository-specific
Move FSMonitor config settings to a new `struct fsmonitor_settings`
structure.  Add a lazily-loaded pointer to `struct repo_settings`.
Create `fsm_settings__get_*()` getters to lazily look up fsmonitor-
related config settings.

Get rid of the `core_fsmonitor` global variable, and add support for
the new `core.useBuiltinFSMonitor` config setting.  Move config code
to lookup the existing `core.fsmonitor` value to `fsmonitor-settings.[ch]`.

The `core_fsmonitor` global variable was used to store the pathname to
the FSMonitor hook and it was used as a boolean to see if FSMonitor
was enabled.  This dual usage will lead to confusion when we add
support for a builtin FSMonitor based on IPC, since the builtin
FSMonitor doesn't need the hook pathname.

Replace the boolean usage with an `enum fsmonitor_mode` to represent
the state of FSMonitor.  And only set the pathname when in HOOK mode.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:06 +02:00
Jeff Hostetler
c624a60d4b fsmonitor-ipc: create client routines for git-fsmonitor--daemon
Create fsmonitor_ipc__*() client routines to spawn the built-in file
system monitor daemon and send it an IPC request using the `Simple
IPC` API.

Stub in empty fsmonitor_ipc__*() functions for unsupported platforms.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:06 +02:00
Jeff Hostetler
e7939e07cb fsmonitor: enhance existing comments
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:06 +02:00
Jeff Hostetler
e4905549b2 Merge branch 'revert-pre-v4-fsmonitor-proper' into revert-pre-v4-fsmonitor 2021-10-02 22:40:05 +02:00
Jeff Hostetler
e96c86e4c2 run-command: create start_bg_command
Create a variation of `run_command()` and `start_command()` to launch a command
into the background and optionally wait for it to become "ready" before returning.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:05 +02:00
Jeff Hostetler
e3ac087f25 simple-ipc/ipc-win32: add Windows ACL to named pipe
Set an ACL on the named pipe to allow the well-known group EVERYONE
to read and write to the IPC server's named pipe.  In the event that
the daemon was started with elevation, allow non-elevated clients
to communicate with the daemon.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:05 +02:00
Jeff Hostetler
e5a1d91095 simple-ipc/ipc-win32: add trace2 debugging
Create "ipc-debug" category events to log unexpected errors
when creating Simple-IPC connections.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:05 +02:00
Jeff Hostetler
a0a6ebeb68 simple-ipc: move definition of ipc_active_state outside of ifdef
From: Carlo Marcelo Arenas Belón <carenas@gmail.com>

Move the declartion of the `enum ipc_active_state` type outside of
the SUPPORTS_SIMPLE_IPC ifdef.

A later commit will introduce the `fsmonitor_ipc__*()` API and stub in
a "mock" implementation that requires this enum in some function
signatures.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:05 +02:00
Jeff Hostetler
f183cc7b0d simple-ipc: preparations for supporting binary messages.
Add `command_len` argument to the Simple IPC API.

In my original Simple IPC API, I assumed that the request would always
be a null-terminated string of text characters.  The `command`
argument was just a `const char *`.

I found a caller that would like to pass a binary command to the
daemon, so I am amending the Simple IPC API to receive `const char
*command, size_t command_len` arguments.

I considered changing the `command` argument to be a `void *`, but the
IPC layer simply passes it to the pkt-line layer which takes a `const
char *`, so to avoid confusion I left it as is.

Note, the response side has always been a `struct strbuf` which
includes the buffer and length, so we already support returning a
binary answer.  (Yes, it feels a little weird returning a binary
buffer in a `strbuf`, but it works.)

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:05 +02:00
Jeff Hostetler
54ddae09f9 trace2: add trace2_child_ready() to report on background children
Create "child_ready" event to capture the state of a child process
created in the background.

When a child command is started a "child_start" event is generated in
the Trace2 log.  For normal synchronous children, a "child_exit" event
is later generated when the child exits or is terminated.  The two events
include information, such as the "child_id" and "pid", to allow post
analysis to match-up the command line and exit status.

When a child is started in the background (and may outlive the parent
process), it is not possible for the parent to emit a "child_exit"
event.  Create a new "child_ready" event to indicate whether the
child was successfully started.  Also include the "child_id" and "pid"
to allow similar post processing.

This will be used in a later commit with the new "start_bg_command()".

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:05 +02:00
Jeff Hostetler
19ee980052 fixup! simple-ipc: preparations for supporting binary messages.
This reverts commit 15b44c3f21.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
f3c5e26fcf fixup! fsmonitor--daemon: man page
This reverts commit e3e590dbc7.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
5c2eb20824 fixup! fsmonitor--daemon: update fsmonitor documentation
This reverts commit 5641d51b02.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
513dee5fd2 fixup! fsmonitor-ipc: create client routines for git-fsmonitor--daemon
This reverts commit a2f5482b4e.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
a63209e7c1 fixup! help: include fsmonitor--daemon feature flag in version info
This reverts commit bd7a1410d3.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
feea1cbe19 fixup! fsmonitor: config settings are repository-specific
This reverts commit 81f7337e3d.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
c6e29c27dc fixup! fsmonitor: use IPC to query the builtin FSMonitor daemon
This reverts commit abc1c46df9.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
669c95526b fixup! fsmonitor--daemon: add a built-in fsmonitor daemon
This reverts commit b441b3720e.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
a9bfc01b1f fixup! fsmonitor--daemon: implement 'stop' and 'status' commands
This reverts commit 1866a62b4f.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
7fbbac3a29 fixup! t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon
This reverts commit f6e27d05bc.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
c1b76c04bc fixup! fsm-listen-win32: stub in backend for Windows
This reverts commit 1b42506aba.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:04 +02:00
Jeff Hostetler
d953cad8a9 fixup! fsm-listen-darwin: stub in backend for Darwin
This reverts commit a6784cb378.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
d5fd4ed9e8 fixup! fsmonitor--daemon: implement 'run' command
This reverts commit 7d39bbb344.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
4ef3173349 fixup! fsmonitor--daemon: implement 'start' command
This reverts commit f5c5085496.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
eee16aa4aa fixup! fsmonitor--daemon: do not try to operate on bare repos
This reverts commit eab07e13cc.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
d27ea8bbab fixup! fsmonitor--daemon: add pathname classification
This reverts commit 89265708a0.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
1be0de12a3 fixup! fsmonitor--daemon: define token-ids
This reverts commit 13cf2b3944.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
f394cb1973 fixup! fsmonitor--daemon: create token-based changed path cache
This reverts commit 68a4ae6032.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
32c3a207fc fixup! fsm-listen-win32: implement FSMonitor backend on Windows
This reverts commit a25a759709.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
6fe4f74050 fixup! fsm-listen-darwin: add macos header files for FSEvent
This reverts commit ccc75e26a1.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
b27da829cd fixup! fsm-listen-darwin: implement FSEvent listener on MacOS
This reverts commit 8657519ee5.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
492c04ea7a fixup! fsmonitor--daemon: implement handle_client callback
This reverts commit d0ed68b4a1.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
b244b5416d fixup! t/helper/test-chmtime: skip directories on Windows
This reverts commit fd9522aa9b.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
43ab88164c fixup! t/perf/p7519: speed up test on Windows
This reverts commit b34d252c70.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:03 +02:00
Jeff Hostetler
62cad115d7 fixup! t/perf: avoid copying builtin fsmonitor files into test repo
This reverts commit 5ab490f926.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:02 +02:00
Jeff Hostetler
b1488adc28 fixup! t/perf/p7519: add fsmonitor--daemon test cases
This reverts commit 41b215e19a.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-10-02 22:40:02 +02:00