Commit Graph

64 Commits

Author SHA1 Message Date
Takuto Ikuta
41834bb21b checkout.c: enable fscache for checkout again
This is retry of #1419.

I added flush_fscache macro to flush cached stats after disk writing
with tests for regression reported in #1438 and #1442.

git checkout checks each file path in sorted order, so cache flushing does not
make performance worse unless we have large number of modified files in
a directory containing many files.

Using chromium repository, I tested `git checkout .` performance when I
delete 10 files in different directories.
With this patch:
TotalSeconds: 4.307272
TotalSeconds: 4.4863595
TotalSeconds: 4.2975562
Avg: 4.36372923333333

Without this patch:
TotalSeconds: 20.9705431
TotalSeconds: 22.4867685
TotalSeconds: 18.8968292
Avg: 20.7847136

I confirmed this patch passed all tests in t/ with core_fscache=1.

Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
2021-06-07 13:17:08 +02:00
Jeff Hostetler
cf1a0fde78 fscache: make fscache_enabled() public
Make fscache_enabled() function public rather than static.
Remove unneeded fscache_is_enabled() function.
Change is_fscache_enabled() macro to call fscache_enabled().

is_fscache_enabled() now takes a pathname so that the answer
is more precise and mean "is fscache enabled for this pathname",
since fscache only stores repo-relative paths and not absolute
paths, we can avoid attempting lookups for absolute paths.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-06-07 13:17:07 +02:00
Jeff Hostetler
249d41d9fd dir.c: make add_excludes aware of fscache during status
Teach read_directory_recursive() and add_excludes() to
be aware of optional fscache and avoid trying to open()
and fstat() non-existant ".gitignore" files in every
directory in the worktree.

The current code in add_excludes() calls open() and then
fstat() for a ".gitignore" file in each directory present
in the worktree.  Change that when fscache is enabled to
call lstat() first and if present, call open().

This seems backwards because both lstat needs to do more
work than fstat.  But when fscache is enabled, fscache will
already know if the .gitignore file exists and can completely
avoid the IO calls.  This works because of the lstat diversion
to mingw_lstat when fscache is enabled.

This reduced status times on a 350K file enlistment of the
Windows repo on a NVMe SSD by 0.25 seconds.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2021-06-07 13:17:07 +02:00
Jeff Hostetler
7f0710f50c fscache: remember not-found directories
Teach FSCACHE to remember "not found" directories.

This is a performance optimization.

FSCACHE is a performance optimization available for Windows.  It
intercepts Posix-style lstat() calls into an in-memory directory
using FindFirst/FindNext.  It improves performance on Windows by
catching the first lstat() call in a directory, using FindFirst/
FindNext to read the list of files (and attribute data) for the
entire directory into the cache, and short-cut subsequent lstat()
calls in the same directory.  This gives a major performance
boost on Windows.

However, it does not remember "not found" directories.  When STATUS
runs and there are missing directories, the lstat() interception
fails to find the parent directory and simply return ENOENT for the
file -- it does not remember that the FindFirst on the directory
failed. Thus subsequent lstat() calls in the same directory, each
re-attempt the FindFirst.  This completely defeats any performance
gains.

This can be seen by doing a sparse-checkout on a large repo and
then doing a read-tree to reset the skip-worktree bits and then
running status.

This change reduced status times for my very large repo by 60%.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-06-07 13:17:07 +02:00
Jeff Hostetler
a4da999ee5 fscache: add key for GIT_TRACE_FSCACHE
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-06-07 13:17:07 +02:00
Karsten Blees
95fc98763e fscache: load directories only once
If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.

On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.

Signed-off-by: Karsten Blees <blees@dcon.de>
2021-06-07 13:17:06 +02:00
Karsten Blees
1de7a44efb mingw: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow
`lstat()` emulation (git calls `lstat()` once for each file in the
index). Windows operating system APIs seem to be much better at scanning
the status of entire directories than checking single files.

Add an `lstat()` implementation that uses a cache for lstat data. Cache
misses read the entire parent directory and add it to the cache.
Subsequent `lstat()` calls for the same directory are served directly
from the cache.

Also implement `opendir()`/`readdir()`/`closedir()` so that they create
and use directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-06-07 13:17:06 +02:00
Karsten Blees
01385e7b2c mingw: make the dirent implementation pluggable
Emulating the POSIX `dirent` API on Windows via
`FindFirstFile()`/`FindNextFile()` is pretty staightforward, however,
most of the information provided in the `WIN32_FIND_DATA` structure is
thrown away in the process. A more sophisticated implementation may
cache this data, e.g. for later reuse in calls to `lstat()`.

Make the `dirent` implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to `readdir()`/`closedir()`
that match the `opendir()` implementation (similar to vtable pointers in
Object-Oriented Programming). Define `readdir()`/`closedir()` so that
they call the function pointers in the `DIR` structure. This allows to
choose the `opendir()` implementation on a call-by-call basis.

Make the fixed-size `dirent.d_name` buffer a flex array, as `d_name` may
be implementation specific (e.g. a caching implementation may allocate a
`struct dirent` with _just_ the size needed to hold the `d_name` in
question).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2021-06-07 13:17:06 +02:00
Karsten Blees
7a45d3e54d Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2021-06-07 13:17:06 +02:00
Johannes Schindelin
b8b2a8839c win32: add a helper to run git.exe without a foreground window
On Windows, there are two kinds of executables, console ones and
non-console ones. Git's executables are all console ones.

When launching the former e.g. in a scheduled task, a CMD window pops
up. This is not what we want for the tasks installed via the `git
maintenance` command.

To work around this, let's introduce `headless-git.exe`, which is a
non-console program that does _not_ pop up any window. All it does is to
re-launch `git.exe`, suppressing that console window, passing through
all command-line arguments as-are.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
2021-06-07 13:16:40 +02:00
Andras Kucsma
05ac8582bc run-command: trigger PATH lookup properly on Cygwin
On Cygwin, the codepath for POSIX-like systems is taken in
run-command.c::start_command(). The prepare_cmd() helper
function is called to decide if the command needs to be looked
up in the PATH. The logic there is to do the PATH-lookup if
and only if it does not have any slash '/' in it. If this test
passes we end up attempting to run the command by appending the
string after each colon-separated component of PATH.

The Cygwin environment supports both Windows and POSIX style
paths, so both forwardslahes '/' and back slashes '\' can be
used as directory separators for any external program the user
supplies.

Examples for path strings which are being incorrectly searched
for in the PATH instead of being executed as is:

- "C:\Program Files\some-program.exe"
- "a\b\c.exe"

To handle these, the PATH lookup detection logic in prepare_cmd()
is taught to know about this Cygwin quirk, by introducing
has_dir_sep(path) helper function to abstract away the difference
between true POSIX and Cygwin systems.

Signed-off-by: Andras Kucsma <r0maikx02b@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-27 11:06:17 -07:00
Johannes Schindelin
67af91c47a Sync with 2.23.1
* maint-2.23: (44 commits)
  Git 2.23.1
  Git 2.22.2
  Git 2.21.1
  mingw: sh arguments need quoting in more circumstances
  mingw: fix quoting of empty arguments for `sh`
  mingw: use MSYS2 quoting even when spawning shell scripts
  mingw: detect when MSYS2's sh is to be spawned more robustly
  t7415: drop v2.20.x-specific work-around
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  ...
2019-12-06 16:31:39 +01:00
Johannes Schindelin
7fd9fd94fb Sync with 2.22.2
* maint-2.22: (43 commits)
  Git 2.22.2
  Git 2.21.1
  mingw: sh arguments need quoting in more circumstances
  mingw: fix quoting of empty arguments for `sh`
  mingw: use MSYS2 quoting even when spawning shell scripts
  mingw: detect when MSYS2's sh is to be spawned more robustly
  t7415: drop v2.20.x-specific work-around
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  ...
2019-12-06 16:31:30 +01:00
Johannes Schindelin
5421ddd8d0 Sync with 2.21.1
* maint-2.21: (42 commits)
  Git 2.21.1
  mingw: sh arguments need quoting in more circumstances
  mingw: fix quoting of empty arguments for `sh`
  mingw: use MSYS2 quoting even when spawning shell scripts
  mingw: detect when MSYS2's sh is to be spawned more robustly
  t7415: drop v2.20.x-specific work-around
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  ...
2019-12-06 16:31:23 +01:00
Johannes Schindelin
fc346cb292 Sync with 2.20.2
* maint-2.20: (36 commits)
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  ...
2019-12-06 16:31:12 +01:00
Junio C Hamano
2d74d28ee0 Merge branch 'dl/compat-cleanup'
Code formatting micronit fix.

* dl/compat-cleanup:
  pthread.h: manually align parameter lists
2019-10-18 11:40:47 +09:00
Junio C Hamano
6d5291be45 Merge branch 'js/azure-pipelines-msvc'
CI updates.

* js/azure-pipelines-msvc:
  ci: also build and test with MS Visual Studio on Azure Pipelines
  ci: really use shallow clones on Azure Pipelines
  tests: let --immediate and --write-junit-xml play well together
  test-tool run-command: learn to run (parts of) the testsuite
  vcxproj: include more generated files
  vcxproj: only copy `git-remote-http.exe` once it was built
  msvc: work around a bug in GetEnvironmentVariable()
  msvc: handle DEVELOPER=1
  msvc: ignore some libraries when linking
  compat/win32/path-utils.h: add #include guards
  winansi: use FLEX_ARRAY to avoid compiler warning
  msvc: avoid using minus operator on unsigned types
  push: do not pretend to return `int` from `die_push_simple()`
2019-10-15 13:48:00 +09:00
Denton Liu
9cad1c4488 pthread.h: manually align parameter lists
In previous patches, extern was mechanically removed from function
declarations without care to formatting, causing parameter lists to be
misaligned. Manually format changed sections such that the parameter
lists are realigned.

Viewing this patch with 'git diff -w' should produce no output.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-11 14:59:35 +09:00
Johannes Schindelin
5b8f9e2417 compat/win32/path-utils.h: add #include guards
This adds the common guards that allow headers to be #include'd multiple
times.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-06 09:07:44 +09:00
Denton Liu
7027f508c7 compat/*.[ch]: remove extern from function declarations using spatch
In 554544276a (*.[ch]: remove extern from function declarations using
spatch, 2019-04-29), we removed externs from function declarations using
spatch but we intentionally excluded files under compat/ since some are
directly copied from an upstream and we should avoid churning them so
that manually merging future updates will be simpler.

In the last commit, we determined the files which taken from an upstream
so we can exclude them and run spatch on the remainder.

This was the Coccinelle patch used:

	@@
	type T;
	identifier f;
	@@
	- extern
	  T f(...);

and it was run with:

	$ git ls-files compat/\*\*.{c,h} |
		xargs spatch --sp-file contrib/coccinelle/noextern.cocci --in-place
	$ git checkout -- \
		compat/regex/ \
		compat/inet_ntop.c \
		compat/inet_pton.c \
		compat/nedmalloc/ \
		compat/obstack.{c,h} \
		compat/poll/

Coccinelle has some trouble dealing with `__attribute__` and varargs so
we ran the following to ensure that no remaining changes were left
behind:

	$ git ls-files compat/\*\*.{c,h} |
		xargs sed -i'' -e 's/^\(\s*\)extern \([^(]*([^*]\)/\1\2/'
	$ git checkout -- \
		compat/regex/ \
		compat/inet_ntop.c \
		compat/inet_pton.c \
		compat/nedmalloc/ \
		compat/obstack.{c,h} \
		compat/poll/

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-05 11:05:45 -07:00
Cesar Eduardo Barros
fe90397604 mingw: embed a manifest to trick UAC into Doing The Right Thing
On Windows >= Vista, not having an application manifest with a
requestedExecutionLevel can cause several kinds of confusing behavior.

The first and more obvious behavior is "Installer Detection" of the
"User Account Control" (also known as "UAC") feature, where Windows
sometimes decides (by looking at things like the file name and even
sequences of bytes within the executable) that an executable is an
installer and should run elevated (causing the well-known popup dialog
to appear). In Git's context, subcommands such as "git patch-id" or "git
update-index" fall prey to this behavior.

The second and more confusing behavior is "File Virtualization". It
means that when files are written without having write permission, it
does not fail (as expected), but they are instead redirected to
somewhere else. When the files are read, the original contents are
returned, though, not the ones that were just written somewhere else.
Even more confusing, not all write accesses are redirected; Trying to
write to write-protected .exe files, for example, will fail instead of
redirecting.

In addition to being unwanted behavior, File Virtualization causes
dramatic slowdowns in Git (see for instance
http://code.google.com/p/msysgit/issues/detail?id=320).

A third unwanted behavior of Windows >= Vista is that it lies about the
Windows version when calling `GetWindowsVersionEx()`.

There are two ways to prevent these unwanted behaviors: Either you embed
an application manifest (which really is an XML document conforming to a
specific schema) within all your executables, or you add an external
manifest (a file with the same name followed by `.manifest`) to all your
executables. Since Git's builtins are hardlinked (or copied), it is
simpler and more robust to embed a manifest.

Recent enough MSVC compilers already embed a working internal manifest,
and building with mingw-w64 (which is the case in Git for Windows' SDK)
does it, too, but for MinGW you have to do so by hand.

In any case, it is better to be explicit about this manifest, that way
changes in the compiler toolchain won't surprise us (as mingw-w64 once
did when it broke `GetWindowsVersionEx()` by mistake).

References:
  - New UAC Technologies for Windows Vista
    http://msdn.microsoft.com/en-us/library/bb756960.aspx
  - Create and Embed an Application Manifest (UAC)
    http://msdn.microsoft.com/en-us/library/bb756929.aspx

Signed-off-by: Cesar Eduardo Barros <cesarb@cesarb.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-27 12:55:45 -07:00
Jeff Hostetler
26c6f251d7 trace2: report peak memory usage of the process
Teach Windows version of git to report peak memory usage
during exit() processing.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:07 +09:00
Jeff Hostetler
353d3d77f4 trace2: collect Windows-specific process information
Add platform-specific interface to log information about the current
process.

On Windows, this interface is used to indicate whether the git process
is running under a debugger and list names of the process ancestors.

Information for other platforms is left for a future effort.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:27:59 -08:00
Junio C Hamano
25d90d1cb7 Merge branch 'tb/use-common-win32-pathfuncs-on-cygwin'
Cygwin update.

* tb/use-common-win32-pathfuncs-on-cygwin:
  git clone <url> C:\cygwin\home\USER\repo' is working (again)
2019-01-14 15:29:32 -08:00
Torsten Bögershausen
1cadad6f65 git clone <url> C:\cygwin\home\USER\repo' is working (again)
A regression for cygwin users was introduced with commit 05b458c,
 "real_path: resolve symlinks by hand".

In the the commit message we read:
  The current implementation of real_path uses chdir() in order to resolve
    symlinks.  Unfortunately this isn't thread-safe as chdir() affects a
      process as a whole...

The old (and non-thread-save) OS calls chdir()/pwd() had been
replaced by a string operation.
The cygwin layer "knows" that "C:\cygwin" is an absolute path,
but the new string operation does not.

"git clone <url> C:\cygwin\home\USER\repo" fails like this:
fatal: Invalid path '/home/USER/repo/C:\cygwin\home\USER\repo'

The solution is to implement has_dos_drive_prefix(), skip_dos_drive_prefix()
is_dir_sep(), offset_1st_component() and convert_slashes() for cygwin
in the same way as it is done in 'Git for Windows' in compat/mingw.[ch]

Extract the needed code into compat/win32/path-utils.[ch] and use it
for cygwin as well.

Reported-by: Steven Penny <svnpenn@gmail.com>
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-26 15:26:17 -08:00
Loo Rong Jie
c73b7ad548 win32: replace pthread_cond_*() with much simpler code
The Win32 CONDITION_VARIABLE has better performance and is easier to
maintain, as the code is a lot shorter now (the semantics of the
CONDITION_VARIABLE matches the pthread_cond_t very well).

Note: CONDITION_VARIABLE is not available in Windows XP and below,
but the declared minimal Windows version required to build and run
Git for Windows is Windows Vista (which is also beyond its
end-of-life, but for less long than Windows XP), so that's okay.

Signed-off-by: Loo Rong Jie <loorongjie@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 15:14:22 +09:00
Johannes Schindelin
db2f7c48cb Win32: simplify loading of DLL functions
Dynamic loading of DLL functions is duplicated in several places in Git
for Windows' source code.

This patch adds a pair of macros to simplify the process: the
DECLARE_PROC_ADDR(<dll>, <return-type>, <function-name>,
...<function-parameter-types>...) macro to be used at the beginning of a
code block, and the INIT_PROC_ADDR(<function-name>) macro to call before
using the declared function. The return value of the INIT_PROC_ADDR()
call has to be checked; If it is NULL, the function was not found in the
specified DLL.

Example:

        DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
                          LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);

        if (!INIT_PROC_ADDR(CreateHardLinkW))
                return error("Could not find CreateHardLinkW() function";

	if (!CreateHardLinkW(source, target, NULL))
		return error("could not create hardlink from %S to %S",
			     source, target);
	return 0;

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-26 11:02:49 +09:00
René Scharfe
149d8cbb2e win32: plug memory leak on realloc() failure in syslog()
If realloc() fails then the original buffer is still valid.  Free it
before exiting the function.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-10 13:57:52 -07:00
Junio C Hamano
b153d2ae92 Merge branch 'jk/push-client-deadlock-fix'
Some Windows SDK lacks pthread_sigmask() implementation and fails
to compile the recently updated "git push" codepath that uses it.

* jk/push-client-deadlock-fix:
  Windows: only add a no-op pthread_sigmask() when needed
  Windows: add pthread_sigmask() that does nothing
2016-05-18 15:11:46 -07:00
Johannes Schindelin
ed84387a6b Windows: only add a no-op pthread_sigmask() when needed
In f924b52 (Windows: add pthread_sigmask() that does nothing,
2016-05-01), we introduced a no-op for Windows. However, this breaks
building Git in Git for Windows' SDK because pthread_sigmask() is
already a no-op there, #define'd in the pthread_signal.h header in
/mingw64/x86_64-w64-mingw32/include/.

Let's wrap the definition of pthread_sigmask() in a guard that skips
it when compiling with MinGW-w64' headers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-11 14:02:10 -07:00
Nguyễn Thái Ngọc Duy
df8e31391d compat/win32/syslog.c: use warning_errno()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-09 12:29:08 -07:00
Johannes Sixt
f924b52a77 Windows: add pthread_sigmask() that does nothing
A previous change introduced a call to pthread_sigmask() in order to block
SIGPIPE in a thread. Since there are no signal facilities on Windows that
are similar to POSIX signals, just ignore the request to block the signal.
In the particular case, the effect of blocking SIGPIPE on POSIX is that
write() calls return EPIPE when the reader closes the pipe. This is how
write() behaves on Windows.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-02 11:22:24 -07:00
Junio C Hamano
3978cd06ff Merge branch 'js/pthread-exit-emu-windows'
* js/pthread-exit-emu-windows:
  Mark win32's pthread_exit() as NORETURN
2016-03-04 13:46:39 -08:00
Johannes Schindelin
296d673733 Mark win32's pthread_exit() as NORETURN
The pthread_exit() function is not expected to return. Ever. On Windows,
we call ExitThread() whose documentation claims: "Ends the calling
thread", i.e. there is no condition in which this function simply
returns: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682659

While at it, fix the return type to be void, as per
http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_exit.html

Pointed out by Jeff King, helped by Stefan Naewe, Junio Hamano &
Johannes Sixt.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-02 12:33:43 -08:00
Junio C Hamano
11529ecec9 Merge branch 'jk/tighten-alloc'
Update various codepaths to avoid manually-counted malloc().

* jk/tighten-alloc: (22 commits)
  ewah: convert to REALLOC_ARRAY, etc
  convert ewah/bitmap code to use xmalloc
  diff_populate_gitlink: use a strbuf
  transport_anonymize_url: use xstrfmt
  git-compat-util: drop mempcpy compat code
  sequencer: simplify memory allocation of get_message
  test-path-utils: fix normalize_path_copy output buffer size
  fetch-pack: simplify add_sought_entry
  fast-import: simplify allocation in start_packfile
  write_untracked_extension: use FLEX_ALLOC helper
  prepare_{git,shell}_cmd: use argv_array
  use st_add and st_mult for allocation size computation
  convert trivial cases to FLEX_ARRAY macros
  use xmallocz to avoid size arithmetic
  convert trivial cases to ALLOC_ARRAY
  convert manual allocations to argv_array
  argv-array: add detach function
  add helpers for allocating flex-array structs
  harden REALLOC_ARRAY and xcalloc against size_t overflow
  tree-diff: catch integer overflow in combine_diff_path allocation
  ...
2016-02-26 13:37:16 -08:00
Jeff King
50a6c8efa2 use st_add and st_mult for allocation size computation
If our size computation overflows size_t, we may allocate a
much smaller buffer than we expected and overflow it. It's
probably impossible to trigger an overflow in most of these
sites in practice, but it is easy enough convert their
additions and multiplications into overflow-checking
variants. This may be fixing real bugs, and it makes
auditing the code easier.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22 14:51:09 -08:00
Johannes Schindelin
2921600afb mingw: uglify (a, 0) definitions to shut up warnings
When the result of a (a, 0) expression is not used, MSys2's GCC version
finds it necessary to complain with a warning:

	right-hand operand of comma expression has no effect

Let's just pretend to use the 0 value and have a peaceful and quiet life
again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-15 14:02:39 -08:00
Johannes Schindelin
83c90da3c1 mingw: squash another warning about a cast
MSys2's compiler is correct that casting a "void *" to a "DWORD" loses
precision, but in the case of pthread_exit() we know that the value
fits into a DWORD.

Just like casting handles to DWORDs, let's work around this issue by
casting to "intrptr_t" first, and immediately cast to the final type.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-15 14:02:37 -08:00
Karsten Blees
0217569bb2 Win32: Unicode file name support (dirent)
Changes opendir/readdir to use Windows Unicode APIs and convert between
UTF-8/UTF-16.

Removes parameter checks that are already covered by xutftowcs_path. This
changes detection of ENAMETOOLONG from MAX_PATH - 2 to MAX_PATH (matching
is_dir_empty in mingw.c). If name + "/*" or the resulting absolute path is
too long, FindFirstFile fails and errno is set through err_win_to_posix.

Increases the size of dirent.d_name to accommodate the full
WIN32_FIND_DATA.cFileName converted to UTF-8 (UTF-16 to UTF-8 conversion
may grow by factor three in the worst case).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-15 11:19:09 -07:00
Junio C Hamano
b0bae7f0e4 Merge branch 'sk/mingw-dirent'
* sk/mingw-dirent:
  Win32 dirent: improve dirent implementation
  Win32 dirent: clarify #include directives
  Win32 dirent: change FILENAME_MAX to MAX_PATH
  Win32 dirent: remove unused dirent.d_reclen member
  Win32 dirent: remove unused dirent.d_ino member
2014-07-09 11:34:27 -07:00
Karsten Blees
d8890ce726 Win32 dirent: improve dirent implementation
Improve the dirent implementation by removing the relics that were once
necessary to plug into the now unused MinGW runtime, in preparation for
Unicode file name support.

Move FindFirstFile to opendir, and FindClose to closedir, with the
following implications:
- DIR.dd_name is no longer needed
- chdir(one); opendir(relative); chdir(two); readdir() works as expected
  (i.e. lists one/relative instead of two/relative)
- DIR.dd_handle is a valid handle for the entire lifetime of the DIR struct
- thus, all checks for dd_handle == INVALID_HANDLE_VALUE and dd_handle == 0
  have been removed
- the special case that the directory has been fully read (which was
  previously explicitly tracked with dd_handle == INVALID_HANDLE_VALUE &&
  dd_stat != 0) is now handled implicitly by the FindNextFile error
  handling code (if a client continues to call readdir after receiving
  NULL, FindNextFile will continue to fail with ERROR_NO_MORE_FILES, to
  the same effect)
- extracting dirent data from WIN32_FIND_DATA is needed in two places, so
  moved to its own method
- GetFileAttributes is no longer needed. The same information can be
  obtained from the FindFirstFile error code, which is ERROR_DIRECTORY if
  the name is NOT a directory (-> ENOTDIR), otherwise we can use
  err_win_to_posix (e.g. ERROR_PATH_NOT_FOUND -> ENOENT). The
  ERROR_DIRECTORY case could be fixed in err_win_to_posix, but this
  probably breaks other functionality.

Removes the ERROR_NO_MORE_FILES check after FindFirstFile (this was
fortunately a NOOP (searching for '*' always finds '.' and '..'),
otherwise the subsequent code would have copied data from an uninitialized
buffer).

Changes malloc to git support function xmalloc, so opendir will die() if
out of memory, rather than failing with ENOMEM and letting git work on
incomplete directory listings (error handling in dir.c is quite sparse).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09 15:10:53 -07:00
Karsten Blees
a8248f4a8d Win32 dirent: clarify #include directives
Git-compat-util.h is two dirs up, and already includes <dirent.h> (which
is the same as "dirent.h" due to -Icompat/win32 in the Makefile).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09 15:10:53 -07:00
Karsten Blees
fa9abe95be Win32 dirent: change FILENAME_MAX to MAX_PATH
FILENAME_MAX and MAX_PATH are both 260 on Windows, however, MAX_PATH is
used throughout the other Win32 code in Git, and also defines the length
of file name buffers in the Win32 API (e.g. WIN32_FIND_DATA.cFileName,
from which we're copying the dirent data).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09 15:10:53 -07:00
Karsten Blees
b0601e6564 Win32 dirent: remove unused dirent.d_reclen member
Remove the union around dirent.d_type and the unused dirent.d_reclen member
(which was necessary for compatibility with the MinGW dirent runtime, which
is no longer used).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09 15:10:53 -07:00
Karsten Blees
1d94c403fd Win32 dirent: remove unused dirent.d_ino member
There are no proper inodes on Windows, so remove dirent.d_ino and #define
NO_D_INO_IN_DIRENT in the Makefile (this skips e.g. an ineffective qsort in
fsck.c).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09 15:10:52 -07:00
Kirill Smelkov
22f4c27e68 mingw: activate alloca
Both MSVC and MINGW have alloca(3) definitions in malloc.h, so by moving
win32-compat alloca.h from compat/vcbuild/include/ to compat/win32/ ,
which is included by both MSVC and MINGW CFLAGS, we can make alloca()
work on both those Windows environments.

In MINGW, malloc.h has explicit check for GNUC and if it is so, defines
alloca to __builtin_alloca, so it looks like we don't need to add any
code to here-shipped alloca.h to get optimum performance.

Compile-tested on Windows in MSysGit.

Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Acked-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-09 10:08:35 -07:00
Ramsay Jones
9c3b051f93 compat/win32/pthread.c: Fix a sparse warning
Sparse issues a 'Using plain integer as NULL pointer' warning when
initializing an pthread_t structure with an '{ 0 }' initializer.
The first field of the pthread_t structure has type HANDLE (void *),
so in order to suppress the warning, we replace the initializer
expression with '{ NULL }'.

Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-28 12:26:45 -07:00
Joachim Schmitz
6d45eb1720 make poll available for other platforms lacking it
move poll.[ch] out of compat/win32/ into compat/poll/ and adjust
Makefile with the changed paths. Adding comments to Makefile about
how/when to enable it and add logic for this

Signed-off-by: Joachim Schmitz <jojo@schmitz-digital.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-17 15:42:57 -07:00
Ramsay Jones
9ba604a9e4 compat/win32/pthread.h: Add an pthread_key_delete() implementation
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-11 11:08:43 -07:00
Erik Faye-Lund
f0bd664977 compat/win32/poll.c: upgrade from upstream
poll.c is updated from revision adc3a5b in
git://git.savannah.gnu.org/gnulib.git

The changes are applied with --whitespace=fix to reduce noise.

poll.h is not upgraded, because the most recent version now
contains template-stuff that breaks compilation for us.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-30 18:45:16 -07:00