Commit Graph

97523 Commits

Author SHA1 Message Date
Johannes Schindelin
ab8bfeeba2 Merge pull request #1934 from benpeart/fscache-thread-safe-enable-gfw
fscache: make fscache_enable() thread safe
2019-05-13 22:57:05 +02:00
Johannes Schindelin
06d3217ac6 Merge remote-tracking branch 'benpeart/fscache-per-thread-gfw'
This brings substantial wins in performance because the FSCache is now
per-thread, being merged to the primary thread only at the end, so we do
not have to lock (except while merging).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:57:05 +02:00
Johannes Schindelin
d3cee078d4 Merge pull request #1910 from benpeart/fscache_statistics-gfw
fscache: add fscache hit statistics
2019-05-13 22:57:04 +02:00
Johannes Schindelin
6e5f7f81b3 Merge pull request #1914 from benpeart/free-fscache-after-add-gfw
At the end of the add command, disable and free the fscache
2019-05-13 22:57:03 +02:00
Johannes Schindelin
3009f8a1f1 Merge pull request #1911 from benpeart/git_test_fscache-gfw
fscache: add GIT_TEST_FSCACHE support
2019-05-13 22:57:03 +02:00
Johannes Schindelin
b97ee24586 Merge pull request #1909 from benpeart/free-fscache-after-status-gfw
status: disable and free fscache at the end of the status command
2019-05-13 22:57:03 +02:00
Johannes Schindelin
b4e55c6f4d Merge pull request #1908 from benpeart/FindFirstFileEx-gfw
fscache: use FindFirstFileExW to avoid retrieving the short name
2019-05-13 22:57:03 +02:00
Johannes Schindelin
a2396dc1a1 Merge pull request #1827 from benpeart/fscache_refresh_index
Enable the filesystem cache (fscache) in refresh_index().
2019-05-13 22:57:03 +02:00
Johannes Schindelin
a01dc7a753 Merge pull request #1468 from atetubou/fscache_checkout_flush
checkout.c: enable fscache for checkout again

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:57:02 +02:00
Johannes Schindelin
2aad8d9011 Merge pull request #1426 from atetubou/fetch_pack
fetch-pack.c: enable fscache for stats under .git/objects
2019-05-13 22:57:02 +02:00
Johannes Schindelin
176d05126b Merge pull request #1344 from jeffhostetler/perf_add_excludes_with_fscache
dir.c: make add_excludes aware of fscache during status
2019-05-13 22:57:02 +02:00
Johannes Schindelin
65b7a081e2 Merge pull request #971 from jeffhostetler/jeffhostetler/add_preload_fscache
add: use preload-index and fscache for performance
2019-05-13 22:57:02 +02:00
Johannes Schindelin
2e616ca605 Merge pull request #994 from jeffhostetler/jeffhostetler/fscache_nfd
fscache: add not-found directory cache to fscache
2019-05-13 22:57:02 +02:00
Johannes Schindelin
cb0f9bd8ba Merge pull request #305 from dscho/msysgit_issues_182
Allow `add -p` and `add -i` with a large number of files
2019-05-13 22:57:01 +02:00
Johannes Schindelin
234d8c5b08 Merge pull request #2091 from dscho/symlink-attr-extra
Touch up symlink .gitattributes support
2019-05-13 22:57:01 +02:00
Johannes Schindelin
dd3fa98395 Merge pull request #1897 from piscisaureus/symlink-attr
Specify symlink type in .gitattributes
2019-05-13 22:57:01 +02:00
Johannes Schindelin
8fb9411165 Merge 'docker-volumes-are-no-symlinks'
This was pull request #1645 from ZCube/master

Support windows container.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:57:01 +02:00
Johannes Schindelin
8c65925877 Merge branch 'kblees/kb/symlinks' 2019-05-13 22:57:01 +02:00
Johannes Schindelin
d42392ea57 Merge branch 'msys2' 2019-05-13 22:57:00 +02:00
Johannes Schindelin
781800f740 Merge branch 'core-longpaths-everywhere'
Git for Windows supports the core.longPaths config setting to allow
writing/reading long paths via the \\?\ trick for a long time now.

However, for that support to work, it is absolutely necessary that
git_default_config() is given a chance to parse the config. Otherwise
Git will be non the wiser.

So let's make sure that as many commands that previously failed to
parse the core.* settings now do that, implicitly enabling long path
support in a lot more places.

Note: this is not a perfect solution, and it cannot be, as there is
a chicken-and-egg problem in reading the config itself...

This fixes https://github.com/git-for-windows/git/issues/1218

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:57:00 +02:00
Johannes Schindelin
445f1b77d9 Merge branch 'long-paths' 2019-05-13 22:57:00 +02:00
Johannes Schindelin
6ffa601f3d Merge branch 'fscache' 2019-05-13 22:57:00 +02:00
Johannes Schindelin
abc31389b9 Merge branch 'gitk-and-git-gui-patches'
These are Git for Windows' Git GUI and gitk patches. We will have to
decide at some point what to do about them, but that's a little lower
priority (as Git GUI seems to be unmaintained for the time being, and
the gitk maintainer keeps a very low profile on the Git mailing list,
too).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:59 +02:00
Johannes Schindelin
6fd07b7dbe Merge branch 'ready-for-upstream'
This is the branch thicket of patches in Git for Windows that are
considered ready for upstream. To keep them in a ready-to-submit shape,
they are kept as close to the beginning of the branch thicket as
possible.
2019-05-13 22:56:59 +02:00
Ben Peart
2d648b810a fscache: make fscache_enable() thread safe
The recent change to make fscache thread specific relied on fscache_enable()
being called first from the primary thread before being called in parallel
from worker threads.  Make that more robust and protect it with a critical
section to avoid any issues.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ben Peart <benpeart@microsoft.com>
2019-05-13 22:56:57 +02:00
Johannes Schindelin
bc8f0a9a62 Merge branch 'busybox-w32'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:57 +02:00
Johannes Schindelin
66b767b47d mingw: add a Makefile target to copy test artifacts
The Makefile target `install-mingit-test-artifacts` simply copies stuff
and things directly into a MinGit directory, including an init.bat
script to set everything up so that the tests can be run in a cmd
window.

Sadly, Git's test suite still relies on a Perl interpreter even if
compiled with NO_PERL=YesPlease. We punt for now, installing a small
script into /usr/bin/perl that hands off to an existing Perl of a Git
for Windows SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
7d0ea1f79c t9350: skip ISO-8859-1 test when the environment is always-UTF-8
In the BusyBox-w32 version that is currently under consideration for
MinGit for Windows (to reduce the .zip size, and to avoid problems with
the MSYS2 runtime), the UTF-16 environment present in Windows is
considered to be authoritative, and the 8-bit version is always in UTF-8
encoding.

As a consequence, the ISO-8859-1 test in t9350-fast-export (which tries
to set GIT_AUTHOR_NAME to a ISO-8859-1 encoded value) *must* fail in
that setup.

So let's detect when it would fail (due to an environment being purely
kept UTF-8 encoded), and skip that test in that case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
a967239b6b t9200: skip tests when $PWD contains a colon
On Windows, the current working directory is pretty much guaranteed to
contain a colon. If we feed that path to CVS, it mistakes it for a
separator between host and port, though.

This has not been a problem so far because Git for Windows uses MSYS2's
Bash using a POSIX emulation layer that also pretends that the current
directory is a Unix path (at least as long as we're in a shell script).

However, that is rather limiting, as Git for Windows also explores other
ports of other Unix shells. One of those is BusyBox-w32's ash, which is
a native port (i.e. *not* using any POSIX emulation layer, and certainly
not emulating Unix paths).

So let's just detect if there is a colon in $PWD and punt in that case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
db8f3c20d3 t7063: when running under BusyBox, avoid unsupported find option
BusyBox' find implementation does not understand the -ls option, so
let's not use it when we're running inside BusyBox.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
4ccd2559d2 t5813: allow for $PWD to be a Windows path
Git for Windows uses MSYS2's Bash to run the test suite, which comes
with benefits but also at a heavy price: on the plus side, MSYS2's
POSIX emulation layer allows us to continue pretending that we are on a
Unix system, e.g. use Unix paths instead of Windows ones, yet this is
bought at a rather noticeable performance penalty.

There *are* some more native ports of Unix shells out there, though,
most notably BusyBox-w32's ash. These native ports do not use any POSIX
emulation layer (or at most a *very* thin one, choosing to avoid
features such as fork() that are expensive to emulate on Windows), and
they use native Windows paths (usually with forward slashes instead of
backslashes, which is perfectly legal in almost all use cases).

And here comes the problem: with a $PWD looking like, say,
C:/git-sdk-64/usr/src/git/t/trash directory.t5813-proto-disable-ssh
Git's test scripts get quite a bit confused, as their assumptions have
been shattered. Not only does this path contain a colon (oh no!), it
also does not start with a slash.

This is a problem e.g. when constructing a URL as t5813 does it:
ssh://remote$PWD. Not only is it impossible to separate the "host" from
the path with a $PWD as above, even prefixing $PWD by a slash won't
work, as /C:/git-sdk-64/... is not a valid path.

As a workaround, detect when $PWD does not start with a slash on
Windows, and simply strip the drive prefix, using an obscure feature of
Windows paths: if an absolute Windows path starts with a slash, it is
implicitly prefixed by the drive prefix of the current directory. As we
are talking about the current directory here, anyway, that strategy
works.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
0fd87d0bfa t5605: special-case hardlink test for BusyBox-w32
When t5605 tries to verify that files are hardlinked (or that they are
not), it uses the `-links` option of the `find` utility.

BusyBox' implementation does not support that option, and BusyBox-w32's
lstat() does not even report the number of hard links correctly (for
performance reasons).

So let's just switch to a different method that actually works on
Windows.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
2f7c6cab27 t5532: workaround for BusyBox on Windows
While it may seem super convenient to some old Unix hands to simpy
require Perl to be available when running the test suite, this is a
major hassle on Windows, where we want to verify that Perl is not,
actually, required in a NO_PERL build.

As a super ugly workaround, we "install" a script into /usr/bin/perl
reading like this:

	#!/bin/sh

	# We'd much rather avoid requiring Perl altogether when testing
	# an installed Git. Oh well, that's why we cannot have nice
	# things.
	exec c:/git-sdk-64/usr/bin/perl.exe "$@"

The problem with that is that BusyBox assumes that the #! line in a
script refers to an executable, not to a script. So when it encounters
the line #!/usr/bin/perl in t5532's proxy-get-cmd, it barfs.

Let's help this situation by simply executing the Perl script with the
"interpreter" specified explicitly.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
7eb67a4ea2 t5003: skip unzip -a tests with BusyBox
BusyBox' unzip is working pretty well. But Git's tests want to abuse it
to not only extract files, but to convert their line endings on the fly,
too. BusyBox' unzip does not support that, and it would appear that
it would require rather intrusive changes.

So let's just work around this by skipping the test case that uses
`unzip -a` and the subsequent test cases expecting `unzip -a`'s output.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:56 +02:00
Johannes Schindelin
b0bab16ff8 t5003: use binary file from t/diff-lib/
At some stage, t5003-archive-zip wants to add a file that is not ASCII.
To that end, it uses /bin/sh. But that file may actually not exist (it
is too easy to forget that not all the world is Unix/Linux...)! Besides,
we already have perfectly fine binary files intended for use solely by
the tests. So let's use one of them instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
4c831e3556 t4124: avoid using "normal" diff mode
Everybody and their dogs, cats and other pets settled on using unified
diffs. It is a really quaint holdover from a long-gone era that GNU diff
outputs "normal" diff by default.

Yet, t4124 relied on that mode.

This mode is so out of fashion in the meantime, though, that e.g.
BusyBox' diff decided not even to bother to support it. It only supports
unified diffs.

So let's just switch away from "normal" diffs and use unified diffs, as
we really are only interested in the `+` lines.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
19a692dccf t1300: mark all test cases with funny filenames as !MINGW
On Windows, it is impossible to create a file whose name contains a
quote character. We already excluded test cases using such files from
running on Windows when git.exe itself was tested.

However, we still had two test cases that try to create such a file, and
redirect stdin from such a file, respectively. This *seems* to work in
Git for Windows' Bash due to an obscure feature inherited from Cygwin:
illegal filename characters are simply mapped into/from a private UTF-8
page. Pure Win32 programs (such as git.exe) *still* cannot work with
those files, of course, but at least Unix shell scripts pretend to be
able to.

This entire strategy breaks down when switching to any Unix shell
lacking support for that private UTF-8 page trick, e.g. BusyBox-w32's
ash. So let's just exclude test cases that test whether the Unix shell
can redirect to/from files with "funny names" those from running on
Windows, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
4bd748a4e6 t0021: use Windows path when appropriate
Since c6b0831c9c (docs: warn about possible '=' in clean/smudge filter
process values, 2016-12-03), t0021 writes out a file with quotes in its
name, and MSYS2's path conversion heuristics mistakes that to mean that
we are not talking about a path here.

Therefore, we need to use Windows paths, as the test-helper is a Win32
program that would otherwise have no idea where to look for the file.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
fa8e16ed2c test-lib: add BUSYBOX prerequisite
When running with BusyBox, we will want to avoid calling executables on
the PATH that are implemented in BusyBox itself.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
dab091353b tests (mingw): remove Bash-specific pwd option
The -W option is only understood by MSYS2 Bash's pwd command. We already
make sure to override `pwd` by `builtin pwd -W` for MINGW, so let's not
double the effort here.

This will also help when switching the shell to another one (such as
BusyBox' ash) whose pwd does *not* understand the -W option.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
8760d9c057 mingw: only use Bash-ism builtin pwd -W when available
Traditionally, Git for Windows' SDK uses Bash as its default shell.
However, other Unix shells are available, too. Most notably, the Win32
port of BusyBox comes with `ash` whose `pwd` command already prints
Windows paths as Git for Windows wants them, while there is not even a
`builtin` command.

Therefore, let's be careful not to override `pwd` unless we know that
the `builtin` command is available.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
486971ae0c tests: use the correct path separator with BusyBox
BusyBox-w32 is a true Win32 application, i.e. it does not come with a
POSIX emulation layer.

That also means that it does *not* use the Unix convention of separating
the entries in the PATH variable using colons, but semicolons.

However, there are also BusyBox ports to Windows which use a POSIX
emulation layer such as Cygwin's or MSYS2's runtime, i.e. using colons
as PATH separators.

As a tell-tale, let's use the presence of semicolons in the PATH
variable: on Unix, it is highly unlikely that it contains semicolons,
and on Windows (without POSIX emulation), it is virtually guaranteed, as
everybody should have both $SYSTEMROOT and $SYSTEMROOT/system32 in their
PATH.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
b23dddb4e1 tests: only override sort & find if there are usable ones in /usr/bin/
The idea is to allow running the test suite on MinGit with BusyBox
installed in /mingw64/bin/sh.exe. In that case, we will want to exclude
sort & find (and other Unix utilities) from being bundled.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
e2f3b80d4d tests: move test PNGs into t/diff-lib/
We already have a directory where we store files intended for use by
multiple test scripts. The same directory is a better home for the
test-binary-*.png files than t/.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:55 +02:00
Johannes Schindelin
84b145c1af gitattributes: mark .png files as binary
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:54 +02:00
Johannes Schindelin
d89972cd2b tests: use t/diff-lib/* consistently
The idea of copying README and COPYING into t/diff-lib/ was to step away
from using files from outside t/ in tests. Let's really make sure that
we use the files from t/diff-lib/ instead of other versions of those
files.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:54 +02:00
Johannes Schindelin
10d76d0623 tests(mingw): if iconv is unavailable, use test-helper --iconv
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:54 +02:00
Johannes Schindelin
8e7a2d8b5d test-tool: learn to act as a drop-in replacement for iconv
It is convenient to assume that everybody who wants to build & test Git
has access to a working `iconv` executable (after all, we already pretty
much require libiconv).

However, that limits esoteric test scenarios such as Git for Windows',
where an end user installation has to ship with `iconv` for the sole
purpose of being testable. That payload serves no other purpose.

So let's just have a test helper (to be able to test Git, the test
helpers have to be available, after all) to act as `iconv` replacement.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:54 +02:00
Johannes Schindelin
4227b7a6d3 tests: replace mingw_test_cmp with a helper in C
This helper is slightly more performant than the script with MSYS2's
Bash. And a lot more readable.

To accommodate t1050, which wants to compare files weighing in with 3MB
(falling outside of t1050's malloc limit of 1.5MB), we simply lift the
allocation limit by setting the environment variable GIT_ALLOC_LIMIT to
zero when calling the helper.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:54 +02:00
Johannes Schindelin
84e183022c test-lib: avoid unnecessary Perl invocation
It is a bit strange, and even undesirable, to require Perl just to run
the test suite even when NO_PERL was set.

This patch does not fix this problem by any stretch of imagination.
However, it fixes *the* Perl invocation that *every single* test script
has to run.

While at it, it makes the source code also more grep'able, as the code
that unsets some, but not all, GIT_* environment variables just became a
*lot* more explicit. And all that while still reducing the total number
of lines.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-05-13 22:56:54 +02:00