On Windows, we have to emulate the fstat() call to fill out information
that takes extra effort to obtain, such as the file permissions/type.
If all we want is the file size, we can use the much cheaper
GetFileSizeEx() function (available since Windows XP).
Suggested by Philip Kelley.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Often we are mmap()ing read-only. In those cases, it is wasteful to map in
copy-on-write mode. Even worse: it can cause errors where we run out of
space in the page file.
So let's be extra careful to map files in read-only mode whenever
possible.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is not really helpful when a `git fetch` fails with the message:
fatal: mmap failed: No error
In the particular instance encountered by a colleague of yours truly,
the Win32 error code was ERROR_COMMITMENT_LIMIT which means that the
page file is not big enough.
Let's make the message
fatal: mmap failed: File too large
instead, which is only marginally better, but which can be associated
with the appropriate work-around: setting `core.packedGitWindowSize` to
a relatively small value.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git for Windows 2.x ships with an executable that starts the Git Bash
with all the environment variables and what not properly set up. It is
also adjusted according to the Terminal emulator option chosen when
installing Git for Windows (while `bash.exe --login -i` would always
launch with Windows' default console).
So let's use that executable (usually C:\Program Files\Git\git-bash.exe)
instead of `bash.exe --login -i` if its presence was detected.
This fixes https://github.com/git-for-windows/git/issues/490
Signed-off-by: Thomas Kläger <thomas.klaeger@10a.ch>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When calling `Repository>Create Desktop Shortcut`, Git GUI assumes
that it is okay to call `wish.exe` directly on Windows. However, in
Git for Windows 2.x' context, that leaves several crucial environment
variables uninitialized, resulting in a shortcut that does not work.
To fix those environment variable woes, Git for Windows comes with a
convenient `git-gui.exe`, so let's just use it when it is available.
This fixes https://github.com/git-for-windows/git/issues/448
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
MSys2 might *look* like Cygwin, but it is *not* Cygwin... Unless it
is run with `MSYSTEM=MSYS`, that is.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We no longer use any of MSVCRT's stat-functions, so there's no need to
stick to a CRT-compatible 'struct stat' either.
Define and use our own POSIX-2013-compatible 'struct stat' with nanosecond-
precision file times.
Note: Due to performance issues when using git variants with different file
time resolutions, this patch does *not* yet enable nanosecond precision in
the Makefile (use 'make USE_NSEC=1').
Signed-off-by: Karsten Blees <blees@dcon.de>
fstat() is the only stat-related CRT function for which we don't have a
full replacement yet (and thus the only reason to stick with MSVCRT's
'struct stat' definition).
Fully implement fstat(), in preparation of implementing a POSIX 2013
compatible 'struct stat' with nanosecond-precision file times.
Signed-off-by: Karsten Blees <blees@dcon.de>
Reusing cached data speeds up git-svn by quite a fair bit. However, if
the YAML module is unavailable, the caches are written to disk in an
architecture-dependent manner. That leads to problems when upgrading,
say, from 32-bit to 64-bit Git for Windows.
Let's just try to read those caches back if we detect the absence of the
YAML module and the presence of the file, and delete the file if it
could not be read back correctly.
Note that the only way to catch the error when the memoized cache could
not be read back is to put the call inside an `eval { ... }` block
because it would die otherwise; the `eval` block should also return `1`
in case of success explicitly since the function reading back the cached
data does not return an appropriate value to test for success.
This fixes https://github.com/git-for-windows/git/issues/233.
[jes: fixed the commit message, made the sign-off explicit]
Signed-off-by: Gavin Lambert <github@mirality.co.nz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, there is no (single) `/etc/` directory. To address that, in
conjunction with the libgit2 project, Git for Windows introduced yet
another level of system-wide config files, located in C:\ProgramData
(and the equivalent on Windows XP).
Let's spell this out in the documentation.
This closes https://github.com/git-for-windows/git/pull/470 (because
there was no reaction in three months in that Pull Request).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Between the libgit2 and the Git for Windows project, there has been a
discussion how we could share Git configuration to avoid duplication (or
worse: skew).
Earlier, libgit2 was nice enough to just re-use Git for Windows'
C:\Program Files (x86)\Git\etc\gitconfig
but with the upcoming Git for Windows 2.x, there would be more paths to
search, as we will have 64-bit and 32-bit versions, and the
corresponding config files will be in %PROGRAMFILES%\Git\mingw64\etc and
...\mingw32\etc, respectively.
Worse: there are portable Git for Windows versions out there which live
in totally unrelated directories, still.
Therefore we came to a consensus to use `%PROGRAMDATA%\Git\config` as the
location for shared Git settings that are of wider interest than just Git
for Windows.
On XP, there is no %PROGRAMDATA%, therefore we need to use
"%ALLUSERSPROFILE%\Application Data\Git\config" in those setups.
Of course, the configuration in `%PROGRAMDATA%\Git\config` has the
widest reach, therefore it must take the lowest precedence, i.e. Git for
Windows can still override settings in its `etc/gitconfig` file.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
'git config' (--add / --unset etc.) automatically creates missing config
files. However, it fails with a misleading error message "could not lock
config file" if the parent directory doesn't exist.
Also create missing parent directories.
This is particularly important when calling
git config -f /non/existing/directory/config ...
We *must not* create the leading directories when locking a config file.
It is the caller's responsibility to ensure that the directory exists,
just like it is the caller's responsibility to call `git init` before
running repository operations.
Point in case: if we simply create all leading directories, calling
`git config user.name me` *outside* of a Git worktree will *create*
.git/!
This fixes https://github.com/git-for-windows/git/issues/643 and
https://groups.google.com/forum/#!topic/git-for-windows/fVRdnDIKVuw
[jes: prevented bogus .git/ directories from being created]
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When developing Git for Windows, we always have to ensure that we do not
break any non-Windows platforms, e.g. by introducing Windows-specific code
into the platform-independent source code.
At other times, it is necessary to test whether a bug is Windows-specific
or not, in order to send the bug report to the correct place. Having
access to a Linux-based Git comes in really handy in such a situation.
Vagrant offers a painless way to install and use a defined Linux
development environment on Windows (and other Operating Systems). We offer
a Vagrantfile to that end for two reasons:
1) To allow Windows users to gain the full power of Linux' Git
2) To offer users an easy path to verify that the issue they are about
to report is really a Windows-specific issue; otherwise they would
need to report it to git@vger.kernel.org instead.
Using it is easy: Download and install https://www.virtualbox.org/, then
download and install https://www.vagrantup.com/, then direct your
command-line window to the Git source directory containing the Vagrantfile
and run the commands:
vagrant up
vagrant ssh
See https://github.com/git-for-windows/git/wiki/Vagrant for details.
As part of switching Git for Windows' development environment from msysGit
to the MSys2-based Git SDK, this Vagrantfile was copy-edited from msysGit:
https://github.com/msysgit/msysgit/blob/0be8f2208/Vagrantfile
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The contract for the stat() and lstat() function is:
> stat(): stats the file pointed to by path and fills in buf.
> lstat(): is identical to stat(), except that if path is a symbolic link,
> then the link itself is stat-ed, not the file that it refers to.
stat() should always return the statistics of the file or directory a
symbolic link is pointing to. The lstat() function is used to get the
stats for the symlink. Hence the check should not be there.
Signed-off-by: Loris Chiocca <loris@chiocca.ch>
This is needed so that `_wchdir()` can be used with drive root
directories, e.g. C:\ (`_wchdir("C:")` fails to switch the directory
to the root directory).
This fixes https://github.com/msysgit/git/issues/359 (in Git for Windows
2.x only, though).
Likewise, `readlink()`'s semantics require a trailing slash for symbolic
links pointing to directories. Otherwise all checked out symbolic links
pointing to directories would be marked as modified even directly after a
fresh clone.
This fixes https://github.com/git-for-windows/git/issues/210
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Symlinks on Windows have a flag that indicates whether the target is a file
or a directory. Symlinks of wrong type simply don't work. This even affects
core Win32 APIs (e.g. DeleteFile() refuses to delete directory symlinks).
However, CreateFile() with FILE_FLAG_BACKUP_SEMANTICS doesn't seem to care.
Check the target type by first creating a tentative file symlink, opening
it, and checking the type of the resulting handle. If it is a directory,
recreate the symlink with the directory flag set.
It is possible to create symlinks before the target exists (or in case of
symlinks to symlinks: before the target type is known). If this happens,
create a tentative file symlink and postpone the directory decision: keep
a list of phantom symlinks to be processed whenever a new directory is
created in mingw_mkdir().
Limitations: This algorithm may fail if a link target changes from file to
directory or vice versa, or if the target directory is created in another
process.
Signed-off-by: Karsten Blees <blees@dcon.de>
Implement symlink() that always creates file symlinks. Fails with ENOSYS
if symlinks are disabled or unsupported.
Note: CreateSymbolicLinkW() was introduced with symlink support in Windows
Vista. For compatibility with Windows XP, we need to load it dynamically
and fail gracefully if it isnt's available.
Signed-off-by: Karsten Blees <blees@dcon.de>
Implement readlink() by reading NTFS reparse points. Works for symlinks
and directory junctions. If symlinks are disabled, fail with ENOSYS.
Signed-off-by: Karsten Blees <blees@dcon.de>
If symlinks are enabled, resolve all symlinks when changing directories,
as required by POSIX.
Note: Git's real_path() function bases its link resolution algorithm on
this property of chdir(). Unfortunately, the current directory on Windows
is limited to only MAX_PATH (260) characters. Therefore using symlinks and
long paths in combination may be problematic.
Note: GetFinalPathNameByHandleW() was introduced with symlink support in
Windows Vista. Thus, for compatibility with Windows XP, we need to load it
dynamically and behave gracefully if it isnt's available.
Signed-off-by: Karsten Blees <blees@dcon.de>
MSVCRT's _wrename() cannot rename symlinks over existing files: it returns
success without doing anything. Newer MSVCR*.dll versions probably do not
have this problem: according to CRT sources, they just call MoveFileEx()
with the MOVEFILE_COPY_ALLOWED flag.
Get rid of _wrename() and call MoveFileEx() with proper error handling.
Signed-off-by: Karsten Blees <blees@dcon.de>
_wunlink() / DeleteFileW() refuses to delete symlinks to directories. If
_wunlink() fails with ERROR_ACCESS_DENIED, try _wrmdir() as well.
Signed-off-by: Karsten Blees <blees@dcon.de>
Symlinks on Windows don't work the same way as on Unix systems. E.g. there
are different types of symlinks for directories and files, creating
symlinks requires administrative privileges etc.
By default, disable symlink support on Windows. I.e. users explicitly have
to enable it with 'git config [--system|--global] core.symlinks true'.
The test suite ignores system / global config files. Allow testing *with*
symlink support by checking if native symlinks are enabled in MSys2 (via
'MSYS=winsymlinks:nativestrict').
Reminder: This would need to be changed if / when we find a way to run the
test suite in a non-MSys-based shell (e.g. dash).
Signed-off-by: Karsten Blees <blees@dcon.de>
The retry pattern is duplicated in three places. It also seems to be too
hard to use: mingw_unlink() and mingw_rmdir() duplicate the code to retry,
and both of them do so incompletely. They also do not restore errno if the
user answers 'no'.
Introduce a retry_ask_yes_no() helper function that handles retry with
small delay, asking the user, and restoring errno.
mingw_unlink: include _wchmod in the retry loop (which may fail if the
file is locked exclusively).
mingw_rmdir: include special error handling in the retry loop.
Signed-off-by: Karsten Blees <blees@dcon.de>
Dynamic loading of DLL functions is duplicated in several places.
Add a set of macros to simplify the process.
Signed-off-by: Karsten Blees <blees@dcon.de>
Git typically doesn't trust the stat.st_size member of symlinks (e.g. see
strbuf_readlink()). However, some functions take shortcuts if st_size is 0
(e.g. diff_populate_filespec()).
In mingw_lstat() and fscache_lstat(), make sure to return an adequate size.
The extra overhead of opening and reading the reparse point to calculate
the exact size is not necessary, as git doesn't rely on the value anyway.
Signed-off-by: Karsten Blees <blees@dcon.de>
Move S_IFLNK detection to file_attr_to_st_mode() and reuse it in fscache.
Implement DT_LNK detection in dirent.c and the fscache readdir version.
Signed-off-by: Karsten Blees <blees@dcon.de>
When obtaining lstat information for reparse points, we need to call
FindFirstFile() in addition to GetFileInformationEx() to obtain the type
of the reparse point (symlink, mount point etc.). However, currently there
is no error handling whatsoever if FindFirstFile() fails.
Call FindFirstFile() before modifying the stat *buf output parameter and
error out if the call fails.
Note: The FindFirstFile() return value includes all the data that we get
from GetFileAttributesEx(), so we could replace GetFileAttributesEx() with
FindFirstFile(). We don't do that because GetFileAttributesEx() is about
twice as fast for single files. I.e. we only pay the extra cost of calling
FindFirstFile() in the rare case that we encounter a reparse point.
Note: The indentation of the remaining reparse point code will be fixed in
the next patch.
Signed-off-by: Karsten Blees <blees@dcon.de>
With the new mingw_stat() implementation, do_lstat() is only called from
mingw_lstat() (with follow == 0). Remove the extra function and the old
mingw_stat()-specific (follow == 1) logic.
Signed-off-by: Karsten Blees <blees@dcon.de>
With respect to symlinks, the current stat() implementation is almost the
same as lstat(): except for the file type (st_mode & S_IFMT), it returns
information about the link rather than the target.
Implement stat by opening the file with as little permissions as possible
and calling GetFileInformationByHandle on it. This way, all link resoltion
is handled by the Windows file system layer.
If symlinks are disabled, use lstat() as before, but fail with ELOOP if a
symlink would have to be resolved.
Signed-off-by: Karsten Blees <blees@dcon.de>
GetFileAttributes cannot handle paths with trailing dir separator. The
current [l]stat implementation calls GetFileAttributes twice if the path
has trailing slashes (first with the original path passed to [l]stat, and
and a second time with a path copy with trailing '/' removed).
With Unicode conversion, we get the length of the path for free and also
have a (wide char) buffer that can be modified.
Remove trailing directory separators before calling the Win32 API.
Signed-off-by: Karsten Blees <blees@dcon.de>
strbuf_readlink() refuses to read link targets that exceed PATH_MAX (even
if a sufficient size was specified by the caller).
As some platforms support longer paths, remove this restriction (similar
to strbuf_getcwd()).
Signed-off-by: Karsten Blees <blees@dcon.de>
strbuf_readlink() calls readlink() twice if the hint argument specifies the
exact size of the link target (e.g. by passing stat.st_size as returned by
lstat()). This is necessary because 'readlink(..., hint) == hint' could
mean that the buffer was too small.
Use hint + 1 as buffer size to prevent this.
Signed-off-by: Karsten Blees <blees@dcon.de>
Since commit 0c499ea60f the send-pack builtin uses the side-band-64k
capability if advertised by the server.
Unfortunately this breaks pushing over the dump git protocol if used
over a network connection.
The detailed reasons for this breakage are (by courtesy of Jeff Preshing,
quoted from ttps://groups.google.com/d/msg/msysgit/at8D7J-h7mw/eaLujILGUWoJ):
----------------------------------------------------------------------------
MinGW wraps Windows sockets in CRT file descriptors in order to mimic the
functionality of POSIX sockets. This causes msvcrt.dll to treat sockets as
Installable File System (IFS) handles, calling ReadFile, WriteFile,
DuplicateHandle and CloseHandle on them. This approach works well in simple
cases on recent versions of Windows, but does not support all usage patterns.
In particular, using this approach, any attempt to read & write concurrently
on the same socket (from one or more processes) will deadlock in a scenario
where the read waits for a response from the server which is only invoked after
the write. This is what send_pack currently attempts to do in the use_sideband
codepath.
----------------------------------------------------------------------------
The new config option "sendpack.sideband" allows to override the side-band-64k
capability of the server, and thus makes the dump git protocol work.
Other transportation methods like ssh and http/https still benefit from
the sideband channel, therefore the default value of "sendpack.sideband"
is still true.
Signed-off-by: Thomas Braun <thomas.braun@byte-physics.de>
It is better to state clearly expectations and intentions than to assume
quietly that everybody agrees.
This Code of Conduct is the Open Code of Conduct as per
http://todogroup.org/opencodeofconduct/ (the only modifications are the
adjustments to reflect that there is no "response team" in addition to the
Git for Windows maintainer, and the addition of the link to the Open Code
of Conduct itself).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The previous implementation said that the filesystem information on
Windows is not reliable to determine whether a file is executable.
To find gather this information it was peeking into the first two bytes
of a file to see whether it looks executable.
Apart from the fact that on Windows executables are usually defined as
such by their extension it lead to slow opening of help file in some
situations.
When you have virus scanner running calling open on an executable file
is a potentially expensive operation. See the following measurements (in
seconds) for example.
With virus scanner running (coldcache):
$ ./a.exe /libexec/git-core/
before open (git-add.exe): 0.000000
after open (git-add.exe): 0.412873
before open (git-annotate.exe): 0.000175
after open (git-annotate.exe): 0.397925
before open (git-apply.exe): 0.000243
after open (git-apply.exe): 0.399996
before open (git-archive.exe): 0.000147
after open (git-archive.exe): 0.397783
before open (git-bisect--helper.exe): 0.000160
after open (git-bisect--helper.exe): 0.397700
before open (git-blame.exe): 0.000160
after open (git-blame.exe): 0.399136
...
With virus scanner running (hotcache):
$ ./a.exe /libexec/git-core/
before open (git-add.exe): 0.000000
after open (git-add.exe): 0.000325
before open (git-annotate.exe): 0.000229
after open (git-annotate.exe): 0.000177
before open (git-apply.exe): 0.000167
after open (git-apply.exe): 0.000150
before open (git-archive.exe): 0.000154
after open (git-archive.exe): 0.000156
before open (git-bisect--helper.exe): 0.000132
after open (git-bisect--helper.exe): 0.000180
before open (git-blame.exe): 0.000718
after open (git-blame.exe): 0.000724
...
This test did just list the given directory and open() each file in it.
With this patch I get:
$ time git help git
Launching default browser to display HTML ...
real 0m8.723s
user 0m0.000s
sys 0m0.000s
and without
$ time git help git
Launching default browser to display HTML ...
real 1m37.734s
user 0m0.000s
sys 0m0.031s
both tests with cold cache and giving the machine some time to settle
down after restart.
Signed-off-by: Heiko Voigt <heiko.voigt@mahr.de>
7ebac8cb94 made launching of .exe
externals work when installed in Unicode paths. But it broke launching
of non-.exe externals, no matter where they were installed. We now
correctly maintain the UTF-8 and UTF-16 paths in tandem in lookup_prog.
This fixes t5526, among others.
Signed-off-by: Adam Roben <adam@roben.org>
If Git were installed in a path containing non-ASCII characters,
commands such as git-am and git-submodule, which are implemented as
externals, would fail to launch with the following error:
> fatal: 'am' appears to be a git command, but we were not
> able to execute it. Maybe git-am is broken?
This was due to lookup_prog not being Unicode-aware. It was somehow
missed in 2ee5a1a14a.
Note that the only problem in this function was calling
GetFileAttributes instead of GetFileAttributesW. The calls to access()
were fine because access() is a macro which resolves to mingw_access,
which already handles Unicode correctly. But I changed lookup_prog to
use _waccess directly so that we only convert the path to UTF-16 once.
Signed-off-by: Adam Roben <adam@roben.org>
On Windows XP3 in git bash
git clone git@github.com:octocat/Spoon-Knife.git
cd Spoon-Knife
git gui
menu Remote\Fetch from\origin
error: cannot spawn git: No such file or directory
error: could not run rev-list
if u run
git fetch --all
it worked normal in git bash or gitgui tools
In second version CreateProcess get 'C:\Git\libexec\git-core/git.exe' in
first version - C:/Git/libexec/git-core/git.exe and not executes (unix
slashes)
after fixing C:\Git\libexec\git-core\git.exe or
C:/Git/libexec/git-core\git.exe it works normal
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
After importing anything with fast-import, we should always let the
garbage collector do its job, since the objects are written to disk
inefficiently.
This brings down an initial import of http://selenic.com/hg from about
230 megabytes to about 14.
In the future, we may want to make this configurable on a per-remote
basis, or maybe teach fast-import about it in the first place.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When calling `git fast-export a..a b` when a and b refer to the same
commit, nothing would be exported, and an incorrect reset line would
be printed for b ('from :0').
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
