This reduces the disk footprint of a full Git for Windows setup
dramatically because on Windows, one cannot assume that hard links are
supported.
The net savings are calculated easily: the 32-bit `git.exe` file weighs
in with 7662 kB while the `git-wrapper.exe` file (modified to serve as a
drop-in replacement for builtins) weighs a scant 21 kB. At this point,
there are 109 builtins which results in a total of 813 MB disk space
being freed up by this commit.
Yes, that is really more than half a gigabyte.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git started out as a bunch of separate commands, in the true Unix spirit.
Over time, more and more functionality was shared between the different
Git commands, though, so it made sense to introduce the notion of
"builtins": programs that are actually integrated into the main Git
executable.
These builtins can be called in two ways: either by specifying a
subcommand as the first command-line argument, or -- for backwards
compatibility -- by calling the Git executable hardlinked to a filename
of the form "git-<subcommand>". Example: the "log" command can be called
via "git log <parameters>" or via "git-log <parameters>". The latter
form is actually deprecated and only supported for scripts; calling
"git-log" interactively will not even work by default because the
libexec/git-core/ directory is not in the PATH.
All of this is well and groovy as long as hard links are supported.
Sadly, this is not the case in general on Windows. So it actually hurts
quite a bit when you have to fall back to copying all of git.exe's
currently 7.5MB 109 times, just for backwards compatibility.
The simple solution would be to install really trivial shell script
wrappers in place of the builtins:
for builtin in $BUILTINS
do
rm git-$builtin.exe
printf '#!/bin/sh\nexec git %s "$@"\n' $builtin > git-builtin
chmod a+x git-builtin
done
This method would work -- even on Windows because Git for Windows ships a
full-fledged Bash. However, the Windows Bash comes at a price: it needs to
spin up a full-fledged POSIX emulation layer everytime it starts.
Therefore, the shell script solution would incur a significant performance
penalty.
The best solution the Git for Windows team could come up with is to extend
the Git wrapper -- that is needed to call Git from cmd.exe anyway, and
that weighs in with a scant 19KB -- to also serve as a drop-in replacement
for the builtins so that the following workaround is satisfactory:
for builtin in $BUILTINS
do
cp git-wrapper.exe git-$builtin.exe
done
This commit allows for this, by extending the module file parsing to
turn builtin command names like `git-log.exe ...` into calls to the main
Git executable: `git.exe log ...`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This prepares the wrapper for modifications to serve as a drop-in
replacement for the builtins.
This commit's diff is best viewed with the `-w` flag.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We take care to embed the manifest, too, because we will modify the
wrapper in the next few commits to serve as a drop-in replacement for
the built-ins, i.e. we will want to call the wrapper under names such
as 'git-patch-id.exe', too.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, Git is faced by the challenge that it has to set up certain
environment variables before running Git under special circumstances
such as when Git is called directly from cmd.exe (i.e. outside any
Bash environment).
This source code was taken from msysGit's commit 74a198d:
https://github.com/msysgit/msysgit/blob/74a198d/src/git-wrapper/git-wrapper.c
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since commit 0c499ea60f the send-pack builtin uses the side-band-64k
capability if advertised by the server.
Unfortunately this breaks pushing over the dump git protocol if used
over a network connection.
The detailed reasons for this breakage are (by courtesy of Jeff Preshing,
quoted from ttps://groups.google.com/d/msg/msysgit/at8D7J-h7mw/eaLujILGUWoJ):
----------------------------------------------------------------------------
MinGW wraps Windows sockets in CRT file descriptors in order to mimic the
functionality of POSIX sockets. This causes msvcrt.dll to treat sockets as
Installable File System (IFS) handles, calling ReadFile, WriteFile,
DuplicateHandle and CloseHandle on them. This approach works well in simple
cases on recent versions of Windows, but does not support all usage patterns.
In particular, using this approach, any attempt to read & write concurrently
on the same socket (from one or more processes) will deadlock in a scenario
where the read waits for a response from the server which is only invoked after
the write. This is what send_pack currently attempts to do in the use_sideband
codepath.
----------------------------------------------------------------------------
The new config option "sendpack.sideband" allows to override the side-band-64k
capability of the server, and thus makes the dump git protocol work.
Other transportation methods like ssh and http/https still benefit from
the sideband channel, therefore the default value of "sendpack.sideband"
is still true.
Signed-off-by: Thomas Braun <thomas.braun@byte-physics.de>
When calculating hashes from pointers, it actually makes sense to cut
off the most significant bits. In that case, said warning does not make
a whole lot of sense.
So let's just work around it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When prefixing a Git call in the test suite with 'TEST_GDB_GIT=1 ', it
will now be run with GDB, allowing the developer to debug test failures
more conveniently.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
With this patch we properly support SOCKS proxies, configured e.g. like
this:
git config http.proxy socks5://192.168.67.1:32767
Without this patch, Git mistakenly tries to use SOCKS proxies as if they
were HTTP proxies, resulting in a error message like:
fatal: unable to access 'http://.../': Proxy CONNECT aborted
This patch was required to work behind a faulty AP and scraped from
http://stackoverflow.com/questions/15227130/#15228479 and guarded with
an appropriate cURL version check by Johannes Schindelin.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This fixes the compilation on an older Linux that was used to debug
test failures when upgrading Git for Windows to Git v2.3.0.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
MinGW builds of cURL does not ship with curl-config unless built
with the autoconf based build system, which is not the practice
recommended by the documentation. MsysGit has had issues with
binaries of that sort, so it has switched away from autoconf-based
cURL-builds.
Unfortunately, broke pushing over WebDAV on Windows, because
http-push.c depends on cURL's multi-threaded API, which we could
not determine the presence of any more.
Since troublesome curl-versions are ancient, and not even present
in RedHat 5, let's just assume cURL is capable instead of doing a
non-robust check.
Instead, add a check for curl_multi_init to our configure-script,
for those on ancient system. They probably already need to do the
configure-dance anyway.
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
The previous implementation said that the filesystem information on
Windows is not reliable to determine whether a file is executable.
To find gather this information it was peeking into the first two bytes
of a file to see whether it looks executable.
Apart from the fact that on Windows executables are usually defined as
such by their extension it lead to slow opening of help file in some
situations.
When you have virus scanner running calling open on an executable file
is a potentially expensive operation. See the following measurements (in
seconds) for example.
With virus scanner running (coldcache):
$ ./a.exe /libexec/git-core/
before open (git-add.exe): 0.000000
after open (git-add.exe): 0.412873
before open (git-annotate.exe): 0.000175
after open (git-annotate.exe): 0.397925
before open (git-apply.exe): 0.000243
after open (git-apply.exe): 0.399996
before open (git-archive.exe): 0.000147
after open (git-archive.exe): 0.397783
before open (git-bisect--helper.exe): 0.000160
after open (git-bisect--helper.exe): 0.397700
before open (git-blame.exe): 0.000160
after open (git-blame.exe): 0.399136
...
With virus scanner running (hotcache):
$ ./a.exe /libexec/git-core/
before open (git-add.exe): 0.000000
after open (git-add.exe): 0.000325
before open (git-annotate.exe): 0.000229
after open (git-annotate.exe): 0.000177
before open (git-apply.exe): 0.000167
after open (git-apply.exe): 0.000150
before open (git-archive.exe): 0.000154
after open (git-archive.exe): 0.000156
before open (git-bisect--helper.exe): 0.000132
after open (git-bisect--helper.exe): 0.000180
before open (git-blame.exe): 0.000718
after open (git-blame.exe): 0.000724
...
This test did just list the given directory and open() each file in it.
With this patch I get:
$ time git help git
Launching default browser to display HTML ...
real 0m8.723s
user 0m0.000s
sys 0m0.000s
and without
$ time git help git
Launching default browser to display HTML ...
real 1m37.734s
user 0m0.000s
sys 0m0.031s
both tests with cold cache and giving the machine some time to settle
down after restart.
Signed-off-by: Heiko Voigt <heiko.voigt@mahr.de>
7ebac8cb94 made launching of .exe
externals work when installed in Unicode paths. But it broke launching
of non-.exe externals, no matter where they were installed. We now
correctly maintain the UTF-8 and UTF-16 paths in tandem in lookup_prog.
This fixes t5526, among others.
Signed-off-by: Adam Roben <adam@roben.org>
If Git were installed in a path containing non-ASCII characters,
commands such as git-am and git-submodule, which are implemented as
externals, would fail to launch with the following error:
> fatal: 'am' appears to be a git command, but we were not
> able to execute it. Maybe git-am is broken?
This was due to lookup_prog not being Unicode-aware. It was somehow
missed in 2ee5a1a14a.
Note that the only problem in this function was calling
GetFileAttributes instead of GetFileAttributesW. The calls to access()
were fine because access() is a macro which resolves to mingw_access,
which already handles Unicode correctly. But I changed lookup_prog to
use _waccess directly so that we only convert the path to UTF-16 once.
Signed-off-by: Adam Roben <adam@roben.org>
On Windows XP3 in git bash
git clone git@github.com:octocat/Spoon-Knife.git
cd Spoon-Knife
git gui
menu Remote\Fetch from\origin
error: cannot spawn git: No such file or directory
error: could not run rev-list
if u run
git fetch --all
it worked normal in git bash or gitgui tools
In second version CreateProcess get 'C:\Git\libexec\git-core/git.exe' in
first version - C:/Git/libexec/git-core/git.exe and not executes (unix
slashes)
after fixing C:\Git\libexec\git-core\git.exe or
C:/Git/libexec/git-core\git.exe it works normal
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
After importing anything with fast-import, we should always let the
garbage collector do its job, since the objects are written to disk
inefficiently.
This brings down an initial import of http://selenic.com/hg from about
230 megabytes to about 14.
In the future, we may want to make this configurable on a per-remote
basis, or maybe teach fast-import about it in the first place.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Again, avoid using echo (which issues DOS line endings on msysGit) to not mix
with Unix line-endings issued by git built-ins, even if this is at the cost of
calling an external executable (cat) instead of a shell built-in (echo).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The CVS tests expect `pwd` to return a POSIX-style directory. Let's skip
our MinGW-specific override to let `pwd` output a Windows-style directory
for that reason.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: 마누엘 <nalla@hamal.uberspace.de>
When calling `git fast-export a..a b` when a and b refer to the same
commit, nothing would be exported, and an incorrect reset line would
be printed for b ('from :0').
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
There are some issues with the git-svn test cases when they are
being run on windows under a MINGW build. Some things are not
available like the changing of the execute flag of shell scripts
via the chmod command. Also there were problems with folder names
that end with a dot on windows.
Signed-off-by: 마누엘 <nalla@users.noreply.github.com>
As per https://msdn.microsoft.com/en-us/library/x99tb11d.aspx:
The set of available locale names, languages, country/region
codes, and code pages includes all those supported by the Windows
NLS API except code pages that require more than two bytes per
character, such as UTF-7 and UTF-8.
Therefore, MinGW gettext cannot cope with UTF-8 at all, because it uses
the Win32 API internally.
However, when the test asks `locale -a` it reports that is_US.utf8 is
available, because that `locale` is actually an *MSys2* program (and MSys2
can cope with UTF-8 alright).
Let's just skip this test for MinGW Git altogether.
Helped-by: 마누엘 <nalla@users.noreply.github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
There is a MinGW gettext.exe, but still no MinGW locale.exe. Instead the
MSys2 locale.exe kicks in, which corresponds to the MSys2 gettext.exe,
however. Therefore some assumptions of t0200 cannot be fulfilled when
running inside MSys2.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
t0027 is marked expensive, but really, for MinGW we want to run these
tests always.
Suggested by Thomas Braun.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
A string of the form "@/abcd" is considered a file path
by the msys layer and therefore translated to a windows path.
Here the trick is to double the slashes.
The msys patch translation can be studied with the following
test program:
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char** argv)
{
unsigned int i;
for(i=1; i < argc; i++)
printf("argv[%d]=%s\n",i, argv[i]);
exit(0);
}
Signed-off-by: Thomas Braun <thomas.braun@byte-physics.de>
MSys works very hard to convert Unix-style paths into DOS-style ones.
*Very* hard.
So hard, indeed, that
git blame -L/hello/,/green/
is translated into something like
git blame -LC:/msysgit/hello/,C:/msysgit/green/
As seen in msys_p2w in src\msys\msys\rt\src\winsup\cygwin\path.cc, line
3204ff:
case '-':
//
// here we check for POSIX paths as attributes to a POSIX switch.
//
...
seemingly absolute POSIX paths in single-letter options get expanded by
msys.dll unless they contain '=' or ';'.
So a quick and very dirty fix is to use '-L/;*evil/'. (Using an equal sign
works only when it is before a comma, so in the above example, /=*green/
would still be converted to a DOS-style path.)
Commit-message-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The test separator char is a colon which means any absolute paths on windows
confuse the tests that use global_excludes.
Suggested-by: Karsten Blees <karsten.blees@gmail.com>
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
Since baaf233 (connect: improve check for plink to reduce false
positives, 2015-04-26), t5601 writes out a `plink.exe` for testing that
is actually a shell script. So the assumption that the `.exe` extension
implies that the file is *not* a shell script is now wrong.
The original idea to special-case `.exe` files was probably to
help performance, but since we are in a code path that involves spawning
a new process (which in and of itself is pretty slow on Windows anyway),
we pursue a better idea to improve performance elsewhere: we try to
convert scripts into builtins and to reduce the number of spawned
processes by adding more internal API calls.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Apparently the signal handling is not quite correct in the fsckobject
handling (most likely we rely on a side effect that lets us still output
some message after receiving a signal 13 but in the BuildHive setup this
fails intermittently).
As a consequence, the push in t5504 does fail as expected, but fails to
output anything (unexpected). Since this is good enough for now, let's
handle an empty output as success, too.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Accessing the Windows console through the special CONIN$ / CONOUT$ devices
doesn't work properly for non-ASCII usernames an passwords.
It also doesn't work for terminal emulators that hide the native console
window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable
indicator for such terminals.
The new shell_prompt() function, on the other hand, works fine for both
MSys1 and MSys2, in native console windows as well as mintty, and properly
supports Unicode. It just needs bash on the path (for 'read -s', which is
bash-specific).
On Windows, try to use the shell to read from the terminal. If that fails
with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback.
Note: To test this, create a UTF-8 credential file with non-ASCII chars,
e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd,
'git credential fill <cred.txt' works (shell version), while calling git
without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt')
mangles non-ASCII chars in both console output and input.
Signed-off-by: Karsten Blees <blees@dcon.de>
This happens only when the corresponding commits are not exported in
the current fast-export run. This can happen either when the relevant
commit is already marked, or when the commit is explicitly marked
as UNINTERESTING with a negative ref by another argument.
This breaks fast-export basec remote helpers.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
The `git_terminal_prompt()` function expects the terminal window to be
attached to a Win32 Console. However, this is not the case with terminal
windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`.
Non-cmd terminals such as `mintty` still have to have a Win32 Console
to be proper console programs, but have to hide the Win32 Console to
be able to provide more flexibility (such as being resizeable not only
vertically but also horizontally). By writing to that Win32 Console,
`git_terminal_prompt()` manages only to send the prompt to nowhere and
to wait for input from a Console to which the user has no access.
This commit introduces a function specifically to support `mintty` -- or
other terminals that are compatible with MSys2's `/dev/tty` emulation. We
use the `TERM` environment variable as an indicator for that: if the value
starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to
let `xterm_prompt()` handle the user interaction.
The most prominent user of `git_terminal_prompt()` is certainly
`git-remote-https.exe`. It is an interesting use case because both
`stdin` and `stdout` are redirected when Git calls said executable, yet
it still wants to access the terminal.
When running inside a `mintty`, the terminal is not accessible to the
`git-remote-https.exe` program, though, because it is a MinGW program
and the `mintty` terminal is not backed by a Win32 console.
To solve that problem, we simply call out to the shell -- which is an
*MSys2* program and can therefore access `/dev/tty`.
Helped-by: nalla <nalla@hamal.uberspace.de>
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
For performance reasons `stdout` is not unbuffered by default. That leads
to problems if after printing to `stdout` a read on `stdin` is performed.
For that reason interactive commands like `git clean -i` do not function
properly anymore if the `stdout` is not flushed by `fflush(stdout)` before
trying to read from `stdin`.
In the case of `git clean -i` all reads on `stdin` were preceded by a
`fflush(stdout)` call.
Signed-off-by: nalla <nalla@hamal.uberspace.de>
HOME initialization was historically duplicated in many different places,
including /etc/profile, launch scripts such as git-bash.vbs and gitk.cmd,
and (although slightly broken) in the git-wrapper.
Even unrelated projects such as GitExtensions and TortoiseGit need to
implement the same logic to be able to call git directly.
Initialize HOME in git's own startup code so that we can eventually retire
all the duplicate initialization code.
Signed-off-by: Karsten Blees <blees@dcon.de>
Lets keep the environment initialization and conversion section as lean as
possible and move recently added tweaks to setup_windows_environment().
This fixes the following potential problems:
* Prevent duplicate TZ variables if both TZ and MSYS2_TZ are set.
* Some of the higher level x* APIs from wrapper.c require a working
getenv(), using e.g. xstrdup() during initialization is dangerous.
* Slashifying the Windows TMP variable may break native Windows programs,
use POSIX TMPDIR instead.
* Properly slashify TMPDIR even if it is already set, and also if we only
have TEMP, but not TMP.
* Reduce complexity from O(n) to O(log n).
Signed-off-by: Karsten Blees <blees@dcon.de>
This developer should really, really have known better. The fact that we
are changing the environment in ways for which the MSVCRT is not
prepared for is bad enough. But then this developer followed the request
to re-enable nedmalloc -- despite the prediction that it would cause an
access violation, predicting it in the same message as the request to
re-enable nedmalloc, no less!
To paper over the issue until the time when this developer finds the
time to re-design the Unicode environment handling from scratch, let's
hope that cURL is the only library we are using that *may* set an
environment variable using MSVCRT's putenv() after we fscked the
environment up.
Note: this commit can serve as no source of pride to anyone, certainly
not yours truly. It is necessary as a quick and pragmatic stop gap,
though, to prevent worse problems.
Note: cURL manages to set the variable CHARSET when nedmalloc is *not*
enabled, without causing an access violation. In that case, it sets it
successfully to the value "cp" + GetACP() (hence it is our choice, too,
cURL may need it, Git does not):
https://github.com/bagder/curl/blob/aa5808b5/lib/easy.c#L157-L162
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git on native Windows exclusively uses UTF-8 for console output (both with
mintty and native console windows). Gettext uses setlocale() to determine
the output encoding for translated text, however, MSVCRT's setlocale()
doesn't support UTF-8. As a result, translated text is encoded in system
encoding (GetAPC()), and non-ASCII chars are mangled in console output.
Use gettext's bind_textdomain_codeset() to force the encoding to UTF-8 on
native Windows.
In this developers' setup, HAVE_LIBCHARSET_H is apparently defined, but
we *really* want to override the locale_charset() here.
Signed-off-by: Karsten Blees <blees@dcon.de>
