The idea of having the Git wrapper in the /cmd/ directory is to allow
adding only a *tiny* set of executables to the search path, to allow
minimal interference with other software applications. It is quite
likely, for example, that other software applications require their own
version of zlib1.dll and would not be overly happy to find the version
Git for Windows ships.
The /cmd/ directory also gives us the opportunity to let the Git wrapper
handle the `gitk` script. It is a Tcl/Tk script that is not recognized
by Windows, therefore calling `gitk` in `cmd.exe` would not work, even
if we add all of Git for Windows' bin/ directories.
So let's use the /cmd/ directory instead of adding /mingw??/bin/ and
/usr/bin/ to the PATH when launching Git CMD.
The way we implemented Git CMD is to embed the appropriate command line
as string resource into a copy of the Git wrapper. Therefore we extended
that syntax to allow for configuring a minimal search path.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We recently added the ability to configure copies of the Git wrapper to
launch custom command-lines, configured via plain old Windows resources.
The main user is Git for Windows' `git-bash.exe`, of course. When the
user double-clicks the `git bash` icon, it makes sense to start the Bash
in the user's home directory.
Third-party software, such as TortoiseGit or GitHub for Windows, may
want to start the Git Bash in another directory, though.
Now, when third-party software wants to call Git, they already have to
construct a command-line, and can easily pass a command-line option
`--no-cd` (which this commit introduces), and since that option is not
available when the user double-clicks an icon on the Desktop or in the
Explorer, let's keep the default to switch to the home directory if the
`--no-cd` flag was not passed along.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When we rewrite the command-line to call the *real* Git, we want to skip
the first command-line parameter. The previous code worked in most
circumstances, but was a bit fragile because it assumed that no fancy
quoting would take place.
In the next commit, we will want to have the option to skip more than
just one command-line parameter, so we have to be much more careful with
the command-line handling.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In the meantime, Git for Windows learned to handle those subcommands
quite well itself; There is no longer a need to special-case them in the
wrapper.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In a push to polish Git for Windows more, we are moving away from
scripts toward proper binaries.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The Git wrapper does one thing, and does it well: setting up the
environment required to run Git and its scripts, and then hand off to
another program.
We already do this for the Git executable itself; in Git for Windows'
context, we have exactly the same need also when calling the Git Bash or
Git CMD. However, both are tied to what particular shell environment you
use, though: MSys or MSys2 (or whatever else cunning developers make
work for them). This means that the Git Bash and Git CMD need to be
compiled in the respective context (e.g. when compiling the
mingw-w64-git package in the MSys2 context).
Happily, Windows offers a way to configure compiled executables:
resources. So let's just look whether the current executable has a
string resource and use it as the command-line to execute after the
environment is set up. To support MSys2's Git Bash better (where
`mintty` should, but might not, be available), we verify whether the
specified executable exists, and keep looking for string resources if it
does not.
For even more flexibility, we expand environment variables specified as
`@@<VARIABLE-NAME>@@`, and for convenience `@@EXEPATH@@` expands into
the directory in which the executable resides.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Otherwise the output of Git commands cannot be caught by, say, Git GUI
(because it is running detached from any console, which would make
`git.exe` inherit the standard handles implicitly).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We are about to use the Git wrapper to call the Git Bash of Git for
Windows. All the wrapper needs to do for that is to set up the
environment variables, use the home directory as working directory and
then hand off to a user-specified command-line.
We prepare the existing code for this change by introducing flags to set
up the environment variables, to launch a non-Git program, and to use
the home directory as working directory.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The original purpose of the Git wrapper is to run from inside Git for
Windows' /cmd/ directory, to allow setting up some environment variables
before Git is allowed to take over.
Due to differences in the file system layout, MSys2 requires some
changes for that to work.
In addition, we must take care to set the `MSYSTEM` environment variable
to `MINGW32` or `MINGW64`, respectively, to allow MSys2 to be configured
correctly in case Git launches a shell or Perl script.
We also need to change the `TERM` variable to `cygwin` instead of
`msys`, otherwise the pager `less.exe` (spawned e.g. by `git log`) will
simply crash with a message similar to this one:
1 [main] less 9832 cygwin_exception::open_stackdumpfile:
Dumping stack trace to less.exe.stackdump
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This reduces the disk footprint of a full Git for Windows setup
dramatically because on Windows, one cannot assume that hard links are
supported.
The net savings are calculated easily: the 32-bit `git.exe` file weighs
in with 7662 kB while the `git-wrapper.exe` file (modified to serve as a
drop-in replacement for builtins) weighs a scant 21 kB. At this point,
there are 109 builtins which results in a total of 813 MB disk space
being freed up by this commit.
Yes, that is really more than half a gigabyte.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git started out as a bunch of separate commands, in the true Unix spirit.
Over time, more and more functionality was shared between the different
Git commands, though, so it made sense to introduce the notion of
"builtins": programs that are actually integrated into the main Git
executable.
These builtins can be called in two ways: either by specifying a
subcommand as the first command-line argument, or -- for backwards
compatibility -- by calling the Git executable hardlinked to a filename
of the form "git-<subcommand>". Example: the "log" command can be called
via "git log <parameters>" or via "git-log <parameters>". The latter
form is actually deprecated and only supported for scripts; calling
"git-log" interactively will not even work by default because the
libexec/git-core/ directory is not in the PATH.
All of this is well and groovy as long as hard links are supported.
Sadly, this is not the case in general on Windows. So it actually hurts
quite a bit when you have to fall back to copying all of git.exe's
currently 7.5MB 109 times, just for backwards compatibility.
The simple solution would be to install really trivial shell script
wrappers in place of the builtins:
for builtin in $BUILTINS
do
rm git-$builtin.exe
printf '#!/bin/sh\nexec git %s "$@"\n' $builtin > git-builtin
chmod a+x git-builtin
done
This method would work -- even on Windows because Git for Windows ships a
full-fledged Bash. However, the Windows Bash comes at a price: it needs to
spin up a full-fledged POSIX emulation layer everytime it starts.
Therefore, the shell script solution would incur a significant performance
penalty.
The best solution the Git for Windows team could come up with is to extend
the Git wrapper -- that is needed to call Git from cmd.exe anyway, and
that weighs in with a scant 19KB -- to also serve as a drop-in replacement
for the builtins so that the following workaround is satisfactory:
for builtin in $BUILTINS
do
cp git-wrapper.exe git-$builtin.exe
done
This commit allows for this, by extending the module file parsing to
turn builtin command names like `git-log.exe ...` into calls to the main
Git executable: `git.exe log ...`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This prepares the wrapper for modifications to serve as a drop-in
replacement for the builtins.
This commit's diff is best viewed with the `-w` flag.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We take care to embed the manifest, too, because we will modify the
wrapper in the next few commits to serve as a drop-in replacement for
the built-ins, i.e. we will want to call the wrapper under names such
as 'git-patch-id.exe', too.
To allow 32-bit and 64-bit builds in the same directory, we let
git-wrapper.o depend on GIT-PREFIX so that it gets recompiled when
compiling for a different architecture.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, Git is faced by the challenge that it has to set up certain
environment variables before running Git under special circumstances
such as when Git is called directly from cmd.exe (i.e. outside any
Bash environment).
This source code was taken from msysGit's commit 74a198d:
https://github.com/msysgit/msysgit/blob/74a198d/src/git-wrapper/git-wrapper.c
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It turns out that the earlier effort to update the heuristics may
want to use a bit more time to mature. Turn it off by default.
* jk/diff-compact-heuristic:
diff: disable compaction heuristic for now
http://lkml.kernel.org/g/20160610075043.GA13411@sigill.intra.peff.net
reports that a change to add a new "function" with common ending
with the existing one at the end of the file is shown like this:
def foo
do_foo_stuff()
+ common_ending()
+end
+
+def bar
+ do_bar_stuff()
+
common_ending()
end
when the new heuristic is in use. In reality, the change is to add
the blank line before "def bar" and everything below, which is what
the code without the new heuristic shows.
Disable the heuristics by default, and resurrect the documentation
for the option and the configuration variables, while clearly
marking the feature as still experimental.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we introduced the
close_all_packs() function.
Earlier, we made sure that the packs are released just before `git gc`
is spawned, in case that gc wants to remove no-longer needed packs.
But this developer forgot that gc itself also needs to let go of packs,
e.g. when consolidating all packs via the --aggressive option.
Likewise, `git repack -d` wants to delete obsolete packs and therefore
needs to close all pack handles, too.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
ASLR interferes with GDB's ability to set breakpoints. A similar issue
holds true when compiling with -O2 (in which case single-stepping is
messed up because GDB cannot map the code back to the original source
code properly). Therefore we simply enable ASLR only when an
optimization flag is present in the CFLAGS, using it as an indicator
that the developer does not want to debug in GDB anyway.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When testing a merge driver which spawns a merge server (for future merges)
I got the following error:
Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed. Should I try again? (y/n)
Only after I stop the merge server the lock is released.
This is caused by windows handle inheritance.
Starting childs with bInheritHandles==FALSE does not work,
because no file handles would be inherited,
not even the hStdXxx handles in STARTUPINFO.
Opening every file with O_NOINHERIT does not work,
Since it is used by git-upload-pack for example,
which expects inherited handles.
This leaves us with only creating temp files with the O_NOINHERIT flag.
Which (currently) only used by lock_file which is exactly what we want.
Signed-off-by: Ben Wijen <ben@wijen.net>
This test was added to verify the behaviour of merge
when a merge server is spawned.
Because file handles are inherited by child processes,
the index.lock is also inherited and - on win32 - can't
be released until the child has stopped.
Signed-off-by: Ben Wijen <ben@wijen.net>
Update tests for "http.extraHeaders=<header>" to be portable back
to Apache 2.2 (the original depended on <RequireAll/> which is a
more recent feature).
* js/http-custom-headers:
submodule: ensure that -c http.extraheader is heeded
t5551: make the test for extra HTTP headers more robust
tests: adjust the configuration for Apache 2.2
Rebased-and-signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch increases the precision of the version recorded in
the resources of the .exe files from major/minor to include also the
micro version and the patch level.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
To support this developer's use case of allowing build agents token-based
access to private repositories, we introduced the http.extraheader
feature, allowing extra HTTP headers to be sent along with every HTTP
request.
This patch verifies that we can configure these extra HTTP headers via the
command-line for use with `git submodule update`, too. Example: git -c
http.extraheader="Secret: Sauce" submodule update --init
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
To test that extra HTTP headers are passed correctly, t5551 verifies that
a fetch succeeds when two required headers are passed, and that the fetch
does not succeed when those headers are not passed.
However, this test would also succeed if the configuration required only
one header. As Apache's configuration is notoriously tricky (this
developer frequently requires StackOverflow's help to understand Apache's
documentation), especially when still supporting the 2.2 line, let's just
really make sure that the test verifies what we want it to verify.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lars Schneider noticed that the configuration introduced to test the
extra HTTP headers cannot be used with Apache 2.2 (which is still
actively maintained, as pointed out by Junio Hamano).
To let the tests pass with Apache 2.2 again, let's substitute the
offending <RequireAll> and `expr` by using old school RewriteCond
statements.
As RewriteCond does not allow testing for *non*-matches, we simply match
the desired case first and let it pass by marking the RewriteRule as
'[L]' ("last rule, do not process any other matching RewriteRules after
this"), and then have another RewriteRule that matches all other cases
and lets them fail via '[F]' ("fail").
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Tested-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We introduce a way to send custom HTTP headers with all requests.
This allows us, for example, to send an extra token from build agents
for temporary access to private repositories. (This is the use case that
triggered this patch.)
This feature can be used like this:
git -c http.extraheader='Secret: sssh!' fetch $URL $REF
Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only
a single list, overriding any previous call. This means we have to
collect _all_ of the headers we want to use into a single list, and
feed it to cURL in one shot. Since we already unconditionally set a
"pragma" header when initializing the curl handles, we can add our new
headers to that list.
For callers which override the default header list (like probe_rpc),
we provide `http_copy_default_headers()` so they can do the same
trick.
Big thanks to Jeff King and Junio Hamano for their outstanding help and
patient reviews.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:
1. It prevented obvious nonsense like using core.worktree
for multiple repos.
2. It could prevent surprise when the user did not mean
for the options to leak to the submodules (e.g.,
http.sslverify=false).
For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).
So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.
Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).
We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Before 14111fc (git: submodule honor -c credential.* from
command line, 2016-02-29), it was sufficient for code which
spawned a process in a submodule to just set the child
process's "env" field to "local_repo_env" to clear the
environment of any repo-specific variables.
That commit introduced a more complicated procedure, in
which we clear most variables but allow through sanitized
config. For C code, we used that procedure only for cloning,
but not for any of the programs spawned by submodule.c. As a
result, things like "git fetch --recurse-submodules" behave
differently than "git clone --recursive"; the former will
not pass through the sanitized config.
We can fix this by using prepare_submodule_repo_env()
everywhere in submodule.c.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
These functions should be used by any code which spawns a
submodule process, which may happen in submodule.c (e.g.,
for spawning fetch). Let's move them there and make them
public so that submodule--helper can continue to use them.
Since they're now public, let's also provide a basic overview
of their intended use.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Commit 14111fc (git: submodule honor -c credential.* from
command line, 2016-02-29) taught git-submodule.sh to save
the sanitized value of $GIT_CONFIG_PARAMETERS when clearing
the environment for a submodule. However, it failed to
export the result, meaning that it had no effect for any
sub-programs.
We didn't catch this in our initial tests because we checked
only the "clone" case, which does not go through the shell
script at all. Provoking "git submodule update" to do a
fetch demonstrates the bug.
Noticed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Right now we test only the cloning case, but there are other
interesting cases (e.g., fetching). Let's pull the setup
bits into their own test, which will make things flow more
logically once we start adding more tests which use the
setup.
Let's also introduce some whitespace to the clone-test to
split the two parts: making sure it fails without our
cmdline config, and that it succeeds with it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Commit 14111fc (git: submodule honor -c credential.* from
command line, 2016-02-29) accidentally wrote $HTTP_URL. It
happened to work because we ended up with "credential..helper",
which we treat the same as "credential.helper", applying it
to all URLs.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The "git -c var=value" option stuffs the config value into
$GIT_CONFIG_PARAMETERS, so that sub-processes can see it.
When the config is later read via git_config() or similar,
we parse it back out of that variable. The parsing end is a
little bit picky; it assumes that each entry was generated
with sq_quote_buf(), and that there is no extraneous
whitespace.
On the generating end, we are careful to append to an
existing $GIT_CONFIG_PARAMETERS variable if it exists.
However, our test for "should we add a space separator" is
too liberal: it will add one even if the environment
variable exists but is empty. As a result, you might end up
with:
GIT_CONFIG_PARAMETERS=" 'core.foo=bar'"
which the parser will choke on.
This was hard to trigger in older versions of git, since we
only set the variable when we had something to put into it
(though you could certainly trigger it manually). But since
14111fc (git: submodule honor -c credential.* from command
line, 2016-02-29), the submodule code will unconditionally
put the $GIT_CONFIG_PARAMETERS variable into the environment
of any operation in the submodule, whether it is empty or
not. So any of those operations which themselves use "git
-c" will generate the unparseable value and fail.
We can easily fix it by catching this case on the generating
side. While we're adding a test, let's also check that
multiple layers of "git -c" work, which was previously not
tested at all.
Reported-by: Shin Fan <shinfan@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Tested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Due to the way that the git-submodule code works, it clears all local
git environment variables before entering submodules. This is normally
a good thing since we want to clear settings such as GIT_WORKTREE and
other variables which would affect the operation of submodule commands.
However, GIT_CONFIG_PARAMETERS is special, and we actually do want to
preserve these settings. However, we do not want to preserve all
configuration as many things should be left specific to the parent
project.
Add a git submodule--helper function, sanitize-config, which shall be
used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs
except a small subset that are known to be safe and necessary.
Replace all the calls to clear_local_git_env with a wrapped function
that filters GIT_CONFIG_PARAMETERS using the new helper and then
restores it to the filtered subset after clearing the rest of the
environment.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
