Since v2.9.0, Git knows about the config variable core.hookspath
that allows overriding the path to the directory containing the
Git hooks.
Since v2.10.0, the `--git-path` option respects that config
variable, too, so we may just as well use that command.
For Git versions older than v2.5.0 (which was the first version to
support the `--git-path` option for the `rev-parse` command), we
simply fall back to the previous code.
This fixes https://github.com/git-for-windows/git/issues/1755
Initial-patch-by: Philipp Gortan <philipp@gortan.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Make use of the new environment variable GIT_ASK_YESNO to support the
recently implemented fallback in case unlink, rename or rmdir fail for
files in use on Windows. The added dialog will present a yes/no question
to the the user which will currently be used by the windows compat layer
to let the user retry a failed file operation.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
"git branch -c/-m new old" was not described to copy config, which
has been corrected.
* jc/branch-copy-doc:
branch (doc): -m/-c copies config and reflog
Consistently use 'directory', not 'folder', to call the filesystem
entity that collects a group of files and, eh, directories.
* ma/doc-folder-to-directory:
gitweb.txt: change "folder" to "directory"
gitignore.txt: change "folder" to "directory"
git-multi-pack-index.txt: change "folder" to "directory"
Drop "git sparse-index" from the list of common commands.
* sg/sparse-index-not-that-common-a-command:
command-list.txt: remove 'sparse-index' from main help
Update "git archive" documentation and give explicit mention on the
compression level for both zip and tar.gz format.
* bs/archive-doc-compression-level:
archive: describe compression level option
Message regression fix.
* ks/submodule-add-message-fix:
submodule: drop unused sm_name parameter from append_fetch_remotes()
submodule--helper: fix incorrect newlines in an error message
Leakfix.
* ab/plug-random-leaks:
reflog: free() ref given to us by dwim_log()
submodule--helper: fix small memory leaks
clone: fix a memory leak of the "git_dir" variable
grep: fix a "path_list" memory leak
grep: use object_array_clear() in cmd_grep()
grep: prefer "struct grep_opt" over its "void *" equivalent
"git for-each-ref" family of commands were leaking the ref_sorting
instances that hold sorting keys specified by the user; this has
been corrected.
* ab/ref-filter-leakfix:
branch: use ref_sorting_release()
ref-filter API user: add and use a ref_sorting_release()
tag: use a "goto cleanup" pattern, leak less memory
"git push" client talking to an HTTP server did not diagnose the
lack of the final status report from the other side correctly,
which has been corrected.
* jk/http-push-status-fix:
transport-helper: recognize "expecting report" error from send-pack
send-pack: complain about "expecting report" with --helper-status
A new feature has been added to abort early in the test framework.
* ab/test-bail:
test-lib.sh: use "Bail out!" syntax on bad SANITIZE=leak use
test-lib.sh: de-duplicate error() teardown code
We already note that we may produce invalid output when we skip calling
iconv() altogether. But we may also do so if iconv() fails, and we have
no good alternative. Let's document this to avoid surprising users.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit fd680bc5 (logmsg_reencode(): warn when iconv()
fails, 2021-08-27). Throwing a warning for each and every commit
that gets reencoded, without allowing a way to squelch, would make
it unpleasant for folks who have to deal with an ancient part of the
history in an old project that used wrong encoding in the commits.
The filter system allows for alterations to file contents when they're
moved between the database and the worktree. We already made sure that
it is possible for smudge filters to produce contents that are larger
than `unsigned long` can represent (which matters on systems where
`unsigned long` is narrower than `size_t`, most notably 64-bit Windows).
Now we make sure that clean filters can _consume_ contents that are
larger than that.
Note that this commit only allows clean filters' _input_ to be larger
than can be represented by `unsigned long`.
This change makes only a very minute dent into the much larger project
to teach Git to use `size_t` instead of `unsigned long` wherever
appropriate.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Matt Cooper <vtbassmatt@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This introduces an additional guard for platforms where `unsigned long`
and `size_t` are not of the same size. If the size of an object in the
database would overflow `unsigned long`, instead we now exit with an
error.
A complete fix will have to update _many_ other functions throughout the
codebase to use `size_t` instead of `unsigned long`. It will have to be
implemented at some stage.
This commit puts in a stop-gap for the time being.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Matt Cooper <vtbassmatt@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
There is mixed use of size_t and unsigned long to deal with sizes in the
codebase. Recall that Windows defines unsigned long as 32 bits even on
64-bit platforms, meaning that converting size_t to unsigned long narrows
the range. This mostly doesn't cause a problem since Git rarely deals
with files larger than 2^32 bytes.
But adjunct systems such as Git LFS, which use smudge/clean filters to
keep huge files out of the repository, may have huge file contents passed
through some of the functions in entry.c and convert.c. On Windows, this
results in a truncated file being written to the workdir. I traced this to
one specific use of unsigned long in write_entry (and a similar instance
in write_pc_item_to_fd for parallel checkout). That appeared to be for
the call to read_blob_entry, which expects a pointer to unsigned long.
By altering the signature of read_blob_entry to expect a size_t,
write_entry can be switched to use size_t internally (which all of its
callers and most of its callees already used). To avoid touching dozens of
additional files, read_blob_entry uses a local unsigned long to call a
chain of functions which aren't prepared to accept size_t.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Matt Cooper <vtbassmatt@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The filter system allows for alterations to file contents when they're
added to the database or workdir. ("Smudge" when moving to the workdir;
"clean" when moving to the database.) This is used natively to handle CRLF
to LF conversions. It's also employed by Git-LFS to replace large files
from the workdir with small tracking files in the repo and vice versa.
Git pulls the entire smudged file into memory. While this is inefficient,
there's a more insidious problem on some platforms due to inconsistency
between using unsigned long and size_t for the same type of data (size of
a file in bytes). On most 64-bit platforms, unsigned long is 64 bits, and
size_t is typedef'd to unsigned long. On Windows, however, unsigned long is
only 32 bits (and therefore on 64-bit Windows, size_t is typedef'd to
unsigned long long in order to be 64 bits).
Practically speaking, this means 64-bit Windows users of Git-LFS can't
handle files larger than 2^32 bytes. Other 64-bit platforms don't suffer
this limitation.
This commit introduces a test exposing the issue; future commits make it
pass. The test simulates the way Git-LFS works by having a tiny file
checked into the repository and expanding it to a huge file on checkout.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Matt Cooper <vtbassmatt@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Allow tests that assume a 64-bit `size_t` to be skipped in 32-bit
platforms and regardless of the size of `long`.
This imitates the `LONG_IS_64BIT` prerequisite.
Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In this developer's tests, producing one gigabyte worth of NULs in a
busy loop that writes out individual bytes, unbuffered, took ~27sec.
Writing chunked 256kB buffers instead only took ~0.6sec
This matters because we are about to introduce a pair of test cases that
want to be able to produce 5GB of NULs, and we cannot use `/dev/zero`
because of the HP NonStop platform's lack of support for that device.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
d5cfd142ec (tests: teach the test-tool to generate NUL bytes and
use it, 2019-02-14), add a way to generate zeroes in a portable
way without using /dev/zero (needed by HP NonStop), but uses a
long variable that is limited to 2^31 in Windows.
Use instead a (POSIX/C99) intmax_t that is at least 64bit wide
in 64-bit Windows to use in a future test.
Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In the basic `[`/`test` command, the string equality operator is a
single `=`. The `==` operator is only available in `[[`, which is a
bash-ism also supported by zsh.
This mix-up was causing the following completion error in zsh:
> __git_ls_files_helper:7: = not found
(That message refers to the extraneous symbol in `==` ← `=`).
This updates that comparison to use a single `=` inside the
basic `[ … ]` test conditional.
Although this fix is inconsistent with the other comparisons in this
file, which use `[[ … == … ]]`, and the two expressions are functionally
identical in this context, that approach was rejected due to a
preference for `[`.
Signed-off-by: Robert Estelle <robertestelle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that we have a `batch` mode, let's be explicit.
This is a follow-up to ce4786fc77 (mingw: change core.fsyncObjectFiles
= 1 by default, 2017-09-04) and will most likely have to be squashed
into it before upstreaming that patch (after the `batch` fsync mode was
upstreamed).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This merges the topic branch (specifically backported onto v2.33.1 to
allow for integrating into Git for Windows' `main` branch) that strikes
a better balance between safety and speed: rather than `fsync()`ing each
and every loose object file, we now offer to do it in a batch.
This will become the new default in Git for Windows.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `xutftowcs_path` function canonicalizes absolute paths using GetFullPathNameW.
This canonicalization may change the length of the string (e.g. getting rid of \.\),
which breaks callers that pass the template string in a strbuf and expect the
length of the string to remain the same.
In my particular case, the tmp-objdir code is passing a strbuf to mkdtemp and is
breaking since the strbuf.len is no longer synchronized with strlen(strbuf.buf).
Signed-off-by: Neeraj K. Singh <neerajsi@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add a basic performance test for "git add" and "git stash" of a lot of
new objects with various fsync settings.
Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add test cases to exercise batch mode for:
* 'git add'
* 'git stash'
* 'git update-index'
* 'git unpack-objects'
These tests ensure that the added data winds up in the object database.
In this change we introduce a new test helper lib-unique-files.sh. The
goal of this library is to create a tree of files that have different
oids from any other files that may have been created in the current test
repo. This helps us avoid missing validation of an object being added due
to it already being in the repo.
Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The unpack-objects functionality is used by fetch, push, and fast-import
to turn the transfered data into object database entries when there are
fewer objects than the 'unpacklimit' setting.
By enabling bulk-checkin when unpacking objects, we can take advantage
of batched fsyncs.
Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The update-index functionality is used internally by 'git stash push' to
setup the internal stashed commit.
This change enables bulk-checkin for update-index infrastructure to
speed up adding new objects to the object database by leveraging the
pack functionality and the new bulk-fsync functionality.
There is some risk with this change, since under batch fsync, the object
files will not be available until the update-index is entirely complete.
This usage is unlikely, since any tool invoking update-index and
expecting to see objects would have to synchronize with the update-index
process after passing it a file path.
Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit adds a win32 implementation for fsync_no_flush that is
called git_fsync. The 'NtFlushBuffersFileEx' function being called is
available since Windows 8. If the function is not available, we
return -1 and Git falls back to doing a full fsync.
The operating system is told to flush data only without a hardware
flush primitive. A later full fsync will cause the metadata log
to be flushed and then the disk cache to be flushed on NTFS and
ReFS. Other filesystems will treat this as a full flush operation.
I added a new file here for this system call so as not to conflict with
downstream changes in the git-for-windows repository related to fscache.
Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
