git-for-windows/git
1
0
Fork 0
mirror of https://github.com/git-for-windows/git.git synced 2026-05-03 02:27:35 -05:00
Code Issues Packages Projects Releases 335 Wiki Activity

t: work around multibyte bug in quoted heredocs with Dash v0.5.13

Browse Source 
When executing our test suite with Dash v0.5.13.2 one can observe
several test failures that all have the same symptoms: we have a quoted
heredoc that contains multibyte characters, but the final data does not
match what we actually wanted to write. One such example is in t0300,
where we see the diffs like the following:

  --- expect-stdout	2026-04-01 07:25:45.249919440 +0000
  +++ stdout	2026-04-01 07:25:45.254919509 +0000
  @@ -1,5 +1,5 @@
   protocol=https
   host=example.com
  -path=perú.git
  +path=perú.git
   username=foo
   password=bar

While seemingly the same, the data that we've written via the heredoc
contains some invisible bytes. The expected hex representation of the
string is:

  7065 72c3 ba2e 6769 74                 per...git

But what we actually get instead is this string:

  7065 7285 02c3 ba02 852e 6769 74       per.......git

What's important to note here is that the multibyte character exists in
both versions. But in the broken version we see that the bytes are
wrapped in a sequence of "85 02" and "02 85". This is the CTLMBCHAR byte
sequence of Dash, which it uses internally to quote multibyte sequences.

As it turns out, this bug was introduced in c5bf970 (expand: Add
multi-byte support to pmatch, 2024-06-02), which adds multibyte support
to more contexts of Dash. One of these contexts seems to be in heredocs,
and Dash _does_ correctly unquote these multibyte sequences when using
an unquoted heredoc. But the bug seems to be that this unquoting does
not happen in quoted heredocs, and the bug still exists on the latest
"master" branch.

For now, work around the bug by using unquoted heredocs instead.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit is contained in:
Patrick Steinhardt
2026-04-02 08:51:18 +02:00
committed by Junio C Hamano
parent 67ad42147a
commit 0c8424c259
6 changed files with 40 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
t/t0300-credentials.sh
View File

@@ -675,7 +675,9 @@ test_expect_success 'match percent-encoded values' '
test_expect_success 'match percent-encoded UTF-8 values in path' '
test_config credential.https://example.com.useHttpPath true &&
test_config credential.https://example.com/perú.git.helper "$HELPER" &&
check fill <<-\EOF
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
check fill <<-EOF
url=https://example.com/per%C3%BA.git
--
protocol=https

6
t/t3430-rebase-merges.sh
View File

@@ -507,9 +507,11 @@ test_expect_success 'octopus merges' '
git rebase -i --force-rebase -r HEAD^^ &&
test "Hank" = "$(git show -s --format=%an HEAD)" &&
test "$before" != $(git rev-parse HEAD) &&
test_cmp_graph HEAD^^.. <<-\EOF
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
test_cmp_graph HEAD^^.. <<-EOF
*-. Tüntenfüsch
|\ \
|\\ \\
| | * three
| * | two
| |/

16
t/t3902-quoted.sh
View File

@@ -60,16 +60,18 @@ With SP in it
"\346\277\261\351\207\216\347\264\224"
EOF
cat >expect.raw <<\EOF
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >expect.raw <<EOF
Name
"Name and a\nLF"
"Name and an\tHT"
"Name\""
"Name and a\\nLF"
"Name and an\\tHT"
"Name\\""
With SP in it
"濱野\t純"
"濱野\n純"
"濱野\\t純"
"濱野\\n純"
濱野 純
"濱野\"純"
"濱野\\"純"
濱野/file
濱野純
EOF

16
t/t4014-format-patch.sh
View File

@@ -1285,7 +1285,9 @@ test_expect_success 'format-patch wraps extremely long from-header (rfc2047)' '
check_author "Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar"
'
cat >expect <<'EOF'
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >expect <<EOF
From: Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar
Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo
Bar Foo Bar Foo Bar Foo Bar <author@example.com>
@@ -1300,7 +1302,9 @@ test_expect_success 'format-patch wraps extremely long from-header (non-ASCII wi
test_cmp expect actual
'
cat >expect <<'EOF'
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >expect <<EOF
Subject: [PATCH] Foö
EOF
test_expect_success 'subject lines are unencoded with --no-encode-email-headers' '
@@ -1312,7 +1316,9 @@ test_expect_success 'subject lines are unencoded with --no-encode-email-headers'
test_cmp expect actual
'
cat >expect <<'EOF'
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >expect <<EOF
Subject: [PATCH] Foö
EOF
test_expect_success 'subject lines are unencoded with format.encodeEmailHeaders=false' '
@@ -1523,7 +1529,9 @@ test_expect_success 'in-body headers trigger content encoding' '
test_env GIT_AUTHOR_NAME="éxötìc" test_commit exotic &&
test_when_finished "git reset --hard HEAD^" &&
git format-patch -1 --stdout --from >patch &&
cat >expect <<-\EOF &&
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >expect <<-EOF &&
From: C O Mitter <committer@example.com>
Content-Type: text/plain; charset=UTF-8

4
t/t4201-shortlog.sh
View File

@@ -105,7 +105,9 @@ test_expect_success 'output from user-defined format is re-wrapped' '
'
test_expect_success !MINGW,ICONV 'shortlog wrapping' '
cat >expect <<\EOF &&
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >expect <<EOF &&
A U Thor (5):
Test
This is a very, very long first line for the commit message to see if

12
t/t9001-send-email.sh
View File

@@ -1649,7 +1649,9 @@ test_expect_success $PREREQ 'To headers from files reset each patch' '
'
test_expect_success $PREREQ 'setup expect' '
cat >email-using-8bit <<\EOF
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >email-using-8bit <<EOF
From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
Message-ID: <bogus-message-id@example.com>
From: author@example.com
@@ -1735,7 +1737,9 @@ test_expect_success $PREREQ '--8bit-encoding overrides sendemail.8bitEncoding' '
'
test_expect_success $PREREQ 'setup expect' '
cat >email-using-8bit <<-\EOF
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >email-using-8bit <<-EOF
From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
Message-ID: <bogus-message-id@example.com>
From: author@example.com
@@ -1764,7 +1768,9 @@ test_expect_success $PREREQ '--8bit-encoding also treats subject' '
'
test_expect_success $PREREQ 'setup expect' '
cat >email-using-8bit <<-\EOF
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
# that contain multibyte chars.
cat >email-using-8bit <<-EOF
From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
Message-ID: <bogus-message-id@example.com>
From: A U Thor <author@example.com>