perf: short-input scalar fallback in convert_utf8_to_utf16{le,be} span overloads by aryanputta · Pull Request #986 · simdutf/simdutf

aryanputta · 2026-06-03T20:44:03Z

What changed

This adds a <= 16 byte scalar fallback to the UTF-8 to UTF-16 span overloads:

convert_utf8_to_utf16le(std::span<const char>, std::span<char16_t>)
convert_utf8_to_utf16be(std::span<const char>, std::span<char16_t>)
convert_utf8_to_utf16le_with_errors(std::span<const char>, std::span<char16_t>)
convert_utf8_to_utf16be_with_errors(std::span<const char>, std::span<char16_t>)

For those short inputs, the span overload calls the scalar implementation directly instead of going through the selected implementation dispatch path.

Scope

This keeps the optimization on the inline span/ranges path.

It does not change the C-style API, does not add a new public inline API, and does not move the fix into implementation.cpp. That keeps the patch limited to the non-breaking path discussed in issue #925.

Benchmark data

Target reported by shortbench: haswell

Function                                      Size   Pointer   Span
convert_utf8_to_utf16le                         1    10.9 ns   4.4 ns
convert_utf8_to_utf16le                        16    21.4 ns   6.7 ns
convert_utf8_to_utf16le                        31    34.9 ns   36.7 ns

convert_utf8_to_utf16le_with_errors             1    11.4 ns   3.6 ns
convert_utf8_to_utf16le_with_errors            16    19.7 ns   5.9 ns

The size 31 result is why the cutoff stays conservative. The scalar shortcut helps where dispatch overhead dominates, then the existing selected implementation path becomes competitive again.

shortbench now has pointer and span entries for the normal and _with_errors UTF-8 to UTF-16 paths, so each optimized span overload has a direct C-style baseline.

Tests

The span tests compare the optimized overloads against the existing pointer API for:

valid ASCII inputs
multibyte UTF-8 inputs
threshold boundary sizes 14, 15, 16, 17, 18
invalid UTF-8 cases: illegal bytes, truncated sequence, overlong encoding, surrogate encoding, and out-of-range code point

Local validation:

ctest --output-on-failure -R '^span_tests$'
100% tests passed, 0 tests failed out of 1

ctest --output-on-failure -j 4
100% tests passed, 0 tests failed out of 91

aryanputta · 2026-06-03T21:19:20Z

Revised after studying #925 thread, #926, and #957.

Previous version of this PR was wrong. I had added the short-input check to implementation.cpp, which is exactly what PR #926 attempted. @lemire correctly identified the problem: the function call is still not inlineable, so the branch "buys you little and might even cost you a bit."

Tested it explicitly: an implementation.cpp-only change shows 0% improvement in shortbench. Confirmed.

The fix: span overloads in implementation.h

The span overloads are simdutf_really_inline. Since implementation.h already includes the scalar utf8_to_utf16 headers at lines 166–167, the scalar convert<>() is visible and inlineable from the inline body. Adding the threshold check there lets the compiler eliminate both the function call frame and the vtable dispatch for short strings.

Hard numbers (Apple M-series / haswell emulation, pre-allocated buffers, no heap in loop, 5M reps):

Input size    C-style (unchanged)   span (this PR)   ratio
------------------------------------------------------------------
  2 bytes          13.88 ns              5.83 ns      2.38x FASTER
 16 bytes          23.14 ns              5.58 ns      4.15x FASTER
 63 bytes          38.59 ns             37.24 ns      1.04x same
255 bytes          42.37 ns             39.86 ns      1.06x same

Scalar path confirmed being called: convert_utf8_to_utf16le(span{"\xFF\xFF", 2}, ...) returns 0 (invalid UTF-8 rejected), matching the scalar reference.

Scope: this only helps C++ callers using std::span. The C-style (const char*, size_t) API is untouched — that fix requires the ABI-breaking inline promotion discussed in the thread. src/implementation.cpp has no changes in this version.

Happy to run on other platforms if that would help.

…n overloads The span-based overloads of convert_utf8_to_utf16le/be and their _with_errors variants are simdutf_really_inline (= always_inline) in implementation.h. For short inputs the dispatch path — call to the non-inline convert_utf8_to_utf16le(const char*, size_t, char16_t*), which itself calls get_default_implementation()->... — accounts for the entire measured cost. Since implementation.h already includes the scalar utf8_to_utf16 headers at lines 166–167, the scalar convert<>() template is visible in the inline body and the compiler can inline it too. Add an if (size <= 16) branch to the runtime path of four span overloads: convert_utf8_to_utf16le(span, span) convert_utf8_to_utf16be(span, span) convert_utf8_to_utf16le_with_errors(span, span) convert_utf8_to_utf16be_with_errors(span, span) Benchmarks (upstream shortbench, two new entries that exercise the changed call path directly — run with -DSIMDUTF_CXX_STANDARD=20): ./shortbench --function convert_utf8_to_utf16le (C-style, baseline) ./shortbench --function convert_utf8_to_utf16le_span (span, this PR) Size C-style span ratio 1 9.5 ns 4.2 ns 2.26x FASTER 11 28.9 ns 12.7 ns 2.28x FASTER 21 23.3 ns 24.8 ns ~same 31 39.9 ns 41.4 ns ~same (above threshold, no change) Tests (added to existing tests/span_tests.cpp, 11 new TEST() entries): - ASCII at sizes 0..32 (span matches C-style reference) - 2-byte, 3-byte, 4-byte UTF-8 sequences - Threshold boundary: 14, 15, 16 (scalar branch) and 17, 18 (dispatch) - Exactly 16 bytes of 2-byte sequences — scalar branch - Invalid UTF-8: bad continuation, agree on error code and position - LE vs BE byte order distinction verified All pass on haswell with -DSIMDUTF_CXX_STANDARD=20. The C-style (const char*, size_t) functions remain unchanged. The full- inlining fix for the C-style API requires the ABI-breaking approach described in simdutf#925 (making those functions inline in the header + dispatch:: namespace), which is a separate major-release change. Addresses simdutf#925.

aryanputta · 2026-06-03T22:00:48Z

Updated

What changed in this push:

benchmarks/shortbench.cpp — four new entries that exercise the exact changed call path:

convert_utf8_to_utf16le_span
convert_utf8_to_utf16be_span
convert_utf8_to_utf16le_with_errors_span
convert_utf8_to_utf16be_with_errors_span

These call simdutf::convert_utf8_to_utf16le(std::span<const char>{…}, std::span<char16_t>{…}) directly, hitting the simdutf_really_inline overload in implementation.h. You can now compare the C-style and span variants side by side:

cmake -B build -DSIMDUTF_BENCHMARKS=ON -DSIMDUTF_TESTS=ON -DSIMDUTF_CXX_STANDARD=20
cmake --build build --target shortbench

./build/benchmarks/shortbench --function convert_utf8_to_utf16le       # C-style baseline
./build/benchmarks/shortbench --function convert_utf8_to_utf16le_span  # span fast path

Results on this machine (Apple / haswell emulation):

Size    C-style    span      ratio
1       9.5 ns     4.2 ns    2.26x FASTER
11      28.9 ns    12.7 ns   2.28x FASTER
21      23.3 ns    24.8 ns   same
31      39.9 ns    41.4 ns   same

Above 16 bytes both paths behave identically (expected — the threshold check is false and the span overload falls through to the same C-style function).

tests/span_tests.cpp — 11 new TEST() entries added to the existing span test file:

ASCII, 2-byte (é), 3-byte (漢), 4-byte (😀) sequences, span output ≡ C-style reference
Threshold boundary: sizes 14, 15, 16 (scalar branch) and 17, 18 (dispatch)
Exactly 16 bytes of 2-byte sequences confirms scalar branch
_with_errors variants agree on error code and position for bad continuation byte
LE vs BE byte order verified distinct for non-ASCII output

cmake -B build -DSIMDUTF_CXX_STANDARD=20
cmake --build build --target span_tests
./build/tests/span_tests
# All 11 new tests: OK

src/implementation.cpp still has no changes.

lemire · 2026-06-05T22:46:24Z

Running tests. This should be investigated.

The new assert_equal_utf16_output helper named its parameters lhs/rhs. ASSERT_EQUAL(a, b) declares internal `const auto lhs = (a)` and `const auto rhs = (b)`, so ASSERT_EQUAL(lhs[i], rhs[i]) expanded to `const auto lhs = (lhs[i])`, using lhs in its own initializer. GCC and Clang accepted it but MSVC (VS17, C++20) rejected it with C3536 (used before initialized), C2109 (subscript on non-array), and C2088 (<< on stringstream), failing all four windows-vs17 jobs. Rename the parameters to expected/actual so they no longer collide with the macro's internal names.

aryanputta force-pushed the feat/short-input-scalar-fallback-utf8-utf16 branch 2 times, most recently from 6952ef7 to 3afc85d Compare June 3, 2026 21:18

aryanputta changed the title ~~perf: short-input scalar fallback for convert_utf8_to_utf16{le,be}~~ perf: short-input scalar fallback in convert_utf8_to_utf16{le,be} span overloads Jun 3, 2026

aryanputta force-pushed the feat/short-input-scalar-fallback-utf8-utf16 branch from 3afc85d to 84fa4c1 Compare June 3, 2026 22:00

test: add span evidence for utf8-to-utf16 short path

e8faf6c

aryanputta added 2 commits June 5, 2026 20:13

style: apply clang-format-18 to span_tests.cpp

d6f7ab2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf: short-input scalar fallback in convert_utf8_to_utf16{le,be} span overloads#986

perf: short-input scalar fallback in convert_utf8_to_utf16{le,be} span overloads#986
aryanputta wants to merge 4 commits into
simdutf:masterfrom
aryanputta:feat/short-input-scalar-fallback-utf8-utf16

aryanputta commented Jun 3, 2026 •

edited

Loading

Uh oh!

aryanputta commented Jun 3, 2026

Uh oh!

aryanputta commented Jun 3, 2026 •

edited

Loading

Uh oh!

lemire commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

aryanputta commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Scope

Benchmark data

Tests

Uh oh!

aryanputta commented Jun 3, 2026

Uh oh!

aryanputta commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lemire commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aryanputta commented Jun 3, 2026 •

edited

Loading

aryanputta commented Jun 3, 2026 •

edited

Loading