Skip to content

CI: disable Boost.Test alternate signal stack on Alpine - #11065

Open
Al2Klimov wants to merge 1 commit into
masterfrom
BOOST_TEST_USE_ALT_STACK
Open

Al2Klimov wants to merge 1 commit into
masterfrom
BOOST_TEST_USE_ALT_STACK

Conversation

@Al2Klimov

@Al2Klimov Al2Klimov commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Boost.Test calls sigaltstack(2) with a stack of SIGSTKSZ bytes. musl defines SIGSTKSZ as a small constant which the kernel rejects with ENOMEM on CPUs with large signal frames (e.g. AVX-512/AMX). This made all test binaries fail with "Test setup error: system_error produced by: exp: Out of memory", already in the --generate_ctest_config build step.

closes #11053

Boost.Test calls sigaltstack(2) with a stack of SIGSTKSZ bytes. musl defines
SIGSTKSZ as a small constant which the kernel rejects with ENOMEM on CPUs with
large signal frames (e.g. AVX-512/AMX). This made all test binaries fail with
"Test setup error: system_error produced by: exp: Out of memory", already in
the --generate_ctest_config build step.
@Al2Klimov Al2Klimov added this to the 2.17.0 milestone Oct 1, 2026
@Al2Klimov Al2Klimov added the area/ci CI/CD label Oct 1, 2026
@cla-bot cla-bot Bot added the cla/signed label Oct 1, 2026
@Al2Klimov Al2Klimov added backport-to-support/2.15 PRs with this label will automatically be backported to the v2.15 support branch. backport-to-support/2.16 PRs with this label will automatically be backported to the v2.16 support branch. labels Oct 1, 2026
@jschmidt-icinga

jschmidt-icinga commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

So according to the boost.test issue this shouldn't be enough because it comes too late. I both retried the Alpine action here and in the other PR a few times and on both came up green repeatedly. Maybe Alpine changed something on their end, maybe GH did and now it doesn't fail anymore, but in any case I'd like to hold off on merging this until we can have a clear AB test.

@julianbrost

Copy link
Copy Markdown
Member

Maybe Alpine changed something on their end, maybe GH did and now it doesn't fail anymore

IIRC the tests were sometimes failing and fixed themselves at some point after a few retries. So given that this PR mentions AVX-512/AMX, it might be the case that the tests failed depending on the machine type the runner VM was running on (as in some have/expose AVX-512, others don't). So if that's indeed the cause, this could help to diagnose this locally (otherwise, one could add something like cat /proc/cpuinfo to the job, retry a few times over a few days and see if there are differences in there and if they correlate with CPU features).

@jschmidt-icinga

Copy link
Copy Markdown
Contributor

So if that's indeed the cause, this could help to diagnose this locally

The LLM (living itself in an Alpine container) already did that for me by faking the environment and also came to the conclusion that the fix in this PR isn't sufficient in that case because the alt stack is already set once before the environment option (or command-line argument) is even parsed.

One thing that does work is compiling the alt stack out entirely with a preprocessor define BOOST_TEST_DISABLE_ALT_STACK, but that only works when using the include version of the library, which we currently only do when the shared/static versions aren't available, so to allow that override we need to add a bit of CMake code to provide a way to force the usage of the header library and pass the define through to our test targets.

@julianbrost

Copy link
Copy Markdown
Member

@jschmidt-icinga I'm not exactly sure how to read your comment and whether it implies that you were able to reproduce that error locally.

@jschmidt-icinga

Copy link
Copy Markdown
Contributor

@jschmidt-icinga I'm not exactly sure how to read your comment and whether it implies that you were able to reproduce that error locally.

  1. I have a small shim library that returns ENOMEM on any call to sigaltstack() that tries to replace the existing stack, otherwise forwards the call.
  2. Then I load it with LD_PRELOAD and observe testbase failing with the same message we observe in our GHA (doesn't really matter if running actual tests or running test discovery).
  3. Then I use the environment variable from this PR to try to disable the alt stack for boost test but it still fails immediately with the same error we observe on the GHA.

Under the assumption that specifically AVX-512 capable GitHub runners fail, and always fail deterministically, not just sometimes, this reproduces the issue exactly and the PR does not fix it as reproduced via this method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci CI/CD backport-to-support/2.15 PRs with this label will automatically be backported to the v2.15 support branch. backport-to-support/2.16 PRs with this label will automatically be backported to the v2.16 support branch. cla/signed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants