errors thrown in describe() do not set non-zero exitCode (and do not fail CI) #58687
Description
Version
v24.2.0
Platform
Darwin 24.4.0 Darwin Kernel Version 24.4.0: Fri Apr 11 18:34:14 PDT 2025; root:xnu-11417.101.15~117/RELEASE_ARM64_T8122 arm64
Subsystem
test
What steps will reproduce the bug?
import { describe, test } from 'node:test'
import assert from 'node:assert/strict'
describe('green', () => {
test('one', () => {
assert(1 === 1)
})
})
describe('red', () => {
test_('one', () => {
assert(1 === 1)
})
test('two', () => {
assert(1 === 2)
})
})node --test foo.js; echo $?
How often does it reproduce? Is there a required condition?
Every time
What is the expected behavior? Why is that the expected behavior?
A test failure printed and a non-zero exit code upon test termination
What do you see instead?
A test failure printed but the exit code is zero, the test is effectively treated as passing by tooling / CI
Additional information
A work-around using a custom reporter is possible, see ExodusOSS/test@7493701
Activity
fix: ensure we set non-0 exit code on errors node:test describe()
ChALkeR commentedon Feb 26, 2026
cc @nodejs/test_runner ?
JakobJingleheimer commentedon Feb 26, 2026
Is this perhaps caused by mixing
testwithdescribe? I think I've not tried to do that; I'm not sure it's supported.ChALkeR commentedon Feb 26, 2026
@JakobJingleheimer No, this assumes legitimate usecase.
E.g. assume some lib uses
node:testfor testsAnd has some preparation steps in
descripe, e.g. a function call or a library import. Or test data creation.Or e.g.
mock.module(call (which Node.js tests do:)That was not expected to throw
And then as a result of or a regression in that lib that code begins to throw (e.g. a file gets renamed and
mock.modulecan't find it anymore)But
node:testin CI will mark it as passing because ifdescribe()itself throws and no tests are run, that counts as passingEven though is reported as failing in the logs.
JakobJingleheimer commentedon Feb 26, 2026
There is no preparation in the reproduction in your OP. Could you please update your OP to reflect what you just described.
ChALkeR commentedon Feb 26, 2026
@JakobJingleheimer There is mistype in
test_. Which demonstrates this just fine once you run it?The test fails but is reported as passing.
Tests should not be marked as passing when the user makes a mistype in
test(.Any error during
describe(reproduces itI'm unsure how adding more examples (of various errors that could happen during describe() under attempted normal usage) to the demo would help if the issue should be clear as is