Skip to content

fix(redis): prevent defer callback accumulation in long-lived coroutines#7734

Merged
limingxinleo merged 11 commits into
hyperf:masterfrom
suyar:fix/redis-defer-memory-leak
Apr 21, 2026
Merged

fix(redis): prevent defer callback accumulation in long-lived coroutines#7734
limingxinleo merged 11 commits into
hyperf:masterfrom
suyar:fix/redis-defer-memory-leak

Conversation

@suyar
Copy link
Copy Markdown
Contributor

@suyar suyar commented Apr 5, 2026

fix: #7733

suyar and others added 3 commits April 5, 2026 18:43
When `pipeline(callback)` or `transaction(callback)` is called repeatedly
in a long-lived coroutine (e.g. while(true) loop), each call registers a
new `defer` closure via `__call()`, but the outer `executeMultiExec()`
finally block immediately clears the context connection. Since
`Context::has()` uses `isset()` which returns false for null values, the
next call sees no existing connection and registers yet another defer.

These defer closures accumulate indefinitely because the coroutine never
terminates, causing a slow memory leak.

Fix: Move the `Context::has()` check into the finally block so it
evaluates after `__call()` has stored the connection. This way the
connection stays in context and is reused on subsequent calls, with only
one defer ever registered per coroutine.

Made-with: Cursor
huangdijia
huangdijia previously approved these changes Apr 13, 2026
@xuanyanwow
Copy link
Copy Markdown
Member

xuanyanwow commented Apr 13, 2026

此改动 如果是一个while true协程 多次调用 transaction/pipeline 没有问题,在defer才释放redis;不会多次注册defer函数;
但在业务逻辑中,会导致redis无及时释放,pool饿死(如CI/CD test不通过的场景)

@suyar
Copy link
Copy Markdown
Contributor Author

suyar commented Apr 13, 2026

此改动 如果是一个while true协程 多次调用 transaction/pipeline 没有问题,在defer才释放redis;不会多次注册defer函数; 但在业务逻辑中,会导致redis无及时释放,pool饿死(如CI/CD test不通过的场景)

确实存在这个问题。

我看了单测的代码,3个连接,但是启动了20个协程并发,因为连接的释放是在defer中才释放,而不是在每次命令执行完成之后释放,所以连接池会耗尽。

看了下连接复用是想要在 pipelinemulti 等场景中使用同一个链接,Redis::__call 内部的复用应该放到外层来。

我先看看代码,大佬们看下有没有什么建议。

@xuanyanwow
Copy link
Copy Markdown
Member

我的思路是在注册defer处做一个标记,不要多次注册即可 可以讨论看看有无其他更优思路

  • 类属性上
  • 上下文标记
image

@suyar
Copy link
Copy Markdown
Contributor Author

suyar commented Apr 13, 2026

我想了一下,如果还是放 defer 释放,那么只要连接复用生效,那么多个协程并发仍然会出现连接池耗尽的情况。之前之所以能通过测试用例,就是每次命令执行完毕之后,链接都释放了。

如此说来,需要在每次执行完毕后,立即归还链接。但是我没想好怎么改,以及影响范围。

顺便mark下,defer泄漏由此次优化引入:#7394

@suyar
Copy link
Copy Markdown
Contributor Author

suyar commented Apr 13, 2026

使用 Context 标记了当前 pipeline 的执行方式,如果是 callback 方式执行的,则不需要设置 defer 来释放连接。
麻烦帮忙再 review 下代码。

Comment thread src/redis/src/Redis.php Outdated
huangdijia
huangdijia previously approved these changes Apr 13, 2026
@limingxinleo
Copy link
Copy Markdown
Member

代码我合进来了,你测试下,我改了一行代码。

@limingxinleo limingxinleo merged commit ce49c50 into hyperf:master Apr 21, 2026
@suyar
Copy link
Copy Markdown
Contributor Author

suyar commented Apr 21, 2026

测试了下,没问题

@suyar suyar deleted the fix/redis-defer-memory-leak branch April 23, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Memory leak: defer callbacks accumulate in long-lived coroutines when using pipeline(callback) / transaction(callback)

4 participants