SEC: Detect multi-hop cyclic /Pages trees in _flatten to prevent SIGSEGV - #3847
stefan6419846 merged 5 commits into
Conversation
_flatten() only checked if a /Pages kid directly referenced its own
parent, missing multi-hop cycles like A→B→C→A. Such cycles caused
infinite recursion that exhausted the C stack before Python could
raise RecursionError, resulting in SIGSEGV in production environments
(e.g. gunicorn workers with partially consumed stacks).
Add a `_seen_refs` set that tracks every /Pages indirect reference
visited during traversal. Any ref seen a second time raises
PdfReadError("Detected cyclic page references.") immediately, without
relying on Python's recursion limit as a backstop.
The existing test for the size-limit case now matches the new (earlier)
error because the NullObject it produces also leads to a cycle; the
match pattern is relaxed to accept both messages.
Fixes #XXXX
stefan6419846
left a comment
There was a problem hiding this comment.
Thanks for the report and PR.
Apart from the failing CI checks, I have added some inline remarks.
- Rename parameter `_seen_refs` to `visited` and its type to `Optional[set[int]]` for consistency with the rest of _doc_common.py - Replace `(ref.idnum, ref.generation)` with `id(obj)` for the same reason: all other cycle-detection loops in this file use id() - Revert manual CHANGELOG edit (updated automatically on release)
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3847 +/- ##
=======================================
Coverage 97.79% 97.79%
=======================================
Files 55 55
Lines 10502 10505 +3
Branches 1954 1956 +2
=======================================
+ Hits 10270 10273 +3
Misses 128 128
Partials 104 104 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-authored-by: Stefan <96178532+stefan6419846@users.noreply.github.com>
|
I was adding an important co-author along with the requested changes. However, it was merged before I had the chance to push. Could you possibly add him so he receives the well deserved credits?
|
|
Sorry, I have not been aware that you still planned to do changes to this PR (I usually consider all non-draft PRs as possibly ready to be reviewed and merged). Due to the size of the project, I want to avoid doing forced pushes to fix the latest commit. I am aware that it is not directly equivalent to having a "regular" commit authored, but I am open to merging a PR which adds the relevant persons to https://github.com/py-pdf/pypdf/blob/main/CONTRIBUTORS.md (where the commit could use the co-authoring approach). |
## What's new ### Security (SEC) - Detect multi-hop cyclic /Pages trees in _flatten to prevent SIGSEGV (#3847) by @fredericoschardong ### Robustness (ROB) - Fix UnboundLocalError in _read_standard_xref_table on a malformed entry (#3841) by @joszamama - Raise PdfStreamError on non-hexadecimal bytes in hex readers (#3832) by @metsw24-max [Full Changelog](6.13.1...6.13.2)
Summary
_flatten()only checked if a/Pageskid directly referenced its own parent node (the single-parent check on line 1219). This missed multi-hop cycles such as A → B → C → A, where no node is its own immediate child.A malformed PDF with such a cycle causes
_flatten()to recurse until Python'sRecursionErroris raised. In production environments with partially-consumed C stacks (e.g. gunicorn workers), the native stack overflows before Python can intercept the recursion, resulting in SIGSEGV and a dead worker process — not a catchable Python exception.Root cause
len(reader.pages)on this PDF triggers the infinite recursion. The existingexcept RecursionErrorblock is an unreliable backstop: it works in an isolated Python process but fails under a heavily-framed call stack.Fix
Add a
_seen_refs: setparameter to_flatten()that accumulates the(idnum, generation)of every/Pagesindirect reference visited during traversal. Any reference seen a second time raisesPdfReadError("Detected cyclic page references.")immediately, without relying on Python's recursion limit.Test
test_cyclic_pages_treeconstructs a minimal in-memory PDF reproducing the exact cycle above and assertsPdfReadErroris raised when accessingreader.pages. The existingtest_get_object_from_stream__size_limittest's match pattern was relaxed because that broken PDF also produces a cycle that is now detected earlier.Reproduction