Generating upcalls fills up java CodeHeap and can lead to OutOfMemoryException #223
Labels
No labels
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
java
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
java-gi/java-gi#223
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This one is a bit more invasive, and I'm a bit out of my depth for fixing it myself.
More easily reproduced when setting a smaller codeheap size, using options like
-XX:NonProfiledCodeHeapSize=10M -XX:ProfiledCodeHeapSize=10M -XX:NonNMethodCodeHeapSize=8MThe individual values as they change can be viewed in the Memory tab of a JMX session like JDK Mission Control (jmc). When all 3 fill up (CodeHeap 'profiled nmethods', CodeHeap 'non-nmethods', and CodeHeap 'non-profiled nmethods'), an OutOfMemoryException will be thrown.
Classes extending
io.github.jwharm.javagi.base.FunctionPointerlikeorg.gnome.glib.SourceFuncrequire memory in the CodeHeap whenever they create a new upcall stub. Instead of creating a new upcall stub for call to functions likeGLib.idleAdd, a generic static upcall stub can be created and the MemorySegment passed to the upcall can be used to reference the actual SourceFunc, similar to howArenas.close_cbworks.I've tested that manually replacing calls to GLib.idleAdd with this and avoid the OutOfMemoryException:
Your proposed solution should work, but it will require large changes in a very complex part of the code generator. I'd really prefer not to go there... And I don't see why the current implementation wouldn't work:
GLib.idleAddcallsg_idle_add_full. Looking at the GIR file, the SourceFunc parameter has "notified" scope. This means the "notify" callback parameter is called immediately after the SourceFunc has finished.Arenas.CLOSE_CB_SYM, and it closes the arena of SourceFunc's upcall stub.I think the logic is sound, so I can't really explain your OOM exception. Can you investigate if the
CLOSE_CB_SYMis run for your SourceFuncs?I can take another look to reconfirm. I believe that
CLOSE_CB_SYMis run properly because I had regular memory issues when dealing with lambdas rather than passing in static classes as SourceFunc. Regular heap usage isn't increasing significantly -- it's the CodeHeap specifically, which by default on my machine has maximums of 116MiB 'profiled-nmethods', 7.25MiB 'non-nmethods', and 116MiB 'non-profiled nmethods'. It normally takes much longer to fill those, which is why I shared the options for limiting them. I also ran with-XX:+ClassUnloading -XX:+UseCodeCacheFlushingin hopes that those caches would clear up, but they didn't.Can you provide a minimal reproducable testcase so I can investigate?
I'm working on a testcase now. I have one that doesn't exhibit this problem, so it looks like this may be the symptom of a different issue and not leaking generally. I'm comparing to my failing case to see where it diverges.
I should be able to isolate it in the next couple of days.
Running this with
-XX:NonProfiledCodeHeapSize=10M -XX:ProfiledCodeHeapSize=10M -XX:NonNMethodCodeHeapSize=8Mwill complain about a full CodeHeap within 10 minutes. Commenting outdraw.invalidateContents()stays stable. AGLib.idleAdddoesn't permanently add to the CodeHeap, but will fail being unable to allocate the upcall after the CodeHeap is full. I didn't hit this issue when I forgot to add theFixedto theWindow.I'm able to reproduce the issue, but haven't found what is clogging the CodeHeap yet. I used
jcmdto repeatedly dump the size and contents of the CodeHeap into a text file until the OOM occured, but I didn't see anything out of the ordinary. The amount of free space fluctuates a bit, but doesn't show a clear trend, until after 8-10 minutes it suddenly goes to zero.