Skip to content

Make j9localmap_DebugLocalBitsForPC the default mapper - #20569

Merged
gacholio merged 1 commit into
eclipse-openj9:masterfrom
babsingh:main13
Nov 12, 2024
Merged

Make j9localmap_DebugLocalBitsForPC the default mapper#20569
gacholio merged 1 commit into
eclipse-openj9:masterfrom
babsingh:main13

Conversation

@babsingh

Copy link
Copy Markdown
Contributor

j9localmap_LocalBitsForPC is more aggressive than
j9localmap_DebugLocalBitsForPC in the way it analyzes and marks local
variables as object references. Aggressiveness relates to determining
whether a local object is being used at a specific point in the
bytecode execution, and this helps optimize garbage collection and
runtime performance. In contrast, j9localmap_DebugLocalBitsForPC
adopts a conservative approach; it overestimates and plays safe by
marking objects alive even if they are not being used.

j9localmap_DebugLocalBitsForPC resolves the issue seen in
ibmruntimes/Semeru-Runtimes#93.

We have run the following benchmarks: Liberty DT8/DT10 Startup &
Footprint, JRuby and Nashorn. No performance difference is seen between
j9localmap_DebugLocalBitsForPC and j9localmap_LocalBitsForPC.

Since there are no side-effects of j9localmap_DebugLocalBitsForPC,
this PR makes it OpenJ9's default mapper.

j9localmap_LocalBitsForPC is more aggressive than
j9localmap_DebugLocalBitsForPC in the way it analyzes and marks local
variables as object references. Aggressiveness relates to determining
whether a local object is being used at a specific point in the
bytecode execution, and this helps optimize garbage collection and
runtime performance. In contrast, j9localmap_DebugLocalBitsForPC
adopts a conservative approach; it overestimates and plays safe by
marking objects alive even if they are not being used.

j9localmap_DebugLocalBitsForPC resolves the issue seen in
ibmruntimes/Semeru-Runtimes#93.

We have run the following benchmarks: Liberty DT8/DT10 Startup &
Footprint, JRuby and Nashorn. No performance difference is seen between
j9localmap_DebugLocalBitsForPC and j9localmap_LocalBitsForPC.

Since there are no side-effects of j9localmap_DebugLocalBitsForPC,
this PR makes it OpenJ9's default mapper.

Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
@babsingh
babsingh marked this pull request as ready for review November 12, 2024 01:11
@babsingh
babsingh requested a review from gacholio November 12, 2024 01:12
@babsingh

Copy link
Copy Markdown
Contributor Author

This PR includes a potentially high-risk change that may affect performance. I've run some benchmarks and haven't observed any performance regressions. To ensure ongoing monitoring, we’d like to alert the perf team to track the impact of these changes in the weekly performance builds after merging. @jdekonin, could you assist with this?

fyi @tajila

@gacholio

Copy link
Copy Markdown
Contributor

jenkins test sanity zlinux jdk21

@pshipton

Copy link
Copy Markdown
Member

Not sure they need alerting. I've seen it, and if a regression is reported that includes this change, I'll call it out.

@gacholio

Copy link
Copy Markdown
Contributor

Test failure is CRIU which would not be affected by this change.

@gacholio
gacholio merged commit a88773c into eclipse-openj9:master Nov 12, 2024
@pshipton

Copy link
Copy Markdown
Member

This is causing asserts, I'm reverting it.
Failures are http://vmfarm.rtp.raleigh.ibm.com/jobs_by_status.php?build_id=81473&status=FAILED&categories=17

j> 10:41:51 000001002A258100: Object neither in heap nor stack-allocated in thread Pooled Thread #141 running com.ibm.jtc.svt.tests.invoke.BasicTest2
j> 10:35:49 15:35:49.674 0x30125a00 j9mm.479 * ** ASSERTION FAILED ** at ../../gc_glue_java/MarkingSchemeRootMarker.cpp:53: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, object, stackLocation, walkState).validate(_env)))

j> 10:49:53 3D053000: Object neither in heap nor stack-allocated in thread Primary|SimpleDriver|com.ibm.jtc.svt.tests.invoke.AsTypeTest.testFloat|3344|Default Invocant
15:41:36.409 0x3001b100 j9mm.479 * ** ASSERTION FAILED ** at ../../../../gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))

000000000001E700: Object neither in heap nor stack-allocated in thread main
15:07:05.299 0x121100 j9mm.479 * ** ASSERTION FAILED ** at ../../../../gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))

000000003001B100: Invalid class pointer in stack allocated object in thread main
15:41:36.409 0x3001b100 j9mm.479 * ** ASSERTION FAILED ** at ../../../../gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))

@gacholio

Copy link
Copy Markdown
Contributor

This doesn't make much sense - if the debug mapper is so flawed, we would certainly have aeen problems in the field.

@pshipton

pshipton commented Nov 12, 2024

Copy link
Copy Markdown
Member

This was the only change in the build, so it seemed the cause. It failed enough that we should have seen it in earlier builds even if it's intermittent and caused by something else.

@pshipton

Copy link
Copy Markdown
Member

Is the JIT usually in a different state when the debug mapper is used?

@gacholio

gacholio commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

The mapper is only used for interpreted frames. The JIT maintains it's own metadata for compiled methods.

@gacholio

gacholio commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

Looking through the code, there is an interaction with DLT. If the problem can be reproduced, it may be worth disabling that in some runs to see if that's the issue.

@gacholio

Copy link
Copy Markdown
Contributor

I don't know if DLT is disabled for debug mode runs.

@pshipton

Copy link
Copy Markdown
Member

I assumed the JIT was involved since stack allocated objects are mentioned.

@gacholio

Copy link
Copy Markdown
Contributor

There won't be any stack-allocated objects in the DLT case.

@babsingh

Copy link
Copy Markdown
Contributor Author

@gacholio Do the below flags need to be set before using j9localmap_DebugLocalBitsForPC?

vm->requiredDebugAttributes |= J9VM_DEBUG_ATTRIBUTE_LOCAL_VARIABLE_TABLE;
vm->requiredDebugAttributes |= J9VM_DEBUG_ATTRIBUTE_CAN_ACCESS_LOCALS;
vm->localMapFunction = j9localmap_DebugLocalBitsForPC;

@gacholio

Copy link
Copy Markdown
Contributor

From the interpreter's POV. no. The CAN_ACCESS_LOCALS almost certainly impacts JIT code generation (I think it may actually force FSD). I still don't understand why conservative object lifetimes would cause problems, but I'm pretty sure that FSD disables stack allocation, so that would explain the difference in behaviours.

@gacholio

Copy link
Copy Markdown
Contributor

For the record, the difference in the local mappers is read vs write. The normal mapper looks forward from the target PC along all paths to see if the a slot is ever read again as an object. The debug mapper is the opposite - it looks from the entry points to the method forward to the PC to see what was stored in the slot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants