Skip to content

Add JMH benchmark for introspection with directives support#4261

Open
andimarek wants to merge 1 commit intomasterfrom
introspection-with-directives-benchmark
Open

Add JMH benchmark for introspection with directives support#4261
andimarek wants to merge 1 commit intomasterfrom
introspection-with-directives-benchmark

Conversation

@andimarek
Copy link
Member

@andimarek andimarek commented Feb 23, 2026

Summary

  • New IntrospectionWithDirectivesBenchmark that measures introspection performance when IntrospectionWithDirectivesSupport is enabled
  • Uses a full introspection query that fetches appliedDirectives { name args { name value } } on __Schema, __Type, __Field, __EnumValue, and __InputValue
  • Same 5 schemas and JMH settings as IntrospectionBenchmark for easy comparison

Baseline results (1 warmup, 1 iteration, 1 fork)

Schema With Directives (ms/op) Without Directives (ms/op)
large-schema-2 24.5 22.6
large-schema-3 83.2 79.2
large-schema-4 511.1 331.7
large-schema-5 839.8 388.9
large-schema-federated-1 680.2 614.1

("Without Directives" numbers from IntrospectionBenchmark PR #4260 for reference)

Test plan

  • ./gradlew jmhClasses compiles successfully
  • Full benchmark run produces 10 rows (5 schemas x 2 modes)

🤖 Generated with Claude Code

@andimarek @claude
Add JMH benchmark for introspection with directives support
5fae331 
Benchmark introspection queries when IntrospectionWithDirectivesSupport
is enabled and appliedDirectives are fetched on all introspection types.
This helps measure the overhead of directive introspection vs standard.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Feb 23, 2026 •
edited
Loading

Test Results

  335 files  ±0    335 suites  ±0   5m 6s ⏱️ ±0s
5 378 tests ±0  5 370 ✅ ±0  8 💤 ±0  0 ❌ ±0 
5 467 runs  ±0  5 459 ✅ ±0  8 💤 ±0  0 ❌ ±0 

Results for commit 5fae331. ± Comparison against base commit 30e0cb4.

This pull request removes 127 and adds 103 tests. Note that renamed tests count towards both. 
	?

	, expected: combo-\"\\\b\f\n\r\t, #4]
                __schema { types { fields { args { type { name fields { name }}}}}}
                __schema { types { fields { type { name fields { name }}}}}
                __schema { types { inputFields { type { inputFields { name }}}}}
                __schema { types { interfaces { fields { type { interfaces { name } } } } } }
                __schema { types { name} }
                __type(name : "t") { name }
                a1: __schema { types { name} }
                a1: __type(name : "t") { name }
…

graphql.execution.instrumentation.TracingInstrumentationTest ‑ do not trace introspection information [testExecutionStrategy: <graphql.execution.AsyncExecutionStrategy@6bcad31f fieldCollector=inaccessible executionStepInfoFactory=inaccessible dataFetcherExceptionHandler=inaccessible resolvedType=inaccessible>, #0]
graphql.execution.instrumentation.TracingInstrumentationTest ‑ do not trace introspection information [testExecutionStrategy: <graphql.execution.AsyncSerialExecutionStrategy@79b34e37 fieldCollector=inaccessible executionStepInfoFactory=inaccessible dataFetcherExceptionHandler=inaccessible resolvedType=inaccessible>, #1]
graphql.execution.instrumentation.TracingInstrumentationTest ‑ tracing captures timings as expected [testExecutionStrategy: <graphql.execution.AsyncExecutionStrategy@be2a568 fieldCollector=inaccessible executionStepInfoFactory=inaccessible dataFetcherExceptionHandler=inaccessible resolvedType=inaccessible>, #0]
graphql.execution.instrumentation.TracingInstrumentationTest ‑ tracing captures timings as expected [testExecutionStrategy: <graphql.execution.AsyncSerialExecutionStrategy@7a63eef1 fieldCollector=inaccessible executionStepInfoFactory=inaccessible dataFetcherExceptionHandler=inaccessible resolvedType=inaccessible>, #1]
graphql.execution.instrumentation.TracingInstrumentationTest ‑ trivial data fetchers are ignored [testExecutionStrategy: <graphql.execution.AsyncExecutionStrategy@2402e7a8 fieldCollector=inaccessible executionStepInfoFactory=inaccessible dataFetcherExceptionHandler=inaccessible resolvedType=inaccessible>, #0]
graphql.execution.instrumentation.TracingInstrumentationTest ‑ trivial data fetchers are ignored [testExecutionStrategy: <graphql.execution.AsyncSerialExecutionStrategy@642f845f fieldCollector=inaccessible executionStepInfoFactory=inaccessible dataFetcherExceptionHandler=inaccessible resolvedType=inaccessible>, #1]
graphql.execution.reactive.CompletionStageMappingPublisherTest ‑ asynchronous mapping works with completion [why: CompletionStageMappingOrderedPublisher, creator: <graphql.execution.reactive.CompletionStageMappingPublisherTest$__spock_feature_0_4prov1_closure10@766be122 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, #1]
graphql.execution.reactive.CompletionStageMappingPublisherTest ‑ asynchronous mapping works with completion [why: CompletionStageMappingPublisher, creator: <graphql.execution.reactive.CompletionStageMappingPublisherTest$__spock_feature_0_4prov1_closure9@4739f8fb delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, #0]
graphql.execution.reactive.ReactiveSupportTest ‑ can get a empty reactive or flow publisher and make a CF from it [reactiveObject: <reactor.adapter.JdkFlowAdapter$PublisherAsFlowPublisher@639c335c pub=inaccessible>, #1]
graphql.execution.reactive.ReactiveSupportTest ‑ can get a reactive or flow publisher and make a CF from it [reactiveObject: <reactor.adapter.JdkFlowAdapter$PublisherAsFlowPublisher@67eb5d40 pub=inaccessible>, #1]
…

♻️ This comment has been updated with latest results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@andimarek