Skip to content

expr: apply target length/truncation to JSON-to-CHAR cast pushdown - #20041

Open
NasitSony wants to merge 1 commit into
tikv:masterfrom
NasitSony:coprocessor-json-char-length-fix
Open

expr: apply target length/truncation to JSON-to-CHAR cast pushdown#20041
NasitSony wants to merge 1 commit into
tikv:masterfrom
NasitSony:coprocessor-json-char-length-fix

Conversation

@NasitSony

@NasitSony NasitSony commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Fixes #19883

cast_json_as_bytes captured only EvalContext and returned the full converted JSON string unconditionally, with no length or truncation handling. TiDB's builtinCastJSONAsStringSig applies ProduceStrWithSpecifiedTp using the return FieldType's flen, which can truncate the result or, under strict mode, abort the statement for an overlong value. Since this conversion backs predicates and DML pushed down to the coprocessor (e.g. CAST(json_col AS CHAR(n)) <> value), a query could select or mutate different rows than the same statement evaluated at the TiDB root -- in the reported case, a DELETE using an overlong cast value removed rows that TiDB's root evaluation would have aborted before touching.

Added extra: &RpnFnCallExtra to cast_json_as_bytes's capture list and routed the converted value through the existing cast_as_string_helper (already used by cast_string_as_string), which applies produce_str_with_specified_tp -- the same flen-based truncation/error logic TiDB uses. This is a pure wiring fix: ret_field_type was already transported via RpnFnCallExtra, just not consumed by this function.

Added a regression test using the exact reproduction value (1234.5 cast to CHAR(4)), confirming it now errors under strict mode rather than silently truncating.

What is changed and how it works?

Issue Number: Close #19883

What's Changed:

expr: apply target length/truncation to JSON-to-CHAR cast pushdown

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Release note

Fix a bug where casting a JSON value to `CHAR(n)` ignored the target length, when the cast was pushed down to TiKV's coprocessor. This could cause a query to select or modify different rows than the same statement evaluated at the TiDB root, since TiDB's root evaluation truncates or, under strict mode, aborts for an overlong value.

Summary by CodeRabbit

  • Bug Fixes
    • Improved JSON-to-string casting to match TiDB-compatible length limits, truncation, and binary padding.
    • Corrected handling of overlong JSON values cast to fixed-length character types.
    • Added strict-mode validation for oversized casts.

Fixes tikv#19883

cast_json_as_bytes captured only EvalContext and returned the full
converted JSON string unconditionally, with no length or truncation
handling. TiDB's builtinCastJSONAsStringSig applies
ProduceStrWithSpecifiedTp using the return FieldType's flen, which can
truncate the result or, under strict mode, abort the statement for an
overlong value. Since this conversion backs predicates and DML pushed
down to the coprocessor (e.g. CAST(json_col AS CHAR(n)) <> value), a
query could select or mutate different rows than the same statement
evaluated at the TiDB root -- in the reported case, a DELETE using an
overlong cast value removed rows that TiDB's root evaluation would
have aborted before touching.

Added extra: &RpnFnCallExtra to cast_json_as_bytes's capture list and
routed the converted value through the existing cast_as_string_helper
(already used by cast_string_as_string), which applies
produce_str_with_specified_tp -- the same flen-based truncation/error
logic TiDB uses. This is a pure wiring fix: ret_field_type was already
transported via RpnFnCallExtra, just not consumed by this function.

Added a regression test using the exact reproduction value (1234.5
cast to CHAR(4)), confirming it now errors under strict mode rather
than silently truncating.

Signed-off-by: Nasit  Sarwar Sony <nasitsony96@gmail.com>
@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/needs-triage-completed dco-signoff: yes Indicates the PR's author has signed the dco. labels Aug 26, 2026
@ti-chi-bot

ti-chi-bot Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign coocood for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the contribution This PR is from a community contributor. label Aug 26, 2026
@ti-chi-bot

ti-chi-bot Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Hi @NasitSony. Thanks for your PR.

I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot added needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 26, 2026
@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 75f1c79d-b177-4d3f-b51b-e0bf80084f4e

📥 Commits

Reviewing files that changed from the base of the PR and between 8ceece5 and a7c800f.

📒 Files selected for processing (1)
  • components/tidb_query_expr/src/impl_cast.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

cast_json_as_bytes now applies target string length, truncation, error, and padding rules. Tests update the helper call and verify that strict mode rejects an overlong JSON string cast.

Changes

JSON string cast behavior

Layer / File(s) Summary
Apply target string semantics
components/tidb_query_expr/src/impl_cast.rs
cast_json_as_bytes now accepts RpnFnCallExtra and uses cast_as_string_helper with return field metadata. Tests cover unspecified length and strict-mode rejection of CAST(1234.5 AS CHAR(4)).

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to a7c80

This is a localized fix to make pushed-down JSON-to-CHAR casts honor target length and strict-mode errors; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: applying target length and truncation behavior to JSON-to-CHAR cast pushdown.
Description check ✅ Passed The description follows the repository template. It includes the linked issue, implementation details, unit-test coverage, side-effect assessment, and release note.
Linked Issues check ✅ Passed The changes address issue #19883 by passing RpnFnCallExtra to cast_json_as_bytes and applying the existing TiDB-compatible length, truncation, padding, and strict-mode error handling. The regression t…
Out of Scope Changes check ✅ Passed The changes are limited to JSON-to-CHAR cast pushdown behavior and its regression tests. No unrelated code changes are identified.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files.
Full details: Linked Issues check

Explanation

The changes address issue #19883 by passing RpnFnCallExtra to cast_json_as_bytes and applying the existing TiDB-compatible length, truncation, padding, and strict-mode error handling. The regression test covers the overlong CHAR(4) cast case.

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. dco-signoff: yes Indicates the PR's author has signed the dco. do-not-merge/needs-triage-completed needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[coprocessor] TiKV JSON-to-CHAR pushdown ignores target length and DELETE can remove rows after TiDB would abort

1 participant