Skip to content

MDEV-40089 Preserve unsigned cached integers in real comparisons - #5769

Open
DerZc wants to merge 1 commit into
MariaDB:11.4from
DerZc:fix-mdev-40089
Open

DerZc wants to merge 1 commit into
MariaDB:11.4from
DerZc:fix-mdev-40089

Conversation

@DerZc

@DerZc DerZc commented Sep 28, 2026 •

Copy link
Copy Markdown

Item_cache_int::val_real() interprets stored unsigned bits as a signed integer before converting to DOUBLE. Values above LLONG_MAX become negative, so a BETWEEN predicate using a cached ~TRUE bound can wrongly discard rows.

Use unsigned_flag to convert unsigned cached values through ulonglong before converting to DOUBLE.

Regression coverage

Add one query to main.func_bit with a bitwise complement and a VARCHAR column. Two rows prevent constant-table optimization from bypassing the cache. Both should satisfy IS FALSE because zero is below the unsigned lower bound; ORDER BY makes the result deterministic.

The five-line test needs no InnoDB prerequisite, SET, database creation, manual prepared statements or duplicate queries. Existing test prerequisites remain unchanged, and MTR supplies prepared-statement and view protocol coverage.

Validation

On current 11.4 at f49e838d367f2154b3edb43effa99ff9eb0d8602:

  • The unmodified server fails main.func_bit only on the new query, returning no rows instead of 0 and 1.
  • The patched Release build succeeds.
  • Normal protocol: main.func_bit, main.select, main.type_bit, main.type_bit_innodb, main.type_decimal and main.type_float pass.
  • Prepared-statement protocol: main.func_bit passes.
  • View protocol: main.func_bit passes.
  • Expected rows are independently derived from unsigned 64-bit complement arithmetic and numeric comparison.
  • The full database regression suite was not run.

Bug report: https://jira.mariadb.org/browse/MDEV-40089

@gkodinov gkodinov added the External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements. label Sep 28, 2026
@gkodinov gkodinov self-assigned this Sep 28, 2026

@gkodinov gkodinov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution! This is a preliminary review.

One general note on the commit message: please do not summarize the changes done.

Good commit messages usually follow the following pattern:

  1. what was wrong exactly
  2. how it was fixed
  3. how was the fix tested.

Comment thread mysql-test/main/func_bit.test Outdated
Comment thread mysql-test/main/func_bit.test Outdated
Comment thread mysql-test/main/func_bit.test Outdated
Item_cache_int::val_real() converts the stored integer to DOUBLE as a
signed value even when unsigned_flag is set. Values above LLONG_MAX
therefore become negative. A BETWEEN comparison using a cached ~TRUE
bound can then discard a row that should match an IS FALSE predicate.

Interpret unsigned cached bits as ulonglong before converting to
DOUBLE, using unsigned_flag to select the correct conversion.

Add a BETWEEN query with a bitwise complement and a VARCHAR column
to func_bit. Use two rows so constant-table optimization cannot bypass
the cache. Check that zero lies below the unsigned lower bound for
both rows.

Bug report: https://jira.mariadb.org/browse/MDEV-40089
@DerZc DerZc changed the title MDEV-40089 Wrong result for BETWEEN predicate involving bitwise negation and implicit type conversion MDEV-40089 Preserve unsigned cached integers in real comparisons Sep 29, 2026

@gkodinov gkodinov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks! Please stand by for the final review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements.

Development

Successfully merging this pull request may close these issues.

2 participants