Dashboard variable values by dynamic finder not returning all results

[KieranP]

Bug description

I setup a dashboard variable to populate values from Traces, sorted alphabetically. It populates 50 results, and then nothing else.

Expected behavior

Should populate the dropdown with all available values.

How to reproduce

1. Setup dashboard variable, populated dynamically by something with more than 50 values
2. Can only select from the first 50 in the dropdown

Version information

• Signoz version: 1.41.1
• Browser version: Brace
• Your OS and version: macOS
• Your CPU Architecture(ARM/Intel): ARM

[Hayyan612]

Traced where the 50 comes from.

It is a backend default, not a dropdown slice. pkg/telemetrymetadata/metadata.go applies the same fallback in six places whenever the caller does not set one, for example at line 1394:

limit := fieldValueSelector.Limit
if limit == 0 {
    limit = 50
}
// query one extra to check if we hit the limit
sb.Limit(limit + 1)

and again in getTagTableValues:

limit := fieldValueSelector.Limit
if limit == 0 {
    limit = 50
}

So any caller that leaves FieldValueSelector.Limit at its zero value silently gets 50 rows. A dashboard variable populated from Traces goes through this path, which matches what you saw: exactly 50, then nothing, with no error.

Note the limit + 1 on the query. The extra row exists so the backend can tell it hit the ceiling, so the truncation is known at that point and just is not surfaced to the variable dropdown.

Three ways this could go, and I would rather you pick than guess:

1. Raise the default. One constant, but it changes every caller of this path, and on a high-cardinality attribute a larger default is a heavier ClickHouse scan for everyone.
2. Have the dashboard variable caller pass an explicit limit. Contained, no effect on other callers, but any value chosen is still a ceiling and the same report comes back at whatever the new number is.
3. Surface the truncation. The limit + 1 already tells the backend it was cut off. Passing that through so the dropdown can say "showing first 50 of more" is honest about the ceiling instead of silently hiding values, and could pair with either of the above.

My instinct is 2 plus 3: give the variable path its own limit and stop the truncation being invisible. But raising the shared default has performance consequences across every field-values caller, so it is your call.

Happy to send the PR once you say which.

[Hayyan612]

Could a maintainer assign this to me once you have picked a direction from the options above? I will wait for the assignment before opening anything.

If it helps the decision: my preference is giving the dashboard-variable caller its own explicit limit and surfacing the truncation that the limit + 1 query already detects, rather than raising the shared default of 50 that every field-values caller in pkg/telemetrymetadata/metadata.go falls back to.

[KieranP]

Another issue this presents: Have say 80 options available, and the dashboard variable defaults to ALL, then deselect one from the dropdown. Instead of getting results for the remaining 79, you now only get the 49 options from the dropdown.

Ideally a combination of options 1 and 2 from @Hayyan612 post. Default to returning all options, which I think most people would expect. But for those who have 1000s of options, allow a limit field so they can limit if they want.

You'd also then need to probably implementation virtual scrolling for the dropdown, so you could have 1000 options returned for the field, but only render the ones in view, to prevent lag.

[srikanthccv]

Hi @KieranP,

• fix(dashboard): restore related values and API search in dynamic variable dropdowns #12935 (in the next release) fixes the part of the issue i.e the dropdown search now queries the server, so you can find all values. When the list is truncated, the dropdown shows "Don't see the value? Use search".
• We will not return all values by default. This is because for high-cardinality attributes, the query cost and the payload size are too large. And no amount limiting is a solution because there is always going to be more values.
• The more the values, the larger the query size becomes, so even if we add support, it will be rejected at the DB layer because query is too big.
• The deselect-one-from-ALL problem is a valid point and remains after fix(dashboard): restore related values and API search in dynamic variable dropdowns #12935. The frontend converts ALL into the loaded 50 values minus one. A larger limit cannot fix this.
• We keep this issue open to track the deselect-{one,multiple}-from-ALL as semantic gap.