All issues

Search results

ScreenSpot: click points in 0-1000 coordinates (Qwen-VL style) are scored as misses

Status: Open.

#1706 In open-compass/VLMEvalKit;

· yepapa-nest opened on Sep 24, 2026

anls_compute measures distance on normalized strings but divides by raw length, so whitespace-padded predictions score near 1.0 on DocVQA/InfoVQA/WildDoc

Status: Open.

#1705 In open-compass/VLMEvalKit;

· shaurya416 opened on Sep 23, 2026

[Bug] Missing images and outdated questions/answers in ZeroBench

Status: Open.

#1703 In open-compass/VLMEvalKit;

· Pulyong opened on Sep 22, 2026

extract_characters_regex: reversed containment test scores empty/missing predictions as 'A'

Status: Open.

#1701 In open-compass/VLMEvalKit;

· rrrxxx0510 opened on Sep 22, 2026

cal_scores: compute_mcq_score scores unparseable GT+prediction as correct (False == False yields 1.0)

Status: Open.

#1697 In open-compass/VLMEvalKit;

· rrrxxx0510 opened on Sep 21, 2026

olmOCRBench: evaluate() returns None, so the score never reaches status.json

Status: Open.

#1690 In open-compass/VLMEvalKit;

· jaeminSon opened on Sep 16, 2026

Structured extra_records output is discarded for the entire dataset if any single sample fails

Status: Open.

#1665 In open-compass/VLMEvalKit;

· williamwuyantao opened on Sep 1, 2026

OpenAI-compat: GPT4V api_base is the full /v1/chat/completions URL, not /v1

Status: Open.

#1661 In open-compass/VLMEvalKit;

· cursor opened on Aug 28, 2026

Interop idea: a VLMEvalKit-openeval-adapter for portable eval results

Status: Open.

#1652 In open-compass/VLMEvalKit;

· adhabnr-ux opened on Aug 23, 2026

Add Pluto-AI-Labs/Apollo-VL-Edge-3B to Leaderboard

Status: Open.

#1632 In open-compass/VLMEvalKit;

· SiddharthFX opened on Aug 14, 2026

Qwen2.5-VL-7B模型，对评测集LogicVista，从推理结果中提取答案错误，导致误判

Status: Open.

#1621 In open-compass/VLMEvalKit;

· suddenlee opened on Aug 6, 2026

MMMU Open-ended问题无法正确评测

Status: Open.

#1617 In open-compass/VLMEvalKit;

· identzz-ai opened on Aug 4, 2026