MoI-BenchA Benchmark for Image Generation and Editing on Mobile Devices

Kailai Feng^1,*Haoxiang Rao^2,*Yuxiang Wei^3,†Bo Chen²
Junsong Li²Yang Pan²Chengkun Cai¹Jagmohan Chauhan^1,†

¹ University College London² ByteDance³ The Hong Kong Polytechnic University

^* Equal contribution ^† Co-corresponding authors

GitHub Coming soon Paper Coming soon

Benchmark overview

MoI-Bench overview with task examples, an Operation × Target × Criteria cube, and the Quality–Deployability map.

1,150samples

64subtasks

16 × 8operations × targets

8evaluation criteria

Beyond a single score.

MoI-Bench organizes everyday image requests by what to change and what to change it on. MoI-Eval assesses task-specific quality, while mobile measurements show which capabilities fit on a device.

Qualitative comparisons

Outputs from nine models across generation, local editing, global editing, text translation, and reference editing.

Capability analysis

Radar charts compare models across operations, targets, and evaluation criteria.

Operation × Target

Operation by target heatmaps for DreamLite, Ovis-U1-3B, and FLUX.2 klein 4B, showing task-specific scores and unsupported input configurations.

Quality & deployability

Dataset overview

Survey results and sample distributions by operation, target, and number of input images.

BibTeX

citation.bib

% BibTeX coming soon.