Skip to main content
Search

# Mobile UI Understanding and Generation

1 related articles

Subscribe to updates
  1. arXiv

    A 4B multimodal model reads and writes mobile UIs: fine-grained perception lifts generation score to 4.92

    UI-UG 2 is a 4B-parameter multimodal large language model that trains mobile UI understanding and generation in one pipeline. It annotates over 30,000 screenshots from RICO, VINS, Android in the Wild and Android Control with an 80-category two-level taxonomy aligned to front-end component semantics, paired with more than 5,000 requirement–screenshot–JSON tuples. Generation is optimized by online render-aware GRPO: sampled DSL outputs are rendered by a Playwright server-side pipeline, then scored by an LLM judge and contrastive preference ranking. On its own 80-category benchmark it gives the strongest referring and dense-grounding results among evaluated models, and its composite generation score of 4.92 is the highest among compared models.

You’re all caught up