How a Small GUI Grounding Model Should Receive the Action Type: Auxiliary Loss, Additive Embedding, and Prompt Word Each Gain 5–7 hit@0.10 Points, but Much of the Gain Comes from a Serialization Artifact
Fine-tuning Qwen2-VL-2B with LoRA on Android in the Wild, the study compares five ways of supplying the action type to a small GUI grounding model under matched data, compute, and decoding, finding that an auxiliary loss, an additive embedding, and a prompt word each raise hit@0.10 by five to seven points, but after retraining without the degenerate class created by the serializer no mechanism beats the baseline on hit rate, and only the auxiliary loss shortens the average miss.