Home
Search
New thread

Discussion

Discussion categories

General
Tech

Alt+Shift+M picks up a node. Use arrows and typeahead to browse destinations. Space cycles before, inside, and after. Enter places; Escape cancels. Tab cancels and leaves move mode.

Library

Library pages

Design
Resources
Websites
Articles
Packs and Templates
Typography
Web development
Cool Libraries
Tools
Articles
Papers and Studies
Articles
Digital Rights and Tech Policy
Software Craft
Society, Science and Culture
Language Models
Articles
Agent Tools
Platforms and Standards
Labs and Products
Searching for a job in the AI era, part 1: the setup and the first 16 days
Tech and Systems
Infrastructure and Hosting
Networking and Protocols
Vector Databases & Object Storage
ManyVector
vec
Amazon S3 Vectors
coder/hnsw
vecgo
Turbopuffer
govector
LanceDB
Articles
Computers
Developer Tools
Ghostty
Nyx
Paseo
Phones and Mobile
Loupe
Windows Tools and Modding
Windhawk
Raycast for Windows
Rainmeter
Mac Apps and Fixes
AltTab
Mac Mouse Fix
Rectangle
Raycast
Space Rabbit
Caffeine
Operating Systems and Desktops
Kando
Plasma 6.7
Haiku
Hardware and Computer Architecture
Fugleramme: E-ink Bird Frame
BROADSONIC Optical Ultrasonic Sensor
IBM introduces the smallest computer chip in the world
Cofactr
Computing History and Culture
Vintage Computing Festival Berlin 2026
Browsers
Aside
Yab
Ardia
Phi
Dia
Zen
Helium
Arcium
Cra
Crest
Search
Stedding
Nook
Park
Random fun stuff
Passport Index Data
Esoteric File Systems
Cool websites
Friends
Unity: Cheaterman's Bar
ChatGPT
SIGBOVIK
Villagers Protest 5G Outage in Indian Village
Database Detective: Minor Crimes Division
Mens T-shirt data
Gentoo is Rice
Indie Fashion
Game Modding
San Andreas Multiplayer / open.mp
KEG
BRUTAL Framework
Storyden
Selfh.st
OpenAlternative
Microlaunch
Peerlist
Glama.ai
AlternativeTo
Brandfetch
Dokploy
PitchHut
Piefed Social

Alt+Shift+M picks up a node. Use arrows and typeahead to browse destinations. Space cycles before, inside, and after. Enter places; Escape cancels. Tab cancels and leaves move mode.

Collections
Links
Members
Roles
RegisterLogin
Login
Library
Papers and Studies
UI-UG: A Unified MLLM for UI Understanding and Generation

UI-UG: A Unified MLLM for UI Understanding and Generation


UI-UG: A Unified MLLM for UI Understanding and Generation

Although Multimodal Large Language Models (MLLMs) have been widely applied across domains, they are still facing challenges in domain-specific tasks, such as User Interface (UI) understanding accuracy and UI generation quality. In this paper, we introduce UI-UG (a unified MLLM for UI Understanding and Generation), integrating both capabilities. For understanding tasks, we employ Supervised Fine-tuning (SFT) combined with Group Relative Policy Optimization (GRPO) to enhance fine-grained understanding on the modern complex UI data. For generation tasks, we further use Direct Preference Optimization (DPO) to make our model generate human-preferred UIs. In addition, we propose an industrially effective workflow, including the design of an LLM-friendly domain-specific language (DSL), training strategies, rendering processes, and evaluation metrics. In experiments, our model achieves state-of-the-art (SOTA) performance on understanding tasks, outperforming both larger general-purpose MLLMs and similarly-sized UI-specialized models. Our model is also on par with these larger MLLMs in UI generation performance at a fraction of the computational cost. We also demonstrate that integrating understanding and generation tasks can improve accuracy and quality for both tasks. Code and Model: https://github.com/neovateai/UI-UG

arxiv.org