Update README.md

README.md

@@ -29,7 +29,7 @@ tags:
   line-height:1.25;
   text-align:center;
   margin:0 0 24px;">
-  OpenCUA: Open Foundations for Computer-Use Agents
+  OpenCUA-7B
 </h1>
 
 <div style="
@@ -38,7 +38,7 @@ tags:
   gap:12px;
   flex-wrap:wrap;
   margin-bottom:28px;">
-  
+
   <a href="https://opencua.xlang.ai/" style="
      display:inline-block;
      padding:8px 24px;
@@ -78,6 +78,22 @@ tags:
 
 <div style="max-width:900px;margin:0 auto;">
 
+# 🚀 vLLM Serve (Recommended)
+
+We recommend using vLLM for production deployment. Requires **vllm>=0.12.0** with `--trust-remote-code`.
+
+```bash
+vllm serve xlangai/OpenCUA-7B \
+  --trust-remote-code \
+  --served-model-name opencua-7b \
+  --host 0.0.0.0 \
+  --port 8000
+```
+
+Adjust `--gpu-memory-utilization` based on your hardware configuration.
+
+---
+
 #  Introduction
 <div style="
   max-width: 880px;              /* 可按需调节整体宽度 */
@@ -85,16 +101,18 @@ tags:
   text-align: justify;          /* 关键：两端对齐 */
   text-justify: inter-word;     /* 优化英文对齐效果 */
   line-height: 1.6;">
-  
-OpenCUA models (OpenCUA-7B and OpenCUA-32B) are end-to-end computer-use foundation models than can produce executable actions in the computer environments.  They are based on the weights of Qwen2.5-VL-7B-Instruction and Qwen2.5-VL-32B-Instruction. 
-They demonstrate superior performance across CUA benchmarks. In particular, <b>OpenCUA-32B</b> achieves an average success rate of **34.8%** on [OSWorld-Verified](https://os-world.github.io/), 
-establishing a new state-of-the-art (SOTA) among open-source models and surpassing OpenAI CUA (GPT-4o). Both models also have strong grounding performance, OpenCUA-32B achieves 59.6% on [OSWorld-G](https://osworld-grounding.github.io/) and 55.3% on [Screenspot-Pro](https://arxiv.org/abs/2504.07981).
+
+OpenCUA models (OpenCUA-7B, OpenCUA-32B, and OpenCUA-72B) are end-to-end computer-use foundation models that can produce executable actions in the computer environments with great planning and grounding capabilities. They are based on the Qwen2.5-VL model family.
+
+With the help of OpenCUA framework, our end-to-end agent models demonstrate strong performance across CUA benchmarks. In particular, <b>OpenCUA-72B</b> achieves an average success rate of **45.0%** on [OSWorld-Verified](https://os-world.github.io/), establishing a new state-of-the-art (SOTA) among open-source models. OpenCUA-72B also has strong grounding ability, achieving 37.3% (SOTA) on [UI-Vision](https://arxiv.org/abs/2504.07981) and 60.8% on [ScreenSpot-Pro](https://arxiv.org/abs/2504.07981).
 </div>
 
 ## 📢 Updates
-- 2025-10-12:  <span style="font-weight:bold">[OpenCUA-7B-exl2](https://huggingface.co/sujitvasanth/OpenCUA-7B-exl2) is now live!</span> ⚡️  
+- 2026-01-17: 🎉 **vLLM now fully supports OpenCUA-7B, OpenCUA-32B, and OpenCUA-72B!** Thanks to the [Meituan EvoCUA Team](https://github.com/meituan) for their contributions to vLLM integration.
+
+- 2025-10-12:  <span style="font-weight:bold">[OpenCUA-7B-exl2](https://huggingface.co/sujitvasanth/OpenCUA-7B-exl2) is now live!</span> ⚡️
   Thanks to [Sujit Vasanth](https://huggingface.co/sujitvasanth) for producing a quantized **exllamav2** version of OpenCUA-7B — enabling much faster inference with lower VRAM usage.
-  
+
 ### Key Features
 
 - **Superior Computer-Use Capablity**: Able to execute multi-step computer-use actions with effective planning and reasoning
@@ -107,9 +125,8 @@ establishing a new state-of-the-art (SOTA) among open-source models and surpassi
 # Performance
 
 ### Online Agent Evaluation
-OpenCUA models achieves strong performance on **[OSWorld-Verified](https://os-world.github.io/)**. 
-OPENCUA-32B achieves the best performance among all open-source models with an average success rate of 34.8%, outperforming prior baselines by large margins. 
-It also closes the gap to proprietary Claude models.
+OpenCUA models achieves strong performance on **[OSWorld-Verified](https://os-world.github.io/)**.
+OpenCUA-72B achieves the best performance among all open-source models with an average success rate of 45.0%, establishing a new state-of-the-art (SOTA).
 <div align="center">
 
 | **Model**                        | **15 Steps** | **50 Steps** | **100 Steps** |
@@ -120,13 +137,14 @@ It also closes the gap to proprietary Claude models.
 | Claude 3.7 Sonnet             | 27.1     | 35.8     | 35.9      |
 | Claude 4 Sonnet               | 31.2     | 43.9     | 41.5      |
 | **Open-Source**               |          |          |           |
-| Qwen 2.5-VL-32B-Instruct       | 3.0      | —        | 3.9       |
-| Qwen 2.5-VL-72B-Instruct       | 4.4      | —        | 5.0       |
+| Qwen 2.5-VL-32B-Instruct      | 3.0      | —        | 3.9       |
+| Qwen 2.5-VL-72B-Instruct      | 4.4      | —        | 5.0       |
 | Kimi-VL-A3B                   | 9.7      | —        | 10.3      |
 | UI-TARS-72B-DPO               | 24.0     | 25.8     | 27.1      |
 | UI-TARS-1.5-7B                | 24.5     | 27.3     | 27.4      |
 | OpenCUA-7B *(Ours)*           | 24.3     | 27.9     | 26.6      |
-| **OpenCUA-32B *(Ours)***      | **29.7** | **34.1** | **34.8**  |
+| OpenCUA-32B *(Ours)*          | 29.7     | 34.1     | 34.8      |
+| **OpenCUA-72B *(Ours)***      | **39.0** | **44.9** | **45.0**  |
 </div>
 
 *OpenCUA scores are the mean of 3 independent runs.*
@@ -134,15 +152,14 @@ It also closes the gap to proprietary Claude models.
 ### GUI Grounding Performance
 <div align="center">
 
-| **Model** | **OSWorld-G** | **ScreenSpot-V2** | **ScreenSpot-Pro** |
-|-------|-----------|---------------|----------------|
-| Qwen2.5-VL-7B | 31.4 | 88.8 | 27.6 |  
-| Qwen2.5-VL-32B | 46.5 | 87.0 | 39.4 |
-| UI-TARS-72B | 57.1 | 90.3 | 38.1 |
-| **OpenCUA-A3B** | 48.6 | 91.4 | 28.5 |
-| **OpenCUA-Qwen2-7B** | 45.7 | 88.5 | 23.7 |
-| **OpenCUA-7B** | 55.3 | 92.3 | 50.0 |
-| **OpenCUA-32B** | **59.6** | **93.4** | **55.3** |
+| **Model** | **OSWorld-G** | **ScreenSpot-V2** | **ScreenSpot-Pro** | **UI-Vision** |
+|-------|-----------|---------------|----------------|----------|
+| Qwen2.5-VL-7B   | 31.4 | 88.8 | 27.6 | 0.85 |
+| Qwen2.5-VL-32B  | 46.5 | 87.0 | 39.4 | - |
+| UI-TARS-72B     | 57.1 | 90.3 | 38.1 | 25.5 |
+| **OpenCUA-7B**  | 55.3 | 92.3 | 50.0 | 29.7 |
+| **OpenCUA-32B** | 59.6 | 93.4 | 55.3 | 33.3 |
+| **OpenCUA-72B** | **59.2** | **92.9** | **60.8** | **37.3** |
 </div>
 
 
@@ -161,164 +178,133 @@ It also closes the gap to proprietary Claude models.
 
 #  🚀 Quick Start
 <div style="border-left: 6px solid #f28c28; background: #fff8e6; padding: 12px 16px; margin: 16px 0;">
-  <strong>⚠️ Important for Qwen-based Models (OpenCUA-7B, OpenCUA-32B):</strong>
-  
+  <strong>⚠️ Important for Qwen-based Models (OpenCUA-7B, OpenCUA-32B, OpenCUA-72B):</strong>
+
   To align with our training infrastructure, we have modified the model in two places:
   <ul style="margin-top: 8px;">
     <li>1. Multimodal Rotary Position Embedding (M-RoPE) has been replaced with 1D RoPE</strong>.</li>
     <li>2. Using the same Tokenizer and ChatTemplate as Kimi-VL.</li>
-    <li>Do not use the default transformers and vllm classes to load the model. Tokenizer and Chat Template should be aligned if training the models.</li>
+    <li>vLLM supported via <code>--trust-remote-code</code> flag. Tokenizer and Chat Template should be aligned if training the models.</li>
   </ul>
 </div>
 
 
 ## Installation & Download
 
-First, install the required transformers dependencies:
+First, install the required dependencies:
 
 ```bash
-conda create -n opencua python=3.10
+conda create -n opencua python=3.12
 conda activate opencua
-pip install -r requirement.txt
+pip install openai>=1.0.0
 ```
 
-Download the model weight from huggingface:
-```bash
+Download the model weight from huggingface (optional, vLLM can download automatically):
+```python
 from huggingface_hub import snapshot_download
 snapshot_download(
     repo_id="xlangai/OpenCUA-7B",
-    local_dir="OpenCUA-7B",                
-    local_dir_use_symlinks=False  
+    local_dir="OpenCUA-7B",
+    local_dir_use_symlinks=False
 )
 ```
 
-## 🎯 GUI Grounding 
+## 🎯 GUI Grounding
+
+First, start the vLLM server:
+
+```bash
+vllm serve xlangai/OpenCUA-7B \
+  --trust-remote-code \
+  --served-model-name opencua-7b \
+  --host 0.0.0.0 \
+  --port 8000
+```
 
-The following code demonstrates how to use OpenCUA models for GUI grounding tasks:
+Then run the following code to test GUI grounding:
 
 ```python
 import base64
-import torch
-from transformers import AutoTokenizer, AutoModel, AutoImageProcessor
-from PIL import Image
-import json
+from openai import OpenAI
+
+# vLLM server configuration
+VLLM_BASE_URL = "http://localhost:8000/v1"
+MODEL_NAME = "opencua-7b"  # Should match --served-model-name in vllm serve
 
 def encode_image(image_path: str) -> str:
-    """Encode image to base64 string for model input."""
+    """Encode image to base64 string."""
     with open(image_path, "rb") as f:
         return base64.b64encode(f.read()).decode()
 
-def load_opencua_model(model_path: str):
-    """Load OpenCUA model, tokenizer, and image processor."""
-    tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
-    model = AutoModel.from_pretrained(
-        model_path, 
-        torch_dtype="auto", 
-        device_map="auto", 
-        trust_remote_code=True
-    )
-    image_processor = AutoImageProcessor.from_pretrained(model_path, trust_remote_code=True)
-    
-    return model, tokenizer, image_processor
+def run_grounding(image_path: str, instruction: str) -> str:
+    """Run GUI grounding inference via vLLM."""
+    client = OpenAI(base_url=VLLM_BASE_URL, api_key="EMPTY")
 
-def create_grounding_messages(image_path: str, instruction: str):
-    """Create chat messages for GUI grounding task."""
     system_prompt = (
         "You are a GUI agent. You are given a task and a screenshot of the screen. "
         "You need to perform a series of pyautogui actions to complete the task."
     )
-    
+
     messages = [
         {"role": "system", "content": system_prompt},
         {
             "role": "user",
             "content": [
-                {"type": "image", "image": f"data:image/png;base64,{encode_image(image_path)}"},
+                {
+                    "type": "image_url",
+                    "image_url": {"url": f"data:image/png;base64,{encode_image(image_path)}"}
+                },
                 {"type": "text", "text": instruction},
             ],
         },
     ]
-    return messages
 
-def run_inference(model, tokenizer, image_processor, messages, image_path):
-    """Run inference on the model."""
-    # Prepare text input
-    input_ids = tokenizer.apply_chat_template(
-        messages, tokenize=True, add_generation_prompt=True
+    response = client.chat.completions.create(
+        model=MODEL_NAME,
+        messages=messages,
+        max_tokens=512,
+        temperature=0,
     )
-    input_ids = torch.tensor([input_ids]).to(model.device)
-    
-    # Prepare image input  
-    image = Image.open(image_path).convert('RGB')
-    image_info = image_processor.preprocess(images=[image])
-    pixel_values = torch.tensor(image_info['pixel_values']).to(
-        dtype=torch.bfloat16, device=model.device
-    )
-    grid_thws = torch.tensor(image_info['image_grid_thw'])
-    
-    # Generate response
-    with torch.no_grad():
-        generated_ids = model.generate(
-            input_ids,
-            pixel_values=pixel_values,
-            grid_thws=grid_thws,
-            max_new_tokens=512,
-            temperature=0
-        )
-    
-    # Decode output
-    prompt_len = input_ids.shape[1]
-    generated_ids = generated_ids[:, prompt_len:]
-    output_text = tokenizer.batch_decode(
-        generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
-    )[0]
-    
-    return output_text
+
+    return response.choices[0].message.content
 
 # Example usage
-model_path = "xlangai/OpenCUA-7B"  # or other model variants
 image_path = "screenshot.png"
 instruction = "Click on the submit button"
 
-# Load model
-model, tokenizer, image_processor = load_opencua_model(model_path)
-
-# Create messages and run inference
-messages = create_grounding_messages(image_path, instruction)
-result = run_inference(model, tokenizer, image_processor, messages, image_path)
-
+result = run_grounding(image_path, instruction)
 print("Model output:", result)
 ```
 
 <div style="border-left: 6px solid #9ca3af; background: #f5f5f5; padding: 12px 16px; margin: 16px 0;">
-  <em>Expected result:</em> ```python
-pyautogui.click(x=1443, y=343)
-```
+  <em>Expected result:</em> ```python\npyautogui.click(x=1443, y=343)\n```
 </div>
 
-You can also run the five grounding examples in [OpenCUA/model/inference/huggingface_inference.py](https://github.com/xlang-ai/OpenCUA/blob/main/model/inference/huggingface_inference.py):
-``` 
+You can also run the grounding examples in [OpenCUA/model/inference/](https://github.com/xlang-ai/OpenCUA/blob/main/model/inference/):
+```bash
 cd ./model/inference/
+
+# vLLM (requires running vLLM server first)
+python vllm_inference.py
+
+# HuggingFace Transformers
 python huggingface_inference.py
 ```
 
 ## 🖥️ Computer Use Agent
 **[OpenCUAAgent](https://github.com/xlang-ai/OSWorld/blob/main/mm_agents/opencua_agent.py)** is developed in the [OSWorld](https://github.com/xlang-ai/OSWorld) environment based on OpenCUA models. It iteratively perceives the environment via screenshots, produces reflective long CoT as inner monologue, and predicts the next action to be executed. OpenCUAAgent uses 3 images in total and L2 CoT format in default.
 
-Command for running OpenCUA-7B and OpenCUA-32B in OSWorld:
+Command for running OpenCUA-7B in OSWorld:
 ```
     python run_multienv_opencua.py \
         --headless \
         --observation_type screenshot \
-        --model OpenCUA-32B \
+        --model OpenCUA-7B \
         --result_dir ./results --test_all_meta_path evaluation_examples/test_all_no_gdrive.json \
         --max_steps 100 \
         --num_envs 30  \
         --coordinate_type qwen25
 ```
-<div style="border-left: 6px solid #9ca3af; background: #f5f5f5; padding: 12px 16px; margin: 16px 0;">
-  <em>Currently we only supports huggingface inference. We are implementing the vLLM supports of OpenCUA models. Please stay tuned.</em>
-</div>
 
 ---
 
@@ -328,7 +314,7 @@ Command for running OpenCUA-7B and OpenCUA-32B in OSWorld:
   <img src="https://cdn-uploads.huggingface.co/production/uploads/67b327cdd4665a0448eef7d5/dw5k183ucDSB2SZuS5f2V.png" width="400" alt="AgentNet Dataset Domain Distribution">
 </div>
 
-AgentNet is the first large-scale desktop computer-use agent trajectory dataset, containing 22.6K human-annotated computer-use tasks across Windows, macOS, and Ubuntu systems. 
+AgentNet is the first large-scale desktop computer-use agent trajectory dataset, containing 22.6K human-annotated computer-use tasks across Windows, macOS, and Ubuntu systems.
 
 👉 **[AgentNet Huggingface Dataset](https://huggingface.co/datasets/xlangai/AgentNet)**
 
@@ -350,17 +336,17 @@ Collecting computer-use agent training data requires 3 steps:
 </div>
 
 
-Our **AgentNetTool** is a cross-platform GUI recorder that runs unobtrusively on annotators’ machines. It captures synchronized **screen video**, **mouse/keyboard events**, and **accessibility trees**, then provides an in-browser UI for reviewing, trimming, and submitting demonstrations. AgentNet Tool is available on Windows, macOS and Ubuntu. 
+Our **AgentNetTool** is a cross-platform GUI recorder that runs unobtrusively on annotators' machines. It captures synchronized **screen video**, **mouse/keyboard events**, and **accessibility trees**, then provides an in-browser UI for reviewing, trimming, and submitting demonstrations. AgentNet Tool is available on Windows, macOS and Ubuntu.
 
 👉 **[AgentNetTool Document](https://agentnet-tool.xlang.ai/)**
 
 
 
 ## 2 DataProcessor – Action Reduction & State–Action Matching
-Raw demonstrations can contain thousands of low-level events that are too dense for model training.  
+Raw demonstrations can contain thousands of low-level events that are too dense for model training.
 The **DataProcessor** module (`./data/data-process/`) performs two key steps:
 
-1.  **Action Reduction** — merges granular signals into concise, semantically meaningful PyAutoGUI actions (e.g., collapsing mouse moves → click, coalescing scrolls, grouping key-press sequences into text or hotkeys).  
+1.  **Action Reduction** — merges granular signals into concise, semantically meaningful PyAutoGUI actions (e.g., collapsing mouse moves → click, coalescing scrolls, grouping key-press sequences into text or hotkeys).
 2.  **State–Action Matching** — aligns every reduced action with the *last visually distinct frame* **before** the action begins, avoiding future-information leakage and yielding compact state–action pairs.
 
 These processed trajectories underlie all downstream training and evaluation.
@@ -368,12 +354,12 @@ These processed trajectories underlie all downstream training and evaluation.
 ---
 
 ## 3 CoTGenerator – Synthesizing Reflective Long Chain-of-Thought Inner Monologue
-To boost robustness and interpretability, we augment each trajectory with **reflective long Chain-of-Thought (CoT) reasoning**.  
+To boost robustness and interpretability, we augment each trajectory with **reflective long Chain-of-Thought (CoT) reasoning**.
 The **CoTGenerator** pipeline (`./data/cot-generator/`) synthesizes step-level reflections that:
 
 *   reflect on the previous action,
-*   explain *why* an action is chosen given the current observation and history,  
-*   note potential alternative actions, and  
+*   explain *why* an action is chosen given the current observation and history,
+*   note potential alternative actions, and
 *   forecast the expected next state.
 
 Empirically, models trained with these rich CoTs scale better with data and generalize across unseen applications.
@@ -390,21 +376,13 @@ Empirically, models trained with these rich CoTs scale better with data and gene
 
 👉 See **[AgentNetBench/README.md](./evaluation/agentnetbench/README.md)** for usage instructions.
 
-# TODO
-## vLLM Support
-We are actively working with the vLLM team to add support for OpenCUA models.
-
-**Workaround:** For now, please use the standard transformers library as shown in the examples above. We will update this section once vLLM support becomes available.
-
-## Training Code
-OpenCUA models are developed based on the training infrastructure of Kimi Team. We are developting the training pipeline based on the open-source infrastructure as well.
-
 # Acknowledge
 <p>
-We thank Su Yu, Caiming Xiong, Binyuan Hui, and the anonymous reviewers for their insightful discussions and valuable feedback. 
-We are grateful to Moonshot AI for providing training infrastructure and annotated data. 
-We also sincerely appreciate Calvin, Ziwei Chen, Jin Zhang, Ze Li, Zhengtao Wang, Yanxu Chen, and Qizheng Gu from the Kimi Team for their strong infrastructure support and helpful guidance. 
-The development of our tool is based on the open-source projects-<a href="https://github.com/TheDuckAI/DuckTrack" target="_blank">DuckTrack</a> and <a href="https://github.com/OpenAdaptAI/OpenAdapt" target="_blank">OpenAdapt</a>. 
+We thank Yu Su, Caiming Xiong, and the anonymous reviewers for their insightful discussions and valuable feedback.
+We are grateful to Moonshot AI for providing training infrastructure and annotated data.
+We also sincerely appreciate Hao Yang, Zhengtao Wang, and Yanxu Chen from the Kimi Team for their strong infrastructure support and helpful guidance.
+We thank Chong Peng, Taofeng Xue, and Qiumian Huang from the <a href="https://github.com/meituan/EvoCUA" target="_blank">Meituan EvoCUA Team</a> for their contributions to vLLM integration.
+The development of our tool is based on the open-source projects-<a href="https://github.com/TheDuckAI/DuckTrack" target="_blank">DuckTrack</a> and <a href="https://github.com/OpenAdaptAI/OpenAdapt" target="_blank">OpenAdapt</a>.
 We are very grateful to their commitment to the open source community. Finally, we extend our deepest thanks to all annotators for their tremendous effort and contributions to this project.
 </p>
 
@@ -414,7 +392,7 @@ This project is licensed under the MIT License - see the LICENSE file in the roo
 
 ## Research Use and Disclaimer
 
-OpenCUA models are intended for **research and educational purposes only**. 
+OpenCUA models are intended for **research and educational purposes only**.
 
 ### Prohibited Uses
 - The model may **not** be used for any purpose or activity that violates applicable laws or regulations in any jurisdiction
@@ -428,52 +406,38 @@ OpenCUA models are intended for **research and educational purposes only**.
 ## Important Notes on Coordinate Systems
 <div style="border-left: 6px solid #9ca3af; background: #f5f5f5; padding: 12px 16px; margin: 16px 0;">
   <ul style="margin: 0;">
-    <li><strong><code>OpenCUA/OpenCUA-A3B</code></strong> – Relative coordinates <em>(not supported in this code)</em></li>
-    <li><strong><code>OpenCUA/OpenCUA-Qwen2-7B</code></strong> – Relative coordinates</li>
     <li><strong><code>OpenCUA/OpenCUA-7B</code></strong> – Absolute coordinates</li>
     <li><strong><code>OpenCUA/OpenCUA-32B</code></strong> – Absolute coordinates</li>
+    <li><strong><code>OpenCUA/OpenCUA-72B</code></strong> – Absolute coordinates</li>
   </ul>
 </div>
 
-**OpenCUA models use different coordinate systems depending on the base model:**
-
-- **OpenCUA-Qwen2-7B**: Outputs **relative coordinates** (0.0 to 1.0 range)
-  ```python
-  # Example output: pyautogui.click(x=0.5, y=0.3)
-  # x=0.5 means 50% from left edge, y=0.3 means 30% from top edge
-  
-  # Convert to absolute coordinates:
-  def qwen2_relative_to_absolute(rel_x, rel_y, original_width, original_height):
-      abs_x = int(rel_x * original_width)
-      abs_y = int(rel_y * original_height)
-      return abs_x, abs_y
-  ```
-
-- **OpenCUA-7B and OpenCUA-32B** (Qwen2.5-based): Output **absolute coordinates** after smart resize
-  ```python
-  # Example output: pyautogui.click(x=960, y=324)  
-  # These are coordinates on the smart-resized image, not the original image
-  
-  # Convert to original image coordinates:
-  # Please refer to the smart_resize function in: https://github.com/huggingface/transformers/blob/67ddc82fbc7e52c6f42a395b4a6d278c55b77a39/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py#L55
-  def qwen25_smart_resize_to_absolute(model_x, model_y, original_width, original_height):
-      # First, calculate the smart-resized dimensions
-      resized_height, resized_width = smart_resize(original_height, original_width, factor = 28, min_pixels = 3136, max_pixels = 12845056)
-      
-      # Convert model output to relative coordinates on original image
-      rel_x = model_x / resized_width
-      rel_y = model_y / resized_height
-      
-      # Then convert to absolute coordinates on original image
-      abs_x = int(rel_x * original_width)
-      abs_y = int(rel_y * original_height)
-      return abs_x, abs_y
-  ```
+**OpenCUA models output absolute coordinates after smart resize:**
+
+```python
+# Example output: pyautogui.click(x=960, y=324)
+# These are coordinates on the smart-resized image, not the original image
+
+# Convert to original image coordinates:
+# Please refer to the smart_resize function in: https://github.com/huggingface/transformers/blob/67ddc82fbc7e52c6f42a395b4a6d278c55b77a39/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py#L55
+def qwen25_smart_resize_to_absolute(model_x, model_y, original_width, original_height):
+    # First, calculate the smart-resized dimensions
+    resized_height, resized_width = smart_resize(original_height, original_width, factor = 28, min_pixels = 3136, max_pixels = 12845056)
+
+    # Convert model output to relative coordinates on original image
+    rel_x = model_x / resized_width
+    rel_y = model_y / resized_height
+
+    # Then convert to absolute coordinates on original image
+    abs_x = int(rel_x * original_width)
+    abs_y = int(rel_y * original_height)
+    return abs_x, abs_y
+```
 
 <div style="border-left: 6px solid #9ca3af; background: #f5f5f5; padding: 12px 16px; margin: 16px 0;">
   <strong>Understanding Smart Resize for Qwen2.5-based Models:</strong>
   <p style="margin: 8px 0 0;">
-    The Qwen2.5-VL models use a “smart resize” preprocessing that maintains aspect ratio while fitting within pixel constraints.
+    The Qwen2.5-VL models use a "smart resize" preprocessing that maintains aspect ratio while fitting within pixel constraints.
     For coordinate conversion, you need the smart resize function from the
     <a href="https://github.com/QwenLM/Qwen2.5-VL/blob/d2240f11656bfe404b9ba56db4e51cd09f522ff1/qwen-vl-utils/src/qwen_vl_utils/vision_process.py#L60">
       official Qwen2.5-VL implementation</a>.
@@ -486,14 +450,14 @@ If you use OpenCUA models in your research, please cite our work:
 
 ```bibtex
 @misc{wang2025opencuaopenfoundationscomputeruse,
-      title={OpenCUA: Open Foundations for Computer-Use Agents}, 
+      title={OpenCUA: Open Foundations for Computer-Use Agents},
       author={Xinyuan Wang and Bowen Wang and Dunjie Lu and Junlin Yang and Tianbao Xie and Junli Wang and Jiaqi Deng and Xiaole Guo and Yiheng Xu and Chen Henry Wu and Zhennan Shen and Zhuokai Li and Ryan Li and Xiaochuan Li and Junda Chen and Boyuan Zheng and Peihang Li and Fangyu Lei and Ruisheng Cao and Yeqiao Fu and Dongchan Shin and Martin Shin and Jiarui Hu and Yuyan Wang and Jixuan Chen and Yuxiao Ye and Danyang Zhang and Dikang Du and Hao Hu and Huarong Chen and Zaida Zhou and Haotian Yao and Ziwei Chen and Qizheng Gu and Yipu Wang and Heng Wang and Diyi Yang and Victor Zhong and Flood Sung and Y. Charles and Zhilin Yang and Tao Yu},
       year={2025},
       eprint={2508.09123},
       archivePrefix={arXiv},
       primaryClass={cs.AI},
-      url={https://arxiv.org/abs/2508.09123}, 
+      url={https://arxiv.org/abs/2508.09123},
 }
 ```
 
-</div>
+</div>