google/skills実行前に内容を確認
SKILL DETAIL
agent-platform-tuning
google/skills/agent-platform-tuning
>-
インストール · 105出典を見る
Installation
npx skills add https://github.com/google/skills --skill agent-platform-tuning
スキルファイル
SKILL.md
最終同期 · 2026/08/29
references/data_prep.md›
# Data Preparation for Agent Platform Model Tuning
Agent Platform Model Tuning requires training data in **JSON Lines (JSONL)**
format stored in Google Cloud Storage (GCS).
## Supported JSONL Formats for Open Models
### 1. Conversational (Messages) Format
Recommended for chat-based models (Llama 3.1/3.2/3.3 Chat, Gemma 3 IT, etc.).
```json
{
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"},
{"role": "assistant", "content": "The capital of France is Paris."}
]
}
```
### 2. Instruction (Prompt/Completion) Format
Suitable for base models or simple completion tasks.
```json
{
"prompt": "Summarize the following text: [TEXT]",
"completion": "[SUMMARY]"
}
```
## Dataset Requirements
- **File Type**: Must be `.jsonl`.
- **Encoding**: UTF-8.
- **Location**: Must be in a GCS bucket (e.g., `gs://my-bucket/train.jsonl`).
- **Validation Split**: A separate validation file is optional but recommended. It must be no more than 25% of the training dataset **file size in bytes**, and no more than 5000 rows.
### Sizing the validation split
The 25% ceiling is measured against the training file, not against the whole
dataset, so a split fraction `s` has to satisfy `s / (1 - s) <= 0.25`, i.e.
`s <= 0.2`. An 80/20 split therefore sits exactly on the limit and is rejected
as soon as the held-out rows are slightly longer than average -- observed
overshoots run from 25.03% to 26.45%. Use `--validation_split 0.1`, which
leaves the validation file at about 11% of the training file.
`scripts/prepare_dataset.py` measures the written files and fails before
upload if the ratio is over the limit, so a rejection here never costs a
tuning job submission.
## Bucket Considerations
Artifacts must live in a bucket the user has chosen. **Never invent a bucket
name, derive one from the project number, or create a bucket unprompted.**
Creating a bucket is a mutating action and requires explicit Tier M
confirmation.
If the user has not named a bucket, stop and ask which one they want to use, and
offer to create one for them as one of the options. Only once they have
confirmed the exact name and location should you run the create command.
Because the default tuning location is `global` — the service picks whichever
region has GPU capacity — a new bucket should be a multi-region so it stays
reachable wherever the job lands. Propose `US` (or `EU` if the data must stay in
Europe):
```bash
# Run only after the user has confirmed the bucket name and location.
# Substitute the confirmed name; never run this with the placeholder as-is.
gcloud storage buckets create gs://CONFIRMED_BUCKET_NAME --location=US
```
Only pin the bucket to a single region when the tuning job itself is pinned to
that region.
## Formatting Best Practices
1. **Quality over Quantity**: 100 high-quality examples often outperform 1,000 noisy ones.
2. **Consistency**: Use consistent formatting for system prompts and instruction styles.
3. **No Empty Values**: Ensure every example has a valid prompt/user message and
completion/assistant response. Use the [preparation script](../scripts/prepare_dataset.py) to validate this.
references/hf_datasets.md›
# Huggingface Dataset References
This document provides references for various datasets used in Agent Platform
Tuning.
## How to use datasets
When using a dataset for tuning, it is recommended to first analyze the dataset to understand its structure and content. For more information on data preparation and formatting, please refer to [Data Preparation for Tuning](data_prep.md).
To download the entire dataset, the following script can be used:
```python
from datasets import load_dataset
dataset = load_dataset("username/dataset_name")
```
To load a specific split, the following script can be used:
```python
from datasets import load_dataset
dataset = load_dataset("username/dataset_name", split="split_name")
```
For larger datasets, it might be helpful to subset the dataset and use only
portions of the dataset as required by the specific task requested. Either use
the splits defined in the huggingface website (ex. `default`) or offer to
partition the dataset for the user. Ensure that the user can see some examples
of the dataset before proceeding.
[!IMPORTANT]
**CRITICAL: Ask for Confirmation and Column Selection.**
Do not proceed with dataset preparation or upload until you perform the
following steps and get user confirmation:
1. **Dataset and Split Confirmation:** Present the dataset and available splits
to the user and have them confirm which to use. Additionally, show a few
samples to the user for preview.
2. **Column Selection:** The tuning entrypoint requires mapping source columns to the target format (either `prompt`/`completion` or `messages` format). You must:
- Provide a list of all available columns in the selected dataset split.
- Recommend which columns should be mapped to `prompt` (or user message) and `completion` (or assistant response), offering a few reasonable options if applicable.
- Ask the user to confirm the column mapping or specify which columns to use.
## Datasets List
Each dataset includes a brief description of the dataset type and some usage
hints. The hints are **not** the only way to use these datasets but offer some
suggestions based on the corresponding dataset.
[!IMPORTANT]
Examine the hints since they include some information about the dataset. These
include comments about the contents of the dataset that may be pertinent to the
users request.
### General and Reasoning Tasks
#### Mathematical Reasoning
| Name | Description | Sample Count (Split) | Usage Hints |
|---|---|---|---|
| [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k) | A dataset of math problems and solutions. | 93,700 (default) - 220,000 (full) | The main columns are problem and solution. Some other helpful columns are answer, problem_type, question_type, and messages. |
| [AI-MO/NuminaMath-TIR](https://huggingface.co/datasets/AI-MO/NuminaMath-TIR) | A dataset for improving model performance in complex logic and calculations. | N/A | Good choice for mathematical reasoning. |
#### Instruction Following
| Name | Description | Sample Count (Split) | Usage Hints |
|---|---|---|---|
| [argilla/ifeval-like-data](https://huggingface.co/datasets/argilla/ifeval-like-data) | A dataset that involves instruction following. | 550,000 (default), 56,000 (filtered) | There are multiple languages in this dataset. Prompt the user with this dataset if specific languages are expected. Filter accordingly |
| [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) | Enhancing broad instruction-following capabilities. | N/A | This needs to be subsetted. as the initial dataset is very large and covers a wide range of tasks. |
#### Multilingual Support
| Name | Description | Sample Count (Split) | Usage Hints |
|---|---|---|---|
| [CohereForAI/aya_dataset](https://huggingface.co/datasets/CohereForAI/aya_dataset) | Expanding linguistic capabilities across diverse languages. | N/A | Contains multi-language instruction following data. |
### Specialized and Technical Tasks
#### Programming & Coding
| Name | Description | Sample Count (Split) | Usage Hints |
|---|---|---|---|
| [ise-uiuc/Magicoder-Evol-Instruct-110K](https://huggingface.co/datasets/ise-uiuc/Magicoder-Evol-Instruct-110K) | Code generation dataset. | 110,000 | Suitable for advancing coding capabilities. |
| [open-r1/datasets](https://huggingface.co/open-r1/datasets) | Specialized programming & reasoning data. | N/A | General source for open reasoning technical data. |
#### Tool Use & Integration
| Name | Description | Sample Count (Split) | Usage Hints |
|---|---|---|---|
| [gorilla-llm/Berkeley-Function-Calling-Leaderboard](https://huggingface.co/datasets/gorilla-llm/Berkeley-Function-Calling-Leaderboard) | Adhere to constraints and use external systems. | N/A | High-quality tool usage and function calling data. |
| [Bingguang/HardGen](https://huggingface.co/datasets/Bingguang/HardGen) | Evaluating handling complex tools and constraints. | N/A | Validated for tool use integration tasks. |
references/models.md›
# Agent Platform Supported Models and Recommendations
This reference catalog provides technical specifications, tuning
recommendations, and deployment hardware requirements for supported models in
Agent Platform.
## Supported Models Catalog
> [!WARNING] **CRITICAL AGENT INSTRUCTION**
> Do NOT use this catalog to recommend a specific model to the user until they
> have explicitly confirmed their **Model Category** as Open Model.
> Furthermore, do NOT recommend any model that is not explicitly listed in this
> catalog, as the tuning service does not support it.
> When a user asks for a model that is not listed, do not stop at refusing it.
> Say it is not supported for tuning, then offer the closest model that is
> listed, preferring the same family and the nearest size, and let the user
> decide. A model missing from this catalog is usually a release that is newer
> than the tuning service supports, so the previous release of that same family
> is normally the right suggestion.
Available open models can be found in Google Cloud [documentation](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning/open-model-tuning.md.txt).
This is the list of open models that are available for tuning; do not suggest
any other open models besides the one listed here.
Each model has some [limitations](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning/open-model-tuning.md.txt) for tuning.
## Model Resource Name Format
The names in the **Model** column below are display names. The tuning API does
not accept them. `--base_model` requires a publisher model **resource name**,
listed in the **Resource name** column of the table in
[Baseline Hyperparameter Recommendations](#baseline-hyperparameter-recommendations):
```
{publisher}/{model_id}@{version_id}
```
The longer form `publishers/{publisher}/models/{model_id}@{version_id}` is also
accepted.
> [!WARNING] Copy the resource name from the table; **do not construct it**.
> The `@{version_id}` suffix is mandatory, `{model_id}` is the *family*
> (`gemma3`, `qwen3`) rather than the variant, and the publisher is lowercase.
> Omitting the suffix is the single most common cause of
> `INVALID_ARGUMENT: Invalid open source publisher model resource name`;
> `google/gemma-3-4b-it` and `Qwen/Qwen3-8B` are both rejected.
The family segment cannot be derived from the display name:
- `meta/llama3_1` uses an underscore, but `meta/llama3-2` and `meta/llama3-3`
use hyphens.
- Qwen 3.5 9B is `qwen/[email protected]` -- hyphen in the family, dot in the
version.
- Medgemma 1.5 4B IT is `google/[email protected]`; the version, not
the family, carries the `1.5`.
This format applies to `--base_model` for supervised tuning. Distillation
teacher models use a different, unversioned form, such as
`qwen/qwen3-next-80b-a3b-thinking-maas`.
## Model Selection Guidelines
**Identify Task**: Check a few samples from the dataset to identify the task.
Choose a model family based on your task type:
- **Qwen**: Best for code generation or complex math-based tasks.
- **Gemma**: Optimized for chat-based interactions, creative writing and multilingual tasks.
- **Llama (Instruct)**: Strong general-purpose chat/instruction models.
- **Llama (Base/Scout)**: Best for continuation tasks or building custom instruction-tuned models.
**Complexity Heuristics**:
- **Simple (QA, Extraction)**: 1B - 3B models.
- **Intermediate (Summarization, Reasoning)**: 8B - 17B models.
- **Complex (Multi-turn, Tool use, Deep reasoning)**: 27B - 70B models.
## Baseline Hyperparameter Recommendations
These values are starting points and should be adjusted based on your dataset
size.
The **Resource name** column is the exact value to pass to `--base_model`.
| Model | Resource name for `--base_model` | Tuning Mode | Learning Rate | Epochs | Adapter Size (PEFT) |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Gemma 4 E2B IT | `google/gemma4@gemma-4-e2b-it` | PEFT | 2.0E-4 | 3 | 16 |
| Gemma 4 E4B IT | `google/gemma4@gemma-4-e4b-it` | PEFT | 2.0E-4 | 3 | 16 |
| Gemma 4 26B A4B IT | `google/gemma4@gemma-4-26b-a4b-it` | PEFT | 2.0E-4 | 3 | 16 |
| Gemma 4 31B IT | `google/gemma4@gemma-4-31b-it` | PEFT | 2.0E-4 | 3 | 16 |
| Gemma 3 1B IT | `google/gemma3@gemma-3-1b-it` | Full | 2.0E-5 | 3 | N/A |
| Gemma 3 4B IT | `google/gemma3@gemma-3-4b-it` | Full | 1.0E-5 | 3 | N/A |
| Gemma 3 12B IT | `google/gemma3@gemma-3-12b-it` | Full | 1.0E-5 | 3 | N/A |
| Gemma 3 27B IT | `google/gemma3@gemma-3-27b-it` | PEFT | 2.0E-4 | 3 | 32 |
| Gemma 3 27B IT | `google/gemma3@gemma-3-27b-it` | Full | 2.0E-4 | 3 | N/A |
| Medgemma 1.5 4B IT | `google/[email protected]` | Full | 1.0E-5 | 3 | N/A |
| Llama 3.1 8B | `meta/[email protected]` | PEFT | 2.0E-4 | 3 | 16 |
| Llama 3.1 8B | `meta/[email protected]` | Full | 2.0E-4 | 3 | N/A |
| Llama 3.1 8B Instruct | `meta/[email protected]` | PEFT | 2.0E-4 | 3 | 16 |
| Llama 3.1 8B Instruct | `meta/[email protected]` | Full | 2.0E-4 | 3 | N/A |
| Llama 3.2 1B Instruct | `meta/[email protected]` | Full | 1.5E-6 | 3 | N/A |
| Llama 3.2 3B Instruct | `meta/[email protected]` | Full | 1.0E-7 | 3 | N/A |
| Llama 3.3 70B Instruct | `meta/[email protected]` | PEFT | 5.0E-5 | 3 | 16 |
| Llama 3.3 70B Instruct | `meta/[email protected]` | Full | 5.0E-5 | 3 | N/A |
| Llama 4 Scout 17B 16E Instruct | `meta/llama4@llama-4-scout-17b-16e-instruct` | PEFT | 2.0E-5 | 3 | 16 |
| Qwen 3.5 9B | `qwen/[email protected]` | Full | 2e-5 | 3 | N/A |
| Qwen 3 4B | `qwen/qwen3@qwen3-4b` | Full | 7.5e-5 | 3 | N/A |
| Qwen 3 8B | `qwen/qwen3@qwen3-8b` | Full | 5e-5 | 3 | N/A |
| Qwen 3 14B | `qwen/qwen3@qwen3-14b` | Full | 4e-5 | 3 | N/A |
| Qwen 3 32B | `qwen/qwen3@qwen3-32b` | PEFT | 2.0E-4 | 3 | 16 |
| Qwen 3 32B | `qwen/qwen3@qwen3-32b` | Full | 2.5e-5 | 3 | N/A |references/requirements.txt›
google-cloud-aiplatform==1.138.0
datasets==2.18.0
google-cloud-storage==3.13.0
google-genai==1.75.0
references/tuning_guide.md›
# Agent Platform Model Tuning Heuristics and Concepts
This guide details the core concepts of fine-tuning and provides heuristics for
adjusting hyperparameters based on your specific dataset.
## Core Tuning Concepts
### Open Models Tuning Modes
- **FULL**: Updates all parameters of the model. Requires more GPU memory and a larger dataset to avoid catastrophic forgetting.
- **PEFT_ADAPTER**: Parameter-Efficient Fine-Tuning. Only a small set of "adapter" weights are trained. Faster, uses less memory, and is less prone to overfitting on small datasets.
### Hyperparameters
- **Epochs**: Number of times the model sees the entire dataset.
- **Learning Rate**: Step size for optimization. Too high can cause instability; too low can lead to very slow convergence.
- **Adapter Size (Rank)**: For PEFT_ADAPTER, this determines the capacity of the adapters. Higher rank allows more complex learning but increases the risk of overfitting.
## Dataset Heuristics
The size and quality of your dataset should dictate your parameter choices. Refer to [Models Catalog](models.md) for baseline values, then adjust as follows:
### 1. Dataset Size Implications
| Dataset Size | Tuning Mode Recommendation | Learning Rate Adjustment | Epochs Recommendation |
| :--- | :--- | :--- | :--- |
| **< 100 examples** | PEFT_ADAPTER (Rank 8) | Lower than baseline | 1-2 |
| **100 - 1000 examples** | PEFT_ADAPTER (Rank 16/32) | Baseline | 3 |
| **> 1000 examples** | FULL or PEFT_ADAPTER (Rank 32) | Higher than baseline | 3-5 |
### 2. General Best Practices
- **Overfitting**: If validation loss starts increasing while training loss decreases, you are overfitting. Reduce epochs or decrease the learning rate.
- **Underfitting**: If both training and validation loss remain high, increase the learning rate or use more epochs.
- **Validation**: Always use a validation set to monitor performance. If not provided, a 10% split is highly recommended; do not go above it, for the reason given in [Data Preparation Guide](data_prep.md#sizing-the-validation-split).
- **Checkpoints**: The final model is always saved to `<output_uri>/postprocess/node-0/checkpoints/final`.
## Hardware and Limitations
For specific hardware recommendations and sequence length limits per model, please refer to the [Models Catalog](models.md).
scripts/calculate_cost.py›
"""Calculate tuning cost for a given dataset and model."""
import argparse
import json
import re
import sys
from typing import TextIO
from google.cloud import storage
# Keyed by the publisher model resource name, the identifier --base_model
# requires and the one references/models.md tells the agent to copy verbatim.
# display_name is the label that table lists it under, kept here so errors can
# name a model the way the docs do; both spellings resolve through
# resolve_model().
#
# The catalog is wider than the price table: an entry with no modes is offered
# for tuning but has no published SKU yet, a routine state worth telling apart
# from an identifier that does not exist.
#
# cost_per_1m_tokens mirrors the public Model Tuning price table:
# https://cloud.google.com/gemini-enterprise-agent-platform/generative-ai/pricing#model-tuning
#
# tokens_per_character is the empirical ratio of billed training tokens to
# dataset characters as counted by count_characters() below, so it absorbs
# chat template overhead. It is measured per tuning mode: PEFT and Full run
# different training containers, so the same dataset can bill a different
# number of tokens in each mode. Do not assume the two modes of a model
# share a value. It is not published anywhere: unlike the price, it has to be
# measured, and the authoritative billable token count is only reported by the
# tuning service after a job completes, which is too late to estimate from.
MODEL_DATA = {
'google/gemma4@gemma-4-e2b-it': {
'display_name': 'Gemma 4 E2B IT',
'modes': {
'PEFT': {'tokens_per_character': 0.236, 'cost_per_1m_tokens': 1.40},
},
},
'google/gemma4@gemma-4-e4b-it': {
'display_name': 'Gemma 4 E4B IT',
'modes': {
'PEFT': {'tokens_per_character': 0.236, 'cost_per_1m_tokens': 1.73},
},
},
'google/gemma4@gemma-4-26b-a4b-it': {
'display_name': 'Gemma 4 26B A4B IT',
'modes': {
'PEFT': {'tokens_per_character': 0.236, 'cost_per_1m_tokens': 2.58},
},
},
'google/gemma4@gemma-4-31b-it': {
'display_name': 'Gemma 4 31B IT',
'modes': {
'PEFT': {'tokens_per_character': 0.236, 'cost_per_1m_tokens': 9.06},
},
},
'google/gemma3@gemma-3-1b-it': {
'display_name': 'Gemma 3 1B IT',
'modes': {
'Full': {'tokens_per_character': 0.231, 'cost_per_1m_tokens': 0.47},
},
},
'google/gemma3@gemma-3-4b-it': {
'display_name': 'Gemma 3 4B IT',
'modes': {
'Full': {'tokens_per_character': 0.231, 'cost_per_1m_tokens': 1.14},
},
},
'google/gemma3@gemma-3-12b-it': {
'display_name': 'Gemma 3 12B IT',
'modes': {
'Full': {'tokens_per_character': 0.231, 'cost_per_1m_tokens': 1.82},
},
},
'google/gemma3@gemma-3-27b-it': {
'display_name': 'Gemma 3 27B IT',
'modes': {
'PEFT': {'tokens_per_character': 0.231, 'cost_per_1m_tokens': 6.83},
'Full': {'tokens_per_character': 0.231, 'cost_per_1m_tokens': 6.83},
},
},
'google/[email protected]': {
'display_name': 'Medgemma 1.5 4B IT',
'modes': {
'Full': {'tokens_per_character': 0.231, 'cost_per_1m_tokens': 1.14},
},
},
'meta/[email protected]': {
'display_name': 'Llama 3.1 8B',
'modes': {
'PEFT': {'tokens_per_character': 0.317, 'cost_per_1m_tokens': 0.67},
'Full': {'tokens_per_character': 0.247, 'cost_per_1m_tokens': 0.67},
},
},
'meta/[email protected]': {
'display_name': 'Llama 3.1 8B Instruct',
'modes': {
'PEFT': {'tokens_per_character': 0.317, 'cost_per_1m_tokens': 0.67},
'Full': {'tokens_per_character': 0.247, 'cost_per_1m_tokens': 0.67},
},
},
'meta/[email protected]': {
'display_name': 'Llama 3.2 1B Instruct',
'modes': {
'Full': {'tokens_per_character': 0.247, 'cost_per_1m_tokens': 0.28},
},
},
'meta/[email protected]': {
'display_name': 'Llama 3.2 3B Instruct',
'modes': {
'Full': {'tokens_per_character': 0.247, 'cost_per_1m_tokens': 0.61},
},
},
'meta/[email protected]': {
'display_name': 'Llama 3.3 70B Instruct',
'modes': {
'PEFT': {'tokens_per_character': 0.317, 'cost_per_1m_tokens': 6.72},
'Full': {'tokens_per_character': 0.247, 'cost_per_1m_tokens': 6.72},
},
},
'meta/llama4@llama-4-scout-17b-16e-instruct': {
'display_name': 'Llama 4 Scout 17B 16E Instruct',
'modes': {
'PEFT': {'tokens_per_character': 0.295, 'cost_per_1m_tokens': 5.77},
},
},
'qwen/[email protected]': {
'display_name': 'Qwen 3.5 9B',
'modes': {}, # Catalogued for tuning, no published price yet.
},
'qwen/qwen3@qwen3-4b': {
'display_name': 'Qwen 3 4B',
'modes': {
'Full': {'tokens_per_character': 0.246, 'cost_per_1m_tokens': 1.35},
},
},
'qwen/qwen3@qwen3-8b': {
'display_name': 'Qwen 3 8B',
'modes': {
'Full': {'tokens_per_character': 0.246, 'cost_per_1m_tokens': 4.18},
},
},
'qwen/qwen3@qwen3-14b': {
'display_name': 'Qwen 3 14B',
'modes': {
'Full': {'tokens_per_character': 0.246, 'cost_per_1m_tokens': 8.46},
},
},
'qwen/qwen3@qwen3-32b': {
'display_name': 'Qwen 3 32B',
'modes': {
'PEFT': {'tokens_per_character': 0.246, 'cost_per_1m_tokens': 6.57},
'Full': {'tokens_per_character': 0.246, 'cost_per_1m_tokens': 6.57},
},
},
}
_LONG_RESOURCE_NAME = re.compile(r'publishers/([^/]+)/models/(.+)')
def _normalize(name: str) -> str:
"""Folds a model identifier to the key form used by _MODEL_ALIASES."""
name = name.strip().lower()
# models.md documents `publishers/{publisher}/models/{model}@{version}` as
# an accepted long form of `{publisher}/{model}@{version}`.
match = _LONG_RESOURCE_NAME.fullmatch(name)
if match:
name = f'{match.group(1)}/{match.group(2)}'
return name
def _build_aliases() -> dict[str, str]:
"""Maps every accepted spelling of a model to its MODEL_DATA key."""
aliases = {}
for resource_name, entry in MODEL_DATA.items():
aliases[_normalize(resource_name)] = resource_name
aliases[_normalize(entry['display_name'])] = resource_name
return aliases
_MODEL_ALIASES = _build_aliases()
def resolve_model(name: str) -> str | None:
"""Resolves any accepted spelling of a model to its MODEL_DATA key.
Args:
name: Either a display name ("Qwen 3 8B") or a publisher model resource name
("qwen/qwen3@qwen3-8b", optionally in the longer
"publishers/qwen/models/qwen3@qwen3-8b" form). Matching ignores case and
surrounding whitespace.
Returns:
The MODEL_DATA key, or None if the model is not in the catalog. A returned
key is not a promise that the model is priced -- an entry with no modes is
catalogued but has no published SKU.
"""
return _MODEL_ALIASES.get(_normalize(name))
def _priced_model_listing() -> str:
"""Renders the priced models as indented `Display name (resource name)`."""
return '\n'.join(
f' {entry["display_name"]} ({resource_name})'
for resource_name, entry in MODEL_DATA.items()
if entry['modes']
)
def _open_jsonl(input_file: str) -> TextIO:
"""Opens a local or ``gs://`` jsonl file for streaming text reads.
Args:
input_file: A local filesystem path or a ``gs://bucket/object`` URI.
Returns:
An open text-mode file object; the caller is responsible for closing it.
Raises:
ValueError: If the path uses a scheme other than ``gs://``, or is a
malformed ``gs://`` URI.
"""
if input_file.startswith('gs://'):
bucket_name, _, blob_name = input_file[len('gs://') :].partition('/')
if not bucket_name or not blob_name:
raise ValueError(
f'Malformed GCS path: {input_file}. Expected gs://<bucket>/<object>.'
)
return storage.Client().bucket(bucket_name).blob(blob_name).open('r')
if '://' in input_file:
raise ValueError(
f'Unsupported file path: {input_file}. '
'Only local paths and gs:// paths are supported.'
)
return open(input_file, 'r')
def count_characters(input_file: str) -> int:
"""Counts the characters in a jsonl dataset.
It is expected that each line in the jsonl file is a json object
with a "messages" key, which is a list of dictionaries. Each
dictionary in the "messages" list should have a "content" key.
This function counts the characters in the "content" field of each
dictionary in the "messages" list.
Args:
input_file: Path to the input jsonl file.
Returns:
Total character count.
"""
total_character_count = 0
with _open_jsonl(input_file) as f:
for line in f:
data = json.loads(line)
for message in data['messages']:
content = message['content']
total_character_count += len(content)
return total_character_count
def calculate_cost(
count: int,
model: str,
tuning_mode: str,
epochs: int,
) -> float:
"""Calculates the tuning cost.
Args:
count: Total character count of the dataset.
model: MODEL_DATA key for the model, as returned by resolve_model().
tuning_mode: Tuning mode.
epochs: Number of epochs.
Returns:
Estimated tuning cost.
"""
model_data = MODEL_DATA[model]['modes'][tuning_mode]
tokens_per_character = model_data['tokens_per_character']
cost_per_1m_tokens = model_data['cost_per_1m_tokens']
num_tokens = count * tokens_per_character * epochs
return (num_tokens / 1000000) * cost_per_1m_tokens
if __name__ == '__main__':
parser = argparse.ArgumentParser(
description='Calculate tuning cost for a given dataset and model.'
)
parser.add_argument('--input', help='Input jsonl file.', required=True)
# No choices= here on purpose. references/models.md and MODEL_DATA are two
# hand-maintained lists that drift, and a model present in the catalog but
# absent from the price table is a routine, expected state while a tuning
# SKU is pending. argparse would reject it with a usage dump; the explicit
# check below reports it in a form the caller can act on.
parser.add_argument(
'--model',
help=(
'Model to use for tuning. Accepts either the display name'
' ("Qwen 3 8B") or the publisher model resource name'
' ("qwen/qwen3@qwen3-8b") that --base_model takes.'
),
required=True,
)
parser.add_argument(
'--tuning_mode',
help='Tuning mode.',
required=True,
choices=['PEFT', 'Full'],
)
parser.add_argument(
'--epochs',
help='Number of epochs.',
required=True,
type=int,
)
args = parser.parse_args()
resolved_model = resolve_model(args.model)
if resolved_model is None:
print(
f'Error: Unrecognized model {args.model}. Pass a display name or the'
' publisher model resource name given to --base_model. Models with a'
f' published tuning price:\n{_priced_model_listing()}'
)
sys.exit(1)
model_entry = MODEL_DATA[resolved_model]
if not model_entry['modes']:
print(
'Error: No published tuning price for model'
f' {model_entry["display_name"]}, so its cost cannot be estimated.'
f' Models with a published price:\n{_priced_model_listing()}'
)
sys.exit(1)
if args.tuning_mode not in model_entry['modes']:
print(
f'Error: Tuning mode {args.tuning_mode} not supported for model'
f' {model_entry["display_name"]}. Supported:'
f' {", ".join(model_entry["modes"])}'
)
sys.exit(1)
character_count = count_characters(args.input)
cost = calculate_cost(
character_count, resolved_model, args.tuning_mode, args.epochs
)
print(f'Total character count: {character_count}')
print(f'Estimated tuning cost: ${cost:.2f}')
scripts/cancel_tuning_job.py›
"""Cancels a Agent Platform Supervised Tuning Job.
This script provides a function to cancel an ongoing SupervisedTuningJob
on Agent Platform given the project, location, and job ID.
"""
import argparse
import vertexai # pyrefly: ignore[missing-import]
from vertexai.tuning import sft # pyrefly: ignore[missing-import]
def cancel_job(project, location, job_id):
vertexai.init(project=project, location=location)
job_resource = f"projects/{project}/locations/{location}/tuningJobs/{job_id}"
job = sft.SupervisedTuningJob(job_resource)
print(f"Cancelling job: {job_resource}")
job.cancel()
print("Cancellation request sent.")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Cancel Vertex AI Supervised Tuning Job"
)
parser.add_argument("--project", required=True)
# Must match the location the job was submitted with, which is `global` for
# open model jobs that did not pin a region.
parser.add_argument("--location", required=True)
parser.add_argument("--job_id", required=True)
args = parser.parse_args()
cancel_job(args.project, args.location, args.job_id)
scripts/list_models.py›
"""Lists Model Garden base model IDs for the tuning skill. See SKILL.md."""
import argparse
import json
import sys
import vertexai # pyrefly: ignore[missing-import]
from vertexai import model_garden # pyrefly: ignore[missing-import]
_TOP_K = 20
def list_model_ids(
project: str, location: str, model_filter: str = ""
) -> dict[str, object]:
"""Returns up to `_TOP_K` Model Garden IDs matching `model_filter`."""
vertexai.init(project=project, location=location)
models = model_garden.list_models(model_filter=model_filter)
selected = models[:_TOP_K]
return {
"models": selected,
"total_count": len(models),
"truncated": len(models) > len(selected),
}
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--project", required=True)
parser.add_argument("--location", default="us-central1")
parser.add_argument("--filter", dest="model_filter", default="")
args = parser.parse_args()
json.dump(list_model_ids(**vars(args)), sys.stdout, indent=2)
sys.stdout.write("\n")
scripts/monitor_tuning_job.py›
"""Monitors a Agent Platform Supervised Tuning Job."""
import argparse
import logging
import time
from google import genai
def monitor_job(
project: str, location: str, job_id: str, poll_interval_secs: int = 60
):
"""Monitors a Agent Platform Supervised Tuning Job.
This function polls the job status until it reaches a terminal state.
Args:
project: The Google Cloud project ID.
location: The Google Cloud location.
job_id: The ID of the SupervisedTuningJob.
poll_interval_secs: The interval in seconds to poll the job status.
"""
job_resource = (
f"projects/{project}/locations/{location}/tuningJobs/{job_id}"
)
logging.info("Starting monitoring for tuning job: %s", job_resource)
logging.info(
"View job in console: https://console.cloud.google.com/agent-platform/"
"tuning/locations/%s/tuningJob/%s/monitor?project=%s",
location,
job_id,
project,
)
with genai.Client(
enterprise=True, project=project, location=location
) as client:
while True:
job = client.tunings.get(name=job_resource)
status = job.state
status_name = status.name if status else "unknown"
if status_name in (
"JOB_STATE_SUCCEEDED",
"JOB_STATE_FAILED",
"JOB_STATE_CANCELLED",
"JOB_STATE_PARTIALLY_SUCCEEDED",
):
logging.info("Job finished with terminal state: %s", status_name)
break
else:
logging.info("Current job status: %s", status_name)
time.sleep(poll_interval_secs)
if __name__ == "__main__":
logging.basicConfig(
level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s"
)
parser = argparse.ArgumentParser(
description="Monitor Agent Platform Supervised Tuning Job"
)
parser.add_argument("--project", required=True)
# Must match the location the job was submitted with, which is `global` for
# open model jobs that did not pin a region.
parser.add_argument("--location", required=True)
parser.add_argument("--job_id", required=True)
parser.add_argument(
"--poll_interval_secs",
type=int,
default=60,
help="Seconds to wait between polls",
)
args = parser.parse_args()
monitor_job(args.project, args.location, args.job_id, args.poll_interval_secs)
scripts/prepare_dataset.py›
"""Module for preparing and validating datasets for Agent Platform model tuning."""
import argparse
import json
import logging
import os
import sys
from typing import Any
import datasets
# The tuning service rejects a job whose validation file exceeds this fraction
# of the training file, measured in bytes rather than rows. 0.2 is the largest
# split that satisfies it and lands exactly on the line, so the default keeps a
# margin -- see references/data_prep.md, "Sizing the validation split".
MAX_VALIDATION_RATIO = 0.25
DEFAULT_VALIDATION_SPLIT = 0.1
def _validate_example(example: dict[str, Any], format_type: str) -> bool:
"""Validates a single example against the expected format."""
if format_type == "messages":
if "messages" not in example or not isinstance(example["messages"], list):
return False
for msg in example["messages"]:
if not all(k in msg for k in ("role", "content")):
return False
if not msg["content"] or str(msg["content"]).strip().lower() == "nan":
return False
else:
if not all(k in example for k in ("prompt", "completion")):
return False
for k in ("prompt", "completion"):
if not example[k] or str(example[k]).strip().lower() == "nan":
return False
return True
def _format_row(
row, format_type: str, prompt_col: str, completion_col: str
) -> dict[str, Any]:
"""Formats a single row into the expected JSON structure."""
prompt_text = str(row[prompt_col])
completion_text = str(row[completion_col])
if format_type == "messages":
return {
"messages": [
{"role": "user", "content": prompt_text},
{"role": "assistant", "content": completion_text},
]
}
return {
"prompt": prompt_text,
"completion": completion_text,
}
def validate_jsonl(file_path: str, format_type: str) -> bool:
"""Validates an existing JSONL file."""
if not os.path.exists(file_path):
logging.error("File not found: %s", file_path)
return False
valid_count = 0
invalid_count = 0
with open(file_path, "r", encoding="utf-8") as f:
for i, line in enumerate(f):
try:
example = json.loads(line)
if _validate_example(example, format_type):
valid_count += 1
else:
invalid_count += 1
logging.warning("Invalid format/empty content at line %d", i + 1)
except json.JSONDecodeError:
invalid_count += 1
logging.warning("Invalid JSON at line %d", i + 1)
logging.info("Validation complete for %s", file_path)
logging.info("Valid: %d, Invalid: %d", valid_count, invalid_count)
return invalid_count == 0
def validation_ratio_error(train_bytes: int, val_bytes: int) -> str | None:
"""Returns a message if the written split would be rejected, else None.
The service weighs the two files by size, so a row-count split fraction
cannot predict the outcome; only the written bytes can.
Args:
train_bytes: Size of the written training file.
val_bytes: Size of the written validation file.
Returns:
An actionable error message, or None if the split is within the ceiling.
"""
if val_bytes <= MAX_VALIDATION_RATIO * train_bytes:
return None
ratio = val_bytes / train_bytes if train_bytes else float("inf")
return (
f"Validation file is {ratio:.1%} of the training file ({val_bytes} vs"
f" {train_bytes} bytes), above the {MAX_VALIDATION_RATIO:.0%} the tuning"
" service allows. Re-run with a smaller --validation_split"
f" (default {DEFAULT_VALIDATION_SPLIT})."
)
def convert_to_jsonl(
input_file: str,
output_file: str,
format_type: str,
prompt_col: str,
completion_col: str,
validation_split: float | None = DEFAULT_VALIDATION_SPLIT,
):
"""Converts a file to JSONL format for Agent Platform tuning.
Supports CSV, JSON, or Parquet files.
Args:
input_file: Path to the input CSV, JSON, or Parquet file.
output_file: Path where the formatted JSONL file will be saved.
format_type: Target format ("messages" or "prompt").
prompt_col: Name of the column containing the prompt/user message.
completion_col: Name of the column containing the completion/response.
validation_split: Optional fraction of data to split for validation.
"""
if not os.path.exists(input_file):
logging.error("Input file not found: %s", input_file)
sys.exit(1)
try:
if input_file.endswith(".csv"):
dataset = datasets.load_dataset(
"csv", data_files=input_file, split="train"
)
elif input_file.endswith(".json"):
dataset = datasets.load_dataset(
"json", data_files=input_file, split="train"
)
elif input_file.endswith(".parquet"):
dataset = datasets.load_dataset(
"parquet", data_files=input_file, split="train"
)
else:
logging.error("Unsupported file format. Use .csv, .json, or .parquet")
sys.exit(1)
except Exception as e: # pylint: disable=broad-exception-caught
logging.exception("Failed to read input file: %s", e)
sys.exit(1)
for col in [prompt_col, completion_col]:
if col not in dataset.column_names: # pyrefly: ignore[not-iterable]
logging.error(
"Column '%s' not found. Available columns: %s",
col,
dataset.column_names,
)
sys.exit(1)
# Remove rows with empty values in critical columns
def is_valid(example):
prompt_text = str(example[prompt_col]).strip().lower()
completion_text = str(example[completion_col]).strip().lower()
return (
len(prompt_text) > 0
and len(completion_text) > 0
and prompt_text != "nan"
and completion_text != "nan"
and prompt_text != "none"
and completion_text != "none"
)
initial_len = len(dataset) # pyrefly: ignore[bad-argument-type]
dataset = dataset.filter(is_valid)
if len(dataset) < initial_len: # pyrefly: ignore[bad-argument-type]
logging.warning(
"Dropped %d rows with empty or NaN values", initial_len - len(dataset) # pyrefly: ignore[bad-argument-type]
)
def format_example(example):
prompt_text = str(example[prompt_col])
completion_text = str(example[completion_col])
if format_type == "messages":
return {
"messages": [
{"role": "user", "content": prompt_text},
{"role": "assistant", "content": completion_text},
]
}
else:
return {
"prompt": prompt_text,
"completion": completion_text,
}
formatted_dataset = dataset.map(
format_example, remove_columns=dataset.column_names # pyrefly: ignore[bad-argument-type]
)
if validation_split and validation_split > 0:
if not (0 < validation_split < 1):
logging.error("validation_split must be between 0 and 1")
sys.exit(1)
# Use the datasets library to perform the split as requested
split_dict = formatted_dataset.train_test_split( # pyrefly: ignore[missing-attribute]
seed=42, test_size=validation_split
)
train_ds = split_dict["train"]
val_ds = split_dict["test"]
val_output_file = output_file.replace(".jsonl", "_validation.jsonl")
if val_output_file == output_file:
val_output_file = output_file + ".validation.jsonl"
train_ds.to_json(output_file, force_ascii=False, lines=True)
val_ds.to_json(val_output_file, force_ascii=False, lines=True)
logging.info(
"Successfully saved %d training examples to %s",
len(train_ds),
output_file,
)
logging.info(
"Successfully saved %d validation examples to %s",
len(val_ds),
val_output_file,
)
# Catch an oversized validation file here rather than letting the tuning
# service reject the job after the dataset has been uploaded to GCS.
ratio_error = validation_ratio_error(
os.path.getsize(output_file), os.path.getsize(val_output_file)
)
if ratio_error:
logging.error("%s", ratio_error)
sys.exit(1)
else:
formatted_dataset.to_json(output_file, force_ascii=False, lines=True) # pyrefly: ignore[missing-attribute]
logging.info(
"Successfully saved %d examples to %s",
len(formatted_dataset), # pyrefly: ignore[bad-argument-type]
output_file,
)
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
parser = argparse.ArgumentParser(
description="Prepare or Validate dataset for Agent Platform Model Tuning"
)
parser.add_argument(
"--input",
help="Input CSV, JSON, Parquet, or JSONL file",
)
parser.add_argument(
"--output",
default="tuning_dataset.jsonl",
help="Output JSONL file (only for conversion)",
)
parser.add_argument(
"--format",
choices=["messages", "prompt"],
default="messages",
help="Target format (messages or prompt/completion)",
)
parser.add_argument(
"--prompt_col",
help="Column name for prompt/user message (CSV/JSON/Parquet only)",
)
parser.add_argument(
"--completion_col",
help=(
"Column name for completion/assistant response (CSV/JSON/Parquet"
" only)"
),
)
parser.add_argument(
"--validation_split",
type=float,
default=DEFAULT_VALIDATION_SPLIT,
help=(
"Fraction of data to hold out for validation. Keep it at or below"
f" {DEFAULT_VALIDATION_SPLIT}; see references/data_prep.md."
),
)
parser.add_argument(
"--validate_only",
action="store_true",
help="Only validate the input JSONL file without converting",
)
args = parser.parse_args()
if args.validate_only:
if not args.input:
logging.error("--input is required for validation")
sys.exit(1)
success = validate_jsonl(args.input, args.format)
sys.exit(0 if success else 1)
else:
if not all([args.input, args.prompt_col, args.completion_col]):
logging.error(
"--input, --prompt_col, and --completion_col are required for"
" conversion"
)
sys.exit(1)
convert_to_jsonl(
args.input,
args.output,
args.format,
args.prompt_col,
args.completion_col,
args.validation_split,
)
scripts/tune_open_model.py›
"""Module for launching Agent Platform model tuning jobs."""
import argparse
import logging
from google import genai
from google.genai import types
def tune_open_model(
project: str,
location: str,
base_model: str,
train_dataset: str,
validation_dataset: str | None,
output_uri: str,
epochs: int,
learning_rate: float,
tuning_mode: str,
adapter_size: int | None = None,
) -> types.TuningJob:
"""Launches a Agent Platform model tuning job."""
training_ds = types.TuningDataset(gcs_uri=train_dataset)
validation_ds = (
types.TuningValidationDataset(gcs_uri=validation_dataset)
if validation_dataset
else None
)
if tuning_mode == "FULL":
mapped_tuning_mode = types.TuningMode.TUNING_MODE_FULL
elif tuning_mode == "PEFT_ADAPTER":
mapped_tuning_mode = types.TuningMode.TUNING_MODE_PEFT_ADAPTER
else:
raise ValueError(
f"Unsupported tuning mode: {tuning_mode}. Supported modes are: FULL,"
" PEFT_ADAPTER."
)
adapter_map = {
1: "ADAPTER_SIZE_ONE",
2: "ADAPTER_SIZE_TWO",
4: "ADAPTER_SIZE_FOUR",
8: "ADAPTER_SIZE_EIGHT",
16: "ADAPTER_SIZE_SIXTEEN",
32: "ADAPTER_SIZE_THIRTY_TWO",
}
mapped_adapter = adapter_map.get(adapter_size) if adapter_size else None
config = types.CreateTuningJobConfig(
epoch_count=epochs,
learning_rate=learning_rate,
validation_dataset=validation_ds,
tuning_mode=mapped_tuning_mode,
adapter_size=mapped_adapter,
output_uri=output_uri,
labels={"mg-source": "agent-platform-tuning-skill"},
)
with genai.Client(
enterprise=True, project=project, location=location
) as client:
tuning_job = client.tunings.tune(
base_model=base_model,
training_dataset=training_ds,
config=config,
)
logging.info("Tuning job launched: %s", tuning_job.name)
job_id = tuning_job.name.split("/")[-1] if tuning_job.name else "unknown"
logging.info(
"View job in console:"
"https://console.cloud.google.com/agent-platform/tuning/locations/%s/tuningJob/%s/monitor?project=%s",
location,
job_id,
project,
)
return tuning_job
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Launch Vertex AI Model Tuning Job"
)
parser.add_argument("--project", required=True)
# `global` lets the service pick a region with available GPU capacity.
parser.add_argument("--location", default="global")
parser.add_argument("--base_model", required=True)
parser.add_argument("--train_dataset", required=True)
parser.add_argument(
"--validation_dataset", help="Optional validation dataset URI"
)
parser.add_argument("--output_uri", required=True)
parser.add_argument("--epochs", type=int, required=True)
parser.add_argument("--learning_rate", type=float, required=True)
parser.add_argument(
"--tuning_mode", choices=("FULL", "PEFT_ADAPTER"), required=True
)
parser.add_argument(
"--adapter_size",
type=int,
choices=(1, 4, 8, 16, 32),
help="Adapter size for PEFT",
)
args = parser.parse_args()
tune_open_model(**vars(args))
SKILL.md›
---
name: agent-platform-tuning
metadata:
category: AiAndMachineLearning
description: >-
Agent Platform Model Tuning. Use when you need to fine-tune open models
or Gemini models using Agent Platform infrastructure. Don't use for model
training outside Agent Platform, model deployment to endpoints (use
`agent-platform-deploy`), or managing serving endpoints (use
`agent-platform-endpoint-management`).
---
# Agent Platform Model Tuning
## Overview
This skill provides procedural knowledge for fine-tuning Large Language Models
(both Open Models and Gemini Models) using Agent Platform's tuning service. It
covers the entire lifecycle from environment setup and data preparation to job
configuration, monitoring, and deployment.
## Workflow Decision Tree
1. **Model Category Identification**: Has the user explicitly stated whether
they want to tune an **Open Model** or a **Gemini Model**?
- **No** → **STOP**. Ask the user if they want to tune an Open Model or a
Gemini Model. **CRITICAL EXCEPTION for Environment Setup Requests:** If
the user is specifically asking for environment setup instructions (e.g.
"What environment setup is needed?"), you **MUST** provide the full
[Phase 0 environment setup](#phase-0) instructions in your initial
response, *simultaneously* with asking clarifying questions about the
model category.
- If the user provides a specific tuning purpose, you should recommend
three models: one Open Model, one Gemini Model, and a third generally
recommended choice. Briefly list the pros and cons of each (e.g., Gemini
models might be more expensive, etc.). **CRITICAL:** You must read
`references/models.md` during this step and only recommend models
explicitly listed in that catalog. Do not recommend unsupported models
like Mistral. If the user names a model that is not in the catalog,
follow the fallback rule in that catalog. Do not proceed with model
configuration until the category is confirmed.
- **Yes** → Proceed.
2. **Environment Check**: Has the environment (Auth, APIs, IAM, Venv) been
initialized?
- **No** → Go to [Phase 0: Environment & IAM Setup](#phase-0).
- **Yes** → Proceed.
3. **Dataset Status**: Is the dataset ready in JSONL format, **is its structure
valid for tuning**, and is it uploaded to Google Cloud Storage?
```
- **No** → Go to [Phase 1: Dataset Preparation & Upload](#phase-1).
- **Yes** → Proceed.
```
4. **Column Selection Confirmation**: Have you presented the columns to the
user and confirmed the mapping?
- **No** → **STOP**. You must show samples and get user confirmation on
column mapping as described in Phase 1.0 before proceeding.
- **Yes** → Proceed.
5. **Configuration**: Has the user provided the target model and
hyperparameters, or explicitly agreed to your recommendations?
- **No** → Go to
[Phase 2: Model Configuration & Recommendation](#phase-2).
- **Yes** → Proceed.
6. **Job Status**: Has the tuning job been submitted?
```
- **No** → Go to
[Phase 3: Tuning Job Execution](#phase-3-tuning-job-execution).
- **Yes** → Proceed.
```
7. **Job Completion**: Is the tuning job complete?
```
- **No** → Go to [Phase 4: Monitoring](#phase-4-monitoring).
- **Yes** → Proceed.
```
8. **Deployment**: Has the tuned model been deployed (if required)?
```
- **No** → Go to [Phase 5: Model Deployment](#phase-5-model-deployment).
- **Yes** → Task Complete.
```
## Phase 0: Environment & IAM Setup {#phase-0}
Ensure the foundational environment is ready before proceeding.
### 0.1 Authentication & Project Context
- Check if `gcloud` CLI is installed. If it is not installed, prompt the user
for permission to install it before proceeding. If it is installed, update
it:
```bash
gcloud components update --quiet > /dev/null 2>&1
```
- Verify `gcloud auth list`. If not authenticated, run `gcloud auth login`.
- Ensure `project` is known. Use `gcloud config get project` to retrieve the
current project.
- **CRITICAL: Ask for Confirmation.** You must prompt the user to confirm the
retrieved project before proceeding, in case they want to switch to a
different one. The location must also be confirmed — see section 0.2 for
which location to propose, which depends on the model category.
### 0.2 Location
Location handling **depends on the model category** you established in the
workflow decision tree. The two categories have different supported locations —
never apply one category's locations to the other.
- **Open models** share one fixed location set, and `global` is the
recommended choice.
- **Gemini models** differ per model and must be looked up. `global` is not
accepted for them today.
If the user names a location that is not valid for their model and category,
STOP. Respond with an error naming the requested location as unsupported, list
the locations that are valid, and do NOT ask for a dataset, do NOT proceed with
any other setup step, and do NOT silently retry elsewhere.
#### Open Models (RECOMMEND: `global`)
**Recommend `global` and confirm it with the user.** Propose it as a single
recommended choice rather than making the user pick a region first, and do not
steer them toward a specific region instead.
These are the only locations available for open model tuning:
- `global` (the recommended choice)
- `us-central1`
- `europe-west4`
- `us-west1`
- `us-east5`
- `asia-southeast1`
The `global` endpoint automatically selects a supported region that has
available capacity, so it is the most likely to be scheduled successfully.
Pinning a region up front restricts the job to that one region's capacity, which
is why `global` is the recommended location for open model tuning.
- **The user named a location** → use it verbatim, provided it is `global` or
one of the regions listed above. Do not talk them out of it.
- **The user asked which locations are supported** → answer the question.
Share the list above and say that `global` is recommended and why. Never
withhold it.
- **The user did not name a location** → propose `global` and ask them to
confirm it before you proceed. Say that `global` lets the service pick a
region with available capacity. Do NOT silently assume `global`.
The point of proposing a single choice is to avoid making region selection a
decision the user must resolve before anything else can happen — that ordering
is what previously blocked people. It is not a reason to hide the list: quote it
whenever the user asks, and quote it when rejecting an unsupported location.
Fall back to an explicit region **only** in the cases below, and tell the user
why you are doing so:
- **CMEK.** Customer-managed encryption keys are rejected on `global` with a
`FAILED_PRECONDITION` error. A CMEK-protected job must name the region that
holds the key.
- **Data residency.** If the user requires the job to stay in a specific
jurisdiction, honor their region. `global` currently runs the job in either
`us-central1` or `europe-west4`.
If a `global` job is accepted but then fails with a `FAILED_PRECONDITION` error
saying the model does not support global endpoint tuning, that model is not
onboarded to the global endpoint yet. The model itself is still tunable:
resubmit once in an explicit region from the list above (`us-central1` is the
safest choice) and tell the user why you switched.
##### Working with a `global` job
- The API host stays `aiplatform.googleapis.com`. There is no
`global-aiplatform.googleapis.com` host.
- The service resolves `global` to a real region at run time. Sub-resources
(the tuned model, checkpoints, TensorBoard) come back with that **real**
region in their resource names, not `global`. Read the location out of the
returned resource name before using it for monitoring or deployment; never
assume it is still `global`.
- Quota is shared across regions, so pinning a region does not grant extra
quota.
#### Gemini Models (per-model, look it up)
`global` is **not accepted for Gemini tuning today** — the service rejects it at
job creation with a `FAILED_PRECONDITION` error, so do not propose it here.
**There is no single region allowlist for Gemini.** Supported tuning regions
vary by model and by model version: some Gemini models are restricted to two
regions while others support many more. Do NOT reuse the open model list above,
and do NOT assume a region carries over from another Gemini model.
Before submitting, look up the chosen model in the supervised fine-tuning
documentation and read its **"Supported endpoint for model tuning"**
row:
[supervised tuning](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning/supervised-tuning)
- **The user asked which regions are supported** → look up that specific model
and tell them what the docs say. Do not answer from memory or from the open
model list, and do not answer for a different Gemini model.
- **The model's row names specific regions** → the user's region must be one
of them. If it is not, STOP and report the supported regions for that model.
- **The model's row is absent or the docs are unclear** → ask the user for the
region rather than guessing one.
Confirm the region with the user before proceeding. Note that some Gemini models
also restrict CMEK and serve tuned models only on the `us` and `eu` multi-region
endpoints, so check the same table for those limits before promising them.
### 0.3 Enable APIs
Ensure `aiplatform.googleapis.com` and `storage.googleapis.com` are enabled.
```bash
gcloud services enable aiplatform.googleapis.com storage.googleapis.com \
--project=YOUR_PROJECT
```
### 0.4 IAM Permissions
Verify the following identities have the required roles.
- **Agent Platform Service Agent**:
`[email protected]`
- **Managed OSS Fine Tuning Service Agent**:
`service-PROJECT_NUMBER@gcp-sa-vertex-moss-ft.iam.gserviceaccount.com`
- **User Identity**: The account running the commands.
### 0.5 Python Dependencies
The scripts in this skill import `vertexai` (from `google-cloud-aiplatform`),
`google-genai`, `google-cloud-storage`, and `datasets`.
**CRITICAL AGENT INSTRUCTION:** Do **not** create a virtual environment, and do
not install anything before checking. A venv starts empty and hides packages the
environment already provides, forcing a redundant several-minute install.
Probe first and install only if the probe fails:
```bash
python3 -c "import vertexai, google.genai, google.cloud.storage, datasets" \
|| pip install -r references/requirements.txt
```
Then run every script with a plain `python3 scripts/...` — no activation prefix.
The `references/requirements.txt` pins are a fallback for an environment that
does not already provide these SDKs. Do not apply them on top of a working
environment: they would downgrade packages other tools may share.
## Phase 1: Dataset Preparation & Upload {#phase-1}
### 1.0 Dataset Discovery & Confirmation
- **User-Provided Dataset Verification:** If the user specifies a dataset
filename or path in their prompt, verify its existence in the workspace
(e.g. via script execution or checking for typos).
* **If the file cannot be found anywhere**, you **MUST** inform the user
that the dataset file does not exist or cannot be accessed. You **MUST**
prompt the user to provide a valid dataset path. Alternatively, if
candidate dataset files are found in the workspace during your search,
you **MUST** present the candidates to the user and ask them to select
one. You **MUST** stop tool execution immediately after reporting the
missing file or presenting candidates, and wait for the user's response.
Do **NOT** ask for 90/10 validation split permission, and do **NOT**
attempt to upload the dataset before receiving a valid dataset file
selection from the user.
* **If the file is found and verified**, proceed to Step 1.1 Formatting &
Validation below.
- **Auto-Discovery: From User Bucket:** If the user does not have a dataset
and no suitable alternative is found in the Hugging Face reference, offer to
search the user's GCS buckets for potential training data. Prioritize
searching for files with extensions like `.jsonl`, `.json`, `.csv`, and
`.parquet`. If such files are found, read the first few lines/records of
each to determine if they contain text-based data suitable for tuning (e.g.,
prompt/completion pairs) that can be modified to follow
[Data Preparation Guide](references/data_prep.md) and is related to the
tuning task requested. **DO NOT** search without prompting first.
- **Auto-Discovery: From Task to Huggingface:** If the user has a specific
task, refer to [Huggingface Datasets Reference](references/hf_datasets.md)
and recommend a dataset from this if one exists. For each dataset
recommended, provide some information about the dataset and provide some
reasonable splits. > [!IMPORTANT] > **CRITICAL: Ask for Confirmation and
Column Selection.** Do not proceed > with dataset preparation or upload
until you perform the following > steps and get user confirmation: > 1.
**Dataset and Split Confirmation:** Present the dataset and > available
splits to the user and have them confirm which to use. > 2. **Column
Selection (Hugging Face or Custom Datasets):** You must: > - Provide a list
of all available columns in the selected dataset > split. > - **Show a few
samples from the dataset** to help the user > understand the content and
make the choice of columns. > - Recommend which columns should be mapped to
`prompt` (or user > message) and `completion` (or assistant response),
offering a few > reasonable options if applicable. > - Ask the user to
confirm the column mapping or specify which > columns to use.
### 1.1 Formatting & Validation
- **Conversion**: If data is in CSV, JSON, or Parquet, use
`scripts/prepare_dataset.py` to convert.
- **Validation Split Confirmation**: If the user only provides a training
dataset, **you must prompt the user** to seek permission to split the
training dataset 90/10 to form a validation dataset (using
`--validation_split 0.1`). If they agree, proceed with the split. If they
decline, just use the training dataset without a validation dataset. Do
**NOT** offer an 80/20 split; the tuning service rejects it, for the reason
given in
[Data Preparation Guide](references/data_prep.md#sizing-the-validation-split).
- **Validation**: If data is already in JSONL, validate it before uploading.
Simply having a `.jsonl` extension is not enough. You must verify that the
content schema is valid for tuning (e.g. correct system/user/model roles).
```bash
python3 scripts/prepare_dataset.py \
--input my_data.jsonl \
--format <messages|messages_gemini> \
--validate_only
```
*(Use `--format messages` for open models and `--format messages_gemini` for
Gemini models.)* - Refer to [Data Preparation Guide](references/data_prep.md)
for required schemas.
### 1.2 Upload
Upload formatted `.jsonl` files to GCS using a unique directory (e.g., with a
datetime timestamp) to avoid overwriting outputs from different runs.
```bash
ARTIFACTS="gs://YOUR_BUCKET/tuning_agent_job_<datetime>/dataset.jsonl"
gcloud storage cp dataset.jsonl "$ARTIFACTS"
```
## Phase 2: Model Configuration & Recommendation {#phase-2}
Help the user choose the best model and parameters. **Always seek user
confirmation before submitting the job.**
- If the user does not specify a specific model in their prompt, calculate
recommendations based on the **Models Catalog**.
- **Prompt for Confirmation:** Present the recommended model to the user and
ask for their confirmation before configuring hyperparameters.
### 2.1 Configuration
#### For Open Models
- Recommend `tuning_mode`, `epochs`, `learning_rate`, and `adapter_size` based
on the [Tuning Guide](references/tuning_guide.md) and model-specific
baselines in the [Models Catalog](references/models.md).
#### Verify the Live Model ID
Before submitting the job, run `scripts/list_models.py` and pick `--base_model`
only from its `models` output. Do not invent IDs or version numbers.
```bash
python3 scripts/list_models.py --project YOUR_PROJECT --filter gemini
```
Output: `{"models": [...], "total_count": N, "truncated": bool}`.
- For Gemini, strip `google/` and `@default` (e.g.
`google/gemini-2.5-flash@default` → `gemini-2.5-flash`); for open models,
pass `publisher/family@version` as-is.
- Skip Gemini variants ending in `-embedding`, `-tts`, `-image`,
`-computer-use`, or `-native-audio`; they are not tunable.
- If `truncated` is `true`, re-run with a tighter `--filter` (e.g.
`gemini-2.5`) before deciding the target version is unavailable.
- If `models` is empty, stop and ask the user.
### 2.2 Calculating Cost (Open Models Only)
- We can calculate a rough estimate of cost of tuning based on the dataset and
the selected model in the [Models Catalog](references/models.md):
```bash
python3 scripts/calculate_cost.py \
--input my_data.jsonl \
--model MODEL_NAME \
--tuning_mode TUNING_MODE \
--epochs epochs
```
`--model` takes either the display name (`Qwen 3 8B`) or the same resource
name you pass to `--base_model` (`qwen/qwen3@qwen3-8b`), so the value chosen
in Step 2.1 can be reused as-is.
> [!NOTE] **Handling Missing Dataset Errors:** If `scripts/calculate_cost.py`
> fails because the dataset file (e.g. `my_data.jsonl` or `dummy_data.jsonl`)
> cannot be found, you **MUST** inform the user that the dataset file does not
> exist or cannot be accessed. You **MUST** prompt the user to provide a valid
> dataset path, and stop tool execution immediately to wait for their response.
> Do **NOT** retry or loop, do **NOT** invent a specific cost number, and do
> **NOT** prompt for job submission approval before receiving a valid dataset
> from the user.
- **Prompt for Confirmation:** Present the recommended hyperparameter
configuration and estimated cost to the user and ask for their approval
before proceeding to job submission. Make sure to note that the estimated
cost is just an estimate and can vary from actual billing costs.
## Phase 3: Tuning Job Execution {#phase-3-tuning-job-execution}
**CRITICAL Pre-Flight Check (GCS Verification):** Before you propose a
confirmation prompt or submit any tuning job, you **MUST** verify that the
specified training dataset GCS URI (e.g. `gs://dummy_bucket/dataset.jsonl` or
`gs://YOUR_BUCKET/...`) actually exists and is accessible. Run `gcloud storage
ls $DATASET_URI` (or `gsutil ls`).
* **If the verification fails** (e.g. `BucketNotFound`, `404`, `AccessDenied`,
or indicating a dummy/missing bucket), you **MUST** inform the user that the
GCS bucket or dataset does not exist or cannot be accessed. You **MUST**
prompt the user to provide a valid GCS URI for the dataset, and stop tool
execution immediately to wait for their response. Do **NOT** propose a
confirmation prompt and do **NOT** execute any tuning scripts before
receiving a valid dataset URI from the user.
* **If the verification succeeds**, proceed to propose the confirmation prompt
below.
### For Gemini Models
Check if `scripts/tune_gemini_model.py` exists.
- **If `scripts/tune_gemini_model.py` exists:** Submit the Gemini model tuning
job using this script.
```bash
python3 scripts/tune_gemini_model.py
```
- **If `scripts/tune_gemini_model.py` does not exist:** Instruct the user to
manually configure and submit the tuning job via the Google Cloud Console UI
or using the Agent Platform SDK for Python.
### For Open Models
Submit the open model tuning job using `scripts/tune_open_model.py`. Identify
the model id using available models documentation
at
[documentation](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/open-model-tuning#supported-models).
`--base_model` takes a publisher model **resource name**
(`{publisher}/{model_id}@{version_id}`), not the display name shown in the
catalog. See "Model Resource Name Format" in `references/models.md` for the
format, verified examples, and how to look up a name you do not have.
```bash
python3 scripts/tune_open_model.py \
--project YOUR_PROJECT \
--location global \
--base_model BASE_MODEL_ID \
--train_dataset gs://YOUR_BUCKET/tuning_agent_job_<datetime>/dataset.jsonl \
--output_uri gs://YOUR_BUCKET/tuning_agent_job_<datetime>/output \
--epochs EPOCHS \
--learning_rate LR \
--tuning_mode MODE
```
This script is open model only, and `--location` falls back to `global` if
omitted. Always pass the location the user confirmed in section 0.2 explicitly,
so it is visible in the command string you present for approval.
> [!WARNING] **`--output_uri` is required for open models.** The Python SDK
> declares it as `output_uri: Optional[str] = None`, but the tuning backend
> rejects open model jobs that omit it with `INVALID_ARGUMENT: The output_uri
> field is required for this model.` Treat the SDK's "optional" signature as
> wrong here and always pass a GCS destination.
Because the flag is mandatory, you must establish where the tuned model is
written before you can submit. **Never invent a bucket name, derive one from the
project number, or run `gcloud storage buckets create` unprompted.** Creating a
bucket is a mutating action and is subject to the Tier M confirmation policy
below.
- **The user named a bucket or URI** → use it, appending a unique per-job
directory as in section 1.2.
- **A bucket was already used for the dataset upload in section 1.2** →
propose reusing it for the output and ask the user to confirm.
- **Neither** → **STOP and ask the user** where they want the tuned model
stored. Offer to create a bucket for them as one of the options. If they
accept, propose the exact bucket name and location, get explicit
confirmation, and only then create it.
> [!IMPORTANT] **Interactive Confirmation Required (Tier M):** Before proceeding
> with job submission, you **MUST** present the proposed command string showing
> all literal flags in a confirmation prompt to the user with 'Yes' and 'No'
> options.
> **CRITICAL:** When presenting this confirmation prompt to the user, you MUST
> output it as a direct plain text response and stop tool execution immediately.
> Do NOT call any command execution or interactive tools in the same turn, as
> unexpected tool calls may be auto-replied by the simulation harness and cause
> an infinite loop. Yield immediately for the user's reply.
## Phase 4: Monitoring {#phase-4-monitoring}
Monitor the job via the Cloud Console link provided in the script output.
`--location` is required and must be the same location you submitted with: an
open model job submitted on `global` is polled with `--location global`, even
though the work runs in a real region behind the scenes.
Additionally, ask the user if they want you to monitor the job status for them
in the background. If they agree, execute `scripts/monitor_tuning_job.py` as a
background task to periodically poll the job status and notify the user to show
the status. If the user declines, leave it completely to the user to check on
the status.
## Phase 5: Model Deployment {#phase-5-model-deployment}
Once the tuning job is `SUCCEEDED`, deploy the model.
Deployment requires a real region — `--region=global` is not valid here. If the
job ran on `global`, read the region out of the tuned model's resource name
(`projects/.../locations/<REGION>/models/...`) and deploy there; do not guess.
```bash
ARTIFACTS="gs://YOUR_BUCKET/tuning_agent_job_<datetime>/output/postprocess/node-0/checkpoints/final"
gcloud ai model-garden models deploy \
--project=YOUR_PROJECT \
--region=YOUR_LOCATION \
--model="$ARTIFACTS" \
--machine-type=MACHINE_TYPE \
--accelerator-type=ACCELERATOR_TYPE \
--accelerator-count=COUNT
```
> [!IMPORTANT] **Interactive Confirmation Required (Tier M):** Before proceeding
> with deployment, you **MUST** present the proposed command string showing all
> literal flags in a confirmation prompt to the user with 'Yes' and 'No'
> options.
> **CRITICAL:** When presenting this confirmation prompt to the user, you MUST
> output it as a direct plain text response and stop tool execution immediately.
> Do NOT call any command execution or interactive tools in the same turn, as
> unexpected tool calls may be auto-replied by the simulation harness and cause
> an infinite loop. Yield immediately for the user's reply.
Refer to [Models Catalog](references/models.md) for hardware recommendations for
specific open models.
## Resources
- [Data Preparation Guide](references/data_prep.md)
- [Models Catalog](references/models.md)
- [Tuning Guide](references/tuning_guide.md)
- `scripts/prepare_dataset.py`: Data conversion & validation.
- `scripts/tune_open_model.py`: Open model tuning job submission.