Instructions to use zamagi/plamo-2-1b-gorilla-chat3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zamagi/plamo-2-1b-gorilla-chat3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zamagi/plamo-2-1b-gorilla-chat3", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("zamagi/plamo-2-1b-gorilla-chat3", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use zamagi/plamo-2-1b-gorilla-chat3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zamagi/plamo-2-1b-gorilla-chat3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zamagi/plamo-2-1b-gorilla-chat3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zamagi/plamo-2-1b-gorilla-chat3

SGLang

How to use zamagi/plamo-2-1b-gorilla-chat3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zamagi/plamo-2-1b-gorilla-chat3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zamagi/plamo-2-1b-gorilla-chat3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zamagi/plamo-2-1b-gorilla-chat3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zamagi/plamo-2-1b-gorilla-chat3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zamagi/plamo-2-1b-gorilla-chat3 with Docker Model Runner:
```
docker model run hf.co/zamagi/plamo-2-1b-gorilla-chat3
```

See axolotl config

axolotl version: 0.7.0



# モデルの設定
base_model: /notebooks/plamo-2-1b-gorilla-chat2              # HuggingFace上のモデル名
model_type: AutoModelForCausalLM         # モデルのロードに使用するクラス
tokenizer_type: AutoTokenizer           # トークナイザのロードに使用するクラス
trust_remote_code: true                 # リモートのカスタムコードを信頼してモデルをロード

hub_model_id: zamagi/fft-1
hub_strategy: "end"
push_dataset_to_hub:
hf_use_auth_token: true

plugins:
  - axolotl.integrations.liger.LigerPlugin
liger_cross_entropy: false
liger_rope: true
liger_rms_norm: true
liger_swiglu: true
liger_fused_linear_cross_entropy: true

# 8bit/4bit設定（8bitモードでメモリ削減）
load_in_8bit: false   #f                      # 8bit量子化されたモデルをロード
load_in_4bit: false                     # 4bit量子化は使用しない
strict: false                           # 重みの厳密な一致を要求しない（追加トークン等がある場合に許容）

chat_template: tokenizer_default

# データセットの設定
datasets:
  - path: Aratako/Magpie-Tanuki-Qwen2.5-72B-Answered
    type: chat_template
    field_messages: messages
    message_field_role: role
    message_field_content: content
    roles_to_train: ["assistant"]       # 学習対象とする役割（アシスタントの発話のみ学習）
    train_on_eos: last
#  - path: Aratako/magpie-qwen2.5-32b-reasoning-100k-formatted
#    type: chat_template
#    field_messages: conversations
#    message_field_role: role
#    message_field_content: content
#  - path: Aratako/magpie-reasoning-llama-nemotron-70b-100k-filtered
#    type: chat_template
#    field_messages: conversations
#    message_field_role: role
#    message_field_content: content
#  - path: Aratako/Open-Platypus-Japanese-masked-formatted
#    type: chat_template
#    field_messages: conversations
#    message_field_role: role
#    message_field_content: content
#  - path: kanhatakeyama/wizardlm8x22b-logical-math-coding-sft_additional-ja
#    type: chat_template
#    field_messages: messages
#    message_field_role: role
#    message_field_content: content
#  - path: kanhatakeyama/ramdom-to-fixed-multiturn-Calm3
#    split: 20240806filtered
#    type: chat_template
#    field_messages: messages
#    message_field_role: role
#    message_field_content: content
#  - path: Aratako/magpie-ultra-v0.1-formatted
#    type: chat_template
#    field_messages: conversations
#    message_field_role: role
#    message_field_content: content
#  - path: Aratako/orca-agentinstruct-1M-v1-selected
#    type: chat_template
#    field_messages: messages
#    message_field_role: role
#    message_field_content: content
#  - path: Aratako/Synthetic-JP-EN-Coding-Dataset-801k-50k
#    type: chat_template
#    field_messages: messages
#    message_field_role: role
#    message_field_content: content

shuffle_merged_datasets: true
dataset_prepared_path: /notebooks/data/fft-data
val_set_size: 0.003
output_dir: /notebooks/data/27b-fft-out-1
dataset_keep_in_memory: false

gpu_memory_limit: 48GiB

sequence_len: 2048
sample_packing: true
eval_sample_packing: false
pad_to_sequence_len: true

adapter:
lora_model_dir:
lora_r:
lora_alpha:
lora_dropout:
lora_target_linear:
lora_fan_in_fan_out:


# トレーニングの設定
gradient_accumulation_steps: 4
micro_batch_size: 8
num_epochs: 2
optimizer: paged_adamw_8bit
lr_scheduler:
cosine_min_lr_ratio: 0.1
learning_rate: 0.00001
max_steps: 5000

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

#wandb: false
#wandb_project: 27b-fft
#wandb_entity: aratako-lm
#wandb_watch:
#wandb_name: attempt-01
#wandb_log_model:

gradient_checkpointing: true
early_stopping_patience:
auto_resume_from_checkpoints: true
local_rank:
logging_steps: 1
xformers_attention: 
flash_attention: 

save_strategy: steps
save_steps: 100
save_total_limit: 2

warmup_steps: 50
eval_steps: 100
eval_batch_size: 1
eval_table_size:
eval_max_new_tokens:

debug:
deepspeed: /notebooks/axolotl/deepspeed_configs/zero3_bf16.json
weight_decay: 0.01
fsdp:
fsdp_config:


# 出力の保存設定
output_dir: /notebooks/output/plamo-2-1b-gorilla-chat3    # チェックポイントや最終モデルの出力先ディレクトリ
hub_model_id: zamagi/plamo-2-1b-gorilla-chat3   # (オプション) Hugging Face Hubにアップロードする場合のリポジトリ名

plamo-2-1b-gorilla-chat3

This model was trained from scratch on the Aratako/Magpie-Tanuki-Qwen2.5-72B-Answered dataset. It achieves the following results on the evaluation set:

Loss: 1.1070

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 8
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
gradient_accumulation_steps: 4
total_train_batch_size: 32
optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 50
training_steps: 750

Training results

Training Loss	Epoch	Step	Validation Loss
1.2715	0.0027	1	1.2753
1.2207	0.2660	100	1.1846
1.1895	0.5319	200	1.1564
1.2285	0.7979	300	1.1334
0.9658	1.0638	400	1.1341
1.0254	1.3298	500	1.1209
0.9521	1.5957	600	1.1129
1.0186	1.8617	700	1.1070

Framework versions

Transformers 4.49.0
Pytorch 2.5.1+cu124
Datasets 3.2.0
Tokenizers 0.21.1

Downloads last month: 1

Safetensors

Model size

1B params

Tensor type

BF16

zamagi
/

plamo-2-1b-gorilla-chat3

plamo-2-1b-gorilla-chat3

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Dataset used to train zamagi/plamo-2-1b-gorilla-chat3