Instructions to use ibm-granite/granite-4.0-micro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ibm-granite/granite-4.0-micro with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ibm-granite/granite-4.0-micro")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ibm-granite/granite-4.0-micro")
model = AutoModelForCausalLM.from_pretrained("ibm-granite/granite-4.0-micro")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ibm-granite/granite-4.0-micro with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ibm-granite/granite-4.0-micro"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibm-granite/granite-4.0-micro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ibm-granite/granite-4.0-micro

SGLang

How to use ibm-granite/granite-4.0-micro with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ibm-granite/granite-4.0-micro" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibm-granite/granite-4.0-micro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ibm-granite/granite-4.0-micro" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibm-granite/granite-4.0-micro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ibm-granite/granite-4.0-micro with Docker Model Runner:
```
docker model run hf.co/ibm-granite/granite-4.0-micro
```

granite-4.0-micro / model.sig

ranarag

adding default system prompt

111f804 verified 8 months ago

raw

history blame

8.74 kB

{"mediaType": "application/vnd.dev.sigstore.bundle.v0.3+json", "verificationMaterial": {"certificate": {"rawBytes": "MIIC5jCCAmygAwIBAgIUYUV8nr2YFPYpKeX/slsWFJxPJjUwCgYIKoZIzj0EAwMwNzEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MR4wHAYDVQQDExVzaWdzdG9yZS1pbnRlcm1lZGlhdGUwHhcNMjUxMDA2MjE0NDA1WhcNMjUxMDA2MjE1NDA1WjAAMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAErXeEqPmhV/bSG2NlHv7xjYCm/0OmXTrlRvOcB0cPSsg2rSPWDkss/8U5DxVVnL51ATG55QqHebeUr5OBUZ4uL6OCAYswggGHMA4GA1UdDwEB/wQEAwIHgDATBgNVHSUEDDAKBggrBgEFBQcDAzAdBgNVHQ4EFgQUwiwUI4hcCt9es1eo0Q3Kk16d7UcwHwYDVR0jBBgwFoAU39Ppz1YkEZb5qNjpKFWixi4YZD8wJAYDVR0RAQH/BBowGIEWR3Jhbml0ZS12ZXJpZnlAaWJtLmNvbTA0BgorBgEEAYO/MAEBBCZodHRwczovL3NpZ3N0b3JlLnZlcmlmeS5pYm0uY29tL29hdXRoMjA2BgorBgEEAYO/MAEIBCgMJmh0dHBzOi8vc2lnc3RvcmUudmVyaWZ5LmlibS5jb20vb2F1dGgyMIGLBgorBgEEAdZ5AgQCBH0EewB5AHcA3T0wasbHETJjGR4cmWc3AqJKXrjePK3/h4pygC8p7o4AAAGZu3tUBwAABAMASDBGAiEAmK3a94POzTOTPBl7oSi9vXXWeZ5viJjt3hRCpzaWfWsCIQDQtXcjv0mzW3YVm/F1adwoaMIgzoXiJZ4CmXeJwT/5HTAKBggqhkjOPQQDAwNoADBlAjEAgXCf95dMeZIaTWn9aN4yfwWusH4wRQP4fZugTvRUwwHzINUqQf3bn0NptAFzhNq2AjAMCeAxLrVxFRfgJiUa+lSb2Vde8gzLUSuCML91sxVjSPDJCMAoSmCGbZoURSZS9vg="}, "tlogEntries": [{"logIndex": "586065829", "logId": {"keyId": "wNI9atQGlz+VWfO6LRygH4QUfY/8W4RFwiT5i5WRgB0="}, "kindVersion": {"kind": "dsse", "version": "0.0.1"}, "integratedTime": "1759787046", "inclusionPromise": {"signedEntryTimestamp": "MEYCIQDzOz9cxHC3Vw6AGKCQyyOSbF3/Vr2Kw0+lQ7YzFiZx5QIhALwa02ODa4BgEJDnpy6+GkmG33SZbtAeK+eaS3ViyYtG"}, "inclusionProof": {"logIndex": "464161567", "rootHash": "4sqcaw4d1fIkjEIyVqPNReFJJwGZjo40DkcgNJZHw/U=", "treeSize": "464161568", "hashes": ["I9BUe6t5+8bcIEEhVNyLu+SvMrRnJcuP/xEDp6i7K2I=", "TpbxOWhVJjNLO/Qmx6W5YNdzE7iOb84qGCqmR9kQAbI=", "2ahweK8UgDxQppU7t5u1tT3f3jjKl0oD17xGN07sMUs=", "HII+KDbN/nvNxCMTr+qwLwT/atHQ9AZ7A2LVI1j13bE=", "trVss6UdVS1xqRUSGrPcdzrVPoa3VHrV1iPTEPcOvn0=", "Aenzo3EuncLmOLSKCNiwt13YnixRsjRz9j5W4VjMDWg=", "yaXm2azwV3nwRFMQohuXYPr70pIzANXkWdqJm8TEhJI=", "j7EZOypG6zah49KEOUG6X+lry26qxWmSMfLWf92s4KI=", "bEd9RBZrMl3sB9+GVKzIknW9kTflFpVSlzQG/XSNoKU=", "bMiYNa7e/10/uvyMIWwNB/Yh9YFw50P0UMIK4zDEuyo=", "eDP2CTarUzd2DhAeWAOCCLq/dFUw6VU8fRiGnB+AkKM=", "POSNGsx0avcIWHYuWJNIMthYZDOfdhmPgzZObRb2d6Y=", "Kcf5veFWCVXREqqw2Q7rOJD3ttDxBxePywSepmjiSGo=", "5J6N5aNEMk6YY8lPNH0zZx95RNICDJ12cMAmpwtvGTo=", "/buh7NyeUdCgJPdaMbTAtiQpMkNHhk+xMsI2shkTieM=", "1mfy94KpcItqshH9+gwqV6jccupcaMpVsF28New8zDY=", "vS7O4ozHIQZJWBiov+mkpI27GE8zAmVCEkRcP3NDyNE="], "checkpoint": {"envelope": "rekor.sigstore.dev - 1193050959916656506\n464161568\n4sqcaw4d1fIkjEIyVqPNReFJJwGZjo40DkcgNJZHw/U=\n\n\u2014 rekor.sigstore.dev wNI9ajBEAiBGPHmx2pRlS1h+TCGVOzeehy6LhscZTuKmzpMgf94bXgIgN2RQ17GNP+rXSyUg9RpEqrG89iH8rx+LIR1MgRWU7HY=\n"}}, "canonicalizedBody": "eyJhcGlWZXJzaW9uIjoiMC4wLjEiLCJraW5kIjoiZHNzZSIsInNwZWMiOnsiZW52ZWxvcGVIYXNoIjp7ImFsZ29yaXRobSI6InNoYTI1NiIsInZhbHVlIjoiNDMyMDVkYWVmY2IyYmU0ZWY5ZWFiOGJmNTA4ZTI1NmIyNTkyODA0NjMyYzZiM2RiYjc5NjM4MTE2ZmU3NjQwMiJ9LCJwYXlsb2FkSGFzaCI6eyJhbGdvcml0aG0iOiJzaGEyNTYiLCJ2YWx1ZSI6Ijg1NjQ4NGQzNDhhYjZkZGNiM2MzN2I0ZmY3ZDlmMzE4NzdjNjUzNmViYjRhYjcyNzdiMDEwYzNkNzI3NjRlYjIifSwic2lnbmF0dXJlcyI6W3sic2lnbmF0dXJlIjoiTUVVQ0lDSU1oT084MzR5ZTRUdmdoRTA3TDVBZW9ielBBZXpvNlk4bElFL1p3N3NxQWlFQXM2aTJNQndMMmtHZVRkS2dsNTVPaUsyWjB0cGtVNjZ0NFh1Z25UcHAza3M9IiwidmVyaWZpZXIiOiJMUzB0TFMxQ1JVZEpUaUJEUlZKVVNVWkpRMEZVUlMwdExTMHRDazFKU1VNMWFrTkRRVzE1WjBGM1NVSkJaMGxWV1ZWV09HNXlNbGxHVUZsd1MyVllMM05zYzFkR1NuaFFTbXBWZDBObldVbExiMXBKZW1vd1JVRjNUWGNLVG5wRlZrMUNUVWRCTVZWRlEyaE5UV015Ykc1ak0xSjJZMjFWZFZwSFZqSk5ValIzU0VGWlJGWlJVVVJGZUZaNllWZGtlbVJIT1hsYVV6RndZbTVTYkFwamJURnNXa2RzYUdSSFZYZElhR05PVFdwVmVFMUVRVEpOYWtVd1RrUkJNVmRvWTA1TmFsVjRUVVJCTWsxcVJURk9SRUV4VjJwQlFVMUdhM2RGZDFsSUNrdHZXa2w2YWpCRFFWRlpTVXR2V2tsNmFqQkVRVkZqUkZGblFVVnlXR1ZGY1ZCdGFGWXZZbE5ITWs1c1NIWTNlR3BaUTIwdk1FOXRXRlJ5YkZKMlQyTUtRakJqVUZOelp6SnlVMUJYUkd0emN5ODRWVFZFZUZaV2JrdzFNVUZVUnpVMVVYRklaV0psVlhJMVQwSlZXalIxVERaUFEwRlpjM2RuWjBkSVRVRTBSd3BCTVZWa1JIZEZRaTkzVVVWQmQwbElaMFJCVkVKblRsWklVMVZGUkVSQlMwSm5aM0pDWjBWR1FsRmpSRUY2UVdSQ1owNVdTRkUwUlVablVWVjNhWGRWQ2trMGFHTkRkRGxsY3pGbGJ6QlJNMHRyTVRaa04xVmpkMGgzV1VSV1VqQnFRa0puZDBadlFWVXpPVkJ3ZWpGWmEwVmFZalZ4VG1wd1MwWlhhWGhwTkZrS1drUTRkMHBCV1VSV1VqQlNRVkZJTDBKQ2IzZEhTVVZYVWpOS2FHSnRiREJhVXpFeVdsaEtjRnB1YkVGaFYwcDBURzFPZG1KVVFUQkNaMjl5UW1kRlJRcEJXVTh2VFVGRlFrSkRXbTlrU0ZKM1kzcHZka3d6VG5CYU0wNHdZak5LYkV4dVdteGpiV3h0WlZNMWNGbHRNSFZaTWpsMFRESTVhR1JZVW05TmFrRXlDa0puYjNKQ1owVkZRVmxQTDAxQlJVbENRMmROU20xb01HUklRbnBQYVRoMll6SnNibU16VW5aamJWVjFaRzFXZVdGWFdqVk1iV3hwWWxNMWFtSXlNSFlLWWpKR01XUkhaM2xOU1VkTVFtZHZja0puUlVWQlpGbzFRV2RSUTBKSU1FVmxkMEkxUVVoalFUTlVNSGRoYzJKSVJWUktha2RTTkdOdFYyTXpRWEZLU3dwWWNtcGxVRXN6TDJnMGNIbG5Remh3TjI4MFFVRkJSMXAxTTNSVlFuZEJRVUpCVFVGVFJFSkhRV2xGUVcxTE0yRTVORkJQZWxSUFZGQkNiRGR2VTJrNUNuWllXRmRsV2pWMmFVcHFkRE5vVWtOd2VtRlhabGR6UTBsUlJGRjBXR05xZGpCdGVsY3pXVlp0TDBZeFlXUjNiMkZOU1dkNmIxaHBTbG8wUTIxWVpVb0tkMVF2TlVoVVFVdENaMmR4YUd0cVQxQlJVVVJCZDA1dlFVUkNiRUZxUlVGbldFTm1PVFZrVFdWYVNXRlVWMjQ1WVU0MGVXWjNWM1Z6U0RSM1VsRlFOQXBtV25WblZIWlNWWGQzU0hwSlRsVnhVV1l6WW00d1RuQjBRVVo2YUU1eE1rRnFRVTFEWlVGNFRISldlRVpTWm1kS2FWVmhLMnhUWWpKV1pHVTRaM3BNQ2xWVGRVTk5URGt4YzNoV2FsTlFSRXBEVFVGdlUyMURSMkphYjFWU1UxcFRPWFpuUFFvdExTMHRMVVZPUkNCRFJWSlVTVVpKUTBGVVJTMHRMUzB0Q2c9PSJ9XX19"}], "timestampVerificationData": {}}, "dsseEnvelope": {"payload": "ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiZ3Jhbml0ZS00LjAtbWljcm8iLAogICAgICAiZGlnZXN0IjogewogICAgICAgICJzaGEyNTYiOiAiNWExNzI3OWZlYWY5NWRhYzYwY2EwODEzYjc4YTM0ZTNkM2E4OWQ5NjM4YTNmYzBmZDQ0YTdlMDMxMTZlOTIyNSIKICAgICAgfQogICAgfQogIF0sCiAgInByZWRpY2F0ZVR5cGUiOiAiaHR0cHM6Ly9tb2RlbF9zaWduaW5nL3NpZ25hdHVyZS92MS4wIiwKICAicHJlZGljYXRlIjogewogICAgInJlc291cmNlcyI6IFsKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiNDJhYTBkZTI0NGEyMGQzNTk5Y2RiYmNhNDE2MDFkYTNlN2ZkZTc1MzFmMTliYjhmMGM3MmY5ZTM2ODM4MDBmZiIsCiAgICAgICAgIm5hbWUiOiAiUkVBRE1FLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiOTUyNGRmNjdiNzdhN2IyNWEyZGZlZTg5OGY3NWIzMTZhMTU3ZWI5ZDg1NWI1MWUzMmFlYWM3OWQ3YzhhODNjZSIsCiAgICAgICAgIm5hbWUiOiAiY2hhdF90ZW1wbGF0ZS5qaW5qYSIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImUxMWEyNDQzNDc2OWYxOTI2YzJiNzNlYTRiNmJmNzExNTUzM2UzNjQ0ZDU1M2Q1YTM0Nzc5NzA4MmU2NWMwZjIiLAogICAgICAgICJuYW1lIjogImNvbmZpZy5qc29uIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiN2MwNGNiOWQyYmE3NzFmNzUyOGZiYTVhNzEwNDk5OWNkYWY3NTY2ZDAyYjVmYmQ1ODQ3MjgyOWY2MjcxNjE3NyIsCiAgICAgICAgIm5hbWUiOiAiZ2VuZXJhdGlvbl9jb25maWcuanNvbiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImI2ZmU0MjRlMzM0OTAzZjdmYjg0ZDNhMTA2ZDk3MzA0NTVmNDc0NGI5ZmUzYzIxZWUxMzZkOTdhMDBlNzI1MDIiLAogICAgICAgICJuYW1lIjogIm1lcmdlcy50eHQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICIxMzFhNzkzYjQ1ZGRlZWU4ZTdjODNhNzc4NDZjMzUwYjUyMzE2MTU4OTE3ZWY1ODljMDA2YzRhMTBmMmRlOTUyIiwKICAgICAgICAibmFtZSI6ICJtb2RlbC0wMDAwMS1vZi0wMDAwMi5zYWZldGVuc29ycyIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogIjZiNjM4M2E1YWNhNzIzOTQwZDllNGJlNjNmYTE1NzVlOWM0NDk0MWU3ZjVkNDg5YjkyM2MwZWFlYWRhMmMwMmYiLAogICAgICAgICJuYW1lIjogIm1vZGVsLTAwMDAyLW9mLTAwMDAyLnNhZmV0ZW5zb3JzIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiMjA4OTU5ZjViZWIyZGQzNjIwZmI1Y2YyN2Q3M2ZiODNkYWIwYmU3NGQ3OTQ4NmVmMGEyOGUwNzQ5YTRiYjBkYSIsCiAgICAgICAgIm5hbWUiOiAibW9kZWwuc2FmZXRlbnNvcnMuaW5kZXguanNvbiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImMwODY3NmM0OWZkNzk2OWEzMTMwZjcyYmU2ZDRiZjM0ZGE2NmFhNDg0YTZlMjFkZmZlMzU5ODkzYTFiZDVmMmUiLAogICAgICAgICJuYW1lIjogInNwZWNpYWxfdG9rZW5zX21hcC5qc29uIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiZTJiYWQ2NjQzOTUzOGNiNGQ1YTc1ODA2ODA5MzI0MzJlZDllY2U5ZDNiODU3N2U2NzU1MTJiZGYxMTU5OTI1MyIsCiAgICAgICAgIm5hbWUiOiAidG9rZW5pemVyLmpzb24iLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICJhNWVjNWRhYWIxMmJhMDkwYTkwZjNkZDE2OWM4ZjljMjc1NTU3MDEzYTg3YjljMTI1OGRjN2NiNDk3YTM1Yzg2IiwKICAgICAgICAibmFtZSI6ICJ0b2tlbml6ZXJfY29uZmlnLmpzb24iLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICI4YWY3MTA3NmRlOGIwYjYyNmVlZDBmNGM5ODRmYWYwYTdjMDYyNDc5MTY0YjJhMzEzMDhhOTQ4NTI0ZDRmNjljIiwKICAgICAgICAibmFtZSI6ICJ2b2NhYi5qc29uIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfQogICAgXSwKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZQogICAgfQogIH0KfQ==", "payloadType": "application/vnd.in-toto+json", "signatures": [{"sig": "MEUCICIMhOO834ye4TvghE07L5AeobzPAezo6Y8lIE/Zw7sqAiEAs6i2MBwL2kGeTdKgl55OiK2Z0tpkU66t4XugnTpp3ks="}]}}