Instructions to use AesSedai/MiniMax-M2.7-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AesSedai/MiniMax-M2.7-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AesSedai/MiniMax-M2.7-GGUF",
	filename="IQ3_S/MiniMax-M2.7-IQ3_S-00001-of-00003.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use AesSedai/MiniMax-M2.7-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use AesSedai/MiniMax-M2.7-GGUF with Ollama:
```
ollama run hf.co/AesSedai/MiniMax-M2.7-GGUF:Q4_K_M
```

Unsloth Studio

How to use AesSedai/MiniMax-M2.7-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiniMax-M2.7-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiniMax-M2.7-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AesSedai/MiniMax-M2.7-GGUF to start chatting

How to use AesSedai/MiniMax-M2.7-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AesSedai/MiniMax-M2.7-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AesSedai/MiniMax-M2.7-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use AesSedai/MiniMax-M2.7-GGUF with Docker Model Runner:
```
docker model run hf.co/AesSedai/MiniMax-M2.7-GGUF:Q4_K_M
```

Lemonade

How to use AesSedai/MiniMax-M2.7-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AesSedai/MiniMax-M2.7-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MiniMax-M2.7-GGUF-Q4_K_M

List all available models

lemonade list

MiniMax-M2.7-GGUF / kld_data /llm_quantization_data.csv

AesSedai

Super-squash branch 'main' using huggingface_hub

d04cac2 about 1 month ago

raw

history blame contribute delete

3.86 kB

	model_name,file_size_gb,bpw,Mean KLD_mean,0.1% KLD,0.1% Δp,1.0% KLD,1.0% Δp,10.0% KLD,10.0% Δp,25.0% Δp,5.0% KLD,5.0% Δp,75.0% Δp,90.0% KLD,90.0% Δp,95.0% KLD,95.0% Δp,99.0% KLD,99.0% Δp,99.9% KLD,99.9% Δp,"Cor(ln(PPL(Q)), ln(PPL(base)))",Maximum KLD,Maximum Δp,Mean KLD_std,Mean PPL(Q)-PPL(base)_mean,Mean PPL(Q)-PPL(base)_std,Mean PPL(Q)/PPL(base)_mean,Mean PPL(Q)/PPL(base)_std,Mean PPL(Q)_mean,Mean PPL(Q)_std,Mean PPL(base)_mean,Mean PPL(base)_std,Mean ln(PPL(Q)/PPL(base))_mean,Mean ln(PPL(Q)/PPL(base))_std,Mean Δp_mean,Mean Δp_std,Median KLD,Median Δp,Minimum KLD,Minimum Δp,RMS Δp_mean,RMS Δp_std,Same top p_mean,Same top p_std,file_path,file_size_gib,ggml_cuda_init,kl_divergence,llama_context,llama_kv_cache,llama_memory_breakdown_print,llama_model_loader,llama_params_fit,llama_params_fit_impl,llama_perf_context_print,load,load_tensors,print_info,sched_reserve,system_info
	MiniMax-M2.7-IQ3_S (aes_sedai),83.60153841664001,2.92,0.28274,-0.0,-95.363,7e-06,-64.924,0.000744,-15.535,-3.641,0.000122,-28.109,0.747,0.68169,7.768,1.16197,15.352,2.855629,38.567,6.940854,76.356,93.68,22.948486,99.967,0.001687,0.954583,0.024081,1.12143,0.003047,8.815764,0.067859,7.861181,0.059535,0.114605,0.002717,-2.459,0.041,0.098503,-0.034,-9e-06,-99.993,15.412,0.074,79.848,0.107,kld/MiniMax-M2.7/wiki-test-raw/aes_sedai/MiniMax-M2.7-IQ3_S.md,77.86,4389000.0,552512819216.0,12.21,128.0,-7374800256.0,-2124.0,0.77,95050.38252,33.0,1.3355,318267.89,256.0,596.984,4.848561200112841e+50
	MiniMax-M2.7-IQ4_XS (aes_sedai),108.5552984064,3.8,0.128807,-1e-06,-81.704,3e-06,-40.411,0.000267,-8.592,-1.755,4.4e-05,-15.826,0.671,0.291244,5.764,0.509207,11.409,1.366539,28.43,4.454674,64.01,97.14,23.78923,99.975,0.00107,0.429493,0.015235,1.054635,0.001922,8.290674,0.063543,7.861181,0.059535,0.053194,0.001823,-0.961,0.028,0.040065,-0.005,-7.9e-05,-99.894,10.471,0.064,86.346,0.092,kld/MiniMax-M2.7/wiki-test-raw/aes_sedai/MiniMax-M2.7-IQ4_XS.md,101.1,4389000.0,552512819216.0,12.21,128.0,-8796220256.0,-462.0,0.52,118701.382508,33.0,1.3355,323718.24,256.0,509.634,4.848561200112841e+50
	MiniMax-M2.7-Q4_K_M (aes_sedai),140.30584414208,4.91,0.059323,-2e-06,-54.805,1e-06,-23.313,0.000104,-4.693,-0.77,1.7e-05,-8.969,0.694,0.12492,4.527,0.213786,8.607,0.600085,21.531,2.745379,48.257,98.66,22.586447,99.979,0.000771,0.090034,0.009895,1.011453,0.001259,7.951215,0.060706,7.861181,0.059535,0.011388,0.001244,-0.105,0.019,0.017084,-0.0,-6.1e-05,-99.871,7.012,0.053,90.711,0.077,kld/MiniMax-M2.7/wiki-test-raw/aes_sedai/MiniMax-M2.7-Q4_K_M.md,130.67,8778000.0,552512819216.0,12.21,128.0,-8796220256.0,-61.0,1.21,158332.770225,33.0,1.3355,713589.83,256.0,91.244,4.848561200112841e+50
	MiniMax-M2.7-Q5_K_M (aes_sedai),168.82442698752,5.91,0.038926,-3e-06,-43.174,0.0,-17.173,6.4e-05,-3.593,-0.572,1.1e-05,-6.809,0.586,0.075408,3.639,0.128819,6.837,0.367792,17.089,2.198945,40.894,99.12,24.088629,99.853,0.000692,0.010697,0.007921,1.001361,0.001008,7.871878,0.059897,7.861181,0.059535,0.00136,0.001006,-0.004,0.015,0.010531,0.0,-3.9e-05,-99.603,5.591,0.052,92.59,0.07,kld/MiniMax-M2.7/wiki-test-raw/aes_sedai/MiniMax-M2.7-Q5_K_M.md,157.23,4389000.0,552512819216.0,12.21,128.0,-8796220256.0,-662.0,0.76,176175.385919,33.0,1.3355,336696.24,256.0,596.374,4.848561200112841e+50
	MiniMax-M2.7-Q8_0 (aes_sedai),243.12736120832002,8.51,0.029715,-3e-06,-35.387,0.0,-14.327,4.5e-05,-2.974,-0.461,7e-06,-5.596,0.507,0.053784,3.089,0.091453,5.783,0.276523,14.613,1.938838,34.987,99.31,22.690596,99.72,0.000649,0.018957,0.007052,1.002412,0.000897,7.880138,0.060034,7.861181,0.059535,0.002409,0.000895,0.034,0.013,0.007496,0.0,-6.2e-05,-99.971,4.783,0.051,93.772,0.064,kld/MiniMax-M2.7/wiki-test-raw/aes_sedai/MiniMax-M2.7-Q8_0.md,226.43,4389000.0,552512819216.0,12.21,128.0,-8796220256.0,-80436.0,0.94,247041.385919,33.0,1.3355,352698.24,256.0,812.494,4.848561200112841e+50