Instructions to use brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
Use Docker
docker model run hf.co/brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
- Ollama
How to use brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF with Ollama:
ollama run hf.co/brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF with Docker Model Runner:
docker model run hf.co/brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
- Lemonade
How to use brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Snorkel-Mistral-PairRM-DPO-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Snorkel-Mistral-PairRM-DPO GGUF
Original model: Snorkel-Mistral-PairRM-DPO Model creator: Snorkel AI
This repo contains GGUF format model files for Snorkel AI’s Snorkel-Mistral-PairRM-DPO.
With this demonstration, we focus on the general approach to alignment. Thus, we use a general-purpose reward model - the performant PairRM model. We use the Mistral-7B-Instruct-v0.2 model as our base LLM.
What is GGUF?
GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. Converted using llama.cpp b1960 (26d6076)
Prompt template: ChatML
<|im_start|>system
{{system_message}}<|im_end|>
<|im_start|>user
{{prompt}}<|im_end|>
<|im_start|>assistant
Download & run with cnvrs on iPhone, iPad, and Mac!
cnvrs is the best app for private, local AI on your device:
- create & save Characters with custom system prompts & temperature settings
- download and experiment with any GGUF model you can find on HuggingFace!
- make it your own with custom Theme colors
- powered by Metal ⚡️ & Llama.cpp, with haptics during response streaming!
- try it out yourself today, on Testflight!
- follow cnvrs on twitter to stay up to date
Original Model Evaluations:
On Alpaca-Eval 2.0:
- The base model: Mistral-7B-Instruct-v0.2 scored 14.72.
After applying the above methodology:
- This model scored 30.22 - ranked 3rd and the highest for an open-source base model at the time of publication.
- Downloads last month
- 189
Model tree for brittlewis12/Snorkel-Mistral-PairRM-DPO-GGUF
Base model
snorkelai/Snorkel-Mistral-PairRM-DPO