Instructions to use DeepMount00/Llama-3.1-8b-ITA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DeepMount00/Llama-3.1-8b-ITA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DeepMount00/Llama-3.1-8b-ITA")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DeepMount00/Llama-3.1-8b-ITA")
model = AutoModelForCausalLM.from_pretrained("DeepMount00/Llama-3.1-8b-ITA", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DeepMount00/Llama-3.1-8b-ITA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DeepMount00/Llama-3.1-8b-ITA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DeepMount00/Llama-3.1-8b-ITA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DeepMount00/Llama-3.1-8b-ITA

SGLang

How to use DeepMount00/Llama-3.1-8b-ITA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DeepMount00/Llama-3.1-8b-ITA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DeepMount00/Llama-3.1-8b-ITA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DeepMount00/Llama-3.1-8b-ITA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DeepMount00/Llama-3.1-8b-ITA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DeepMount00/Llama-3.1-8b-ITA with Docker Model Runner:
```
docker model run hf.co/DeepMount00/Llama-3.1-8b-ITA
```

Free Beta Model Hosting

by carnalimr - opened Jul 7, 2025

Discussion

carnalimr

Jul 7, 2025

We’re running a little experiment with our new servers and wanted to invite a few Model communities to test it out. If you've got a fine-tuned LLaMA 3B or 8B model, we’re offering free hosting for a few months — unlimited tokens and solid performance (80 TPS for 10-20 concurrent requests).

After the pilot, it’s just $30/month for 3B or $60/month for 8B — but no pressure, we're mainly looking for folks who are excited to try this out and help shape how we roll it out more broadly.

Some communities are already using it to run chatbots or embed their models into their websites — it’s a lightweight way to showcase your work and let others engage with it.

If you’re interested in being part of the pilot, just give us a shout!

AIutoBrain

Jul 8, 2025

Hi,
I’m interested. Please send more details.
Thanks.

carnalimr

Jul 8, 2025

Do you have an email I can contact you on?

AIutoBrain

Jul 8, 2025

Hi,
for a first exchange, you can write me at: alt.ni-an515ue@yopmail.com
Please mention “AIutoBrain” in the subject so I can spot your message quickly.

Looking forward to your message!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment