Instructions to use HuggingFaceM4/idefics-80b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics-80b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-80b")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-80b")
model = AutoModelForMultimodalLM.from_pretrained("HuggingFaceM4/idefics-80b", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HuggingFaceM4/idefics-80b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics-80b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics-80b

SGLang

How to use HuggingFaceM4/idefics-80b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics-80b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics-80b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics-80b with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics-80b
```

idefics-80b

Commit History

Aug 19, 2023

20:23
updates to the readme

VictorSanh

02787cf

Aug 18, 2023

23:16
revert

VictorSanh

338e8a2
20:41
add tokenizer.json for fast tokenizer

VictorSanh

3f2d3ab
13:50
add latest colab link and description

Leyo

395412b
11:06
update_all_readme_script (#15)⁠

Leyo

47f3ffb

Aug 17, 2023

21:45
update the generation args

VictorSanh

8a29e64

Aug 15, 2023

08:51
update colab link

Leyo

1552af8

Aug 14, 2023

13:31
fix link stable bias

Leyo

0fc4c61
12:47
Add more people to contributors

Leyo

6982895

Aug 13, 2023

19:51
update .gitattributes

VictorSanh

eea9385
19:40
integrate Giada's comments

VictorSanh

a6e5964

Aug 11, 2023

15:01
update links to bias sections (#14)⁠

davanstrien HF Staff

40ba628
14:33
update_colab_image (#13)⁠

Leyo

1e7399a
14:22
add logo

Leyo

fa3b42d
14:15
add_assets (#12)⁠

Leyo

ffce151
08:21
Update model cards authors

Leyo

a62b219

Aug 10, 2023

12:18
fix_details_model_card (#11)⁠

Leyo

4edcc87
12:11
Update README.md

HugoLaurencon

697ab43

Aug 9, 2023

09:26
Upload Figure_Evals_IDEFICS.png

HugoLaurencon

7ec80bb
09:26
Delete assets/Figure_Evals_IDEFICS.png

HugoLaurencon

37dffe3

Aug 8, 2023

18:21
update_tables (#10)⁠

Leyo

c5cc8c2
00:31
update_vic_2 (#8)⁠

VictorSanh

6c9a197

Aug 7, 2023

17:53
add column to distinguish dataset part of finetuning mix

Leyo

599076c
17:33
Model card bias section (#6)⁠

VictorSanh +1: davanstrien

e7e7632

Aug 6, 2023

21:43
Update README.md

HugoLaurencon

d999531
21:41
Update README.md

HugoLaurencon

5d9b2ec
21:10
Update README.md

HugoLaurencon

4ece772
21:10
Upload Figure_Evals_IDEFICS.png

HugoLaurencon

6a31588
21:10
Delete assets/Figure_Evals_IDEFIX.png

HugoLaurencon

827ae2f

Aug 4, 2023

22:40
Add stars for evals in instruction dataset

Leyo

9053c22
22:06
Fix model naming (idefix/idefics)

Leyo

96555cb
22:00
Update Idefics instruct eval table with new merged table

Leyo

28fdffe
09:30
switch Base vs Instruct

Leyo

8ded392
09:29
add separator

Leyo

7e556e9
09:28
Add fairness evals for Idefics instruct

Leyo

455a3a6
09:19
Add idefics instruct comparison table

Leyo

89c2eea
08:46
Add Idefics instruct evals

Leyo

e23861b

Aug 3, 2023

21:11
small wording issue

VictorSanh

d464935
21:09
add instruct model card

VictorSanh

a2ee636
19:22
update figure

VictorSanh

e434afa
17:43
fix rendering issue

VictorSanh

da5defa
17:35
images

VictorSanh

91b973f
17:24
add assets path

VictorSanh

9d30b49
17:19
update_readme (#7)⁠

VictorSanh

79d1def
11:13
add new lines + imagenet

Leyo

9d73c99
10:46
Add Fairness eval table

Leyo

dcb7082
10:35
Update Table with random shots numbers

Leyo

a1d95e2

Aug 2, 2023

14:04
update transformers version in generation_config to match idefics-9b

VictorSanh

809846f

Jul 31, 2023

17:37
enable legacy=True

stas

936694b

Jul 27, 2023

17:54
Update config.json (#5)⁠

stas

44215ea

Commit History

Aug 19, 2023

updates to the readme

Aug 18, 2023

revert

add tokenizer.json for fast tokenizer

add latest colab link and description

update_all_readme_script (#15)⁠

Aug 17, 2023

update the generation args

Aug 15, 2023

update colab link

Aug 14, 2023

fix link stable bias

Add more people to contributors

Aug 13, 2023

update .gitattributes

integrate Giada's comments

Aug 11, 2023

update links to bias sections (#14)⁠

update_colab_image (#13)⁠

add logo

add_assets (#12)⁠

Update model cards authors

Aug 10, 2023

fix_details_model_card (#11)⁠

Update README.md

Aug 9, 2023

Upload Figure_Evals_IDEFICS.png

Delete assets/Figure_Evals_IDEFICS.png

Aug 8, 2023

update_tables (#10)⁠

update_vic_2 (#8)⁠

Aug 7, 2023

add column to distinguish dataset part of finetuning mix

Model card bias section (#6)⁠

Aug 6, 2023

Update README.md

Update README.md

Update README.md

Upload Figure_Evals_IDEFICS.png

Delete assets/Figure_Evals_IDEFIX.png

Aug 4, 2023

Add stars for evals in instruction dataset

Fix model naming (idefix/idefics)

Update Idefics instruct eval table with new merged table

switch Base vs Instruct

add separator

Add fairness evals for Idefics instruct

Add idefics instruct comparison table

Add Idefics instruct evals

Aug 3, 2023

small wording issue

add instruct model card

update figure

fix rendering issue

images

add assets path

update_readme (#7)⁠

add new lines + imagenet

Add Fairness eval table

Update Table with random shots numbers

Aug 2, 2023

update transformers version in generation_config to match idefics-9b

Jul 31, 2023

enable legacy=True

Jul 27, 2023

Update config.json (#5)⁠