Instructions to use HuggingFaceM4/idefics-80b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics-80b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-80b")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-80b")
model = AutoModelForMultimodalLM.from_pretrained("HuggingFaceM4/idefics-80b", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HuggingFaceM4/idefics-80b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics-80b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics-80b

SGLang

How to use HuggingFaceM4/idefics-80b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics-80b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics-80b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics-80b with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics-80b
```

idefics-80b / README.md

Commit History

Oct 12, 2023

18:45
tgi infos

VictorSanh

48d1fa4

Aug 28, 2023

19:57
Adding carbon footprint information (#20)⁠

VictorSanh +1: sasha

b2fa6b4
12:27
Update README.md

patrickvonplaten

01698d0

Aug 22, 2023

17:59
Update README.md (#17)⁠

VictorSanh +1: stas

eddaa07
14:47
Disable inference (#16)⁠

VictorSanh +1: osanseviero

2ccbe9d

Aug 21, 2023

15:14
fix - thanks @joaogante !

VictorSanh

1c86537
14:06
more specific wording

VictorSanh

0f909ad
14:00
more specific wording

VictorSanh

34a2e3b
13:47
change order

VictorSanh

39b3f74

Aug 20, 2023

21:08
Update model card bias section (#9)⁠

VictorSanh +1: davanstrien

7b2c3a2
21:04
Update README.md

VictorSanh

39115ab

Aug 19, 2023

20:23
Merge branch 'main' of https://huggingface.co/HuggingFaceM3/idefics-80b into main

VictorSanh

86ca379
20:23
updates to the readme

VictorSanh

02787cf

Aug 18, 2023

23:42
Update README.md

VictorSanh

cb7b149
13:50
add latest colab link and description

Leyo

395412b

Aug 17, 2023

21:45
update the generation args

VictorSanh

8a29e64

Aug 15, 2023

08:51
update colab link

Leyo

1552af8

Aug 14, 2023

13:31
fix link stable bias

Leyo

0fc4c61
12:47
Add more people to contributors

Leyo

6982895

Aug 13, 2023

19:40
integrate Giada's comments

VictorSanh

a6e5964

Aug 11, 2023

15:01
update links to bias sections (#14)⁠

davanstrien HF Staff

40ba628
14:22
add logo

Leyo

fa3b42d
08:21
Update model cards authors

Leyo

a62b219

Aug 10, 2023

12:18
fix_details_model_card (#11)⁠

Leyo

4edcc87
12:11
Update README.md

HugoLaurencon

697ab43

Aug 8, 2023

18:21
update_tables (#10)⁠

Leyo

c5cc8c2
00:31
update_vic_2 (#8)⁠

VictorSanh

6c9a197

Aug 7, 2023

17:53
add column to distinguish dataset part of finetuning mix

Leyo

599076c
17:33
Model card bias section (#6)⁠

VictorSanh +1: davanstrien

e7e7632

Aug 6, 2023

21:43
Update README.md

HugoLaurencon

d999531
21:41
Update README.md

HugoLaurencon

5d9b2ec
21:10
Update README.md

HugoLaurencon

4ece772

Aug 4, 2023

22:40
Add stars for evals in instruction dataset

Leyo

9053c22
22:06
Fix model naming (idefix/idefics)

Leyo

96555cb
22:00
Update Idefics instruct eval table with new merged table

Leyo

28fdffe
09:30
switch Base vs Instruct

Leyo

8ded392
09:29
add separator

Leyo

7e556e9
09:28
Add fairness evals for Idefics instruct

Leyo

455a3a6
09:19
Add idefics instruct comparison table

Leyo

89c2eea
08:46
Add Idefics instruct evals

Leyo

e23861b

Aug 3, 2023

21:11
small wording issue

VictorSanh

d464935
21:09
add instruct model card

VictorSanh

a2ee636
17:43
fix rendering issue

VictorSanh

da5defa
17:35
images

VictorSanh

91b973f
17:24
add assets path

VictorSanh

9d30b49
17:19
update_readme (#7)⁠

VictorSanh

79d1def
11:13
add new lines + imagenet

Leyo

9d73c99
10:46
Add Fairness eval table

Leyo

dcb7082
10:35
Update Table with random shots numbers

Leyo

a1d95e2

Jul 25, 2023

03:05
Update README.md

VictorSanh

4a3d381

Commit History

Oct 12, 2023

tgi infos

Aug 28, 2023

Adding carbon footprint information (#20)⁠

Update README.md

Aug 22, 2023

Update README.md (#17)⁠

Disable inference (#16)⁠

Aug 21, 2023

fix - thanks @joaogante !

more specific wording

more specific wording

change order

Aug 20, 2023

Update model card bias section (#9)⁠

Update README.md

Aug 19, 2023

Merge branch 'main' of https://huggingface.co/HuggingFaceM3/idefics-80b into main

updates to the readme

Aug 18, 2023

Update README.md

add latest colab link and description

Aug 17, 2023

update the generation args

Aug 15, 2023

update colab link

Aug 14, 2023

fix link stable bias

Add more people to contributors

Aug 13, 2023

integrate Giada's comments

Aug 11, 2023

update links to bias sections (#14)⁠

add logo

Update model cards authors

Aug 10, 2023

fix_details_model_card (#11)⁠

Update README.md

Aug 8, 2023

update_tables (#10)⁠

update_vic_2 (#8)⁠

Aug 7, 2023

add column to distinguish dataset part of finetuning mix

Model card bias section (#6)⁠

Aug 6, 2023

Update README.md

Update README.md

Update README.md

Aug 4, 2023

Add stars for evals in instruction dataset

Fix model naming (idefix/idefics)

Update Idefics instruct eval table with new merged table

switch Base vs Instruct

add separator

Add fairness evals for Idefics instruct

Add idefics instruct comparison table

Add Idefics instruct evals

Aug 3, 2023

small wording issue

add instruct model card

fix rendering issue

images

add assets path

update_readme (#7)⁠

add new lines + imagenet

Add Fairness eval table

Update Table with random shots numbers

Jul 25, 2023

Update README.md