gnitoahc/ceed-b1-gemma4-e4b-it-0729

CEED B1 — supervised fine-tune of Gemma-4-E4B-it

A standalone checkpoint of google/gemma-4-e4b-it with a LoRA adapter folded into the weights. Same architecture, config, and processor as the base model — loadable with plain transformers, no PEFT and no CEED code required.

This is Group B1 of the CEED baselines: supervised fine-tuning on gold answers with no teacher signal (kd_weight: 0). It is the no-distillation control, not the primary result.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained(
    "gnitoahc/ceed-b1-gemma4-e4b-it-0729", dtype="float16", device_map="auto"
)
processor = AutoProcessor.from_pretrained("gnitoahc/ceed-b1-gemma4-e4b-it-0729")

Prompt it the way it was trained and scored — a single user turn containing the image followed by the question — or the reported number will not reproduce.

Training

Base	google/gemma-4-e4b-it
Method	LoRA, rank 4 — 2,269,184 trainable of 7,941,100,832 parameters
Steps	2,000 (one example per step)
Corpus	DocVQA (source validation split, re-split 80/10/10 by example id, seed 0)
Final train loss	1.405 (pure cross-entropy; KD term 0.0)

Evaluation

Greedy decoding, max_new_tokens=64, harness ceed-direct-1.

Dataset	Split	Metric	n	Score
DocVQA	validation	ANLS	565	0.809

Held-out validation, not test. No comparison against the untuned base model is claimed here.

Provenance and limitations

ceed_provenance.json in this repo records the source run's config_hash, param_efficiency, and metrics beside the weights.

• These are LoRA-derived weights. Merging folds W' = W + (alpha/r)BA into the base tensors; it does not make the result a full fine-tune. Cite it as a rank-4 LoRA run.
• Weights are fp16; the merge arithmetic itself ran in fp32 on CPU.
• Single-domain. Trained and evaluated on DocVQA only. Behaviour on natural images, charts, or non-English documents is untested and inherits whatever the base model does.
• Inherits the base model's limitations and biases, and is subject to the Gemma Terms of Use and the Gemma Prohibited Use Policy.