Add CPU and GPU usage examples

README.md

@@ -45,6 +45,103 @@ This checkpoint is intended for research on:
 - recurrent-state contamination during long generation
 - Reference-State Reset with Rolling Replay, a GDN/RNN adaptation of the R-SWA idea
 
+## Usage
+
+These checkpoints use the FLA Gated DeltaNet implementation. In the current
+Long-GDN environment, plain `GatedDeltaNetForCausalLM.from_pretrained()` can
+hit a Transformers 5.x tied-weight metadata issue. The robust path is to patch
+the FLA tied-weight metadata before loading.
+
+Install/runtime requirements:
+
+```bash
+pip install torch transformers safetensors huggingface_hub
+# plus an FLA package/source tree that provides:
+#   fla.models.gated_deltanet.GatedDeltaNetForCausalLM
+```
+
+### CPU Example
+
+```python
+import torch
+from transformers import AutoTokenizer
+from fla.models.gated_deltanet import GatedDeltaNetForCausalLM
+
+repo_id = "LLM-OS-Models/gdn1-longctx-1b-32k-answer-ft"
+
+# Transformers 5.x compatibility patch for the installed FLA class.
+if isinstance(getattr(GatedDeltaNetForCausalLM, "_tied_weights_keys", None), list):
+    GatedDeltaNetForCausalLM._tied_weights_keys = {
+        "lm_head.weight": "model.embeddings.weight"
+    }
+
+tokenizer = AutoTokenizer.from_pretrained(repo_id, use_fast=True)
+model = GatedDeltaNetForCausalLM.from_pretrained(
+    repo_id,
+    torch_dtype=torch.float32,
+)
+model.eval()
+
+prompt = "A special magic number is 12345. What is the special magic number?"
+inputs = tokenizer(prompt, return_tensors="pt")
+with torch.no_grad():
+    output = model.generate(
+        **inputs,
+        max_new_tokens=32,
+        do_sample=False,
+    )
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+### Single-GPU bf16 Example
+
+```python
+import torch
+from transformers import AutoTokenizer
+from fla.models.gated_deltanet import GatedDeltaNetForCausalLM
+
+repo_id = "LLM-OS-Models/gdn1-longctx-1b-32k-answer-ft"
+
+if isinstance(getattr(GatedDeltaNetForCausalLM, "_tied_weights_keys", None), list):
+    GatedDeltaNetForCausalLM._tied_weights_keys = {
+        "lm_head.weight": "model.embeddings.weight"
+    }
+
+tokenizer = AutoTokenizer.from_pretrained(repo_id, use_fast=True)
+model = GatedDeltaNetForCausalLM.from_pretrained(
+    repo_id,
+    torch_dtype=torch.bfloat16,
+).to("cuda")
+model.eval()
+
+prompt = "Reference facts:\n- key_alpha: value_123\n\nQuestion: key_alpha?\nAnswer:"
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+with torch.no_grad():
+    output = model.generate(
+        **inputs,
+        max_new_tokens=32,
+        do_sample=False,
+    )
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+### Long-GDN Local Loader
+
+The project repository includes a more defensive loader at
+`scripts/gdn1_common.py::load_gdn1_causal_lm`. It handles the compatibility
+patch and older public-checkpoint key conversion used in local experiments.
+
+```python
+from pathlib import Path
+import torch
+from transformers import AutoTokenizer
+from scripts.gdn1_common import load_gdn1_causal_lm
+
+repo_or_local_path = Path("path/to/downloaded/checkpoint")
+tokenizer = AutoTokenizer.from_pretrained(repo_or_local_path, use_fast=True)
+model = load_gdn1_causal_lm(repo_or_local_path, torch_dtype=torch.bfloat16).to("cuda")
+```
+
 ## Known Results
 
 1B 32K answer-focused run. MQAR likelihood improved 4K/16K but damaged 1K and did not transfer to 32K/64K: final 1K 0.0000, 2K 0.0625, 4K 0.1562, 8K 0.0625, 16K 0.1875, 32K 0.0000, 64K 0.0000.