YihangSun/hw1-hc3-detector

HW1 — HC3 AI-Generated Text Detector

Fine-tuned sentence-transformers/all-MiniLM-L6-v2 for binary AI-generated text detection on the HC3 (English) corpus.

Labels: 0 = human, 1 = ChatGPT.

Results (held-out HC3 test split, 4668 answers)

Model	Test accuracy
Frozen sentence-transformers/all-MiniLM-L6-v2 embeddings + logistic regression (baseline)	0.8449
Fine-tuned encoder + classification head (this model)	0.9901

Confusion matrix of the fine-tuned model (rows = true, cols = predicted; order human, ChatGPT):

[[2288   46]
 [   0 2334]]

Training setup

• Data: HC3 English, revision 4d0ff18143b5a7e1b1e79beb540c04549d1e59d3, one human and one ChatGPT answer per question, split 80/10/10 by question with seed 42 so no question crosses splits.
• Objective: cross-entropy, num_labels=2.
• Optimizer: AdamW, learning rate 2e-05, batch size 32, 5 epochs, max_length=256 with dynamic padding.

Intended use and limitations

Coursework artifact for CS546. HC3 was collected from ChatGPT in late 2022 under a single prompting setup, and part of what this classifier separates is collection artifacts rather than authorship. It is not a reliable detector for contemporary AI-generated text and must not be used to make accusations about real student work.