mradermacher

/

model_requests

like 168

crashed 2 weeks ago, let's try again
It's queued!

You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#EnceladusHyperStock-24B-GGUF for quants to appear.

It seems to have crashed again 😒

seems something is wrong, either tokenizer is modified or the model is not supported, the usual NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre() from llama cpp

seems something is wrong, either tokenizer is modified or the model is not supported, the usual NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre() from llama cpp

How am I not surprised? We are currently running inference on this model and BPE was a total shitshow. It is in fact so broken I had to write the following function to postprocess the vllm output which is the first model we ever tried that required postprocessing:

def nuke_llm_garbage(text: str) -> str:
    """Forcefully converts byte-level artifacts back to normal whitespace."""
    if not text:
        return ""

    # Map the entire known family of BPE character artifacts
    garbage_map = str.maketrans({
        'Ġ': ' ',   # The annoying space replacement
        'Ċ': '\n',  # The annoying newline replacement
        'ĉ': '\r',  # Carriage return
        'ċ': '\t',  # Tab fallback
    })
    return text.translate(garbage_map)

Seems like @ShyliaSafetensors has created a white crow!

should i take this as a compliment or roast? i genuinely dont know about tokenizer and stuff, im just doing 'random bullshit go!' while merging models to achieve good-ish model, then testing little bit by myself. how this happen? lol

should i take this as a compliment or roast?

Both? There may be a structural issue with Mistral 24B merges.