Hugging Face tokenizers up to 0.23.2 BpeBuilder model.rs BpeBuilder::build buffer overflow

Summaryinfo

A vulnerability was found in Hugging Face tokenizers up to 0.23.2. It has been classified as problematic. Affected is the function BpeBuilder::build of the file tokenizers/src/models/bpe/model.rs of the component BpeBuilder. The manipulation leads to buffer overflow. This vulnerability is uniquely identified as CVE-2026-85670. The attack is possible to be carried out remotely. No exploit exists.

Detailsinfo

A vulnerability, which was classified as problematic, has been found in Hugging Face tokenizers up to 0.23.2. This issue affects the function BpeBuilder::build of the file tokenizers/src/models/bpe/model.rs of the component BpeBuilder. The manipulation with an unknown input leads to a buffer overflow vulnerability. Using CWE to declare the problem leads to CWE-120. The product copies an input buffer to an output buffer without verifying that the size of the input buffer is less than the size of the output buffer, leading to a buffer overflow. Impacted is availability. The summary by CVE is:

tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.

The advisory is shared at github.com. The identification of this vulnerability is CVE-2026-85670 since 09/04/2026. The exploitation is known to be easy. The attack may be initiated remotely. No form of authentication is needed for a successful exploitation. It demands that the victim is doing some kind of user interaction. Technical details are known, but no exploit is available. The price for an exploit might be around USD $0-$5k at the moment (estimation calculated on 09/04/2026).

There is no information about possible countermeasures known. It may be suggested to replace the affected object with an alternative product.

If you want to get the best quality for vulnerability data then you always have to consider VulDB.

Productinfo

Vendor

• Hugging Face

Name

• tokenizers

Version

• 0.23.0
• 0.23.1
• 0.23.2

CPE 2.3info

• 🔒
• 🔒
• 🔒

CPE 2.2info

• 🔒
• 🔒
• 🔒

CVSSv4info

CVSSv3info

CVSSv2info

Exploitinginfo

Threat Intelligenceinfo

Countermeasuresinfo

Timelineinfo

Sourcesinfo

Entryinfo

If you want to get the best quality for vulnerability data then you always have to consider VulDB.

Discussion