{"id":"CVE-2026-85670","state":"PUBLISHED","title":"tokenizers BpeBuilder Buffer Overflow via merge token","description":"tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.","publishedAt":"2026-09-04T14:32:25.391Z","updatedAt":"2026-09-04T17:49:42.263Z","severity":{"score":7.1,"label":"High","version":"4.0","vector":"CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N","source":"VulnCheck"},"assigner":"VulnCheck","products":["huggingface tokenizers"],"affected":[{"vendor":"huggingface","product":"tokenizers","packageURL":"pkg:pypi/tokenizers","defaultStatus":"unaffected","versions":[{"version":"0","status":"affected","lessThanOrEqual":"0.23.2","versionType":"semver"}]}],"references":["https://github.com/huggingface/tokenizers/issues/2094","https://github.com/huggingface/tokenizers","https://github.com/huggingface/tokenizers/blob/v0.23.2/tokenizers/src/models/bpe/model.rs","https://www.vulncheck.com/advisories/tokenizers-bpebuilder-buffer-overflow-via-merge-token"],"weaknesses":["CWE-787"],"ingestedAt":"2026-10-29T20:13:15.269Z","source":{"repository":"CVEProject/cvelistV5","path":"cves/2026/85xxx/CVE-2026-85670.json","commit":"556c9d3a8cd17e218bb8e452ba7cd619a384ff47"},"raw":{"dataType":"CVE_RECORD","dataVersion":"5.2","cveMetadata":{"cveId":"CVE-2026-85670","state":"PUBLISHED","datePublished":"2026-09-04T14:32:25.391Z","dateUpdated":"2026-09-04T17:49:42.263Z","assignerShortName":"VulnCheck","assignerOrgId":"83251b91-4cc7-4094-a5c7-464a1b83ea10","dateReserved":"2026-09-04T13:32:36.617Z"},"containers":{"cna":{"title":"tokenizers BpeBuilder Buffer Overflow via merge token","descriptions":[{"lang":"en","value":"tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1."}],"affected":[{"vendor":"huggingface","product":"tokenizers","packageURL":"pkg:pypi/tokenizers","defaultStatus":"unaffected","versions":[{"version":"0","status":"affected","lessThanOrEqual":"0.23.2","versionType":"semver"}]}],"references":[{"url":"https://github.com/huggingface/tokenizers/issues/2094","tags":["issue-tracking"],"name":"GitHub Issue #2094"},{"url":"https://github.com/huggingface/tokenizers","tags":["product"]},{"url":"https://github.com/huggingface/tokenizers/blob/v0.23.2/tokenizers/src/models/bpe/model.rs","tags":["technical-description"]},{"url":"https://www.vulncheck.com/advisories/tokenizers-bpebuilder-buffer-overflow-via-merge-token","name":"VulnCheck Advisory: tokenizers BpeBuilder Buffer Overflow via merge token","tags":["third-party-advisory"]}],"metrics":[{"format":"CVSS","cvssV4_0":{"version":"4.0","vectorString":"CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N","attackVector":"NETWORK","attackComplexity":"LOW","attackRequirements":"NONE","privilegesRequired":"NONE","userInteraction":"PASSIVE","vulnConfidentialityImpact":"NONE","vulnIntegrityImpact":"NONE","vulnAvailabilityImpact":"HIGH","subConfidentialityImpact":"NONE","subIntegrityImpact":"NONE","subAvailabilityImpact":"NONE","baseScore":7.1,"baseSeverity":"HIGH"}},{"format":"CVSS","cvssV3_1":{"version":"3.1","vectorString":"CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H","attackVector":"NETWORK","attackComplexity":"LOW","privilegesRequired":"NONE","userInteraction":"REQUIRED","scope":"UNCHANGED","confidentialityImpact":"NONE","integrityImpact":"NONE","availabilityImpact":"HIGH","baseScore":6.5,"baseSeverity":"MEDIUM"}}],"problemTypes":[{"descriptions":[{"cweId":"CWE-787","lang":"en","description":"Out-of-bounds Write","type":"CWE"}]}],"providerMetadata":{"shortName":"VulnCheck","orgId":"83251b91-4cc7-4094-a5c7-464a1b83ea10","dateUpdated":"2026-09-04T14:32:25.391Z"},"datePublic":"2026-06-10T00:00:00.000Z","credits":[{"lang":"en","value":"George Chen","type":"finder"}],"x_generator":{"engine":"vulncheck-endgame"}},"adp":[{"title":"CISA ADP Vulnrichment","references":[{"url":"https://github.com/huggingface/tokenizers/issues/2094","tags":["exploit"]}],"metrics":[{"other":{"type":"ssvc","content":{"timestamp":"2026-09-04T17:49:21.636135Z","id":"CVE-2026-85670","options":[{"Exploitation":"poc"},{"Automatable":"no"},{"Technical Impact":"partial"}],"role":"CISA Coordinator","version":"2.0.3"}}}],"providerMetadata":{"shortName":"CISA-ADP","orgId":"134c704f-9b21-4f2e-91b3-4a467353bcc0","dateUpdated":"2026-09-04T17:49:42.263Z"}}]}}}