| Titel | AkariAsai self-rag 1fcdc420e48f50a7d7ab1ece5494221b93252e99 Unsafe Deserialization / Integrity Bypass |
|---|
| Beschreibung | ## Vulnerability Title
Supply Chain Integrity Bypass via Weak Checksum Validation in Retriever Index Cache
## Affected Component
`retrieval_lm/src/index.py` and `retrieval_lm/passage_retrieval.py`
Repository: https://github.com/AkariAsai/self-rag
## Summary
An attacker with the ability to supply or replace retriever index cache files can craft a malicious `index_meta.faiss` artifact. The cache loader performs no trusted checksum, signature, or provenance validation and then deserializes the metadata with `pickle.load()`. This leads to arbitrary code execution in the retriever process and negatively impacts systems that load untrusted Self-RAG retriever artifacts.
## Technical Details
The vulnerable path is an artifact integrity failure in the retriever index cache. No security-relevant hash, checksum, signature, or authenticated manifest is validated before loading executable pickle metadata. The application trusts the file based merely on its location in the retriever cache directory and deserializes it directly.
**Vulnerable Cache Loader**
In `retrieval_lm/src/index.py`, the `deserialize_from()` method loads the `index_meta.faiss` file directly using `pickle.load()`:
```python
def deserialize_from(self, dir_path):
index_file = os.path.join(dir_path, 'index.faiss')
meta_file = os.path.join(dir_path, 'index_meta.faiss')
# ...
with open(meta_file, "rb") as reader:
self.index_id_to_db_id = pickle.load(reader)
assert len(self.index_id_to_db_id) == self.index.ntotal, '...'
```
Because `pickle.load()` is inherently unsafe when handling untrusted data, a malicious artifact can execute arbitrary code during the unpickling process, long before the `assert` length check ever occurs.
**How the Attacker Constructs a Conflicting Object**
Because the application has no authenticated artifact identity, two cache states can occupy the same directory and filename while being security-distinct:
1. A benign `index_meta.faiss` containing only the expected ID mapping.
2. A malicious `index_meta.faiss` containing pickle opcodes that execute arbitrary code and then return a list-like value to bypass subsequent scrutiny.
The application makes a trust decision based purely on cache path existence (`if self.args.save_or_load_index and os.path.exists(index_path):`). If an attacker places a crafted `index_meta.faiss` in this directory, it will be blindly executed.
## Impact
This vulnerability allows attackers to:
- Execute arbitrary code as the user running the retriever process when a malicious cache artifact is loaded.
- Steal local secrets such as API keys, model access tokens, environment variables, or internal service credentials.
- Poison downstream retrieval behavior by replacing or corrupting retriever cache artifacts.
*(Note: The attack is persistent, as a malicious `index_meta.faiss` can remain in a shared or downloaded cache directory and execute whenever the affected cache-loading path is triggered).*
## Proof of Concept
The PoC proves that code embedded in `index_meta.faiss` executes during `pickle.load()` before any validation or length assertion checks happen.
```python
# A malicious pickle generation script can create an exploit payload:
import builtins
import pathlib
import pickle
class Exploit:
def __reduce__(self):
payload = (
"(__import__('pathlib').Path('/tmp/agentscan_pickle_marker')"
".write_text('pickle executed'), [])[1]"
)
return builtins.eval, (payload,)
# Write this payload into 'index_meta.faiss'
pathlib.Path("index_meta.faiss").write_bytes(pickle.dumps(Exploit()))
```
If this generated `index_meta.faiss` is placed alongside an `index.faiss` file and loaded via `self.index.deserialize_from(embeddings_dir)`, the code immediately executes and writes the marker file to `/tmp`, confirming arbitrary code execution.
## Remediation
Replace executable `pickle` metadata with a safe data format and authenticate retriever cache artifacts before loading them.
Example implementation direction:
```python
# In retrieval_lm/src/index.py
# Replace pickle with a non-executable format like JSON.
import json
with open(meta_file, "r") as reader:
self.index_id_to_db_id = json.load(reader)
```
Additional mitigations:
1. Complete Field Coverage: Bind `index.faiss`, `index_meta` metadata, schema version, generator version, and provenance in a trusted manifest using SHA-256 or cryptographic signatures.
2. Read-Time Revalidation: Validate metadata type, length, schema, and binding to the FAISS index before use.
3. State Invalidation: Regenerate or remove existing `pickle`-based `index_meta.faiss` files from untrusted cache locations.
## References
- Vulnerable deserialization: `https://github.com/AkariAsai/self-rag/blob/1fcdc420e48f50a7d7ab1ece5494221b93252e99/retrieval_lm/src/index.py#L57-L68`
- CLI cache-loading path: `https://github.com/AkariAsai/self-rag/blob/1fcdc420e48f50a7d7ab1ece5494221b93252e99/retrieval_lm/passage_retrieval.py#L138-L143`
- Demo cache-loading path: `https://github.com/AkariAsai/self-rag/blob/1fcdc420e48f50a7d7ab1ece5494221b93252e99/retrieval_lm/passage_retrieval.py#L187-L192`
- Cache-loading flag: `https://github.com/AkariAsai/self-rag/blob/1fcdc420e48f50a7d7ab1ece5494221b93252e99/retrieval_lm/passage_retrieval.py#L252-L254` |
|---|
| Quelle | ⚠️ https://github.com/AkariAsai/self-rag/issues/105 |
|---|
| Benutzer | Dem00000 (UID 98563) |
|---|
| Einreichung | 10.06.2026 11:12 (vor 1 Monat) |
|---|
| Moderieren | 12.07.2026 19:56 (1 month later) |
|---|
| Status | Akzeptiert |
|---|
| VulDB Eintrag | 377885 [AkariAsai self-rag bis 1fcdc420e48f50a7d7ab1ece5494221b93252e99 retrieval_lm index.py Indexer.deserialize_from index_meta.faiss erweiterte Rechte] |
|---|
| Punkte | 20 |
|---|