Skip to content
← All work
activeJan 2026 — Present

Modular Memory Unit

Hybrid retrieval RAG framework

A retrieval framework that fuses dense and lexical signals with Reciprocal Rank Fusion, benchmarked on legal and compliance documents across five custom metrics including faithfulness and cross-document reasoning.

Retrieval signals fused
3
BGE-M3 · FAISS · BM25
Custom eval metrics
5
incl. faithfulness, cross-doc reasoning
Target venue
EMNLP/ACL
Student Research Workshop

The problem

Retrieval-augmented generation over legal and compliance corpora punishes the usual shortcuts. Dense retrievers are good at paraphrase and bad at the exact statutory reference a clause hinges on. Lexical retrievers are the reverse: they will find §4(2)(b) every time and miss the paragraph three documents away that qualifies it.

Picking one means accepting a known failure mode. On documents where being wrong has consequences, that is not a trade worth making silently.

The approach

MMU runs dense and lexical retrieval side by side and fuses the ranked lists with Reciprocal Rank Fusion.

  • BGE-M3 provides the dense embedding signal.
  • FAISS serves the dense index.
  • BM25 provides the lexical signal.
  • Reciprocal Rank Fusion combines the ranked lists without needing the two retrievers' scores to be on a comparable scale — which is the whole reason RRF is the right tool rather than a weighted score blend.

The framework is deliberately modular: the retrievers sit behind a common interface so a signal can be swapped or added without touching the fusion layer or the evaluation harness.

Evaluation

Rather than reporting a single retrieval score, MMU is benchmarked across five custom metrics on legal and compliance documents. Two of them matter most for this domain:

  • Faithfulness — whether the generated answer is actually supported by the retrieved context, rather than merely consistent with it.
  • Cross-document reasoning — whether the system can combine evidence that is split across documents, which is where single-retriever setups tend to quietly fail.

TODO —

Add the measured numbers per metric, and the comparison against dense-only and BM25-only baselines, once the benchmark run is final.

Serving

The framework ships behind a FastAPI backend:

  • HybridRetriever as the entry point, so callers do not need to know how many signals are being fused.
  • An async/sync thread-pool pattern, so the synchronous retrieval libraries do not block the event loop.
  • SSE streaming, so answers surface token by token instead of after the full generation completes.

Status

Active. The work is being written up with the EMNLP/ACL Student Research Workshop as the target venue.