University of Pennsylvania · Prof. Chris Callison-Burch · May 2025 – Aug 2025
Retrieval-Augmented Generation for DARPA SciFy and OpenScholar
Designed and evaluated retrieval-augmented generation workflows for scientific question answering on DARPA SciFy and OpenScholar, with a focus on retriever ranking quality and reproducible evaluation.
Contributions
- Implemented score-based filtering on top of dense retrievers to reduce irrelevant context passed to the generator.
- Prepared and standardized domain datasets so retriever swaps and generator swaps could be evaluated on the same evidence pool.
- Wrote reusable training and evaluation scripts and lab-facing documentation so other researchers could reproduce results.
Methods
- Contrastive retriever fine-tuning
- Score-based passage filtering
- RAG evaluation harness
- Domain dataset curation
PythonPyTorchHugging FaceRetrievalRAGNLP