LatteReview 🤖☕

A framework for multi-agent review workflows using large language models.

Overview

LatteReview is a Python framework that enables you to create and manage multi-agent review workflows using various large language models. It provides a flexible and extensible architecture for implementing different types of review processes, from simple single-agent reviews to complex multi-stage workflows with multiple agents.

Features

• Multi-agent review system with customizable roles and expertise levels for each reviewer
• Support for multiple review rounds with hierarchical decision-making workflows
• Review diverse content types including article titles, abstracts, custom texts, and images using LLM-powered reviewer agents
• Define reviewer agents with specialized backgrounds and distinct evaluation capabilities
• Create flexible review workflows where multiple agents operate in parallel or sequential arrangements
• Enable reviewer agents to analyze peer feedback, cast votes, and propose corrections to other reviewers' assessments
• Enhance reviews with item-specific context integration, supporting use cases like Retrieval Augmented Generation (RAG)
• Broad compatibility with LLM providers through LiteLLM, including OpenAI and Ollama
• Model-agnostic integration supporting OpenAI, Gemini, Claude, Groq, and local models via Ollama
• High-performance asynchronous processing for efficient batch reviews
• Standardized output format featuring detailed scoring metrics and reasoning transparency
• Robust cost tracking and memory management systems
• Extensible architecture supporting custom review workflow implementation

Quick Links

• Installation Guide
• Quick Start Guide
• Tutorial notebooks
• API Reference
• GitHub Repository

License

This project is licensed under the MIT License - see the LICENSE file for details.

👨‍💻 Authors

Pouria Rouzrokh, MD, MPH, MHPE Medical Practitioner and Machine Learning Engineer Incoming Radiology Resident @Yale University Former Data Scientist @Mayo Clinic AI Lab

Support LatteReview

If you find LatteReview helpful in your research or work, consider supporting its continued development. Since we're already sharing a virtual coffee break while reviewing papers, maybe you'd like to treat me to a real one? ☕ 😊

Ways to Support:

• Become my sponsor on GitHub
• Treat me to a cup of coffee on Ko-fi ☕
• Star the repository to help others discover the project
• Submit bug reports, feature requests, or contribute code
• Share your experience using LatteReview in your research

Acknowledgement

I would like to express my heartfelt gratitude to Moein Shariatnia for his invaluable support and contributions to this project.

📚 Citation

If you use LatteReview in your research, please cite our paper:

@misc{rouzrokh2025lattereview,
    title={LatteReview: A Multi-Agent Framework for Systematic Review Automation Using Large Language Models},
    author={Pouria Rouzrokh and Moein Shariatnia},
    year={2025},
    eprint={2501.05468},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}