README.md

## Modularized LLM-Reranking Library: 
This modularized LLM-Reranking project provides a flexible framework for reranking search results using large language models (LLMs). 
It allows users to easily experiment and integrate different components from different neural reranking methods. 

[July 4, 2025] We consider `sorting algorithm`, `prompting template` and corresponding `rankign parser` with different `LLM backend`.

## Installation
- Environment
```
conda create -n rerank python=3.10
pip install uv
```
- Dependency Installation
```
git clone https://github.com/DylanJoo/APRIL.git
cd APRIL
uv pip install -e .
uv pip install vllm==0.11.1 ftfy ir_datasets ir_measures 
```
The final `requirements.txt` will be provided in the future release.

## Strcutre [TODO: update to the beta version]
```
APRIL/ # the proposed new method using `autollmrerank`.
├── unittest/li_textlist.py
├── src/
│   └── autollmrerank/
│       ├── __init__.py
│       ├── config_manager.py
│       ├── utils.py
│       ├── prompt_builder/
│       │   ├── base.py
│       │   └── _rank_gpt.py
│       ├── llm_provider/
│       │   ├── base.py
│       │   └── _rank_gpt.py
│       ├── result_parser/
│       │   ├── base.py
│       │   └── _rank_gpt.py
│       └── tests/
│           └── test_model.py
└── examples/
    ├── run_trec-dl-2020.sh
    └── log.vllm/
```

#### Wrapper/main functions
- ModularReranker 
    * A wrapper class that defines the reranking types for class factory, which can integrates all the following 4 components.

#### Four modules 
- InputAssebler (rename? scheduler? handler? ...)
    * Input: query and results
    * Output: list of query-documents pairs

- PromptBuilder
    * Input: query and documents 
    * Output: text prompts for LLM

- LLMProvider
    * Input: text prompts
    * output: text outputs or list of numbers

- ResultParser
    * Input: text outputs or list of numbers
    * Output: Result object with sorted results

#### Utililty functions/classes
- Result: the class of retrieval/ranking results.
- PromptMode: the class of reranking mode, including the prompt, llm calling and parsing

#### Top 100 pool
Average pool size 255.1627906976744
Average pool size 145.625
Average pool size 325.32
Average pool size 278.23839009287923
Average pool size 278.408
Average pool size 291.54
Average pool size 264.40816326530614

#### Top 20 pool
Average pool size 97.90697674418605
Average pool size 41.815
Average pool size 144.32
Average pool size 122.30030959752322
Average pool size 116.86
Average pool size 123.94
Average pool size 127.40816326530613