Instructions to use llm-agents/tora-13b-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use llm-agents/tora-13b-v1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="llm-agents/tora-13b-v1.0")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("llm-agents/tora-13b-v1.0")
model = AutoModelForCausalLM.from_pretrained("llm-agents/tora-13b-v1.0")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use llm-agents/tora-13b-v1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "llm-agents/tora-13b-v1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llm-agents/tora-13b-v1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/llm-agents/tora-13b-v1.0

SGLang

How to use llm-agents/tora-13b-v1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "llm-agents/tora-13b-v1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llm-agents/tora-13b-v1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "llm-agents/tora-13b-v1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llm-agents/tora-13b-v1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use llm-agents/tora-13b-v1.0 with Docker Model Runner:
```
docker model run hf.co/llm-agents/tora-13b-v1.0
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

ToRA: A Tool-Integrated Reasoning Agent
for Mathematical Problem Solving

[🌐 Website] • [📜 Paper] • [🤗 HF Models] • [🐱 GitHub]
[🐦 Twitter] • [💬 Reddit] • [🍀 Unofficial Blog]

Repo for "ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving"

🔥 News

[2023/10/08] 🔥🔥🔥 All ToRA models released at HuggingFace!!!
[2023/09/29] ToRA paper, repo, and website released.

💡 Introduction

ToRA is a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical reasoning problems by interacting with tools, e.g., computation libraries and symbolic solvers. ToRA series seamlessly integrate natural language reasoning with the utilization of external tools, thereby amalgamating the analytical prowess of language and the computational efficiency of external tools.

Model	Size	GSM8k	MATH	AVG@10 math tasks^†
GPT-4	-	92.0	42.5	78.3
GPT-4 (PAL)	-	94.2	51.8	86.4
ToRA-7B	7B	68.8	40.1	62.4
ToRA-Code-7B	7B	72.6	44.6	66.5
ToRA-13B	13B	72.7	43.0	65.9
ToRA-Code-13B	13B	75.8	48.1	71.3
ToRA-Code-34B^*	34B	80.7	51.0	74.8
ToRA-70B	70B	84.3	49.7	76.9

^*ToRA-Code-34B is currently the first and only open-source model to achieve over 50% accuracy (pass@1) on the MATH dataset, which significantly outperforms GPT-4’s CoT result (51.0 vs. 42.5), and is competitive with GPT-4 solving problems with programs. By open-sourcing our codes and models, we hope more breakthroughs will come!
^†10 math tasks include GSM8k, MATH, GSM-Hard, SVAMP, TabMWP, ASDiv, SingleEQ, SingleOP, AddSub, and MultiArith.

⚡️ Training

The models are trained on ToRA-Corpus 16k, which contains tool-integrated reasoning trajectories of MATH and GSM8k from GPT-4.

We use imitation learning (i.e., SFT) to fine-tune the models, and then apply our proposed output space shaping to improve tool-integrated reasoning behaviors. Please refer to the paper for more details.

🪁 Inference & Evaluation

Please refer to ToRA's GitHub repo for inference, evaluation, and training code.

☕️ Citation

If you find this repository helpful, please consider citing our paper:

@misc{gou2023tora,
      title={ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving}, 
      author={Zhibin Gou and Zhihong Shao and Yeyun Gong and yelong shen and Yujiu Yang and Minlie Huang and Nan Duan and Weizhu Chen},
      year={2023},
      eprint={2309.17452},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Downloads last month: 1,018

Model tree for llm-agents/tora-13b-v1.0

Quantizations

3 models

Dataset used to train llm-agents/tora-13b-v1.0

Paper for llm-agents/tora-13b-v1.0

ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving

Paper • 2309.17452 • Published Sep 29, 2023 • 3