docs/cli_usage.md

# CLI Usage

LLMPromptKit provides a command-line interface (CLI) for managing prompts, versions, tests, and evaluations.

## Basic Commands

### Prompt Management

```bash
# Create a prompt
llmpromptkit prompt create "Weather Forecast" --content "Provide a weather forecast for {location} on {date}" --tags "weather,forecast"

# List all prompts
llmpromptkit prompt list

# Get prompt details
llmpromptkit prompt get <prompt_id>

# Update a prompt
llmpromptkit prompt update <prompt_id> --content "New content" --tags "new,tags"

# Delete a prompt
llmpromptkit prompt delete <prompt_id>
```

### Version Control

```bash
# Commit a version
llmpromptkit version commit <prompt_id> --message "Version description"

# List versions
llmpromptkit version list <prompt_id>

# Check out (revert to) a specific version
llmpromptkit version checkout <prompt_id> <version_number>

# Compare versions
llmpromptkit version diff <prompt_id> <version1> <version2>
```

### Testing

```bash
# Create a test case
llmpromptkit test create <prompt_id> --input '{"location": "New York", "date": "tomorrow"}' --expected "Expected output"

# List test cases
llmpromptkit test list <prompt_id>

# Run a specific test case
llmpromptkit test run <test_case_id> --llm openai

# Run all test cases for a prompt
llmpromptkit test run-all <prompt_id> --llm openai

# Run an A/B test between two prompts
llmpromptkit test ab <prompt_id_a> <prompt_id_b> --inputs '[{"var": "value1"}, {"var": "value2"}]' --llm openai
```

### Evaluation

```bash
# Evaluate a prompt
llmpromptkit eval run <prompt_id> --inputs '[{"var": "value1"}, {"var": "value2"}]' --llm openai

# List available metrics
llmpromptkit eval metrics

# Register a custom metric
llmpromptkit eval register-metric <metric_file.py>
```

## Environment Configuration

The CLI supports environment variables for configuration:

- `LLMPROMPTKIT_STORAGE`: Path to store prompts and related data
- `LLMPROMPTKIT_OPENAI_API_KEY`: OpenAI API key for built-in LLM support
- `LLMPROMPTKIT_DEFAULT_LLM`: Default LLM to use for testing and evaluation

You can also create a config file at `~/.llmpromptkit/config.json`:

```json
{
  "storage_path": "/path/to/storage",
  "default_llm": "openai",
  "api_keys": {
    "openai": "your-openai-key"
  }
}
```

## Advanced Usage

### Multiple Storage Locations

```bash
# Specify a storage location for a command
llmpromptkit --storage /path/to/storage prompt list

# Export a prompt to another storage
llmpromptkit prompt export <prompt_id> --output /path/to/output.json

# Import a prompt from a file
llmpromptkit prompt import /path/to/prompt.json
```

### Automation and Scripting

```bash
# Get output in JSON format
llmpromptkit --json prompt list

# Use in shell scripts
PROMPT_ID=$(llmpromptkit --json prompt create "Script Prompt" --content "Content" | jq -r '.id')
echo "Created prompt with ID: $PROMPT_ID"
```