SPIGA/spiga/eval/benchmark/readme.md

# SPIGA: Benchmark
The benchmark evaluator can be found at ```./eval/benchmark/evaluator.py``` and it allows
to extract an extended report of metrics for each dataset. For further details,
check the parser and complete the interactive terminal procedure to specify the evaluation
characteristics.

In order to use the benchmark evaluation, the prediction file must follow the same data structure
and file extension as the ground truth annotations available in ```./data/annotations/<database_name>```. 
The data structure consist on a list of dictionaries where each one represents an image sample,
similar to the previous dataloader configuration:

```
sample = {"imgpath": Relative image path, 
          "bbox": Bounding box [x,y,w,h] (ref image), 
          "headpose": Euler angles [yaw, pitch, roll], 
          "ids": Landmarks database ids,
          "landmarks": Landmarks (ref image), 
          "visible": Visibilities [0,1, ...] (1 == Visible)
          }
```

Finally, is worth to mention that the benchmark can be easily extended for other task by 
inheriting the class structure available in ```./eval/benchmark/metrics/metrics.py``` and 
developing a new task file like the available ones: landmarks and headpose.