# MetaCAT Model Evaluation

**URL:** <https://discourse.cogstack.org/t/metacat-model-evaluation/135>\
**Category:** MedCAT\
**Created:** [January 9, 2023, 5:41pm UTC](https://discourse.cogstack.org/t/metacat-model-evaluation/135 "2023-01-09T17:41:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jamesbrandreth](https://avatars.discourse-cdn.com/v4/letter/j/9e8a1a/32.png) [@jamesbrandreth](https://discourse.cogstack.org/u/jamesbrandreth)\
**Post date:** [January 9, 2023, 5:41pm UTC](https://discourse.cogstack.org/t/metacat-model-evaluation/135/1 "2023-01-09T17:41:36Z")

</div>

When training a MedCAT model with some Meta-Annotations you get Precision/Recall/F1 scores outputted (generated by the `medcat.metacat.MetaCAT.eval()` method).

What is this evaluating - is it performance of the MedCAT model identifying concepts, or is it evaluating the performance of meta models choosing categories?

I’m trying to evaluate the performance of our meta models.

---

<div class="post-metadata">

**Author:** ![jamesbrandreth](https://avatars.discourse-cdn.com/v4/letter/j/9e8a1a/32.png) [@jamesbrandreth](https://discourse.cogstack.org/u/jamesbrandreth)\
**Post date:** [January 17, 2023, 10:06am UTC](https://discourse.cogstack.org/t/metacat-model-evaluation/135/2 "2023-01-17T10:06:50Z")

</div>

Answered my own question - if you have a `metacat` model object, and call `.eval()` on it with a filepath to a MedCATtrainer project export as the argument, it evaluates the one meta model.

`meta_model.eval('path/to/annotation/project/export.json')`

---

<div class="post-metadata">

**Author:** ![anthony.shek](https://avatars.discourse-cdn.com/v4/letter/a/b782af/32.png) [@anthony.shek](https://discourse.cogstack.org/u/anthony.shek)\
**Post date:** [January 18, 2023, 1:07pm UTC](https://discourse.cogstack.org/t/metacat-model-evaluation/135/3 "2023-01-18T13:07:10Z")

</div>

Hey!  
Yes correct it needs a “gold standard” to compare its predictions against in order to produce metrics. Glad you found it