MetaCAT Model Evaluation

When training a MedCAT model with some Meta-Annotations you get Precision/Recall/F1 scores outputted (generated by the medcat.metacat.MetaCAT.eval() method).

What is this evaluating - is it performance of the MedCAT model identifying concepts, or is it evaluating the performance of meta models choosing categories?

I’m trying to evaluate the performance of our meta models.

Answered my own question - if you have a metacat model object, and call .eval() on it with a filepath to a MedCATtrainer project export as the argument, it evaluates the one meta model.


Yes correct it needs a “gold standard” to compare its predictions against in order to produce metrics. Glad you found it