I am using medcat snomed model. I checked programs memory before loading the model (1063.73 MB), after loading the model (5773.49MB), after inferring 1st batch of input ( 5922.69 MB) and after 2nd batch of input(6054.17 MB). The memory is increasing. Normally, i work with deep learning frameworks so i can address this problem. I am using medcat library to load and infer, and i cant find anything in documentation that is helpful to me.
Hi. Thank you for your question.
I’ve been looking into this potential issue quite a bit as of lately.
Most of my observations so far were regarding training, however.
But now before replying I also looked at little closer on inference. In the example I ran*, the overall memory usage (after GC) rose from 12GB to 12.8GB in around 2h (in fact, it reached 12.75GB within 30 minutes - after the first GC run). So an increase of only around 6.7% in this case.
Now, of course, before garbage is collected, the memory usage does increase (in my example quite consistently up to 14GB). But that shouldn’t really be surprising.
So all in all, I can see a relatively small increase in memory usage during inference, but it seems to top out relatively quickly.
There could be several reasons for this small memory footprint increase:
- It could be related to internal caching of some dependencies
- It could be related to dynamic imports within some dependencies
But all in all, I do not believe this is a memory leak. Not in the traditional sense at least.
* Was using an internal Snomed model with medcat==1.12.0, training on MIMIC-IV, and using an M1 MacBook.