Temporal information extraction using Medcat

Nehal · December 15, 2023, 6:52am

I am trying to get medcat to extract dates ( for e.g. 10.10.2010, 10-10-2010, 10/10/2010, 10th Oct 2010 etc). from unstructured text. Spacy itself extracts dates but ‘cat.get_entities’ filters out these dates. I tried adding the timexy module to the medcat pipeline

cat.pipe.nlp.add_pipe(“timexy”, config=config)
config = {
“kb_id_type”: “timex3”,
“label”: “concept”, # default: ‘time’
“overwrite”: False # default: False
}

But the results were not as expected. The original entities extraction itself got messed up.

Would like to know if there are any recommended approaches to get medcat extract dates ?

sangeetabose · December 19, 2023, 6:03am

Would be great to get a response on this… we are working on getting temporal information from unstructured text.

jkgenser · December 19, 2023, 2:37pm

I believe medcat stores the detected entities in ._.ents and then the non-overlapping ones in .ents.

You may need to add a pipe that runs before medcat, takes the normal spacy .ents which are the date time type, put those into ._.times or some other namespace, then be able to recover it later.

Otherwise it will get clobbered by the medspacy .ents which stores the CUI entities.

This line is where the linker resets the .ents attribute

github.com

CogStack/MedCAT/blob/master/medcat/linking/context_based_linker.py#L58


      
              else:
              """
              # Always train
              self.context_model.train(cui, entity, doc, negative=False)
              if add_negative and self.config.linking.negative_probability >= random.random():
                  self.context_model.train_using_negative_sampling(cui)
              self.train_counter[name] = self.train_counter.get(name, 0) + 1
          
          # Override
          def __call__(self, doc: Doc) -> Doc:
              # Reset main entities, will be recreated later  
              doc.ents = []  # type: ignore
              cnf_l = self.config.linking
              linked_entities = []
          
              if cnf_l.train:
                  # Run training
                  for entity in doc._.ents:
                      # Check does it have a detected name
                      if entity._.detected_name is not None:
                          name = entity._.detected_name

Here is where the cat is instantiated and components are added to the pipeline. Some of these are spacy components some of them are medcat specific:

github.com

CogStack/MedCAT/blob/master/medcat/cat.py#L105


      
                  # Take config from the cdb
                  self.config = cdb.config
              else:
                  # Take the new config and assign it to the CDB also
                  self.config = config
                  self.cdb.config = config
              self._meta_cats = meta_cats
              self._addl_ner = addl_ner if isinstance(addl_ner, list) else [addl_ner]
              self._create_pipeline(self.config)
          
          def _create_pipeline(self, config: Config):
              # Set log level
              logger.setLevel(config.general.log_level)
          
              # Build the pipeline
              self.pipe = Pipe(tokenizer=spacy_split_all, config=config)
              self.pipe.add_tagger(tagger=tag_skip_and_punct,
                                   name='skip_and_punct',
                                   additional_fields=['is_punct'])
          
              if self.vocab is not None:

Also I believe medcat disables spacy NER by default. So you may want to re-enable it and put it earlier in the pipeline then have your own custom component that reads from spacy doc.ents before the medcat NER+Linker runs/

Nehal · January 16, 2024, 6:26pm

Thank you @jkgenser for answering in such details with code, I am still investigating on it, will come back again for further query if have, thanks again

Topic		Replies	Views
Negative accuracy in annotation suggestion? MedCAT	7	233	July 3, 2023
Reenabling pipeline components MedCAT	6	149	August 18, 2022
MedCAT sentencisation and chunking MedCAT	1	162	December 23, 2022
Accessing MedCAT entities' concept embeddings MedCAT	10	267	January 3, 2024
Using non word2vec vectors (floret or word-piece) MedCAT	0	105	June 6, 2023

Temporal information extraction using Medcat

Related Topics