# Accessing MedCAT entities' concept embeddings

**URL:** <https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182>\
**Category:** MedCAT\
**Created:** [April 14, 2023, 10:41am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182 "2023-04-14T10:41:16Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hideaki](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@Hideaki](https://discourse.cogstack.org/u/Hideaki)\
**Post date:** [April 14, 2023, 10:41am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/1 "2023-04-14T10:41:16Z")

</div>

Hello. Is there a way to access the embeddings of the concepts that have been linked to the CDB?

---

<div class="post-metadata">

**Author:** ![Hideaki](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@Hideaki](https://discourse.cogstack.org/u/Hideaki)\
**Post date:** [April 18, 2023, 9:48am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/2 "2023-04-18T09:48:43Z")

</div>

Dear @zeljko , is there a way to access the context embeddings of the entities linked?

---

<div class="post-metadata">

**Author:** ![zeljko](https://avatars.discourse-cdn.com/v4/letter/z/7ab992/32.png) [@zeljko](https://discourse.cogstack.org/u/zeljko)\
**Post date:** [April 18, 2023, 11:43am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/3 "2023-04-18T11:43:43Z")

</div>

Hi @Hideaki , unfortunately, that is not possible, it would consume too much memory for big runs so the context embeddings are just calculated but not stored anywhere during the Linking phase. You can of course access embeddings of concepts from the CDB.

---

<div class="post-metadata">

**Author:** ![Hideaki](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@Hideaki](https://discourse.cogstack.org/u/Hideaki)\
**Post date:** [April 19, 2023, 10:30am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/6 "2023-04-19T10:30:01Z")

</div>

Thank you, @zeljko. I’ve used the cat.cdb.cui2context\_vectors on a few CUIs. For most of them, I am able to access the four vector types. However, I get a key error with CUI 840539006 and 91637004. Not sure what I’ve done wrong

---

<div class="post-metadata">

**Author:** ![Hideaki](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@Hideaki](https://discourse.cogstack.org/u/Hideaki)\
**Post date:** [April 21, 2023, 8:40am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/7 "2023-04-21T08:40:38Z")

</div>

Also, may I please check, where does the original embedding for concepts in the CDB come from?

---

<div class="post-metadata">

**Author:** ![zeljko](https://avatars.discourse-cdn.com/v4/letter/z/7ab992/32.png) [@zeljko](https://discourse.cogstack.org/u/zeljko)\
**Post date:** [April 21, 2023, 4:47pm UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/8 "2023-04-21T16:47:13Z")

</div>

Hi @Hideaki,

re the key error: not all CUIs have embeddings, depends did the CUI receive any training.

The original embeddings for concepts in the CDB come from the unsupervised training, have a look at the MedCAT paper it explains the training procedure and how the Vocab (word embeddings) are used to make concept embeddings.

---

<div class="post-metadata">

**Author:** ![Hideaki](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@Hideaki](https://discourse.cogstack.org/u/Hideaki)\
**Post date:** [April 22, 2023, 6:42am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/9 "2023-04-22T06:42:23Z")

</div>

Thank you, @zeljko! We received a private model from a different Trust and so I used cat.multiprocessing to annotate then checked the embeddings of the concepts. I suppose those CUIs did not receive training from the original organisation.

---

<div class="post-metadata">

**Author:** ![anthony.shek](https://avatars.discourse-cdn.com/v4/letter/a/b782af/32.png) [@anthony.shek](https://discourse.cogstack.org/u/anthony.shek)\
**Post date:** [April 26, 2023, 9:13am UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/10 "2023-04-26T09:13:39Z")

</div>

@Hideaki just to double check if the concepts have recieved any training you can explore:

`cat.cdb.cui2count_train['<CUI OF INTREREST>']`

---

<div class="post-metadata">

**Author:** ![Jthteo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jthteo](https://discourse.cogstack.org/u/Jthteo)\
**Post date:** [November 13, 2023, 2:25pm UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/11 "2023-11-13T14:25:10Z")

</div>

@Hideaki thanks for joining the call. apologies, for delay as we are in usa this week. We can go into the many-to-1 map and context vectors next week when we are back

@mart.ratas @anthony.shek

---

<div class="post-metadata">

**Author:** ![Hideaki](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@Hideaki](https://discourse.cogstack.org/u/Hideaki)\
**Post date:** [November 13, 2023, 2:43pm UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/12 "2023-11-13T14:43:24Z")

</div>

Thank you, @Jthteo. What I’m interested to find out are:

1. Where the four types of vectors come from when we use cat.cdb.cui2context\_vectors[CUI]. My understanding from Equation 10 of the 2021 paper is that there can be many context vectors (Vcntx) but only one concept vector (Vconcept). The Public and King’s 1.4 models have 4 types of of these vectors and the King’s 1.2 model has 6.

2. Whether the vectors of cat.cdb.cui2context\_vectors map to preferred names in a one-one fashion? I found that there are duplicates of the preferred names when I run the below code for the Public model. There seems to be 29674 trained CUIs but 29314 unique preferred names.

---

<div class="post-metadata">

**Author:** ![anthony.shek](https://avatars.discourse-cdn.com/v4/letter/a/b782af/32.png) [@anthony.shek](https://discourse.cogstack.org/u/anthony.shek)\
**Post date:** [January 3, 2024, 12:15pm UTC](https://discourse.cogstack.org/t/accessing-medcat-entities-concept-embeddings/182/13 "2024-01-03T12:15:37Z")

</div>

Hi @Hideaki,

Although we spoke about this. Just to have the answer in the public domain.

As you have correctly pointed out, currently no single concept vector that is stored. Instead there are multiple different vectors which represent different contexts sizes. A single concept vector can be calculated from these and is therefore never stored. The parameters for the different concept context vectors are specified in the [medcat config file](https://github.com/CogStack/MedCAT/blob/22e22550ba0da762351212a21bb5203a58016861/medcat/config.py#L490).

1. **Context Vector Sizes:**

- `context_vector_sizes` : This is a dictionary specifying the sizes of different context vectors that will be calculated and used for linking. Each key represents a context type (‘xlong’, ‘long’, ‘medium’, ‘short’), and the corresponding value is the size of the vector associated with that context type.
- For example: ‘xlong’ context vectors have a context window size of 27, ‘long’ context vectors have a context window size of 18, and so on. Not to be confused with the size of the vectors.

1. **Context Vector Weights:**

- `context_vector_weights`: This is a dictionary specifying the weight of each vector in the similarity score. The weights are used when calculating the overall similarity score based on multiple context types. Each key corresponds to a context type, and the value is the weight assigned to that context type.
- The weights should add up to **1**. In this case, the ‘long’ and ‘medium’ context vectors contribute more to the overall similarity score, with weights of 0.4 each, while ‘xlong’ and ‘short’ contribute less with weights of 0.1 each.

If you want to combine multiple vectors into a single vector that represents them all, you can perform a weighted sum of the individual vectors, where each vector is multiplied by its corresponding weight.

```auto
# First initialise your cat object, then proceed with the following code:

# Retrieve the multiple different vectors from your cat object, here I will mention 4 explicitly. 
v_xlong = ... # Replace with the actual vector for 'xlong'
v_long = ... # Replace with the actual vector for 'long'
v_medium = ... # Replace with the actual vector for 'medium'
v_short = ... # Replace with the actual vector for 'short'

# Retrieve the weights
weights = cat.config.linking['context_vector_weights'][context_type]

# Combine vectors into a single vector
v_combined_unitvec = weights['xlong'] * unitvec(v_xlong) + weights['long'] * unitvec(v_long) + weights['medium'] * unitvec(v_medium) + weights['short'] * unitvec(v_short)

```

Here the `v_combined` is your single vector representation.

This is part of the process in how we then calculate the prediction/similarity of the target word/phrase to the concept vectors.

## Calculation of similarity:

The [similarity calculated](https://github.com/CogStack/MedCAT/blob/master/medcat/linking/vector_context_model.py#L125C19-L125C19) in this context is a weighted sum of cosine similarities between vectors associated with a given Clinical Unique Identifier (CUI) and input vectors for different context types. Cosine similarity is a measure that ranges from -1 to 1, where:

- 1 indicates perfect similarity.
- -1 indicates perfect dissimilarity.
- 0 indicates orthogonality (no similarity).

Given this range, the weighted sum of cosine similarities should also fall within the range of -1 to 1, and it shouldn’t exceed 1.