# Sentence embeddings from C

Compute sentence embeddings from a C program with the Kwker Runtime - a sentence-transformers model folder in, vectors out, no Python and no PyTorch.

Web page: https://kwker.io/docs/?p=hf-c-embed - every Kwker documentation page for AI assistants: https://kwker.io/llms.txt

The Kwker Runtime (`libkwker_rt`) runs KwkEncoder's models from any program that can call C. Give it a model folder,
such as a downloaded sentence-transformers model. It reads the tokenizer, the weights and the pooling settings, and turns
texts into embedding vectors: the same vectors sentence-transformers computes in Python.

It loads BERT, RoBERTa, XLM-RoBERTa, CamemBERT, DistilBERT, MPNet, ModernBERT, NomicBERT and EuroBERT models, and the
embedding models built on them, such as all-MiniLM-L6-v2, all-mpnet-base-v2, BGE, E5, GTE and nomic-embed-text-v1.5 - multilingual ones too, such as
multilingual-e5 and paraphrase-multilingual-MiniLM-L12-v2.

## What you need

- The library and its header, as in [Run a language model from C](https://kwker.io/docs/md/hf-c.md#what-you-need).
- A model folder with `config.json`, `tokenizer.json` and `model.safetensors`, for example a downloaded Hugging Face
  snapshot of `sentence-transformers/all-MiniLM-L6-v2`.

## The program

The program loads the tokenizer and the encoder, turns every text into token ids, embeds them all with one call and
prints how similar each text is to the first one (1 is identical in meaning, 0 unrelated).

```c
// Sentence embeddings with the Kwker Runtime from C: load a sentence-transformers model folder (weights and tokenizer),
// embed a few texts and print how similar each one is to the first. No Python, no PyTorch.
//
//   rt_embed <model folder> <text> <text> [<text> ...]
//
// The embeddings use the folder's own pooling and normalization, as sentence-transformers does.
//
// Build: cc rt_embed.c -I<kwker include dir> -L<lib dir> -lkwker_rt -lm -o rt_embed
#include <math.h>
#include <stdio.h>
#include <stdlib.h>

#include "kwker_rt.h"

int main(int argc, char** argv) {
    if (argc < 4) {
        fprintf(stderr, "usage: %s <model folder> <text> <text> [<text> ...]\n", argv[0]);
        return 2;
    }
    const char* dir = argv[1];
    const int64_t n = argc - 2;

    kwker_rt_tokenizer* tok = NULL;
    kwker_rt_encoder* enc = NULL;
    if (kwker_rt_tokenizer_load(dir, &tok) != 0) {
        fprintf(stderr, "tokenizer: %s\n", kwker_rt_tokenizer_error());
        return 1;
    }
    if (kwker_rt_encoder_load(dir, "float32", &enc) != 0) {
        fprintf(stderr, "encoder: %s\n", kwker_rt_last_error());
        return 1;
    }
    const int64_t hidden = kwker_rt_encoder_hidden(enc);

    // Token ids for every text, with the model's start and end tokens; rows padded to the longest one.
    int64_t* lens = malloc(sizeof(int64_t) * n);
    int64_t longest = 0;
    for (int64_t i = 0; i < n; i++) {
        kwker_rt_tokenizer_encode(tok, argv[2 + i], -1, 1, NULL, 0, &lens[i]);
        if (lens[i] > longest) longest = lens[i];
    }
    int64_t* ids = calloc((size_t)(n * longest), sizeof(int64_t));
    for (int64_t i = 0; i < n; i++) kwker_rt_tokenizer_encode(tok, argv[2 + i], -1, 1, ids + i * longest, longest, &lens[i]);

    // One call embeds them all: n rows of hidden floats.
    float* emb = malloc(sizeof(float) * (size_t)(n * hidden));
    if (kwker_rt_encoder_embed(enc, n, longest, ids, lens, emb, 0) != 0) {
        fprintf(stderr, "embed: %s\n", kwker_rt_last_error());
        return 1;
    }

    // Cosine similarity of each text to the first.
    for (int64_t i = 1; i < n; i++) {
        double dot = 0, a = 0, b = 0;
        for (int64_t j = 0; j < hidden; j++) {
            dot += (double)emb[j] * emb[i * hidden + j];
            a += (double)emb[j] * emb[j];
            b += (double)emb[i * hidden + j] * emb[i * hidden + j];
        }
        printf("%.4f  %s\n", dot / sqrt(a * b), argv[2 + i]);
    }

    kwker_rt_encoder_free(enc);
    kwker_rt_tokenizer_free(tok);
    free(emb);
    free(ids);
    free(lens);
    return 0;
}
```

Build it against the library, then run it on a model folder with a few texts:

```sh
cc rt_embed.c -I<include dir> -L<library dir> -lkwker_rt -lm -Wl,-rpath,<library dir> -o rt_embed
./rt_embed ~/models/all-MiniLM-L6-v2 "How do I bake bread?" "A recipe for sourdough loaves" "The stock market fell today"
```

It prints each text's similarity to the first:

```text
0.5845  A recipe for sourdough loaves
0.0645  The stock market fell today
```

The recipe is close in meaning to the question about bread; the stock market is not.

## The calls

| Call | What it does |
|---|---|
| `kwker_rt_encoder_load(dir, precision, &e)` | Load a model folder. `precision` is `"float32"`, `"bf16"` (faster, slightly less exact) or `"int8"` (faster still; check the results on your data). |
| `kwker_rt_encoder_embed(e, n, len, ids, lens, out, threads)` | Embed `n` texts given as rows of `len` token ids; row `i` holds `lens[i]` real tokens, the rest is ignored. Writes `n` rows of `hidden` floats. `threads` 0 uses every core. |
| `kwker_rt_encoder_forward(e, n, len, ids, lens, types, out, threads)` | The model's last hidden state instead: `n` x `len` rows of `hidden` floats, rows past a text's length zero. `types` is NULL or the token type ids. |
| `kwker_rt_encoder_hidden(e)` | The length of one embedding. |
| `kwker_rt_encoder_free(e)` | Release the encoder. |
| `kwker_rt_last_error()` | The reason the last encoder call on this thread returned -1. |

The tokenizer calls are listed in [Run a language model from C](https://kwker.io/docs/md/hf-c.md#the-calls).

## Notes

- The pooling follows the folder's sentence-transformers settings (`modules.json` and `1_Pooling/config.json`): mean,
  first token (CLS), max or last token, then normalization when the folder has a Normalize step. A folder without
  `modules.json` gets mean pooling. Other steps, such as an extra Dense layer, are refused by `kwker_rt_encoder_embed`
  with a message.
- `"float32"` keeps the checkpoint's weights exactly; `"bf16"` and `"int8"` match KwkEncoder's results in Python.
- On CPUs with AMX (Intel Xeon 4th generation, Sapphire Rapids, and later) `"bf16"` and `"int8"` run on the AMX units,
  as KwkEncoder does. `KWKER_AMX=0` turns that off.
- ModernBERT, NomicBERT and EuroBERT models load in `"float32"` or `"bf16"`.

## Related

- [Run a language model from C](https://kwker.io/docs/md/hf-c.md): the decoder and the tokenizer from C.
- [ONNX models](https://kwker.io/docs/md/onnxruntime.md): the same encoders inside ONNX Runtime.