Kwker

Image models from C

The Kwker Runtime (libkwker_rt) runs KwkCNN's image models from any program that can call C. Save the model once in Python with kwker.cnn.save(); a C program then loads the file and runs it, with the same outputs as KwkCNN in Python.

It runs the torchvision families KwkCNN runs - ResNet, ResNeXt, MobileNet, EfficientNet, RegNet, ConvNeXt, DenseNet, GoogLeNet, ShuffleNet and others - in float32 or int8.

What you need

Save the model

PythonRuns on your machine.
import torchvision
import kwker.cnn

model = torchvision.models.mobilenet_v2(weights="DEFAULT").eval()
kwker.cnn.save(model, "mobilenet_v2.kwk")

The file holds the model's operators and weights for 224 x 224 images. int8=True saves the faster int8 model, calibrated on calib images as KwkCNN(..., int8=True) is.

The program

The program loads the model file, runs one image and prints the five highest-scoring classes.

CInclude kwker.h; link with -lkwker_c.
// Classify an image from C with the Kwker Runtime: load a model file that kwker.cnn.save() wrote in Python, run it on one
// image and print the five highest-scoring classes. No Python, no PyTorch.
//
//   rt_classify <model file> [<image file>]
//
// The image file holds the preprocessed image as raw float32 values, [channels][height][width] - for an ImageNet model,
// resized and normalized with the model's mean and standard deviation. Without one, the program runs a gray image.
//
// Build: cc rt_classify.c -I<kwker include dir> -L<lib dir> -lkwker_rt -o rt_classify
#include <stdio.h>
#include <stdlib.h>

#include "kwker_rt.h"

int main(int argc, char** argv) {
    if (argc != 2 && argc != 3) {
        fprintf(stderr, "usage: %s <model file> [<image file>]\n", argv[0]);
        return 2;
    }
    kwker_rt_cnn* m = NULL;
    if (kwker_rt_cnn_load(argv[1], &m) != 0) {
        fprintf(stderr, "load failed: %s\n", kwker_rt_last_error());
        return 1;
    }

    // The image: [channels][height][width] floats, the size the model was saved for.
    int64_t shape[3];
    kwker_rt_cnn_input_shape(m, shape);
    const int64_t n_in = shape[0] * shape[1] * shape[2];
    float* image = calloc((size_t)n_in, sizeof(float));
    if (argc == 3) {
        FILE* f = fopen(argv[2], "rb");
        if (!f || fread(image, sizeof(float), (size_t)n_in, f) != (size_t)n_in) {
            fprintf(stderr, "%s: expected %lld float32 values (%lld x %lld x %lld)\n", argv[2], (long long)n_in, (long long)shape[0],
                    (long long)shape[1], (long long)shape[2]);
            return 1;
        }
        fclose(f);
    }

    // One image in, one row of scores out.
    const int64_t n_out = kwker_rt_cnn_output_size(m);
    float* scores = malloc(sizeof(float) * (size_t)n_out);
    if (kwker_rt_cnn_run(m, 1, image, scores, 0) != 0) {
        fprintf(stderr, "run failed: %s\n", kwker_rt_last_error());
        return 1;
    }

    // The five highest scores and their class numbers.
    for (int k = 0; k < 5 && k < n_out; k++) {
        int64_t best = -1;
        for (int64_t i = 0; i < n_out; i++)
            if (scores[i] == scores[i] && (best < 0 || scores[i] > scores[best])) best = i;
        printf("class %lld: %.4f\n", (long long)best, scores[best]);
        scores[best] = 0.0f / 0.0f;  // (NaN: skipped from now on)
    }

    kwker_rt_cnn_free(m);
    free(scores);
    free(image);
    return 0;
}

Build it against the library. Prepare the image in Python the way the model expects (resized to 224 x 224, normalized with the model's mean and standard deviation) and write it as raw float32 values, then run the program:

ShellOn your machine.
cc rt_classify.c -I<include dir> -L<library dir> -lkwker_rt -Wl,-rpath,<library dir> -o rt_classify
./rt_classify mobilenet_v2.kwk photo.f32

On a photo of a truck it prints:

Text
class 867: 5.7185
class 675: 5.5233
class 569: 4.2484
class 660: 4.1655
class 856: 3.2134

Class 867 is "trailer truck" in the ImageNet class list. The scores are the model's logits, the same numbers KwkCNN and the PyTorch model give.

The calls

Call What it does
kwker_rt_cnn_load(path, &m) Load a model file that kwker.cnn.save() wrote.
kwker_rt_cnn_input_shape(m, shape) The image shape the model was saved for: [channels, height, width].
kwker_rt_cnn_output_size(m) The number of floats one image's output holds (1000 for an ImageNet classifier).
kwker_rt_cnn_run(m, n, images, out, threads) Run n images, [n][channels][height][width] float32, into n rows of outputs. threads 0 uses every core.
kwker_rt_cnn_free(m) Release the model.
kwker_rt_last_error() The reason the last call on this thread returned -1.

Notes