I implemented multitheading for miRack audio engine back in 2018 when it was a project targeting single-board computers because with slower CPUs it's essentially a requirement to be able to run any decently-sized patches. The implementation (available here and is used in miRack app with minor modifications) is based on idea of having arrays for module input and output values, and a lock-free concurrent work queue implementation by Cameron Desrochers.

For each rendering cycle, say 512 samples (steps), all rack modules are pushed into to a work queue, also with start and end steps to process (so initially that is 1 to 512), then worker threads are woken up. The worker threads dequeue modules from the work queue and check that values for the step being processed are present for all module inputs (for disconnected inputs this is always true). If all values are available, the module is processed and output values are saved straight in input arrays of modules connected to each output for the next step number. The process continues until the end step is reached or until any of the input values are not available, in which case the module is pushed back into the work queue (updating the start step if needed) and another module is pulled from the work queue. Once there is no more modules in the work queue, the workers pause, and the rendering cycle completes. This implementation ensures that workers don't wait unless they have to.

Until recently I never looked at the multithreading implementation that later appeared in VCV Rack (available here), but wanted to run some benchmarks at some point.

During normal opearation, VCV Rack implementation uses spinlocks only. For each step in a rendering cycle, workers process only that single step for each module. Once there are no more modules for a worker to pick up, it will spinwait until all workers have finished, then values are transferred from outputs to connected inputs, and the workers are woken up to process the next single step. This implementation causes the workers to wait a lot instead of possibly processing next steps for some modules.

Now to the benchmark. I used the current miRack code and the latest VCV Rack code. All graphics rendering was disabled, as well as audio output. For VCV Rack, updating port lights was also disabled - it involves a lot of computations that substantially affect the results while not being related to audio processing.

The audio engines were told to process 1024 samples (steps) as fast as they can, and it was repeated 1000 times for a single thread then for 2, 3, and 4 worker threads. The tests were performed on a CPU with 4 physical cores. The following patches (by VCV Rack Ideas were used):

Patch 1

Patch 2

1st Patch Results

2nd Patch Results

"%" column shows time difference to the single-threaded case, and "Ideal %" shows the best theoretically achievable improvement of N times for N threads.

Also I should note that initially it was about comparing multithreaded speed increase, not absolute values (at least because miRack and VCV Rack use different versions of some of the patch modules), but absolute values turned out to be quite interesting as well. As I mentioned above, port lights update code adds about another second to VCV Rack results.

Great news! I've managed to implement loading of closed-source VCVRack plugins into miRack. This means it now makes sense to build packages for desktop operating systems. And soon you will be able to enjoy all the benefits of miRack, including lower CPU usage, more responsive UI and multithreaded processing - and still use all the same plugins you have, including commercial ones you purchased.

Of course this does not affect miRack running on ARM boards - only open-source plugins can be used in that case because they need to be compiled for ARM in first place.

Here is a windowRun() function which is an event handling/rendering loop. It runs at VSync rate (if it's supported/enabled or at 90 FPS otherwise). Let's assume it's 60 FPS. Each frame, cursorPosCallback() is called. That functions doesn't check whether the cursor position has actually changed or not. It does number of things but we're now interested in this line where onDragMove() is called for a gDraggedWidget if it's not NULL. gDraggedWidget is set here whenever you press left mouse button over a widget. Suppose, you pressed it over a module background (not a control), then requestModuleBoxNearest() will be called here to find a new position for the module so that it doesn't overlap other modules.

So far we see that some function is being called even when it's not really neccessary - that's bad but not the end of the world, right? Then now on to the interesting part. I'll provide full code of this function here:

bool RackWidget::requestModuleBoxNearest(ModuleWidget *m, Rect box) {
    // Create possible positions
    int x0 = roundf(box.pos.x / RACK_GRID_WIDTH);
    int y0 = roundf(box.pos.y / RACK_GRID_HEIGHT);
    std::vector positions;
    for (int y = max(0, y0 - 8); y < y0 + 8; y++) {
        for (int x = max(0, x0 - 400); x < x0 + 400; x++) {
            positions.push_back(Vec(x * RACK_GRID_WIDTH, y * RACK_GRID_HEIGHT));
        }
    }

    // Sort possible positions by distance to the requested position
    std::sort(positions.begin(), positions.end(), [box](Vec a, Vec b) {
        return a.minus(box.pos).norm() < b.minus(box.pos).norm();
    });

    // Find a position that does not collide
    for (Vec position : positions) {
        Rect newBox = box;
        newBox.pos = position;
        if (requestModuleBox(m, newBox))
            return true;
    }
    return false;
}

Isn't this wonderful? Create an array of 12800 vectors, then sort it, computing more than 100k square roots (not counting other operations), then iterate over these vectors again until we find a non-overlapping position (requestModuleBox() will iterate over all the modules too each time) - and all this at 60 FPS when you just pressed a mouse button and didn not even move the cursor (and of course it can be done much faster and simpler when you do move)!

Of course, this is not the only place with such... um... coding style. On the bright side, now I know why modules are moving not quite as smooth as desired (on Tinker Board this mess drops FPS to less than 20 when dragging a module).

And even better news is that the overall low FPS I was experiencing on Tinker Board can be fixed by just using full-screen window. There's a bug in GPU driver, Xorg or somewhere. Anyway, I'll just default to fullscreen on ARM (Raspberry Pi doesn't have this problem but it won't hurt anyway) and won't have to work on running without an X server for now.

miRack - an optimised fork of VCVRack primarily targeting Raspberry Pi, ASUS Tinker Board and similar hardware. But will keep your desktop cooler too.