Asio link pro is a great piece of software. I think what comes way closer to a matrix based Voicemeeter and in some regard covers most features that ASIO Link Pro offers is
R3lay VirtualPatchBay.
From the feature set it is more or less what an matrix based unlimited Voicemeeter would be. It can support any number of KS, WASAPI and ASIO devices. It has a number of (Stereo only) virtual WDM devices. It supports network streams similar to VBAN (although via AES67). Also DAWs can use a virtual ASIO client driver to connect directly to it as well.
All of these devices, network streams, virtual WDM devices, DAWs and even VST plugins appear as individual endpoints in the matrix and you can route audio arbitrarily between any of their in- and output channels.
It is quite a powerful tool and probably unobtainable by us mortals. Their target customers are broadcasting companies and this piece of software is only one little cog in a whole suite of hardware and software they offer.
There is a trial version that can be downloaded though. It has a time limit of 15 minutes (and then has to be restarted), so i had a little play around with it.
As their backend they use pulseaudio. Compared to Voicemeeter it had way more issues with audio devices. I would throw some errors with some of my ASIO devices and a few WDM devices always had severe crackling, wouldn't work at all or would get lost after a few minutes. All of which work fine with Voicemeeter by the way.
Between any two endpoinds there is a buffer that the software automatically determines in samples, which can also be overridden manually. I couldn't really achieve low latencies without running into various audio issues. With Voicemeeter in that regard i also could achieve (measured) way lower latencies.
From the feature set and GUI it is probably very close what i imagine a matrix based Voicemeeter could work like.