This is a follow-up to my previous article on Multi-GPU (Nvidia) P2P capabilities and debugging tips, which you can read here: https://morgangiraud.medium.com/multi-gpu-nvidia-p2p-capabilities-and-debugging-tips-fb7597b4e2b5
Be sure to check it out to understand the context.
TL;DR
If you have multi-GPU builds with Nvidia consumer graphics cards in the 40** series. The tinygrad Patch allows you to enable some nice hacky P22P capabilities that are compatible with all ML frameworks(TF, Pytorch, etc.)
Tinygrad Patched driver
The Tinygrad patched driver codebase can be found here: https://github.com/tinygrad/open-gpu-kernel-modules
You can check the different commits added geohot & friends to hack their way through P2P capabilities.
As I had to upgrade my server today, I also had to properly reinstall the patched version of the nvidia open driver version 550. As I ran into some troubles, I wrote a tiny GIST to help one get through the journey of overriding a driver module on Ubuntu.
All, right! less talking, more doing!
Following is a gist containing all you need to know to install this patched driver!
Cheers!
