Ever considered DietPi?!
DietPi is an extremely lightweight Debian OS, highly optimised for minimal CPU and RAM resource usage [...]
You can install either LXDE, LXQt, MATE, Xfce or GNUstep as a desktop. Maybe Chromium in Kiosk mode could be suitable for you too?!
They provide ready-to-go images for a bunch of different hypervisors.
Did you met his SO?!