After installing Docker package docker.io from official Ubuntu repo and then removing it (to install the latest version from Docker's own repo instead), I found that my server could not resolve hostnames anymore (back then I didn't know yet it was caused by Docker though). After reboot, /etc/resolv.conf was blank, although files in /etc/network/interfaces.d/* files were intact and contained the correct DNS servers.
/etc/resolv.conf nowadays is actually a symlink to a file managed by resolvconf, which is being passed DNS server list by ifup on reboot.
What happened is, docker.io package depends on ubuntu-fan package which installs /etc/network/if-up.d/ubuntu-fan script to be called by ifup (and other scripts for ifdown, etc.). When I removed Docker and then ran apt autoremove, it removed ubuntu-fan package, but left the scripts. On next reboot, ifup tried to execute them, but the actual fanctl executable was no longer there, ifup was half-failing and not calling resolvconf, although the interface was up (I'm not sure why, as /etc/network/if-up.d/000resolvconf should be called first and not affected by later scripts).
The solution is to run apt-get purge ubuntu-fan which will delete residual configuration files, and restart the server.