I've periodically removed Cloudflare because of issues with reissuing SSL certs, Cloudflare being down, and other reasons, and haven't noticed any problems.
The biggest benefit I get from Cloudflare is blocking scraper robots, which I've just been too lazy to figure out how to do myself.
But have those webservers supported a small or medium-sized business?