You can keep both, you'll need it if you want to make changes. My suggestion would be to put rules in .htaccess to restrict who can reach it. If it's blocked for everyone except you, it will produce far less load than a WP site that is seeing lots of traffic.