There are two parts to this setup:
1. On your registrar's site (such as Namecheap.com, GoDaddy.com, etc., the one you bought your domain from) set up the appropriate records (normally A and AAAA are enough, just A if you do not care about ipv6) to point to your host. You should have received the appropriate addresses in your VPS invitation. These records have to be set up or each subdomain.
2. On your server (Apache or nginx) set up virtual hosts (again, one for each subdomain). Let's Encrypt certificates have to be generated for each virtual host if you want the pages to be served over https. See some examples here. These virtual hosts will be served from the same IP, and Apache will use the request headers to decide which host to direct the request to.
Alex