I have a VPC in AWS with multiple public and private subnets in different availability zones.
I want to host my own DNS on this network to use single label hostnames. For example, I want to use mydatabase instead of mydatabase.company.com.
I deployed a VM in one subnet, and installed and configured dnsmasq with a CNAME to translate the hostnames.
I validated on the DNS server returned the expected response to dig mydatabase. I also validated the networking and dnsmasq running on the right port because the expected response is received on another client machine on the network for dig mydatabase @172.31.1.36 where 172.31.1.36 is the IP address of my DNS server.
I set the the DHCP Option Set on the VPC to one with options of domain-name-servers: 172.31.1.36, expecting that machines on this network will be sent to my DNS server instead of the Amazon provided DNS. resolvectl status on these machines (in either a public or private subnet) returns:
ubuntu@ip-172-31-1-58:~$ resolvectl status
Global
Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Link 2 (enX0)
Current Scopes: DNS
Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 172.31.1.36
DNS Servers: 172.31.1.36
However, dig mydatabase on these client machines returns no response. Logs on the DNS server record activity when, on a different machine, I run dig mydatabase@172.31.1.36 but doesn't record anything when I run dig mydatabase. The same is true for dig google.com@172.31.1.36 and dig google.com. Clearly, the client machines aren't using my DNS server for resolution.
I was expecting that when I updated the DHCP Option Set on the VPC that the network router would then forward all DNS queries on the network to my DNS server.
What have I missed?
Is there any way I can see which DNS server is answering my client machines when they query google.com? (as I've observed that mine is not)
What is the difference between the Global and Link 2 sections reported by resolvectl status?
Are my client machines preferring Global over Link 2 for some reason?
I have seen that there might be the option to set the DNS server on each client machine, involving installing resolvconf. I'd rather not do that, as I would need to do it on all machines deployed into the network. I would rather handle this at the network level.
My VPC has the Enable DNS resolution option set (as per AWS defaults). I tried disabling this without success.