Hello everyone
We want to move some of the systems for a co-location into AWS. IPA systems are some of our candidate servers.
I have attempted to get this working by setting up a replica server in the cloud and attempting to setup replication - over VPN - and its not working. This is due to DNS issue on AWS being biased toward AWS DNS. If I use nmap, it verify I can reach port 53 (TCP and UDP) on the co-location from AWS, but if I do a dig against existing DNS, it doesn't seem to resolve.
Have anyone gone through the exercise recently and managed to figure how to work around this limitation? Would be grateful if someone can share how the worked around this problem.
Regards, William
I run a 4-node IPA cluster on AWS spanning a few global regions and tied into a particularly complex AD forest -- never had the DNS issues you mention but I've never had to talk to IPA on-prem either. And our setup is strange as we could never get the global AD admins to make DNS entries for us among other issues so we ended up choosing a totally new TLD domain name to run IPA on and bind our servers against; this works fine except we can't leverage kerberos based features because our realm is different from our domain. All we needed was RBAC login using AD usernames and passwords though so our needs are met.
For DNS on AWS though ...
- You can override or alter DHCP level nameserver assignments at a VPC-level if you want to adjust DHCP option sets. This is a big decision as it is VPC-wide in scope but works well to push out "global" DNS and nameserver preferences
- If VPC level changes are too much then it's pretty easy on an EC2 instance to override or alter DNS resolution, nameservers and search order in ways that persist beyond reboot. Specific method depends OS you are using and if you want to this on the CLI via ansible or manual operations or if you want to do this at the cloud-init "just booting up" level
I did have a nasty DNS issue in the past with an old school client that believed all DNS should be served out of AD and that pre-creating reverse PTR records was "nonsense" and "not needed" which broke a ton of AWS HPC and EMR use cases and other projects where software does a reverse lookup on it's IP to learn more about its self. In that setting we built our own custom DNS server based on DNSMasq -- DNSMasq is perfect because you can construct a totally bespoke DNS environment combining delegated nameservers, local name servers and you can even have DNSMasq read DNS entries from an /etc/hosts style file so you don't have to maintain a complex bind-format zone file
I don't know the exact scope of your situation but this seems like a case for overriding the DNS settings on the EC2 hosts running IPA to talk to your colo nameservers rather than the AWS designated ones that show up via your DHCP Option Set
Chris
William Muriithi via FreeIPA-users wrote on 5/27/20 10:30 AM:
Hello everyone
We want to move some of the systems for a co-location into AWS. IPA systems are some of our candidate servers.
I have attempted to get this working by setting up a replica server in the cloud and attempting to setup replication - over VPN - and its not working. This is due to DNS issue on AWS being biased toward AWS DNS. If I use nmap, it verify I can reach port 53 (TCP and UDP) on the co-location from AWS, but if I do a dig against existing DNS, it doesn't seem to resolve.
Have anyone gone through the exercise recently and managed to figure how to work around this limitation? Would be grateful if someone can share how the worked around this problem.
Regards, William _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Hello Chris,
I run a 4-node IPA cluster on AWS spanning a few global regions and tied into a particularly complex AD forest -- never had the DNS issues you mention but I've never had to talk to IPA on-prem either.
Okay, may be I will have to investigate this.
And our setup is strange as we could never get the global AD admins to make DNS entries for us among other issues so we ended up choosing a totally new TLD domain name to run IPA on and bind our servers against; this works fine except we can't leverage kerberos based features because our realm is different from our domain.
You can enrol IPA client through through Kerberos, this correct?
- If VPC level changes are too much then it's pretty easy on an EC2
instance to override or alter DNS resolution, nameservers and search order in ways that persist beyond reboot. Specific method depends OS you are using and if you want to this on the CLI via ansible or manual operations or if you want to do this at the cloud-init "just booting up"
I am dropping a file /etc/NetworkManager/conf.d/90-dns-none.conf using ansible and then adding IPA server IP on /etc/resolve.conf
I don't know the exact scope of your situation but this seems like a case for overriding the DNS settings on the EC2 hosts running IPA to talk to your colo nameservers rather than the AWS designated ones that show up via your DHCP Option Set
I wish its this, but I dont think so. If it was this, wouldn't doing dig @192.168.30.8 neptune.external.example.com work at least? The host 192.168.30.8 being in the office? How is your VPC? Do you have public and private and NAT between? Or just a flat public? I went with the later as I assumed IPA don't like working over NAT.
Thanks again for the feedback.
Regards, William
Replies inline ...
And our setup is strange as we could never get the global AD admins to make DNS entries for us among other issues so we ended up choosing a totally new TLD domain name to run IPA on and bind our servers against; this works fine except we can't leverage kerberos based features because our realm is different from our domain.
You can enrol IPA client through through Kerberos, this correct?
Yep! Our oddball IPA setup looks like this, due to a combination of senior IT leadership being frightened of the cloud and global active directory domain admins doing their usual global admin thing and refusing to even take meetings with mortals. So we had to self-support and only got a grudging 1-way trust set with the domain after escalating to very high levels. It works though.
AD Domain (complex multi-child forest) == company-x.com with child domains at nafta.company-x.com, apac.company-x.com etc. etc. Our FreeIPA domain name is == company-x-ipa.com
So our REALM is COMPANY-X-IPA and that is where we bind our client machines to when enrolling but we set the local domain name and hostname to "company-x.com" when we enroll the host
This works well but from what I've been told we lose the ability to do things like kerberized authentication for filessytem mounts etc. -- since all we really needed was RBAC managed SSH login with a small smattering of custom sudo roles and a bunch of local SSH key storage it works fine for us.
I wish its this, but I dont think so. If it was this, wouldn't doing dig @192.168.30.8 neptune.external.example.com work at least? The host 192.168.30.8 being in the office? How is your VPC? Do you have public and private and NAT between? Or just a flat public? I went with the later as I assumed IPA don't like working over NAT.
Yeah most of this DNS is on-premise; again because of a corporate desire to use AD for DNS. We basically point all our nameservers at the on-prem domain controllers and don't use FreeIPA for DNS records really much at all (even the _SVR records that enable FreeIPA auto-discovery) -- but no network, DNS query or technical issues at all with DNS being outside of AWS -- works fine for us even DNS queries made on the IPA masters. For rare cases where that did not work (HPC use cases) we did the DNSMasq thing so we could inject custom reverse DNS responses while still delegating other queries back to the domain controllers.
Our VPC is direct connected back to the corporate WAN; no NAT on the private AWS subnets so it would be flat/private. Lots of security controls and firewalls doing SSL intercept and decrypt at edges that blocked
Thanks again for the feedback.
Regards, William
William,
Once I had to setup an IPA master and a few clients on AWS, and have issues with its DNS, since the external name do not match the internal name, hence, clients could not enroll (which I believe is similar to what you are facing with replicas).
What I did, using Ansible (and ansible-freeipa), was to retrieve the server name with `dig -x`, and using this name for the master FQDN.
I'm not sure it is the same issue you are having, but looks similar.
Rafael
On Wed, May 27, 2020 at 11:39 AM William Muriithi via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Hello everyone
We want to move some of the systems for a co-location into AWS. IPA systems are some of our candidate servers.
I have attempted to get this working by setting up a replica server in the cloud and attempting to setup replication - over VPN - and its not working. This is due to DNS issue on AWS being biased toward AWS DNS. If I use nmap, it verify I can reach port 53 (TCP and UDP) on the co-location from AWS, but if I do a dig against existing DNS, it doesn't seem to resolve.
Have anyone gone through the exercise recently and managed to figure how to work around this limitation? Would be grateful if someone can share how the worked around this problem.
Regards, William _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Hi Rafael,
Once I had to setup an IPA master and a few clients on AWS, and have issues with its DNS, since the external name do not match the internal name, hence, clients could not enroll (which I believe is similar to what you are facing with replicas).
What I did, using Ansible (and ansible-freeipa), was to retrieve the server name with `dig -x`, and using this name for the master FQDN.
Possibly this is the case. I will try to create a fresh IPA as the realm will change and will update the list on the result. Then figure out how to migrate the data manually.
I'm not sure it is the same issue you are having, but looks similar.
Something different. For example, I setup an IPA client, with the server on the co-location - over VPN. The ipa-client enrol fine, and I can even ssh in. Cool, 4 days later, no change, we can't authenticate. When debugging, all looks like a DNS issue, but oddly, it had worked a couple of day earlier. Was assuming AWS infrastructure are hijacking DNS traffic as the rest - ssh using IP address still works from AWS to co-location over VPN.
Thanks again Rafael. William
freeipa-users@lists.fedorahosted.org
-
Chris Dagdigian -
Rafael Jeffman -
William Muriithi