DNS: Everything You've Ever Needed to Know! (28)
The Domain Name System (DNS) translates human-readable addresses like stmforum.com, to actual server locations via their IP address.
As a non-techy in the IM world the abbreviation DNS is probably synonymous with mindfuck and #noidea.
But, when you get into running your own VPS or dedicated servers, and even when you do something as simple as buying a domain, you need to deal with it.
Furthermore, DNS configuration is easy to mess up.
In this tutorial, I’m going to teach you the basics of what the DNS involves, how it works and how to go all the way from buying a domain name to configuring your DNS nameservers for your server, CNAMEing domains and all that jazz.
Hopefully this might clarify some things for more experienced folk as well. A video will follow showing how to buy and setup your domain name from start to finish with NameCheap and will plug that in in a bit when I finish all this.
This tutorial will even introduce you to a snazzy platform that we will use in the near future in a more advanced tutorial. Wicked!
If you just want to know how to buy a domain and set up your DNS configuration, skip this stuff below. But if you ask a question that I’ve answered in it, I’ll ban you. Just kidding. But I will use ALL CAPS @ YOU.
The Basics
Computers facing the outward internet address each other using something called an IP address.
However, humans use the Roman alphabet (generally and at least in browsers!).
Result? We need some middle-man system to translate a human readable address – a domain name – to an IP address that points to some computer somewhere.
Thus the domain name system and its importance.
Let’s first outline several parts that make up this system:
1. A registrar – e.g. GoDaddy, NameCheap. You register a domain name here.
2. A DNS server a.k.a. resolver – this is just a piece of software on some computer that responds to DNS queries, e.g. please take me to wherever stmforum.com is meant to point!
These are the points where caching occurs, and there will always be multiple resolvers involved – in your browser, your OS, and at your ISP.
Caching is where something saves a result for later use to avoid having to calculate or look up that result again. It is a way of saving time and effort. Caching generally occur for a set period of time in systems.
3. Nameservers – these are the libraries of the DNS world that hold onto records and provide the answers.
4. Records – these are the pieces of information that nameservers hold on to, and the things that get cached. Think of them as contact details for a person like you have on a smartphone.
There are several types of pertinent records:
A records - These point to server IP addresses.
CNAME records - These point to other domain names and can be used for redirection and domain masking.
NS records - These restate the nameservers for a domain and are not so important for you!
MX records - These point to the server that will handle email traffic whatever@yourdomain.com
Now, let’s get into the basics of how the DNS works.
This process is partially summarised in this diagram: http://i.imgur.com/6BgozJA.jpg
But, I’ll present an in-depth diagram below that I whipped up so that we can cover everything.
Now, I know this probably (definitely) looks equally awe-inspiring and frightening for non-techies.
Holy shit Zeno you crazy!
But let’s dissect it and highlight several key things that WILL be useful things to know in your AM endeavours.
Step 1
You enter a URL into your browser and hit enter. At this point DNS lookups happen in your browsers cache and an OS cache.
Sometimes, this cache is holding on to values which are no longer useful – e.g. if 5 minutes ago you changed a DNS record to point somewhere else.
These cached values get stored for X amount of time and have a specific expiry date, after which they won’t get used anymore. The length of this caching is determined by something called a time-to-live (TTL), and you set this for all records (in seconds).
Think of it like putting a countdown timer on all records. The moment some cache somewhere stores it, the timer starts, and once it hits zero it has to look it up again.
Caching is important because it prevents clients from having to lookup the same answer time in time out – and you can imagine if you have 100,000 visits a second to youtube.com, doing a DNS lookup that takes 50 ms every time would be a lot of wasted time, no?
Tip on TTL: if you are using a domain and playing around with DNS entries, set the TTL as low as possible e.g. 60 seconds. This will make changes propagate everywhere as fast as possible. When you are no longer playing, set the TTL to e.g. 3600 or 86400 (an hour or a day) – this will have a small performance impact on your traffic cf. having it set to 60 s.
Clearing caches
This is useful to know.
Sometimes you can’t access sites and it can be DNS-related. For example, a site changes their DNS records to load-balance traffic to a backup server.
However your browser or computer is still caching the old values and preventing you from getting to the new site location.
No problemo!
To clear browser DNS cache in Firefox:
Go to about:config in the address bar, accept the message.
Search for dnscache in the input box.
Modify network.dnsCacheExpiration to 0
If this key doesn’t exist, right click in the white space and hit new > integer, input “network.dnsCacheExpiration” as the name and 0 as the value.
Set this and your cache will expire. You can then reset firefox to be sure. If you want to turn DNS caching back on, set this back to 3600 or reset it. I have it permanently set at 0.
To clear browser DNS cache in Chrome:
Go to chrome://net-internals/#dns in the address bar.
Hit clear host cache. Done.
Now this will clear the browser cache – but note we also have an operating system cache as well!
To clear this in Windows, hit windows key + R (or manually open a run dialog), type cmd, run that. Then type “ipconfig /flushdns” and hit enter > all done. You should close all browsers before you do this.
Same goes for Macs – close all browsers, then open terminal and enter “dscacheutil –flushcache”. If you are on an older version of OS X you may need to do “lookupd –flushcache” instead.
You should get used to doing this routinely if you are messing with DNS settings for your sites, modifying them, or if sites you visit have gone down for no apparent reason.
Tip: if you think a site is down, google “is it down” or “down or just me” – there are plenty of online checkers that will tell you if it’s just you. If it is, your first port of call should be a browser and OS DNS flush.
Now, aside from your router (no instructions for this, will be too router specific and not so likely to cause issues), your ISP will cache DNS entries.
There is nothing you can do to flush those.
However, we can use other DNS servers and this is very common – it happens when we are using a VPN service.
When you connect to a VPN and create a private tunnel, you often use the VPN’s DNS servers – not your ISPs.
If a site is down, consider trying to access it via a VPN instead!
One caveat: quite often if you access a site and then disconnect > try to reload the site with your browser on your native connection, you can get timeouts and ‘could not find’ errors.
This can be due to the DNS caching going on, so if you encounter this, flush your browser cache and then close it > flush your OS cache > cross fingers. Sometimes it’s easier to just reconnect to the VPN or go make a coffee.
Learned anything yet?
Step 2
Your ISPs DNS resolver has to try and find out where stmforum.com points.
The first thing it does is query something called a root nameserver. There are 13 of these and they are globally distributed. One is even in an unknown location! Welcome to the super-secret underworld, the DNS darknet.
Anyway, this root nameserver responds with an address for the nameserver for your TLD – e.g. ‘.com’. This is called a recursive lookup because we continually dig deeper with our questions until we find the right answer.
Step 3
We then bounce to the .com nameserver, and it tells us what the authoritative nameservers are for stmforum.com.
You know how at a registrar you have to set some nameserver values if you want to use your own? This is where those details get pushed to, and these are the answers Mr .com provides to your ISP.
Because we have to push these details to a .com nameserver and propagate them around the world, changing your nameservers at your registrar takes a while (e.g. up to 24 hours), but it’s not something you’d change often.
Step 4
We now bounce to the authoritative nameserver that we set at the registrar, in this case ns1.nameserver.com. This nameserver we would generally have hosted with a service like Amazon S3 or Rackspace Cloud DNS since they are very cheap.
Soon I will show you how to use NSOne, a really cool DNS platform.
Moving on, it is at this authoritative nameserver that we mess around with our DNS records – the bits of information that tell us what server the domain should point to as well as subdomains, email service records, etc.
We will discuss the different types of records later. For now, just know that it is an “A-record” that points to an IP address of a server.
So, our friendly nameserver looks up stmforum.com and replies with “go to this IP address: 123.123.123.123”.
Perfect, we have a final answer!
Step 5
This result gets stored by your ISP and forwarded on to you, where both your OS and browser will likely cache the result – for as long as the TTL of the A-record dictates.
Step 6
Your browser tries to go to IP 123.123.123.123 and carries with it the hostname “stmforum.com” in the address bar – that is the site we are requesting.
The server receives our request for stmforum.com and says yes, I host that!
Let me just forward you on to the correct folder on my system that stores all that site content….
And that’s how the internet works.
Fun side fact: if you want to know what the faster DNS servers for you to use are, check out this: https://code.google.com/p/namebench/downloads/list
The result will often be your default i.e. what is set in your router, but sometimes switching your router to use Google DNS or something else may be faster than using your ISPs.
Now, hopefully you’re all following along. Does DNS make a little more sense now? I hope so.
DNS Configuration
Let’s move on to using a DNS service to configure all your DNS records – these are the all-important things that you have control over.
In this tutorial we are going to use NSOne, my favourite DNS service. Note that they cost money if you are pushing over 1 million queries a month.
If you are, and don’t need their advanced functions, use a cheaper provider. These guys provide some wicked functionality but it’s not needed for a non-techy.
To summarise, you can set advanced DNS rules and can use feeds/uptime monitoring to change DNS answers in real time – I will do a tutorial in the near future about using these to build something cool.
Why use NSOne? Not all DNS providers are made equal. I am not sure how good Namecheap DNS is, but there is a reason people pay $$ for DNS services. Check out http://blog.cloudharmony.com/2012/08...naged-dns.html for testing results.
This tutorial will help you do DNS configuration no matter the service. If you are just starting out, just use NSOne. You can easily switch later.
Head over to https://nsone.net/ and register an account. Sign in. At your dashboard, click on zones.
This is where you add your root domains – i.e. the ones you have bought.
Add some domain by clicking on “Create a New Zone” – in this example I’ll use domain.com.
Hit manual setup and fill out the details:
Don’t worry about any of the values, the defaults are fine. Note the NX TTL – this is our default TTL for our records that determines how long results may be cached for by our ISP, OS, and browser.
Now you should have a screen that looks like this:
See those Nameservers listed on the left hand side? Those are the nameservers you want to configure at your registrar.
If you are with NameCheap for example, go there now and click on your domain name > domain name server setup > put those values in:
Note use your values not whatever you see in my screenshots!
These will get pushed out to the .com or whichever relevant nameserver, though it may take a while.
Now when we try to go to domain.com, the recursive DNS lookup eventually points to dns1.p07.nsone.net, and this is where our records are stored.
Let’s now add the records we need to our DNS setup
Firstly, note that there is already a record there of type NS. This notes what the authoritative nameservers are for the domain in question (kind of like declaring it in multiple places for consistency).
Click Create New Record.
We want to add the base A-record that points the domain to our server.
We can do this one of two ways – we can put a * for the subdomain part, which will make every single subdomain go to this server, or we can leave it blank.
Let’s leave it blank for now.
Record type should be A.
You can leave the TTL blank to take the default value of 3600 (1 h).
Now put in your server IP. I’m going to use 123.123.123.123
Hit create record and it will now appear in the list of records for the domain.
All done! If you load domain.com it will now send people to the server IP listed.
However, what if we put www.domain.com? That’s a subdomain. If we try to go to it now, the DNS lookup will fail as there are no records for www.
Note that www is a subdomain of domain.com as much as track.domain.com is, or landers.domain.com. The only difference is that webservers tend to assume www.domain.com is meant for the root domain.com and they redirect the traffic there anyway.
We can do the same at the DNS level in two ways – we can make another A-record for www that points to the server IP, or we can use a CNAME record to point www.domain.com > domain.com. Both will accomplish the same thing.
A CNAME record is an alias. Imagine it as being like having 10 different email addresses that all forward to one original email account.
With CNAMEing, the address in the browser URL doesn’t change but underneath the URL loaded is that of the destination we set.
For example, if we add a CNAME record for cdn.domain.com and put the alias to abcdefg123.cloudfront.net, our CDN address, then going to cdn.domain.com/image.jpg would actually load abcdefg123.cloudfront.net/image.jpg – but in the browser we would still see the cdn.domain.com address in the address bar.
CNAMEing is an easy way to mask your URLs and use a different domain without any backend changes.
Have a tracking system link that got burned? You could add a CNAME record for track.domain2.com that points to track.domain1.com, allowing you to use a completely new domain name masked over the old one.
The only issue comes with SSL, i.e. trying to load pages via https – due to the way certificates and authentication works. This is why you can’t use SSL with you custom domains with
I digress, let’s add a CNAME record for www that points to the root domain:
Add that, and you’re done – www.domain.com and domain.com now resolve to the same server and everyone is happy.
Now, if you set up subdomains on your server, you don’t want to use CNAMEs for that.
Why? Because we don’t want track.domain.com to head to domain.com, we want it to just head to the server IP instead so that the server sends the users to the right virtual host, subdomain, whatever you want to call it.
The example for www above was unique because it is generally considered the default subdomain.
Let’s add an A-record for track.domain.com:
Good, all set.
If you look at your records list you will notice they have a static label.
If you click a record you can play with advanced settings – like filter chains for multiple results. Don’t worry about this for now or if you are not so inclined.
In the records for any domain you should have 3 minimum – an NS record, an A record for the root domain, and a CNAME record to point www > root domain.
Now, Shall We Add Some Uptime Monitoring?
First head to the notifications tab.
Create a new list called “Uptime Monitoring”
Click “Add notifier to list” > select email > input your email address.
Save and go to the monitors tab.
Add a new monitor:
I recommend selecting a region that’s not where your server is – if it’s in NY or Dallas, don’t select them.
I recommend this because there is then a small chance that a check from the same data center would not detect downtime caused by external network issues.
Note that selecting a 60 s frequency will give you 43,200 checks per month – and that will presumably dig into your query quota of 1 million (so not so much). So, when you scroll down, use the IP address of your server as the check point rather than domain.com – that way we check the server uptime without going through the DNS.
This works if the server has a default hostname/site on it. In some cases you might need to use a domain, so be weary of how many lookups you make especially if using other services that do so from many locations.
Now, if you reboot or power down your server, you should get a message from NSOne telling you that your server went down.
These monitoring feeds can be used to load-balance.
What's load balancing? It's where you spread the load across multiple endpoints i.e. servers. It is very common. You can load balance by just splitting traffic equally. You can load balance to specific servers based on where users are located. Often, load balancers are used so that you can take servers down for maintenance and have all traffic handled by the remaining servers, before putting the server back in the load-balancing circuit. Redundancy is good for the internet!
Practically, with DNS load-balancing, you would set a low TTL on your DNS entries and set rules to balance between multiple servers IF they are online. Server goes down > NSone removes server from rotation pool > new DNS lookups only go to online servers > cached entries expire within a few minutes and are sent to the new online server.
This kind of DNS-level load balancing is not super-fast (cf. a real load balancer) but is useful and as I said we will use it in a future tutorial for some cool stuff.
Quick Video Tutorial
Pasword is 'stmrocks'
Since VBulletin isn't letting your full-screen the video, you can also view it here -
As a newbie this awesome value Zeno! I can actually make a bit more sense about what the more techy guys are talking about now 
Cheers mate!
Just a question though, where does hosting come into all of this?
In the past i'd set up a website using basic hosting (say a shared server from hostgator) & a domain name from godaddy and that was all i needed, i'd simply copy and paste the name-servers from hostgator to godaddy and my website is up and running. Would my hosting have had a DNS server already installed in it? I've just never had to separately go to a 3rd party (eg DNS made easy or NSOne) to organise my DNS, so i'm a bit confused.
How would one link up their hosting here? Would it be done via NSOne or just by adding the hosting nameservers to the registrar?
Hosting comes in at the very end - it provides the server (computer) with IP of 123.123.123.123 as in the example diagram.
This is the end destination that all the DNS lookups provide.
So, the DNS system is basically the phonebook-like process that leads a users browser from some arbitrary name like "stmforum.com" to a server IP address.
I would recommend using NSOne or similar for all your DNS needs from now. If you just use the hosting companies nameservers, then those nameservers have to point to your server. This only works if your are on shared hosting (e.g. with HostGator) or the hosting company specifically adds entries for your servers - this isn't something I would rely on.
You can also use your server itself as the nameserver (e.g. ns1.mydomain.com) as servers can run DNS software on them, but again, not something I would recommend - why tie DNS resolution to some endpoint to that endpoint? That's sort of like having your street address as your name in the phone book.
Going with a third party = safer, more reliable DNS infrastructure, and when you want to make changes you can do so without worrying about your servers. E.g. you want domain.com to point to a different server for a week? Easy. Just change the A-record at NSOne. If you had the registrar using your server or the hosting companies nameservers... well, this is going to take more time and be a PITA!
Hi Zeno,
if I use Rackspace cloud I have to add only CNAME and put in the rackspace URL? (because for A Record I would need an IP?)
Thanks
Markus
For Rackspace Cloud Files?
Yes, just add a CNAME entry that goes to your container URL (without a protocol, i.e. no http://).
For CNAMEing you can either do:
blah.domain.com > CNAME to Racksapce
domain.com > CNAME to Rackspace
The latter is a special situation which is not possible with all DNS providers, and is only allowed if the CNAME entry is the only record for that domain zone.
hi zeno,
what exactly did you mean when you say below? Is it still on the way or...? thanks!
I'll just use NSOne for some other stuff in the future, though with an option to use Amazon R53 as well (so stay tuned, it'll be cool I assure ya!).
Hello,
Noobie question here, when I add an A-Record type that points to the domain of my server, what IP address do I use to point to?
Would this be my IP address from my local computer (from cmd > ipconfig/all) or my server? Also, how would one find this IP address?
My apologies for the lack of understanding.
Thanks
Another noob question -- I have just set up my first landing page. The hosting and domain are at namecheap, but the website is uploaded to XCDN. How does DNS play into all of this? My delivery address is cdn.domain.com, so do I use the original domain.com that I purchased from namecheap to set up zones or do I use the CNAMEd cdn.domain.com? Or do I not even need to worry about DNS?
Originally Posted by grindtimer
Originally Posted by mcsteve
http://stmforum.com/forum/showthread...l=1#post191210
https://www.namecheap.com/support/kn...a-cname-record
Originally Posted by vision
@vision Ahhh Ok thanks.
That is the step i am missing, not purchasing a server.
I will get on that ASAP.
Cheers!
Originally Posted by mcsteve
Usually, I would use a DNS service like NSOne rather than Namecheap's DNS but you can use whatever.
You should have an A-record pointing to your server IP.
You could then add a CNAME record such as cdn.domain.com that points to your XCDN edge hostname, which is something that they would supply. On their end, they may need you to add a published host (cdn.domain.com) so that when traffic comes to their edge hostname with a domain of 'cdn.domain.com', they know this is you and that the traffic should route through their XCDN hostname to your account.
The CDN may use containers that you directly upload to, or it may pull directly from folders on your server - and you upload there instead.
Originally Posted by zeno
One last question -- it shouldn't matter that my web server is shared hosting on namecheap if the content is being served via the CDN and the DNS is being handled by Cloudflare (or NSOne), right?
CloudFlare will be fine. Your DNS provider will in practice have a very small impact compared to hosting provider, server location, user connectivity, etc.
If you are sending users to the CDN then yes, your other hosting is irrelevant and isn't involved. Of course I would suggest you leave the shared hosting just like that - uninvolved with anything AM!
Hi, I've documented my problem in getting my domain to load here. Appreciate if someone could point to me my mistake.
http://stmforum.com/forum/showthread...130#post234130
Thanks!
Hi guys,
I'm really struggling to get this working. I'm following everything to a T I believe.
I'm using Hover as my register, which is a little different I suppose but it shouldn't make a big difference, right?
I've used my Hover IP address (shown in my Hover DNS panel) in NSone A Record (is that correct?). I've also made a www > www.domain .com CNAME.
At the Hover side; I've set the name servers at hover to the NSone servers. I've also created a CNAME record at hover with the 'Host Name' as CDN and the 'Target Host' as my Rackspace cloud files container without the http://. Should this be done with NSone instead of Hover?
I just tried it with NSone and I'm getting
The requested URL "/", is invalid.
Reference #9.d5b7d117.1439264037.385cfce7
Thanks for any help
I am confused by your setup here as you seem to be clashing Hover and NSOne against each other.
You have one registrar where you set authoritative nameservers for a domain.
You then have a DNS service like NSOne that hosts DNS records.
You do not have different systems trying to do the same thing, so if NSOne is being used, you should have zero DNS-related config at Hover other than setting the Nameservers to NSOne's.
Using an A-record supplied by hover makes no sense unless Hover is actually hosting a website for you on that server.
As for the error you are receiving, perhaps you have a / character in a DNS entry. There should never be any protocol (http, ftp, etc.) or file paths (anything after the .com or equivalent) in a DNS entry.
Originally Posted by zeno
Hover has default DNS stuff
Should I just delete all of that?
The IP I need for the A Records are from Rackspace, correct? Where do I acquire this?
Sorry for wasting your time, I really appreciate the help
If you are not using Hover as your DNS service then their records are completely irrelevant.
Wherever you bought the domain is the registrar. There you will set nameservers and should set those to NSOne's provided ones.
Then in NSOne you can set your actual DNS records.
If you want your site to load from the CDN you will need to create a CNAME record that points www.domain.com to the Rackspace hostname, e.g. "12345abc.rackcdn.net"
You cannot CNAME domain.com to that as, within standard DNS specs, you cannot CNAME the root domain. There are some workarounds but I don't believe NSOne provides these, but I could be wrong, I haven't tried.
Great Visual of how DNS works.
https://howdns.works/ (Just for the visually curious, that don't know what DNS is or how it works)
Hi Zeno,
When i try to load up my website, it gives a Index of /
Is there anything I missing here??
@zabs11230 - How do you mean? Are you seeing the root directory of your server, or a blank page with "/" on it and nothing else, or something else?
If you can give us some more details we can probably help out.
Hi Caumen,
Thank you for responding, I went to the Cpanel for beyond hosting and uploaded the index.html file to the public_html folder an now it loads.
Hi Zeno,
This tutorial is awesome! It took me from knowing nothing about DNS etc to having an understanding to how Internet works.
I would like to ask some question though as I am still not clear on the following areas:
1. What is the difference between dns1.p07.nsone.net and dns2.p07.nsone.net, dns3, dns4? The type of records they hold onto?
2. Why would we need to add a CNAME records to point www to our domain since it was said that webserver tends to regard them the same?
3. What is actually NS record? Could you elaborate on this?
Thank you!
Home > Programming, Servers & Scripts >