Many people first learned about the DNS system in October of 2021, when all of Facebook's apps and websites went down at the same time, due to a catastrophic DNS misconfiguration.

What is an IP Address?

An IP address is a unique identifier for a device on a network. They are used to route traffic to the correct device on a network.

The primary IP address of google.com, for instance, is 172.217.165.14.

IP addresses can be hard to remember. Especially if they are long and complicated. Human-readable names are much easier to remember.

What are the main types of DNS Servers?

There are many different types of DNS servers, each with its own unique capabilities.

The most common type of DNS server is the recursive DNS server. This is responsible for performing DNS lookups on behalf of its clients.

How Recursive DNS Servers Work

A client – typically a web browser – sends a DNS query (what's the IP address of this domain name?) to a recursive DNS server. That server resolves the query, then returns the answer to the client.

Recursive DNS servers are typically run by Internet Service Providers (ISPs). These are the companies you pay for your internet access each month.

How Authoritative DNS Servers Work

Another type of DNS server is the Authoritative DNS server. These are responsible for storing the DNS records for a domain. They contain a database of public IP addresses and corresponding hostnames.

Authoritative DNS servers are responsible for translating domain names to IP addresses. This allows users to access websites using domain names instead of IP addresses.

Authoritative DNS servers are typically provided by domain registrars.

What are the Ways a DNS Servers Can be Configured?

You can configure a DNS servers using one of these approaches:

• Static IP Address servers – permanent IP addresses that have been assigned to specific computers. Static IP addresses are ideal for computers that need to be accessible at all times – such as servers.
• Dynamic IP Address servers – these are useful when devices aren't permanently connected to the network (such as with public Wi-Fi networks). You can also use these to balance network traffic, or assign temporary IP addresses to devices that only infrequently connect to the network.
• Round Robin servers – these resolves domain names by returning a list of IP addresses – each corresponding to a server able to provide the requested information. Round Robin server can distribute traffic evenly across a group of servers. This ensures that no single server is overloaded with requests, and that other servers receive their fair share of traffic, too.
• Load Balancing servers – these figure out the most efficient way to distribute requests across servers. freeCodeCamp.org uses load balancing servers (also called "Load Balancers") and I imagine most major websites do, too.

You can also configure DNS servers to use different types of caching, which can improve performance.

What is caching?

Caching is a technique where you store data from past requests in a temporary memory location. The thinking is: if someone needs this information, someone else will probably need this information as well.

When someone requests data from your server, you can then first check to see whether the data is stored in your cache. If it is, you can retrieve it from cache rather than the original location.

This is how Content Delivery Networks (CDNs) work. Caching can dramatically speed up the performance of your website or service.

Does DNS change your IP address?

No. Switching DNS servers will not change your IP address.

DNS servers translate domain names to IP addresses. By default, all the web browsers come with the option to automatically detect the DNS settings of their current network.

So when you connect to a Virtual Private Network (VPN), the DNS server of your VPN replaces the DNS server of your ISP.

How do I setup a DNS server?

If you want to set up your own DNS server for your company or organization, here are some steps to get started:

1. Choose the right DNS server software. Some popular options include BIND, ISC DHCP, and PowerDNS.
2. Install the DNS server software on a dedicated server. This will help you ensure that your server has the resources it needs to run reliably. If you use the cloud, you won't have to worry as much about a power outage or network outage taking down your DNS.
3. Configure the DNS server software. This includes setting up the DNS zones and records.
4. Test the DNS server. Once it's is up and running, you might stress test it by simulating traffic to make sure it doesn't "fall over."

There are also plenty of hosted DNS server tools you can use, which should work out of the box and save you some time. These cost a bit of money each month, but require less expertise to supervise.

I hope you learned a lot about DNS Servers.

I hope you've found this helpful. If you want to learn more about programming and technology, try freeCodeCamp's core coding curriculum. It's free.

Ping and traceroute are common commands you can use to troubleshoot network problems.

Ping is a simple command that can test the reachability of a device on the network.

Traceroute is a command you use to 'trace' the route that a packet takes when traveling to its destination. It's useful for tracing network problems, discovering where connections fail, and tracking down latency problems.

How does ping work?

Ping uses ICMP (Internet Control Message Protocol) Echo messages to see if a remote host is active or inactive, how long a round trip message takes to reach the target host and return, and any packet loss.

It sends a request and waits for a reply (which it receives if the destination responds back within the timeout period).

It's basically a quick, easy way to verify that you can reach a destination on the internet. If you can, great! If not, you can use traceroute to investigate what's happening at every step between your device and the destination.

Example ping command and results:

hostname ~ % ping -c 5 www.google.com

PING www.google.com (216.58.212.228): 56 data bytes

The ping command, set to send 5 packets to google.com.

64 bytes from 216.58.212.228: icmp_seq=0 ttl=113 time=42.262 ms

64 bytes from 216.58.212.228: icmp_seq=1 ttl=113 time=34.796 ms

64 bytes from 216.58.212.228: icmp_seq=2 ttl=113 time=35.805 ms

64 bytes from 216.58.212.228: icmp_seq=3 ttl=113 time=45.299 ms

64 bytes from 216.58.212.228: icmp_seq=4 ttl=113 time=150.292 ms

This shows the results from each individual ping, with their round trip time in milliseconds.

--- www.google.com ping statistics ---

5 packets transmitted, 5 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 34.796/61.691/150.292/44.474 ms

The stats from the entire test - the minimum time it took to reach the destination, the average, the maximum, and the standard deviation.

How does traceroute work?

By default, traceroute sends three packets of data to test each 'hop' (when a packet is passed between routers it is called a 'hop').

It will first send 3 packets to an unreachable port on the target host, each with a Time-To-Live (TTL) value of 1. This means that as soon as it hits the first router in the path (within your network), it will timeout. The first router will respond with an ICMP Time Exceeded Message (TEM), as the datagram has expired.

Then another 3 datagrams are sent, with the TTL set to 2, causing the second router (your ISP) in the path to respond with an ICMP TEM.

This continues until the datagrams eventually have a TTL long enough to reach the destination. When it does, as the messages are being sent to an invalid port, an ICMP port unreachable message is returned, signaling that the traceroute is finished.

In this case, an error message is actually expected behavior, not a sign that something has gone wrong.

The most important part of a traceroute is usually the round trip times. Ideally you're looking for consistent times over the course of the trace.

If you see times suddenly increase (elevated latency) on a specific hop, and continue to increase as the trace approaches the target, this may indicate a problem starting with the sudden increase.

However, if there is elevated latency in the middle, but it remains consistent toward the end, or if the elevated latency decreases toward the end, that doesn't necessarily indicate a problem.

If you see high latency at the beginning of the trace, it may indicate a problem with your local network. You should work with your local admin (or yourself, if you are your own local admin) to fix it. By default, Windows uses ICMP to transmit the data while Linux uses UDP.

Example traceroute command and result:

hostname ~ % traceroute www.google.com

traceroute to www.google.com (216.58.212.228), 64 hops max, 52 byte packets

The command to traceroute to google.

1 homerouter.cpe (192.168.8.1) 10.129 ms 1.528 ms 1.373 ms

The first hop is within a local network. Here, we have the hop number (1), the domain name/IP address (in this case a home router), then RTT1, RTT2, and RTT3 (Round Trip Time - the time it takes for a packet to get to the hop and back to the computer, in milliseconds). This is the latency of the hop.

There are three numbers because, by default, the command sends three data packets. In general, times over 150ms are unusual for a trip within the continental US, though signals crossing an ocean may exceed this time.

2 *

Hop 2: There are two possibilities for stars like this - either ICMP/UDP were not configured on the receiving device and it did not respond, or the packets were dropped due to a network issue (such as a firewall or packet timeouts).

In this case, as it's very close to the beginning of the trace, it's likely that this is due to the device not being configured to send responses to a traceroute.

3 192.168.213.21 (192.168.213.21) 26.641 ms 31.671 ms 26.824 ms

4 192.168.213.22 (192.168.213.22) 20.294 ms 22.496 ms 19.922 ms

5 *

6 *

These stars, further along in the trace, are more likely to be due to a target's firewall blocking requests (though HTTP requests should still be able to be processed in most cases), a possible connection problem, or a return path issue (that is, the signal is reaching the router, but isn't getting a response).

The trace will then continue until it reaches the target.

Wrapping Up

In summary, ping is a (very) fast way to tell if a host is reachable over a network, while traceroute can help you diagnose connectivity problems.

They're both useful commands to know, as understanding how they work, and what the output means, can be very helpful when troubleshooting network connectivity.

You should also know how to use them for networking or security interviews, where questions like 'what port does ping work over (it's a trick question as ping uses ICMP)?' are commonly asked.

To make your life easier, the freeCodeCamp community has made this simple cheat sheet. Just scroll or use Ctrl/Cmd + f to find the value you're looking for.

Here are the charts, followed by some explanations of what they mean.

• /31 is a special case detailed in RFC 3021 where networks with this type of subnet mask can assign two IP addresses as a point-to-point link.

And here's a table of the decimal to binary conversions for subnet mask and wildcard octets:

Note that the wildcard is just the inverse of the subnet mask.

If you are new to network engineering, you can get a better idea of how computer networks work here.

Finally, this cheat sheet and the rest of the article is focused on IPv4 addresses, not the newer IPv6 protocol. If you'd like to learn more about IPv6, check out the article on computer networks above.

How Do IP Address Blocks Work?

IPv4 addresses like 192.168.0.1 are really just decimal representations of four binary blocks.

Each block is 8 bits, and represents numbers from 0-255. Because the blocks are groups of 8 bits, each block is known as an octet. And since there are four blocks of 8 bits, every IPv4 address is 32 bits.

For example, here's what the IP address 172.16.254.1 looks like in binary:

Source: IPv4

To convert an IP address between its decimal and binary forms, you can use this chart:

The chart above represents one 8 bit octive.

Now lets say you want to convert the IP address 168.210.225.206. All you need to do is break the address into four blocks (168, 210, 225, and 206), and convert each into binary using the chart above.

Remember that in binary, 1 is the equivalent to "on" and 0 is "off". So to convert the first block, 168, into binary, just start from the beginning of the chart and place a 1 or 0 in that cell until you get a sum of 168.

For example:

128 + 32 + 8 = 168, which in binary is 10101000.

If you do this for the rest of the blocks, you'd get 10101000.11010010.11100001.11001110.

What is Subnetting?

If you look at the table above, it can seem like the number of IP addresses is practically unlimited. After all, there are almost 4.2 billion possible IPv4 addresses available.

But if you think about how much the internet has grown, and how many more devices are connected these days, it might not surprise you to hear that there's already a shortage of IPv4 addresses.

Because the shortage was recognized years ago, developers came up with a way to split up an IP address into smaller networks called subnets.

This process, called subnetting, uses the host section of the IP address to break it down into those smaller networks or subnets.

Generally, an IP address is made up of network bits and host bits:

Source: What is IPv4

So generally, subnetting does two things: it gives us a way to break up networks into subnets, and allows devices to determine whether another device/IP address is on the same local network or not.

A good way to think about subnetting is to picture your wireless network at home.

Without subnetting, every internet connected device would need its own unique IP address.

But since you have a wireless router, you just need one IP address for your router. This public or external IP address is usually handled automatically, and is assigned by your internet service provider (ISP).

Then every device connected to that router has its own private or internal IP address:

Source: What Is My IP Address?

Now if your device with the internal IP address 192.168.1.101 wants to communicate with another device, it'll use the IP address of the other device and the subnet mask.

The combination of the IP addresses and subnet mask allows the device at 192.168.1.101 to figure out if the other device is on the same network (like the device at 192.168.1.103), or on a completely different network somewhere else online.

Interestingly, the external IP address assigned to your router by your ISP is probably part of a subnet, which might include many other IP addresses for nearby homes or businesses. And just like internal IP addresses, it also needs a subnet mask to work.

How Subnet Masks Work

Subnet masks function as a sort of filter for an IP address. With a subnet mask, devices can look at an IP address, and figure out which parts are the network bits and which are the host bits.

Then using those things, it can figure out the best way for those devices to communicate.

If you've poked around the network settings on your router or computer, you've likely seen this number: 255.255.255.0.

If so, you've seen a very common subnet mask for simple home networks.

Like IPv4 addresses, subnet masks are 32 bits. And just like converting an IP address into binary, you can do the same thing with a subnet mask.

For example, here's our chart from earlier:

Now let's convert the first octet, 255:

Pretty simple, right? So any octet that's 255 is just 11111111 in binary. This means that 255.255.255.0 is really 11111111.11111111.11111111.00000000 in binary.

Now let's look at a subnet mask and IP address together and calculate which parts of the IP address are the network bits and host bits.

Here are the two in both decimal and binary:

With the two laid out like this, it's easy to separate 192.168.0.101 into network bits and host bits.

Whenever a bit in a binary subnet mask is 1, then the same bit in a binary IP address is part of the network, not the host.

Since the octet 255 is 11111111 in binary, that whole octet in the IP address is part of the network. So the first three octets, 192.168.0, is the network portion of the IP address, and 101 is the host portion.

In other words, if the device at 192.168.0.101 wants to communicate with another device, using the subnet mask it knows that anything with the IP address 192.168.0.xxx is on the same local network.

Another way to express this is with a network ID, which is just the network portion of the IP address. So the network ID of the address 192.168.0.101 with a subnet mask of 255.255.255.0 is 192.168.0.0.

And it's the same for the other devices on the local network (192.168.0.102, 192.168.0.103, and so on).

What Does CIDR Mean and What is CIDR Notation?

CIDR stands for Classless Inter-Domain Routing, and is used in IPv4, and more recently, IPv6 routing.

_Source: Classless Inter-Domain Routing_

CIDR was introduced in 1993 as a way to slow the usage of IPv4 addresses, which were quickly being exhausted under the older Classful IP addressing system that the internet was first built on.

CIDR encompasses a couple of major concepts.

The first is Variable Length Submasking (VLSM), which basically allowed network engineers to create subnets within subnets. And those subnets could be different sizes, so there would be fewer unused IP addresses.

The second major concept CIDR introduced is CIDR notation.

CIDR notation is really just shorthand for the subnet mask, and represents the number of bits available to the IP address. For instance, the /24 in 192.168.0.101/24 is equivalent to the IP address 192.168.0.101 and the subnet mask 255.255.255.0.

How to Calculate CIDR Noation

To figure out the CIDR notation for a given subnet mask, all you need to do is convert the subnet mask into binary, then count the number of ones or "on" digits. For example:

Because there's three octets of ones, there are 24 "on" bits meaning that the CIDR notation is /24.

You can write it either way, but I'm sure you'll agree that /24 is a whole lot easier to write than 255.255.255.0.

This is usually done with an IP address, so let's take a look at the same subnet mask with an IP address:

The first three octets of the subnet mask are all "on" bits, so that means that the same three octets in the IP address are all network bits.

Let's take a look at the last forth octet in a bit more detail:

In this case, because all the bits for this octet in the subnet mask are "off", we can be certain that all of the corresponding bits for this octet in the IP address are part of the host.

When you write CIDR notation it's usually done with the network ID. So the CIDR notation of the IP address 192.168.0.101 with a subnet mask of 255.255.255.0 is 192.168.0.0/24.

To see more examples of how to calculate the CIDR notation and network ID for a given IP address and subnet mask, check out this video:

Classful IP Addressing

Now that we've gone over some basic examples of subnetting and CIDR, let's zoom out and look at what's known as Classful IP addressing.

Back before subnetting was developed, all IP addresses fell into a particular class:

Source: Subnetting for dummies

Note that there are class D and E IP addresses, but we'll go into these in more detail a bit later.

Classful IP addresses gave network engineers a way to provide different organizations with a range of valid IP addresses.

There were a lot of issues with this approach that eventually lead to subnetting. But before we get into those, let's take a closer look at the different classes.

Class A IP Addresses

For Class A IP addresses, the first octet (8 bits / 1 byte) represent the network ID, and the remaining three octets (24 bits / 3 bytes) are the host ID.

Class A IP addresses range from 1.0.0.0 to 127.255.255.255, with a default mask of 255.0.0.0 (or /8 in CIDR).

This means that Class A addressing can have a total of 128 (2⁷) networks and 16,777,214 (2²⁴-2) usable addresses per network.

Also, note that the range 127.0.0.0 to 127.255.255.255 within the Class A range is reserved for host loopback address (see RFC5735).

Class B IP Addresses

For Class B IP addresses, the first two octets (16 bits / 2 bytes) represent the network ID and the remaining two octets (16 bits / 2 bytes) are the host ID.

Class B IP addresses range from 128.0.0.0 to 191.255.255.255, with a default subnet mask of 255.255.0.0 (or /16 in CIDR).

Class B addressing can have 16,384 (2¹⁴) network addresses and 65,534 (2¹⁶) usable addresses per network.

Class C IP Addresses

For Class C IP addresses, the first three octets (24 bits / 3 bytes) represent the network ID and the last octet (8 bits / 1 bytes) is the host ID.

Class C IP Addresses range from 192.0.0.0 to 223.255.255.255, with a default subnet mask of 255.255.255.0 (or /24 in CIDR).

Class C translates to 2,097,152 (2²¹) networks and 254 (2⁸-2) usable addresses per network.

Class D and Class E IP Addresses

The last two classes are Class D and Class E.

Class D IP addresses are reserved for multicasts. They occupy the range from 224.0.0.0 through 239.255.255.255.

Class E IP addresses are experimental, and are anything over 240.0.0.0.

The Issue with Classful IP Addresses

The main issue with classful IP addresses is that it wasn't efficient, and could lead to a lot of wasted IP addresses.

For example, imagine that you're part of a large organization back then. Your company has 1,000 employees, meaning that it would fall into class B.

But if you look above, you'll see that a class B network can support up to 65,534 usable addresses. That's way more than your organization would likely need, even if each employee had multiple devices with a unique address.

And there was no way your organization could fall back to class C – there just wouldn't be enough usable IP addresses.

So while classful IP addresses were used around the time IPv4 addresses became widespread, it quickly became clear that a better system would be necessary to ensure we wouldn't use up all of the ~4.2 billion usable addresses.

Classful IP addresses haven't been used since they were replaced by CIDR in 1993, and are mostly studied to understand early internet architecture, and why subnetting is important.

I hope this cheat sheet has been a helpful reference for you

If you found this helpful, please share it with your friends so more people can benefit from it.

Also, feel free to reach out on Twitter and let me know what you think.

IT Managers earn an average of $91,000 a year. This salary bracket can actually range from $21,000 to $180,000 annually, depending on the industry of the IT Manager. In each case, though, the job entails problem-solving and project management.

Specifically, an IT Manager position has the following job description:

• Analyze, plan, and design the company’s information system needs and execute such plans and designs.
• Monitor and improve such a system (which may also include the company’s full telecommunications infrastructure) and recommend any upgrades required to its top management.
• Manage the company’s IT policies, including information policy, data/information access, and regulatory compliance.
• Hire and lead system administrators and other personnel who will be part of their team.
• Identify the training needs of the team and be able to arrange for such training and measure the benefits.

The first three above, as you will see, require the IT Manager to be astute in the technical aspects of the job. In particular, they must be competent in network-related areas. They should master these skills if they want to reach the higher tier of the salary scale.

Luckily, there are many training programs designed for IT Management professionals. These aren’t just regular training programs but fully-fledged certification courses. What follows are some of the top programs.

Cisco Certified Network Associate (CCNA)

The Cisco Certified Network Associate (CCNA) course is probably the most widely respected certification out there for IT professionals. It is an intermediate-level certification course. A CCNA holder has high competence in configuring, operating, and troubleshooting routed networks.

To get this certification, you must take and pass the CCNA 200-301 examination, which deals with fundamentals on network security, internet protocols, automation, and other security-related skills.

The exam is composed of 120 questions where each question has an associated score. To pass the exam, you must have a score of at least 800 (the highest attainable is 1,000). The cost for this exam is $300.

Cisco Certified Network Professional (CCNP)

Whereas CCNA covers the basic to intermediate skill level, the Cisco Certified Network Professional certification is for those who want to achieve a higher level of certification.

Unlike the CCNA, which requires the candidate to pass one exam, CCNP certification requires two exams: a core exam, covering fundamental networking concepts, and a concentration exam in any one area of intended focus (network design, automation, and so on).

It’s best to get a CCNP certificate after at least three years of experience in this industry. You don't need to take the CCNA before the CCNP examination. The costs for CCNP examinations range anywhere from $900 to $1,200.

Secure Access Service Edge (SASE)

SASE is a new technology (as described in Gartner’s “The Future of Network Security is in the Cloud”). This architecture integrates Wide Area Network (WAN) and some native security protocols of the cloud.

Cato Networks is a company that offers a cloud-based SASE platform, and they also offer a certification course for SASE so you can get familiarized with this new technology.

For this particular architecture, the center of a company’s network access is its physical data centers. For SASE, that center transfers over to the cloud.

In a nutshell, SASE makes it more efficient for IT Managers to administer the network as it removes some of the complexities associated with traditional physical data center setup.

To get a SASE certification, you need to take the SASE Expert Level 1 course and pass the associated examination (minimum score must be 85%).

CompTIA cloud+

CompTIA’s cloud+ certifications are for those who want to validate their system administration, cloud services maintenance, and other network-related knowledge and skills.

Currently, there are two such exams you can take to get a certification.

The CV0-002 (launched in February of 2018) is designed for candidates whose tasks include integrating various solutions into a business-specific system. It has 90 questions max and can be taken in 90 minutes. The passing score (from 100 to 900) is 750. The cost to take this exam is $329.

The CV0-003 (still in beta mode) is designed to get the candidate vendor-agnostic in creating solutions for their organizations. It has 110 questions that can be answered in 165 minutes. At the moment, there isn’t any finalized passing mark, and since it’s in beta, the cost to take this particular certification is just $50.

In both cases, it is recommended that the candidate has at least two years of experience.

AWS Certified Solutions Architect

The AWS Certified Solutions Architect certification is geared for professionals who design network architecture involving AWS. It requires that the candidate has at least two years of experience in operating and administering AWS applications.

It is one of the more sought-after certifications, as Amazon is the top vendor for cloud computing services. Hence, more and more organizations hire professionals that can navigate through this infrastructure.

With a price tag of $300 to take the exam, you should prepare for it thoroughly. The great thing is Amazon provides some preparatory materials for those interested in acquiring this prized certification. And so does freeCodeCamp.

Salesforce Certified Development Lifecycle and Deployment

As Salesforce is a key enterprise CRM solution, organizations often count on this SaaS platform to manage their relationship with customers as well as provide centralized access for related departments.

This exam also has a hefty price tag - $400. The exam itself has 60 multiple choice questions and requires the candidate to answer at least 60% of them correctly. Those failing the exam are allowed to retake it for a $200 fee.

In conclusion

These certifications are by no means the only ones that can enhance an IT Manager’s skills (and hence, career prospects). There are also others related to project management.

What is important to note here is that in the ever-changing technology landscape, it’s always essential to put yourself into continuous learning mode.

Sometimes, when you face a challenge, you might be able to solve it with routine processes. But other times you need to try something completely new, something that you know nothing about.

Usually in these scenarios you should apply engineering thinking. For me, these moments are the most insightful and I want to share some of mine with the community.

Here I will guide you through the steps that my team and I took when we connected existing AWS infrastructure to a large private network using Direct Connect.

Along the way, I will provide Terraform code snippets that will show you how to implement all of these components as "infrastructure as code" with accompanying design schemes.

What we'll cover

1. Problems to Solve
2. What is Direct Connect?
3. How to embed it
4. Transit VPC using Terraform
5. Direct Connect using Terraform
6. Peering between main and transit VPCs
7. Do you use OpenVPN (optional)?
8. Router Service
9. Closing thoughts

Problems to Solve

We had services within our VPC that should be able to communicate with other services in a separate virtual private network

In order to establish the connection, we needed to accept an AWS hosted connection from a network provider as part of a signed contract to grant access to the VPN using AWS Direct Connect.

So how were we to implement all of this? How were we going to embed it a current solution that was managed using Terraform? Were there any best practices for doing that?

What is Direct Connect?

AWS Direct Connect makes it easy to establish a dedicated network connection from your premises to your Amazon VPC or among Amazon VPCs. This option can potentially reduce network costs, increase bandwidth throughput, and provide a more consistent network experience than the other VPC-to-VPC connectivity options. (source)

Essentially you have a network provider who has AWS facilities in a shared data centre. Then you both can make a direct connection between your AWS network components and the network using the provider's hardware (literally a patchcord in the nest) with subsequent access.

Generic implementation in terms of AWS looks like the following:

• You configure one or two (reserved) Direct Connections in the console, which creates a Direct Connect Gateway.
• Then you attach a private VIF (one per connection) to the gateway.
• Once you make a few calls with the provider's network engineers and exchange routing policies, it is done.

Usually all instructions regarding how to enable the connection will be sent over to you by the provider.

How to embed it

Our first assumption was that we would enable the connection in the VPC and create the routing configuration to direct connect gateway for the required requests (for example, we'd distinguish them by the header "Host" or by IPs).

On high level, it would look something like this:

AWS Direct Connect

During a call with the provider's network engineers, they asked us about our IP range that we advertised to the network. We wondered why. It was because Direct Connect work is declared by a protocol called BGP. If you want more info, there are a lot of videos that will teach you about one of the major Internet protocols that are running under the hood.

Our initial thought was that it needed to be a subnet which contained services that we wanted to access the network. After that, we were asked to configure the subnet 10.1.2.0/24 as an allowed prefix in our Direct Connect configuration.

Long story short, "allowed prefixes" here stand for an IP range that we were going to advertise to the network provider that they would register in the routing policies.

Well, after all that, it did not work. The provider did not "see" our advertised routes despite the fact that we could see them.

A bit of investigation and voilà:

AWS will allocate private IPs (/30) in the 169.x.x.x range for the BGP session and will advertise the VPC CIDR block over BGP. You can advertise the default route via BGP.

Additionally, we found other folks who seemed faced the same issue:

we ended up with creating a new VPC with smaller CIDR our partner wanted.

So basically, the IP's range that you can advertise over Direct Connect is limited up to /30. Also, you can not advertise subnets – rather you should advertise the whole VPC CIDR.

Our network CIDR was 10.1.0.0/16 and we had an issue with that - it was too large to accept for the network provider. On top of that, during the call we discovered another thing we had to do when connecting to the network: we needed to contact the network IP access management department (if the network was large enough, I suppose) to ask them to provide us with a unique range within the network. Subsequently, it should be our new VPC CIDR.

We decided to create a separate VPC. To get some proofs of work we found some official guides form AWS such as this one. Shortly after this, we learned that the AWS community would start using separate words for that separate VPC - they'd call it a transit VPC.

Direct Connect using transit VPC

Before getting a reply to a request a unique IP range in the network, we asked the provider about currently unused IP ranges so we could implement it quickly on our side. This would give us the proof of work we needed for a solution. Everything worked perfectly.

The next step was to implement everything (Direct Connect configurations + VPCs peering) in our existing Terraform configuration.

Transit VPC using Terraform

First of all, before we start to dig into the code, I want to say that you can find all the code below on GitHub here.

Let's first recap what we discussed before. We have conditions where we had an existing VPC. And we wanted some services within it to be able to communicate through the network that we connected to using Direct Connect.

We were granted two AWS-hosted connections (primary and secondary, in order to ensure connection fallback). The main idea was to extend our existing infrastructure somehow. Somehow meant Transit VPC – the solution that helped us integrate with such connections.

Now let's look at some code to represent what we have discussed. The first thing to define is going to be our main VPC. I want to present it for illustration purposes only, so it makes all further steps seem more consistent.

```hashicorp configuration language variable "main_vpc_name" { description = "Name of your main VPC" } variable "main_vpc_cidr" { description = "CIDR of your main VPC, e.g. 10.1.0.0/16" } variable "public_subnet" { description = "pubic subnet of your main VPC (if you have), e.g. 10.1.1.0/24" } variable "private_app_subnet" { description = "private subnet of your main VPC (if you have), e.g. 10.1.2.0/24" } variable "main_vpc_key_name" { default = "main-vpc-key" description = "Name of SSH key of your main VPC" } variable "aws_availability_zone" { description = "Your AWS AZ of your main VPC" }

provider "aws" { profile = "your-profile" region = "your-region" }

terraform { backend "s3" { bucket = "your-terraform-states-bucket" key = "terraform.tfstate" profile = "your-profile" region = "your-region" } }

module "vpc" { version = "~> v2.0" source = "terraform-aws-modules/vpc/aws" name = var.main_vpc_name cidr = var.main_vpc_cidr

azs = [ var.aws_availability_zone, ]

private_subnets = [ var.private_app_subnet ]

public_subnets = [ var.public_subnet, ]

single_nat_gateway = true one_nat_gateway_per_az = false enable_nat_gateway = true enable_vpn_gateway = false

tags = { Terraform = "true" } } /* bellow could be defined any other resources from you infrastructure e.g. OpenVPN server, instances, security configuration, key pairs etc.

... */

Next, some of the main VPC's parameters are going to be used in the transit VPC. So let's define them as output:

```hashicorp configuration language
output "main_vpc_id" {
  value = module.vpc.vpc_id
}
output "main_vpc_range" {
  value = module.vpc.vpc_cidr_block
}
output "main_vpc_az" {
  value = module.vpc.azs.0
}
output "main_vpc_key_name" {
  value = var.main_vpc_key_name
}
output "main_public_routing_table_id" {
  value = module.vpc.public_route_table_ids.0
}
output "main_private_routing_table_id" {
  value = module.vpc.private_route_table_ids.0
}

Now we can start to configure our transit VPC. Just for sake of good design, we decided to manage it in a separate state under a separate folder (e.g. tranist-vpc/). Let's first import above outputs as locals:

```hashicorp configuration language locals { main_private_routing_table = data.terraform_remote_state.main.outputs.main_private_routing_table_id

main_public_routing_table = data.terraform_remote_state.main.outputs.main_public_routing_table_id

main_vpc_id = data.terraform_remote_state.main.outputs.main_vpc_id

main_vpc_range = data.terraform_remote_state.main.outputs.main_vpc_range

main_vpc_az = data.terraform_remote_state.main.outputs.main_vpc_az

main_vpc_key_name = data.terraform_remote_state.main.outputs.main_vpc_key_name }

Next, we can start defining the transit VPC configuration. First, I want to list all variables that we need (_pay attention to the IPs of the DNS servers in the network that we want to connect to. You should know them to specify as DNS servers in transit VPC_):

```hashicorp configuration language
variable "transit_vpc_name" {
  default = "transit-vpc"
}
variable "transit_vpc_cidr" {
  description = "Transit VPC CIDR. Your unique IP range in the network e.g. 10.10.14.0/24"
}
variable "transit_private_subnet" {
  description = "Transit VPC private subnet e.g 10.10.14.0/25"
}
variable "transit_public_subnet" {
  description = "Transit VPC public subnet for the NAT gateway e.g. 10.10.14.128/25"
}
variable "network_dns_server" {
  description = "IP of one of DNS servers in the network. Distributed by provider"
}
variable "network_dns_server_2" {
  description = "IP of one of DNS servers in the network. Distributed by provider"
}
variable "dhcp_options_domain_name" {
  description = "DHCP option domain name depending on your AWS region e.g. {your_region}.compute.internal"
}

And, secondly, the configuration:

```hashicorp configuration language module "transit-vpc" { version = "~> v2.0" source = "terraform-aws-modules/vpc/aws" name = var.transit_vpc_name cidr = var.transit_vpc_cidr

azs = [ local.main_vpc_az, ]

private_subnets = [ var.transit_private_subnet, ]

public_subnets = [ var.transit_public_subnet, ]

single_nat_gateway = true one_nat_gateway_per_az = false enable_nat_gateway = true enable_vpn_gateway = false enable_dhcp_options = true dhcp_options_domain_name = var.dhcp_options_domain_name dhcp_options_domain_name_servers = [var.network_dns_server, var.network_dns_server_2]

tags = { Terraform = "true" } }

## Direct Connect using Terraform

Let's continue with the Direct Connect configuration. First, let's define all variables that we need in order to continue. You should get all these values from your network provider. I assume they will be sent over to you (the same worked for us) in a separate document like a spreadsheet:

```hashicorp configuration language
variable "bgp_provider_asn" {
  description = "BGP autonomous system number of the provider. Distributed by provider"
}
variable "provider_vln_id" {
  description = "BGP VLN ID of the provider. Distributed by provider"
}
variable "primary_bgp_key" {
  description = "BGP auth key for primary virtual interface. Distributed by provider"
}
variable "secondary_bgp_key" {
  description = "BGP auth key for secondary virtual interface. Distributed by provider"
}
variable "primary_connection_id" {
  description = "BGP auth key for primary virtual interface. Distributed by provider"
}
variable "secondary_connection_id" {
  description = "IP range distributed by provider"
}
variable "primary_amazon_address" {
  description = "IP range distributed by provider"
}
variable "secondary_amazon_address" {
  description = "IP range distributed by provider"
}
variable "primary_customer_address" {
  description = "IP range distributed by provider"
}
variable "secondary_customer_address" {
  description = "IP range distributed by provider"
}

And now we can do the rest of the configuration:

```hashicorp configuration language resource "aws_dx_gateway" "provider-gateway" { name = "provider-dc-gateway" amazon_side_asn = "64512" // usually it's a default value }

resource "aws_dx_gateway_association" "transit" { dx_gateway_id = aws_dx_gateway.provider-gateway.id associated_gateway_id = aws_vpn_gateway.transit_vpn_gw.id allowed_prefixes = [ var.transit_vpc_cidr ] }

resource "aws_dx_private_virtual_interface" "primary" { connection_id = var.primary_connection_id name = "provider-vif-primary" vlan = var.provider_vln_id address_family = "ipv4" bgp_asn = var.bgp_provider_asn amazon_address = var.primary_amazon_address customer_address = var.primary_customer_address dx_gateway_id = aws_dx_gateway.provider-gateway.id bgp_auth_key = var.primary_bgp_key

}

resource "aws_dx_private_virtual_interface" "secondary" { connection_id = var.secondary_connection_id name = "provider-vif-secondary" vlan = var.provider_vln_id address_family = "ipv4" bgp_asn = var.bgp_provider_asn amazon_address = var.secondary_amazon_address customer_address = var.secondary_customer_address dx_gateway_id = aws_dx_gateway.provider-gateway.id bgp_auth_key = var.secondary_bgp_key }

Now, if you go to your AWS console, next to Direct Connection you should see something like this:

![Image](https://www.freecodecamp.org/news/content/images/2020/05/dc-connection.png)
_configured direct connections_

## Peering between main and transit VPCs

The last issue to solve is to configure connectivity between our services and transit VPC in order to establish access to the network. 

To accomplish this, we decide to use [VPC peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html). Here we will need some of the _locals_' variables that we imported before:

```hashicorp configuration language
resource "aws_vpc_peering_connection" "main-to-transit" {
  peer_vpc_id = module.transit-vpc.vpc_id
  vpc_id      = local.main_vpc_id
  auto_accept = true

  tags = {
    Name = "VPC Peering between main and transit VPC"
  }
}

resource "aws_route" "from-main-to-transit" {
  route_table_id            = local.main_private_routing_table
  destination_cidr_block    = var.transit_vpc_cidr
  vpc_peering_connection_id = aws_vpc_peering_connection.main-to-transit.id
}

resource "aws_route" "from-main-public-to-transit" {
  route_table_id            = local.main_public_routing_table
  destination_cidr_block    = var.transit_vpc_cidr
  vpc_peering_connection_id = aws_vpc_peering_connection.main-to-transit.id
}

resource "aws_route" "from-transit-to-main" {
  route_table_id            = module.transit-vpc.private_route_table_ids.0
  destination_cidr_block    = local.main_vpc_range
  vpc_peering_connection_id = aws_vpc_peering_connection.main-to-transit.id
}

Next we need to allow inbound HTTP traffic from the main VPC. That configuration can be done like this:

```hashicorp configuration language resource "aws_security_group" "transit_vpc_sg" { name = "transit-vpc-sg" description = "Transit VPC SG" vpc_id = module.transit-vpc.vpc_id

ingress { description = "Allow HTTP from main VPC" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = [local.main_vpc_range] }

egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] }

tags = { Name = "transit-vpc" } }

Great. Now we have two VPCs that are peered and coexist together.

## Do you use OpenVPN (optional)?

In our case, we have an OpenVPN server to manage access (SSH) to the main VPC's internal resources. And we wanted to access the transit VPCs resources in the same way. In order to do that we needed to create few additional resources within the transit VPC:

```hashicorp configuration language
resource "aws_vpn_gateway" "transit_vpn_gw" {
  tags = {
    Name = "transit-vpn-gw"
  }
}

resource "aws_vpn_gateway_attachment" "vpn_attachment" {
  vpc_id         = module.transit-vpc.vpc_id
  vpn_gateway_id = aws_vpn_gateway.transit_vpn_gw.id
}

resource "aws_vpn_gateway_route_propagation" "transit" {
  vpn_gateway_id = aws_vpn_gateway.transit_vpn_gw.id
  route_table_id = module.transit-vpc.private_route_table_ids.0
}

And then add an ingress rule to transit-vpc-SG that was created on the previous step:

```hashicorp configuration language ingress { description = "Allow SSH from main VPC" from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = [local.main_vpc_range] }

To make all of this work you need to specify the transit VPC's CIDR along with the main VPC's CIDR in the OpenVPN server routing setting under the VPN Setting section:

![Image](https://www.freecodecamp.org/news/content/images/2020/05/open-VPC-server-settings.png)
_OpenVPN server routing settings_

So now we are almost there. The last thing to do is to design and configure how our services within the main VPC will be able to programmatically access the network.

## Router Service

To recap, the main reason why we've done all of that is that we need to be able to access other services in the network (for example request or submit data). We found two possible ways to achieve that here:

* Migrate required services to the transit VPC and use them there, assigned with new private IPs. Main VPC internal routing should be adjusted. On top of that, any access to DB servers, logs' storage, and so on should be managed as well.
* Create router service (running HAproxy or NGingx) within the transit VPC. Add router private IP to the `hosts` file in each service in the main VPC that wants to access the network so the IP will be resolved behind the required domain name.

We choose the second option as it seemed to be the most aligned with the [open-close principle](https://en.wikipedia.org/wiki/Open–closed_principle). Here how it approximately looks:

![Image](https://www.freecodecamp.org/news/content/images/2020/05/transit-vac-router-service-3-2.png)
_transit VPC router service_

Let's configure it in Terraform:

```hashicorp configuration language
variable "router_private_ip" {
  description = "Private IP of router instance in transit VPC t route request back and forward e.g. 10.10.14.90"
}

resource "aws_instance" "router" {
  ami               = "ami-0eb89db7593b5d434" // any AMI you prefer
  instance_type     = "t2.micro" //any type you prefer
  availability_zone = local.main_vpc_az
  key_name          = local.main_vpc_key_name
  subnet_id         = module.transit-vpc.private_subnets.0
  private_ip        = var.router_private_ip


  vpc_security_group_ids = [
    aws_security_group.router_sg.id,
  ]

  user_data = file("router_init.sh")

  associate_public_ip_address = false
  tags = {
    Name    = "transit-vpc-router"
    Managed = "terraform"
  }
}

resource "aws_security_group" "router_sg" {
  name        = "router_security_group"
  description = "router_security_group"

  ingress {
    from_port = 80
    to_port   = 80
    protocol  = "tcp"

    cidr_blocks = [
      local.main_vpc_range,
      var.transit_private_subnet
    ]
  }

  ingress {
    from_port = 22
    to_port   = 22
    protocol  = "tcp"

    cidr_blocks = [
      local.main_vpc_az,
    ]
  }

  egress {
    from_port = 0
    to_port   = 0
    protocol  = "-1"

    cidr_blocks = [
      "0.0.0.0/0",
    ]
  }

  vpc_id = module.transit-vpc.vpc_id

  tags = {
    Managed = "terraform"
  }
}

Here router_init.sh contains a script to configure and launch the HAproxy service in a container. For illustration purposes, let's assume that we want to access two internal domain names in the network:

• domain-name-1.internal.com
• domain-name-2.internal.com

#! /bin/bash

# Install Docker
apt-get update

apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

apt-key fingerprint 0EBFCD88

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

apt-get update

apt-get install -y docker-ce

usermod -a -G docker ubuntu

chown -R ubuntu:ubuntu /home/ubuntu/.docker/

# Create HAproxy configuration
cat > /home/ubuntu/haproxy.cfg <<- "EOF"
       global
           log stdout local0
           daemon
           maxconn 4000

       defaults
           log               global
           mode              http
           option            httplog
           timeout connect   5s
           timeout check     5s
           timeout client    60s
           timeout server    60s
           timeout tunnel    3600s

       frontend http-in
           bind *:80

           #hosts acls
           acl domain1_acl             hdr(host) -i domain-name-1.internal.com
           acl domain2_acl             hdr(host) -i domain-name-2.internal.com


           use_backend domain1         if domain1_acl
           use_backend domain2         if domain2_acl

       backend domain1
           mode http
           option forwardfor
           http-request replace-header Host .* domain-name-1.internal.com
           server domain1 domain-name-1.internal.com:443 ssl verify none

       backend domain2
           mode http
           option forwardfor
           http-request replace-header Host .* domain-name-2.internal.com
           server domain2 domain-name-2.internal.com:443 ssl verify none
EOF

#Launch router
docker run -d --restart always --name haproxy --net=host -v /home/ubuntu:/usr/local/etc/haproxy:ro haproxy:2.1-alpine

The last step is to check that our domains were added to the hosts file on the instances in the main VPC and start making requests over HTTP.

Closing Thoughts

In this article, I showed you how to integrate Direct Connect into your existing AWS infrastructure. I also talked about how you can efficiently manage it using Terraform.

Then I discussed what approach would be appropriate for a network routing configuration that would make the solution transparent and easy to maintain as much as possible.

Transit VPC, which is recommended by AWS to solve such challenges, was indeed easy to configure. And the approach we tried with router service within transit VPC to access the private network showed its proof of work. But it didn't seem to be any better than other alternatives.

Lastly, I introduced Terraform code snippets are will hopefully be useful for anyone who wants to do something similar.

I hope you enjoyed this article and found it helpful!