AGI refers to a type of artificial intelligence that has the ability to understand, learn, and apply knowledge across a wide range of tasks at a level comparable to, or even beyond, human intelligence.

Unlike narrow AI, which excels at specific tasks (like image recognition or playing chess), AGI would be able to perform any cognitive task that a human can, adapt to new situations, and improve its capabilities over time without human intervention.

The emergence of AGI would fundamentally change how we think about and interact with technology. For engineers, it means preparing for a world where intelligent systems can perform tasks autonomously, requiring new skills and approaches to software development.

Here are some key areas of work you can focus on now to help prepare you for the AGI era.

1. Mastering Machine Learning and Deep Learning

Engineers with expertise in these fields will be at the forefront of AGI development. Machine learning and deep learning are the building blocks of AGI, as they enable systems to learn from data, identify patterns, and make decisions.

To prepare for AGI, you need to go beyond the basics of supervised learning and explore more advanced areas like reinforcement learning, where agents learn by interacting with their environment, and unsupervised learning, which allows systems to find hidden patterns in data without explicit guidance. Neural networks, particularly deep neural networks, will play a critical role in enabling AGI to generalize across tasks.

Why should you learn these skills? AGI will require systems that can adapt and improve autonomously. Mastering these skills will help you understand how to create models that can handle complex, unstructured data and make decisions in real-time, which is essential for AGI.

How to prepare

• Take courses: Consider taking advanced courses on platforms like freeCodeCamp, Coursera, edX, or Udacity that focus on reinforcement learning, neural networks, and deep learning.

• Build projects: Build your own machine learning models and experiment with different types of data. Participating in Kaggle competitions can also help you hone your skills.

• Job titles to watch: Machine Learning Engineer, AI Research Scientist.

2. Software Engineering with a Focus on AI Integration

Traditional software engineering roles will evolve to integrate AI components seamlessly. This means developing frameworks that allow AGI to be incorporated into existing systems or creating entirely new systems designed around AGI capabilities.

What does this look like? Engineers might develop APIs that allow AGI to communicate with other software, create microservices that enable modular AGI deployment, or design platforms that facilitate continuous learning for AGI systems.

For example, integrating AGI into a customer service platform could involve building an interface where AGI handles complex queries while human agents focus on more nuanced tasks.

How to prepare

• Study: Learn how to design and implement AI components in software through courses and hands-on experience. Understanding cloud-based AI services like AWS SageMaker or Google AI Platform will also be beneficial.

• Practice: Work on projects where you integrate AI models into existing applications, such as adding a chatbot to a web service or incorporating predictive analytics into a mobile app.

• Job titles to watch: Full Stack Developer with AI specialization, AI Software Engineer.

3. Navigating Ethics and AI Governance

As AGI could pose significant ethical and governance challenges, roles focusing on the ethical implications, policy-making, and regulatory compliance will be crucial. This includes ensuring AGI systems operate within legal and ethical frameworks. Public as well as private sector experience will be valuable.

Key ethical concerns include issues like accuracy, accountability, and transparency. You can benefit from developing skills in critical thinking and understanding how to interpret data and statistics. These skills can be helpful when collaborating with policymakers.

How to learn

• Read up: Explore literature on how policy is formed at the institutional and governmental level.

• Take courses: Consider taking courses on statistics and ethics to grow a deeper understanding of model results. Here's one on the ethics of AI and ML for starters.

• Job titles to watch: AI Ethics Analyst, Policy Advisor for AI, Compliance Officer for AI Systems.

4. Evolving Human-Computer Interaction (HCI)

HCI will quickly transform into Human-AI Interaction Design. As AGI systems become more prevalent, they will need to interact with humans in intuitive and seamless ways. Companies will need interfaces where humans can interact with AGI systems effectively, built by engineers who understand cognitive psychology and UX/UI design for AI systems.

Engineers will need to design interfaces where AGI can explain its decisions, ask for clarification when needed, and understand human emotions and context.

For example, AGI in healthcare might need to provide doctors with explanations of its diagnoses while considering the doctor's expertise and the patient's emotions. Building skills in designing intuitive interfaces and interactions between humans and intelligent systems will help you to be highly successful in AGI integration.

How to prepare

• Take courses: Study HCI and UX design with a focus on AI systems. Platforms like Interaction Design Foundation and Coursera offer relevant courses.

• Build projects: Experiment with designing user interfaces for AI-powered applications. This could include developing conversational agents or creating dashboards that visualize AI decision-making processes.

• Job titles to watch: Interaction Designer for AI, User Experience Researcher for AI Systems.

5. Enhancing Autonomous Systems and Robotics

If AGI leads to more autonomous robots, engineers who can design, build, and program robots with AGI capabilities will be in demand. This includes understanding how AGI can enhance robotic functionality.

AGI has the potential to revolutionize autonomous systems and robotics by enabling machines to learn and adapt in real-time. This could lead to more advanced self-driving cars, drones, and robots that can perform complex tasks without human intervention.

AGI could allow robots to understand and navigate unstructured environments, learn from experience, and collaborate with humans more effectively. For example, an AGI-powered robot could assist in disaster relief by autonomously adapting to changing conditions and coordinating with human teams.

Working on autonomous systems, whether in robotics, self-driving vehicles, or drones, can provide practical experience with highly independent systems. These skills will be transferrable to managing and optimizing AGI-based autonomous agents.

How to prepare

• Take courses: Take robotics courses that cover autonomous systems, computer vision, and AI integration.

• Build projects: Work on robotics projects, such as building an autonomous vehicle or programming a robot to perform complex tasks.

• Job titles to watch: Robotics Engineer, Automation Specialist.

6. Pioneering Hardware Development for AGI

We'll need engineers working on specialized hardware that can support AGI. Technologies like neuromorphic computing chips or quantum computing might be necessary for the computational power AGI would require.

Neuromorphic computing involves designing chips that replicate the structure and function of the human brain's neurons and synapses. These chips could enable more efficient and powerful AI systems by processing information in ways that are closer to how the human brain works. Also, quantum computing could provide the processing power needed for AGI's complex calculations.

How to prepare

• Study: Learn about neuromorphic computing and quantum computing through specialized courses and research papers. Staying updated with developments from companies like IBM and Intel, which are working on neuromorphic chips, can also be helpful.

• Build projects: Experiment with hardware design, such as working with FPGAs (Field Programmable Gate Arrays) or exploring quantum computing platforms like IBM Q.

• Job titles to watch: Hardware Engineer for AI, Quantum Computing Engineer.

7. Securing the Future: Cybersecurity for AGI

AGI systems will introduce new security challenges. Engineers with expertise in cybersecurity will be in high demand to protect AGI systems from national security threats, ensure data privacy, and secure AI-driven decision-making processes against manipulation. There are also concerns about data privacy, as AGI systems will likely handle sensitive information across various domains.

How to prepare

• Take courses: Take cybersecurity courses focused on AI and machine learning security. Platforms like freeCodeCamp, Cybrary, and Coursera offer relevant courses. Here's a handbook that has some helpful insights, too.

• Practice: Engage in cybersecurity challenges, such as Capture the Flag (CTF) competitions, to develop hands-on skills in securing computing systems.

• Job titles to watch: AI Security Specialist, Cybersecurity Analyst for AI.

8. Data Engineering: Fueling AGI with Information

Handling large-scale data systems will be critical for AGI, as it will require vast amounts of data to learn and operate effectively. Data engineers will play a crucial role in building and maintaining the infrastructure that feeds AGI with the information it needs.

Data engineers will need expertise in big data technologies, such as Hadoop and Spark, and real-time data processing systems, like Apache Kafka. They will also need to ensure data quality and integrity, as AGI systems will rely heavily on accurate and comprehensive data to function effectively.

How will data engineers work with AGI systems? Data engineers will design data pipelines that can handle the immense scale of data AGI requires. This includes everything from data ingestion, storage, processing, and ensuring that the data is of high quality and usable for AGI models. They'll also need to implement systems for continuous data updates, enabling AGI to learn and adapt in real-time.

How to prepare

• Take courses: Take courses on big data technologies, data pipeline architecture, and real-time data processing. Platforms like freeCodeCamp, Udemy, and Coursera offer courses on tools like Apache Kafka, Spark, and Hadoop.

• Projects: Work on projects that involve large-scale data processing and pipeline development. Contributing to open-source big data projects can also be a great way to gain experience.

• Job titles to watch: Data Engineer, Big Data Architect.

9. Building Infrastructure for AGI

AGI will require robust and scalable infrastructure on a never-before-seen scale. Engineers with experience in cloud computing, distributed systems, and infrastructure as code (IaC) will be crucial in building the systems that support AGI.

AGI systems will likely operate on a global scale, requiring vast amounts of computational power and data storage. Engineers will need to design cloud-based infrastructure that can scale dynamically, handle high volumes of data, and ensure low latency for real-time processing. They will also need to consider the security and reliability of these systems.

How to prepare

• Take courses: Study cloud computing platforms like AWS, Google Cloud, or Microsoft Azure. Learning about distributed systems and IaC tools like Terraform and Ansible will also be beneficial.

• Obtain certifications: Earning certifications in cloud architecture (like the AWS Certified Solutions Architect) can help solidify your knowledge. freeCodeCamp has many free courses to help you study for AWS certs – like this one.

• Build projects: Work on setting up and managing cloud infrastructure for applications, experimenting with scalability and load balancing.

• Job titles to watch: Cloud Infrastructure Engineer, Systems Architect for AI.

10. Cross-Disciplinary Collaboration in the AGI Era

Working in roles that involve cross-disciplinary collaboration, such as roles in research or innovation labs, can provide engineers with the ability to think broadly and integrate knowledge from various fields. Knowledge in other fields can give you the ability to engineer products that help people in a niche you care about.

Combining skills from fields such as biology (for bioinformatics or synthetic biology with AGI) and psychology (for understanding human-AI interaction) will be vital in the AGI era. Engineers who can think broadly and collaborate across disciplines will be better equipped to tackle complex problems that require diverse perspectives. For instance, combining AGI with neuroscience could advance brain-computer interfaces.

How to prepare

• Networking: Engage with professionals from different fields by attending interdisciplinary conferences and joining relevant online communities.

• Take courses: Take courses or workshops in complementary fields, such as biology, psychology, or environmental science, to broaden your understanding. Here's a course on bioinformatics if you're curious.

• Build projects: Collaborate on interdisciplinary projects, such as developing AI models that incorporate insights from other domains.

• Job titles to watch: Bioinformatics Engineer, Environmental Data Scientist.

11. Education and Training for an AGI-Ready Workforce

As AGI transforms industries, there will be a growing need for educational programs that teach engineers how to work with AGI systems.

Education and training programs should cover a range of topics, from yet-undeveloped AGI techniques, safety protocols, policy-making, to interdisciplinary collaboration. Preparation for creating or leading education in AGI means becoming a continuous learner yourself. Also, training should emphasize lifelong learning, as AGI technology will continue to evolve rapidly.

How to prepare

• Create content: If you're an educator, consider developing courses or workshops focused on AGI-related topics. Collaborate with industry experts to ensure the content is relevant and up-to-date.

• Enroll in programs: Participate in advanced AI or AGI training programs, either through universities or industry-led initiatives. Stay updated with emerging trends by attending seminars and conferences.

• Job titles to watch: AI Curriculum Developer, Training Specialist for AI Technologies.

12. Shaping Regulations in an AGI-Driven World

Engineers working on regulatory technology (RegTech) will gain insight into compliance and governance, which will be critical as AGI evolves within legal frameworks. Understanding how to navigate and shape regulations will be vital.

Regulations could cover areas such as data privacy, transparency, accountability, and the use of AGI in various industries. Engineers working in this field will need to collaborate with policymakers, legal experts, and industry leaders to develop guidelines that balance innovation with responsibility.

How to prepare

• Study: Stay informed about current AI regulations and legal frameworks. If you're interested, consider pursuing a certification or degree in law or public policy with a focus on AI governance.

• Networking: Join industry groups like IEEE or think tanks that focus on AI policy and ethics. Engaging in discussions with policymakers can provide valuable insights into the regulatory landscape.

• Job titles to watch: Regulatory Engineer, Compliance Specialist for AI.

13. Research and Development (R&D) in AGI-related Areas

Finally, engineers who are involved in cutting-edge research in AGI, cognitive computing, or advanced AI labs will be directly contributing to and understanding the frontiers of AGI technology, giving them a head start in a world where AGI is a reality. These roles offer the opportunity to shape the future of AGI and explore new possibilities in artificial intelligence.

Get involved in R&D by joining research institutions, universities, or tech companies that focus on AGI development. Contributing to open-source AI projects or publishing papers on AGI-related topics can also help you establish yourself in the field.

How to prepare

• Research: Stay updated with the latest advancements in AGI by reading academic papers, attending conferences, and following thought leaders in the field.

• Collaborate: Work with academic or industry researchers on AGI projects. Participating in hackathons or research competitions can also provide hands-on experience.

• Job titles to watch: AGI Research Scientist, Cognitive Computing Engineer.

Future Job Titles

The transition to an AGI world will likely see a blend of these roles, where engineers might need to be polymaths, understanding not just one but multiple areas of technology and science. It's important to grow your technical skills, but also practice adaptability and continuous learning.

Be on the lookout for roles that might not directly mention AGI but are foundational in AI, machine learning, and related technologies. As you choose your next role, think ahead to how you can tailor your focus and future-proof your work for the AGI era.

Actionable steps

• Learn continuously: Make lifelong learning a priority by regularly updating your skills and knowledge through courses, certifications, and hands-on projects.

• Network: Build relationships with professionals in various fields to stay informed about emerging trends and opportunities.

• Adapt: Stay flexible and open to new challenges, as the AGI era will require engineers to adapt to rapidly changing technologies and environments.

Be on the lookout for roles that might not directly mention AGI but are foundational in AI, machine learning, and related technologies. As you choose your next role, think ahead to how you can tailor your focus and future-proof your work for the AGI era.

• How do I become a software developer?
• What language or framework should I learn first?
• Where do I start?

While I’m certain there’s no one right answer for everyone, I’m also certain that the world needs more software developers and systems thinkers.

The best thing I can do to help you lead yourself, learn to code, and become a software developer is to share the most efficient parts of how I did it myself. This is the article I wish I had read when I started coding.

Depth of knowledge matters

Software is exceedingly complex. Like a good novel that you wish you’d never finish reading, there’s always more to discover and learn.

If you don’t want to miss the best parts, don’t be satisfied with surface-level explanations. Always go deeper! Ask why, why, and why again until you get to the fundamentals. Soon enough, you’ll start to see patterns.

By digging deeper, you’ll begin to understand the fundamentals of how things connect, what makes things “fast,” and facets of software operation that you probably can’t even imagine exist. It’s like peeking behind the curtain and seeing a whole world of systems and processes that most people are never aware of.

Going in-depth can expand your mind and your capacity for learning. Keep asking why. Follow every link. Let your curiosity guide you.

Hard stuff matters

Giving yourself the chance to be delighted through discovery doesn’t come for free. It takes a lot of hard work to read and compress complicated ideas into your meat brain.

It’s important not to gloss over the hard stuff. In fact, if something seems too hard to understand, you might benefit from doing it first. You might have to get creative to find ways to explain things to yourself, but when you succeed, it makes everything else easier later on.

Analogies are helpful for understanding hard concepts, but they’ll only help you start to understand concepts at a surface level. Remember to go in-depth. Don’t stop at the analogy.

Writing matters

Write right away. Create a habit of explaining everything you learn to yourself in long-form writing. Better than bullet points, writing with a conversational tone engages parts of your brain that help you to process and remember new information. It’s why humans like and remember stories, and it’s a superpower you get for free.

Start by writing for yourself. Write about what interests you. Try something new, even if it seems rudimentary, and write in-depth about what you learn. (One of my most popular posts is about iteration in Python. When I first wrote it, I considered myself a complete beginner.)

If you want to go a step further, share your writing with the world. Learn in public, like I do. I often get questions like, “how do I choose a theme for my blog?” or “what platform should I use?” or “what popular language/framework/topic should I focus on?” My answer is: don’t worry about it.

Don’t fret too much about your blog theme or platform. Pick the easiest option for you to get started with for now. All of that will change and improve as you learn, practice, and find your focus. Just start writing, ideally, yesterday.

Write for yourself by explaining what you’re doing, as if it were past-you teaching future-you — because it is. You will be your first reader, and the first judge of how useful your blog can be. Seek to impress yourself!

The language, framework, or version doesn’t matter

Why pigeonhole your abilities before you even start? Pick any software language, framework, or technology that seems to make sense to you when you first read it. Start there.

Remember that it’s important to dig deep and understand the fundamentals. Basic concepts of software transcend languages.

Whichever first language you choose, understand functions, variables, return values, iteration, and how immutability works. You’ll find that learning these concepts will make it easier to recognize them in your second language, and learn that too.

Your portfolio doesn’t matter

If your first objective is to build a portfolio, you may be trying to run before you walk. Building a portfolio to showcase to potential employers is a great goal, but a terrible first step.

If you think of creating a polished portfolio as a first step, you’re liable to spend too much time making it pretty and presentable before focusing on the content. As someone who hires software developers, I can tell you wholeheartedly that I’d rather see clean and well-written code than a flashy front page.

Don’t confuse building a portfolio with building projects. Absolutely build projects, right from the beginning. There’s no better way to see the practical application of what you’re learning. Just treat them as first drafts, as training ground, and don’t worry about packaging them up for professional consumption.

By allowing yourself to build some draft projects first, you allow yourself the breathing room to learn from them. Focus on iteration, on making one small thing better each time, and you’ll build a portfolio without even realizing it.

Focus on what matters

Don’t follow this advice blindly – rather, incorporate it into your own systems. Experiment, make it work better than when you found it, then pay it forward by writing down what you’ve learned for someone else to read!

Here are my favorite books for reading or listening to if you want to cultivate a learning mindset. See non-coding books for coders.

If this article benefits you in some way, I encourage you to write about it! The process of learning how to learn is never finished. You can be the next iteration.

If you enjoyed this post, I'd love to know. Join the thousands of people who learn along with me on victoria.dev! Visit and subscribe for more about building your coding skill stack.

If you’ve ever half-written a software project before taking a few days off, this is the article you’ll discover you needed when you reopen that IDE.

Don't worry, it'll all come together by Friday again... (Comic by author)

In the technology teams I lead, we make a constant effort to document all the things. Documentation lives alongside the code as an equal player.

This helps ensure that no one needs to make assumptions about how something works, or is calling lengthy meetings to gain working knowledge of a feature. Good documentation saves us a lot of time and hassle.

That said, and contrary to popular belief, the most valuable software documentation is not primarily written for other people. As I said in this well-received tweet:

The secret to good documentation is to write it while you're writing the code. You are your first audience. Explain what you're doing to yourself. Future you will thank you!

— Victoria Drake November 24, 2020

Here are three concrete steps you can take to write good documentation before it’s too late.

1. Start with accurate notes

As you work out ideas in code, ensure you don’t soon forget important details by starting with accurate notes. While you will want to explain things to yourself in long-form later, short-form notes will suffice to capture details without interrupting your coding session flow.

Keep a document open alongside your code and write down things like commands, decisions, and sources you use. This can include:

• Terminal commands you typed in
• Why you chose a particular method over another
• Links you visited for help or _cough_copy-pastecough inspiration
• The order in which you did things

Don’t worry about full sentences at this point. Just ensure you accurately capture context, relevant code snippets, and helpful URLs. It can also be helpful to turn on any auto-save option available.

2. Explain decisions in long form

The ideal time to tackle this step is when you take a break from coding, but before you completely go out to lunch on whatever it is you’re working on at the moment.

You want to ensure that context, ideas, and decisions are all still fresh in your mind when you explain them to yourself.

Go over the short-form notes you took and start expanding them into conversational writing. Be your own rubber duck. Describe what you’re doing as if you were teaching it to someone else. You might cover topics such as:

• Quirky-looking decisions: “I would normally do it this way, but I chose to do something different because…”
• Challenges you ran into and how you overcame them
• Architectural decisions that support your project goals

Stick to the main points. Long-form writing doesn’t mean you’ll be paid by the word! Just use full sentences, and write as if explaining your project to a colleague. You’re explaining to future you, after all.

3. Don’t neglect prerequisite knowledge

This step is best done after a long lunch break, or even the next day (but probably not two). Re-read your document and fill in any blanks that become apparent after putting some distance between yourself and the project.

Take extra care to fill in or at least link to prerequisite knowledge, especially if you frequently use different languages or tools. Even an action as small as pasting in a link to the API documentation you used can save hours of future searching.

Write down or link to READMEs, installation steps, and relevant support issues. For frequently performed command-line actions, you can use a self-documenting Makefile to avoid having to man common tasks each time you come back to a project.

It’s easy to forget supporting details after even just a short break from your project. Capture anything you found helpful this time around.

Document all the things!

The next time you catch yourself thinking, “I’m sure I’ll remember this part, no need to write it down,” just recall this emoji: 🤦‍♀️

Software projects are made up of a lot more than just their code. To best set up your future self for success, document all the things! Whether it’s a process you’ve established, Infrastructure as Code, or a fleeting future roadmap idea — write it down! Future you will thank you for it.

If you enjoyed this post, I'd love to know. Join the thousands of people who learn along with me on victoria.dev! Visit and subscribe for more about building your coding skill stack.

Whether you’re leading a team of people or leading yourself, it’s important to take account of all the important things that need doing in your organization. And to realize that this does not mean that everything can be equally important.

Logically, everything can’t be. Tasks are typically interdependent, and there’s always one task on which another depends. Tasks can be time-sensitive. Certain tasks might block a logical path towards a goal.

It’s the duty of a leader to make hard calls and decide which tasks are most important out of everything that needs doing. This necessitates comparing one to another, which is much easier to do with a centralized to-do list.

Here’s how this one simple change to your perspective on to-do lists can help to build happier and more productive teams.

Keep a central prioritized to-do list

Avoid working in silos. A single centralized list can make it easier for you and your team members to see what’s being worked on. With all tasks out in the open, it’s easier for people to spot opportunities for helping each other out and where they can contribute.

Encouraging a culture of openness can help people feel more comfortable asking questions, asking for help, and proposing ideas and improvements. Tracking work in the open also means that no one is left wondering what status a task is currently in.

For team leaders, a single list makes it easier to compare and prioritize tasks. This benefits team members by providing a completely unambiguous and transparent accounting of what needs doing next. Whichever task is most important, for the whole organization, is on top.

Priorities with autonomy

A single priority doesn’t necessarily pigeonhole someone into doing a task they don’t feel cut out for. Each member of your team has different strengths, skill sets, and diverse ways of thinking. You can take full advantage of this by encouraging autonomy in task selection.

Have people choose whichever task is nearest to the top that they’d like to tackle. They might pick the highest priority task that’s in their wheelhouse, or experiment with a higher one that’s in a domain they’d like to improve their skills at.

Embrace opportunities for cross-training. If tasks high up on the list fall in a category that only one or a few people on your team are experts in, have your experts partner up with another team member who’s taking on the task.

By pooling your resources to cross-train across domains, you multiply the capabilities of each team member and your team as a result.

When a task is especially time-sensitive, have several team members swarm on it and distribute the work according to their interests or strengths.

Make yourself redundant

Working off a single prioritized to-do list works best when your team members can take on tasks as independently as possible. This is especially important in remote teams where people work asynchronously.

If you’re a leader and find that your team members frequently ask you what they should do next, you could be making your team dependent on you. Ask yourself if you’re unnecessarily gatekeeping information that would let your team be more autonomous.

A team that overly depends on their leader is not an efficient one. Individual people, such as yourself, don’t scale. Don’t become a bottleneck to your team’s productivity.

A successful leader should be able to take several days off on short notice without productivity grinding to a halt.

To support your team’s ability to work without you, make your team, product, and company goals painfully available.

Put them where people hang out – your team’s message board, chat channel, or document repository, for example. No one should be at a loss when asked what the team wants to achieve next, and why.

Make any applicable resources, style guides, product documents, or links to external documentation painfully available as well.

If your team makes a decision about how something should be done, write it down. Don’t rely on yours or anyone else’s meat brain to remember an important decision, nor make yourself the only resource for recalling it.

Make yourself redundant when it comes to day-to-day work. Doing so empowers your team members to do work without you, think through solutions on their own, and propose paths of action that you probably wouldn’t have thought of yourself.

Build happier and more productive teams

From first-hand experience as both a team member and leader, I’ve seen how encouraging a culture of openness, cross-training, and autonomy makes for happier team members and more productive teams.

A single prioritized to-do list, coupled with available documentation and resources, opens the gates to let your technical team be maximally productive.

By removing bottlenecks, you allow people to make more decisions on their own and take ownership of their work. That’s a technical team I’d be proud to lead.

If you enjoyed this post, I'd love to know. Join the thousands of people who learn along with me on victoria.dev! Visit or subscribe via RSS for more about building happy and productive technical teams.

If you happen to drop by my website, you may notice I’ve spruced it up a bit. Victoria.dev can now better respond to your devices and preferences.

Most modern devices and web browsers allow users to choose either a light or dark theme for the user interface. With CSS media queries, you can have your own website's styles change to match this user setting!

Media queries are also a common way to have elements on web pages change to suit different screen sizes. This is an especially powerful tool when combined with custom properties set on the root element.

Here's how to use CSS media queries and custom properties to improve your visitor's browsing experience with just a few lines of CSS.

How to Cater to People's Color Preferences

The prefers-color-scheme media feature can be queried to serve up your user’s color scheme of choice. The light option is the go-to version if no active preference is set, and it has decent support across modern browsers.

Additionally, users reading on certain devices can also set light and dark color themes based on a schedule. For example, my phone uses light colors throughout its UI during the daytime, and dark colors at night. You can make your website follow suit.

Avoid repeating a lot of CSS by setting custom properties for your color themes on your :root pseudo-class. Create a version for each theme you wish to support. Here’s a quick example you can build on:

:root {
    color-scheme: light dark;
}

@media (prefers-color-scheme: light) {
    :root {
        --text-primary: #24292e;
        --background: white;
        --shadow: rgba(0, 0, 0, 0.15) 0px 2px 5px 0px;
    }
}

@media (prefers-color-scheme: dark) {
    :root {
        --text-primary: white;
        --background: #24292e;
        --shadow: rgba(0, 0, 0, 0.35) 0px 2px 5px 0px;
    }
}

As you can see, you can use custom properties to set all kinds of values. To use these as variables with other CSS elements, use the var() function:

header {
    color: var(--text-primary);
    background-color: var(--background);
    box-shadow: var(--shadow);
}

In this quick example, the header element will now display your user’s preferred colors according to their browser settings.

Preferred color schemes are set by the user in different ways, depending on the browser. Here are a couple examples.

Light and Dark Mode in Firefox

You can test out light and dark modes in Firefox by typing about:config into the address bar. Accept the warning if it pops up, then type ui.systemUsesDarkTheme into the search.

Choose a Number value for the setting, then input a 1 for dark or 0 for light.

Light and Dark Mode in Brave

If you’re using Brave, find color theme settings in Settings > Appearance > Brave colors.

How to Use Variable Scaling

You can also use a custom property to effortlessly adjust the size of text or other elements depending on your user’s screen size. The width media feature tests the width of the viewport.

While width: _px will match an exact size, you can also use min and max to create ranges.

Query with min-width: _px to match anything over _ pixels, and max-width: _px to match anything up to _ pixels.

Use these queries to set a custom property on the :root to create a ratio:

@media (min-width: 360px) {
    :root {
        --scale: 0.8;
    }
}

@media (min-width: 768px) {
    :root {
        --scale: 1;
    }
}

@media (min-width: 1024px) {
    :root {
        --scale: 1.2;
    }
}

Then make an element responsive by using the calc() function. Here are a few examples:

h1 {
    font-size: calc(42px * var(--scale));
}

h2 {
    font-size: calc(26px * var(--scale));
}

img {
    width: calc(200px * var(--scale));
}

In this example, multiplying an initial value by your --scale custom property allows the size of headings and images to magically adjust to your user’s device width.

The relative unit rem will have a similar effect. You can use it to define sizes for elements relative to the font size declared at the root element.

h1 {
    font-size: calc(5rem * var(--scale));
}

h2 {
    font-size: calc(1.5rem * var(--scale));
}

p {
    font-size: calc(1rem * var(--scale));
}

Of course, you can also multiply two custom properties. For example, setting the --max-img as a custom property on the :root can help to save you time later on by not having to update a pixel value in multiple places:

img {
    max-width: calc(var(--max-img) * var(--scale));
}

Raise your responsiveness game

Try out these easy wins for a website that caters to your visitor’s devices and preferences. I’ve put them to good use now on victoria.dev. I invite you to let me know how you like it!

If you enjoyed this post, there's a lot more where it came from. Subscribe on victoria.dev to see new articles first.

This skill was instrumental to the creation of the Internet. If no one had imagined the underlying technology that most now take for granted every day, there would be no cat memes.

To make the Internet possible, two things that needed imagining were layers and protocols.

Layers are conceptual divides that group similar functions together. The word “protocol,” means “the way we’ve agreed to do things around here,” more or less.

In short, both layers and protocols can be explained to a five-year-old as “ideas that people agreed sounded good, and then they wrote them down so that other people could do things with the same ideas.”

The Internet Protocol Suite is described in terms of layers and protocols. Collectively, the suite refers to the communication protocols that enable our endless scrolling.

It’s often called by its foundational protocols: the Transmission Control Protocol (TCP) and the Internet Protocol (IP). Lumped together as TCP/IP, these protocols describe how data on the Internet is packaged, addressed, sent, and received.

Here’s why the Internet Protocol Suite, or TCP/IP, is an imaginary rainbow layer cake.

Layers are imaginary

If you consider the general nature of a rainbow layer sponge cake, it’s mostly made up of soft, melt-in-your mouth vanilla-y goodness. This goodness is in itself comprised of something along the lines of eggs, butter, flour, and sweetener.

There isn’t much to distinguish one layer of a rainbow sponge cake from another. Often, the only difference between layers is the food-coloring and a bit of frosting. When you think about it, it’s all cake from top to bottom. The rainbow layers are only there because the baker thought they ought to be.

Similar to cake ingredients, layers in the context of computer networking are mostly composed of protocols, algorithms, and configurations, with some data sprinkled in.

It can be easier to talk about computer networking if its many functions are split up into groups, so certain people came up with descriptions of layers, which we call network models. TCP/IP is just one network model among others. In this sense, layers are concepts, not things.

Some of the people in question are part of the Internet Engineering Task Force (IETF). They created the RFC-1122 publication, discussing the Internet’s communications layers. Half of a whole, the standard:

…covers the communications protocol layers: link layer, IP layer, and transport layer; its companion RFC-1123 covers the application and support protocols.

The layers described by RFC-1122 and RFC-1123 each encapsulate protocols that satisfy the layer’s functionality. Let’s look at each of these communications layers and see how TCP and IP stack up in this model of the Internet layer cake.

Link layer protocols

The link layer is the most basic, or lowest-level, classification of communication protocol. It deals with sending information between hosts on the same local network, and translating data from the higher layers to the physical layer.

Protocols in the link layer describe how data interacts with the transmission medium, such as electronic signals sent over specific hardware. Unlike other layers, link layer protocols are dependent on the hardware being used.

Internet layer protocols

Protocols in the Internet layer describe how data is sent and received over the Internet. The process involves packaging data into packets, addressing and transmitting packets, and receiving incoming packets of data.

The most widely known protocol in this layer gives TCP/IP its last two letters. IP is a connectionless protocol, meaning that it provides no guarantee that packets are sent or received in the right order, along the same path, or even in their entirety.

Reliability is handled by other protocols in the suite, such as in the transport layer.

There are currently two versions of IP in use: IPv4, and IPv6. Both versions describe how devices on the Internet are assigned IP addresses, which are used when navigating to cat memes.

IPv4 is more widely used, but has only 32 bits for addressing, allowing for about 4.3 billion (ca. 4.3×10⁹) possible addresses. These are running out, and IPv4 will eventually suffer from address exhaustion as more and more people use more devices on the Internet.

The successor version IPv6 aims to solve address exhaustion by using 128 bits for addresses. This provides, um, a lot more address possibilities (ca. 3.4×10³⁸).

Transport layer protocols

In May 1974, Vint Cerf and Bob Kahn (collectively often called “the fathers of the Internet”) published a paper entitled A Protocol for Packet Network Intercommunication.

This paper contained the first description of a Transmission Control Program, a concept encompassing what would eventually be known as the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). (I had the pleasure of meeting Vint and can personally confirm that yes, he does look exactly like The Architect in the Matrix movies.)

The transport layer presently encapsulates TCP and UDP. Like IP, UDP is connectionless and can be used to prioritize time over reliability.

TCP, on the other hand, is a connection-oriented transport layer protocol that prioritizes reliability over latency, or time. TCP describes transferring data in the same order as it was sent, retransmitting lost packets, and controls affecting the rate of data transmission.

Application layer protocols

The application layer describes the protocols that software applications interact with most often. The specification includes descriptions of the remote login protocol Telnet, the File Transfer Protocol (FTP), and the Simple Mail Transfer Protocol (SMTP).

Also included in the application layer are the Hypertext Transfer Protocol (HTTP) and its successor, Hypertext Transfer Protocol Secure (HTTPS).

HTTPS is secured by Transport Layer Security, or TLS, which can be said to be the top-most layer of the networking model described by the Internet protocol suite.

If you’d like to further understand TLS and how this protocol secures your cat meme viewing, I invite you read my article about TLS and cryptography.

The Internet cake is still baking

Like a still-rising sponge cake, descriptions of layers, better protocols, and new models are being developed every day. The Internet, or whatever it will become in the future, is still in the process of being imagined.

If you enjoyed learning from this post, there’s a lot more where this came from! I write about computing, cybersecurity, and building great technical teams. Join the thousands of people who learn from my articles on victoria.dev! Visit and subscribe by email or RSS to see new articles first.

If you want to see it in action, you can now subscribe to my email list on victoria.dev.

Now, I'll show you how I built it.

Introducing Simple Subscribe

If you’re interested in managing your own mailing list or newsletter, you can set up Simple Subscribe on your own AWS resources to collect email addresses.

This open source API is written in Go, and runs on AWS Lambda. Visitors to your site can sign up to your list, which is stored in a DynamoDB table, ready to be queried or exported at your leisure.

When someone signs up, they’ll receive an email asking them to confirm their subscription. This is sometimes called “double opt-in,” although I prefer the term “verified.”

Simple Subscribe works on serverless infrastructure and uses an AWS Lambda to handle subscription, confirmation, and unsubscribe requests.

You can find the Simple Subscribe project, with its fully open-source code, on GitHub. I encourage you to pull up the code and follow along!

In this post I’ll share each build step, the thought process behind the API’s single-responsibility functions, and security considerations for an AWS project like this one.

How to build a verified subscription flow

A non-verified email sign up process is straightforward. Someone puts their email into a box on your website, then that email goes into your database.

However, if I’ve taught you anything about not trusting user input, the very idea of a non-verified sign up process should raise your hackles. Spam may be great when fried in a sandwich, but it's no fun when it’s running up your AWS bill.

While you can use a strategy like a CAPTCHA or puzzle for is-it-a-human verification, these can create enough friction to turn away your potential subscribers.

Instead, a confirmation email can help to ensure both address correctness and user sentience.

To build a subscription flow with email confirmation, create single-responsibility functions that satisfy each logical step. Those are:

1. Accept an email address and record it.
2. Generate a token associated with that email address and record it.
3. Send a confirmation email to that email address with the token.
4. Accept a verification request that has both the email address and token.

To achieve each of these goals, Simple Subscribe uses the official AWS SDK for Go to interact with DynamoDB and SES.

At each stage, consider what the data looks like and how you store it. This can help to handle conundrums like, “What happens if someone tries to subscribe twice?” or even threat-modeling such as, “What if someone subscribes with an email they don’t own?”

Ready? Let’s break down each step and see how the magic happens.

Subscribing

The subscription process begins with a humble web form, like the one on my site’s main page. A form input with attributes type="email" required helps with validation, thanks to the browser. When submitted, the form sends a GET request to the Simple Subscribe subscription endpoint.

Simple Subscribe receives a GET request to this endpoint with a query string containing the intended subscriber’s email. It then generates an id value and adds both email and id to your DynamoDB table.

The table item now looks like:

The confirm column, which holds a boolean, indicates that the item is a subscription request that has not yet been confirmed. To verify an email address in the database, you’ll need to find the correct item and change confirm to true.

As you work with your data, consider the goal of each manipulation and how you might compare an incoming request to existing data.

For example, if someone made a subsequent subscription request for the same email address, how would you handle it?

You might say, “Create a new line item with a new id." However, this might not be the best strategy when your serverless application database is paid for by request volume.

Since DynamoDB Pricing depends on how much data you read and write to your tables, it’s advantageous to avoid piling on excess data.

With that in mind, it would be prudent to handle subscription requests for the same email by performing an update instead of adding a new line.

Simple Subscribe actually uses the same function to either add or update a database item. This is typically referred to as, “update or insert.”

In a database like SQLite this is accomplished with the UPSERT syntax. In the case of DynamoDB, you use an update operation. For the Go SDK, its syntax is UpdateItem.

When a duplicate subscription request is received, the database item is matched on the email only. If an existing line item is found, its id and timestamp are overridden, which updates the existing database record and avoids flooding your table with duplicate requests.

How to verify email addresses

After submitting the form, the intended subscriber then receives an email from SES containing a link. This link is built using the email and id from the table, and takes the format:

<BASE_URL><VERIFY_PATH>/?email=subscriber@example.com&id=uuid-xxxxx

In this set up, the id is a UUID that acts as a secret token. It provides an identifier that you can match that is sufficiently complex and hard to guess. This approach deters people from subscribing with email addresses they don’t control.

Visiting the link sends a request to your verification endpoint with the email and id in the query string.

This time, it’s important to compare both the incoming email and id values to the database record. This verifies that the recipient of the confirmation email is initiating the request.

The verification endpoint ensures that these values match an item in your database, then performs another update operation to set confirm to true, and update the timestamp. The item now looks like:

How to query for emails

You can now query your table to build your email list. Depending on your email sending solution, you might do this manually, with another Lambda, or even from the command line.

Since data for requested subscriptions (where confirm is false) is stored in the table alongside confirmed subscriptions, it’s important to differentiate this data when querying for email addresses to send to. You’ll want to ensure you only return emails where confirm is true.

How to provide unsubscribe links

Similar to verifying an email address, Simple Subscribe uses email and id as arguments to the function that deletes an item from your DynamoDB table in order to unsubscribe an email address.

To allow people to remove themselves from your list, you’ll need to provide a URL in each email you send that includes their email and id as a query string to the unsubscribe endpoint. It would look something like:

<BASE_URL><UNSUBSCRIBE_PATH>/?email=subscriber@example.com&id=uuid-xxxxx

When the link is clicked, the query string is passed to the unsubscribe endpoint. If the provided email and id match a database item, that item will be deleted.

Proving a method for your subscribers to automatically remove themselves from your list, without any human intervention necessary, is part of an ethical and respectful philosophy towards handling the data that’s been entrusted to you.

How to care for your data

Once you decide to accept other people’s data, it becomes your responsibility to care for it. This is applicable to everything you build. For Simple Subscribe, it means maintaining the security of your database, and periodically pruning your table.

In order to avoid retaining email addresses where confirm is false past a certain time frame, it would be a good idea to set up a cleaning function that runs on a regular schedule. This can be achieved manually, with an AWS Lambda function, or using the command line.

To clean up, find database items where confirm is false and timestamp is older than a particular point in time. Depending on your use case and request volumes, the frequency at which you choose to clean up will vary.

Also depending on your use case, you may wish to keep backups of your data. If you are particularly concerned about data integrity, you can explore On-Demand Backup or Point-in-Time Recovery for DynamoDB.

Build your independent subscriber base

Building your own subscriber list can be an empowering endeavor! Whether you intend to start a newsletter, send out notifications for new content, or want to create a community around your work, there’s nothing more personal or direct than an email from me to you.

I encourage you to start building your subscriber base with Simple Subscribe today. Like most of my work, it’s open source and free for your personal use. Dive into the code at the GitHub repository or learn more at SimpleSubscribe.org.

If you enjoyed this post, I'd love to know. Join the thousands of people who learn along with me on victoria.dev. Visit or subscribe via RSS for more projects like this one.

A password is a password, so what’s the difference? About 60 seconds to billions of years, as it turns out.

All Wi-Fi encryption is not created equal. Let’s explore what makes these four acronyms so different, and how you can best protect your home and organization Wi-Fi.

Wired Equivalent Privacy (WEP)

In the beginning, there was WEP.

Wired Equivalent Privacy is a deprecated security algorithm from 1997 that was intended to provide equivalent security to a wired connection. “Deprecated” means, “Let’s not do that anymore.”

Even when it was first introduced, it was known not to be as strong as it could have been, for two reasons:

• its underlying encryption mechanism, and
• World War II.

During World War II, the impact of code breaking (or cryptanalysis) was huge. Governments reacted by attempting to keep their best secret-sauce recipes at home.

Around the time of WEP, U.S. Government restrictions on the export of cryptographic technology caused access point manufacturers to limit their devices to 64-bit encryption. Though this was later lifted to 128-bit, even this form of encryption offered a very limited possible key size.

This proved problematic for WEP. The small key size resulted in being easier to brute-force, especially when that key doesn’t often change.

WEP’s underlying encryption mechanism is the RC4 stream cipher. This cipher gained popularity due to its speed and simplicity, but that came at a cost.

It’s not the most robust algorithm. WEP employs a single shared key among its users that must be manually entered on an access point device. (When’s the last time you changed your Wi-Fi password? Right.)

WEP didn’t help matters either by simply concatenating the key with the initialization vector – which is to say, it sort of mashed its secret-sauce bits together and hoped for the best.

Initialization Vector (IV): fixed-size input to a low-level cryptographic algorithm, usually random.

Combined with the use of RC4, this left WEP particularly susceptible to related-key attack. In the case of 128-bit WEP, your Wi-Fi password can be cracked by publicly-available tools in a matter of around 60 seconds to three minutes.

While some devices came to offer 152-bit or 256-bit WEP variants, this failed to solve the fundamental problems of WEP’s underlying encryption mechanism.

So, yeah. Let’s not do that anymore.

Wi-Fi Protected Access (WPA)

A new, interim standard sought to temporarily “patch” the problem of WEP’s (lack of) security. The name Wi-Fi Protected Access (WPA) certainly sounds more secure, so that’s a good start. However, WPA first started out with another, more descriptive name.

Ratified in a 2004 IEEE standard, Temporal Key Integrity Protocol (TKIP) uses a dynamically-generated, per-packet key. Each packet sent has a unique temporal 128-bit key, (See? Descriptive!) that solves the susceptibility to related-key attacks brought on by WEP’s shared key mashing.

TKIP also implements other measures, such as a message authentication code (MAC). Sometimes known as a checksum, a MAC provides a cryptographic way to verify that messages haven’t been changed.

In TKIP, an invalid MAC can also trigger rekeying of the session key. If the access point receives an invalid MAC twice within a minute, the attempted intrusion can be countered by changing the key an attacker is trying to crack.

Unfortunately, in order to preserve compatibility with the existing hardware that WPA was meant to “patch,” TKIP retained the use of the same underlying encryption mechanism as WEP – the RC4 stream cipher.

While it certainly improved on the weaknesses of WEP, TKIP eventually proved vulnerable to new attacks that extended previous attacks on WEP.

These attacks take a little longer to execute by comparison: for example, twelve minutes in the case of one, and 52 hours in another. This is more than sufficient, however, to deem TKIP no longer secure.

WPA, or TKIP, has since been deprecated as well. So let’s also not do that anymore.

Which brings us to…

Wi-Fi Protected Access II (WPA2)

Rather than spend the effort to come up with an entirely new name, the improved Wi-Fi Protected Access II (WPA2) standard instead focuses on using a new underlying cipher.

Instead of the RC4 stream cipher, WPA2 employs a block cipher called Advanced Encryption Standard (AES) to form the basis of its encryption protocol.

The protocol itself, abbreviated CCMP, draws most of its security from the length of its rather long name (I’m kidding): Counter Mode Cipher Block Chaining Message Authentication Code Protocol, which shortens to Counter Mode CBC-MAC Protocol, or CCM mode Protocol, or CCMP. ?

CCM mode is essentially a combination of a few good ideas. It provides data confidentiality through CTR mode, or counter mode. To vastly oversimplify, this adds complexity to plaintext data by encrypting the successive values of a count sequence that does not repeat.

CCM also integrates CBC-MAC, a block cipher method for constructing a MAC.

AES itself is on good footing. The AES specification was established in 2001 by the U.S. National Institute of Standards and Technology (NIST). They made their choice after a five-year competitive selection process during which fifteen proposals for algorithm designs were evaluated.

As a result of this process, a family of ciphers called Rijndael (Dutch) was selected, and a subset of these became AES.

For the better part of two decades, AES has been used to protect every-day Internet traffic as well as certain levels of classified information in the U.S. Government.

While possible attacks on AES have been described, none have yet been proven to be practical in real-world use. The fastest attack on AES in public knowledge is a key-recovery attack that improved on brute-forcing AES by a factor of about four. How long would it take? Some billions of years.

Wi-Fi Protected Access III (WPA3)

The next installment of the WPA trilogy has been required for new devices since July 1, 2020. Expected to further enhance the security of WPA2, the WPA3 standard seeks to improve password security by being more resilient to word list or dictionary attacks.

Unlike its predecessors, WPA3 will also offer forward secrecy. This adds the considerable benefit of protecting previously exchanged information even if a long-term secret key is compromised.

Forward secrecy is already provided by protocols like TLS by using asymmetric keys to establish shared keys. You can learn more about TLS in this post.

As WPA2 has not been deprecated, so both WPA2 and WPA3 remain your top choices for Wi-Fi security.

If the other ones are no good, why are they still around?

You may be wondering why your access point even allows you to choose an option other than WPA2 or WPA3. The likely reason is that you’re using legacy hardware, which is what tech people call your mom’s router.

Since the deprecation of WEP and WPA occurred rather recently, it’s possible in large organizations as well as your parent’s house to find older hardware that still uses these protocols. Even newer hardware may have a business need to support these older protocols.

While I may be able to convince you to invest in a shiny new top-of-the-line Wi-Fi appliance, most organizations are a different story. Unfortunately, many just aren’t yet cognizant of the important role cybersecurity plays in meeting customer needs and boosting that bottom line.

Additionally, switching to newer protocols may require new internal hardware or firmware upgrades. Especially on complex systems in large organizations, upgrading devices can be financially or strategically difficult.

Boost your Wi-Fi security

If it’s an option, choose WPA2 or WPA3. Cybersecurity is a field that evolves by the day, and getting stuck in the past can have dire consequences.

If you can’t use WPA2 or WPA3, do the best you can to take additional security measures.

The best bang for your buck is to use a Virtual Private Network (VPN). Using a VPN is a good idea no matter which type of Wi-Fi encryption you have. On open Wi-Fi (coffee shops) and using WEP, it’s plain irresponsible to go without a VPN.

It's kind of like shouting out your bank details as you order your second cappuccino.

Choose a VPN provider that offers a feature like a kill switch that blocks your network traffic if your VPN becomes disconnected. This prevents you from accidentally transmitting information on an insecure connection like open Wi-Fi or WEP. I wrote more about my top three considerations for choosing my VPN in this post.

When possible, ensure you only connect to known networks that you or your organization control.

Many cybersecurity attacks are executed when victims connect to an imitation public Wi-Fi access point, also called an evil twin attack, or Wi-Fi phishing.

These fake hotspots are easily created using publicly accessible programs and tools. A VPN can help mitigate damage from these attacks as well, but it’s always better not to take the risk.

If you travel often, consider purchasing a portable hotspot that uses a cellular data plan, or using data SIM cards for all your devices.

Much more than just acronyms

WEP, WPA, WPA2, and WPA3 mean a lot more than a bunch of similar letters – in some cases, it’s a difference of billions of years minus about 60 seconds.

On more of a now-ish timescale, I hope I’ve taught you something new about the security of your Wi-Fi and how you can improve it!

If you enjoyed this post, I'd love to know. Join the thousands of people who learn along with me on victoria.dev! Visit or subscribe via RSS for more programming, cybersecurity, and cartoon dad jokes.

The Django framework in particular offers your team the opportunity to create an efficient testing practice, based on the Python standard library unittest.

Proper tests in Django are fast to write, faster to run, and can offer you a seamless continuous integration solution for taking the pulse of your developing application.

With comprehensive tests, developers have higher confidence when pushing changes. I’ve seen firsthand in my own teams that good tests can boost development velocity as a direct result of a better developer experience.

In this article, I’ll share my own experiences in building useful tests for Django applications, from the basics to the best possible execution. If you're using Django or building with it in your organization, you might like to read my Django series on Victoria.dev.

What to test

Tests are extremely important. Far beyond simply letting you know if a function works, tests can form the basis of your team’s understanding of how your application is intended to work.

Here’s the main goal: if you hit your head and forgot everything about how your application works tomorrow, you should be able to regain most of your understanding by reading and running the tests you write today.

Here are some questions that may be helpful to ask as you decide what to test:

• What is your customer supposed to be able to do?
• What is your customer not supposed to be able to do?
• What should this method, view, or logical flow achieve?
• When, how, or where is this feature supposed to execute?

Tests that make sense for your application can help build developer confidence.

With these sensible safeguards in place, developers make improvements more readily, and feel confident introducing innovative solutions to product needs. The result is an application that comes together faster, and features that are shipped often and with confidence.

A cartoon created by a programmer and a mathematician, if you couldn't tell.

Where to put tests

If you only have a few tests, you may organize your test files similarly to Django’s default app template by putting them all in a file called tests.py. This straightforward approach is best for smaller applications.

As your application grows, you may like to split your tests into different files, or test modules. One method is to use a directory to organize your files, such as projectroot/app/tests/. The name of each test file within that directory should begin with test, for example, test_models.py.

Besides being aptly named, Django will find these files using built-in test discovery based on the unittest module. All files in your application with names that begin with test will be collected into a test suite.

This convenient test discovery allows you to place test files anywhere that makes sense for your application. As long as they’re correctly named, Django’s test utility can find and run them.

How to document a test

Use docstrings to explain what a test is intended to verify at a high level. For example:

def test_create_user(self):
    """Creating a new user object should also create an associated profile object"""
    # ...

These docstrings help you quickly understand what a test is supposed to be doing. Besides navigating the codebase, this helps to make it obvious when a test doesn’t verify what the docstring says it should.

Docstrings are also shown when the tests are being run, which can be helpful for logging and debugging.

What a test needs to work

Django tests can be quickly set up using data created in the setUpTestData() method. You can use various approaches to create your test data, such as utilizing external files, or even hard-coding silly phrases or the names of your staff.

Personally, I much prefer to use a fake-data-generation library, such as faker.

The proper set up of arbitrary testing data can help you ensure that you’re testing your application functionality instead of accidentally testing test data. Because generators like faker add some degree of unexpectedness to your inputs, it can be more representative of real-world use.

Here is an example set up for a test:

from django.test import TestCase
from faker import Faker

from app.models import MyModel, AnotherModel

fake = Faker()


class MyModelTest(TestCase):
    def setUpTestData(cls):
        """Quickly set up data for the whole TestCase"""
        cls.user_first = fake.first_name()
        cls.user_last = fake.last_name()

    def test_create_models(self):
        """Creating a MyModel object should also create AnotherModel object"""
        # In test methods, use the variables created above
        test_object = MyModel.objects.create(
            first_name=self.user_first,
            last_name=self.user_last,
            # ...
        )
        another_model = AnotherModel.objects.get(my_model=test_object)
        self.assertEqual(another_model.first_name, self.user_first)
        # ...

Tests pass or fail based on the outcome of the assertion methods. You can use Python’s unittest methods, and Django’s assertion methods.

For further guidance on writing tests, see Testing in Django.

Best possible execution for running your tests

Django’s test suite is manually run with:

./manage.py test

I rarely run my Django tests this way.

The best, or most efficient, testing practice is one that occurs without you or your developers ever thinking, “I need to run the tests first.”

The beauty of Django’s near-effortless test suite set up is that it can be seamlessly run as a part of regular developer activities. This could be in a pre-commit hook, or in a continuous integration or deployment workflow.

I’ve previously written about how to use pre-commit hooks to improve your developer ergonomics and save your team some brainpower. Django’s speedy tests can be run this way, and they become especially efficient if you can run tests in parallel.

Tests that run as part of a CI/CD workflow, for example, on pull requests with GitHub Actions, require no regular effort from your developers to remember to run tests at all. I’m not sure how plainly I can put it – this one’s literally a no-brainer.

Testing your way to a great Django application

Tests are extremely important, and under-appreciated. They can catch logical errors in your application. They can help explain and validate how concepts and features of your product actually function. Best of all, tests can boost developer confidence and development velocity as a result.

The best tests are ones that are relevant, help to explain and define your application, and are run continuously without a second thought. I hope I’ve now shown you how testing in Django can help you to achieve these goals for your team!

I’ve been developing with Django for years, and I’ve never been happier with my Django project set up than I am right now.

In this article, I'll explain how I make a day of developing with Django the most relaxing and enjoyable development experience possible for myself and my engineering team.

A custom CLI tool for your Django project

Instead of typing:

python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
python3 manage.py makemigrations
python3 manage.py migrate
python3 manage.py collectstatic
python3 manage.py runserver

Wouldn’t it be much nicer to type:

make start

…and have all that happen for you? I think so!

We can do that with a self-documenting Makefile. Here’s one I frequently use when developing my Django applications, like ApplyByAPI.com:

SHELL := /bin/bash

include .env

.PHONY: help
help: ## Show this help
    @egrep -h '\s##\s' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

.PHONY: venv
venv: ## Make a new virtual environment
    python3 -m venv $(VENV) && source $(BIN)/activate

.PHONY: install
install: venv ## Make venv and install requirements
    $(BIN)/pip install -r requirements.txt

migrate: ## Make and run migrations
    $(PYTHON) manage.py makemigrations
    $(PYTHON) manage.py migrate

db-up: ## Pull and start the Docker Postgres container in the background
    docker pull postgres
    docker-compose up -d

db-shell: ## Access the Postgres Docker database interactively with psql
    docker exec -it container_name psql -d $(DBNAME)

.PHONY: test
test: ## Run tests
    $(PYTHON) $(APP_DIR)/manage.py test application --verbosity=0 --parallel --failfast

.PHONY: run
run: ## Run the Django server
    $(PYTHON) $(APP_DIR)/manage.py runserver

start: install migrate run ## Install requirements, apply migrations, then start development server

You’ll notice the presence of the line include .env above. This ensures make has access to environment variables stored in a file called .env.

This allows Make to utilize these variables in its commands, for example, the name of my virtual environment, or to pass in $(DBNAME) to psql.

What’s with that weird “##” comment syntax? A Makefile like this gives you a handy suite of command-line aliases you can check in to your Django project. It’s very useful so long as you’re able to remember what all those aliases are.

The help command above, which runs by default, prints a helpful list of available commands when you run make or make help:

help                 Show this help
venv                 Make a new virtual environment
install              Make venv and install requirements
migrate              Make and run migrations
db-up                Pull and start the Docker Postgres container in the background
db-shell             Access the Postgres Docker database interactively with psql
test                 Run tests
run                  Run the Django server
start                Install requirements, apply migrations, then start development server

All the usual Django commands are covered, and we’ve got a test command that runs our tests with the options we prefer. Brilliant.

You can read my full post about self-documenting Makefiles here, which also includes an example Makefile using pipenv.

Save your brainpower with pre-commit hooks

I previously wrote about some technical ergonomics that can make it a lot easier for teams to develop great software.

One area that’s a no-brainer is using pre-commit hooks to lint code prior to checking it in.

This helps maintain the quality of the code your developers check in. But most importantly, it ensures that no one on your team is spending time trying to remember if it should be single or double quotes or where to put a line break.

The confusingly-named pre-commit framework is an otherwise fantastic way to keep hooks (which are not included in cloned repositories) consistent across local environments.

Here is my configuration file, .pre-commit-config.yaml, for my Django projects:

fail_fast: true
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v3.1.0
    hooks:
      - id: detect-aws-credentials
  - repo: https://github.com/psf/black
    rev: 19.3b0
    hooks:
      - id: black
  - repo: https://github.com/asottile/blacken-docs
    rev: v1.7.0
    hooks:
      - id: blacken-docs
        additional_dependencies: [black==19.3b0]
  - repo: local
    hooks:
      - id: markdownlint
        name: markdownlint
        description: "Lint Markdown files"
        entry: markdownlint '**/*.md' --fix --ignore node_modules --config "./.markdownlint.json"
        language: node
        types: [markdown]

These hooks check for accidental secret commits, format Python files using Black, format Python snippets in Markdown files using blacken-docs, and lint Markdown files as well.

There are likely even more useful hooks available for your particular use case: see supported hooks to explore.

Useful gitignores

An under-appreciated way to improve your team’s daily development experience is to make sure your project uses a well-rounded .gitignore file.

This can help prevent files containing secrets from being committed, and can additionally save developers hours of tedium by ensuring you’re never sifting through a git diff of generated files.

To efficiently create a gitignore for Python and Django projects, Toptal’s gitignore.io can be a nice resource for generating a robust .gitignore file.

I still recommend examining the generated results yourself to ensure that ignored files suit your use case, and that nothing you want ignored is commented out.

Continuous testing with GitHub Actions

If your team works on GitHub, setting up a testing process with Actions is low-hanging fruit.

Tests that run in a consistent environment on every pull request can help eliminate “works on my machine” conundrums. They can also help ensure that no one’s sitting around waiting for a test to run locally.

A hosted CI environment like GitHub Actions can also help when running integration tests that require using managed services resources.

You can use encrypted secrets in a repository to grant the Actions runner access to resources in a testing environment, without worrying about creating testing resources and access keys for each of your developers to use.

I’ve written on many occasions about setting up Actions workflows, including using one to run your Makefile, and how to integrate GitHub event data. GitHub even interviewed me about Actions once.

For Django projects, here’s a simple GitHub Actions workflow that runs tests with a consistent Python version.

name: Run Django tests

on: push

jobs:
  test:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.8'
      - name: Install dependencies
        run: make install
      - name: Run tests
        run: make test

For the installation and test commands, I’ve simply utilized the Makefile that’s been checked in to the repository.

A benefit of using your Makefile commands in your CI test workflows is that you only need to keep them updated in one place – your Makefile! No more “why is this working locally but not in CI??!?” headaches.

If you want to step up your security game, you can add Django Security Check as an Action too.

Set up your Django project for success

Want to help keep your development team happy? Set them up for success with these best practices for Django development.

Remember, an ounce of brainpower is worth a pound of software.

Successful applications that are growing are a lovely problem to have. As a product develops, it tends to accumulate complication the way your weekend cake project accumulates layers of frosting.

Thankfully, Django, my favorite batteries-included framework, handles complexity pretty well.

Django models help humans work with data in a way that makes sense to our brains. And the framework offers plenty of classes you can inherit to help you rapidly develop a robust application from scratch.

As for developing on existing Django applications, there’s a feature for that, too.

In this article, we’ll cover how to use Django migrations to update your existing models and database.

What’s under the hood

Django migrations are Python files that help you add and change things in your database tables to reflect changes in your Django models.

To understand how Django migrations help you work with data, it may be helpful to understand the underlying structures we’re working with.

What’s a database table?

If you’ve laid eyes on a spreadsheet before, you’re already most of the way to understanding a database table.

In a relational database, for example, a PostgreSQL database, you can expect to see data organized into columns and rows. A relational database table may have a set number of columns and any number of rows.

In Django, each model is its own table. For example, here’s a Django model:

from django.db import models


class Lunch(models.Model):
    left_side = models.CharField(max_length=100, null=True)
    center = models.CharField(max_length=100, null=True)
    right_side = models.CharField(max_length=100, null=True)

Each field is a column, and each row is a Django object instance of that model.

Here’s a representation of a database table for the Django model “Lunch” above. In the database, its name would be lunch_table.

The model Lunch has three fields: left_side, center, and right-side. One instance of a Lunch object would have “Fork” for the left_side, a “Plate” for the center, and “Spoon” for the right_side.

Django automatically adds an id field if you don’t specify a primary key.

If you wanted to change the name of your Lunch model, you would do so in your models.py code.

For example, change “Lunch” to “Dinner,” then run python manage.py makemigrations. You’ll see:

python manage.py makemigrations
Did you rename the backend.Lunch model to Dinner? [y/N] y
Migrations for 'backend':
  backend/migrations/0003_auto_20200922_2331.py
    - Rename model Lunch to Dinner

Django automatically generates the appropriate migration files. The relevant line of the generated migrations file in this case would look like:

migrations.RenameModel(old_name="Lunch", new_name="Dinner"),

This operation would rename our “Lunch” model to “Dinner” while keeping everything else the same.

But what if you also wanted to change the structure of the database table itself, its schema, as well as make sure that existing data ends up in the right place on your Dinner table?

Terrible cartoon by author.

Let’s explore how to turn our Lunch model into a Dinner model that looks like this:

from django.db import models


class Dinner(models.Model):
    top_left = models.CharField(max_length=100, null=True)
    top_center = models.CharField(max_length=100, null=True)
    top_right = models.CharField(max_length=100, null=True)
    bottom_left = models.CharField(max_length=100, null=True)
    bottom_center = models.CharField(max_length=100, null=True)
    bottom_right = models.CharField(max_length=100, null=True)

…with a database table that would look like this:

How to manipulate data with Django migrations

Before you begin to manipulate your data, it’s always a good idea to create a backup of your database that you can restore in case something goes wrong.

There are various ways to do this depending on the database you’re using. You can typically find instructions by searching for <your database name> and keywords like backup, recovery, or snapshot.

In order to design your migration, it’s helpful to become familiar with the available migration operations.

Migrations are run step-by-step, and each operation is some flavor of adding, removing, or altering data. Like a strategic puzzle, it’s important to make model changes one step at a time so that the generated migrations have the correct result.

We’ve already renamed our model successfully. Now, we’ll rename the fields that hold the data we want to retain:

class Dinner(models.Model):
    bottom_left = models.CharField(max_length=100, null=True)
    bottom_center = models.CharField(max_length=100, null=True)
    top_center = models.CharField(max_length=100, null=True)

Django is sometimes smart enough to determine the old and new field names correctly. You’ll be asked for confirmation:

python manage.py makemigrations
Did you rename dinner.center to dinner.bottom_center (a CharField)? [y/N] y
Did you rename dinner.left_side to dinner.bottom_left (a CharField)? [y/N] y
Did you rename dinner.right_side to dinner.top_center (a CharField)? [y/N] y
Migrations for 'backend':
  backend/migrations/0004_auto_20200914_2345.py
    - Rename field center on dinner to bottom_center
    - Rename field left_side on dinner to bottom_left
    - Rename field right_side on dinner to top_center

In some cases, you’ll want to try renaming the field and running makemigrations one at a time.

Now that the existing fields have been migrated to their new names, add the remaining fields to the model:

class Dinner(models.Model):
    top_left = models.CharField(max_length=100, null=True)
    top_center = models.CharField(max_length=100, null=True)
    top_right = models.CharField(max_length=100, null=True)
    bottom_left = models.CharField(max_length=100, null=True)
    bottom_center = models.CharField(max_length=100, null=True)
    bottom_right = models.CharField(max_length=100, null=True)

Running makemigrations again now gives us:

python manage.py makemigrations
Migrations for 'backend':
  backend/migrations/0005_auto_20200914_2351.py
    - Add field bottom_right to dinner
    - Add field top_left to dinner
    - Add field top_right to dinner

You’re done! By generating Django migrations, you’ve successfully set up your dinner_table and moved existing data to its new spot.

Additional complexity

You’ll notice that our Lunch and Dinner models are not very complex. Out of Django’s many model field options, we’re just using CharField. We also set null=True to let Django store empty values as NULL in the database.

Django migrations can handle additional complexity, such as changing field types, and whether a blank or null value is permitted. I keep Django’s model field reference handy as I work with varying types of data and different use cases.

De-mystified migrations

I hope this article has helped you better understand Django migrations and how they work!

Now that you can change models and manipulate existing data in your Django application, be sure to use your powers wisely.

Backup your database, research and plan your migrations, and always run tests before working with customer data. By doing so, you have the potential to enable your application to grow – with manageable levels of complexity.

But if you want to send data confidentially over the Internet, you might have a few more considerations to cover.

TLS, or Transport Layer Security, refers to a protocol. "Protocol" is a word that means, "the way we've agreed to do things around here," more or less.

The "transport layer" part of TLS simply refers to host-to-host communication, such as how a client and a server interact, in the Internet protocol suite model.

The TLS protocol attempts to solve these fundamental problems:

• How do I know you are who you say you are?
• How do I know this message from you hasn't been tampered with?
• How can we communicate securely?

Here's how TLS works, explained in plain English. As with many successful interactions, it begins with a handshake.

Getting to know you

The basic process of a TLS handshake involves a client, such as your web browser, and a server, such as one hosting a website, establishing some ground rules for communication.

It begins with the client saying hello. Literally. It's called a ClientHello message.

The ClientHello message tells the server which TLS protocol version and cipher suites it supports.

While "cipher suite" sounds like a fancy hotel upgrade, it just refers to a set of algorithms that can be used to secure communications.

The server, in a similarly named ServerHello message, chooses the protocol version and cipher suite to use from the choices offered. Other data may also be sent, for example, a session ID, if the server supports resuming a previous handshake.

Cartoon of a browser window and server saying hello, by author.

Depending on the cipher suite chosen, the client and server exchange further information in order to establish a shared secret.

Often, this process moves the exchange from asymmetric cryptography to symmetric cryptography with varying levels of complexity. Let's explore these concepts at a general level and see why they matter to TLS.

Asymmetric beginnings

This is asymmetry:

Small egg, big egg.

Asymmetric cryptography is one method by which you can perform authentication. When you authenticate yourself, you answer the fundamental question, "How do I know you are who you say you are?"

In an asymmetric cryptographic system, you use a pair of keys in order to achieve authentication. These keys are asymmetric. One key is your public key, which, as you would guess, is public. The other is your private key, which – well, you know.

Typically, during the TLS handshake, the server will provide its public key via its digital certificate, sometimes still called its SSL certificate, though TLS replaces the deprecated Secure Sockets Layer (SSL) protocol.

Digital certificates are provided and verified by trusted third parties known as Certificate Authorities (CA), which are a whole other article in themselves.

While anyone may encrypt a message using your public key, only your private key can then decrypt that message.

The security of asymmetric cryptography relies only on your private key staying private, hence the asymmetry.

It's also asymmetric in the sense that it's a one-way trip. Alice can send messages encrypted with your public key to you, but neither of your keys will help you send an encrypted message to Alice.

Symmetric secrets

Asymmetric cryptography also requires more computational resources than symmetric cryptography.

Thus when a TLS handshake begins with an asymmetric exchange, the client and server will use this initial communication to establish a shared secret, sometimes called a session key. This key is symmetric, meaning that both parties use the same shared secret and must maintain that secrecy for the encryption to be secure.

Wise person say: share your public key, but keep your shared keys private.

By using the initial asymmetric communication to establish a session key, the client and server can rely on the session key being known only to them. For the rest of the session, they'll both use this same shared key to encrypt and decrypt messages, which speeds up communication.

Secure sessions

A TLS handshake may use asymmetric cryptography or other cipher suites to establish the shared session key. Once the session key is established, the handshaking portion is complete and the session begins.

The session is the duration of encrypted communication between the client and server. During this time, messages are encrypted and decrypted using the session key that only the client and server have. This ensures that communication is secure.

The integrity of exchanged information is maintained by using a checksum. Messages exchanged using session keys have a message authentication code (MAC) attached. This is not the same thing as your device's MAC address. The MAC is generated and verified using the session key.

Because of this, either party can detect if a message has been changed before being received. This solves the fundamental question, "How do I know this message from you hasn't been tampered with?"

Sessions can end deliberately, due to network disconnection, or from the client staying idle for too long. Once a session ends, it must be re-established via a new handshake or through previously established secrets called session IDs that allow resuming a session.

TLS and you

Let's recap:

• TLS is a cryptographic protocol for providing secure communication.
• The process of creating a secure connection begins with a handshake.
• The handshake establishes a shared session key that is then used to secure messages and provide message integrity.
• Sessions are temporary, and once ended, must be re-established or resumed.

This is just a surface-level skim of the very complex cryptographic systems that help to keep your communications secure. For more depth on the topic, I recommend exploring cipher suites and the various supported algorithms.

The TLS protocol serves a very important purpose in your everyday life. It helps to secure your emails to family, your online banking activities, and the connection by which you're reading this article.

The HTTPS communication protocol is encrypted using TLS. Every time you see that little lock icon in your URL bar, you're experiencing firsthand all the concepts you've just read about in this article.

So now you know the answer to the last question: "How can we communicate securely?"

While a multitude of methods exist to search for and replace words in a single file, what do you do when you’ve got a string to update across multiple unrelated files, all with different names? You harness the power of command line tools, of course!

First, you’ll need to find all the files you want to change. Stringing together what are effectively search queries for find is really only limited by your imagination.

Here’s a simple example that finds Python files:

find . -name '*.py'

The -name test searches for a pattern, such as all files ending in .py. But find can do a lot more with other test conditions, including -regex tests. Run find --help to see the multitude of options.

Further tune your search by using grep to get only the files that contain the string you want to change, such as by adding:

grep -le '\<a whale\>'

The -l option gives you just the file names for all files containing a pattern (denoted with -e) that match “a whale”.

You can also use Vim’s impressive :bufdo which lets you run the same command across multiple buffers. It interactively works with all of these files without the tedium of opening, saving, and closing each file, one at a time.

Let’s plug your powerful find+grep results into Vim with:

vim `find . -name '*.py' \
-exec grep -le '\<a whale\>' {} \;`

Using backtick-expansion to pass our search to Vim opens up multiple buffers ready to go. (Do :h backtick-expansion in Vim for more.)

Now you can apply the Vim command :bufdo to all of these files and perform actions such as interactive search-and-replace:

:bufdo %s/a whale/a bowl of petunias/gce

The g for “global” will change occurrences of the pattern on all lines. The e will omit errors if the pattern is not found. The c option makes this interactive. If you’re feeling confident, you can omit it to make the changes without reviewing each one.

When you’ve finished going through all the buffers, save all the work you’ve completed with:

:bufdo wq!

Then bask in the glory of your saved time and effort.

The most recent integration between Visual Studio Code and GitHub really helps make development accessible and welcoming.

Now in beta, GitHub Codespaces provide an online, in-the-browser IDE powered by Visual Studio Code.

This lets you use this full-featured IDE, complete with extensions, terminal, Git commands, and all the settings you’re accustomed to, on any machine. You can now bring your development workflow anywhere using a tablet or other browser-based device.

Codespaces is great news for open source contributors, too. Adding a codespace configuration to your project is a great way to invite new folks to easily start contributing.

A new open source contributor or new hire at your organization can quickly fire up a codespace and get hacking on a good first issue with no local environment set up or installations necessary.

We’ve added codespace configuration settings over at the OWASP Web Security Testing Guide (WSTG). Want to take it for a spin? See our open issues.

Configuring Codespaces

You can use Visual Studio Code’s .devcontainer folder to configure a development container for your repository as well.

Many pre-built containers are available – just copy the .devcontainer you need to your repository root. If your repository doesn’t have one, a default base Linux image will be used.

Here’s a reason to remove .vscode from your .gitignore file. Any new codespaces created in your repository will now respect settings found at .vscode/settings.json. This means that your online IDE can have the same Workspace configuration as you have on your local machine. Isn’t that useful!

Making Codespaces Personal

For next-level dotfiles personalization, consider committing relevant files from your local dotfiles folder as a public GitHub repository at yourusername/dotfiles.

When you create a new codespace, this brings in your configurations, such as shell aliases and preferences, by creating symlinks to dotfiles in your codespace $HOME. This personalizes all the codespaces you create in your account.

Need some inspiration? Browse my dotfiles repository on GitHub.

Developing in Codespaces

Developing in a codespace is a familiar experience for Visual Studio Code users, right down to running an application locally.

Thanks to port forwarding, when I run an application in a codespace terminal, clicking on the resulting localhost URL takes me to the appropriate port as output from my codespace.

When I’m working on my blog in my codespace, for example, I run hugo serve then click the provided localhost:1313 link to see a preview of my changes in another browser tab.

Want to stay in sync between devices? There’s an extension for that. You can connect to your codespace from Visual Studio Code on your local machine so you can always pick up right where you left off.

Develop Anywhere

Codespaces is a super exciting addition to my GitHub workflow. It allows me to access my full development process pretty much anywhere, using devices like my iPad.

It’ll also make it easier for new open source contributors or new hires at your organization to hit the ground running with a set-up IDE.

If you have access to the limited beta, I invite you to spin up a codespace and try contributing to the WSTG, or to an issue on one of my open source projects.

I’m looking forward to general availability and seeing what the open source community will dream up for GitHub Codespaces next!

And yes – codespaces support your favorite Visual Studio Code theme. ?

Screenshot of a codespace with the [Kabukichō](https://marketplace.visualstudio.com/items?itemName=VictoriaDrake.kabukicho" rel="noopener) theme for Visual Studio Code

Can a Makefile improve your DevOps and keep developers happy? How awesome would it be if a new developer working on your project didn’t start out by copying and pasting commands from your README? What if instead of:

pip3 install pipenv
pipenv shell --python 3.8
pipenv install --dev
npm install
pre-commit install --install-hooks
# look up how to install Framework X...
# copy and paste from README...
npm run serve

… you could just type:

make start

…and then start working?

Making a difference

I use make every day to take the tedium out of common development activities like updating programs, installing dependencies, and testing.

To do all this with a Makefile (GNU make), we use Makefile rules and recipes. Similar parallels exist for POSIX flavor make, like Target Rules. Here’s a great article on POSIX-compatible Makefiles.

Here’s some examples of things we can make easier (sorry):

update: ## Do apt upgrade and autoremove
    sudo apt update && sudo apt upgrade -y
    sudo apt autoremove -y

env:
    pip3 install pipenv
    pipenv shell --python 3.8

install: ## Install or update dependencies
    pipenv install --dev
    npm install
    pre-commit install --install-hooks

serve: ## Run the local development server
    hugo serve --enableGitInfo --disableFastRender --environment development

initial: update env install serve ## Install tools and start development server

Now we have some command-line aliases that you can check in. Great idea! If you’re wondering what’s up with that weird ## comment syntax, it gets better.

A self-documenting Makefile

Aliases are great, if you remember what they all are and what they do without constantly typing cat Makefile. Naturally, you need a help command:

.PHONY: help
help: ## Show this help
    @egrep -h '\s##\s' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

With a little command-line magic, this egrep command takes the output of MAKEFILE_LIST, sorts it, and uses awk to find strings that follow the ## pattern. It then prints a helpful formatted version of the comments.

We’ll put it at the top of the file so it’s the default target. Now to see all our handy shortcuts and what they do, we just run make, or make help:

help                 Show this help
initial              Install tools and start development server
install              Install or update dependencies
serve                Run the local development server
update               Do apt upgrade and autoremove

Now we have our very own personalized and project-specific CLI tool!

The possibilities for improving your DevOps flow with a self-documenting Makefile are almost endless. You can use one to simplify any workflow and produce some very happy developers.