This absence of direction makes the journey significantly harder. Faced with an overwhelming abundance of online resources, many beginners become confused about where to start and often attempt to learn everything at once.

This is where the struggle with knowledge retention begins.

Not because they lack intelligence or effort, but because they're learning in a way that contradicts how the human brain actually works.

They dive into tutorials and courses without understanding the mechanism of the brain – that is, how the brain processes, stores, and retrieves information. As a result, much of what they learn simply doesn’t stick.

How the Brain Processes Information

So what's the connection between the brain and learning how to code, you might ponder?

The connection is direct and unavoidable.

Coding isn't learned through willpower or motivation — though both matter — or by spending countless hours watching tutorials.

It's learned through the brain’s ability to process, store, and retrieve information.

Every variable, function, data structure, or debugging pattern must pass through the brain’s cognitive systems before it becomes usable knowledge.

If your learning process doesn't align with how the brain naturally acquires and organises information, your retention will collapse, no matter how determined you are.

Now imagine you’re trying to fill a bucket with water. You keep pouring and pouring, but the bucket has tiny holes at the bottom. No matter how much effort you put in, the water keeps leaking out. You might blame yourself for not pouring fast enough, or you might try switching to a bigger jug, but the real problem isn’t your effort — it’s the bucket.

The water is the information you’re trying to learn.

The bucket is your brain’s memory system.

The holes in the bucket are the natural forgetting mechanisms of the brain: cognitive overload, limited working memory, and other constraints that make retention difficult.

If you don’t understand these mechanisms, you can pour in as much information as you want, but most of it will leak out.

Not because you’re incapable, but because you’re learning in a way that contradicts how the brain actually retains knowledge.

The Role of Academic Learning Theories

Since learning ultimately takes place in the brain, an important question is: How does the human brain acquire, organize, and apply knowledge and why does the typical self‑taught learning process clash with these principles?

This is where academic learning theories become indispensable. These frameworks explain how the brain actually acquires, retains, and applies complex information and they offer a scientific roadmap for learning more effectively. Without understanding these principles, self‑taught developers unintentionally work against the brain’s natural architecture.

The purpose of this article is to unpack these essential learning theories and apply them directly to the beginner self‑taught developer’s journey.

By understanding how the brain processes information, beginners can structure their learning more intentionally, retain knowledge more reliably, and move toward becoming competent developers with far greater confidence and clarity.

Table of Contents

• Cognitive Load Theory (CLT)

  • Working Memory

  • Miller's Law

  • Chunking

  • Long-term memory

  • Schema

  • Intrinsic Load: The Natural Difficulty of the Task

  • Extraneous Load: The Mental Noise

  • Germane Load: The Construction Work

• Ebbinghaus Forgetting Curve.

• How the Theory of Spaced Repetition Works

• Theory of Deliberate Practice

• What is Bloom's Taxonomy?

  • Focused Mode vs Diffuse Mode

• Conclusion

• References

Cognitive Load Theory (CLT)

Learning a new concept often requires mental effort by the brain to process newly acquired information. This effort exerted by the brain is known as cognitive load, a term coined by Australian educational psychologist John Sweller in 1988 during his study on how the brain acquires and retains information (Sweller, 1988).

Since then, his work has been expanded upon by other researchers. Notably, Dylan Wiliam famously tweeted in 2017 that Cognitive Load Theory (CLT) is "the single most important thing for teachers to know" (Dylan William, 2017).

You might wonder again: What does this have to do with me? As a beginner self‑taught developer, the answer is simple: you're both the teacher and the student.

So this is the most important theory you should know. In this self-tutoring journey, you're tasked with designing your own curriculum, choosing your own resources, pace your own learning, and evaluating your own progress.

Without understanding how cognitive load affects your ability to absorb and retain information, you may unintentionally overload your brain and sabotage your own learning.

Before we get into the nitty-gritty of CLT, there are important concepts masterminded by David Geary that you'll need to grasp to sufficiently understand this concept : “that which can be learnt” (biologically primary knowledge), “that which can be taught”(biologically secondary knowledge) (Geary, 2007, 2008).

According Geary (2007, 2008), "biologically primary knowledge" consists of "instinctual" skills that the brain is evolved to pick up naturally without formal schooling.

Examples include learning a first language, recognizing faces, or basic social navigation.

"Biologically secondary knowledge", on the other hand, consists of cultural and technical skills, like reading and writing, that are necessary for society but don't come naturally to the brain.

This is because we aren't "wired" to pick these up automatically. Instead, they require formal instruction and schools to pass them down.

Therefore, coding is a prime example of biologically secondary knowledge. The human brain is remarkably plastic, but it didn't evolve to interpret syntax, manage memory allocation, or debug logical loops.

These are cultural inventions, not natural instincts. Unlike learning to walk or speak your native language (which are biologically primary skills) you can't learn to code simply by “being around” computers.

Recognising that the human brain is not instinctively prepared for coding allows you to change your strategy. Once you accept that coding concepts are not “natural,” you can finally approach them with the structured, deliberate effort they require.

The second set of concepts beginner self-taught developers should know and understand are working memory, Miller's Laws, chunking, long-term memory, and schemas.

Working Memory

Working memory is where thinking happens. It's the active mental workspace where you hold information while you process it. When you encounter concepts like syntax, loops, functions, or an if/elseif statement for the first time, all of that information sits inside your working memory. The problem is that working memory is extremely limited and fragile.

When you first learn to code, your working memory functions like a small mental desk where only a few items can be placed at once.

Imagine trying to assemble a piece of IKEA furniture on a tiny coffee table. If you spread out the instruction manual, the screws, the wooden panels, and the tools all at the same time, the table becomes cluttered instantly. You start losing track of which part goes where, not because you’re incapable, but because the surface you’re working on is too small to hold everything at once.

Working memory behaves the same way. When you’re learning new concepts – like arrays, loops, functions, or error handling – each idea takes up space on that mental desk. If you then overload it, the desk becomes overcrowded.

Once it exceeds its capacity, things begin to fall off, and your ability to retain collapses.

It’s not a lack of intelligence. It’s simply the natural limit of working memory.

Now this collapse happens because you went against the threshold your working memory can hold. This is backed up by research that shows that working memory can typically process only 5–9 pieces of information at any given time (Miller, 1956). This is known as Miller’s Law.

Miller's Law

In 1956, George Miller found that the average human can hold about seven items (plus or minus two) in working memory at once, even some recent research has stated the number is even lower about four item (Nelson Cowan, 2001).

So imagine you encounter a tutorial that introduces the following concepts all at the same time: a Route, a Controller, a Model, a Migration, a View, a Request, Helper files, Jobs and Queues, Middleware, Roles and Permissions, and a Service Provider. If you attempt to hold all of these in your mind simultaneously, you'll inevitably hit Miller’s Wall, as your working memory becomes overloaded, and you'll likely forget the first concept long before you reach the last.

So how do you handle complex tasks if the brain can only juggle 4–9 items at once?

You use chunking — the process of grouping small pieces of information into a single, meaningful unit.

Chunking

Chunking is the brain’s strategy for compressing complexity. Instead of forcing working memory to hold a dozen unrelated items, you reorganise them into a few coherent structures. This reduces cognitive load, prevents overload, and allows you to work with far more information than your raw working‑memory limits would normally allow.

Let's consider an example:

A beginner learning Laravel might see Route, Controller, Model, Migration, and View as five separate, overwhelming items. To a beginner, each one feels like a distinct cognitive burden. But an experienced developer doesn't treat them as isolated concepts. Instead, they're understood as a single meaningful unit: the MVC pattern. Instead of holding five items in working memory, the expert holds one.

This raises an important question: how does a beginner know that these five elements belong together when they have only just encountered them?

It's crucial to emphasise that chunking isn't automatic. It depends on recognising meaningful relationships between concepts, and beginners typically lack the prior knowledge needed to perceive those relationships early on.

But as learners repeatedly encounter the same sequence during the learning process, they begin to notice consistent patterns. Over time, the brain’s natural tendency to seek structure enables them to identify which components reliably operate together, allowing these elements to gradually fuse into a single, meaningful chunk.

For example, when I first followed a Laravel e-commerce tutorial, I noticed that for every new resource the tutor created – Payment, Cart, KYC, and Contact – the same pattern was repeated: a Controller, a Model, and a View were always created together.

After encountering this sequence several times, it became clear that these components consistently belonged together as a set. Over time, I began to perceive the Controller, Model, and View not as separate elements, but as a single, integrated unit.

So beginners may not be able to chunk effectively on day one because they lack the prior knowledge needed to recognise what belongs together. But with time, and repeated encounters across different contexts, these individual pieces fuse into stable mental units stored in long‑term memory.

What feels overwhelming at first eventually becomes effortless, not because the task became simpler, but because your internal representation became more organised.

This is the power of chunking: it transforms scattered pieces of information into organised units that fit comfortably within the limits of working memory.

Without chunking, beginners drown in details. With chunking, they gain the cognitive space needed to understand, retain, and apply what they learn.

Long-term memory

Unlike working memory, long‑term memory has virtually infinite capacity. The goal of all study is to move information from the cramped working memory into the vast long‑term memory.

Here is the real secret: you don’t learn in working memory – you only process there.

True learning is the permanent change that happens in long‑term memory.

Schema

Once stored in long-term memory, information becomes part of a schema — a mental map or filing system that organizes related ideas.

For example, when you finally learn that Laravel is an MVC framework, you aren’t just memorizing three letters. You're building a schema that tells your brain: Models handle data, Views handle presentation, and Controllers handle logic.

Once a schema is built, it can be pulled into working memory as a single chunk, effectively bypassing Miller’s Law.

This is how experts think effortlessly while beginners feel overwhelmed.

And this is why Garnett (2020) argues that "being competent or lacking competence in something depends entirely on how secure the retrieval of knowledge held in the schema is".

Now that the foundations of working memory, long‑term memory, schemas, and chunking are clear, we can turn to another set of concepts every self‑taught developer must understand: intrinsic load, extraneous load, and germane load. These three components make up the full structure of Cognitive Load Theory, and they determine whether learning feels manageable or overwhelming.

Intrinsic Load: The Natural Difficulty of the Task

Intrinsic load refers to the inherent complexity of the material itself. Some concepts are simply harder than others because they contain more interacting elements that must be processed at the same time.

In Laravel, understanding a simple Route has low intrinsic load.

But concepts like Dependency Injection or Polymorphic Relationships have high intrinsic load because they involve multiple layers of abstraction and interdependent ideas.

You can't change the intrinsic load of a concept, but you can manage it by breaking the idea into smaller, more digestible sub‑tasks. This is why good teaching — and good self‑teaching — always begins with simplification and sequencing.

Simplification means stripping a concept down to its essential parts so the learner isn't overwhelmed by unnecessary detail.

Sequencing means introducing parts in a logical order, where each step builds on the previous one. This helps reduce unnecessary cognitive load and allows learners to devote more mental effort (germane load) to building schemas.

It’s like meeting someone new, and they tell you their name and it happens to be your mother’s name. Instantly, your brain forms a connection. You associate this new person’s name with the strong, deeply stored memory of your mother.

Because that schema already exists in your long-term memory, the new information “attaches” to it. Later, when you try to recall the name, you don’t struggle, you simply think of your mother, and the name comes back easily.

While many believe self‑taught developers struggle because they lack immediate, reliable, personal guidance, there's actually a hidden advantage in this predicament. When a teacher explains a concept, even if they try their best to “chunk” the information, they can't truly know the student’s internal limits — how much intrinsic load the learner can handle, how quickly they can process new ideas, or how much prior knowledge they can activate.

This is where self‑taught developer quietly shines. Because you are both the teacher and the student, you know your own cognitive limits better than anyone else. You can slow down when something feels heavy, pause when working memory is overloaded, and chunk information in a way that perfectly matches your personal capacity.

You can simplify a concept to its bare essentials and sequence it at a pace that aligns with your own understanding.

Extraneous Load: The Mental Noise

Extraneous load is the enemy of the self‑taught developer. It's the mental effort wasted on tasks that don't contribute to actual learning. This is where a self-taught developer's strength must truly shine.

A teacher in a classroom is responsible for removing any distractions that might derail a child or slow down their assimilation of knowledge. As a self-taught developer, that responsibility falls entirely on you. You must identify these distractions and eliminate them.

As a self-taught developer myself, I use specific strategies to ensure I stay focused. Before starting any course, I spend time reading the comment section to see what others have experienced. If I see complaints about low audio quality, unclear explanations, or tutorials that move too fast, I immediately abandon that course and look for one with better reviews.

Anything that might derail my progress must be removed. If you spend half of your mental energy trying to figure out all of these, you only have the remaining half available for understanding the logic of the code. And remember: when learning new concepts, we use working memory, which is fragile.

As your own “inner teacher,” you must eliminate this noise so your limited working memory can focus entirely on the material that matters.

Germane Load: The Construction Work

Germane load is the productive mental effort used to build and refine schemas — the mental structures that make future learning easier.

This is the “Aha!” moment when new information connects meaningfully to what you already know.

For example, germane load appears when you realise that a Database Migration is essentially a version‑control system for your table structure.

That insight is schema construction in action.

Teachers are often advised to help manage a child's germane load. One way they do this is by connecting the new idea being taught to an existing concept.

By doing this, they help the student build schemas: mental frameworks that organise and interpret information.

For a self-taught developer, this means instead of memorizing a new syntax in isolation, you look for a 'hook' in something you already understand.

For example, if you already know how a physical filing cabinet works, understanding Arrays or Objects in code becomes much easier.

You aren't learning from scratch – you're simply "plugging" new data into an old socket. This reduces the mental strain and makes the new knowledge stick permanently.

But this can only happen when intrinsic load is properly managed and extraneous load is removed.

It's important to note that, unlike intrinsic and extraneous load, germane load isn't an independent type of cognitive load.

Instead, it represents the portion of your working memory that remains available to handle the element interactivity associated with intrinsic load.

In other words, germane load is the mental energy you have left for learning once the unnecessary noise is stripped away.

Understanding cognitive load explains why learning can feel overwhelming in the moment, but it doesn't explain why knowledge fades after the moment has passed. For that, we turn to another foundational principle in learning science: the Ebbinghaus Forgetting Curve.

Ebbinghaus Forgetting Curve

If you remember the bucket analogy, this curve represents one of the holes at the bottom — the brain’s natural tendency to let information leak away unless it's reinforced.

In the late 19th century, Hermann Ebbinghaus discovered that human memory follows a predictable pattern of decline. After learning something new, we forget most of it astonishingly quickly — often within hours — unless the information is revisited. The forgetting curve shows that memory retention drops sharply at first and then continues to decline more slowly over time.

Studies based on Ebbinghaus’ forgetting curve found that without a conscious effort to retain newly acquired information, we lose approximately 50% of new information within 24 hours, and up to 90% within a week (Clearwater, 2024).

In other words, the brain is designed to discard information that isn't reinforced.

For self‑taught developers, this has profound implications.

You may understand a Laravel controller today, spatial roles and permission concepts, and so on – but if you don't revisit it, practice it, or apply it within the next few hours, your brain will naturally let it fade.

This is not a sign of weakness or lack of talent. It's simply how human brain works.

The forgetting curve also explains why tutorials feel deceptively easy the moment you're going through them.

While watching, everything seems clear — but a week later, the same concepts feel unfamiliar.

The knowledge never made it into long‑term memory because you didn't revisit, practice, or connect it to existing schemas.

Since the human brain is designed to forget anything that isn't repeated, repetition becomes the signal that tells the brain, “This matters — keep it.”

This is why, when you meet someone for the first time and they tell you their name, you'll almost certainly forget it unless you consciously repeat it to yourself several times. If you don’t reinforce it, you end up asking — often with embarrassment — “Sorry, what was your name again?”

The same principle applies to learning code: without deliberate repetition, the brain simply lets the information fade. However, with a technique called spaced repetition, retention is significantly improved .

How the Theory of Spaced Repetition Works

Spaced repetition is a learning technique grounded in cognitive psychology that involves reviewing information at increasingly spaced intervals to strengthen long‑term memory retention.

It's based on the principle that memory decays predictably over time — as demonstrated by the Ebbinghaus Forgetting Curve — and that strategically timed reviews or repetition interrupt this decay, making the memory more durable with each repetition.

This idea is what gave birth to Anki-Flash cards.

Imagine you're trying to memorise the time complexity of different algorithms.

This is a classic "dry" academic topic that's easy to forget.

To understand why spaced repetition is so powerful, consider a familiar scenario. You spend Sunday night staring at a chart of Big‑O complexities for four hours. By Monday’s review, you can recall most of them. By Friday, only a few remain. Two weeks later, the entire chart has vanished from memory.

Spaced repetition reverses this process by reviewing information at the precise moment it's about to be forgotten. Instead of cramming Big‑O notation in a single session, you revisit it across expanding intervals:

1. Day 1 (Initial Learning): You study the Big‑O chart and understand each complexity class.

2. Day 2 (First Review): You test yourself. If you recall an item correctly, you schedule the next review three days later. If you miss it, you review it again the following day.

3. Day 5 (Second Review): You encounter the material again. Because you still remember it, the interval expands to ten days.

4. Day 15 (Third Review): Your memory has begun to fade, but the moment you see the prompt, the concept resurfaces. This slight struggle to retrieve the information is precisely what strengthens long‑term retention.

5. Day 45 (Fourth Review): By now, the memory is deeply consolidated. Concepts like O(log⁡n) feel as natural and accessible as your own phone number.

Through this process, spaced repetition transforms fragile, short‑term awareness into durable, long‑term knowledge. Each review interrupts the forgetting curve, reinforces the schema, and reduces the cognitive load required to recall the concept in the future.

For self-taught developers, spaced repetition can take many forms. You might rewrite code from memory, re‑implement a feature days later, build small variations of the same concept, or return to the concept after working on different tasks.

Every review strengthens the schema and reduces the cognitive load required to recall it. Over time, what once felt complex becomes automatic — not because the concept changed, but because your brain reorganised it into a stable, efficient structure.

As you can see, learning isn't a single event but a cycle of exposure, forgetting, and reinforcement.

Mastery comes not from seeing something once, but from returning to it until it becomes part of your cognitive architecture.

But we must be careful with repetition. Doing the same thing over and over again doesn't guarantee improvement. In fact, mindless repetition can trap you at the same level indefinitely.

This is where the theory of deliberate practice becomes essential, as it emphasises increasing the level of challenge, focusing on specific weaknesses, and actively seeking feedback so that each repetition leads to measurable improvement rather than just familiarity.

Theory of Deliberate Practice

Developed by psychologist K. Anders Ericsson, it argues that expertise is not the result of talent but of high‑quality, intentional practice (Ericsson, 1993). This type of practice is fundamentally different from simply doing something repeatedly.

He coined the term "deliberate practice" while researching how people become experts. Studying experts from several different fields, he dismantled the myth that expert performers have unusual innate talents.

Instead, he discovered that experts attain their high performance through how they practice: it's a deliberate effort to become an expert. This effort is characterized by breaking down required skills into smaller parts and practicing these parts repeatedly.

According to Anders Ericsson, Deliberate Practice requires:

1. Clear goals

2. Immediate feedback

3. Tasks that stretch your ability just beyond your comfort zone

4. Full concentration and effort

The main tenet of Deliberate Practice is that tasks must stretch your ability just beyond your comfort zone. This is paramount to the advancement of learning.

Imagine a child being taught 1+1 every day. That child will never grow beyond basic arithmetic. Anders Ericsson calls this "Arrested Development" (Ericsson, Nandagopal and Roring, 2005). For that child to grow to become a mathematician, their knowledge must be stretched.

The takeaway for developers is a play on the DRY principle (Don’t Repeat Yourself): If you are only repeating what you already know without stretching yourself, you aren't growing. This "stretch" is the extra edge that Deliberate Practice adds to Spaced Repetition.

Building a simple to-do list, a calculator, or a weather app over and over again won't take you anywhere. You already know how to do those.

To truly grow, you must stretch yourself. Instead, try a project that integrates new ideas, like building a mini-app where the weather data affects your to-do list. For example, if the API shows it's raining, the app automatically hides outdoor tasks and calculates the time you'll save or the indoor tasks you should prioritize instead.

This forces you to handle complex logic and state management, moving you beyond simple repetition into true mastery.

This ability to create brings me to the last theory: Bloom's Taxonomy.

What is Bloom's Taxonomy?

Bloom’s Taxonomy provides a hierarchy of cognitive skills that learners move through as they develop mastery. It begins with the simplest tasks and progresses toward the most complex:

1. Remember – recalling facts or syntax

2. Understand – explaining concepts in your own words

3. Apply – using knowledge in real situations

4. Analyze – breaking problems into parts

5. Evaluate – judging solutions or comparing approaches

6. Create – building original systems or applications

Most self‑taught developers get stuck in the first two levels. They memorize syntax and understand examples, but they struggle to apply, analyze, or create.

This isn't because they lack ability. Rather, it's because they haven't been taught that learning must progress through these stages.

Bloom’s Taxonomy gives structure to the learning journey.

It reminds self-taught developers that mastery isn't achieved by watching tutorials but by climbing the ladder from remembering → understanding → applying → analyzing → evaluating → creating (with an emphasis on Creation).

Creation is one of the most difficult yet most transformative experiences in your journey as a developer. It forces you to think abstractly, confront ambiguity, and notice dimensions of a problem that tutorials rarely reveal.

When you build something real, the neatness of the theory in your head collapses, and you begin to see its true complexity. You must then devise strategies to navigate these challenges, and through this process, you learn.

And as with anything worthwhile, the process isn't smooth. You'll encounter bugs — not just one or two, but hundreds. Yet this is precisely how real knowledge is built. Every bug you solve becomes a permanent entry in your long‑term memory.

The next time you see that error, you won’t panic. Instead, you’ll recognise it instantly and know exactly where it’s coming from and how to fix it.

Some self‑taught developers encounter a few bugs and never return to their projects again, concluding that “coding isn’t for me.”

After trying several fixes and seeing no progress, they abandon the work and look for something else. But this is the wrong conclusion. The problem is rarely a lack of talent — it's a misunderstanding of how the brain behaves under cognitive strain.

Focused Mode vs Diffuse Mode

When you spend a long time wrestling with a bug, you may be experiencing mental fixation or functional fixedness.

This is when your brain becomes locked into a single line of reasoning, repeating the same logic path over and over because it feels like the right direction. The longer you stare at the problem, the deeper the cognitive rut becomes. You develop tunnel vision, making it almost impossible to see alternative solutions.

This is where understanding how the brain operates becomes essential.

According to Oakley (2014), the brain works in two primary modes:

1. Focused Mode: Ideal for executing a known formula or following a clear procedure, terrible for discovering a new approach or breaking out of a mental rut.

2. Diffuse Mode: This is activated when you step away — walking, showering, relaxing, or sleeping.

In this second mode, the brain enters a “big‑picture” state where neural connections stretch across different regions.

The background processes continue working on the problem without the restrictive tunnel vision of conscious focus.

This phenomenon is known as incubation.

This is why solutions often appear when you’re not actively thinking about the problem. You step away, and suddenly the answer emerges, not because you stopped working, but because a different part of your brain started working for you.

The reality is that many developers never allow for incubation. While you step away from the problem, your brain performs subconscious synthesis: it clears out the noise (Extraneous Load) and lets the core logic (Germane Load) settle. When you return, the “wrong” paths you were obsessing over have faded, and the correct path which was there all along often finally becomes visible.

This is why developers must deliberately allow for incubation. We can take some lessons from great minds of the past:

Henri Poincaré famously struggled with Fuchsian functions for weeks. It was only during a geological excursion when he had completely forgotten about the mathematics that the solution appeared with “perfect certainty” the moment he stepped onto an omnibus. His breakthrough did not come from more effort, but from stepping away long enough for diffuse mode to take over.

Friedrich August Kekulé experienced something similar after years of wondering why benzene’s carbon atoms didn't fit a linear structure.

If some of the greatest minds in history stepped away from their problems and found solutions in diffuse mode, why should developers treat themselves any differently?

Now that you're familiar with some key learning strategies – Cognitive Load Theory, Spaced Repetition, and Bloom's Taxonomy – creating or building a project from the ground up should be your next task. It will help you curate, retrieve, organise, and seal in all that diverse knowledge you've gathered as a self-taught developer.

Conclusion

In this article, we explored why the human brain isn't instinctively wired to understand programming. Coding is a biologically secondary skill, which means it doesn't develop naturally through immersion but requires explicit instruction, structure, and patience.

We also talked about the limits of working memory, the importance of chunking, and the need to manage cognitive load so that learning remains possible rather than overwhelming.

We then analyzed the three components of Cognitive Load Theory – intrinsic, extraneous, and germane load – and discussed how each influences the learning process. Reducing extraneous load is especially crucial for self‑taught developers, as it frees up mental resources for meaningful understanding.

From there, we turned to the Ebbinghaus Forgetting Curve, which demonstrates how quickly newly learned information fades without reinforcement.

To counter this natural forgetting, we introduced Spaced Repetition, a method that strengthens memory by reviewing material at expanding intervals. We also examined Deliberate Practice, which pushes learners just beyond their comfort zone to promote genuine skill development, and Bloom’s Taxonomy, which outlines the stages of cognitive growth from remembering to creating.

Finally, we emphasized the importance of knowing when to step back. The brain operates in both focused and diffuse modes, and effective learning requires movement between the two. Breaks are not signs of weakness but essential components of consolidation and insight.

Together, these theories form a comprehensive framework for learning to code with scientific precision. When self‑taught developers understand how their brain learns, forgets, and grows, they can design a learning process that isn't only more efficient but far more sustainable.

With all this new knowledge, one truth is certain: focus, determination, and consistency are the forces that transform theory into mastery.

Learning science can guide the process, but only sustained effort turns knowledge into skill.

References

1. Clearwater, L. (2024). Understanding the Science Behind Learning Retention | Reports | What We Think | Indegene. [online] www.indegene.com. Available at: https://www.indegene.com/what-we-think/reports/understanding-science-behind-learning-retention.

2. Dylan Wiliam [@dylanwiliam]. (2017, January 25). I’ve come to the conclusion Sweller’s Cognitive Load Theory is the single most important thing for teachers to know [Tweet]. X. https://x.com/dylanwiliam/status/824682504602943489

3. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993).
   The role of deliberate practice in the acquisition of expert performance.
   Psychological Review, 100(3), 363–406.

4. Garnett, S. (2020.). Cognitive Load Theory A handbook for teachers. [online] Available at: https://www.crownhouse.co.uk/assets/look-inside/9781785835018.pdf.

5. Geary, D. C. (2007). An evolutionary perspective on learning disability in mathematics. Developmental Neuropsychology, 32(1), 471–519. https://doi.org/10.1080/87565640701360924

6. Geary, D. C. (2008). An evolutionarily informed education science. Educational Psychologist, 43(4), 179–195. https://doi.org/10.1080/00461520802392133

7. George A. Miller (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81–97. https://doi.org/10.1037/h0043158

8. Nelson Cowan (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87–114. https://doi.org/10.1017/S0140525X01003922

9. Oakley, B. (2014). A Mind for Numbers: How to Excel at Math and Science (Even If You Flunked Algebra). New York: TarcherPerigee.

10. Sweller, J. (1988). Cognitive Load during Problem Solving: Effects on Learning. Cognitive Science, [online] 12(2), pp.257–285. doi:https://doi.org/10.1207/s15516709cog1202_4.

‌

‌

You’ve been grinding through tutorials. You've built a portfolio site, maybe deployed a few projects on GitHub. And now you're trying to land a job or join a team.

Then the interviews start.

Suddenly, people ask:

• "Are you familiar with Agile?"

• "Have you worked in a Scrum environment?"

• "What’s your experience with sprints?"

Cue the imposter syndrome. Because no one teaches this stuff in JavaScript 101.

This guide is for you.

I’ll help make the Scrum process – a very common way developers work together – make actual sense. I’ll walk you through the basics, but also tell you what developers actually do, how standups feel when you're new, and what’s expected of you when you’re no longer coding in a vacuum.

Let’s break it down.

Here’s what we’ll cover:

• What Even Is Scrum?

• The Three Roles in Scrum (and Who Does What)

• The Scrum Rhythm: What a Sprint Actually Looks Like

• Who attends the Ceremonies:

• Standups: Where You Talk Like a Human, Not a Robot

• Sprint Planning

• What’s a User Story and Why Does It Sound Like a Children’s Book?

• What Counts as “Done”? Definition of Done and Why It’s Important

• Demos, Retros, and Saying the Hard Things

• Tools You Might Encounter

• If You’re Preparing for a Job, Here’s What You Can Do

• Final Thoughts

What Even Is Scrum?

Scrum is not a tool. It’s not a software. It’s not some elite thing only PMs care about.

It’s a lightweight framework that helps software teams build things incrementally, together, in short focused cycles called sprints.

Scrum is used by everyone from FAANG teams to indie dev shops because it helps:

• Keep teams aligned

• Deliver working software fast

• Course-correct often

• Spot problems early (before they go nuclear)

It’s the opposite of the old-school “build for a year and pray it works” model.

The Three Roles in Scrum (and Who Does What)

Scrum officially defines three roles. Here's what that means in practice:

1. Product Owner (PO)

Think: Vision-holder. They decide what the team builds and why. A product owner:

• Writes user stories (think of these as feature requests written from a user’s point of view)

• Prioritizes the work

• Clarifies what success looks like

• Says “yes” or “not yet” to features

2. Scrum Master (SM)

Think: Air-traffic controller meets therapist. They make sure the process works. The are master Facilitators, like between Dev and PO’s. A Scrum Master:

• Facilitates meetings

• Removes blockers (“Your AWS access is stuck? I’ll escalate it.”)

• Coaches the team on Scrum practices

• Doesn’t manage people – manages flow

3. Developers (YOU!)

Think: Builders. You write code, test it, ship it, fix it, and improve it. You also:

• Break down stories into tasks

• Pick up work from the team board (like Jira or Trello)

• Communicate progress

• Demo what you’ve built at the end of the sprint

You might also work with designers, testers, or DevOps folks – but within Scrum, you’re all “developers” building a product together.

The Scrum Rhythm: What a Sprint Actually Looks Like

Image Source: https://www.invensislearning.com/blog/what-are-scrum-ceremonies/

Understanding the Scrum Cycle

So, what does it actually look like when a team uses Scrum to build software?

Let’s walk through a full sprint – not just the buzzwords, but what really happens when a group of humans tries to plan, build, review, and improve together. Think of this as your backstage pass to the rhythm of modern teamwork.

📦 Step 1: Build and Refine the Product Backlog

Before any coding starts, the team needs to agree on what they might build – not just this week, but in the near future too.

That’s where the Product Backlog comes in. This is a big, running list of everything the product might need – features, bug fixes, improvements, ideas, and maybe a few wild dreams. It’s like the wishlist for the product, but more organized (ideally).

The Product Owner is responsible for maintaining and prioritizing this list. They decide what’s most important to work on based on customer needs, business goals, and feedback.

But the PO doesn’t do this in isolation. Enter the Backlog Refinement meeting.

In these sessions, the Scrum Team – that’s the PO, the Scrum Master (SM), and the Developers – come together to:

• Review the most important upcoming items

• Clarify any vague or confusing parts of each task

• Break big items down into smaller, buildable pieces called user stories

• Estimate effort (how much time or complexity is involved for each story)

This meeting makes sure the team isn’t caught off guard in the sprint – that they understand the work ahead and can actually start sprinting when the time comes.

🧭 Step 2: Sprint Planning – What Are We Building This Time?

Now that we’ve got a solid backlog, it’s time to pick what to build right now.

At the start of each sprint (which typically lasts 1 to 4 weeks), the team holds a Sprint Planning meeting. This meeting sets the stage for the entire sprint – it’s like the huddle before the big game.

In Sprint Planning, the team:

• Reviews the top items from the backlog

• Discusses what can realistically be completed based on their availability and capacity

• Chooses a handful of these stories to commit to

• Defines a Sprint Goal – a simple statement that captures the purpose of this sprint

For example, the Sprint Goal might be:
🎯 “Allow users to reset their passwords.”

Every user story chosen should contribute to that goal. The collection of these stories becomes the Sprint Backlog – basically, the to-do list for the sprint.

So when we say:

“The team selects an ordered list of user stories to comprise the Sprint Backlog for the next sprint, which will be achievable to satisfy the Sprint Goal...”

We’re really just saying:
👉 “Pick a realistic number of important tasks that, if completed, will help us hit our target for the sprint.”

Not too vague. Not too ambitious. Just achievable and focused.

☀️ Step 3: Daily Standups – Stay in Sync

Now the sprint is underway! But how does everyone stay aligned and avoid working in silos?

That’s where the Daily Standup comes in. Every day – usually in the morning – the team has a quick check-in (about 15 minutes) where each person answers three questions:

1. What did I do yesterday?

2. What am I working on today?

3. Is anything blocking me? (that is, am I stuck?)

Example:

“Yesterday I set up the login API integration. Today I’ll work on the UI validation. I’m blocked on getting access to the staging database — may need help.”

These standups keep the team in sync and surface blockers early so they can be addressed quickly. They’re not about micromanaging or showing off. They’re about visibility and support.

📉 What’s a Sprint Burndown Chart?

You might hear your team mention a “burndown chart.” No, this isn’t about things going down in flames (hopefully).

A Sprint Burndown Chart is a graph that shows how much work is left in the sprint – day by day.

• The y-axis is the amount of work remaining (often measured in story points or tasks)

• The x-axis is the number of days left in the sprint

The line should ideally trend downward as work gets completed – hence “burning down.” If it flattens out or slopes up, that’s a red flag that the team might be stuck, behind schedule, or not updating the board.

Think of it as a visual heartbeat of the sprint. You can learn more via a practical example in this video.

🖥️ Step 4: Sprint Review – Show What You’ve Built

At the end of the sprint, the team holds a Sprint Review (also called a “demo”). This is where you show what was actually built during the sprint.

• The Developers demo working features – live, not just screenshots

• The Product Owner reviews whether the Sprint Goal was achieved

• Stakeholders may ask questions, give feedback, or suggest tweaks

This meeting isn’t just for show – it’s a feedback loop. It helps the team validate that what they built is useful, usable, and meets expectations. If changes are needed, those get added to the backlog for future sprints.

🔍 Step 5: Sprint Retrospective – Look Back to Move Forward

Once the review is done, the team shifts focus from what they built to how they worked together.

Enter the Sprint Retrospective – a meeting to reflect on the process, not the product.

The team discusses:

• ✅ What went well

• ❌ What didn’t go so well

• 🔁 What could be improved next time

This isn’t about pointing fingers. It’s about learning, adapting, and continuously improving how the team collaborates.

The Scrum Master often facilitates this meeting and helps turn feedback into action items for the next sprint. For example:

“We underestimated testing time. Next sprint, let’s budget for QA earlier.”

The best teams take retros seriously. Why? Because even if your code is perfect, your process needs tuning too – and small process changes often lead to big gains.

♻️ Scrum Is a Loop

Here’s the rhythm:

1. Plan the sprint

2. Check in daily

3. Build and demo the product

4. Reflect and improve

Then do it all over again – with slightly better coordination and slightly more trust each time.

It’s not about being fast. It’s about being intentional, consistent, and collaborative.

Example Sprint

Let’s say, for example, that your team does 4-week sprints. (Keep in mind that Sprints can differ by team, nature of product, release cycles, and so on.)

Here’s the rough beat:

Scrum works in loops. Every 2-4 weeks (depending on your cadence and sprint cycle), your team should have working, demo-able software to show for it – even if it’s small.

And no, it’s not about “speed.” It’s about consistency, communication, and collaboration.

Who attends the Ceremonies:

Now lets dive deeper and understand practically how each of these ceremonies work:

Standups: Where You Talk Like a Human, Not a Robot

So how does the team actually stay connected day to day? That’s where standups come in.

Every morning, your team meets briefly – usually on Zoom or in a circle – and you answer 3 questions:

1. What did I work on yesterday?

2. What will I work on today?

3. What’s blocking me? Any impediments?

Example:

"Yesterday I cleaned up the signup validation logic. Today I’m working on the email verification flow. I’m stuck on SendGrid config – might need help setting up credentials."

It’s not about impressing anyone. It’s about keeping everyone in sync. Some days you’ll say, “I spent the whole day debugging a CSS bug that turned out to be a semicolon.” That’s okay.

How does it work?

The Scrum Master gathers everyone in a huddle room, the PO and Dev Team included, and opens the the Standup. They are the facilitator of the ceremony. Everyone gets a chance to answer the 3 questions above (usually about 2-5 minutes each). It’s not a full report – it’s quick. When one person is done, they pass it on to someone else.

This ensures there is team cohesion and transparency.

Here is a video example of a standup.

Sprint Planning

The goal of the planning meeting is to answer the questions “What are we going to work on, and how are we going to do it?” It is critical for the team to have a shared goal and a shared commitment to this goal before beginning this ceremony.

Participants should:

• Measure growth

• Sync with the Scrum Master

• Sync with the Product Owner

Sprint planning happens just before the sprint starts, and usually lasts for an hour or two. In this meeting, the team goes over a collection of user stories and discuss, plan, measure, and prioritize. This is where they decide what is going to be in scope for their upcoming sprint cycle.

The Product Owner will have a prioritized view of things in the backlog. They work with the team on each object or customer experience. Together, as a group they go through and make calculations, deciding to what they can commit.

What’s a User Story and Why Does It Sound Like a Children’s Book?

So you might be wondering: how do you know what to work on? What to build? So much work, so little time? Thats where user stories come in.

In Scrum, teams don’t just write vague tasks like “code the login.” Instead, they write user stories – short, human-centered feature descriptions that describe what the user needs, why they need it, and what success looks like.

Here’s an example:

As a user, I want to be able to reset my password, so I can access my account if I forget it.

User stories are the scaffolding of teamwork. They’re written with empathy, not just efficiency. And each one comes with acceptance criteria – a checklist that clarifies what “done” actually means:

• A “Forgot Password” link is visible

• Clicking it shows a form

• An email gets sent with a reset link

Once a story is agreed upon, developers break it down into tasks, like “build form,” “hook into backend,” or “handle email validation.” It’s collaborative, not prescriptive. And user stories have priority so you know what’s the most important and what’s the least.

A helpful rule of thumb many teams use is the Gherkin-style "Given–When–Then" format:

• Given some initial context

• When an event occurs

• Then a specific outcome should happen

This ensures that everyone – devs, testers, and product owners – shares the same understanding of behavior and expectations.

Here is a great video example thats outlines how to draft effective and powerful user stories.

What Counts as “Done”? Definition of Done and Why It’s Important

Now you might be wondering – how do I know when a task is done and can be closed out?

The Definition of Done is a type of documentation in the form of a team agreement. The Definition of Done identifies the conditions that need to be achieved in order for the product to be considered done (as in potentially shippable).

This is how we know that we "did the thing right". Meaning, we built the correct level of quality into the product. The Definition of Done is not the same as the acceptance criteria, which are written by the product owner to help us know we did the "right thing".

Every team has a Definition of Done – it’s not just “I pushed code.” It could mean:

• Code is written

• Reviewed by a peer

• Merged into main

• Tested on staging

• Possibly deployed

This clarity keeps teams honest and accountable. No “it works on my machine” energy here. The DoD sets a quality bar. It prevents ambiguity, rework, and “it works on my machine” moments. When every card on the board passes the same finish line, teams move faster – and trust each other more.

Everyone should know what done is in a team. Either its Done as per DoD standards or its not.

Here is a beautiful video highlighting the impotence of DoD.

Demos, Retros, and Saying the Hard Things

Once you’ve built the product, then comes demos (showcasing your work) and retros (analysis as a team on what when well and what areas to improve on).

In the retro, everyone’s encouraged to speak up:

• What went well?

• What didn’t?

• What should we try next time?

Example:

“We missed a lot of stories because we didn’t account for testing time. Maybe we buffer next sprint with fewer tasks.”

The goal is not to blame – it’s to improve. Over time, this feedback loop becomes gold. The Scrum Master usually facilitates, collects feedback (via tools like Parabol, Miro, or sticky notes), and helps turn insights into actionable experiments for the next sprint.

Over time, retros become the heartbeat of team evolution.

Here is a video highlighting the importance of a Retro and Sprint Review.

🧠 Why Retrospection Matters More Than You Think

The Sprint Retrospective is more than just another meeting. It’s a mirror for your team – a safe, structured space to pause, reflect, and improve together.

You discuss:

✅ what went well

❌ what did not go well

🔁 what could we do better next time

Great teams don't just deliver great software, they continually deliver better ways of working.

This is why many experienced Scrum practitioners consider the retro to be the most important event in Scrum. Code is deployed once, but process improvements grow exponentially, sprint after sprint.

Tools You Might Encounter

Scrum doesn’t require software, but real-world teams use a variety of tools:

• Jira – Tracks sprints, issues, velocity

• Trello – Simple board, good for small teams

• Slack – Where standups often happen if async

• Notion / Confluence – Docs, retros, notes

• GitHub Projects – Lightweight planning for devs

Don’t worry if you’re not fluent in these yet. They’re tools – you’ll learn them on the job.

If You’re Preparing for a Job, Here’s What You Can Do

• ✍️ Practice writing user stories from your side projects

• 🧪 Run a mini-sprint: Plan your weekend project, set goals, and “review” it at the end

• 🤝 Contribute to an open-source project that uses Scrum or Agile workflows

• 🧾 Write about what you learned – maybe as a tutorial (hint hint)

Final Thoughts

So to recap, Scrum is a simple yet powerful way for teams to work together, stay organized, and deliver results quickly. It runs in short cycles called sprints, where the team plans what to do, checks in daily, shows their progress at the end, and reflects on how to improve.

The four key ceremonies – Sprint Planning, Daily Scrum, Sprint Review, and Sprint Retrospective – help keep everyone aligned and focused. With clear roles and regular feedback, Scrum makes it easier to handle changes, solve problems early, and continuously get better as a team.

But scrum isn’t a magic spell. It’s just a way for humans to build complex things – together – without falling apart.

You don’t need to be a Scrum Master. You don’t need a certification. But if you understand how sprints work, what’s expected of you, and how to show up to meetings with clarity and candor, you’re 10 steps ahead of most.

Scrum helps teams talk, plan, build, and learn. And now? You can too.

If you liked this, please do share. You never know who it might help out.

Until then…keep learning, unlearning, and relearning!!!

This shift away from reliance on cloud-based APIs offers significant advantages in terms of privacy, cost-effectiveness, and offline accessibility. Developers and enthusiasts can now experiment with and deploy sophisticated AI capabilities without sending data externally or incurring API fees.

This tutorial serves as a practical, hands-on guide to harnessing this local AI power. It focuses on leveraging the Qwen 3 family of LLMs, a state-of-the-art open-source offering from Alibaba, combined with Ollama, a tool that dramatically simplifies running LLMs locally.

Prerequisites

Before diving into this tutorial, you should have a foundational understanding of Python programming and be comfortable using the command line or terminal. Make sure you have Python 3 installed on your system.

While prior experience with AI or Large Language Models (LLMs) is beneficial, it's not essential, as I’ll introduce and explain core concepts like Retrieval-Augmented Generation (RAG) and AI agents throughout the guide.

This tutorial serves as a practical, hands-on guide to harnessing this local AI power. It focuses on leveraging the Qwen 3 family of LLMs, a state-of-the-art open-source offering from Alibaba, combined with Ollama, a tool that dramatically simplifies running LLMs locally.

Table of Contents

1. Local AI Power with Qwen 3 and Ollama

   • Ollama: Your Local LLM Gateway

   • Tutorial Roadmap

2. How to Set Up Your Local AI Lab

   • Install Ollama

   • Choose Your Qwen 3 Model

   • Pull and Run Qwen 3 with Ollama

   • Set Up Your Python Environment

3. How to Build a Local RAG System with Qwen 3

   • Step 1: Prepare Your Data

   • Step 2: Load Documents in Python

   • Step 3: Split Documents

   • Step 4: Choose and Configure Embedding Model

   • Step 5: Set Up Local Vector Store (ChromaDB)

   • Step 6: Index Documents (Embed and Store)

   • Step 7: Build the RAG Chain

   • Step 8: Query Your Documents

4. How to Create Local AI Agents with Qwen 3

   • Step 1: Define Custom Tools

   • Step 2: Set up the Agent LLM

   • Step 3: Create the Agent Prompt

   • Step 4: Build the Agent

   • Step 5: Create the Agent Executor

   • Step 6: Run the Agent

5. Advanced Considerations and Troubleshooting

   • Controlling Qwen 3's Thinking Mode with Ollama

   • Managing Context Length (num_ctx)

   • Hardware Limitations and VRAM

6. Conclusion and Next Steps

Local AI Power with Qwen 3 and Ollama

Running LLMs locally addresses several key concerns associated with cloud-based AI services.

• Privacy is paramount – data processed locally never leaves the user's machine.

• Cost is another major factor – utilizing open-source models and tools like Ollama eliminates API subscription fees and pay-per-token charges, making advanced AI accessible to everyone.

• Local execution enables offline functionality – crucial for applications where internet connectivity is unreliable or undesirable.

Ollama: Your Local LLM Gateway

Ollama acts as a bridge, making the power of models like Qwen 3 accessible on local hardware. It's a command-line tool that simplifies the download, setup, and execution of various open-source LLMs across macOS, Linux, and Windows.

Ollama handles the complexities of model configuration and GPU utilization, providing a straightforward interface for developers and users. It also exposes an OpenAI-compatible API endpoint, allowing seamless integration with popular frameworks like LangChain.

Tutorial Roadmap

This tutorial will guide you through the process of:

1. Setting up a local AI environment: Installing Ollama and selecting/running appropriate Qwen 3 models.

2. Building a local RAG system: Creating a system that allows chatting with personal documents using Qwen 3, Ollama, LangChain, and ChromaDB for vector storage.

3. Creating a basic local AI agent: Developing a simple agent powered by Qwen 3 that can utilize custom-defined tools (functions).

How to Set Up Your Local AI Lab

The first step is to prepare your local machine with the necessary tools and models.

Install Ollama

Ollama provides the simplest path to running LLMs locally.

• Linux / macOS: Open a terminal and run the official installation script:

    curl -fsSL https://ollama.com/install.sh | sh
  

• Windows: Download the installer from the Ollama website (https://ollama.com/download) and follow the setup instructions.

After installation, verify it by opening a new terminal window and running:

ollama --version

Ollama typically stores downloaded models in ~/.ollama/models on macOS and /usr/share/ollama/.ollama/models on Linux/WSL.

Choose Your Qwen 3 Model

Selecting the right Qwen 3 model is crucial and depends on your intended task and available hardware, primarily system RAM and GPU VRAM. Running larger models requires more resources but generally offers better performance and reasoning capabilities.

Qwen 3 offers two main architectures available through Ollama:

• Dense Models: (like qwen3:0.6b, qwen3:4b, qwen3:8b, qwen3:14b, qwen3:32b) These models activate all their parameters during inference. Their performance is predictable, but resource requirements scale directly with parameter count.

• Mixture-of-Experts (MoE) Models: (like qwen3:30b-a3b) These models contain many "expert" sub-networks but only activate a small fraction for each input token. This allows them to achieve the performance characteristic of their large total parameter count (for example, 30 billion) while having inference costs closer to their smaller active parameter count (for example, 3 billion). They offer a compelling balance of capability and efficiency, especially for reasoning and coding tasks.

Recommendation for this tutorial: For the examples that follow, qwen3:8b strikes a good balance between capability and resource requirements for many modern machines. If resources are more constrained, qwen3:4b is a viable alternative. The MoE model qwen3:30b-a3b offers excellent performance, especially for coding and reasoning, and runs surprisingly well on systems with 16GB+ VRAM due to its sparse activation.

Pull and Run Qwen 3 with Ollama

Once you’ve chosen a model, you’ll need to download it (pull it) via Ollama.

Pull the model: Open the terminal and run (replace qwen3:8b with the desired tag):

ollama pull qwen3:8b

This command downloads the model weights and configuration.

Run interactively (optional test): To chat directly with the model from the command line:

ollama run qwen3:8b

Type prompts directly into the terminal. Use /bye to exit the session. Other useful commands within the interactive session include /? for help and /set parameter <name> <value> (for example, /set parameter num_ctx 8192) to temporarily change model parameters for the current session. Use ollama list outside the session to see downloaded models.

Run as a server: For integration with Python scripts (using LangChain), Ollama needs to run as a background server process, exposing an API. Open a separate terminal window and run:

ollama serve

Keep this terminal window open while running the Python scripts. This command starts the server, typically listening on http://localhost:11434, providing an OpenAI-compatible API endpoint.

Set Up Your Python Environment

A dedicated Python environment is recommended for managing dependencies.

Create a virtual environment:

python -m venv venv

Activate the environment:

• macOS/Linux: source venv/bin/activate

• Windows: venv\Scripts\activate

Install necessary libraries:

pip install langchain langchain-community langchain-core langchain-ollama chromadb sentence-transformers pypdf python-dotenv unstructured[pdf] tiktoken

• langchain, langchain-community, langchain-core: The core LangChain framework for building LLM applications.

• langchain-ollama: Specific integration for using Ollama models with LangChain.

• chromadb: The local vector database for storing document embeddings.

• sentence-transformers: Used for an alternative local embedding method (explained later).

• pypdf: A library for loading PDF documents.

• python-dotenv: For managing environment variables (optional but good practice).

• unstructured[pdf]: An alternative, powerful document loader, especially for complex PDFs.

• tiktoken: Used by LangChain for token counting.

The local setup involves coordinating several independent components: Ollama itself, the specific Qwen 3 model weights, the Python environment, and various libraries like LangChain and ChromaDB. Ensuring compatibility between these pieces and correctly configuring parameters (like Ollama's context window size or selecting a model appropriate for the available VRAM) is key to a smooth experience.

While this modularity offers flexibility – allowing components like the LLM or vector store to be swapped – it also means the initial setup requires careful attention to detail. This tutorial aims to provide clear steps and sensible defaults to minimize potential friction points.

How to Build a Local RAG System with Qwen 3

Retrieval-Augmented Generation (RAG) is a powerful technique that enhances LLMs by providing them with external knowledge.

Instead of relying solely on its training data, the LLM can retrieve relevant information from a specified document set (like local PDFs) and uses that information to answer questions. This significantly reduces "hallucinations" (incorrect or fabricated information) and allows the LLM to answer questions about specific, private data without needing retraining.

The core RAG process involves:

1. Loading and splitting documents into manageable chunks.

2. Converting these chunks into numerical representations (embeddings) using an embedding model.

3. Storing these embeddings in a vector database for efficient searching.

4. When a query comes in, embedding the query and searching the vector database for the most similar document chunks.

5. Providing these relevant chunks (context) along with the original query to the LLM to generate an informed answer.

Let's build this locally using Qwen 3, Ollama, LangChain, and ChromaDB.

Step 1: Prepare Your Data

Create a directory named data in the project folder. Place the PDF document that you intend to query into this directory. For this tutorial, using a single, primarily text-based PDF (like a research paper or a report) for simplicity.

mkdir data
# Copy your PDF file into the 'data' directory
# e.g., cp ~/Downloads/some_paper.pdf./data/mydocument.pdf

If you don’t have a PDF readily available that you’d like to use, you can download a sample PDF (the Llama 2 paper) for this tutorial using the following command in your terminal:

wget --user-agent "Mozilla" "https://arxiv.org/pdf/2307.09288.pdf" -O "data/llama2.pdf"

This command creates the data directory and downloads the PDF, saving it as llama2.pdf inside the data directory. If you prefer to use your own document, place your PDF file into the data directory and update the filename in the subsequent Python code.

Step 2: Load Documents in Python

Use LangChain's document loaders to read the PDF content. PyPDFLoader is straightforward for simple PDFs. UnstructuredPDFLoader (requires unstructured[pdf]) can handle more complex layouts but has more dependencies.

# rag_local.py
import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader # Or UnstructuredPDFLoader

load_dotenv() # Optional: Loads environment variables from.env file

DATA_PATH = "data/"
PDF_FILENAME = "mydocument.pdf" # Replace with your PDF filename

def load_documents():
    """Loads documents from the specified data path."""
    pdf_path = os.path.join(DATA_PATH, PDF_FILENAME)
    loader = PyPDFLoader(pdf_path)
    # loader = UnstructuredPDFLoader(pdf_path) # Alternative
    documents = loader.load()
    print(f"Loaded {len(documents)} page(s) from {pdf_path}")
    return documents

# documents = load_documents() # Call this later

Step 3: Split Documents

Large documents need to be split into smaller chunks suitable for embedding and retrieval. The RecursiveCharacterTextSplitter attempts to split text semantically (at paragraphs, sentences, and so on) before resorting to fixed-size splits. chunk_size determines the maximum size of each chunk (in characters), and chunk_overlap specifies how many characters should overlap between consecutive chunks to maintain context.

# rag_local.py (continued)
from langchain_text_splitters import RecursiveCharacterTextSplitter

def split_documents(documents):
    """Splits documents into smaller chunks."""
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
        length_function=len,
        is_separator_regex=False,
    )
    all_splits = text_splitter.split_documents(documents)
    print(f"Split into {len(all_splits)} chunks")
    return all_splits

# loaded_docs = load_documents()
# chunks = split_documents(loaded_docs) # Call this later

Step 4: Choose and Configure Embedding Model

Embeddings transform text into vectors (lists of numbers) such that semantically similar text chunks have vectors that are close together in multi-dimensional space.

Option A (Recommended for Simplicity): Ollama Embeddings

This approach uses Ollama to serve a dedicated embedding model. nomic-embed-text is a capable open-source model available via Ollama.

First, ensure the embedding model is pulled:

ollama pull nomic-embed-text

Then, use OllamaEmbeddings in Python:

# rag_local.py (continued)
from langchain_ollama import OllamaEmbeddings

def get_embedding_function(model_name="nomic-embed-text"):
    """Initializes the Ollama embedding function."""
    # Ensure Ollama server is running (ollama serve)
    embeddings = OllamaEmbeddings(model=model_name)
    print(f"Initialized Ollama embeddings with model: {model_name}")
    return embeddings

# embedding_function = get_embedding_function() # Call this later

Option B (Alternative): Sentence Transformers

This uses the sentence-transformers library directly within the Python script. It requires installing the library (pip install sentence-transformers) but doesn't need a separate Ollama process for embeddings. Models like all-MiniLM-L6-v2 are fast and lightweight, while all-mpnet-base-v2 offers higher quality.

# Alternative embedding function using Sentence Transformers
from langchain_community.embeddings import HuggingFaceEmbeddings

def get_embedding_function_hf(model_name="all-MiniLM-L6-v2"):
     """Initializes HuggingFace embeddings (runs locally)."""
     embeddings = HuggingFaceEmbeddings(model_name=model_name)
     print(f"Initialized HuggingFace embeddings with model: {model_name}")
     return embeddings

embedding_function = get_embedding_function_hf() # Use this if choosing Option B

For this tutorial, we’ll use Option A (Ollama Embeddings with nomic-embed-text) to keep the toolchain consistent.

Step 5: Set Up Local Vector Store (ChromaDB)

ChromaDB provides an efficient way to store and search vector embeddings locally. Using a persistent client ensures the indexed data is saved to disk and can be reloaded without re-processing the documents every time.

# rag_local.py (continued)
from langchain_community.vectorstores import Chroma

CHROMA_PATH = "chroma_db" # Directory to store ChromaDB data

def get_vector_store(embedding_function, persist_directory=CHROMA_PATH):
    """Initializes or loads the Chroma vector store."""
    vectorstore = Chroma(
        persist_directory=persist_directory,
        embedding_function=embedding_function
    )
    print(f"Vector store initialized/loaded from: {persist_directory}")
    return vectorstore

embedding_function = get_embedding_function()
vector_store = get_vector_store(embedding_function) # Call this later

Step 6: Index Documents (Embed and Store)

This is the core indexing step where document chunks are converted to embeddings and saved in ChromaDB. The Chroma.from_documents function is convenient for the initial creation and indexing. If the database already exists, subsequent additions can use vectorstore.add_documents.

# rag_local.py (continued)

def index_documents(chunks, embedding_function, persist_directory=CHROMA_PATH):
    """Indexes document chunks into the Chroma vector store."""
    print(f"Indexing {len(chunks)} chunks...")
    # Use from_documents for initial creation.
    # This will overwrite existing data if the directory exists but isn't a valid Chroma DB.
    # For incremental updates, initialize Chroma first and use vectorstore.add_documents().
    vectorstore = Chroma.from_documents(
        documents=chunks,
        embedding=embedding_function,
        persist_directory=persist_directory
    )
    vectorstore.persist() # Ensure data is saved
    print(f"Indexing complete. Data saved to: {persist_directory}")
    return vectorstore

#... (previous function calls)
vector_store = index_documents(chunks, embedding_function) # Call this for initial indexing

To load an existing persistent database later:

embedding_function = get_embedding_function()
vector_store = Chroma(persist_directory=CHROMA_PATH, embedding_function=embedding_function)

Step 7: Build the RAG Chain

Now, assemble the components into a LangChain Expression Language (LCEL) chain. This involves initializing the Qwen 3 LLM via Ollama, creating a retriever from the vector store, defining a suitable prompt, and chaining them together.

A critical parameter when initializing ChatOllama for RAG is num_ctx. This defines the context window size (in tokens) that the LLM can handle. Ollama's default (often 2048 or 4096 tokens) might be too small to accommodate both the retrieved document context and the user's query/prompt.

Qwen 3 models (8B and larger) support much larger context windows (for example, 128k tokens), but practical limits depend on your available RAM/VRAM. Setting num_ctx to a value like 8192 or higher is often necessary for effective RAG.

# rag_local.py (continued)
from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

def create_rag_chain(vector_store, llm_model_name="qwen3:8b", context_window=8192):
    """Creates the RAG chain."""
    # Initialize the LLM
    llm = ChatOllama(
        model=llm_model_name,
        temperature=0, # Lower temperature for more factual RAG answers
        num_ctx=context_window # IMPORTANT: Set context window size
    )
    print(f"Initialized ChatOllama with model: {llm_model_name}, context window: {context_window}")

    # Create the retriever
    retriever = vector_store.as_retriever(
        search_type="similarity", # Or "mmr"
        search_kwargs={'k': 3} # Retrieve top 3 relevant chunks
    )
    print("Retriever initialized.")

    # Define the prompt template
    template = """Answer the question based ONLY on the following context:
{context}

Question: {question}
"""
    prompt = ChatPromptTemplate.from_template(template)
    print("Prompt template created.")

    # Define the RAG chain using LCEL
    rag_chain = (
        {"context": retriever, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
    )
    print("RAG chain created.")
    return rag_chain

#... (previous function calls)
vector_store = get_vector_store(embedding_function) # Assuming DB is already indexed
rag_chain = create_rag_chain(vector_store) # Call this later

The effectiveness of the RAG system hinges on the proper configuration of each component. The chunk_size and chunk_overlap in the splitter affect what the retriever finds. Your choice of embedding_function must be consistent between indexing and querying. The num_ctx parameter for the ChatOllama LLM must be large enough to hold the retrieved context and the prompt itself. A poorly designed prompt template can also lead the LLM astray. Make sure you carefully tune these elements for optimal performance.

Step 8: Query Your Documents

Finally, invoke the RAG chain with a question related to the content of the indexed PDF.

# rag_local.py (continued)

def query_rag(chain, question):
    """Queries the RAG chain and prints the response."""
    print("\nQuerying RAG chain...")
    print(f"Question: {question}")
    response = chain.invoke(question)
    print("\nResponse:")
    print(response)

# --- Main Execution ---
if __name__ == "__main__":
    # 1. Load Documents
    docs = load_documents()

    # 2. Split Documents
    chunks = split_documents(docs)

    # 3. Get Embedding Function
    embedding_function = get_embedding_function() # Using Ollama nomic-embed-text

    # 4. Index Documents (Only needs to be done once per document set)
    # Check if DB exists, if not, index. For simplicity, we might re-index here.
    # A more robust approach would check if indexing is needed.
    print("Attempting to index documents...")
    vector_store = index_documents(chunks, embedding_function)
    # To load existing DB instead:
    # vector_store = get_vector_store(embedding_function)

    # 5. Create RAG Chain
    rag_chain = create_rag_chain(vector_store, llm_model_name="qwen3:8b") # Use the chosen Qwen 3 model

    # 6. Query
    query_question = "What is the main topic of the document?" # Replace with a specific question
    query_rag(rag_chain, query_question)

    query_question_2 = "Summarize the introduction section." # Another example
    query_rag(rag_chain, query_question_2)

Run the complete script (python rag_local.py). Make sure that the ollama serve command is running in another terminal. The script will load the PDF, split it, embed the chunks using nomic-embed-text via Ollama, store them in ChromaDB, build the RAG chain using qwen3:8b via Ollama, and finally execute the queries. It’ll print the LLM's responses based on the document content.

How to Create Local AI Agents with Qwen 3

Beyond answering questions based on provided text, LLMs can act as the reasoning engine for AI agents. Agents can plan sequences of actions, interact with external tools (like functions or APIs), and work towards accomplishing more complex goals assigned by the user.

Qwen 3 models were specifically designed with strong tool-calling and agentic capabilities. While Alibaba provides the Qwen-Agent framework, this tutorial will continue using LangChain for consistency and because its integration with Ollama for agent tasks is more readily documented in the provided materials.

We will build a simple agent that can use a custom Python function as a tool.

Step 1: Define Custom Tools

Tools are standard Python functions that the agent can choose to execute. The function's docstring is crucial, as the LLM uses it to understand what the tool does and what arguments it requires. LangChain's @tool decorator simplifies wrapping functions for agent use.

# agent_local.py
import os
from dotenv import load_dotenv
from langchain.agents import tool
import datetime

load_dotenv() # Optional

@tool
def get_current_datetime(format: str = "%Y-%m-%d %H:%M:%S") -> str:
    """
    Returns the current date and time, formatted according to the provided Python strftime format string.
    Use this tool whenever the user asks for the current date, time, or both.
    Example format strings: '%Y-%m-%d' for date, '%H:%M:%S' for time.
    If no format is specified, defaults to '%Y-%m-%d %H:%M:%S'.
    """
    try:
        return datetime.datetime.now().strftime(format)
    except Exception as e:
        return f"Error formatting date/time: {e}"

# List of tools the agent can use
tools = [get_current_datetime]
print("Custom tool defined.")

Step 2: Set Up the Agent LLM

Instantiate the ChatOllama model again, using a Qwen 3 variant suitable for tool calling. The qwen3:8b model should be capable of handling simple tool use cases.

It's important to note that tool calling reliability with local models served via Ollama can sometimes be less consistent than with large commercial APIs like GPT-4 or Claude. The LLM might fail to recognize when a tool is needed, hallucinate arguments, or misinterpret the tool's output. Starting with clear prompts and simple tools is recommended.

# agent_local.py (continued)
from langchain_ollama import ChatOllama

def get_agent_llm(model_name="qwen3:8b", temperature=0):
    """Initializes the ChatOllama model for the agent."""
    # Ensure Ollama server is running (ollama serve)
    llm = ChatOllama(
        model=model_name,
        temperature=temperature # Lower temperature for more predictable tool use
        # Consider increasing num_ctx if expecting long conversations or complex reasoning
        # num_ctx=8192
    )
    print(f"Initialized ChatOllama agent LLM with model: {model_name}")
    return llm

# agent_llm = get_agent_llm() # Call this later

Step 3: Create the Agent Prompt

Agents require specific prompt structures that guide their reasoning and tool use. The prompt typically includes placeholders for user input (input), conversation history (chat_history), and the agent_scratchpad. The scratchpad is where the agent records its internal "thought" process, the tools it decides to call, and the results (observations) it gets back from those tools. LangChain Hub provides pre-built prompts suitable for tool-calling agents.

# agent_local.py (continued)
from langchain import hub

def get_agent_prompt(prompt_hub_name="hwchase17/openai-tools-agent"):
    """Pulls the agent prompt template from LangChain Hub."""
    # This prompt is designed for OpenAI but often works well with other tool-calling models.
    # Alternatively, define a custom ChatPromptTemplate.
    prompt = hub.pull(prompt_hub_name)
    print(f"Pulled agent prompt from Hub: {prompt_hub_name}")
    # print("Prompt Structure:")
    # prompt.pretty_print() # Uncomment to see the prompt structure
    return prompt

# agent_prompt = get_agent_prompt() # Call this later

Step 4: Build the Agent

The create_tool_calling_agent function combines the LLM, the defined tools, and the prompt into a runnable unit that represents the agent's core logic.

# agent_local.py (continued)
from langchain.agents import create_tool_calling_agent

def build_agent(llm, tools, prompt):
    """Builds the tool-calling agent runnable."""
    agent = create_tool_calling_agent(llm, tools, prompt)
    print("Agent runnable created.")
    return agent

# agent_runnable = build_agent(agent_llm, tools, agent_prompt) # Call this later

Step 5: Create the Agent Executor

The AgentExecutor is responsible for running the agent loop. It takes the agent runnable and the tools, invokes the agent with the input, parses the agent's output (which could be a final answer or a tool call request), executes any requested tool calls, and feeds the results back to the agent until a final answer is reached. Setting verbose=True is highly recommended during development to observe the agent's step-by-step execution flow.

# agent_local.py (continued)
from langchain.agents import AgentExecutor

def create_agent_executor(agent, tools):
    """Creates the agent executor."""
    agent_executor = AgentExecutor(
        agent=agent,
        tools=tools,
        verbose=True # Set to True to see agent thoughts and tool calls
    )
    print("Agent executor created.")
    return agent_executor

# agent_executor = create_agent_executor(agent_runnable, tools) # Call this later

Step 6: Run the Agent

Invoke the agent executor with a user query that should trigger the use of the defined tool.

# agent_local.py (continued)

def run_agent(executor, user_input):
    """Runs the agent executor with the given input."""
    print("\nInvoking agent...")
    print(f"Input: {user_input}")
    response = executor.invoke({"input": user_input})
    print("\nAgent Response:")
    print(response['output'])

# --- Main Execution ---
if __name__ == "__main__":
    # 1. Define Tools (already done above)

    # 2. Get Agent LLM
    agent_llm = get_agent_llm(model_name="qwen3:8b") # Use the chosen Qwen 3 model

    # 3. Get Agent Prompt
    agent_prompt = get_agent_prompt()

    # 4. Build Agent Runnable
    agent_runnable = build_agent(agent_llm, tools, agent_prompt)

    # 5. Create Agent Executor
    agent_executor = create_agent_executor(agent_runnable, tools)

    # 6. Run Agent
    run_agent(agent_executor, "What is the current date?")
    run_agent(agent_executor, "What time is it right now? Use HH:MM format.")
    run_agent(agent_executor, "Tell me a joke.") # Should not use the tool

Running python agent_local.py (with ollama serve active) will execute the agent. The verbose=True setting will print output resembling the ReAct (Reasoning and Acting) framework, showing the agent's internal "Thoughts" on how to proceed, the "Action" it decides to take (calling a specific tool with arguments), and the "Observation" (the result returned by the tool).

Building reliable agents with local models presents unique challenges. The LLM's ability to correctly interpret the prompt, understand when to use tools, select the right tool, generate valid arguments, and process the tool's output is critical.

Local models, especially smaller or heavily quantized ones, might struggle with these reasoning steps compared to larger, cloud-based counterparts. If the qwen3:8b model proves unreliable for more complex agentic tasks, consider trying qwen3:14b or the efficient qwen3:30b-a3b if hardware permits.

For highly complex or stateful agent workflows, exploring frameworks like LangGraph, which offers more control over the agent's execution flow, might be beneficial.

Advanced Considerations and Troubleshooting

Running LLMs locally offers great flexibility but also introduces specific configuration aspects and potential issues.

Controlling Qwen 3's Thinking Mode with Ollama

Qwen 3's unique hybrid inference allows switching between a deep "thinking" mode for complex reasoning and a faster "non-thinking" mode for general chat. While frameworks like Hugging Face Transformers or vLLM might offer explicit parameters (enable_thinking), the primary way to control this when using Ollama appears to be through "soft switches" embedded in the prompt.

Append /think to the end of a user prompt to encourage step-by-step reasoning, or /no_think to request a faster, direct response. You can do this via the Ollama CLI or potentially within the prompts sent via the API/LangChain.

# Example using LangChain's ChatOllama
from langchain_ollama import ChatOllama

llm_think = ChatOllama(model="qwen3:8b")
llm_no_think = ChatOllama(model="qwen3:8b") # Could also set system prompt

# Invoke with prompt modification
response_think = llm_think.invoke("Solve the equation 2x + 5 = 15 /think")
print("Thinking Response:", response_think)

response_no_think = llm_no_think.invoke("What is the capital of France? /no_think")
print("Non-Thinking Response:", response_no_think)

# Alternatively, set via system message (might be less reliable turn-by-turn)
llm_system_no_think = ChatOllama(model="qwen3:8b", system="/no_think")
response_system = llm_system_no_think.invoke("What is 2+2?")
print("System No-Think Response:", response_system)

Note that the persistence of these tags across multiple turns in a conversation might require careful prompt management.

Managing Context Length (num_ctx)

The context window (num_ctx) determines how much information (prompt, history, retrieved documents) the LLM can consider at once. Qwen 3 models (8B+) support large native context lengths (for example, 128k tokens), but Ollama often defaults to a much smaller window (like 2048 or 4096). For RAG or conversations requiring memory of earlier turns, this default is often insufficient.

Set num_ctx when initializing ChatOllama or OllamaLLM in LangChain:

# Example setting context window to 8192 tokens
llm = ChatOllama(model="qwen3:8b", num_ctx=8192)

Be mindful that larger num_ctx values significantly increase RAM and VRAM consumption. But setting it too low can lead to the model "forgetting" context or even entering repetitive loops. Choose a value that balances the task requirements with hardware capabilities.

Hardware Limitations and VRAM

Running LLMs locally is resource-intensive.

• VRAM: A dedicated GPU (NVIDIA or Apple Silicon) with sufficient VRAM is highly recommended for acceptable performance. The amount of VRAM dictates the largest model size that can run efficiently. Refer to the table in Section 2 for estimates.

• RAM: System RAM is also crucial, especially if the model doesn't fit entirely in VRAM. Ollama can utilize system RAM as a fallback, but this is significantly slower.

• Quantization: Ollama typically serves quantized models (for example., 4-bit or 5-bit), which reduce the model size and VRAM requirements significantly compared to full-precision models, often with minimal performance degradation for many tasks. The tags like :4b, :8b usually imply a default quantization level.

If performance is slow or errors occur due to resource constraints, consider:

• Using a smaller Qwen 3 model (like 4B instead of 8B).

• Ensuring Ollama is correctly detecting and utilizing the GPU (check Ollama logs or system monitoring tools).

• Closing other resource-intensive applications.

Conclusion and Next Steps

This tutorial gave you a practical walkthrough for setting up your local AI environment using the powerful and open Qwen 3 LLM family with the user-friendly Ollama tool.

If you’ve followed these steps, you should have successfully:

1. Installed Ollama and downloaded/run Qwen 3 models locally.

2. Built a functional Retrieval-Augmented Generation (RAG) pipeline using LangChain and ChromaDB to query local documents.

3. Created a basic AI agent capable of reasoning and utilizing custom Python tools.

Running these systems locally unlocks significant advantages in privacy, cost, and customization, making advanced AI capabilities more accessible than ever. The combination of Qwen 3's performance and open license with Ollama's ease of use creates a potent platform for experimentation and development.

Official Resources:

• Qwen 3: GitHub, Documentation

• Ollama: Website, Model Library, GitHub

• LangChain: Python Documentation

• ChromaDB: Documentation

• Sentence Transformers: Documentation

By leveraging these powerful, free, and open-source tools, you can continue to push the boundaries of what's possible with AI running directly on your own hardware.

It is better to be a samurai in a garden than an agricultural worker in a war - Miyamoto Musashi

In this article, I’ll share my story.

When I was younger, I thought I was destined to be a failure in life. To be isolated from everyone. But years later, I realized I was actually destined for success.

I went from wasting thousands of hours playing video games to giving a lecture to medical professionals called “Trustworthy AI: The Role of Small Models in Critical Systems.”

And I went from being told I was dumber than most people based on an IQ test I took as a 14 year old low self-esteem kid, to becoming a frequent contributor to freeCodeCamp. I’ve written articles on interpretable AI, applied math, and advanced tech. And these articles have now reached more than 200,000 people worldwide.

And this is just the beginning.

When it comes to education, I owe gratitude to three people and the organizations they lead:

• Salman Khan, founder of Khan Academy

• Quincy Larson, founder of freeCodeCamp

• Andrew Ng, co-founder of Coursera and founder of DeepLearning.AI

I also owe a lot to the great author of the book Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones – James Clear.

I’m sharing my story to inspire others who are struggling in their lives just like I was.

I’m also writing this for those who know me or will know me personally, so that they can understand where my determination comes from and why I am relentless.

Here’s what I’ll cover:

1. Where I Was: Misery, Depression, and Isolation

2. My Transformation: Learning Triple Integrals and Programming

3. One of the Best Choices in My Life: Why I Chose Electrical and Computer Engineering

4. Being Restless and Determined: My Work Ethic in University

5. My Projects while in NOVA FCT: AI Projects, International Student Organizations, and freeCodeCamp Articles

6. My Personal Philosophy at 21 Years Old and View on Envy and Negativity

7. Where I Am Today: A Fraction of What I Have Achieved

8. Final Thoughts: Have an Adaptive Grand Strategy for Your Life

Where I Was: Misery, Depression, and Isolation

Photo by Nikko Balanial on Unsplash

"As it turns out, it was that very rock bottom that became the firmest foundation I had ever planted my feet on." — Mandy Hale

Five years ago, I was in a different place and was a completely different person.

Like many teenagers, I started playing video games and became addicted to them. Over time, games became an escape from reality and all my problems, including my bad grades and many other issues.

At age 14, I still held ambitions in my heart. I dreamed of being someone who would help others, maybe as a doctor or an engineer.

But after an IQ and vocational guidance test, I was told that I was incapable of doing these things. That I lacked the intelligence needed. That it was unrealistic for me to pursue these types of degrees.

Eventually, and because of many comments, opinions, and expectations of others, I began to believe in this lie for years.

Over time, depressed and constantly escaping reality, my grades plummeted and I got worse and worse. And this only made the prospect of going to college less and less likely.

By 11th grade, I was:

• Extremely shy and anxious

• Struggling academically

• Over 2000 hours in video games on two games alone:

  • 1000 hours in GTA V

  • 1000 hours in Destiny 2

2000 hours equals nearly 83 days.

This means that in these two games, I lost more than two months of my life.

But from these wasted hours, I learned English. This became crucial when learning online.

In January 2020, I was tired of everything. In particular, I was sick of the misery of always being at the bottom and of so much negativity towards and around me.

So I made these vows to myself for the rest of my life:

• Never again would I worry or care about what other people say about me.

• I would no longer accept the limitations imposed by others or myself on my growth.

• And as for the limitations I imposed on myself, I would rethink to see if they were really impossible or if I could actually conquer them.

As a result, I started relearning and learning everything by myself to make sure I succeeded in the national exams.

My Transformation: Learning Triple Integrals and Programming

Photo by Joshua Earle on Unsplash

"The expert in anything was once a beginner."
— Helen Hayes

I started going through chemistry exercises in the 10th and 11th grades using books from school and YouTube videos. In two weeks, I relearned or learned most of the material I needed to know.

I started doing the same for mathematics, something I always found hard due to a lack of basic foundational mathematics knowledge.

I found it hard, that was, until I discovered Khan Academy.

With the Khan Academy, I rebuilt myself, going from struggling with basic math to mastering double and triple integrals, all within five to six months.

My method was simple:

• Study a little bit every day.

• Take detailed notes

• Redo quizzes or units tests until I scored more than 90%

  • For topics that I found harder or failed to understand, I did the practice exercizes

• Use YouTube to close any knowledge gaps

For example, for Algebra I, where I started to relearn math, I say how many units there were. Each unit had a certain number of topics. As of 2025, Algebra I has, from units with mastery exercises, 89 topics.

For those 89 topics, I watched the videos and did the quizzes. According to my scores, I would either go on to the next video (if I felt confident), or stop, rewatch the video, go through the same material on YouTube, do practice exercises, and then try to do the quiz again.

I decided that I needed to do at least three topics every day. This way, by doing 3 topics per day, I could finish Algebra I by the end of one month.

But I was so motivated and so focused on it that I did more than 3 topics per day.

I did the same for Algebra II, and all the others until College Calculus BC.

Some days, I completed more than 8 topics. Other days, I struggle to even do 2. But I made sure that I mastered mathematics and its foundation for the rest of my life.

This was not just about grades. It was about regaining belief and confidence in myself.

I also read many books, primarily self-help, to make myself better. Over the years, I have started reading fewer self-help books and have started focusing on non-fictional books that explain to me how the world works.

COVID-19: Accelerating My Learning in Programming and Machine Learning

When the pandemic hit, I started accelerating my learning in other areas, like programming and physics. In many online classes, I didn’t pay attention as well as I should’ve – and I found myself prioritizing self study on topics I found more important. And I always used my time to learn more about programming.

I learned Python and C through free YouTube courses for beginners on freeCodeCamp’s channel.

This was where I first learned Python.

And C:

Soon after exploring C programming, I realized that programming languages are just tools. Once you master one, others come more naturally.

I studied data science tutorials on the web and on YouTube. This way, I learned how to import Python libraries in virtual environments. I also began building projects with Python libraries I found interesting and made it a habit to explain every line of code to myself as if I were the teacher.

For example, I started working with the scikit learn Python library to make simple linear and logistic models that could make make predictions.

I also decided to explore Deep Learning and taught myself how to work with Arduino and circuits. In other words, I learned how to train the architecture of neural networks to predict things.

I found this hard to master compared to triple integrals at the time!

But this way, I understood one very important thing about Deep Learning: to truly understand and master it, I needed to know, deeply, some difficult mathematics concepts. And I also needed to learn quickly about the new research coming out.

One of the Best Choices in My Life: Why I Choose Electrical and Computer Engineering

Photo by Nicolas Thomas on Unsplash

"Opportunities multiply as they are seized."
— Sun Tzu

After completing the Portuguese national mathematics exam in the 12th grade, I chose Electrical and Computer Engineering (ECE). I choose this area, because it would challenge me and allow me to gain the skills to learn and apply new mathematics by myself without anyone teaching me.

It was also broad:

• If I liked, I could follow an electrical engineering area. Like circuits, power systems, or telecommunications.

• If an electrical engineering subarea was not in my best interest, I could follow a computer science path or apply math in banking or other areas where people who know applied math work.

The ECE degree also allwed me to unite the following areas:

• Advanced Mathematics

• Programming (from low-level like assembly and C to high-level like Python)

• Physics (circuits, robotics, communication systems)

I wanted to become someone who not only mastered knowledge but could also create new systems and ideas from it.

I knew that I was laying the foundation for something greater than just academic success.

What Did I Gain Exactly?

Over time, I learned the many skills I needed to understand all the new AI research coming out after completing AI specializations.

I also learned hard math and applied mathematics areas such as:

• Partial differential equations: how they can represent and model real phenomena, like the economy of a country.

• Pure harmonic analysis: Fourier and Laplace transformations and how integral transformations allow us to see problems in other ways.

• Complex analysis: application of derivatives and integrals in a complex domain, with real and imaginary numbers.

• Numerical analysis: how math used to approximate analytical math is used by computers to get faster results.

• Signal and control theory: how the architecture of systems is studied to ensure rocket, train, and car control systems are stable, despite possible disturbances in the systems.

Not to mention physics classes like:

• Classical mechanics

• Electromagnetism

While these topics may not be applied in depth to AI, learning them helped me develop an incredible intuition into systems thinking. It also greatly improved my ability to learn hard STEM concepts on my own.

Being Restless and Determined: My Work Ethic in University

Me at 18 years old

"Success is the sum of small efforts, repeated day in and day out." — Robert Collier

I adopted a very rigorous work ethic.

When my work ethic failed to achieve what I wanted, I adapted with more knowledge and learned very deeply what I did wrong so as not to repeat it.

For example, in the first semester, my first grades were not the best. So, I read:

• Deep Work

• The Effective Executive: The Definitive Guide to Getting the Right Things Done

• How to Become a Straight-A Student: The Unconventional Strategies Real College Students Use

These books taught me how to focus and prioritize what needed to be done. This became essential as I entered one of the most demanding phases of my life.

In addition, I used Notion as a management system and Google Calendar as a schedule system.

Every week, I transferred next week's tasks from Google Calendar to Notion. This way, I never forgot anything and never worried about forgetting anything.

I had two simple scalable systems that worked very well for managing everything I did.

In the scheduling system, I would place certain events on repeat, for example:

1. Every week:

   • Read the top articles of the week on Subreddits dedicated to programming and others topics so I could keep learning and growing. Same with communities on Stack Exchange Network.

• Read new articles on IEEE Spectrum and learn as much as possible about what is happening currently.

2. Every two weeks:

   • Plan my studying according to time available, as well as all class and other resources I could get for tests, projects, and exams.

3. Every month:

   • Review my annual objectives and prioritize what was important and urgent to do in that month. Also, review new opportunities that appeared that aligned with my objectives this year and in my life.

This way, I was always aligned and efficient. And all this was from a Notion database.

Very often, I started working at 8:00am and continued until around 9:00 or 10:30am, when my classes often started. At that time, I studied, did student organization work, completed online courses and specializations, worked on AI projects, wrote freeCodeCamp articles, and many tasks.

I went beyond studying just the subjects from my degree:

• I also studied history, economics, and geopolitics to understand the hidden incentives that shape the world.

• I developed the habit of studying the architecture of things, from political systems to technology, understanding how they work to design better systems.

• I attended many free online and university events to learn as much as possible.

I also treated weekends as opportunities to grow and work, and did not stop. This was not possible 100% of the time, but most days I was able to do so.

In this way, I completed Coursera’s prestigious Deep Learning Specialization, a very important achievement in my journey.

I also read many books and listened to podcasts while taking public transportation, ensuring that no time was wasted.

My Projects While in NOVA FCT: AI Projects, International Student Organizations, and freeCodeCamp Articles

Me at 21 years old.

"The strength of the team is each individual member. The strength of each member is the team."
— Phil Jackson

International student organizations often offer opportunities that are rarely found in local college clubs.

These student organizations are also often better managed than some local clubs, which can sometimes suffer from internal politics focused on titles rather than making a real contribution to society.

For this reason, I sought international organizations that pushed members towards real impact and development.

After a while, I became interested in BEST (Board of Engineering Students of Technology), a large international organization of student organizations spread over 80 groups around European universities. I joined the local group, BEST Almada, one of the 80 local BEST groups across Europe that helps foster the development of students through courses and events.

I also became deeply involved in the IEEE, the world’s largest non-profit professional association, where I served as the Vice-Chair of the IEEE NOVA Student Branch. Currently, I contribute nationally in the IEEE Portugal Section by creating videos for social media.

Thanks to IEEE, I was able to go to the IEEE Melecon conference in Porto last year to speak with some amazing scientists and researchers.

Here’s a key thing I learned from IEEE that I want to share: Communication, alignment of expectations between everybody, and knowing how to navigate social dynamics is crucial for any project or initiative to succeed. Of course, the culture of the organization and a lot of other variables are important as well. But I believe communication is one of the most important and critical factors.

Along this path, I worked on projects like Eurostatify AI, which aimed to provide European public data insights and hidden patterns that are accessible to researchers and policymakers. I also led the Doctor AI Project as part of a Hackathon in March 2023, where I developed two AI bots using Flutter and the ChatGPT API to help doctors make better decisions.

Each step helped me forge myself into someone capable of inspiring and leading others. I also taught complex topics in my freeCodeCamp articles, such as how CPUs work in depth, interpretable AI, quantum AI, and even how to design a control system for rockets.

I was involved in local student clubs before I realized the value of joining international organizations. In Europe, these organizations bring unique opportunities, are usually better managed than local groups. They’re a great place for developing soft skills as well.

So in the end, joining international student organizations was one of the best decisions of my university life.

My Personal Philosophy at 21 Years Old and View on Envy and Negativity

Photo by Giammarco Boscaro on Unsplash

"Freedom lies in being bold."
— Robert Frost

Here’s one thing I’ve learned over the years: You need to make your own path. Chasing social status and falling prey to social pressures isn’t worth it, and you shouldn’t be blinded by these things. True freedom comes from defining your own path. Developing relationships with professors and mentors, learning from books, and taking advantage of solid free learning resource are all things that can help you go further in life.

But what about envy and negativity from others?

Well, unfortunately these will always be part of our lives. Being envious is human nature, and various forms of negativity will likely continue to exist. Anyone who works and achieves any level of success will inevitably attract envy and negativity.

The best response is not to react and to ignore it completely. Just keep growing.

Some people will disappear, mock you, envy you, or hate you – but just try to let it all go. Keep walking your path.

Time is precious. Don’t waste it on:

• Meaningless opinions

• Video games

• Distractions

I find it sad that, despite living in such an exciting time, and despite unprecedented access to knowledge and education, advances in technology, and immense global connectivity, some people still choose to hate and be envious of others. But as I said before, it’s human nature and there is little we can do about it.

Just remember: you have opportunities today that previous generations could only dream of. Take advantage of them to the fullest and worry about your own personal growth.

Where I am Today: A Fraction of What I Have Achieved

Me in a Tesla factory in Silicon Valley

"I am not a product of my circumstances. I am a product of my decisions."
— Stephen R. Covey

At 21, I am finishing my degree in Electrical and Computer Engineering at NOVA FCT.

So far:

• I’ve been accepted into the Silicon Valley Fellowship Program: Only 18 out of 600 are accepted to visit Silicon Valley's top companies and universities.

• I’ve delivered a talk to doctors about AI called "Trustworthy AI - The Role of Small AI Models in Critical Systems.". Before this, I delivered other smaller talks.

• I’ve completed Coursera AI specializations such as the Deep Learning Specialization from DeepLearning.AI and Reinforcement Learning Specialization from the University of Alberta.

• In IEEE (the largest non-profit professional association in the world), I served as the vice chair of my faculty IEEE NOVA SB student branch, and I am now an IEEE PT Officer, creating videos for social media.

• I’ve had twenty articles published on freeCodeCamp since 2023 that have accumulated around 200,000 views. They are related to advanced applied math, AI, and technology. (Link below)

• I’ve been recognized as a Top Open Source Contributor for freeCodeCamp in 2022, 2023, and 2024

Final Thoughts: Have an Adaptive Grand Strategy for Your Life

Silicon Valley Fellowship post about me

"What you do makes a difference, and you have to decide what kind of difference you want to make."
— Jane Goodall

My life objective was and still the same when I was 14, 7 years ago:

• Help as many people as possible. In their opportunities to make their life’s better and in making society better for future generations that will come after mine.

My strongest advice for anyone: Have a grand strategy for your life.

A grand strategy is a type of long-term strategy in which nations align power and resources to achieve their objectives. You must align your actions, skills, and knowledge towards your purpose.

I used to be afraid of public speaking and so many other things. Not anymore.

Now, I know I am destined to contribute, inspire, and leave a mark on other people's lives for the better.

If you feel stuck, remember:

• You can change! It will be hard. Many people will not want it.

• Ignore all that and focus on yourself.

It takes effort, patience and courage, but it is possible.

Thanks to all organizations for the opportunity to contribute to society and grow as a person:

• NOVA School of Science and Technology and its student association, AEFCT

• IEEE Portugal Section

• Silicon Valley Fellowship

• BEST and BEST Almada

• Magma Studio

I also want to thank all the university professors at NOVA FCT who taught me, especially the ones from the department of electrical and computer engineering.

Finally, I want to express my gratitude to Portuguese society. Not long ago, in Portugal, pursuing higher education, especially in STEM, was inaccessible to many. Thanks to the efforts of past generations, today, young people like me can pursue these opportunities and contribute back to society.

This is just the beginning of my impact on society.

My FreeCodeCamp blog:

By the end of this tutorial, you will know how an IDS works and be able to build your own real-time network monitoring system using Python.

Table of Contents

• Understanding the Types of IDS

• How to Setup Your Development Environment

• Building the Core IDS Components

  • Building the Packet Capture Engine

  • Building the Traffic Analysis Module

  • Building the Detection Engine

  • Building the Alert System

  • Putting It All Together

• Ideas to Extend the IDS

• Security Considerations

• Testing the IDS on Mock Data

• Wrapping Up

Understanding the Types of IDS

Before we jump into the coding part, let’s understand the types of IDS:

1. Network-based IDS (NIDS): This system monitors network traffic for suspicious activity.

2. Host-based IDS (HIDS): This system monitors system logs and file changes on individual hosts and is not directly deployed in the network.

3. Signature-based IDS: This system is either in the network or on the host and identifies attack patterns based on known patterns.

4. Anomaly-based IDS: This system identifies unusual behavior using heuristics and prediction algorithms that are trained on previously seen attack patterns.

For this tutorial, you will be building a hybrid system that combines signature-based and anomaly-based detection systems to monitor network traffic.

How to Setup Your Development Environment

Let’s start by setting up our Python environment (I’m using Python 3) and installing the following prerequisites:

pip install scapy
pip install python-nmap
pip install numpy
pip install sklearn

Building the Core IDS Components

Our IDS will comprise of four main components:

1. A packet capture system

2. Traffic analysis module

3. A detection engine

4. An alert system

Building the Packet Capture Engine

Let’s start with the packet capture engine. We’ll use Scapy for this. Scapy is a networking library that allows us to perform network and network-related operations using Python.

First, we’ll define our PacketCapture class that will serve as the basis of our IDS.

from scapy.all import sniff, IP, TCP
from collections import defaultdict
import threading
import queue

class PacketCapture:
    def __init__(self):
        self.packet_queue = queue.Queue()
        self.stop_capture = threading.Event()

    def packet_callback(self, packet):
        if IP in packet and TCP in packet:
            self.packet_queue.put(packet)

    def start_capture(self, interface="eth0"):
        def capture_thread():
            sniff(iface=interface,
                  prn=self.packet_callback,
                  store=0,
                  stop_filter=lambda _: self.stop_capture.is_set())

        self.capture_thread = threading.Thread(target=capture_thread)
        self.capture_thread.start()

    def stop(self):
        self.stop_capture.set()
        self.capture_thread.join()

Let’s quickly walk through the code and understand what these functions do. For this, you will be using threading and queues to efficiently process and capture network packets.

The init method initializes the class by creating a queue.Queue to store captured packets and a threading Event to control when the packet capture should stop. The packet_callback method acts as a handler for each captured packet and checks if the packet contains both IP and TCP layers. If so, it adds it to the queue for further processing.

The start_capture method begins capturing packets on a specified interface (defaulting to eth0 to capture packets from the Ethernet interface). Run ifconfig to understand the available interfaces and select the appropriate interface from the list.

The function spawns a separate thread to run Scapy’s sniff function, which continuously monitors the interface for packets. The stop_filter parameter ensures the capture stops when the stop_capture event is triggered.

The stop method stops the capture by setting the stop_capture event and waits for the thread to finish execution, ensuring the process terminates cleanly. This design allows for seamless real-time packet capturing without blocking the main thread.

Building the Traffic Analysis Module

Now, let’s write the traffic analysis module. This module will process captured packets and extract relevant features.

class TrafficAnalyzer:
    def __init__(self):
        self.connections = defaultdict(list)
        self.flow_stats = defaultdict(lambda: {
            'packet_count': 0,
            'byte_count': 0,
            'start_time': None,
            'last_time': None
        })

    def analyze_packet(self, packet):
        if IP in packet and TCP in packet:
            ip_src = packet[IP].src
            ip_dst = packet[IP].dst
            port_src = packet[TCP].sport
            port_dst = packet[TCP].dport

            flow_key = (ip_src, ip_dst, port_src, port_dst)

            # Update flow statistics
            stats = self.flow_stats[flow_key]
            stats['packet_count'] += 1
            stats['byte_count'] += len(packet)
            current_time = packet.time

            if not stats['start_time']:
                stats['start_time'] = current_time
            stats['last_time'] = current_time

            return self.extract_features(packet, stats)

    def extract_features(self, packet, stats):
        return {
            'packet_size': len(packet),
            'flow_duration': stats['last_time'] - stats['start_time'],
            'packet_rate': stats['packet_count'] / (stats['last_time'] - stats['start_time']),
            'byte_rate': stats['byte_count'] / (stats['last_time'] - stats['start_time']),
            'tcp_flags': packet[TCP].flags,
            'window_size': packet[TCP].window
        }

In this code section, we define the TrafficAnalyzer class to analyze network traffic. Here we track connection flows and calculate statistics for packets in real time. We use the defaultdict data structure in Python to manage connections and flow statistics by organizing data by unique flows.

The __init__ method initializes two attributes: connections, which stores lists of related packets for each flow, and flow_stats, which stores aggregated statistics for each flow, such as packet count, byte count, start time, and the time of the most recent packet.

The analyze_packet method processes each packet. If the packet contains IP and TCP layers, it extracts the source and destination IPs and ports, forming a unique flow_key to identify the flow. It updates the statistics for the flow by incrementing the packet count, adding the packet’s size to the byte count, and setting or updating the start and last time of the flow. Eventually, it calls extract_features to calculate and return additional metrics.

The extract_features method computes detailed characteristics of the flow and the current packet. These include the packet size, flow duration, packet rate, byte rate, TCP flags, and the TCP window size. These metrics are quite useful to identify patterns, anomalies, or potential threats in network traffic.

Building the Detection Engine

Now we will define our detection engine that will implement both the signature as well as the anomaly-based detection mechanisms:

from sklearn.ensemble import IsolationForest
import numpy as np

class DetectionEngine:
    def __init__(self):
        self.anomaly_detector = IsolationForest(
            contamination=0.1,
            random_state=42
        )
        self.signature_rules = self.load_signature_rules()
        self.training_data = []

    def load_signature_rules(self):
        return {
            'syn_flood': {
                'condition': lambda features: (
                    features['tcp_flags'] == 2 and  # SYN flag
                    features['packet_rate'] > 100
                )
            },
            'port_scan': {
                'condition': lambda features: (
                    features['packet_size'] < 100 and
                    features['packet_rate'] > 50
                )
            }
        }

    def train_anomaly_detector(self, normal_traffic_data):
        self.anomaly_detector.fit(normal_traffic_data)

    def detect_threats(self, features):
        threats = []

        # Signature-based detection
        for rule_name, rule in self.signature_rules.items():
            if rule['condition'](features):
                threats.append({
                    'type': 'signature',
                    'rule': rule_name,
                    'confidence': 1.0
                })

        # Anomaly-based detection
        feature_vector = np.array([[
            features['packet_size'],
            features['packet_rate'],
            features['byte_rate']
        ]])

        anomaly_score = self.anomaly_detector.score_samples(feature_vector)[0]
        if anomaly_score < -0.5:  # Threshold for anomaly detection
            threats.append({
                'type': 'anomaly',
                'score': anomaly_score,
                'confidence': min(1.0, abs(anomaly_score))
            })

        return threats

This code defines a hybrid system that combines the signature-based and anomaly-based detection methods. We use the Isolation Forest model to detect anomalies and also use pre-defined rules for identifying specific attack patterns. If you would like to know more about how the Isolation Forest model works, check out this article.

In this code snippet, the train_anomaly_detector method trains the Isolation Forest model using a dataset of normal traffic features. This enables the model to differentiate typical traffic patterns from anomalies.

The detect_threats method evaluates network traffic features for potential threats using two approaches:

1. Signature-Based Detection: It iteratively goes through each of the predefined rules, applying the rule’s condition to the traffic features. If a rule matches, a signature-based threat is recorded with high confidence.

2. Anomaly-Based Detection: It processes the feature vector (packet size, packet rate, and byte rate) through the Isolation Forest model to calculate an anomaly score. If the score indicates unusual behavior, the detection engine triggers it as an anomaly and produces a confidence score proportional to the anomaly’s severity.

Finally, we return the aggregated list of identified threats with their respective annotation (either signature or anomaly), the rule or score that triggered the anomaly, and a confidence score that suggests how likely it is that the identified pattern is a threat.

Building the Alert System

Now let’s build the last component of our IDS which is the alert system. It will process and log detected threats in a structured way. You will also have the option to extend the system to include additional notification mechanisms like Slack, Jira tickets, and so on

import logging
import json
from datetime import datetime

class AlertSystem:
    def __init__(self, log_file="ids_alerts.log"):
        self.logger = logging.getLogger("IDS_Alerts")
        self.logger.setLevel(logging.INFO)

        handler = logging.FileHandler(log_file)
        formatter = logging.Formatter(
            '%(asctime)s - %(levelname)s - %(message)s'
        )
        handler.setFormatter(formatter)
        self.logger.addHandler(handler)

    def generate_alert(self, threat, packet_info):
        alert = {
            'timestamp': datetime.now().isoformat(),
            'threat_type': threat['type'],
            'source_ip': packet_info.get('source_ip'),
            'destination_ip': packet_info.get('destination_ip'),
            'confidence': threat.get('confidence', 0.0),
            'details': threat
        }

        self.logger.warning(json.dumps(alert))

        if threat['confidence'] > 0.8:
            self.logger.critical(
                f"High confidence threat detected: {json.dumps(alert)}"
            )
            # Implement additional notification methods here
            # (e.g., email, Slack, SIEM integration)

The init method sets up a logger named IDS_Alerts with an INFO logging level to capture alert information. It writes logs to a specified file, ids_alerts.log by default. A FileHandler directs logs to the file, while the Formatter ensures the logs follow a consistent format.

The generate_alert method is responsible for creating structured alert entries. Each alert includes key information such as the timestamp of detection, the type of threat, the source and destination IPs involved, the confidence level of the detection, and additional threat-specific details. These alerts are logged as WARNING level messages in JSON format.

If the confidence level of a detected threat is high (greater than 0.8), the alert is escalated and logged as a CRITICAL level message. Note that this method is designed to be extensible, allowing for additional notification mechanisms, such as sending alerts via email or integrating with third-party systems like Slack or SIEM solutions.

Putting it All Together

Now let’s integrate all the components together into our fully functional IDS solution:

class IntrusionDetectionSystem:
    def __init__(self, interface="eth0"):
        self.packet_capture = PacketCapture()
        self.traffic_analyzer = TrafficAnalyzer()
        self.detection_engine = DetectionEngine()
        self.alert_system = AlertSystem()

        self.interface = interface

    def start(self):
        print(f"Starting IDS on interface {self.interface}")
        self.packet_capture.start_capture(self.interface)

        while True:
            try:
                packet = self.packet_capture.packet_queue.get(timeout=1)
                features = self.traffic_analyzer.analyze_packet(packet)

                if features:
                    threats = self.detection_engine.detect_threats(features)

                    for threat in threats:
                        packet_info = {
                            'source_ip': packet[IP].src,
                            'destination_ip': packet[IP].dst,
                            'source_port': packet[TCP].sport,
                            'destination_port': packet[TCP].dport
                        }
                        self.alert_system.generate_alert(threat, packet_info)

            except queue.Empty:
                continue
            except KeyboardInterrupt:
                print("Stopping IDS...")
                self.packet_capture.stop()
                break

if __name__ == "__main__":
    ids = IntrusionDetectionSystem()
    ids.start()

In this code, the IntrusionDetectionSystem class sets up its core components: PacketCapture for capturing packets from a network interface, TrafficAnalyzer for extracting and analyzing packet features, DetectionEngine for identifying threats using both signature-based and anomaly-based methods, and AlertSystem for logging and escalating detected threats. The interface parameter specifies the network interface to monitor, defaulting to eth0 (the generally named ethernet interface on most systems).

The start function initiates the IDS. It begins by starting packet capture on the specified interface and enters a loop to continuously process incoming packets. For each packet captured, the system extracts its features using the TrafficAnalyzer and analyzes them for potential threats using the DetectionEngine. If any threats are detected, the system generates detailed alerts through the AlertSystem.

The system runs in a loop until interrupted by either of the two key exceptions: queue.Empty, which occurs if no packets are available for processing, and KeyboardInterrupt, which stops the IDS gracefully by halting packet capture and exiting the loop.

Ideas to Extend the IDS

To enhance or extend the IDS, you can consider designing or implementing the following features / improvements:

1. Machine Learning enhancements: You can enhance the IDS capabilities by incorporating deep learning models like Auto Encoders for anomaly detection and using RNNs for sequential pattern analysis. This will improve the system’s ability to identify complex and evolving threats by leveraging advanced feature engineering.

2. Performance optimizations: You can optimize the IDS using PyPy for faster execution, packet sampling to handle high-traffic networks, and parallel processing to scale the system efficiently.

3. Integration capabilities: You can extend the IDS by considering support for a REST API for remote monitoring, enabling seamless interaction with external systems.

Security Considerations

When deploying the IDS, note that the system is a proof-of-concept and is not intended for production use-cases. Also keep the following things in mind:

• Run the system with appropriate permissions (root/admin required for packet capture)

• Secure the alert logs and implement proper log rotation

• Regularly update signature rules and retrain anomaly detection models

• Monitor system resource usage, especially in high-traffic environments

• Implement proper access controls for the IDS configuration and alerts

Testing the IDS on Mock Data

To validate the functionality of your IDS, you can test it using mock data that will simulate real-world network traffic. This will allow you to observe how the system processes packets, analyzes traffic, and generates alerts without requiring a live network environment.

Use the following function to test the IDS:

from scapy.all import IP, TCP

def test_ids():
    # Create test packets to simulate various scenarios
    test_packets = [
        # Normal traffic
        IP(src="192.168.1.1", dst="192.168.1.2") / TCP(sport=1234, dport=80, flags="A"),
        IP(src="192.168.1.3", dst="192.168.1.4") / TCP(sport=1235, dport=443, flags="P"),

        # SYN flood simulation
        IP(src="10.0.0.1", dst="192.168.1.2") / TCP(sport=5678, dport=80, flags="S"),
        IP(src="10.0.0.2", dst="192.168.1.2") / TCP(sport=5679, dport=80, flags="S"),
        IP(src="10.0.0.3", dst="192.168.1.2") / TCP(sport=5680, dport=80, flags="S"),

        # Port scan simulation
        IP(src="192.168.1.100", dst="192.168.1.2") / TCP(sport=4321, dport=22, flags="S"),
        IP(src="192.168.1.100", dst="192.168.1.2") / TCP(sport=4321, dport=23, flags="S"),
        IP(src="192.168.1.100", dst="192.168.1.2") / TCP(sport=4321, dport=25, flags="S"),
    ]

    ids = IntrusionDetectionSystem()

    # Simulate packet processing and threat detection
    print("Starting IDS Test...")
    for i, packet in enumerate(test_packets, 1):
        print(f"\nProcessing packet {i}: {packet.summary()}")

        # Analyze the packet
        features = ids.traffic_analyzer.analyze_packet(packet)

        if features:
            # Detect threats based on features
            threats = ids.detection_engine.detect_threats(features)

            if threats:
                print(f"Detected threats: {threats}")
            else:
                print("No threats detected.")
        else:
            print("Packet does not contain IP/TCP layers or is ignored.")

    print("\nIDS Test Completed.")

if __name__ == "__main__":
    test_ids()

This will test the system against a variety of attacks like SYN flooding and port scanning.

Wrapping Up

Now you know how to build a basic intrusion detection system with Python and a few open-source libraries! This IDS demonstrates some core concepts of network security and real-time threat detection.

Keep in mind that this tutorial is for educational purposes only. There are professionally designed enterprise-grade systems like Snort and Suricata that can handle advanced threats and large-scale deployments.

I hope you gained insights into network security fundamentals and learned how Python can be used to build practical security solutions.

When you give someone the answer, you are depriving them of that “aha” moment. You are removing the opportunity for them to learn how to reach the conclusion through their own thinking, and instead allowing them to progress with minimal effort or thought.

So how can you best help someone with their code?

What is the Socratic Method?

The Greek philosopher Plato was a student of another philosopher, Socrates. In Plato’s works, he often writes about the debates that Socrates would have with his colleagues and students.

Socrates would begin these discussions by raising common beliefs and scrutinizing them to question their compatibility with other beliefs, and guide people to reach the truth.

But how does that translate to our modern day interactions in a digital age?

Let’s consider a real example from one of our communities:

You might be tempted to give them the direct answer, such as this code example:

const sum = nums.reduce((acc, el) => acc + el, 0);

But in doing so, you have performed the logical reasoning on behalf of the learner.

Instead, you should focus on asking pointed questions to guide them to the answer. For example, you might start by asking:

❓ How would you find the sum of a list of numbers on pen and paper, without code?

It may seem counter-intuitive to ask the learner a question that requires them to step away from the code, but you are actually pointing them to the underlying logic behind their question.

Let’s assume the learner responds with something like:

I would go through each number, one by one, and add it to the sum of the previous numbers.

As the learner begins answering your questions, the dialogue should progress in this call-and-response format. Your questions should become increasingly narrow and pointed with the learner’s progress toward the solution.

For example, a series of questions might look like:

Instructor: “Great, now what is an array of numbers?”
Student: “A list?”
Instructor: “Good! How would you iterate through that list?”
Student: “With a for loop.”
Instructor: “And what does your loop need to do on each iteration?”
Student: “Add the current number to the sum.”
Instructor: “Where can you find the sum?”
Student: “I could put it in a variable outside the loop.”
Instructor: “Great, now you’re ready to try writing the code.”
— original example written for this article

At this point, the learner will likely connect the dots and reach the final solution.

Socrates in Modern Culture

Learning the Socratic method is hard. In fact, many do not encounter it until they reach their university studies. But there are examples found in modern pop culture that can help you understand how the Socratic method works.

The TV show House, M.D. is rife with examples. Take this exchange from the episode titled “Three Stories”:

House: “Kidney stones would cause what?”
Student: “Blood in urine.”
House: “What colour is your pee?”
Student: “Yellow.”
House: “What colour is your blood?”
Student: “Red.”
House: “What colours did I use?”
Student: “Red, yellow, and brown.
House: “And brown. What causes brown?”
Student: “Waste”.
— (Frapier, 2008)¹

You’ll notice how this exchange took place. House’s goal here was not to give the learner the answer, but to ask deductive questions to guide the learner to reach the answer on their own.

Consider the popular movie The Matrix:

Morpheus: “Have you ever had a dream, Neo, that you were so sure was real?”
Neo: “This can't be…”
Morpheus: “Be what? Be real?”
Morpheus: “What if you were unable to wake from that dream, Neo? How would you know the difference between the dreamworld and the real world?”
— (Wachowski & Wachowski, 1999)²

In this scene, Morpheus is applying the Socratic method to lead Neo to question his perceptions of reality. This is a rather dramatic example, but the premise remains the same: rather than telling the learner how they should think, you guide them to reach the conclusion through their own volition.

Finally, let’s look at an example from Legally Blonde:

Elle: “Your father was shot while you were in the shower?”
…
Chutney: “Yes. I was washing my hair.”
Elle: “Miss Windham, can you tell us what you'd been doing earlier in the day?”
Chutney: “I got up, went to Starbucks, went to the gym, got a perm, and came home.”
Elle: “Where you got in the shower.”
Chutney: “Yes.”
…
Elle: “…Had you ever gotten a perm before, Miss Windham?”
Chutney: “Yes.”
Elle: “How many, would you say?”
Chutney: “Two a year since I was twelve. You do the math.”
…
Elle: “Chutney, why is it that Tracy Marcinko's curls were ruined when she got hosed down?”
Chutney: “Because they got wet.”
Elle: “That's right. Because isn't the first cardinal rule of perm maintenance that you are forbidden to wet your hair for at least twenty-four hours after getting a perm at the risk of de-activating the ammonium thiglycolate? And wouldn't someone who's had -- thirty perms ? -- throughout her lifetime, be well aware of this rule? And if you, in fact, were not washing your hair, as I suspect you were not, since your curls are still intact, wouldn't you have heard the gunshot?”
—(Luketic, 2001)³

The Socratic method can often be seen in law, and this serves as an excellent example. Through this exchange, Elle is not trying to guide Chutney to reach a conclusion, but rather the spectators (in this case, the jury). This is an important distinction for a community such as ours: where engaging in a Socratic discussion with one member can actually benefit current and future members who may observe or revisit your conversation.

Goal of the Socratic Method

It is important to recognize that the goal of the Socratic method is not to present a quick exchange of information. Instead, the aim is to first get the learner to realize that they know less than they believe they do, then guide them to the answer through questions that elicit certain thought processes.⁴

When applying this to learning programming, then, it’s important to remember the phrase “challenge your assumptions”. We often assume that we know what the code we have written is doing, so our first step when it is not working is to examine those assumptions.

As we guide learners through the process of debugging, we want to ask questions that do the same. A common question in your repertoire should be “What does this line of code do?”. When the learner answers with information that is inaccurate, respond with questions that drill down into smaller components of the code – break the problem into pieces, per se.

Consider this example:

Learner: “but maybe that wouldnt work because I need to find the book not the id in the book”
Instructor: “Correct. Now, the good news is that we have a function that would ostensibly do that. The bad news is that function doesn't do that. What does that function do instead?”
Learner: “returns bookId”
— sourced from our Discord community⁵

By asking the question “What does that function do instead?”, the instructor has provided the direction for the learner to reach the correct conclusion without having to provide any specific information. The learner already knew that the function did not perform as expected, and the instructor asked the question to challenge the learner’s assumption that the function did perform.

Here’s another example:

Instructor: “This is the example from the Chalenge (sic). Did you follow that example they gave?”
Learner: “yes but i didnt understand the for-in loop, i didnt understand "(const food in refigerator)", having to state the string property name with const or let was confusing (it isnt anymore)”
Instructor: “Ok, so lets start here instead of the full solution to the Challenge”
Instructor: “What does that code do when you run it?”
…
Learner: “each of the object's keys were set to the variable food which was made in the for-in loop, it then iterates through the object and logs the key and value assigned to the key?”
— sourced from our Discord community⁶

Again we can see how the instructor focuses on asking pointed questions that guide the user toward the solutions through their own reasoning.

❗ That is the ultimate goal of the Socratic method: to guide the learner to reach the logical conclusion solely through their own reasoning.

Long-Term Benefits of the Socratic Method

The Socratic method is not solely beneficial for resolving the immediate solution. The deductive reasoning process that is applied in this approach can also serve the learner well over the long-term.

The process of asking a learner questions to challenge their assumptions and knowledge, and guide them to the solution, is something that can be internalized as well. It’s a powerful tool to have in your repertoire for approaching debugging, isolating assumptions, and even learning to articulate an issue.

That is, by walking a learner through this call-and-response conversation, the learner can also walk away with the ability to ask themselves these questions. They can take the same process of asking increasingly narrowing questions as a logical pathway and run through it mentally when they encounter future issues with their code (or even in other aspects of life)!

Conclusion

Providing a solution directly to a learner inhibits their intellectual growth and deprives them of the rewarding experience that comes from reaching their own logical conclusion. By leveraging techniques like the Socratic method, we can foster a stronger and more effective educational environment that allows learners to grow and thrive.

💡 If you were sent this article in response to a forum post, Reddit comment, Discord message, or other communication where you gave a fellow learner a working solution, please consider the merits of this method instead.

This approach can seem tedious and lengthy, and at times it is. It is far quicker (and arguably easier) to hand someone the code that resolves their problem.

But that approach, more often than not, harms the learner in the end. And, if you’ll forgive this author for an anecdote, I have seen people get frustrated when they are the subject of a Socratic approach, or lose patience with the back and forth conversation. But I have never seen them upset with or disappointed by that end result – the “aha” moment where they deduce the solution to the problem through their own reasoning.

Sources:

1. ¹ Frapier, M. (2008). Being Nice is Overrated: House and Socrates on the Necessity of Conflict. In House and Philosophy (pp. 100–101). John Wiley & Sons, Inc.

2. ² Wachowski, L., & Wachowski, L. (Directors). (1999, March 24). The Matrix (Z. Staenberg, Ed.). Warner Bros.

3. ³ Luketic, R. (Director). (2001, July 13). Legally Blonde (A. Brandt-Burgoyne, Ed.). MGM Distribution Co.

4. ⁴ Frapier, M. (2008). Being Nice is Overrated: House and Socrates on the Necessity of Conflict. In House and Philosophy (p. 103). John Wiley & Sons, Inc.

5. ⁵ lightskingeneral & plamoni (2024, January 4). [Discord conversation]. Discord. Retrieved 2025, January 6, from https://discord.com/channels/692816967895220344/718214639669870683/1192488621043888198

6. ⁶ lightskingeneral & jeremylt (2023, November 13). [Discord conversation]. Discord. Retrieved 2025, January 6, from https://discord.com/channels/692816967895220344/718214639669870683/1173733715109761065

Special thanks to ArielLeslie and JeremyLT for their help with this article.

SSH is widely used for accessing remote servers and managing code repositories securely, particularly in environments like GitHub. You’ll learn about the fundamental concepts of SSH, how it works, and practical steps to set it up on Windows. We will cover everything from generating SSH keys to testing your connection and cloning repositories.

By the end of this guide, you will have a solid understanding of SSH and be equipped to enhance your workflow with secure connections.

What we’ll cover:

1. What is SSH?

2. How Does the Secure Shell Protocol Work?

• TCP/IP

• The SSH Connection Process

3. How to Use SSH in Windows

• Windows PowerShell or Command Prompt

• Windows Subsystem for Linux (WSL)

• Third-Party SSH Clients (for example, PuTTY)

4. How to Use SSH In Windows To Connect To GitHub

• Install Through Windows Optional Features

• Download And Install From Github

• How To Generate SSH Keys

• How To Add The Public Key To GitHub

• Testing The SSH Connection

• Cloning A Repository Using SSH

5. Conclusion

Prerequisites

Before you begin this tutorial, ensure you have the following:

• Basic Understanding of Command Line: You should be familiar with using the command line interface (CLI) on Windows, including how to open PowerShell or Command Prompt.

• Windows Operating System: This tutorial is specifically tailored for users running Windows 10 or later versions.

• OpenSSH Client Installed: Ensure that you have the OpenSSH client installed on your system. You can check this by typing ssh in your command line interface.

• GitHub Account: You’ll need a GitHub for testing SSH connections and managing repositories.

• Text Editor: You’ll also need a code editor (like Visual Studio Code, Notepad++, or similar) to edit configuration files if needed.

What is SSH?

SSH stands for secure shell (protocol). The secure shell protocol is a cryptographic network protocol for operating network services securely over an unsecured network.

In simpler terms, SSH is a way to safely use services on a network, like accessing a remote computer, even when the network itself isn’t fully secure. It protects your data and communications by scrambling (encrypting) everything, so no one can eavesdrop or tamper with it, even if they intercept it.

This is especially useful when connecting to services like GitHub, where you want to securely manage code on a remote server.

How Does The Secure Shell Protocol Work?

SSH runs on the TCP/IP set of rules that computers use to communicate over the internet.

TCP/IP

TCP stands for Transmission Control Protocol. It’s one of the core protocols in the TCP/IP suite, which is used to send data over the internet. The IP stands for Internet Protocol. It's part of the TCP/IP set of rules also, and its job is to guide packets of data to the right destination on the internet.

You can read more about the TCP/IP protocol here if you’d like to go deeper.

The SSH Connection Process

The Secure Shell protocol operates by establishing a secure connection between a client (your local machine) and a remote server.

First, the client initiates the connection by specifying the server's address (IP or domain) and login credentials. During the authentication phase, the server verifies the identity of the client, typically using a password or, more securely, SSH keys. These are a pair consisting of a private key stored on the client and a public key on the server.

Once authenticated, SSH encrypts all data transferred between the client and the server, ensuring confidentiality and integrity. This secure channel allows the client to execute commands, transfer files, and manage the server remotely as if physically present while safeguarding the data from potential interception or tampering.

IP is responsible for addressing and routing the initial connection request from the client to the server. It ensures that the request, which includes the server's address, is sent to the correct destination.

Once the SSH connection request reaches the server, TCP establishes a reliable communication channel. It manages data transmission between the client and the server, ensuring that all packets are delivered accurately and in the correct order.

During the SSH session, TCP ensures that data packets are correctly transmitted and reassembled, while IP handles the addressing. SSH then uses this secure and reliable channel to authenticate the client, encrypt all transferred data, and maintain a secure connection. This allows remote management and file transfer with confidentiality and integrity.

How to Use SSH in Windows

Windows provides a few different ways to use SSH:

Windows PowerShell or Command Prompt

Windows comes with a built-in OpenSSH client. You can access this by opening PowerShell or Command Prompt and typing ssh This method is simple and requires no additional software.

Windows Subsystem for Linux (WSL)

You can enable Windows Subsystem for Linux to run a Linux distribution directly on your Windows PC. Many Linux distros come with OpenSSH pre-installed, allowing you to use ssh commands from the Linux terminal. This method is great if you prefer a Linux-like environment.

Third-Party SSH Clients (for example, PuTTY)

If you prefer a standalone application, tools like PuTTY offer a graphical interface to manage SSH connections. PuTTY is widely used for connecting to servers and supports advanced options like saving connection profiles.

Windows 10 and later versions come with a native OpenSSH client, which allows you to securely connect to remote systems over an SSH (Secure Shell) connection.

How to Use SSH in Windows to Connect to GitHub

If you're using Windows 10 or a later version, you can take advantage of the built-in SSH capabilities.

To get started, open your Windows Terminal or PowerShell, preferably as an administrator. In the command line interface, you can check if SSH is installed by typing the command ssh. This will confirm if the SSH client is available on your system.

If you got the above image, it means SSH is available on your local system.

The first thing you'll need to set up an SSH connection is a pair of SSH keys – a public key and a private key. These are typically located in the .ssh directory in your home directory if generated already.

On Windows, your home directory is usually C:\Users\owner\.ssh. This is where you will find your SSH key files if they've already been generated.

You can easily install OpenSSH on your Windows system. There are a couple of ways to do this:

Install Through Windows Optional Features

1. Open the Windows Start menu and type "optional features" in the search bar.

2. Click on "Add or remove Windows optional features" to open the Optional Features window.

3. In the Optional Features window, scroll down and find the "OpenSSH Client" option.

4. Check the box next to it and click the "Install" button at the bottom to install the OpenSSH client.

Download and Install from GitHub

1. Visit the GitHub repository for the Win32-OpenSSH (32-bit) or Win64-OpenSSH (64-bit) release, depending on your Windows version:

   • 32-bit: https://github.com/PowerShell/Win32-OpenShell/releases

   • 64-bit: https://github.com/PowerShell/Win64-OpenShell/releases

2. Download the latest .msi installer file that matches your Windows architecture.

3. Run the .msi installer to install the OpenSSH client on your system.

After installing OpenSSH, you can now generate SSH keys by opening the Windows Terminal or PowerShell and running the ssh-keygen command. This will walk you through the process of creating a new public-private key pair that you can use to authenticate your SSH connections, such as to your GitHub account.

The public key can then be added to your GitHub account settings, allowing you to connect to your repositories using the SSH protocol instead of HTTPS. Remember to keep your private key safe and secure, as it provides access to your GitHub account.

By installing OpenSSH, you've set up the necessary foundation to work with SSH keys and establish secure connections to remote systems and services like GitHub.

Now you can check if the OpenSSH server is running by accessing the Windows Services utility. To do this, press the Windows key + R to open the Run dialog, then input "services.msc" and press Enter. This will open the services manager.

The services manager:

Scroll and look for the openSSH SSH server:

If the OpenSSH Server service is running in the Services window, it means that the OpenSSH server component is properly set up on your Windows system. This is an important first step in enabling secure SSH connections.

To ensure the OpenSSH server starts automatically whenever your PC is turned on, you can further configure the service's startup type. In the Services window, right-click on the "OpenSSH Server" service and select "Properties."

In the service properties, look for the "Startup type" dropdown and select "Automatic." This will ensure the OpenSSH Server service starts up automatically whenever your computer is powered on, rather than requiring you to manually start it each time.

By setting the OpenSSH Server service to start automatically, you can be confident that the server will be ready to accept incoming SSH connections at all times, without needing to remember to start it manually. This streamlines the process of using SSH-based authentication and remote access on your Windows machine.

Remember, having the OpenSSH Server service running and set to start automatically is just the first step. You'll still need to generate SSH keys and configure your GitHub account (or other services) to use the public key for authentication. But this initial setup of the OpenSSH server lays the foundation for seamless SSH usage going forward.

Now that you've confirmed the OpenSSH Server is running on your Windows system, the next step is to ensure it can communicate through the Windows Firewall. The easiest way to do this is by using a PowerShell command.

Open PowerShell as an administrator and run the following command:

New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH SSH Server' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22 -Program "C:\Program Files\OpenSSH\sshd.exe"

This PowerShell command will create a new firewall rule that allows inbound TCP traffic on port 22 (the default SSH port) for the OpenSSH Server executable located at "C:\Program Files\OpenSSH\sshd.exe".

Here's a breakdown of the different parameters used in the command:

• -Name sshd: Assigns the name "sshd" to the new firewall rule.

• -DisplayName 'OpenSSH SSH Server': Provides a descriptive display name for the rule.

• -Enabled True: Ensures the rule is active and enabled.

• -Direction Inbound: Specifies that the rule applies to inbound network traffic.

• -Protocol TCP: Configures the rule to allow TCP protocol connections.

• -Action Allow: Instructs the firewall to allow the specified traffic.

• -LocalPort 22: Sets the local port to 22, which is the default SSH port.

• -Program "C:\Program Files\OpenSSH\sshd.exe": Identifies the specific program (the OpenSSH Server executable) that the rule should apply to.

By running this PowerShell command, you're creating a dedicated firewall rule that permits the OpenSSH Server to communicate through the Windows Firewall, enabling you to establish secure SSH connections to your system.

Remember, it's always a good practice to review your firewall settings periodically, especially if you make changes to your network configuration or want to refine access to the OpenSSH Server further.

If successful, you should get this:

Summary of the above CLI response:

• Rule Name: sshd

• Display Name: OpenSSH SSH Server

• Direction: Inbound

• Action: Allow

• Enabled: True

• Profile: Any

The rule is designed to allow inbound traffic, enabling external clients to connect to the server. The "Enabled" status confirms that the rule is currently active, while the "Profile" set to "Any" indicates that it applies to all network types, including public, private, and domain.

Security Policies:

• Edge Traversal Policy: Block

• Loose Source Mapping: False

• Local Only Mapping: False

The "EdgeTraversalPolicy" is configured to "Block," preventing traffic from traversing the network boundary, specifically disallowing connections from public networks to private ones. This helps enhance security by mitigating potential vulnerabilities.

The settings for "LooseSourceMapping" and "LocalOnlyMapping" are both set to false, ensuring that only direct connections are allowed without additional mapping.

Status and Enforcement:

• Primary Status: OK

• Status: The rule was parsed successfully from the store.

• Enforcement Status: NotApplicable

• Policy Store Source: PersistentStore

• Policy Store Source Type: Local

The "Status" field confirms that the rule has been successfully parsed from the policy store, indicating no configuration issues. The "EnforcementStatus" marked as "NotApplicable" suggests there are no restrictions that would prevent the rule from being enforced.

Remember, to set up an SSH connection we need a pair of SSH keys – a public key and a private key. Let’s see how to do that now.

How to Generate SSH Keys

• Open your Windows Terminal or PowerShell.

• Run the command ssh-keygen to generate a new SSH key pair.

• Follow the prompts to specify the save location and passphrase (optional) for your keys.

You will get what you see in the above image if you’re successful.

The public key will be saved as id_rsa.pub in the .ssh directory of your user's home folder (for example, C:\Users\you\.ssh\id_rsa.pub).

How to Add the Public Key to GitHub

• Log in to your GitHub account and go to your account settings.

• Navigate to the "SSH and GPG keys" section.

• Click on the "New SSH key" button. Give your key a descriptive title, and then copy the contents of the id_rsa.pub file (your public key) into the "Key" field.

• Click "Add SSH key" to save the new key.

Testing the SSH Connection

In your Windows Terminal or PowerShell, execute the following command to test your SSH connection to GitHub:

bashCopy codessh -T git@github.com

If prompted, enter your passphrase if you set one during the key generation process. If you did not set a passphrase, simply type "yes" when asked for a response to the handshake.

If the connection is successful, you should see a message like "Hi username! You've successfully authenticated, but GitHub does not provide shell access."

You can solve this by following the process here.

Cloning a Repository Using SSH

Navigate to one of your GitHub repositories and copy the SSH clone URL, which typically starts with git@github.com:

In your local terminal, run the command git clone git@github.com:username/repository.git to clone the repository using the SSH protocol.

Conclusion

You now have SSH working between your Windows computer and your GitHub account. This means you can work with your GitHub projects more easily and securely.

Why SSH is Cool:

1. It's super secure. SSH scrambles your data so others can't snoop on it.

2. It's convenient. No more typing passwords all the time!

3. It's flexible. You can use it for more than just GitHub.

4. You can have different "keys" for different projects.

5. It works well with automatic coding tools.

But it’s not perfect:

1. Keeping track of many SSH keys can be a headache.

2. If someone gets your private key, they could access your stuff.

3. Setting it up the first time can be tricky.

4. You need to remember to change your keys yourself for extra safety.

5. Some networks might block SSH, which can be annoying.

Remember to keep your private SSH key safe. It's like a special password for your GitHub account, so guard it well.

Even with these small downsides, SSH is still awesome for most coders. You are now all set to work on your GitHub projects more easily and safely. Have fun with your improved coding setup, and keep learning about ways to keep your work secure.

There are several popular ways to create reusable code, and each has its strengths and weaknesses. But they all share a common goal: making it easier for you to avoid reinventing the wheel.

And now, with the improvements in AI, you can take advantage of the available technology to make the development process easier, faster, and more efficient.

In this article, we’ll explore the benefits of code reuse, discuss popular tools for finding reusable code (which, in my opinion, are the best tools to use), and examine how different solutions can help you work more efficiently.

Table of Contents

1. Benefits of Code Reuse

2. Popular Options for Code Reuse

3. Conclusion

The Benefits of Code Reuse

Reusing code brings many advantages that make software development faster and better. Using code that already works helps you avoid doing the same tasks again and again. It also helps you follow the “DRY” principle, and lets you focus on more important parts of your projects.

Time Savings and Faster Development Cycles

A key benefit of reusing code is the amount of time it saves. You no longer need to spend hours rewriting common functions or tasks that have already been solved. This allows you to quickly add these pieces to new projects and spend more time building new features that make your product unique.

Improved Code Quality

Code that has already been tested and proven tends to have fewer mistakes. Using reusable code that has been checked before means you can trust that it will work properly, reducing the chances of bugs and errors in the final product.

Increased Consistency

Reusing code helps create consistency across different projects. The same functions and logic are applied in every project, ensuring everything works in a similar way. This makes the codebase easier to understand and maintain.

Enhanced Collaboration and Knowledge Sharing

When teams share and reuse components, collaboration becomes easier. Team members can work together more efficiently, using familiar code pieces. This also helps spread knowledge across the team, as everyone benefits from understanding and using the same code.

Cost Efficiency

Reusing existing solutions helps reduce costs. Since you spend less time creating new code from scratch, the development process becomes cheaper. This can also lead to faster project completion, which saves money in the long run.

Popular Options for Code Reuse

There are several tools and platforms that developers rely on to find reuseable code. There are widely used options like Google, StackOverflow, and documentation. And on the other hand there are newer AI tools like ChatGPT, Gemini, and Codiumate to mention a few.

Each option has its own strengths and weaknesses, but they all aim to make coding easier and faster by offering pre-existing solutions. Below, we’ll look at some of the most popular options and compare how they help with code reuse.

Google Search

Google is one of the most widely-used tools for finding code snippets and tutorials. Developers often search for specific solutions, reading through blog posts, forums, and official documentation to find what they need.

Advantages:

• Access to a vast amount of information, tutorials, and resources across the web.

• Great for finding niche solutions or specific examples.

• Helpful for discovering documentation and learning new technologies.

Disadvantages:

• Finding the right solution often involves sifting through a lot of content.

• Not all resources are trustworthy or up-to-date.

• It can take longer to find specific, reusable code compared to other platforms.

Google in Action: A quick goggle search will show results comprised of articles and publications related to your question. Going through these articles might help with the exact code you are searching for.

Stack Overflow

Stack Overflow is a popular online forum where developers ask questions and share solutions to programming problems, and it’s been a reliable friend to developers over the years.

It has a massive library of answers and code snippets provided by the developer community. It also now has an AI feature which we’ll discuss in the next section. You get access to different solutions to the same problem provided by developers.

Advantages:

• Large collection of real-world code examples and solutions.

• Answers are often reviewed and voted on by other users, making it easier to find high-quality code.

• Covers a wide range of coding topics and issues.

Disadvantages:

• Requires manual searching, which can be time-consuming.

• Some solutions may be outdated or not relevant to modern development practices.

• It’s not always easy to find code that fits perfectly with your specific project needs.

StackOverflow in Action: You can just open up the site in your browser and search what ever question you need and the highest rated answers will be provided:

Multi-Purpose AI Tools (like ChatGPT and Gemini)

ChatGPT and Gemini are AI tools that help developers by providing code snippets and answering programming questions. They are known for their versatility, supporting a wide range of programming languages and offering quick suggestions on how to solve coding problems.

These tools are growing each day and have truly revolutionised the tech space. They can provide code snippets quickly and with sources.

Advantages:

• Can generate code for various languages and frameworks.

• Provides quick explanations and examples and sources.

• Easy to use for beginners and experienced developers alike.

Disadvantages:

• These AI toools are general-purpose, so they may not always provide the most optimized or accurate code for complex tasks and sometimes struggle with context.

• Since they’re not focused solely on coding, they might miss important details or suggest outdated solutions.

ChatGPT in Action: Let’s see ChatGPT in action with a simple question and the sample code it gives us:

Question: “Can you generate a function to validate form inputs in JavaScript?"

Response: Here’s a reusable JavaScript function to validate form inputs. This function will validate if inputs are not empty, and optionally check for conditions such as valid email addresses, minimum character length, and numeric validation. You can expand it depending on the form requirements.

function validateForm(inputs) {
  let errors = {};

  inputs.forEach(input => {
    const { name, value, type, required, minLength, pattern } = input;

    // Check if the input is required and empty
    if (required && !value) {
      errors[name] = `${name} is required`;
      return;
    }

    // Check for minimum length
    if (minLength && value.length < minLength) {
      errors[name] = `${name} must be at least ${minLength} characters`;
      return;
    }

    // Check for valid email format
    if (type === 'email' && !/\S+@\S+\.\S+/.test(value)) {
      errors[name] = 'Invalid email address';
      return;
    }

    // Check if the input matches the specified pattern (e.g., phone number)
    if (pattern && !new RegExp(pattern).test(value)) {
      errors[name] = `${name} is invalid`;
      return;
    }
  });

  return Object.keys(errors).length ? errors : true;
}

// Example usage:
const inputs = [
  { name: 'username', value: 'john_doe', required: true, minLength: 5 },
  { name: 'email', value: 'john@example', type: 'email', required: true },
  { name: 'password', value: '1234', required: true, minLength: 6 },
];

const validation = validateForm(inputs);
console.log(validation);

This response will save you the trouble of thinking about the logic and still gives space for customization. Just make sure to test the code to make sure it works for your purposes.

Overflow AI (from Stack Overflow)

Overflow AI is a generative AI tool integrated into the Stack Overflow platform where users can ask questions in natural language and receive summarized answers. The answers include proper citations for existing content from Stack Overflow’s massive library.

Overflow AI can provide code snippets and solutions to your problems using the highest rated answers. Just keep in mind that there doesn’t appear to be a free version at the moment, so you may have to pay for the service.

Advantages:

• Access to Stack Overflow’s large collection of real-world code examples and solutions.

• Answers provided are the highest voted answers to your specific question.

• Code provided is usually of the highest quality.

• It can be integrated into your IDE and Slack platform for teams.

Disadvantages:

• Some answers may not suit your specific needs for more complex problems or code.

• It struggles with context as other multi-purpose AI tools do as there is no way to train it specifically for your project.

OverFlow AI in Action: OverFlow AI is currently only available for Stack Overflow for Teams Enterprise. Companies are required to have a subscription for their teams.

Codiumate

Codiumate is an open-source AI-powered coding assistant designed specifically to enhance the software development process. It’s trained solely for the purpose of assisting developers by providing high quality and reusable code, iterative tests, PR reviews, and code completion.

It also has a chat feature which lets you ask questions. But the most impressive feature it has, in my view, is the option to input your entire codebase or chunks of it so that it has context for the code you need from it.

When it comes to reusing code effectively, having a tool specifically designed for that purpose makes all the difference. And it’s great that Codiumate is an open-source tool, too.

Advantages:

• Codiumate is specialized. While other tools like ChatGPT are useful for many tasks, Codiumate’s focus allows it to deliver more precise and relevant recommendations for code reuse.

• It can scan large codebases quickly, identifying opportunities to reuse code, and offering seamless integration options.

• It possesses features like automatic test creation and real-time bug detection.

• It has smart code completion, a coding assistant, and a PR-agent for Pull Request reviews(chrome extension) which makes it a one-stop shop.

Limitations:

• As an open-source tool, Codiumate may require some initial setup or customization to fit your specific development environments. This can take a little extra time upfront, especially for developers who aren't familiar with the tool. But once it's integrated, the time savings and improvements to code quality make it well worth the effort. You can find the link to setup here.

• Since Codiumate is designed specifically for code reuse, it may not be the best tool for handling broader queries outside the realm of coding. If you’re looking for more general advice or help with non-code-related tasks, you might still need to use other tools like ChatGPT. But for focused, efficient code reuse, Codiumate is a great choice.

Codiumate in Action: Let’s see Codiumate in action:

After setting Codiumate up (which takes about 2 minutes), you can access the chat feature where you can add the entire codebase or highlight part of it for context to get more specific results:

Question: “Can you generate a function to validate form inputs in JavaScript?"

Response:

You can expand this function to include more validations as needed, such as checking for special characters in the password, matching passwords, and so on by clicking the “continue this chat” button.

Additional Resources

There are a ton of AI tools and platforms available with similar features and we’re almost spoiled for choice at this point.

Here are a few additional resources about some of my favorite tools I discussed:

• Read more on Codiumate

• Read more on OverflowAI

• How to reuse code with codiumate (Youtube)

• Using AI to improve code quality

Conclusion

Reusing code is a useful practice that makes development faster and easier. It allows developers to focus on creating new features while keeping their projects consistent. Reusing code also helps teams work together better and share knowledge more easily.

There are a ton of AI platforms available like to help with code reuse, and each has its own benefits (and I haven’t covered them all here.

You should take advantage of these AI tools and make your choice based on which tools suits you more. Codiumate and OverflowAI stand out above the rest to me, but the right tool will depend on what you need at that moment.

In the end, AI tools are there to make the development process more streamlined, cheaper, and to ultimately make your life easier as a developer.

I recently finished reading The Pragmatic Programmer 20th Anniversary Edition (2019) and The Clean Coder (2011). You'll find these books on almost every “top 10 Software Development books” list out there.

My goal was to learn, improve, and get the kind of knowledge that a teacher from mine used to say “can’t be obtained from just reading articles.”

When you are developing software, you may get stuck at a point where YouTube videos and StackOverflow answers don’t help. You end up reaching for the official docs, or the source code of that technology to find the answer.

The same thing happens when you want to understand the subject really deeply. Articles can sometimes fall short, and reading well-known books is the often best approach.

These books focus not only on how to write code, but on teaching you best practices for developing software and even useful life lessons. I’ll share some lessons I learned from them in this article.

Table of Contents

• How to Take Responsibility
• Testing is Important
• Teamwork Makes the Dream Work
• How to Estimate
• Tracer Bullet Development
• How to Handle Pressure
• The Importance of Refactoring
• Main Differences Between These Books
• Conclusion
• Resources

How to Take Responsibility

As a software developer, you are responsible for the code you create. You must ensure that it not only works now but will do so in the best possible way for a long time (nothing lasts forever).

The best way to make sure the code won’t fail is by testing it – having automated tests that you run each time you write new lines to be sure everything is still working.

Take Responsibility. Responsibility is something you actively agree to. — The Pragmatic Programmer

Responsibility is not only related to coding, though. You have to take responsibility for improving yourself too and get better by scheduling time for it.

Professionals spend time caring for their profession. Presumably, you became a software developer because you are passionate about software and your desire to be a professional is motivated by that passion. — The Clean Coder

Your knowledge and experience are your most important day-to-day professional assets. — The Pragmatic Programmer

Testing is Important

The importance of testing in software development is so great that both books focus on this topic.

You have to look at tests as the first users of your code, so they are the best feedback that guides your coding.

Practice TDD Test Driven Development. How does it work? Here are its three main steps:

1. Choose a feature to add and write a test that will pass after implementing it. Now, all tests but the new one should pass.
2. Write the code required to pass it.
3. Refactor the code and check that all tests still pass.

This said, it is important to look at the big picture and not to miss the main goal by writing too many tests.

It is easy to become seduced by the green "tests passed" message, writing lots of code that doesn’t actually get you closer to a solution. — The Pragmatic Programmer

There are three ways of testing: First, During, and Never. First (TDD) is the best one. During is a fallback when First is not useful. And Never is often called “Test Later” but sadly in most cases Later means Never.

The need to test first forces you to think about "good design." — The Clean Coder

Having tests give you the confidence to refactor code more often because you can check to make sure that the tests still pass after you've made your changes.

Tests should be run as frequently as possible to provide maximum feedback and to ensure that the system remains continuously clean. — The Clean Coder

Use acceptance tests to define when a requirement is done collaborating with the stakeholders.

Developers take the responsibility to ensure that tests are always automated for a simple reason: cost.

Developers have should have the goal that “QA should find nothing”. You can accomplish this by implementing different kinds of tests, in different measures from unit to exploratory tests.

Teamwork Makes the Dream Work

When working on a team, you have to be a "team player", communicate frequently, keep an eye out for your teammates, and execute your responsibilities as effectively as possible.

Good communication is key to avoiding these problems. And by "good" we mean instant and frictionless. Frictionless means it’s easy and low-ceremony to ask questions, share your progress, your problems, your insights and learnings, and to stay aware of what your teammates are doing. — The Pragmatic Programmer

Teams should be small, less than 10-12 members, where everyone knows and trusts each other. This team environment is hard to achieve therefore once you get it you have to care for it changing the projects the team works on rather than the members.

As team size grows, communication paths grow at the rate of O(n^2), where n is the number of team members. On larger teams, communication begins to break down and becomes ineffective. — The Pragmatic Programmer

To form teams around projects is a foolish approach. Individuals are only on the project for a short time and therefore never learn how to deal with each other. Teams are harder to build than projects. Therefore, it is better to form persistent teams that move together from one project to the next and can take on more than one project at a time. — The Clean Coder

Furthermore, great teams will face and solve issues together where each individual will provide their best effort. They get things done as a unit. In the end, they will be known for their performance and quality of work.

Quality can come only from the individual contributions of all team members. Quality is built in, not bolted on. — The Pragmatic Programmer

A gelled team can work miracles, anticipate each other, cover for each other, support each other, and demand the best from each other. They make things happen. — The Clean Coder

Great project teams have a distinct personality. People look forward to meetings with them, because they know that they’ll see a well-prepared performance that makes everyone feel good. The documentation they produce is crisp, accurate, and consistent. — The Pragmatic Programmer

How to Estimate

This lesson, like the majority from both books, is as important in software development as in real life. The more you practice and develop it, the more intuitive will be your ability to determine the feasibility of any task.

First, I want to clarify what estimating means by sharing The Clean Coder's definition of an estimate which refers to it as a probability distribution.

An estimate is not a number. An estimate is a probability distribution, the likelihood of completion. — The Clean Coder

To help you understand it, here is a figure of the likelihood of completion of a task for the next 11 days.

One of the basics to make great estimations is to have proper knowledge of what you are estimating.

The first part of any estimation exercise is building an understanding of what’s being asked. You need to have a grasp of the scope of the domain. — The Pragmatic Programmer

Don’t estimate alone, communicate with other people to be as accurate as possible.

The most important estimation resource you have are the people around you. They can see things that you don’t. They can help you estimate your tasks more accurately than you can estimate them on your own. — The Clean Coder

A basic estimating trick that always gives good answers: ask someone who’s already done it. — The Pragmatic Programmer

When asked for an estimation choose the units that better reflect the accuracy you intend to convey. This Estimation times scale from The Pragmatic Programmer may help you.

Although Business likes to view estimates as commitments, remember that an estimate is just a guess therefore no commitment is implied.

A commitment is something you must achieve. If you commit to getting something done by a certain date, then you simply have to get it done by that date. Professionals don’t make commitments unless they know they can achieve them. Missing a commitment is an act of dishonesty only slightly less onerous than an overt lie. — The Clean Coder

Thus said, to help Business measuring requirements and making appropriate plans, you have to remove the requirement’s ambiguity before estimating. Afterwards, keep them updated about the progress.

The trick to managing lateness is early detection and transparency. Regularly measure your progress against your goal. Be as honest as you can about all dates. Do not incorporate hope into your estimates! — The Clean Coder

Do not reinvent the wheel, use well-known estimation techniques for tasks. Here I made a summary of some techniques mentioned in both books.

• PERT
• Wideband Delphi
• Flying Fingers
• Planning Poker
• Affinity Estimation
• Trivariate Estimates
• The Law of Large Numbers

The more experience you have on a certain project, the better you will estimate its tasks. Therefore don’t worry if the first estimates you make are not as accurate as they could be. It is an incremental process as with every long term goal you want to achieve.

Like one of my favorite quotes states:

There is only one way to eat an elephant: a bite at a time. — Desmond Tutu

However, there is no way you can eat such cuteness:

Tracer Bullet Development

Tracer bullets are a special kind of bullet used in movies to mark the path they’ve taken as feedback for the shooter to aim better the next time. Therefore, the main goal of Tracer Bullet Development is to "shoot" new features into the project and get quick feedback to "aim" better on the next ones.

Tracer development is consistent with the idea that a project is never finished: there will always be changes required and functions to add. It is an incremental approach. — The Pragmatic Programmer

This method helps developers focus on the main features to be implemented so others can be built on.

In addition, it serves as proof the architecture is compatible and feasible by providing a functional and demonstrable skeleton to work from the beginning of the development process.

Look for the important requirements, the ones that define the system. Look for the areas where you have doubts, and where you see the biggest risks. Then prioritize your development so that these are the first areas you code. — The Pragmatic Programmer

Lastly, Tracer Bullet Development should not be confused with prototyping. The code from prototypes isn’t supposed to be part of the project, whereas the code from the tracer bullets isn’t thrown away. It works and is improved each iteration with new features.

Prototyping generates disposable code. Tracer code is lean but complete, and forms part of the skeleton of the final system. Think of prototyping as the reconnaissance and intelligence gathering that takes place before a single tracer bullet is fired. — The Pragmatic Programmer

How to Handle Pressure

I liked this one as well, since it helps you outside software development too. Soon or later you will be under pressure, and the best tricks to handling it are to avoid it when you can, and weather it when you can’t.

The best way to stay calm under pressure is to avoid the situations that cause pressure. — The Clean Coder

You mainly avoid it by managing commitments, keeping clean, and following your disciplines.

The best way to manage commitments is by saying no to those deadlines you aren’t sure you can meet. Keeping clean basically means that you have no mess on your systems, code, and design.

The way to go fast, and to keep the deadlines at bay, is to stay clean. Professionals do not succumb to the temptation to create a mess in order to move quickly. “Quick and dirty” is an oxymoron. Dirty always means slow! — The Clean Coder

Follow disciplines you truly believe in and stick to them all the time no matter the situation. Crisis times will come, and that’s when you have to pay attention to how you behave. If you follow your disciplines it means you believe in them.

Changing your behavior and not following your disciplines would mean you don’t truly believe in your normal behavior, and you have to shift those disciplines to improve it.

If you keep your code clean during normal times but make messes in a crisis, then you don’t really believe that messes slow you down. If you pair in a crisis but don’t normally pair, then you believe pairing is more efficient than non-pairing. — The Clean Coder

Choose disciplines that you feel comfortable following in a crisis. Then follow them all the time. Following these disciplines is the best way to avoid getting into a crisis. Don’t change your behavior when the crunch comes. If your disciplines are the best way to work, then they should be followed even in the depths of a crisis.

But you can’t always avoid pressure, so you have to learn how to get through it. You weather it by staying calm, communicating, following your disciplines, and getting help.

To stay calm, don’t panic, manage your stress, and think the problem through to find the best possible outcome. Then go for it at a steady pace, like eating an elephant. Make sure you're communicating all the time with your team and superiors to let them know when you are in trouble so you can get input and guidance. This way there won’t be any unexpected surprises in the end.

Avoid creating surprises. Nothing makes people more angry and less rational than surprises. Surprises multiply the pressure by ten. — The Clean Coder

In the same way you relied on your disciplines to avoid pressure, you should also rely on them when the moment gets tough. In fact, these times you have to pay special attention to them and neither question nor abandon them.

The communication tip includes asking for help from teammates to pair, superiors, or internet sites and forums.

Don't forget to be there for others too when they are under pressure and need help.

When the heat is on, find an associate who is willing to pair program with you. You will get done faster, with fewer defects. Your pair partner will help you hold on to your disciplines and keep you from panicking. — The Clean Coder

The Importance of Refactoring

The term Refactoring is defined by Martin Fowler as a:

Disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior. — Martin Fowler

Sometimes you will find code that does not feel right and should be fixed or improved. And you have to keep in mind the best moment to do so is now, when you find it.

It is inevitable: a program’s code needs to grow, evolve, and improve. To do so, you'll need to rethink some decisions, and the code will need to change. So be sure to have it covered with automated tests to guarantee that the external behavior does not change.

Rather than construction, software is more like gardening – it is more organic than concrete. You plant many things in a garden according to an initial plan and conditions. You constantly monitor the health of the garden, and make adjustments as needed. — The Pragmatic Programmer

When should you refactor the code? Here is a list of situations that qualify:

• To remove code duplication.
• To make some parts of the code more orthogonal.
• To update code and/or documentation that's outdated.
• To improve performance.

And here are Martin Fowler's tips on how to refactor without doing more harm than good:

1. Don’t try to refactor and add functionality at the same time.
2. Make sure you have good tests before you begin refactoring. Run the tests as often as possible.
3. Take short, deliberate steps: move a field from one class to another, split a method, rename a variable. Refactoring often involves making many localized changes that result in a larger-scale change.

The most important thing to keep in mind is that refactoring is not a certain task, it is a habit. And, as with most things in life, it is easier to do while the issues are small, as an ongoing activity while coding.

The less you refactor now the greater time you’ll have to invest to fix the problem down the road.

Refactoring as "a growth." Removing it requires invasive surgery. You can take it out while it is still small. Or, you could wait while it grows and spreads – but removing it then will be both more expensive and more dangerous. Wait even longer, and you may lose the patient entirely. — The Pragmatic Programmer

Main Differences Between These Books

I did not want this article to be a comparison – but although these books focus on similar subjects, the content and the way each is narrated are not the same.

Here are the main impressions I got from them that are not related to the content itself. These observations can help you have an idea of what to expect while reading them.

• The Clean Coder speaks about the developer in a more day by day at work way. It makes references to common situations that are given in such an environment like the relationship with sales or business people, working in a team, or saying no to clients. The developer in the Pragmatic Programmer isn’t really placed in job situations. Rather, it grants an overview of the field, structuring the book on advice: topics and tips, for any situation.
• The Clean Coder refers to the figure of the Software Developer as a Professional Programmer while The Pragmatic Programmer uses the totally unexpected term Pragmatic Programmer.
• In general terms, The Clean Coder has a more subjective sense since it shares more personal experiences from the author. On the other hand, The Pragmatic Programmer feels more objective, focusing mostly on the advice itself.
• The Pragmatic Programmer contains more code examples in different programming languages than the Clean Coder, which helps you understand the concepts discussed.

Conclusion

Keep in mind that what I've discussed here is just my own personal impressions and lessons I got from reading these two books. Each book has much more to offer, and the best thing you can do is read both yourself to form your own opinion and conclusions.

If you are interested in software development and want to get better, you should read both of them. It's worth it – they're very different and each is worth your time. And they'll bring you different knowledge and best practices for your career.

Having said that, if you still want to go the lazy way, I made this open-source project with the core sentences from each book.

Resources

The main resources are both books, which you can find easily on the internet and the project where I noted down the sentences I found most important.

I hope you enjoyed this article. You can read it too on my site along with others! If you've got any questions, suggestions, or feedback in general, don't hesitate to reach out on any of the social networks from my site.

After spending 12+ years in software development, I've found it challenging to keep learning consistently. But if you identify what you're passionate about and follow a solid learning path, you'll learn new things all the time.

This article will discuss three important areas of developer education that you may find useful in your learning journey. I have been following this path for the last two years and I've seen very positive results.

TL;DR

Become a better learner and developer by:

• Being an active part of the developer community
• Starting your own blog
• Building side projects

This article is all about experience sharing and how to make these things happen. Just chill and read it with an open mind 🧘.

Be an Active Part of the Developer Community

When we focus all our attention on something, learning becomes easy. The challenging part, though, is to know what to learn.

Where should we find help? Is there a better way to solve certain problems? How do we not reinvent the wheel? And finally, how do we make it happen continuously?

Developer Community to grow together

Being part of one or more developer communities in your areas of interest helps solve these challenges in a significant way.

Don't worry if you do not have anything to contribute in the early days. Just hang in there and soak up information, connect with like-minded people, observe best practices, and so on. You will eventually contribute, and that comes naturally.

Here are the developer communities, learning platforms, and social media outlets that I've found immensely helpful. Please note that opinions are mine, and some of you may not agree or might want to add more to the list. That's perfectly fine, and you are welcome to do so.

• Twitter: When it comes to staying relevant with technologies, soft mentoring, and getting quick tips and tricks, Twitter is a great platform. If you don't have one already, create a Twitter account and join the platform. Follow topics that interest you (using hashtags) and people who post about things you want to learn more about to get started.
  I've had a dormant Twitter account since 2009! I started using it actively a year ago and can tell you how useful it is.
• freeCodeCamp: If you are looking for focused learning, give freeCodeCamp a try. There are thousands of hours of study materials available, and it's free! You should join the forum to get insights, answers, and solutions. You can earn free verified certifications as well. If you are reading this article, you are probably aware of the freeCodeCamp curriculum already.
• Hashnode: This is one of the fastest-growing developer communities out there today. Once you join Hashnode, you can follow the tags and community members of your choice to see articles, discussions, and more in your feed. Another USP is that you get a free blogging domain where you can write your articles and publish them.
• Dev.to: a mature developer community with many contributors who write articles, discusses topics, and share thoughts. You can follow the topics of your interest to create a personalized feed.
• daily.dev: This is not precisely a community yet, but rather an aggregator of many. It does an excellent job of bringing you a list of hot articles you might be interested in without looking for them. Just install their browser extension, and you are all set to go.

Apart from those I mentioned above, LinkedIn, Reddit, Hackernoon, Code Newbie, Women who code, StackOverflow, and Hackernews are other fantastic communities to look out for.

Start Your Own Blog (or Something Equivalent)

Teaching and sharing knowledge is the best way to gain more knowledge. It is a universal truth that you can not teach a topic well without learning it well.

Having a technology blog (or something like a YouTube channel) can help you participate in the learning-sharing cycle.

Start Blogging

My key takeaways from blogging for the last couple of years are:

• Your research on a topic helps you gain more knowledge about it.
• You get better visibility in the developer community by contributing to that community.
• You create a series of knowledge bases for yourself that you can refer to or use (maybe as a speaker, in a video, and so on).
• If you are a non-English speaker and write your articles in English, it helps you get a better grasp of the language.
• You can create a strong portfolio for yourself by mentioning your blog. I found that this is beneficial when you pitch to writing for publications like freeCodeCamp News.
• Last but not least, you can earn money from your articles. Some publications pay you per article. You can be a guest writer for certain organization's blogging programs. If you are just getting started with blogging, do not put much emphasis on earning money. Focus on delivering useful content to the community. Then financial opportunities will come naturally.

If you are thinking about starting a blog with your own domain, check out Hashnode. It has everything you need as a blogging platform, and it is free. If you have an existing domain, you can map that to it too. So give it a try.

Alternatively, you can publish on dev.to, Hackernoon, and other communities I mentioned above. You can also apply to become a writer on freeCodeCamp News.

Build Some Side Projects

An effective way to polish your learning and fuel your passion is by doing side projects. In many cases, you may not be able to work on any technology you like at work. But no one can stop you from doing a side project using that tech.

Do Side Projects

A side project can help you:

• Live your passion
• Generate ideas
• Learn new skills
• Grow in new areas
• Contribute to the open-source community
• Earn rewards and money

So how do you get started? First, you should create an account with GitHub (or any other source code repository manager).

Next, you need an idea of what to build. Start small, and plan to do many projects. Try making something that you may want to use.

Then identify what you've learned from it, document it as an article, and publish it.

Before we go

I hope you've found this article insightful, and that it motivates you to check out these learning areas. I would like to hear from you. Let's connect.

You will find me active on Twitter (@tapasadhikary). Please feel free to follow me. You can read some of the other articles from my blog (blog.greenroots.info). All my side projects are in GitHub (atapas), and they are open-source. Please feel free to try/fork/follow.

You may also like:

• How to find blog content ideas effortlessly?
• Why do you need to do Side Projects as A Developer?
• 16 side project GitHub repositories you may find useful
• How to write consistently, a guide for technical writers

The answer may seem obvious: you’re done with a problem once you’ve solved it.

That’s how I approached problem-solving when I began learning to code. I was on a problem-solving treadmill: solving as many problems as quickly as possible.

And why not? There’s no shortage of problems to solve. Besides, don’t you get better by solving more problems? More to the point: what else can you do once you have the answer? As it turns out, quite a bit. The fallacy of my approach soon surfaced.

Although I solved the problem, I didn’t learn much from it. That’s because a few days or weeks later when I tried to re-solve the problem or when I came across a related one, I got really stuck. Mistakes were made. Concepts were confused. Progress was stalled.

I now realize that getting the solution is only part of the problem-solving process. Then, in the words of a mathematician named George Pólya, it’s time to “look back.”

Looking Back

Pólya writes about the problem-solving process in his book, How to Solve It, through the lens of mathematical problem-solving. But his ideas are applicable to programming. What’s particularly interesting to me is his fourth phase: looking back.

“By looking back at the completed solution, by reconsidering and reexamining the result and the path that led to it, [students] could consolidate their knowledge and develop their ability to solve problems,” Pólya writes.

In some ways, solving a problem is like creating a piece of art. There’s always something more we could do. “We could improve any solution, and, in any case, we can always improve our understanding of the solution,” explains Pólya.

For me, “looking back” is a practice of self-improvement and learning. The aim is to:

• Learn from my successes: understand what you wrote and why.
• Solidify my learning of new concepts.
• See patterns and understand the context for using a particular data structure or algorithm.

Consider a basketball player who takes 1,000 shots each day. That sounds admirable. But as he rushes to get the 1,000 shots in, his form gets sloppy. He uses the wrong technique.

He’d benefit more from taking a few hundred shots, then evaluating his performance: watching a video recording of his form, seeing the flaws, and correcting them. Then, he'd hit the court again.

Now he’ll be more informed, since he looked back and evaluated his performance. He’ll practice better.

The same is true with solving problems. The idea isn’t to check a box so you can claim you solved “x” number of problems. Instead it’s doing your best work each time and learning as much as possible along the way.

There are three reasons why looking back matters.

Reason #1: See the Patterns and Understand the Context

You’ll see similar patterns over and over again in the problems you solve.

Understand how to use a particular algorithm, like binary search. Train your eye so you know when and how to apply it. So when you encounter a related problem in the future, you’ll be ready. Doing so will save time (and frustration) in the long run.

Reason #2: Solidify Your Learning

Say you used something that’s new to you to solve a problem, like a stack or queue.

Do you really know how to use it again? Do you feel comfortable using a stack in a related problem? Take the time to understand anything new you used so you can use it again in the future.

Reason #3: Learn from Your Successes

Mathematician Richard Hamming gets to the heart of the matter with this quote from his book, The Art of Doing Science and Engineering.

“I regard the study of successes as being basically more important than the study of failures...there are so many ways of being wrong and so few of being right, studying successes is more efficient.”

As programmers, we deal with our fair share of errors. And then (many tries later) we run the program and it works. Now is a great time to put Hamming’s words to practice and study your success.

Do you understand how your program works? Do you understand what you wrote and why you wrote it?

By looking back⁠—when the information is still fresh in your mind⁠—you’re preparing your future self. It’ll help you bridge your understanding and solidify your mental models. It’ll help you improve and prevent repeating the same mistakes over again. In short, it’ll help you get better.

Four Ways to Look Back

There are a few ways that I “look back” at problems. Give them a try.

Teach Yourself

A fantastic way to help solidify your mental models is to teach yourself. After you complete a program or problem, go through your code and explain it line by line. It’s one of the best ways of “looking back” when you’re learning something new.

I’ve found this process invaluable while learning web development. After I complete a project, I copy my code into a Google Doc. Starting at the top, I make comments throughout to teach myself about important concepts.

Here’s an example of some code and some of the comments I wrote.

export default function ManageTeamMembersPage(props) {

    const [teammate, setTeammate] = useState({
       name:"",
       email: "",
       role: "",
   })

   ...

• Use props to access data passed down from the parent component.
• Add state hook. The hook takes a default, which is an object that contains everything I need for the form: name, email, role.

This method of “looking back” is about understanding. In this example, I was learning about state, props, and forms in React.

Writing out comments to explain your code will help you solidify concepts in your mind. If you can’t type a short explanation of it on the spot, then revisit the topic.

This method is equally useful for future problems and projects. I regularly pull up old problems and programs I’ve notated. I use them as a reference when writing related programs or solving related problems. Doing so reinforces key ideas, and to Hamming’s point, it helps me remember my successes: what to keep doing.

Study Solutions of Great Programmers

It’s not only useful to study your own code, but also the code of others who have solved the same problem. There are a lot of great programmers out there and we can learn from them.

After I solve a problem, I apply a learning technique that Ben Franklin used to become a better writer. His process involved trying to reproduce an article from a publication he admired after he’d forgotten the details of it.

I follow a similar process to become a better programmer.

Here’s how it works:

• Solve a problem.
• Find a programmer who’s better than you and who’s solved the same problem.
• Study their solution: read each line of code and type a comment in your editor to explain it.
• Re-solve the program after some time has passed. Use the comments you typed out as hints to guide you along the way.
• Compare your program to the one you studied.

To be clear, this practice isn’t about memorizing or copying someone else’s code—far from it. Rather, it’s about learning: get practice reading code; see another way to solve the same problem; experiment with new parts of a language; and get practice teaching yourself. It’s also about applying what you’ve learned by putting it into your own style.

Add a Constraint

See how different techniques apply to the same problem when you add a constraint. For example, you solved the problem using a hash table. Now try solving it using an array.

The idea is to gain another perspective, and adding a constraint can do just that. It’ll get you out of your comfort zone, forcing you to think creatively.

As a result, you may find a slicker approach and cut the length of your program in half. Or may realize what data structure not to use, which is equally important.

Here’s the point: you’ll have another approach at your ready when you’re faced with a related problem in the future.

Solve a Related Problem

The programming website LeetCode is great for many reasons. One is providing similar questions for problems that you solve.

In one problem on LeetCode you are given an array of integers and a target number. The aim is to find two numbers that add up to the target and return their indices.

You solve the problem.

Now solve a related one, which LeetCode provides. This time you’re given an array of integers that’s sorted in ascending order, along with a few additional constraints to differentiate this problem from the previous one.

Solving a related problem is a great way to get practice using a similar technique, data structure, or algorithm in a different context.

Looking back focuses on the process, instead of the end result. And revisiting the process matters. It’s getting out of your comfort zone, trying something new whether that’s a data structure or algorithm. It’s realizing there are different ways to solve the same problem. It’s understanding how to write better code. It’s about learning.

Yes, it takes some time to look back. But it’s time well spent: it’s how we get better.

I write about the programming skills you need to master and the concepts you need to learn, and the best ways to learn them (amymhaddad.com).

Design as a developer

Design is a term that can describe many parts of the process. Most often, it's probably thought of as the visual designers or UX designers who are actively working to make your product usable or beautiful.

A question that I commonly get asked or see online is where should one start to learn the basics of design?

This question makes me happy. It's an important part of the process that is constantly a secondary thought. And when I asked this on Twitter, I received some thoughtful responses!

We're living in an age where good UX is increasingly seen as important, but it's still nowhere near the top of the list for those who are learning their way through development. It's still an afterthought that's tossed over to the design team when you're at the 2nd half of your last sprint before launch.

So before I share with you some resources, I'm going to start by explaining why these things are important.

Giving something for your visitors to use

Starting from the top – you can have the best, most ingenious product that blows all competition away, but if no one knows how to use it, it realistically has no value.

Fictional Pied Piper's app tanked because of a bad user experience

Our fictional friends at Pied Piper learned this the hard way when their product launch with a groundbreaking compression algorithm didn't go over too well because no one knew how to actually use it.

But this is a realistic scenario. We learned from Craigslist that your website or app doesn't need to have a slick design, but it has to be usable.

Craigslist in 2001 vs 2020

For nearly 20 years, Craigslist really hasn't changed that much. A few tweaks to make things a little bit cleaner, but overall it's the same basic layout and experience.

Unfortunately, budget is a concern, and sometimes you can only stretch it so much which doesn't always include a designer. But try to at least get some outside perspective – it can even be a family member or a friend! You would be surprised how much insight you can gather by trying to get your family over the holiday to try to use your app.

The goal is ultimately to get your product in front of someone that hasn't been heads down in it for the past year to get a pair of fresh eyes and a new perspective.

Becoming more productive

Putting on my process hat – another overlooked aspect of prioritizing design time is the amount of overall time you can possibly save.

Designers don't always get it right the first time, let alone leaving developers to create the design. That's not even including the possibility that your customer or product owner doesn't like the direction you took.

Why spend a couple sprints building out a solution only to start from scratch when you have to throw it all away? Not only have you wasted your customer's time, you've wasted your whole team's time.

Get design feedback early. Work with the design resources you have to iterate and work in feedback as you go. It's important to solve the UI problems before you develop the solutions.

Design sketches from Unsplash.com

And this can generally help your team flesh out ideas. Visually seeing an idea, even as a rough sketch on a piece of paper or a disposable prototype, can completely change the way you look at the solution.

All of this early feedback and brainstorming will help you achieve the end-goal of a better product and user experience.

Developers are users

Most of people reading this post probably have written code in one form or another. But have you ever thought about how your code impacts other developers?

There are many layers to how your code can provide a better experience for another developer. The basic being – is the documentation that you created readable and easy to understand?

_Stripe's API documentation_

While it's certainly important for your fellow teammates, they should typically be able to message you on Slack or hop on a call when they run into trouble. But how about those developers that only have your documentation?

Beautiful documentation isn't that which has a beautiful design, it's one that thoroughly walks through the code that you've created so that others can understand what it does and how they can use it. Just as a visitor to your app needs to know how to use the UI, a developer working with your services needs to know how to use the API.

A little extra delight

But of course everyone wants something nice to look at. Not only can design treats make people happy and smile, it can also help avoid stress and tension.

Josh W. Comeau includes cute, happy graphics throughout his site

Having a page that's ugly or overly thick with data can be unsettling. Not only can it hurt the usability, but the clutter and color choices can make people feel uneasy, stressed, and full of anxiety.

While your team might not have the resources for full-time designers, taking a little time to learn the basics of design can help set you on your journey of becoming a well rounded developer.

Resources for learning design basics

Like it or not – we're all "designers" in a way. We're all building tools and systems that ultimately will be used by another human being. We should do what we can to make our creations friendlier to use for everyone who comes next.

Since you're now inevitably inspired to spend some time and learn about design, there are luckily a lot of resources available for you to get started.

Videos

• Abstract: The Art of Design (Netflix)
• Objectified (Amazon)

Books

• Design for Hackers (Amazon)
• The Non-Designer's Design Book (Amazon)

Email Course

• Design for Hackers
• Design Academy
• Hack Design

Misc

• Design Tips from @steveschoger

What's your take on design as a developer?

Have any of these worked particularly well for you? Am I missing something that made design click for you? Please share with me on Twitter!

Join the conversation!

Tom is no longer in your extended network

It's been almost 10 years since we saw massive changes underway at MySpace. If you don't remember what MySpace was, it was the popular social network before Facebook was king (though technically it still exists).

Tom's MySpace Profile

This was when everyone started with a new friend named Tom who was the co-founder of MySpace. Everyone could set their own "top 8" that could make or break the best friendships if you snubbed someone.

You could also set a song for your profile which would be added as a little player, back when it was “okay” to automatically play a song when your page loaded.

But MySpace is now a thing of the past, so why am I talking about it?

Making MySpace, your space

MySpace had a unique characteristic about it that you don't see in any of the popular social networks these days – it allowed you to add HTML and CSS into the page so that you could really do whatever you wanted with the your profile.

Example of custom code in a MySpace profile from dummies.com

Though from a simplistic perspective, this gives you some limited options. Maybe you can add some CSS that updates how the headers look and change a background color.

But realistically, this opened the door to adding full webpages that you could overlay on top of your profile by positioning your new layout above the original content with some z-indexing.

Custom MySpace profile for the band Filmore

The trick was, you had a limited amount of characters that you could put in each input. That had limitations, but these limitations helped promote creativity whether it was figuring out how to strip all of the whitespace in the code or by figuring out a way to add something with less overall code.

Learning through hobbies

Building MySpace profiles was addicting! It was a social medium that you could customize to make your own and stamp your name on it to show everyone what you did.

While you could do the same with your own website, it wasn't as easy at the time. And building out a profile on MySpace was easy, free, and you could connect with all of your friends.

One of my portfolio custom profiles

Each time I made a new page, I tried to think about what else I could do. I was even involved with small communities where we would share our work and help each other figure out how to hack things into a profile page (shoutout DM!).

This whole time, as I was trying to learn what more I could customize, I was learning how to code! I learned about how I could squeeze every inch of HTML I could out of a document and I also developed an understanding of how CSS selectors worked. I also learned how positioning allowed me to do cool things on the page and how to center things back when we didn’t have the option to flexbox all the things.

Flexbox all the things!

And this led to “business Colby” developing a sense of entrepreneurship. At this point, I was still in high school, but I was able to spin off a little business from building these pages.

While a modest $50 per profile seems low, that's a lot for a highschooler in 2004, not to mention that was a lot of money for another highschooler to pay to get one created!

STK Counter-Strike 1.6 team website

But this was the start of my code journey. Building off of MySpace, I started creating actual websites for Counter-Strike teams (CAL anyone?) and bands which would ultimately lead me to make a career out of coding.

Turning hobbies into jobs

I've heard others with a similar story, even others who similarly started with MySpace that eventually turned their interest into a coding career. While the MySpace era is something we all look back and laugh at, it really inspired a lot of people to find what they love to do.

Finding this kind of inspiration can be important to discovering how you want to spend your time. Whether it's a hobby or learning on the side to turn it into a career, keeping engaged with something you're actually interested in is important to learning a new craft.

But it's important to pay attention to how you engage with that new interest. You don't want to force it into a career and quickly burn yourself out. That will lead to regret and resentment.

While I got lucky that my interest in coding ended up turning into my career, I make sure to not "take my work home" as it's important to separate the work from your personal interests like random OSS projects on the side.

Find what motivates you

The important thing is to find something that can motivate and inspire you to develop your passion whether for a career or hobby. Each of us have our own journey, whether code related or not, and we should try to find something that will make us happy.

What was your inspiration for your interests? Share with me on Twitter!

Why should we write in the first place?

Maybe you've never thought about writing before or maybe you're looking for a little more motivation. Whatever the case, there are a lot of good reasons why writing can help us grow.

Helping others to learn

Coding can be tough. And we all started somewhere at the beginning.

It's a long journey that each of us go through and it's even harder to go at it alone. Luckily, we've had websites like freecodecamp.org and css-tricks.com along the way that have made that journey more manageable, but there's always going to be a challenge that just doesn't make sense.

CSS-Tricks around 2011 when I got my first official coding job via Way Back Machine

This is where we have an opportunity to share our experiences to help others grow. While our favorite coding sites have tons of amazing tutorials, maybe they're missing a key point someone needed to fully grasp the solution.

Whether you're at the beginning or middle of your journey, every experience, big or small, can help another developer on their own journey.

And this is where it can be rewarding. One of my favorite parts of writing has been getting a Twitter DM from someone about how one of my tutorials helped them learn. It's a great feeling knowing you helped someone else on their journey.

Take this as a way for you to give back to the community that helped you learn and grow.

Reinforcing your learning

Thinking back to your grade school Spanish class, one of the tricks to helping the students learn was to reinforce learning. Teachers use different methods like reading, writing, speaking, flashcards, or even that Cinco de Mayo fiesta where you learned about and celebrated Mexico's heritage to help make the language stick.

This applies to development concepts as well. While your solution is still fresh in your head, write it down! Being able to teach what you've learned by writing about it will help those concepts stick in your head.

And just because you write it, it doesn't mean you have to share it. If you're not comfortable sharing it with the world, just keep a copy in a private notebook. You'll always be able to reference it yourself and still reinforce what you know by writing it down.

Documenting our experiences

Doubling up on reinforcing your learning, by writing about your experiences, you're kind of creating your own personal StackOverflow with a list of problems you've already solved.

As much as we had hoped we would remember how to speak Spanish from the classes we took, 10 years goes by pretty quickly and you suddenly realize you can only correctly say "hola" in Spanish.

_Hola sign from Jon Tyson on Unsplash_

If you were the ambitious type, maybe you saved all of your notes. If not, maybe you remember a good website that has flashcards.

Either way, having resources to go back to, whether it's your own notes or that website you still have bookmarked, helps us to quickly reference challenges we've already come across. By writing down your experiences, you're adding to that digital notebook that you know you can always use to remember that quirk you fixed.

Growing your personal network

Everyone has their own reasons for wanting to write and share content – and that's okay! While some of us might be doing this to help others learn, others might be simply trying to get a job or grow their Twitter following.

One of the benefits of sharing your experiences is that you're immediately growing the network of developers that have a little more insight into who you are and the work you've done.

This doesn't guarantee a job, but it certainly can help. All teams are a little different in how they recruit. Smaller teams tend to include other developers more in the beginning of the process who are probably more likely to open your blog and not only read a little bit, but actually understand it.

But even if you don't immediately get hired, you're playing the long game. Opportunities will start to come in small ways that lead to bigger opportunities down the road.

Why you might be hesitating and how we can look at it another way

Sometimes motivation isn't enough. We all have our personal challenges that might prevent us from doing something we want to do. But sometimes it's a matter of finding a different perspective to give you that bit of inspiration to finally dive in.

Imposter syndrome – who am I to write about this?

This is something I've personally struggled with for the majority of my career. Whether I was confident in my ability to do my job or not, I never felt like it was enough to be one of the people writing guides for others.

My website and blog

It wasn't until another team member brought another perspective to this argument (thanks Marquet) – everyone learns differently and maybe my way of telling a story can help someone learn.

If someone doesn't read my work, what are the consequences beyond a little bit of wasted time? On the other hand, maybe my blog post resonated with someone in a way that other blog posts didn't and they were finally able to solve their problem.

And this goes for any level of experience. As Swyx says, the fastest way to learn is in public. If you're not helping someone else, you're at least helping yourself grow.

Finding time – it takes a while to write a post

Time is a funny thing – it becomes an excuse that we use when we don't want to admit the real reason we're holding back.

Of course that's not always the case. Realistically a lot of us have a ton on our plates already whether we work, take care of a family, or are volunteering to help others. It can be a struggle to not want to just sit on the couch after a hard day's work.

But consider the free time you do have. Can you spare 10 minutes jotting down a few notes or even starting an outline of some things you want to say?

While 10 minutes won't write a whole post, if you start the habit and write another paragraph or two every day, you'll slowly end up with a blog post that you can feel good about sharing.

Just don't set the bar too high. If you start off with an unrealistic goal, you're likely to burn out fast and end up back in the same position you're at now.

No topic – not knowing what to write about

From speaking with other developers, I've noticed this seems to be tied a little bit to imposter syndrome. Maybe you're getting confident enough that you're more willing to write, but you're struggling to find what to write about. Is it because you can't think of a topic or is it because every topic you think about, you don't think you're enough of an expert to write about it?

Not having a topic can be a legitimate problem, but there's inspiration all around us. If you're working every day as a developer, you're working on problems that you ultimately need to solve. Why not write about those problems? And if you can't write about the problems due to contract concerns, what about writing about them in an abstract way without any sensitive information?

Inspiration for me comes from my experience talking with others. I'm lucky to be on a team that encourages growth of junior developers and during my time helping them out, their questions lead me to think about topics that I took for granted.

Whenever a topic like that comes up, I immediately note it down for later. This topic was on that list! Over time, my list of topics has grown to a point where I don't think I'll even finish it by the end of the year.

My personal list of topic ideas

While that could feel overwhelming, it's also motivates me to have a bigger selection of what I'm in the mood to write about rather than forcing myself to both think of a new topic and write about something I don't want to write about each week.

Whatever the case, find your source of inspiration. You don't need to be an expert on it. It doesn't matter if someone more experienced already wrote about it. Write what you know. Share your experiences. Maybe you'll help someone see the problem from a different perspective.

Finding what motivates you

At the end of the day, writing is something that each of us will need to find our own motivation to do.

Try to evaluate your goals. What are you looking to achieve? If writing is something you truly want to do, you'll need to find your own source of inspiration.

While some reasons might be more valid than others, it's easier to find reasons not to do something than it is to do it.

You got this! ?

Summary of this article:

• The backstory: why CompTIA Linux+ exam.
• Review of learning resources: what is good and what is not good.
• Retrospective on studying: what worked and did not work.
• My experience during the exam: how the test was conducted and how I felt.
• My approach during the exam: what strategies to use and mindset to have.
• Getting an awesome badge: what it looks like and what's in it for me?
• Bonus: as most of my materials are sponsored by the school, I've looked for free content for you to study for the exam, such as filesystem hierarchy, basic linux commands, using AWS EC2 Ubuntu as your Linux environment. I also created a flash card in Trello, available here. Disclaimer: free content might not be sufficient.
• Exam objectives: Official CompTIA Linux+ Exam Objectives.

TLDR exam tips (I had no professional experience with Linux):

• Focus on the Activity Section of the CompTIA Study Guide and follow along with your own Linux environment.
• Practice on Performance-Based Questions (PBQs). They show around 10 choices that all look correct.
• Play around with Linux command combinations using piping (|)
• Configure IP tables, firewall, and troubleshooting.
• Learn the configuration steps, boot process, and backing up files.
• Basic bash scripting and regular expressions.

Read until the end for tips on how to have the correct mindset and strategies to answer difficult questions. Also, a video of me summarizing this article during a club meeting at our school is available at the bottom of the article. Do check the video out as it has a Q&A section towards the end.

I'd like to start off with a big thank you to School of Technology and Computing of City University of Seattle (CityU). They provided me with a job, career growth, and sponsored my CompTIA learning materials and exam voucher.

As an F-1 Student Visa holder with limited job opportunities, this allowed me to earn extra income, of which I donate a portion to freeCodeCamp on a monthly basis.

This is a loose continuation from my story: Why I abandoned my MBA to get a master's in Computer Science.

City University of Seattle Campus

Why CompTIA Linux+?

In July 2019, our school, with the help of our director of external relations, landed a contract with Amazon Web Services (AWS) and Washington Technology Industry Association (WTIA), called the AWS Apprenticeship Program (AAP).

We had the opportunity to train 23 military veterans and veteran spouses. Their requirement was to pass CompTIA Linux+ Exam and CityU's Full Stack System Developer Program which includes CompTIA Linux+ Exam preparation. Once they met both criteria, they'd be able to transition to on-the-job training with AWS.

My role at the time was a Lead Teaching Assistant (TA) and I was tasked to help build the Dean's program proposal. I coordinated with 6 other TAs to build the courses, which consisted the following: Linux I, Linux II, Networking, Web Development, JavaScript/TypeScript, Full Stack - MEAN Stack, Python, and Full Stack - Django.

As this was our first contract, we were building the plane while trying to fly it.

Picture looks fun! But not in real life lol!

We utilized the following resources for our Linux Operating System course:

• Textbook: Pro Bash Programming: Scripting the GNU/Linux Shell, Second Edition, by Chris F. A. Johnson, Jayant Varma
• Student's Linux environment: AWS EC2 Instance with Ubuntu
• Virtual Lab: InfoSec Learning

What I learned

• A lot of bash scripting exercises and coding challenges
• How to SSH to a remote server (EC2 Instance)
• Various Linux commands and configuration setups related to CompTIA Linux+

After a few weeks into the program, we researched more and listened to student feedback. Turns out we were doing too much bash scripting and lacked on CompTIA Linux+ Exam preparation.

The students were in hell during the first few weeks of bash scripting

We immediately partnered with CompTIA to get CompTIA Linux+ Exam Study materials at bulk price. We pivoted hard and changed parts of our program to integrate the new study materials.

Cool badge

Good thing our department was practicing Agile methodology and it made us very nimble.

The new resources are as follows:

• CertMaster Learn - includes eBook, Performance-Based Question (PBQ), and CompTIA Videos (Note: This was last year. If you purchased this year, you won't have the PBQs, but you get a new interface, and an additional practice test).
• CertMaster Practice - multiple-choice questions on steroids. Once you've accumulated a good amount of wrong answers, you'd get feedback on why your answers are wrong and it explains the other choices as well. Then, it pushes your progress bar back.
• CertMaster Labs - shorter lab activities compared to InfoSec Labs.

The CompTIA Linux+ Learning bundle

My thoughts on the new resources

I read the eBook from top to bottom and it was helpful to obtain an overview. However, I wasn't able to retain any knowledge from reading it. I breezed through both the CertMaster Practice and CertMaster Labs and I gained nothing as well.

What's wrong with me? Hmm..

Because of an opportunity to teach the JavaScript/TypeScript course in the AAP program and the unfortunate event of my dad's passing, my road to Linux+ Exam was eventually derailed and forgotten.

Man.... What a bummer...

Fast forward to the end of the AWS Apprenticeship Program

In November 2019, we successfully transitioned all 23 students! A 100%! We were the only partner school to have done that.

Most credit goes to the School of Technology & Computing Dean's repeated quotes of "leave no man behind" - similar to the soldier's creed: "I will never leave a fallen comrade behind", and "we stay together, we survive" (from the movie Gladiator).

The program's success was largely attributed to TAs, who supported the students since Day 1.

We asked for feedback from our students on what worked and what did not in regards to the CompTIA Linux+ materials. We got the following:

• Focus on Activity Section of the Study Guide
• Do the virtual labs repeatedly
• Going through the practice exam is not effective
• Watch ITPro.tv videos (which we were not able to provide that time)

I did not study for the Linux+ Exam for several months as I focused on learning the MEAN Stack (especially the Angular front end framework). I regularly tackled leetcode coding challenges to prepare for software developer job interviews.

I was also finishing my capstone project: Software Documentation and Architectural Analysis of Full Stack Development, and teaching two classes: Information Systems, and Data Management Communications and Networking.

No time... no time at all... or was it just excuses?

When I was nearing the end of my last quarter, I felt guilty and started studying again for the Linux+ Exam.

This time, a new plan of attack.

I started reading the eBook again from top to bottom. What was different the second time around was that for every theory, definition, command, option, subcommands, etc. that I'd encounter, I would create a flash card in Trello (available here). I used this to train with friends to do a pop quiz and execute commands in the Linux terminal as well.

On my own, I would have my own Linux environment following along with the Activity Section of the eBook. This really helped me retain my knowledge on Linux commands and configurations. I also incorporated using Linux commands in my personal laptop with MacBook Pro, as MacOS has Unix-like terminal.

I utilized the CertMaster Practice Exam again. And this time, I had a friend to go over and discuss why each choice was right or wrong, using the process of elimination.

It seems like doing hands-on and having a discussion with a friend truly helps you master a topic.

I really didn't like the CertMaster Labs and did not use it during my study.

I hesitated to study the following:

• bash scripting - I'm a software developer and I know how to code already. I have built bash scripts to boost my productivity, from class notes to web applications.
• version control - I've been using git and GitHub in my classes and personal projects.
• containers and orchestration - I have a good foundation as I've been reading DevOps books.

The Binge-Worthy ITPro.tv Videos

I accidentally discovered it. It was timely and effective.

where you can binge-watch IT videos

At the start of April 2020, we got news of incoming AAP students. The Dean asked me to check the CompTIA store again for materials, and find out what these ITPro.tv videos were all about.

At first, I was reluctant to even purchase the videos because I remembered the CompTIA Study Guide having the ITPro.tv logo at the start of their videos. I started calculating the total hours of the CompTIA Study Guide videos. The total duration was only 22 minutes.

This was a big difference compared to the standalone ITPro.tv videos with 16 hours of content. I went ahead and purchased it (with reimbursement of course).

And boy, I did not regret it. Those videos were fantastic. They dove deeper into the topics and explained the commands throughly with the help of a co-host to ask why we needed to use those commands.

The average duration of a video is about 15 minutes. A big contrast compared to the other resource, which averaged about 2 minutes per video.

I started picking videos to watch based on the topics that I knew I couldn't explain to anyone. I combined this strategy with the multiple-choice-on-steroids CertMaster Practice to fill in my knowledge gaps.

In the middle of April 2020, I scheduled my CompTIA Linux+ Exam Online Testing. They created the online testing environment due to the pandemic. As it was nearing the scheduled date, I relaxed my study patterns and limited myself to watching the ITPro.tv videos.

Exam Procedures

The online exam requires you to abide by the following procedures:

• have an internet connection
• have an external webcam or laptop webcam
• have an external microphone or laptop microphone
• have a photo of valid ID
• have pictures of your front, back, left, and right of your own testing environment
• No headphones, and smartphones and electronics should be out of reach

View the full details here.

Exam Time

instagram post of my ever supportive partner

I totally appreciate the support!

My Experience with the Online Test App

• The exam was scheduled 12:15 midnight. This was my intention to avoid any noise in the house.
• The proctor can see you through the webcam, and you can't see them.
• The online testing app slowed and crashed several times. The proctor said it was due to the high volume of test takers.

What I felt during the exam

The first few questions I got in the exam were Performance-Based Questions. It was freaking hard. I really couldn't answer with 100% confidence. I encountered a problem using Linux commands grep and awk with piping to another command.

Another problem was about investigating an SELinux policy violation, which requires you have to inspect each command and their respective output to pinpoint which file or directory needed additional configuration.

If only I had that professional experience, this would have been much easier.

As I went through more questions, I really felt so dumb and lost my confidence. I didn't know how to answer the questions about network, system, or user configurations. In the bash scripting portion, I knew I wasn't really good at RegEx (regular expressions).

I had 30 minutes left in my exam and felt like failing...

My thoughts of plan B were coming back – I would just take the exam again. This was partly due to my own perspective in life. I always tell people I'm used to failure and I'm fine failing as long as I can be persistent until I reach my goal.

Around the 28-minute mark I was about to call it quits, when I heard another voice in my brain (I'm not crazy I tell you. It's the Jedi Mind trick, read How to teach yourself to learn again).

The voice said: "don't think about failure now, use the remaining minutes and do your best!"

So I used the remaining time to jump around to different questions until I'd covered them all. This time with more focus. I changed almost 80% of my answers and I started to do the following strategies to increase my chances of getting the right answer:

• Process of elimination - I removed choices that I knew were totally wrong.
• I stuck with what I knew - I didn't choose an answer if I didn't know what it meant.
• Deeper focus - I read word by word and tried to use both comprehension and logical thinking.
• Eyes closed, mind opened - I convinced myself that I'd studied really hard for this exam so it must be stored in my brain somewhere. With that in mind, I tried to retrieve those memories to achieve 100% confidence in some questions that I had around 70% confidence answering.

I kept repeating these strategies until the last minute.

Exam Results

The exam ended and I passed – 739 out of 1000! A passing grade point is 720. After 30 minutes, I got an email from CompTIA.

my cool instagram post celebrating my achievement

I will still be humbled and say to people that I got lucky. But for myself, I won't say I got lucky because I changed most of my answers using a better mindset and more effective strategies.

Lessons Learned

• I should really stop taking these exams when I don't have professional experience yet. Why? I encountered the same kind of difficulty and failed my AWS Developer Certification Exam before (here's the article: I failed my AWS Developer Exam. What now?).
• In studying, I should've done more assessment early on to focus on my knowledge gap instead of speeding through all the materials.
• During the exam, I needed to remind myself this is my reality now. What can I do to be great in this situation? This mindset is something I can apply to my current work as well.
• Remember to have study sessions with friends or colleagues, as you'll have the chance to grow your knowledge exponentially.

quote from Gladiator (2000) movie

What now, Clark?

I rejoined CityU as a full-timer with a position of Program Manager for AWS Apprentice Program (Full Stack Web Systems Developer) last April 20, 2019.

With the Linux+ Certified under my belt, I have gained additional experience, along with my previous experience being a TA Lead with the same program. It's allowed me to better lead, teach, and mentor the new cohort. This includes my team that includes new TAs that I know for sure will be more successful than I was.

I will do my best to keep teaching and mentoring students while sharpening my software development skills, as I'm still on a journey to find a software developer job in the tech industry. I know it's hard coming from a non-tech background to transition, but we'll all get there in time.

To my readers, thank you for reading my article. I really want to focus more on technical articles but I couldn't let a good story slip away.=)

Connect with me in LinkedIn here.

I couldn't decide on my LinkedIn headline lol

Article Video Summary and Q&A