engineering - freeCodeCamp.org

Product-Led Research: A Practical Guide for R&D Leaders [Full Book]

Omer Rosenbaum — Wed, 18 Feb 2026 22:39:21 +0000

Your team needs to solve a problem, and there's no clear solution path. Multiple approaches might work, but you're not sure which. Success isn't guaranteed.

This is Research, not Development. And if you manage it like Development, things aren't going to go well for you.

Maybe you've seen this: A talented engineer spends three weeks trying different approaches, switching between them until the original problem is no longer relevant. Or a researcher declares after two days "it's impossible" and gives up, though later it turns out the problem was actually solvable. Or worse and most frustrating of all: Research succeeds technically but doesn't impact the product, and you realize you've been solving the wrong problem.

Here's the thing: managing Research is fundamentally different from managing Development.

Development has known solution paths, relatively predictable timelines, and measurable progress. You probably learned to manage Development, and know how to do it well. Managing Development is challenging, yet there are established best practices that are well known in the industry.

But Research? Research is inherently uncertain. The techniques that work for Development fail spectacularly for Research. Most organizations either approach Research by deploying techniques devised for managing Development, or entirely give up on managing Research and treat it as mystical work by brilliant people, where the best you can do is not interrupt.

It doesn't have to be this way.

What You'll Learn

By reading this book, you’ll gain practical tools for managing Research that create real product impact.

You’ll learn your two critical roles as a Research leader:

Ensure Research connects to product impact.
Ensure Research is done effectively.

You’ll understand why Research is different from Development. You’ll get concrete tools for managing Research execution, and learn proven heuristics for ensuring product impact.

Throughout the book, you'll see the mentioned methods applied to real challenges.

You’ll feel confident managing Research. You’ll understand when to use which tools. And you’ll know how to keep Research connected to product value while managing its inherent uncertainty.

Who Is This Book For?

Any engineering leader who manages Research, or has researchers on their team.

If you're a CTO, VP of Engineering, R&D Director, or Engineering Manager responsible for Research initiatives – this book is for you.

This book is for product-focused leaders who need their Research to ship value, not just produce interesting findings.

You’ll also notice that I use a casual style throughout the book. I believe that learning Research management should be insightful and practical. These are hard problems, and writing in an overly academic style wouldn't serve you well. This book is for you, written to help you succeed.

Who Am I?

This book is about you, and your journey managing Research. But let me tell you why I think I can contribute to that journey.

I am the CTO and co-founder of Swimm.io, a knowledge management platform for code. At Swimm, I've led multiple Research initiatives — from automatically keeping documentation in sync with code changes, to extracting business rules from legacy COBOL systems. We've faced genuine Research challenges where the path to success wasn't clear, where we didn't know if solutions were even possible.

I've managed Research in product environments where "interesting findings" aren't enough — where Research must ship and create measurable value. I've experienced the failure modes firsthand: Research that succeeded technically but didn't impact the product, teams that got stuck exploring endlessly, and the challenge of managing uncertainty while maintaining velocity.

I've also managed teams of researchers — brilliant people who needed guidance not on technical capability, but on connecting their work to product value and working systematically through uncertainty.

This book combines my experience leading product-focused Research with my background in teaching and making complex ideas practical and actionable.

The Approach of This Book

This is not an academic piece on Research methodology. When writing it, I had three principles in mind:

1. Practical: In this book, you will learn how to accomplish things in Research management. You will understand frameworks not just for the sake of understanding, but with a practical mindset. I sometimes refer to this as the "practicality principle" – which guides me in deciding whether to include certain topics, and to what extent.

2. Based on proven theory: While practical, the methods are grounded in Alan Schoenfeld's problem-solving research — a framework developed by studying how people actually solve uncertain problems. You'll see how Schoenfeld's components (knowledge, heuristics, control, beliefs) map directly to practical Research management tools.

3. Real examples: You will see these methods applied to actual Research challenges. Not toy problems, but real initiatives involving complex problems with genuine uncertainty. These examples show the methods in action, including when they're messy, when approaches fail, and how to adapt.

Structure of This Book

The book is organized in three parts:

Part 1: Foundations: Read this to understand what makes Research different and why it needs specialized management. This part is short — just enough to establish the framework that organizes everything else.

Part 2: Research Management Methods: These are tools that work for any Research.

Part 3: Ensuring Product Impact: Methods specifically for connecting Research to product value.

Why Is This Book Publicly Available?

In short, I'd like this book to get to as many people as possible.

If you would like to support this book, you are welcome to buy E-Book version, Paperback, Hardc over , or buy me a coffee. Thank you!

Feedback Is Welcome

I created this book to help you and people like you manage Research effectively and ensure it creates product impact.

From the beginning, I asked for feedback from experienced leaders and researchers to make sure the book achieves these goals. If you found something valuable, felt something was missing, or thought something needed improvement — I would love to hear from you.

Your feedback helps make this book better for everyone. Please reach out at: gitting.things@gmail.com.

Now, let's begin. In Chapter 1, you'll learn exactly what makes Research different from Development, and why managing them differently matters.

Note

This book is provided for free on freeCodeCamp as described above and according to Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International.

If you would like to support this book, you are welcome to buy E-Book version, Paperback, Hardc over , or buy me a coffee. Thank you!

Part 1: Foundations

Chapter 1 - What is Research?

To manage research effectively, you first need to understand what research is, and what it is not.

From now on, I will use Research (capital R) when referring to our specific concept of research in this book, to distinguish it from general uses of the word.

Consider this scenario: Your team needs to optimize a critical API endpoint. It's slow, users complain, and you know exactly what to do: profile the code, identify bottlenecks, apply standard optimization techniques. This is challenging work, but it's not Research.

Now consider this: Your team needs to automatically extract business rules from 40-year-old COBOL codebases, consisting of millions of lines of code where the original developers are long retired. You're not even sure if extracting these rules automatically is possible. Multiple approaches might work. Or none might. This is Research.

The Difference

The distinction isn't about difficulty or technical sophistication. It's about uncertainty of approach.

Throughout this book, I will adopt the following definition: Research is confronting problems where you don't know if solutions exist or which approaches will work.

Research confronts problems where:

You don't know if a solution exists.
Multiple approaches might work, but you don't know which.
The path to success is not immediately clear.
You may need to invent new techniques.

Development involves:

Applying known techniques to build specific features.
Following established approaches, even if complex.
Clear success criteria tied to working software.

I've found Alan Schoenfeld's model of problem solving [1] to be a useful framework for defining and analyzing research. Schoenfeld, studying how people solve mathematical problems, identified four** components that determine success when facing genuinely uncertain problems. His framework applies directly to software Research:

Schoenfeld's Framework for Problem Solving

1. Knowledge Base — What you know

Are you familiar with relevant tools, algorithms, and techniques?

For COBOL business rule extraction: Do you understand COBOL syntax? Static analysis? Program comprehension techniques?

Without the right knowledge, you will have to spend time acquiring it before making progress, and might miss options that could be obvious to someone with more background.

2. Heuristics — Strategies for approaching problems

We’ll cover heuristics in much more detail later. For now, here are some examples of effective heuristics:

"Work backwards from the desired output".
"Break the problem into smaller pieces".
"Try a simpler version first".
"List all assumptions and test each one".

For our COBOL business rule extraction case: "Start by manually extracting rules from one small program to understand what 'success' looks like".

3. Control — Monitoring and adjusting your approach

Recognizing when your current strategy isn't working. Deciding when to pivot to a different approach. Managing your time and resources effectively.

This is what separates experienced researchers from novices: it's not just what you know, and the heuristics that you may deploy, but when and how you use them. If you choose one approach, reflect on its effectiveness, and decide to try something different when needed, that's an example of control.

4. Beliefs and Attitudes — Your mindset toward the problem

Schoenfeld found that successful problem solvers held certain beliefs that helped them persist through challenges. Examples include:

"I can figure this out" vs. "I'm not good at this kind of thing".
"Problems have multiple solutions" vs. "There's one right answer".
"I should write things down and work systematically" vs. "I should solve this in my head".

These beliefs profoundly affect your ability to persist and succeed.

Why This Matters for R&D Leaders

You've likely seen this: A capable engineer spends three weeks on a Research task, trying approach after approach, until the original problem is no longer relevant. Or they declare after two days "it's impossible" and give up, though it turns out later that the problem was actually solvable.

The issue isn't (necessarily) capability. It's that Research requires different management, and different skills, than Development:

Knowledge base: Did they have the right background, or access to people who did?
Heuristics: Were they using effective problem-solving strategies, or just "trying things"?
Control: Did they know when to pivot? When to ask for help? When to break the problem down differently?
Beliefs: Did they believe the problem was solvable? That systematic approaches work better than random attempts?

The good news: All four components can be improved. People get better at Research through practice, exposure to effective heuristics, and environments that support good Control and healthy Beliefs.

The rest of this book provides concrete tools, like using a Research Tree, drawing backwards, and time-boxing methods, that put Schoenfeld's framework into action in a Product-led environment. These tools help you and your team apply better heuristics, maintain effective control, and build the beliefs that sustain successful Research.

But first, let's make sure we're clear on when you actually need these tools. The next chapter dives deeper into distinguishing Research from Development work.

Notes

** Actually, Schoenfeld (1992) described five components (which he terms "categories"), but I focused on four of them. For the curious reader – the one I skipped is called "Practices" – the habits and cultural norms of the mathematical environment that shape how a student approaches a problem. I chose to skip it as applying it to Research felt artificial.

References

[1] Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving, metacognition, and sense-making in mathematics. In D. Grouws (Ed.), Handbook for Research on Mathematics Teaching and Learning (pp. 334-370). New York: MacMillan.

Chapter 2 - Research and Development

Most R&D departments have plenty of Development work. Everyone agrees on what Development is. But Research? That's murkier.

Some claim every development task involves "research" – you have to test your code, try different things, read documentation. Is this Research?

Let's be precise.

Defining Research vs. Development

We established in chapter 1 the core distinction: Research involves fundamental uncertainty about whether solutions exist and which approaches will work, while Development applies known techniques to build specific features.

With this foundation, let's explore how to identify Research in practice.

A Quick Test

You're asked to reverse-engineer a specific compiled function: disassemble it and provide the equivalent code in C language. You know assembly, you know C, you have a disassembler. Is this Research?

No. You know how to proceed. It might take three days of careful work, especially if the function is complex, but it's not Research. You're applying known techniques, and know how to progress to a solution.

But if you need to understand how an entire program operates, and one possible approach is reverse engineering its compiled form, and you're not sure if that approach is even feasible time-wise or whether it will yield the insights you need? That's Research.

Key Indicators You're Doing Research

1. Fundamental Uncertainty About Solution Viability

You're asking "Can this even be done?" rather than "How should we do this?" This isn't about implementation details – rather, it's about whether the approach itself is viable.

2. Multiple Competing Approaches Without Clear Superiority

Research often means exploring several paths simultaneously, knowing that many will fail, to discover which approach (if any) can solve the problem.

3. Need for New Fundamental Techniques

You may need to invent new methods rather than adapting existing ones. Note: Not all Research creates new techniques, but the possibility exists.

Common Misconceptions

Misconception 1: Technical Complexity = Research

Many challenging Development tasks involve sophisticated algorithms, large-scale systems, or cutting-edge technologies without requiring Research approaches.

Building a distributed system with complex consensus algorithms? Challenging Development. Figuring out whether a distributed system can meet your latency requirements given your unusual constraints? Might be Research.

Technical complexity is not the same as Research.

Misconception 2: Using Advanced Algorithms = Research

Implementing machine learning pipelines with random forests or neural networks isn't Research – even though the underlying algorithms are sophisticated. The Research happened when those algorithms were first developed. However, if you are using those algorithms when trying to solve a problem where it's unclear if they will work at all, that could be Research.

Misconception 3: Research Can Be Managed Like Development

Perhaps the most damaging misconception. This leads to:

Demanding precise time estimates for uncertain work.
Expecting steady, measurable progress on fixed schedules.
Evaluating Research with Development metrics.

Research requires different approaches. This is exactly what the rest of this book addresses.

Misconception 4: Research Cannot Be Managed

The opposite extreme: treating Research as mystical work by brilliant people. The best a manager can do is not interrupt.

I've had the pleasure and privilege to work with many extremely skilled researchers. I can confidently say that this is simply not the case, as even the most talented researcher can benefit from skillful guidance.

Specifically, even the most talented researcher benefits from:

Clear connections between their work and product goals.
Structured approaches to exploring alternatives.
Regular checkpoints to assess direction.
Team collaboration and brainstorming.

Research is not magic, it can be managed effectively.

Your Role as Research Leader

When leading Product-led Research, your job has two parts:

1. Ensure the Research makes the biggest possible impact on the product

This is your most important responsibility. "Successful" Research that doesn't impact the product is a failed project. Your job is to maintain the connection between Research work and product value – not just at the start, but continuously.

This means:

Starting with clear product needs, not interesting technical questions.
Regularly validating that the Research still serves the product goal.
Making trade-offs between thorough exploration and shipping impact.

We'll cover this in detail in Part 3.

2. Ensure the Research is done in the most effective manner

Even brilliant researchers benefit from structured approaches. Your role is to help the team work systematically rather than randomly.

This means:

Helping identify which questions are worth answering.
Introducing better heuristics when the team is stuck ("Let's work backwards," "Let's time-box this investigation").
Preventing common failure modes like endless exploration or premature commitment.

(We'll cover this in detail in Part 2.)

Responsibility (1) is defining the right goals. Responsibility (2) is reaching these goals effectively.

The rest of this book provides concrete tools for both responsibilities.

Part 1 - Summary

Research is work where the path to success is not immediately obvious. It requires:

Knowledge of relevant domains.
Effective problem-solving strategies (heuristics) .
Monitoring and adjusting approaches (control).
Healthy beliefs and persistence.

Your role as a Research leader is to:

Ensure Research connects to product impact.
Ensure Research is done effectively.

You need different tools to manage Research versus Development – in order to make sure Research is done effectively. Part 2 provides those tools.

Part 3 will focus on connecting Research to product impact.

Part 2: Research Management Methods

Chapter 3 - Why Methodology Matters: A True Story

It was a late evening, and the classroom was filled with three dozen students. They were all sitting in front of their computers, working in silence. I was leading a cybersecurity training focused on reverse engineering. Each exercise included a single compiled program without its source code, with one goal: "understand how this program works." The output would be either a detailed document, an equivalent program implemented in a high-level language, or both.

This particular evening, the students were furiously working on reverse-engineering a game. The instruction was: "understand the game's rules, and document them thoroughly." The game had a user interface with two dimensions and could be played against the computer. Moving behind the students, I could see their screens with various reverse engineering tools open.

At some point we asked them to stop, turn around and look at the instructor. The instructor then provided a guided solution – this was a technique we used quite frequently, showing the students the "right" way to approach a problem they had spent some time tackling. The instructor opened the game, looked at the screen, opened the "File" menu, clicked on "Help" – and there it was, the entire description of the game rules.

The Lesson

The room erupted in nervous laughter. Some students looked embarrassed. Others seemed frustrated. But everyone understood the point.

These were capable people. They had the relevant knowledge base, in Schoenfeld's terms, as presented in chapter 1. That is, they had the relevant technical skills – they knew both assembly and C, they knew how to use disassemblers, debuggers, and all the sophisticated tools of reverse engineering. Yet they had missed something fundamental: checking if there was a simpler solution first.

This happens all the time in Research work.

A team spends weeks diving deep into a complex technical approach, when a simpler path existed that they never explored. Or they try one thing, then another, then another – never systematically evaluating which approaches make sense before starting.

The problem isn't capability. It's approach.

This is exactly why you need structured methods for Research management. Without them, even your most talented people will waste time, miss obvious solutions, and burn out trying random approaches.

As a reminder, in chapter 2, we discussed your role as a Research leader:

Focusing on responsibility (2), the illustration here shows how the same peak can be reached via difficult climbing (reverse engineering) or by using a hot air balloon (reading the Help menu). Part of your job as a Research leader is to help your team find the easiest path to the goal.

What This Part Covers

The rest of Part 2 provides concrete tools to prevent these problems. You'll learn:

The Research Tree – A visual framework for systematically exploring solution paths.
Time-boxing methods – How to limit exploration without killing creativity.

These aren't abstract concepts. They're battle-tested techniques that directly address the failure modes you've probably seen: teams spinning their wheels, giving up too early, or getting stuck on the wrong approach.

Let's get started.

Chapter 4 - The Research Tree

You've probably seen this: An engineer/researcher starts down one path, hits an obstacle, tries something else, hits another obstacle, then tries a third approach. Three weeks later, they're still stuck – or worse, they've built something that technically works but doesn't solve the actual problem.

The issue isn't persistence. It's that they never mapped out the solution space. They never visualized which approaches might work, which questions need answering, and how everything relates to each other.

The Research Tree solves this problem.

It’s a way to visualize and manage the Control component of Schoenfeld's framework from chapter 1 – monitoring and adjusting your approach. This is where most Research efforts struggle.

What Is a Research Tree?

A Research Tree is a living visual representation of your Research journey. It captures three things:

Possible solution paths – different approaches you might take.
Open questions – what you need to learn.
Closed questions – what you've already discovered.

Unlike a static plan, the Research Tree grows and changes as you learn (which is one reason I like the name "tree" 😇). You start with what you know, then update it continuously as you investigate. Dead ends get marked. New branches appear. Questions get answered and new questions emerge.

Think of it like this: You're exploring a cave system. You don't have a map – you're creating the map as you explore. You mark passages you've tried. You note which ones are dead ends. You write down questions: "Does this passage connect to the main chamber?" "Is there water down this route?" As you explore, you answer some questions and discover new ones you hadn't considered. You write down the answers you found, and the experiments you conducted to find them ("I tried going left, hit a wall after 50 feet").

Research works the same way. The Research Tree is both your map and your log.

Your First Research Tree

Let's build one together. Consider a common engineering challenge:

Goal: Reduce API response time from 800ms to under 100ms

(Note: despite not being a real Research task, I chose this example for its simplicity to illustrate the process of creating a Research Tree. It can also show you how Research Trees can be useful in a variety of scenarios.)

You start with a fundamental question that needs answering. What's the first thing you need to know?

Take a moment to think about this before reading on.

The first question is usually: Where is the bottleneck?

Until you answer this, you don't know which approaches make sense. Let's draw this:

You can use a simple pen and paper, a whiteboard, or a digital tool to draw this out. When creating your very first tree, I highly recommend doing it by hand – the physical act of drawing will help you feel comfortable with the process.

Now, how can you answer this question? What approaches might tell you where the bottleneck is?

You might identify:

Profile the application with a performance monitoring tool.
Add detailed logging to measure each operation.
Use database query analysis tools.

Let's add these as branches:

(Note: the Brown status means "uncertain, needs investigation" – more on this later.)

Each approach is an investigation you could run to answer the question.

The Power of Stopping to Think

Before we go further, notice what just happened. You stopped.

Instead of immediately jumping into "Let's add more logging!" or "I bet it's the database, let's check queries," you identified multiple possible approaches. You're looking at three different ways to answer the same question.

This is already valuable. Most engineers would have jumped straight into whichever approach came to mind first. Maybe you've done this yourself: spent two days adding detailed logging, only to discover later that a profiler would have given you the answer in 30 minutes.

By creating this tree, you've avoided that trap. You can see all the approaches before committing to any of them. It doesn't guarantee you will choose the "right" path - you can't always do that in advance – but it will minimize the chances of you omitting it completely.

Remember the reverse engineering students from chapter 3? They never created this tree. They jumped straight to the first approach they knew: disassemblers and debuggers. They didn't stop to think: "What are all the ways we could understand this game's rules?" If they had, they would have listed approaches like: reverse engineer the binary, check the Help menu, just play the game, examine config files, watch network traffic. And if they'd evaluated those approaches using the framework you're about to learn, "check the Help menu" would have scored perfectly: fastest feedback (30 seconds), lowest cost (zero), best coverage (complete rules). Instead, they spent hours on complex reverse engineering when a simple menu click would have worked. Don't be those students.

Now comes the critical question: Which branch do you try first?

Consider our tree again:

Choosing Your First Path

You're looking at three approaches: Profile, Logging, DB Analysis. Each is a heuristic (in Schoenfeld's terms, as presented in chapter 1) – a problem-solving strategy that might work. How do you decide which one to try?

Sometimes, the answer is obvious (I wouldn't really use a framework to check if there's a "Help" menu). Let me show you the framework I use when it's not so clear which path to choose. Ask yourself these questions:

1. Which gives the fastest feedback?

How long until you get an answer?

Profile: Can run in 10 minutes. Setup is probably 5 minutes.
Logging: Need to add logging code, deploy, wait for traffic – maybe 4 hours.
DB Analysis: Need to enable slow query log, wait for data – maybe 2 hours.

Fastest feedback wins. You want to learn quickly.

2. Which has the lowest cost?

What do you need to set up or change?

Profile: Just attach a profiler – no code changes.
Logging: Need to modify code, test, deploy.
DB Analysis: Need database permissions, might need config changes.

Lower cost wins. Why spend hours adding logging if you can get the answer without changing any code?

(In your environment, it may be different. Perhaps logging is really easy, while profiling is super hard to set up. I am not claiming that profiling is a better heuristic than logging – it depends on your circumstances.)

3. Which answers the most questions?

Some approaches answer not just your immediate question, but related questions, too.

Profile: Shows you CPU, memory, database, network – a complete picture.
Logging: Only shows what you logged.
DB Analysis: Only shows database queries.

Broader coverage wins. A profiler might show you that 70% of time is database queries and that 20% is network latency – information you wouldn't get from narrow approaches.

So this is an easy one. Profiling wins on all three criteria. That's your first approach to try.

Update your tree:

This doesn't mean the other approaches are bad, or that this one will necessarily turn out to be the best. It means that profiling is the best starting point given what you know right now.

What If Your First Choice Doesn't Work?

Let's say you try profiling and hit a problem: Your profiler can't attach to the production environment due to security restrictions.

This is valuable information. Mark Profile as Red and move to your next best option:

Now you try Logging. But notice: You didn't waste days trying Profile in production. You tried it, hit a blocker, immediately pivoted to Logging. The tree helped you move quickly.

Choosing When Everything Seems Equal

Sometimes multiple approaches look equally good. Profile and DB Analysis might both be fast, low-cost, and have decent coverage. How do you choose?

Pick one and move on. Don't spend an hour analyzing which approach to try for 30 minutes. The meta-work (deciding) shouldn't take longer than the actual work (trying it).

When approaches seem equal, ask: "Which one am I more familiar with?" or "Which one does the team have experience with?" Use your judgment, make a choice, and start learning.

The worst decision is no decision.

In general, this approach might feel like overkill. Should you really sketch out trees and compare branches before actually doing something?

The surprising answer is that while almost always it feels like overkill, almost every single time, it turns out to be worth it. Try it a few times and you will see for yourself.

Heuristics Can Be Combined

Sometimes you don't have to choose just one. You might run Profile and enable DB Analysis simultaneously. If they don't conflict and you have the time, parallel investigations can be powerful.

But be careful: Don't try to do everything at once. Start with your best option. If that doesn't fully answer your question, then add another approach.

How Answers Lead to New Questions

After marking "Profile" as red, you moved on to "Logging". You added a few indicative log messages and let the system run for a day. You discover: 70% of response time is database queries.

This answer eliminates the need for other approaches (you don't need DB Analysis now – you found the answer). But more importantly, it reveals new questions:

See how the tree grows? One answered question spawns two new questions. Each of these new questions will have its own approaches for answering them.

And you'll apply the same framework to choose which question to answer first: Which gives fastest feedback? Which has lowest cost? Which answers the most questions?

Let's expand "Which queries are slowest?":

Again, you'd evaluate: Which approach gives fastest feedback? Enabling slow query log is probably fastest, as you’d just flip a config flag and wait a few minutes.

You decide to enable the slow query log. After investigating, you discover: User profile queries are slowest: they make 15 separate database calls (N+1 problem).

(What is the N+1 problem in this context? It means that when fetching user profiles, the code first queries for a list of users (1 query), then for each user, it makes an additional query to fetch related data (N queries). If there are 15 users, that's 16 queries total. This is inefficient and slows down response time.)

This answer leads to a new question:

Now you have three solution approaches. Again, evaluate them:

Rewrite queries with JOINs: Fast to implement, proven pattern.
Add Eager Loading: Depends on your ORM, might be quick.
DataLoader Pattern: Requires learning new pattern, takes longer.

Rewrite queries with JOINs probably gives the fastest feedback if your team knows SQL well.

The Complete Picture

Let's see how the full tree looks after a few days of investigation:

(Note: for the "How many queries per request?" node – I skipped the approaches for brevity.)

Reading this tree:

We started with "Where is the bottleneck?" and chose profiling (fastest feedback).
We found out the profiler can't attach to the production environment due to security restrictions, so we pivoted to logging.
Logging revealed that 70% of response time is database queries.
That answer led to two new questions about specific queries.
For "Which queries are slowest?", we chose slow query log (fastest to enable).
Answering it led to another question about fixing the N+1 problem.
For "How can we fix N+1?", we evaluated the approaches and chose "Rewrite queries with JOINs" (team knows SQL, fastest to implement).
Meanwhile, "Can we reduce query count?" is still open and has its own approaches to investigate.

Color-Coding Status

When you're creating your Research Tree, you'll mark both questions and approaches with a particular status. Of course, the following specific colors are just suggestions – the important thing is to keep something consistently so you can quickly see the status at a glance.

For Questions:

Open: Not yet answered.
Closed: Answered (show the answer).
Blocking: Must answer before proceeding with an approach.

For Approaches:

Green: Viable, worth pursuing.
Brown: Uncertain, needs investigation.
Red: Dead end or not viable.

In our example above:

"Rewrite with Joins" is Green because we've identified that it addresses the specific N+1 problem and that the team is confident in implementing it.
"Redesign API" is Red because it would take too long for this project.
Other approaches are Brown because we haven't investigated them yet.

Additional Tips

Keeping the tree clean and simple is important, and obsessing over its looks and details really misses the point. That said, some readers will find benefits by adding a few more details to the tree, specifically:

Order: add a number next to a specific branch when tackling it, so it is easy to track which direction you tried first, which one followed and so on.
Pivot Explanations: if you chose to pivot from one branch to another, write why. This might help when you revise your decisions later, or when reviewing with your team (as described later in this chapter).

The Research Tree Prevents Common Pitfalls

The Research Tree with this decision-making framework addresses five critical failure modes:

1. Jumping on First Idea: Without a tree, people implement the first approach they think of. The tree forces you to identify alternatives and evaluate them systematically before starting.

2. Tunnel Vision: Even when considering alternatives in the beginning, people tend to lock onto one approach and not pivot from it even when it turns out to be the wrong choice. The tree makes alternatives visible and helps you not only choose the best starting point, but also reevaluate continuously.

3. Inefficient Learning: Teams might try expensive, slow approaches first when faster, cheaper ones exist. The decision framework helps you learn quickly.

4. Answering Questions You Don't Need To: Teams waste time investigating interesting but irrelevant questions. The tree shows how questions connect – you only need to answer questions that lead to your goal.

Time to Practice

Open your favorite drawing tool, or just grab a piece of paper. Think of a Research problem you're currently facing or recently faced.

Now answer these questions:

What's your goal? Write it at the top.
What's the first question you need to answer? Write it below the goal.
What are 2-4 approaches to answer that question? Draw them as branches.
For each approach, evaluate:
- How fast is the feedback?
- What's the cost?
- How much does it answer?
Which approach scores best? Mark it "TRY FIRST".

Actually do this. Don't just read and think "I understand." Drawing the tree and evaluating approaches forces you to be explicit, and you'll immediately see gaps in your thinking.

I'll wait.

...

Don't worry about me, I'm enjoying some really great coffee in the meanwhile.

...

Done? Good. You now have your first Research Tree with a clear starting point.

Using the Tree with Your Team

Research Trees become even more powerful when shared with a team. It actually provides you, the Lead, with a way to see what directions the team is executing upon, and why. Your job here is to make sure the framework is used. Help your team stop and ask: are we asking the right questions? Are there approaches that we missed? Are we choosing the right approach?

During Planning:

Draw the tree together as a group.
Brainstorm questions and approaches.
Evaluate approaches using the framework (fastest feedback, lowest cost, best coverage).
Everyone sees why you're trying a particular approach first.

During Execution:

Update the tree as you learn.
When stuck, revisit the tree to identify alternative approaches.
Make sure you consider whether you are asking all of the important questions, and whether you are considering all relevant approaches.
If you pivoted from a branch, explain your reason and ask if someone can challenge your logic.
Conduct regular tree reviews (weekly or bi-weekly).

Note that the Research Tree is also useful for one-on-one sessions: you can review the tree with individual team members to understand their progress and help them choose next steps. It actually makes the Control component of Schoenfeld's framework much easier to manage - as you see the variolus questions and approaches laid out visually.

Tools

Pen and paper (seriously, this works great).
Whiteboard (for team sessions).
Miro, Mural, or similar digital whiteboards.
Mind mapping software (XMind, MindNode, and so on).
Even a simple text file with indentation.

The tool doesn't matter. What matters is that the tree exists, is visible, and gets updated.

Pro Tips

Start with the most important question

Don't try to list all possible questions upfront. Start with the one question that, if answered, would most clarify your path forward. Answer it, then see what new questions emerge. More on finding the questions to start from in chapter 7.

Show how answers lead to new questions

When you close a question, immediately ask: "What new questions does this answer reveal?" Draw those as branches from the answer.

Update questions weekly

In your weekly check-ins, explicitly review: Which questions did we close? What did we learn? Which new questions emerged? Which questions are blocking progress?

Re-evaluate when context changes

If you learn something new that changes the evaluation (maybe a tool you thought was fast turns out to be slow), re-evaluate your approaches. The tree is living – update it.

Recap - The Research Tree

The Research Tree is a living visual framework that:

Shows questions you need to answer to reach your goal.
Maps approaches for answering each question.
Helps you choose the best starting point for each question.
Prevents jumping on the first idea without considering alternatives.
Captures how answers lead to new questions.
Tracks status of questions (open/closed/blocking) and approaches (green/brown/red).
Documents the investigation path so the team understands why decisions were made.
Evolves as you learn – questions get answered, new questions emerge.

Key structure:

Goal → Question → Approaches to answer it → Answer → New questions

Decision framework for choosing approaches:

Which gives fastest feedback?
Which has lowest cost?
Which answers the most questions?

In the next chapter, you'll learn how to manage execution using time-boxing and decision points to keep your Research moving forward without getting stuck.

Chapter 5 - Time-Boxing Research Explorations

In the previous chapter, you learned about the Research Tree, a powerful tool for visualizing and managing Research efforts. The tree helps you systematically explore different solution paths and keep track of open questions. It also helps you decide which path to try first.

But once you've chosen a branch, how long should you pursue it before stepping back to reconsider?

The Problem: Research Without Time Limits

A researcher investigates whether a machine learning model can predict code complexity. Day one goes well. Day two, they need more features. Day three, a different architecture might work better. They switch. Day four, the new architecture needs different preprocessing.

Two weeks later, they're still on this path. When you ask about trying alternatives, they say "I'm close. Just need a few more days."

Three weeks in, they admit this approach isn't viable. Meanwhile, a simpler approach sat unexplored on the Research Tree.

Without a defined checkpoint, there's no natural moment to ask: "Given what I've learned, is this still the best path?" The sunk cost fallacy (continuing an approach because you've already invested time, rather than because it's still the best option) takes over. This is extremely common by Researchers, who tend to be dedicated, brilliant people who get fixated on problems they try to solve.

Time-Boxing: Creating Decision Points

Let's face it: it's hard, even impossible, to estimate how long a Research task will take. But you should still provide time limits based on how long you're willing to invest before reconsidering.

Time-boxing provides mandatory decision points.

Note that we are not talking about deadlines, but decision points. These are moments where you stop and evaluate: What did I learn? Is this still the most promising path?

As a rule, for every task longer than a day, define a time limit. For example: "I'll spend three days investigating whether we can cluster files into logical folders based on their I/O operations."

Three things can happen:

Early success: The researcher figures it out in a few hours. Done early, move to the next question.
Early blocker: After one hour, they discover they don't have access to filenames. They can immediately reconsider: "Without filenames, is this viable?"
Time box expires: Three days pass with partial progress. Now comes the mandatory decision point.

The Decision Point

When your time box expires, stop. Pull out your Research Tree (yes, you already love that Tree). Ask three questions:

1. What was your goal in pursuing this direction?

Which question were you trying to answer? Why did you choose this approach over alternatives?

2. What did you learn?

Document your discoveries, even if incomplete:

"The approach works but needs more sophisticated algorithms than we thought."
"We need data we don't currently have."
"This is harder than expected – would take at least 2 more weeks."
"We're 70% there – just need to handle edge cases."

3. Given what you learned, is this still the most promising path?

Look at your Research Tree. You have other branches. Given what you now know, is continuing the best use of time?

You have three options:

You can continue with a new time box: "I've solved the core challenge. One more day for edge cases." Define the new time box. The key: you consciously decided to continue based on what you learned, not inertia.

You can pivot: "This would take two more weeks, and I'm not confident it'll work. There's a simpler approach on my tree." Mark this branch Red. Move to a different branch.

Or you can reconsider the question: "Files in this codebase don't have clear I/O patterns. Maybe I should try clustering by function dependencies instead." Go back to your tree and identify a different question.

Example: Detecting God Objects

You're detecting God Objects (classes that do too much) using static analysis. Your Research Tree shows three approaches: complexity metrics, method naming patterns, or machine learning on AST features.

You choose complexity metrics (fastest feedback, lowest cost) and set a 2-day time box.

Day 1-2: You implement cyclomatic complexity and SLOC metrics. Results: 65% accuracy, but 40% false positives.

Decision point: You stop. Looking at your tree, method naming patterns might provide complementary information. You decide to spend 1 day exploring whether combining both approaches improves accuracy.

Day 3: Combined approach: 78% accuracy, 25% false positives. Promising.

New decision: Time-box 3 more days to refine and test on larger codebases.

Notice what happened: The 2-day time box forced assessment before perfecting the first approach. You learned combining approaches was better. Without time-boxing, you might have spent a week perfecting complexity metrics alone.

How to Set Time Boxes

When choosing a branch on your Research Tree, ask: Is this shorter than a day? A few days? Longer than a week?

Shorter than a day: Just do it. Don't overthink time-boxing for tasks this short.

A few days (2-7 days): Time-box for 2-5 days. I usually time-box for slightly less than my estimate – if I think 4 days, I set 3 days. This forces reflection based on learning, not arbitrary completion.

Longer than a week: Time-box for one week maximum. Even if you're making progress, a week is long enough that stepping back is valuable.

Don't overthink the exact duration. The goal is creating natural stopping points. The specific number matters less than having some checkpoint.

Critical: Time-boxing is not about pressure. You're not failing if the time box expires without solving the problem. That's expected in Research. What time-boxing does is force moments of reflection that prevent weeks spent on directions you'd have abandoned if you'd stopped to reconsider.

Using Time-Boxing with Your Team

When managing researchers, set time boxes together during planning. Make sure they understand it's a decision point, not a deadline. Schedule the review explicitly.

At the review, look at the Research Tree together. Ask the three questions. Make the decision collaboratively. This prevents researchers from getting stuck without asking for help, and makes pivoting feel like a positive decision rather than failure.

Integration with the Research Tree

Time-boxing combines naturally with the Research Tree (chapter 4): each branch gets a time box, and when it expires, the tree shows your alternatives.

Time-boxing creates moments to ask "given what I've learned, what should I do next?" The Research Tree helps you answer that question.

Recap

For tasks longer than a day, set a time limit (2-5 days for medium tasks, max 1 week). When time expires, stop and evaluate using your Research Tree: What was my goal? What did I learn? Is this still the best path?

Time boxes acknowledge that research estimation is hard. They prevent sunk cost fallacy and endless exploration. They're not about pressure or finishing "on time" – they're about forcing reflection instead of momentum-driven continuation.

Combined with the Research Tree, time-boxing gives you control over research execution while respecting its inherent uncertainty.

Part 2 Summary

Effective Research requires structured approaches, not just technical skill:

The Research Tree visualizes solution paths, open questions, and closed questions.
Time-boxing prevents endless exploration while preserving flexibility.
Systematic evaluation helps you choose the best path forward.

In chapter 2, I argued that your role as a Research leader is to:

Ensure Research connects to product impact.
Ensure Research is done effectively.

This part handled the latter part of your role: how to ensure Research is done effectively.

Within this part, your role is to:

Ensure the team uses these methods consistently.
Help identify when to pivot, when to persist, and which questions matter.

These tools prevent common failure modes: jumping on the first idea, tunnel vision, inefficient learning, and wasting time answering irrelevant questions.

Part 3 shows how to connect Research to product impact.

Part 3: Ensuring Product Impact

In chapter 2, I argued that your role as a Research leader is to:

Ensure Research connects to product impact.
Ensure Research is done effectively.

Part 2 addressed (2) above – ensuring Research is done effectively – with tools that work in ANY research context. This part provides the complete answer to (1) – ensuring Research connects to product impact:

First, choose research that matters (chapter 6).
Then, work backwards from product value (chapter 7).
Continuously validate with end-to-end iterations (chapter 8).

Chapter 6 - How to Choose Research Initiatives

The very first step in making sure your Research impacts the product is choosing the right thing to research. And, just as important, avoiding Research that won't impact the product.

Research initiatives can start from two different places:

From a problem the product is facing.
From a technological opportunity.

1. Starting From a Concrete Problem

The most promising way to find research initiatives that will have a big impact on the product is to start from an acute problem that the product is facing.

At Swimm, we allowed users to write documents about their code, but inevitably, the code changed, and the documentation became outdated. This made writing documentation not worth the effort in the first place. We needed to find a way to make sure the documentation stayed up to date automatically, with a good user experience. This was a clear problem we faced, and we didn't know if it was even possible to solve.

Consider a different example: a medical company that wants to diagnose a disease based on a few blood samples. Currently, they have an algorithm in place, but it's not very accurate. Specifically, it yields too many false positives. They need to find a way to improve their prediction accuracy. This is a clear problem the product is facing, and it doesn't have a clear technological solution.

In both cases, the problem is clear, and its impact on the product or company is clear. At the same time, the solution is not clear, and it's not certain that a solution will be technologically feasible.

2. Starting From a Technological Opportunity

When generative AI became popular, many companies started exploring how to leverage it to improve their products. This is an example of an emerging technology that can enable new product features.

The same can happen with smaller, more specific technologies, and not necessarily new technologies – sometimes technologies that the relevant teams just familiarized themselves with. For example, if a researcher reads a paper about a new way to parse source code, that researcher might have an idea for a new product feature that can leverage this technology.

While many good ideas come from technological opportunities, it's important to remember that the real impact of Research is determined by the product, not the technology. It's far more risky to pursue a technological opportunity than a concrete problem. If you do start a Research based on a technological opportunity, your responsibility is to make sure that the technological opportunity, if pursued successfully, will indeed have a big impact on the product.

Problem-Driven vs. Opportunity-Driven: A Comparison

Aspect	Problem-Driven Research	Opportunity-Driven Research
Starting Point	Customer pain point or product limitation	New technology or technique becomes available
Product Impact Clarity	High – you know exactly what problem you're solving	Low to Medium – you're searching for problems this technology can solve
Risk Level	Lower – you know there's demand if you succeed	Higher – solution might not match any important problem
Validation	Problem already validated through user feedback	Needs validation that the solution matters to users
Examples	Swimm's auto-updating docs; Reducing false positives in medical diagnosis	"How can we use LLMs in our product?"; "This new parsing technique could enable..."
Success Criteria	Did we solve the problem?	Did we find a valuable use case AND solve the problem?

Problem-driven research starts with validated demand: you know that the goal is worth pursuing. Opportunity-driven research starts with a hammer looking for nails. Sometimes you find valuable nails, but it's riskier.

Should You Pursue a Research Initiative?

Say you have a research initiative – an idea you'd like to research. To know whether you should pursue it, you should be able to answer a simple set of questions:

1. Product Impact – If the Research succeeds, how big will the impact be?

This is the most crucial question.

For problem-driven research, this is usually straightforward: "If we solve automatic doc updates, we'll retain 40% more customers who currently churn due to outdated docs."

For opportunity-driven research, you need to work harder: "If we use LLMs for code analysis, we could enable X feature, which would help Y users save Z hours per week, translating to $W in additional revenue."

If you're not convinced a successful Research result would make a huge impact on the product, it's probably not worth pursuing at the moment (until you are convinced). "Huge impact" means:

Solving a top-3 customer pain point, OR
Enabling a new product capability that significantly expands your market, OR
Reducing a major cost or risk factor

Anything less, and your resources are better spent on Development work with clearer ROI.

2. Time to Impact – How long until we see product value?

Time estimation is always hard in software. This is true for Development tasks, and even more so for Research. You learned ways to manage this uncertainty during Research in chapter 5, but even at this early pre-Research stage, you should consider the timeline.

Ask yourself two related questions:

How long for the Research itself? (days? weeks? months?)
How long from successful Research to product impact? (immediate integration? requires significant Development? needs market validation?)

The total timeline matters because:

Research that takes 6 months but delivers immediate product value might be worthwhile.
Research that takes 2 months but requires 8 more months of Development might not be worth it if your product roadmap can't accommodate that.
Research that takes 1 year with uncertain outcomes probably isn't worth pursuing unless the potential impact is transformational.

Of course, the actual timespans vary greatly by context (and specifically by the company you work for).

Consider also whether you can achieve incremental value. Can you get some product impact in 3 months even if the full solution takes 9 months? This significantly de-risks longer Research initiatives.

3. Resources – Do you have what you need?

Research requires specific resources beyond just "engineering time":

Knowledge: Do you have team members familiar with the relevant:

Technical domain (for example, NLP, compiler design, distributed systems)?
Business domain (for example, medical diagnostics, financial regulations)?
Similar problems solved elsewhere?

If not, can you acquire this knowledge in reasonable time? (Reading papers for a week: reasonable. Earning a PhD: not reasonable.)

Capacity: Can you dedicate someone (or multiple people) for the expected duration? Research requires sustained focus – splitting someone 10% on Research and 90% on urgent product work rarely succeeds.

Dependencies: Do you need:

Access to specific data or systems?
Collaboration from other teams?
External expertise or consulting?
Budget for tools, cloud resources, or datasets?

If critical resources are unavailable or expensive to obtain, the initiative may not be viable.

Pre-Research Checks

You might not have clear answers to all three questions above. In that case, it makes sense to spend time answering them before actually pursuing the Research. This phase should be limited in time – ideally a few days, at most a week or two.

In this stage you might:

Interview customers and business stakeholders to understand the real impact of solving the problem.
Read about the technological aspects and previous research done in this field to assess feasibility and timeline.
Consult with people who have faced similar challenges (internal experts, academic contacts, or practitioners in your network).
Run quick feasibility tests – not full Research, but simple checks like "Can we even access the data we'd need?" or "Does this library exist in our language?"

After pre-research checks, you should have a clear answer: "Yes, we should pursue this because the impact is X, the timeline is roughly Y, and we have (or can get) the resources we need."

If you're not confident about substantial product impact, don't start the Research. This might sound harsh, but it's crucial: unfocused Research that doesn't connect to product needs wastes your most valuable resource – talented engineers' time and attention. One way to enhance your certainty about product impact is described in chapter 7.

The next chapters assume you understand the product impact of successful Research outcomes, and will help you ensure you actually achieve this impact as quickly as possible.

How to Choose Research Initiatives - Summary

Research initiatives start from either:

Problem-driven: A clear product need (strongly preferred)
Opportunity-driven: A new technology (higher risk)

Before pursuing Research, answer three questions:

Product Impact: Will success create huge value?
Time to Impact: How long until we see product results?
Resources: Do we have the knowledge, capacity, and dependencies?

Run pre-research checks (days, not weeks) to answer these questions if unclear.

Only pursue Research when you're confident about substantial product impact.

The next chapter shows how to maintain that product connection throughout the Research process.

Chapter 7 - Drawing Backwards

So you've chosen a research initiative, and done so correctly (following chapter 6). Now, how do you start working on it?

Most teams start by diving into technical challenges: parsing COBOL, building callgraphs, implementing algorithms. But there's a more powerful approach that ensures your Research actually impacts the product: start from the end and work backwards.

This heuristic – working backwards from your goal – is one of the most valuable problem-solving strategies you can use. Let me show you why with a simple game.

The Spiral Game

Consider this game:

The rules are simple:

The pawn starts on spot 41.
On each turn, a player moves the pawn backward between 1 to 6 spots. That is, if the pawn is on spot 41, you can move it to any spot from 35 to 40.
The player who moves the pawn to spot 1 wins.

Take a moment: If you go first, how would you play to guarantee a win?

(I do encourage you to take a moment and try this for yourself first.)

Most people start thinking from the current position (spot 41) and try to calculate forward: "If I move 3 spaces, they can move 2, then I can move 4..." This quickly becomes overwhelming – too many possible moves to track.

But if you work backwards, the solution becomes clear:

Starting from the end (spot 1):

To win, you need the pawn on spot 1 on your turn.
Your opponent just moved, so the pawn could be on spots 2-7 (since they moved 1-6 backward from wherever it was).
For any spot from 2-7, you can move directly to spot 1.
Conclusion: If the pawn is on spots 2-7 at the start of your turn, you win.

Continue working backwards (from spots 2-7):

You want your opponent to land on spots 2-7.
From spot 8, no matter what they do (move 1-6), they land on spots 2-7.
Conclusion: If you get the pawn to spot 8, you guarantee a win.

Continue working backwards from spot 8:

Notice that you've just created a "new" game: you no longer need to land on spot 1 in order to win. It's enough that you land on spot 8 – as from there, you can win no matter what your opponent does.

So, how do you ensure you can land on spot 8?

Spots 9-14 all allow moving to spot 8.
So if your opponent starts their turn with pawn on a spot between 9-14, you can force it to 8.
Which means you do not want to land on spots 9-14, but you do want to land on 15 (which will force your opponent, in turn, to land somewhere between 9-14).

Again, you've just created a "new" game: where your goal is to land on spot 15. From there, you already know how to win.

You can keep going like this, drawing backwards – would you want to land on spot 16 or not? How about 17? Until, at some point...

The pattern emerges:

Safe spots for you: 1, 8, 15, 22, 29, 36
From any of these, your opponent cannot avoid giving you another safe spot
These are all numbers of the form: 1 + 7n

The winning strategy:

From spot 41, move 5 spots backward to reach spot 36 (a "safe spot").
No matter what your opponent does, you can always move to the next "safe spot".
Eventually, you reach spot 1 and win.

Notice what happened: By working backwards from the goal, you discovered the systematic solution. Working forward from the start position would have been much, much harder.

You actually deployed a second, powerful heuristic here: considering specific cases (1, 8, 15, 22, 29, 36) – and generalizing (1+7n). This heuristic is also very common in research, though not specifically when aiming to ensure product impact.

How to Apply Drawing Backwards to Product-led Research

Let's connect this to Product-led Research. When you face a complex Research challenge, the question isn't "What technical problem should I solve first?" but rather:

"If the Research succeeds, what would the result look like?"

This forces you to:

Connect to product impact: You must envision the end state that creates value.
Work systematically: Like the spiral game, you identify the chain of dependencies backward.
Validate assumptions: Before solving sub-problems, ensure they lead to your goal.

Let me show you how this worked in practice at Swimm.

Case Study: Extracting Business Rules from COBOL

At Swimm, we wanted to automatically generate documents from COBOL codebases that included all the extracted business rules.

Quick context on business rules: Business rules are the constraints, conditions, and actions embedded within software that reflect organizational policies. For example, in money transfer logic:

A customer cannot transfer more than their available balance (overdraft limits notwithstanding).
High-value transfers require additional verification.
Cross-currency transfers must apply current exchange rates.

Some sources define business rules with three elements: Event, Condition, Action:

ON 
IF 
THEN 
ELSE

Our goal was to extract all business rules from a COBOL codebase. While challenging in any codebase, it's particularly acute in legacy COBOL code. (If you're interested in the technical details, see this post.) For this book, it's sufficient to know that many research attempts over the last few decades have tried different approaches to face this challenge.

Where Do You Even Start?

Faced with this problem, you might think:

"Should I build a callgraph of all functions? That means I need to parse COBOL code..."
"Should I create a COBOL parser first?"
"Maybe I should read academic papers about program comprehension?" (Good practice during the pre-Research checks phase from chapter 6)
"Perhaps I should track COBOL variables through the codebase?"
"Should I distinguish business conditions ('if requested transfer amount > available balance') from technical conditions ('if variable not initialized, show error')?"

Each of these might require its own research effort. Where do you start?

Drawing Backwards: Start with the End Result

Drawing backwards made us ask:

"If the Research succeeds, what would the result look like?"

When we first asked ourselves this question, we weren't sure. We knew from our clients that they wanted extracted business rules, but we couldn't know what the "ideal" output would look like.

So we decided, before tackling any technical challenges, to manually create documents showing extracted business rules from sample programs. We did this completely manually: no parsing, no algorithms, just understanding COBOL code ourselves and writing documentation.

We did this for various types of applications from different codebases, and learned:

There's no single "right" way to construct such a document.
The output structure differs from one program to another.
Certain patterns appear consistently across business logic.

By creating these documents manually, we formed a hypothesis: "This is what the output should look like, which means this output would make the biggest impact on the product. This is our north star."

But was it actually the north star?

Validating the Hypothesis

Once we manually wrote the documents, it was time to verify our hypothesis. With concrete output in hand, we could:

Discuss internally within the team – get feedback from engineers who understand both COBOL and our product.
Reach out to clients – show them the actual output and ask: "Does this solve your problem?"

We deliberately refrained from solving hard technological challenges before knowing where we were aiming.

Working Backwards Through Sub-Problems

The manual documents also gave us something crucial: a concrete example to analyze backwards.

For instance, we saw that our documents listed many conditions. This made us realize we would (most probably) need to:

Find conditions in the code.
Filter out business-related conditions (vs. technical conditions).
Explain the condition in the document.

Here's where drawing backwards becomes powerful: We tackled (3) before (2), and (2) before (1).

Why? Because we needed to solve (3) to reach our goal – creating impactful documents.

This means we mocked the first two steps – we assumed we already had a way to find conditions and filter business-related ones. So we already had our list of business conditions, and now the challenge was: for each condition, explain it clearly in the output document.

This approach prevented us from spending weeks on tasks (1) and (2), only to discover that our explanation approach (3) didn't actually work. If we can't accomplish (3) assuming (1) and (2) are solved, then perhaps this entire approach isn't viable, and we need to reconsider alternatives.

The logic is clear: While solving (3) ultimately requires solving (1) and (2), if we fail at (3) even with (1) and (2) accomplished, then pursuing (1) and (2) might not be worth the effort at all.

What does mocking a dependency look like in practice? In our case, mocking steps (1) and (2) meant manually identifying a handful of business conditions from the COBOL code ourselves, rather than building automated systems to find them. We created a small, hand-crafted list of conditions that we knew were business-relevant and existed in our sample programs, and used that as input for testing our explanation approach in step (3).

Why Drawing Backwards Is So Powerful

The advantages of drawing backwards become clear from both the game examples and the COBOL case study:

1. Forces Connection to Product Impact

Like working backwards from spot 1 in the spiral game, starting with "what does successful output look like?" forces you to think about end-user value. You can't start drawing backwards without a clear goal. This prevents the common failure mode of technically interesting Research that doesn't impact the product.

2. Provides a System for Progressing

When Research seems like a huge, daunting task with endless options, working backwards gives you a systematic approach. Just as the spiral game became trivial once you worked backwards to identify the safe spots (1, 8, 15, 22...), Research becomes more manageable when you work backwards from the desired output to identify the dependencies.

3. Validates Each Step Before Investment

Working backwards lets you validate that each sub-problem actually contributes to your goal before you invest significant effort. In the COBOL example, we could verify that our explanation approach (step 3) worked before spending weeks on finding and filtering conditions (steps 1 and 2).

Practical Application: Your Research Tree

Drawing backwards integrates naturally with the Research Tree from chapter 4. When you create your tree:

Start with the end:

Root of tree: "Generate impactful business rule documentation".
First question: "What should successful output look like?"
Approach: Create manual examples.

Then work backwards:

Once you have output, ask: "What do we need to produce this?".
This reveals the actual sub-questions and their dependencies.
Each branch represents a prerequisite you need to solve.

Validate before going deeper:

Before pursuing any branch deeply, ask: "If I solve this, does it actually get me closer to the goal?"
Mock out dependencies to test approaches cheaply.
Use time-boxing (from chapter 5) to limit exploration of any branch.

Summary: Drawing Backwards

Drawing backwards is the heuristic of starting from your desired end state and working systematically toward your current position.

In Product-led Research, drawing backwards means:

Start by defining what successful output looks like (often manually or semi-manually).
Validate the output with stakeholders before technical work.
Work backwards through dependencies, solving them in reverse order.
Validate that each step contributes to the goal before major investment.

This heuristic ensures that Research connects to product impact, since you start with the product goal. It provides systematic progress even when problems seem overwhelming, and makes you validate each step before you invest heavily.

Integration with other tools:

Use with Research Tree (chapter 4) to map backwards dependencies.
Apply time-boxing (chapter 5) to limit exploration of each branch.
Combine with pre-Research checks (chapter 6) to validate product impact.

In the next chapter, you’ll learn about two limitations of drawing backwards and how to address them with continuous end-to-end iterations.

Chapter 8 - End-to-End Iterations

Your most important role is ensuring that Research impacts the product. One major risk: spending weeks on research questions that seem vital, only to discover they don't actually impact the product.

In chapter 7, I advocated for drawing backwards – starting from product impact and working your way back to research questions. This is indeed powerful, especially because it forces you to focus on the end result and validate it with users.

But drawing backwards alone has limitations. Two risks emerge:

Risk 1: Infeasibility in practice Your manually-created "ideal output" might be technically infeasible or impossibly expensive to generate. You won't discover this until you try to build it.

Risk 2: Lack of real-world validation Since you haven't completed an end-to-end process, you probably haven't run your solution on clients' actual data (assuming you can't access it during the manual phase). Continuing our COBOL example from the previous chapter, what works on carefully-chosen examples might fail on real codebases.

These two risks are why I advocate for continuous end-to-end iterations.

Drawing Backwards + End-to-End: A Combined Approach

These aren't competing approaches – they're complementary:

Heuristic	Purpose	What It Gives You	Strength	Limitation
Drawing Backwards (chapter 7)	Define target and path	1. Target output\ 2. Hypothesized chain of steps to get there\ 3. Order of dependencies	Ensures product focus; reveals what you need to build	Hypotheses may be wrong; doesn't validate feasibility on real data
End-to-End Iterations (this chapter)	Validate and build incrementally	1. Proof the chain works\ 2. Learning from real data\ 3. Prioritized improvements	Validates feasibility; discovers what actually works	Can lose direction without clear target and chain

The recommended flow:

Use drawing backwards to:
- Define your target output (manually create examples, validate with users).
- Identify the chain of intermediary steps needed to produce that output.
- Understand the order of dependencies.
Switch to end-to-end iterations to:
- Test whether your hypothesized chain actually works.
- Validate on real data (not just your manual examples).
- Incrementally build toward the target.
Throughout iterations, keep both the target AND the chain from step 1 as your guide.

Drawing backwards already outlines your end-to-end process (Principle 1 below). End-to-end iterations validate and build that process incrementally, learning what works and what doesn't.

The Five Principles of End-to-End Iterations

The end-to-end approach relies on five principles:

Outline the end-to-end process.
Get to end-to-end by simplifying.
Ship it as fast as you can.
Gradually replace steps.
Get frequent feedback on results.

Let me explain each principle using the COBOL business rules example from chapter 7. Since this book isn't about COBOL, I'll keep explanations short while focusing on the methodology.

Principle 1: Outline the End-to-End Process

Start by outlining the entire process from input to output. More accurately, outline the hypothesized end-to-end process – you can't know for certain until you test with users.

If you followed drawing backwards from chapter 7, you already have this. Drawing backwards naturally produces this chain: you started with the target output, asked "what do I need to produce this?", then "what do I need for that?", working your way back to the starting input. That chain IS your end-to-end process outline.

For the COBOL business rules example: We used drawing backwards to identify that our document needed business rule sections, which meant we needed to explain conditions, which meant we needed to filter business conditions, which meant we needed to find conditions, which meant we needed to parse COBOL. Working backwards gave us this chain:

Start with a COBOL program.
Parse the COBOL program into an Abstract Syntax Tree (AST).
Traverse the AST to find conditions.
Filter out business-related conditions (vs. technical conditions).
For every business rule, create a document section explaining the condition.
Sort document sections according to business rule dependencies.

This is your first draft: the hypothesized process that drawing backwards revealed.

How to outline:

Draw boxes on a whiteboard.
Use a flowchart if the process isn't linear.
Keep it visible throughout the research.
Update it as you learn.

The outline serves as your map – it shows you where you are and what you're building toward.

Principle 2: Get to End-to-End by Simplifying

Your goal: make the process work end-to-end. Start with input (a COBOL program), reach the output (a document with business rules), while passing through the intermediate stages.

This sounds like too much. Don't you need to complete the whole research to achieve this?

Definitely not. The trick is to find the easiest way to get end-to-end by taking shortcuts and making assumptions that are definitely too generous for production. You should fight your inner engineer who wants to "do it right" from the start. Your goal is to get an end-to-end process working, as this heuristic is far more valuable than perfecting one step.

This means that some of the steps can be completed manually, or with very simple implementations that you know won't work in production. Remember it is an intermediate milestone, not the final product.

For our COBOL example:

Start with a single, known COBOL program.
Skip parsing – just manually write a list of conditions for the next stage.
The filtering function returns true if business-related, false otherwise.
- First implementation: a simple mapping between input conditions (that you know of) and whether to return true or false.
- Alternative: always return true – yes, you'll get non-relevant rules, but that's a problem for later.
The document generation might also be manual for the first pass.

End result: A rough-looking document, generated by a combination of manual work and code that works solely on a single program. This is far from shippable, but it's an extremely important milestone for ensuring research impacts the product.

You might argue that it's overkill, and why waste this time on manual steps when you'll need to automate them eventually? Even if you don't, I promise from experience that many researchers and engineers feel that way.

From my own experience, I learned (the hard way) that this pays off. Getting to a working end-to-end makes sure you:

Validate that the entire flow works.
Identify bottlenecks and blockers early.
Have something concrete to show and get feedback on.
Prevent spending months on one component that turns out to be unnecessary.

Principle 3: Ship It as Fast as You Can

By the end of Principle 2, you have a working end-to-end process for a single case. You definitely can't ship it yet – it only works on one specific program, certainly not the client's program.

Next milestone: make it shippable.

Does "shippable" mean it works on any program? That's too hard and takes too long. You need shortcuts.

This is where creativity matters. For our COBOL example:

Information gathering:

If running on a client's program, get as much information as possible.
Perhaps assume it's less than 1,000 lines of code.
Perhaps you know which COBOL dialect the client uses, so you don't need to support others (fun fact: COBOL has more than 300 dialects. Well, it's fun for you, not for those who need to support them).

Algorithmic shortcuts:

Use regular expressions to find conditions instead of a full parser.
Yes, you'll miss some conditions – that's a problem for later.

UX shortcuts:

Skip the document generation. Just print rules to the console.
Run from the command line without a GUI.
Manual configuration file instead of user interface.

Your goal is clear: ship it.

It doesn't need to be perfect. It needs to be enough to learn from this iteration. If you don't ship, you can only learn from your intuition – a very bad idea.

Unlike in Principle 2, note that here you don't generate an output (partially) manually - you need a working software that can run on real data.

What "shippable" means:

Works on the client's actual data (even if imperfectly).
Produces output you can get real feedback on.
Doesn't require your manual intervention for each run.

What "shippable" doesn't mean:

Perfect accuracy.
Handles all edge cases.
Production-quality code.
Beautiful user interface.

You can't ship just anything, though. In our example, if you only have a solution that works on one specific test program you created, generating an irrelevant document for the client wastes time and teaches you nothing.

You can't learn from iteration without shipping. And you can't ship without a working end-to-end process. I know it sounds obvious, but many teams miss this in practice as they get into the rabbit hole of solving one step "the right way" before validating the entire flow.

Principle 4: Gradually Replace Steps, While Carefully Prioritizing

Now you have a working end-to-end process. You can start replacing various steps' implementations with better ones:

Replace manual steps with automated ones.
Remove shortcuts and add more robust implementations.
Improve accuracy and coverage

After shipping, you'll have many things you think you must replace immediately. But remember: the goal is making research impact the product. To do that, you need careful prioritization.

The Prioritization Framework

Prioritize changes based on three criteria:

1. Learned Necessity Did you learn that something doesn't work in the current implementation and must be fixed for the product to be viable?

Example: "Regular expressions miss nested conditions, and 60% of the client's business rules are in nested conditions. We must use a real parser."

2. Learning Potential Will changing this implementation help you learn more about product impact in the next iteration?

Example: "If we improve the filtering accuracy from 40% to 80%, we'll learn whether the document format is actually useful when it contains mostly-correct content."

3. Effort Estimation How much time will this change take?

Example: "Building a full parser: 3 weeks. Improving regex to handle nested conditions: 2 days. The latter gives us 80% of the value for 10% of the effort."

Continuing with our COBOL example, you may consider and prioritize these changes following the first iteration:

Changes after first iteration:

- Fix parser to handle nested conditions [Learned: Critical, 2 days] → DO FIRST
- Add GUI for document generation [Nice-to-have, 1 week] → DEFER
- Improve filtering accuracy [Learning: Critical, 3 days] → DO SECOND  
- Support additional COBOL dialects [No evidence needed, very long time...] → DEFER
- Better document formatting [Client mentioned it, 1 week] → DEFER (validate content first)

Iterate fast: change something, ship again, get feedback. Don't solve every issue you find, even issues clients mention. Ask: "What's the most important thing to change to learn something in the very next iteration?"

Principle 5: Get Frequent Feedback on Results

The trick is to be obsessed with getting as much feedback as you can, on each and every iteration.

On each iteration:

Get feedback on the end result
- Show the actual output to users.
- When applicable, don't just ask "does this work?" – also watch them try to use it.
- Identify what works, what doesn't, what's missing.
Understand the next questions to answer
- What did you learn about product impact?
- What assumptions were validated or invalidated?
- What new questions emerged?
Plan the next iteration accordingly
- Use the prioritization framework above.
- Focus on learning, not building.
Implement minimal changes to answer questions
- Don't fix everything.
- Make the smallest changes that will answer your next most important question.
- Keep the cycle fast (days to weeks, not months).

Example iteration cycle:

Iteration 1:
- Shipped: Regex-based condition finder, always-true filter, console output.
- Learned: Document structure works, but too many false positives (noise).
- Question: Is filtering accuracy the blocker to usefulness?
- Next: Improve filtering to 80% accuracy, ship again.

Iteration 2:
- Shipped: Same regex finder, smarter filtering (80% accurate), console output.
- Learned: With better filtering, users found the output useful!
- Question: Do we need a parser, or is regex sufficient?
- Next: Test on larger programs to see where regex breaks down.

Iteration 3:
- Shipped: Same pipeline, tested on 10 real programs.
- Learned: Regex fails on 4/10 programs (nested conditions).
- Question: Parser worth the investment now?
- Next: Build parser for nested conditions, ship again.

Notice: Each cycle is fast, focused on one question, and based on real learning.

In real life, you may want to tackle a few questions per iteration. If two things are clear, fix them before shipping again so you can actually gain valuable feedback rather than hearing the same complaints.

Also, when working with real clients, they might not be as receptive to trying things so many times – so you’ll need to consider that aspect as well. Regardless, the key remains the same: keep cycles short and focused on learning.

Integration with Other Tools

End-to-end iterations work best when combined with other research management tools:

Research Tree (chapter 4):

The outlined process becomes a branch in your tree.
Each iteration tests different approaches on branches.
Failed iterations mark branches red, successful ones mark green.

Time-boxing (chapter 5):

Time-box each iteration.
If you can't ship in the time box, you're building too much.

Drawing Backwards (chapter 7):

Drawing backwards (chapter 7) defines both:
- The target output.
- The hypothesized chain of intermediary steps to reach it.
End-to-end iterations test whether that chain actually works on real data.
The target from drawing backwards acts as your north star throughout iterations.
Each iteration validates or refines the steps that drawing backwards identified.
Use both: drawing backwards reveals what to build, while end-to-end iterations prove it works and let you test it incrementally.

Summary: End-to-End Iterations

End-to-end iterations ensure Research impacts the product by continuously validating feasibility and learning from real data.

The five principles:

Outline the process: Draw backwards already gives you this: the chain from input to output.
Simplify to get end-to-end: Use shortcuts and manual steps to make the whole chain work.
Ship fast: Real data teaches what theory can't.
Prioritize carefully: Use the three-criteria framework:
- Learned necessity (is it broken?)
- Learning potential (what will you learn?)
- Effort estimation (how long will it take?)
Get frequent feedback: Fast cycles (days to weeks) focused on learning.

Part 3 Summary

Back in chapter 2, you learned that your role as a Research leader is to:

Ensure Research connects to product impact.
Ensure Research is done effectively.

Part 2 handled ensuring Research is done effectively.

Part 3 handled ensuring Research connects to product impact – your most important responsibility. This part provided a complete methodology for ensuring product impact through three complementary stages:

The Three Stages of Product-Led Research

Stage 1: Choose research that matters (chapter 6)

Before starting any Research, answer three critical questions:

Product impact: Will success create huge value?
Time to impact: How long until we see product results?
Resources: Do we have the knowledge, capacity, and dependencies?

You learned to distinguish between problem-driven research (starting from validated customer pain points, which is strongly preferred) and opportunity-driven research (starting from new technologies).

Run focused pre-research checks to answer these questions. Only pursue Research when confident about substantial product impact.

Stage 2: Start from product value and work backwards (chapter 7)

Once you've chosen what to research, the drawing backwards heuristic ensures that you start right. Instead of diving into technical challenges, start from the end:

Manually create the desired output: What should successful Research produce? Create it by hand before solving any technical problems.
Validate with stakeholders or customers: Show them the output and confirm it solves their problem.
Work backwards through dependencies: From that validated output, identify what you need to produce it, then what you need for that, working your way back to your starting point.
Solve in reverse order: Tackle the final step first (with earlier steps mocked), validating that each step contributes to the goal before investing heavily in earlier dependencies.

The spiral game example showed why this works: working backwards from the goal reveals systematic solutions that working forward obscures.

Drawing backwards forces product connection because you literally start with product output. It integrates with the Research Tree (chapter 4): While drawing backwards identifies what questions matter, the Tree helps you explore approaches for answering them.

Stage 3: Validate and build iteratively (chapter 8)

Drawing backwards alone has two limitations:

Your manually-created "ideal output" might be technically infeasible to generate.
You haven't validated on real user data.

End-to-end Iterations address both limitations. These two tools aren't competing approaches – rather, they complement one another:

How the Three Stages Work Together

These chapters form a complete methodology for product-led Research:

Choose → Start from the end → Validate iteratively

Chapter 6 ensures you choose research that COULD have huge impact (strategic decision).
Chapter 7 ensures you start from product value (planning backward from validated output).
Chapter 8 ensures you continuously validate with real users.

Each stage prevents different undesired, but painfully common outcomes:

Chapter 6 prevents pursuing research that won't matter (even if successful).
Chapter 7 prevents building technically correct solutions that don't create product value.
Chapter 8 prevents building infeasible solutions or solutions that fail on real data.

You now have the complete answer to "How do I ensure Research impacts the product?":

Choose wisely: Only pursue research with clear, huge product impact.
Start from product value: Manually create and validate desired output before technical work.
Work backwards: Identify dependencies from output back to starting point.
Build end-to-end fast: Get entire chain working with shortcuts and manual steps.
Ship to real users: Validate on actual data, not just examples.
Iterate based on learning: Improve the chain incrementally, prioritizing what teaches you most.

This methodology keeps product impact central at every stage: choosing, planning, and executing. It prevents the most expensive failure: "successful" Research that doesn't affect the product.

Book Summary

You picked up this book because managing Research is different from managing Development – and you needed concrete tools to handle that difference.

What You Learned About Research

In chapter 1, you learned that Research isn't about difficulty or technical sophistication. It's about uncertainty of approach – that is, confronting problems where you don't know if a solution exists, where multiple approaches might work but you're not sure which, and where the path to success isn't immediately clear.

You saw how Alan Schoenfeld's problem-solving framework breaks down the Research process into four components:

Knowledge base – what you know.
Heuristics – strategies for approaching problems.
Control – monitoring and adjusting your approach.
Beliefs – your mindset toward the problem.

The good news: all four can be improved with the right management and methods.

In chapter 2, you explored the distinction between Research and Development more deeply. You learned that your role as a Research leader has two parts:

Ensure Research connects to product impact: "Successful" Research that doesn't affect the product is a failed project. This is the most important part in Product-led companies.
Ensure Research is done effectively: Even brilliant researchers benefit from structured approaches.

This two-part framework organized everything that followed.

How to Do Research Effectively

Part 2 gave you concrete methods for effective Research execution – tools that work in any research context (whether it's Product-led or not).

In chapter 3, I shared with you a personal reverse engineering classroom story. Students with sophisticated technical skills missed the obvious solution (checking the Help menu) because they lacked structured methodology. The story clearly showed that the problem isn't capability, it's often approach. This illustrated why you need the methods that follow.

Chapter 4 introduced the Research Tree method – a living visual framework for systematically exploring solution paths. You learned:

How to map questions you need to answer and approaches for answering them.
A decision framework for choosing which approach to try first: fastest feedback, lowest cost, best coverage.
How using the tree helps avoid common failure modes: jumping on the first idea, tunnel vision, inefficient learning, answering questions you don't need to, and lost context.

The Research Tree helps you implement Schoenfeld's "control" component – helping you monitor and adjust your approach systematically rather than randomly trying things. It is helpful for a researcher, but as a Research leader, it allows you to guide your team effectively.

In chapter 5, you learned how to manage exploration without killing creativity. Since Research estimation is inherently difficult, time-boxing provides structure by setting time limits for specific research directions. After the allocated time, you stop to reconsider: What did you learn? Is this still the most promising path? This tool acknowledges uncertainty while preventing endless exploration, or diving too deep into rabbit holes that don't necessarily help your Product goals.

How to Ensure Product Impact

Part 3 focused on your most important responsibility: ensuring Research creates product value.

Chapter 6 showed you how to choose what directions to research – and more importantly, what not to pursue. You learned the distinction between:

Problem-driven research – starting from customer pain points (strongly preferred, lower risk).
Opportunity-driven research – starting from new technologies (higher risk, needs validation).

Before pursuing any Research initiative, you need to answer three questions:

Product impact: Will success create huge value?
Time to impact: How long until we see product results?
Resources: Do you and your team have the knowledge, capacity, and dependencies needed?

You saw how to run focused pre-research checks to answer these questions.

Chapter 7 introduced a powerful heuristic for ensuring product connection: start from the end and work backwards. Through the spiral game example, you saw how working backwards reveals systematic solutions that working forward obscures.

In the COBOL business rules case study, you saw a practical application:

Start by manually creating the desired output (before solving any technical challenges).
Validate that output with stakeholders.
Work backwards through dependencies, solving them in reverse order.
Validate that each step contributes to the goal before major investment.

Drawing backwards forces connection to product impact because you must start with the product goal.

Chapter 8 showed you how to address two limitations of the drawing backwards heuristic through continuous end-to-end iterations. Your manually-created "ideal output" might be infeasible to generate, and you haven't validated it on real data. End-to-end iterations solve both problems.

You learned five principles to run effective end-to-end iterations:

Outline the end-to-end process: Drawing backwards already gives you this chain.
Get to end-to-end by simplifying: Use shortcuts and manual steps to make the whole chain work.
Ship it as fast as you can: Real data teaches what theory can't.
Gradually replace steps: Prioritize based on learned necessity, learning potential, and effort.
Get frequent feedback: Fast cycles focused on learning.

Drawing backwards reveals what to build and in what order, while end-to-end iterations prove it works and build it incrementally.

Your Toolkit for Research Management

You now have a complete toolkit:

From Part 2 (works for any research):

Research Tree – maps solution space, chooses approaches systematically.
Time-boxing – manages exploration with structure.

From Part 3 (specifically for product impact):

Choosing frameworks – decides what deserves Research effort.
Drawing backwards – forces product connection from the start.
End-to-end iterations – validates feasibility and learns from real users.

These methods work together. Drawing backwards identifies your goal and the chain of steps. The Research Tree maps approaches for each step. Time-boxing prevents getting deep into a rabbit hole when you should reconsider based on what you've learned, while acknowledging the inability to provide exact time estimates on Research tasks. End-to-end iterations validate and build incrementally.

My Message To You

Research is fundamentally uncertain work. Applying traditional Development management practices fails because it assumes known solution paths, predictable timelines, and steady progress.

But Research doesn't have to be mystical or random. With the right frameworks, you can manage it systematically while maintaining focus on what matters: creating product value. You learned to ensure that Research is done effectively (part 2) and that it connects to product impact (part 3). You have concrete tools, real examples, and a clear framework for both responsibilities.

Now go turn your team's uncertain Research into systematic progress toward measurable product impact. I am confident that you can lead Research teams to success, and I would be happy to hear about your experiences applying these methods.

If you liked this book, please share it with more people.

Acknowledgements

I am extremely lucky to have such wonderful people supporting me in this journey.

Abbey Rennemeyer has been a wonderful editor. Abbey had edited my posts for freeCodeCamp over the past few years, as well as my previous book Gitting Things Done, so I knew she was the perfect fit for this book as well. Her insights both on the content and the writing style have greatly improved this book.

Quincy Larson founded the amazing community at freeCodeCamp. I thank him for starting this incredible community, his ongoing support, and for his friendship.

Estefania Cassingena Navone designed the cover of this book. I am grateful for her professional work and her patience with my perfectionism and requests.

Beta readers who contributed their time and mind to read through an unfinished version of this book to improve it for all of you have helped me get it to its current shape. Specifically, I would like to thank Jason S. Shapiro and Omer Gull for their insights.

Dr. David Ginat introduced me first to Alan Schoenfeld's problem-solving research during my time at Tel Aviv University. His teaching inspired me to apply these ideas in practical contexts, including Research management.

I was privileged to work with many brilliant researchers and engineering leaders over the years, too many to name here.

To readers of my previous book, Gitting Things Done, who were kind enough to provide feedback and support – you are awesome. Receiving your emails and comments made me feel like there is a real reason to keep writing.

If You Wish to Support This Book

If you would like to support this book, you are welcome to buy E-Book version, Paperback, Hardc over , or buy me a coffee. Thank you!

Contact Me

If you liked something about this book, or felt that something was missing or needed improvement – I would love to hear from you. Please reach out at: gitting.things@gmail.com.

Thank you for learning and allowing me to be a part of your journey.

- Omer Rosenbaum

About the Author

Omer Rosenbaum is an established technologist and writer. He's the author of the Brief YouTube Channel, and the books Gitting Things Done and Computer Networks (in Hebrew). He's also a cyber training expert and founder of Checkpoint Security Academy.

The Case for End-to-End Engineering Education: Preparing Institutions for a Dynamic Future

Vahe Aslanyan — Fri, 01 Aug 2025 22:30:02 +0000

The pace of innovation in artificial intelligence, automation, and hyper-connected systems is accelerating, placing software engineers at the very center of a global transformation. They are the architects of our digital future, wielding the code that powers everything from global logistics to personal devices.

Yet, a critical paradox lies at the heart of their software engineer training: most university programs still prepare them for “middle-layer” duties – wiring together pre-built libraries, cloud services, and hardware they rarely see or touch, treating the physical world as a distant abstraction.

This narrow educational focus has tangible consequences. It can blunt creativity and problem-solving skills, leaving graduates ill-prepared to design the complete, resilient solutions that society urgently needs.

This disconnect is reflected in surprising employment statistics, where computer science graduates can face higher unemployment rates than those in some non-technical fields. More importantly, it creates a generation of specialists who understand software in isolation but may lack the holistic perspective to build systems that are secure, robust, and seamlessly integrated with the physical world.

This handbook argues for a necessary evolution: a new, end-to-end engineering education that fuses software, hardware, robotics, mechanics, and cybersecurity into a single, coherent toolkit. It provides a blueprint for educators, industry leaders, and aspiring engineers to build a new generation of creators who can think across disciplines, solve complex problems from concept to deployment, and drive meaningful, sustainable progress. The moment demands not just programmers, but true system architects.

By the end of this handbook, you’ll be able to:

Articulate why traditional "middle integration" software education is no longer sufficient for today's technological challenges.
Define the core principles of End-to-End Engineering and how it integrates software with hardware, robotics, and mechanics.
Analyze the economic, societal, and demographic forces that demand a new, more versatile type of engineer.
Incorporate cybersecurity and ethical design as foundational pillars of system development, not as afterthoughts.
Develop a framework for overseeing and validating AI-driven systems to ensure they are reliable and secure.
Outline a practical, year-by-year curriculum for implementing an end-to-end engineering program.
Identify the benefits of this holistic approach for graduates, industry, and society as a whole.
Formulate strategies for overcoming common challenges in implementation, from faculty training to infrastructure investment.

Inspiration for this Handbook
Why End‑to‑End Engineering Matters
Understanding End-to-End vs. Middle Integration in Engineering
Economic Challenges and Opportunities for Software Engineers
The Role of Institutions in Cultivating End‑to‑End Engineers
Proposed Reforms: Designing End-to-End Programs
Benefits for Graduates and Society
Overcoming Challenges in Implementation
Conclusion: A Path Forward for Engineering Education
Further Resources

Inspiration for this Handbook

The current educational landscape

In our complex and rapidly evolving digital world, the role of higher education as a foundation for innovation and societal progress is more crucial than ever. The rigorous systems established by universities are essential for cultivating the expertise that drives our economies forward.

At the same time, the current educational landscape presents significant opportunities for growth and adaptation. The financial model for higher education is a subject of ongoing discussion, as substantial investments from grants and endowments exist alongside rising levels of student debt. This is causing many to wonder how to best align resources with student outcomes and evolving industry needs.

This dynamic is contributing to a noticeable shift in how learners approach higher education. University degrees are no longer always seen as the exclusive pathway to a skilled career – and this trend is reflected in enrollment data across the globe.

In regions from the United States to Canada and Armenia and beyond, a significant number of university positions that were once highly competitive now remain unfilled. In response, many prospective students are diversifying their educational portfolios, pursuing industry-recognized certifications from technology leaders like Google, AWS, and Microsoft, or engaging in self-directed learning.

This suggests a broader re-evaluation of educational return on investment, as the traditional assumption of a guaranteed path from a degree to employment comes under greater scrutiny.

Evolving educational systems

Established institutions, by their nature, often take a measured approach to curricular change. This can sometimes create a gap between traditional programs and the fast-paced innovation occurring in the technology sector, where open-source knowledge and new learning platforms are becoming increasingly prevalent.

We should consider diverse global strategies in this conversation. For example, China’s model of offering extensive scholarships to international students highlights an approach focused on attracting global talent. Likewise, its emergence as a leading contributor to open-source projects and academic research demonstrates a powerful commitment to widespread knowledge sharing.

The ultimate goal of any educational system is to equip graduates with durable and relevant skills. A student’s education can be viewed as their professional operating system. A strong foundation provides the essential hardware, while a modern, integrated curriculum installs the powerful, adaptable software needed to solve complex problems and create value.

This presents a compelling opportunity for a strategic evolution in higher education. By fostering greater collaboration between academia and industry and thoughtfully integrating new hands-on learning models, we can enhance the impact and accessibility of our educational systems. The path forward lies in building a more responsive, inclusive, and sustainable framework that empowers the next generation of innovators to meet the challenges of the future.

You can download a free copy of the ebook version of this handbook here.

And you can listen to it as a podcast here:

Why End‑to‑End Engineering Matters

Employment data tell a cautionary tale. Computer science graduates currently face about 6.1% unemployment, while computer engineering majors experience a 7.5% rate – higher than fields like art history (3%) or journalism (4.4%). This mismatch stems from curricula that prize isolated coding skills over the interdisciplinary fluency modern industry expects.

Big-tech titans such as Apple, Amazon, Alphabet, Meta, Microsoft, Nvidia, and Tesla push the frontier of AI and automation, but they also expose society to new vulnerabilities – from misinformation cascades to brittle supply-chain software. And there are valid criticisms of universities – such as outdated approaches that reinforce these vulnerabilities in various ways. For example, many university programs focus courses on stitching together third-party APIs or cloud SDKs, leading students to depend on vendor ecosystems rather than building foundational technologies themselves. But, these institutions remain invaluable assets for any country.

MIT is still MIT, and Stanford continues to produce some of the world's best engineers, driving innovation through cutting-edge programs. Universities overall generate a massive workforce that transforms fields, along with groundbreaking research papers that advance global knowledge.

But many universities are being left behind due to insufficient investment in the education system and systemic inefficiencies, which are causing huge troubles for the entire world. For instance, nations need to keep pace with aging populations, where rising old-age dependency ratios – projected to increase significantly by 2055 – mean fewer workers supporting more retirees. This will potentially requiring two people to effectively pay for one non-worker through higher taxes and social security burdens.

This is evident in aging societies like Japan, Denmark, and Finland, where top personal income tax rates exceed 55%, and citizens face mounting fiscal pressures to fund pensions and healthcare.

Security is another critical concern: even nuclear agencies are being hacked, as seen in the July 2025 breach of the U.S. National Nuclear Security Administration (NNSA) by Chinese state-sponsored hackers exploiting Microsoft SharePoint vulnerabilities.

These issues highlight the urgent need for universities to foster resilient, skilled talent that can safeguard economies and societies. What this likely means is a shift away from traditional models – like over-relying on international student tuition and exorbitant fees – toward hands-on, open-source styles that democratize learning.

For example, organizations like freeCodeCamp, alongside tech giants such as Google, Microsoft, and Amazon, are open-sourcing vast engineering content that rivals entire university curricula, all without massive endowments or campus infrastructures.

Google's AI tools, like NotebookLM for generating educational content, OpenAI's agents for interactive learning, and productivity boosters such as Cursor (despite its limitations in studies showing 19% slower task completion due to bugs) are unlocking doors previously locked by institutional barriers.

These innovations allow single engineers to achieve more, as industry can no longer afford inefficiencies. This has been made clear by companies rapidly adopting alternatives to traditional systems, swapping locked gates for open pathways to boost output and adaptability.

In the context of educational institutions, end-to-end curricula offer a different path. By combining rigorous software foundations with hardware prototyping, robotics labs, mechanical design, and embedded security, universities can graduate engineers who understand an entire system’s life cycle – from concept sketches and circuit diagrams to secure deployment in the field.

Such breadth does more than widen a résumé. It also empowers graduates to spot hidden failure points, slash integration overhead, and create novel products that are both robust and ethically sound. The payoff is twofold. First, students gain adaptability: a graduate who can write control firmware, machine-learning inference code, and penetration tests is far harder to automate or outsource.

Second, industry gains innovators who can push technology forward without leaning exclusively on closed-source toolchains. This reduces systemic risk and diversifies the ecosystem.

This handbook sets out the full case for such a transformation. We will examine the economic and societal forces demanding new skills, survey pioneering institutions already leading the charge, and map a practical blueprint for universities ready to pivot.

The goal is simple: equip tomorrow’s engineers to build end-to-end solutions that drive progress responsibly – and ensure they share equitably in the value they create.

Understanding End-to-End vs. Middle Integration in Engineering

The Scope of Traditional Software Engineering

Traditional software engineering education focuses on intermediary roles, where engineers develop software to bridge users and systems – such as connecting databases to applications, devices to networks, or algorithms to outputs.

This "middle" integration approach often involves working with pre-existing hardware, such as laptops from manufacturers like Dell or Apple, and leveraging APIs or cloud services provided by leading tech companies.

While it’s effective in specific contexts, this focus can lead to inefficiencies, as engineers dedicate significant time to managing integrations rather than creating innovative solutions. Also, reliance on third-party tools can introduce complexities, including compatibility issues or security vulnerabilities, which require ongoing maintenance and can limit creative problem-solving.

For example, engineers working with cloud platforms may spend considerable effort resolving version conflicts or debugging third-party APIs, diverting resources from developing new features. This dynamic can also expose systems to risks, as external tools may contain outdated libraries or vulnerabilities that require constant updates.

The 2020 SolarWinds hack, which compromised organizations through a supply chain attack, illustrates the challenges of fragmented development, where reliance on external components can introduce unforeseen risks.

The Vision of End-to-End Engineering

End-to-end engineering education adopts a holistic approach, training students to oversee every stage of system development, from ideation to deployment. This encompasses software development, hardware prototyping, mechanical engineering for physical systems like robotics, and cybersecurity to ensure system integrity.

For instance, an end-to-end engineer might design a robotic arm’s software, optimize its mechanical components for precision and durability, and embed security protocols to protect against cyber threats. This comprehensive skill set helps engineers create integrated, resilient systems that minimize reliance on external tools and enhance system reliability.

The benefits of this approach are multifaceted. Robotics training equips engineers to address physical constraints, such as sensor accuracy, motor efficiency, or material strength, fostering innovation in fields like autonomous vehicles, industrial automation, and medical robotics.

Mechanical engineering bridges the digital and physical realms, enabling engineers to design systems that interact seamlessly with the real world.

Cybersecurity integration is critical in an era of increasing connectivity, as devices like robots and IoT systems face growing risks of cyber threats. For example, industrial robots designed with embedded security can prevent disruptions like the Stuxnet attack, which targeted control systems, ensuring operational continuity and safety.

Addressing Curriculum Gaps

Current software engineering curricula, typically spanning 120-130 credits over four years, cover foundational topics such as mathematics (calculus, linear algebra), programming languages (Python, Java, C++), data structures, and software design principles. While these are essential, programs often include courses like introductory chemistry or unrelated electives that may not align with modern industry needs, consuming valuable time and resources.

Meanwhile, key interdisciplinary skills – robotics, mechanical engineering, and cybersecurity – are often underrepresented, leaving graduates less prepared for real-world challenges where software must integrate with hardware under security constraints.

This curriculum gap can impact graduates’ economic outcomes. At companies like Meta, engineers earn competitive salaries ($210,000 to $3.67 million annually, including bonuses and stock), yet the broader distribution of corporate profits, such as Meta’s $39 billion in 2023, tends to favor executives and shareholders.

Similarly, Vivaro, an online casino platform based in Armenia, has leveraged the country’s relatively low labor costs and favorable government relations to achieve rapid growth with minimal regulatory oversight, highlighting how companies can benefit from localized economic advantages.

This dynamic underscores how reliance on integration-focused roles can limit engineers’ ability to capture the full value of their work, as companies maximize profits through strategic labor and regulatory practices.

End-to-end education addresses this by equipping engineers with versatile skills to innovate independently, pursue entrepreneurial ventures, or lead multidisciplinary projects, enabling them to contribute meaningfully and share more equitably in the value they create.

Pioneering Models for the Future

Institutions like MIT are leading the way with programs that integrate computer science, electrical engineering, robotics, and cybersecurity.

MIT’s Department of Electrical Engineering and Computer Science (EECS) offers courses like "Robotics: Science and Systems," where students design complete robotic solutions, blending software, hardware, and security. These programs produce graduates who excel in diverse roles, from developing secure autonomous systems to founding innovative startups.

Similarly, Stanford’s AI and Robotics track combines software development with mechanical engineering and cybersecurity, preparing students for complex challenges like secure drone navigation.

By adopting such models, educational institutions can better prepare students for a rapidly evolving industry, ensuring they are equipped to navigate and contribute to a technology-driven world.

Economic Challenges and Opportunities for Software Engineers

Today’s software engineers face a complex landscape of economic and societal pressures that are fundamentally reshaping their roles. Much of the work has shifted from pure invention to integration, often centering on stitching together proprietary clouds and third-party APIs.

This moves engineering effort toward upkeep – resolving version conflicts, debugging vendor libraries, and managing deployment pipelines – rather than creating foundational technology. This dynamic not only suppresses an engineer's individual earning potential, as disproportionate profits flow to leadership and investors, but also leaves businesses vulnerable to vendor lock-in and supply-chain shocks.

Dual Nature of AI

Compounding this challenge is the dual nature of modern artificial intelligence. While AI tools promise to accelerate code generation, their practical application reveals significant limitations and challenges. Real-world studies, such as the METR study, show that developers often overestimate AI's productivity benefits and can face slowdowns of nearly 20% due to the time spent fixing flawed or inefficient code.

This highlights that human oversight remains indispensable, especially when AI outputs must interface with custom hardware or meet strict safety standards.

The opportunity lies with engineers who understand the full system – electronics, mechanics, and secure architecture – and can effectively validate and harden AI-driven solutions.

Societal Challenges

Simultaneously, society is placing new and urgent demands on the engineering profession. An aging global population and declining birth rates are tightening the economic noose, with fewer workers supporting more retirees. This demographic headwind necessitates greater automation in manufacturing, food production, and healthcare.

The engineers who can deliver these solutions – by designing robotic arms for harvesting, smart greenhouses for urban farming, or humanoid helpers for elder care – will be at the forefront of tackling this challenge and opening new economic frontiers.

Beyond this, in a world flooded by misinformation and clickbait, engineers have an ethical duty to build systems that prioritize truth and transparency, embedding features like content-verification protocols and secure data handling to foster a trustworthy digital environment.

Changing Demands

These evolving demands expose a critical disconnect in traditional education. Costly four-year degrees too often leave graduates with narrow skill sets and surprisingly high unemployment rates (6-7.5%) that rival non-technical fields. This mismatch arises from curricula that prioritize isolated foundational skills or include unrelated electives over the practical, interdisciplinary training modern industry requires.

The path forward is through a more streamlined and relevant education that acts as a catalyst for resilience. By replacing less applicable courses with accelerated, hands-on projects, institutions can transform learners from passive code-integrators into formidable innovators.

Globally, leading institutions are already recognizing this need. In Nordic countries like Sweden and Finland, programs that integrate sustainability, ethics, and interdisciplinary skills are producing graduates who excel at innovation.

By adopting similar approaches – offering real-world modules in robotics prototyping, embedded security, and end-to-end system integration – we can empower engineers to meet today's complex demands and build the resilient, automated, and trustworthy systems our world urgently needs.

The Role of Institutions in Cultivating End‑to‑End Engineers

As technology shifts ever faster – reintegrating software with custom hardware, AI-driven automation, and secure connected systems – traditional universities risk obsolescence unless they reinvent themselves. Beyond breaking down academic silos, forward‑looking institutions will need to embrace four key strategies:

1. Embrace Agility Through Continuous Curriculum Evolution

Modular, Stackable Credentials
Universities should offer micro‑certificates in robotics prototyping, embedded security, or systems integration alongside full degrees. Students and professionals can assemble just the modules they need, when they need them – mirroring the on‑demand model of platforms like Coursera or Google’s own AI toolkits.

Real‑Time Industry Feedback Loops
They should also have rolling curriculum reviews with employer advisory boards. If a new sensor technology or cloud‑native inference engine emerges, courses can pivot within months, not years, ensuring graduates never learn outdated tools.

2. Partner with EdTech Leaders – Don’t Compete Alone

Leverage Existing Toolchains
Rather than ignoring Google’s free AI labs or Microsoft’s cloud credits, universities can integrate them directly into their coursework. Assignments could require deploying a hardware‑accelerated model on Google Coral or securing an Azure‑hosted IoT network.

Co‑Create Open Educational Resources
Institutions could also collaborate on open‑source textbooks, interactive labs, and tutorial videos – both to amplify institutional reach and to demonstrate that the university is part of, not apart from, today’s creator economy.

3. Prioritize User‑Centric Design in Education

Student and Employer Needs First
Schools should also treat their “customers” (students and hiring companies) as co‑designers. Conduct regular surveys and job‑task analyses: What exact blend of Linux kernel debugging, CAD design, and cryptographic key management does the next‑gen engineer need? Then build courses to match.

Flexible Delivery Modalities
They could also combine in‑person maker‑space workshops with online simulators (for example, Gazebo robotics, virtual FPGA labs) so that learners worldwide can participate – reducing geographic and economic barriers.

4. Cultivate an Ecosystem of Lifelong Learning

Alumni‑for‑Credit Programs
Universities could offer discounted, advanced modules for graduates to return and upskill as hardware standards or threat landscapes evolve. This continuous‑learning pathway turns one‑off degrees into multi‑decade partnerships.

Innovation Incubators and Industry Challenges
They could also host hackathons, sponsored capstone projects, and startup incubators right on campus. When students design and pitch end‑to‑end solutions for real companies – say, a secure medical‑robotics prototype – they graduate not just with a diploma, but with market‑tested experience and potential investors.

5. Staying Relevant – and Un-gatekeeping

With Google, Apple, and a legion of online platforms freely distributing cutting‑edge AI, robotics toolkits, and interactive tutorials, any institution that clings to century‑old lecture halls and fixed curricula looks increasingly like a barrier, not a gateway. To avoid that fate:

Shift from “Seat Time” to “Skill Proof”: Replace rigid credit hours with outcomes‑based assessments – portfolios, live demos, and secure system audits prove mastery far better than final exams.

Align incentives around impact, not enrollment: Reward faculty for evolving courses, publishing open resources, and mentoring student startups rather than gatekeeping admissions or ballooning class sizes.

By viewing themselves not as ivory‑tower knowledge guardians but as agile partners in an ever‑changing tech ecosystem, educational institutions can remain indispensable. They’ll graduate engineers who wield software and hardware with equal fluency, who adapt on the fly, and who drive innovation – and who never fear being “left behind” by the next big Google toolkit.

Proposed Reforms: Designing End-to-End Programs

Curriculum Transformation

To implement end-to-end engineering education, institutions should redesign curricula to prioritize interdisciplinary skills across a structured timeline like the following:

Year 1: Core Foundations – Focus on mathematics (calculus, linear algebra, probability) and programming (Python, C++, JavaScript), introducing systems thinking, basic robotics concepts, and an overview of cybersecurity principles. This foundational year ensures students build a strong technical base while gaining exposure to interdisciplinary applications.
Year 2: Software and Hardware Integration – Combine software development with mechanical engineering, emphasizing hands-on projects like robot prototyping. Courses might include designing simple robotic systems, such as a sensor-based navigation device, to connect digital and physical systems and introduce students to hardware constraints.
Year 3: Cybersecurity and Ethics – Teach cybersecurity principles, such as encryption and secure system design, alongside AI ethics to promote responsible technology development. Projects could involve securing IoT devices or analyzing AI-generated code for vulnerabilities, preparing students for real-world challenges.
Year 4: Capstone Projects – Require students to design and deploy real-world systems, such as secure IoT devices, autonomous robots, or energy-efficient automation systems, integrating all learned disciplines. These projects should involve collaboration with industry partners or research labs to ensure practical relevance.

This structure prioritizes practical, relevant skills, replacing less applicable courses with interdisciplinary modules that align with industry needs.

Faculty and Resources

Recruiting faculty with expertise in robotics, mechanical engineering, and cybersecurity is essential for delivering a robust curriculum. Institutions can support collaboration through training programs, workshops, and incentives like joint research grants. For example, faculty from computer science and mechanical engineering could co-teach courses on robotic system design, fostering an interdisciplinary approach.

Investments in infrastructure, such as robotics labs, 3D printing facilities, and cybersecurity simulation environments, are necessary but can be costly. Institutions can implement phased rollouts, starting with virtual simulations or open-source tools to reduce initial expenses.

Grants from organizations like the National Science Foundation (NSF) or partnerships with industry can offset costs, ensuring long-term sustainability. For instance, virtual robotics platforms like Gazebo allow students to simulate robot designs before building physical prototypes, making training more accessible.

Industry Collaboration

Partnerships with industry provide hands-on experience, ensuring students gain practical skills aligned with market needs. These collaborations should prioritize ethical practices, focusing on projects that address societal challenges, such as sustainable technology, secure systems, or healthcare robotics.

For example, joint labs with companies developing energy-efficient automation systems can enhance learning while fostering responsible development. Institutions must ensure partnerships emphasize student development and societal benefit, avoiding scenarios where corporate priorities overshadow educational goals.

Accessible and Flexible Pathways

To make end-to-end education accessible, institutions can offer accelerated programs, such as three-year degrees or modular bootcamps, incorporating AI tools to enhance efficiency.

For example, once they’ve learned key programming concepts, students could use AI-assisted coding platforms to prototype systems, learning to validate outputs for accuracy and security. Online platforms can broaden access, enabling diverse populations to benefit from comprehensive training. Partnerships with community colleges and vocational programs can create pathways for underrepresented groups, fostering an inclusive engineering workforce.

Continuous Curriculum Evolution

To remain relevant, institutions must continuously evolve their curricula to reflect emerging technologies and industry trends. This includes incorporating advancements in AI, such as generative models or reinforcement learning, and addressing new cybersecurity threats, like quantum computing risks. Regular feedback from alumni, industry partners, and students can ensure curricula stay aligned with real-world needs, preparing graduates for long-term success.

Benefits for Graduates and Society

Enhancing Graduate Outcomes

End-to-end education prepares graduates for a competitive market, reducing unemployment risks and enabling higher earnings. With skills in AI oversight, robotics, and hardware design, graduates can pursue roles in high-demand fields like healthcare robotics, secure IoT systems, or autonomous vehicle development, commanding 10-20% higher salaries due to their interdisciplinary expertise.

For example, engineers trained in robotics and cybersecurity can design secure medical robots, addressing the growing demand for healthcare automation.

By launching startups or freelancing, end-to-end engineers can innovate independently, bypassing traditional corporate structures and sharing more directly in the value they create.

Societal Contributions

Society benefits significantly from resilient, secure systems designed by end-to-end engineers. Secure robots and IoT devices protect critical infrastructure, such as manufacturing plants, hospitals, or transportation networks, from cyber threats.

For example, a secure robotic system in a hospital could ensure reliable operation of surgical robots, improving patient outcomes. Training in AI ethics ensures graduates prioritize societal good, mitigating risks like misinformation by designing platforms with robust content verification.

Accessible, accelerated programs promote equity, fostering diverse talent pools and countering job polarization, where AI enhances 25% of roles but automates others. By making education more inclusive, institutions can reduce disparities, ensuring underrepresented groups have access to high-demand careers in engineering.

Sustainability and Global Impact

Sustainability is a key benefit of end-to-end education. Engineers trained in holistic design can create energy-efficient systems, such as optimized robots for logistics or manufacturing, aligning with global environmental goals.

For instance, a robotic system designed to minimize energy consumption in a warehouse could reduce carbon emissions, contributing to sustainability efforts. Institutions adopting this model produce leaders who drive innovation and inclusive growth, addressing global challenges like climate change and digital equity.

Ethical Technology Development

End-to-end education fosters ethical awareness, equipping graduates to combat societal challenges like misinformation and system vulnerabilities. By integrating AI ethics and cybersecurity, graduates can design technologies that prioritize public good, ensuring platforms and systems are trustworthy and resilient. This approach aligns with the growing demand for ethical technology, as emphasized by many in the field who believe in the importance of critical thinking and responsibility in engineering.

Overcoming Challenges in Implementation

Faculty Engagement and Training

Transitioning to end-to-end programs may face resistance from faculty accustomed to traditional, siloed teaching. Institutions can address this through training workshops, collaborative research opportunities, and incentives like joint research grants.

For example, as mentioned above, faculty from computer science and mechanical engineering could co-develop courses on robotic system design, fostering interdisciplinary collaboration. Hiring experts in robotics, cybersecurity, and mechanical engineering ensures a capable teaching staff equipped to deliver comprehensive curricula.

Infrastructure Investment

The cost of infrastructure, such as robotics labs, 3D printing facilities, and cybersecurity simulation environments, poses a significant hurdle. Institutions can implement phased rollouts, starting with virtual simulations using tools like ROS (Robot Operating System) or Gazebo, which allow students to prototype systems without physical hardware. Grants from organizations like the NSF or partnerships with industry can offset costs, while open-source tools enhance accessibility, ensuring equitable access to training.

Curriculum and Accreditation

Redesigning curricula to meet accreditation standards, such as those set by ABET, requires a modular approach that integrates interdisciplinary skills while maintaining compliance. Institutions can pilot programs to test reforms, gradually incorporating modules like robotics or cybersecurity into existing curricula.

For example, a pilot program might introduce a robotics module in year two, allowing institutions to assess outcomes before full implementation. Regular reviews ensure curricula remain aligned with industry needs and accreditation requirements.

Building Stakeholder Support

Securing stakeholder support requires demonstrating the benefits of end-to-end education, including lower unemployment rates (potentially dropping below 3% with holistic training), higher graduate earnings (10-20% above traditional programs), and societal impact through secure, sustainable systems.

Engaging alumni, industry partners, and students in curriculum design builds trust and ensures relevance. For instance, advisory boards with industry representatives can provide insights into emerging trends, aligning programs with market demands.

Promoting Equity and Access

To ensure equitable access, institutions should leverage online platforms and modular degrees, reducing costs and reaching diverse populations. Partnerships with community colleges and vocational programs can create pathways for underrepresented groups, fostering an inclusive engineering workforce.

For example, online courses in robotics or cybersecurity can provide access to students in remote or underserved areas, while modular bootcamps allow working professionals to upskill efficiently.

Addressing Scalability

Scaling end-to-end programs requires strategic planning to balance quality and accessibility. Institutions can start with small cohorts, refining curricula based on feedback before expanding. Collaborations with other universities or online education platforms can share resources, reducing costs and increasing reach. For instance, a consortium of universities could develop shared virtual labs, enabling cost-effective training across institutions.

Conclusion: A Path Forward for Engineering Education

The case for end-to-end engineering education is compelling in a world shaped by AI, interconnected systems, and evolving societal needs. Traditional software engineering programs, with their focus on intermediary roles, must evolve to prepare graduates for the complexities of modern industries.

By integrating software development with robotics, mechanical engineering, and cybersecurity, institutions can produce versatile, innovative engineers who lead in a technology-driven world.

Reforms require bold action: transforming curricula to prioritize interdisciplinary skills, investing in faculty and infrastructure, fostering ethical industry partnerships, and promoting accessible pathways.

Case studies from MIT, Stanford, Vanderbilt, and global institutions like those in Nordic countries demonstrate the transformative potential of this approach, with graduates excelling in diverse roles, founding startups, and building resilient systems. Emerging programs at institutions like ETH Zurich and the University of Toronto further highlight the global applicability of end-to-end education.

Challenges like faculty resistance, infrastructure costs, and accreditation hurdles can be addressed through strategic planning, including phased rollouts, grants, and stakeholder engagement. Online platforms and partnerships with community colleges ensure equity, fostering a diverse talent pool that drives inclusive growth.

End-to-end education is not just an opportunity – it’s a necessity for equipping engineers to navigate a complex, technology-driven world. By embracing this model, institutions can empower the next generation to build innovative, secure, and sustainable systems that benefit society, ensuring a resilient and equitable future for all.

Further Resources:

Ready to become an End-to-End Engineer – mastering software, hardware, AI deployment, robotics, and cybersecurity to build complete systems from the ground up?

Don't just integrate – innovate and lead. You can enroll in LUNARTECH AI for Executives, tailored for leaders who want to strategize, fund, and deploy cutting-edge AI solutions without falling behind in the fast-evolving tech landscape.

LunarTech AI for Executives

For leaders and frontline professionals who feel the pressure to “get AI” but don’t speak code, this 1- to 3-day program delivers exactly what you need: no fluff, no jargon. In clear language, we unpack how generative AI, large-language models, and regulatory frameworks such as the EU AI Act are reshaping compliance, risk, and client service.

Next, we roll up our sleeves. You’ll practice with ChatGPT, Phoenix, Gemini, and other curated tools to summarize 200-page reports in minutes, flag hidden risks, and automate repetitive workflows. Expect live demos, breakout labs, and case studies drawn straight from banking, asset management, and insurance.

By the final session you’ll have a road-ready playbook for piloting AI safely – from data-governance checklists to ROI metrics your CFO will love. Graduates leave with a certificate, a toolkit of prompts, and the confidence to champion AI initiatives inside their own departments.

Format: Online or on-site, 1–3 days
Cost: $997 per participant

Apply Here: https://www.lunartech.ai/programs/ai-for-executives

Other Resources

Lens | LUNARTECH - https://lens.lunartech.ai/
YouTube | LUNARTECH - https://www.youtube.com/@lunartech_ai
Linkedin | LUNARTECH - https://www.linkedin.com/company/lunartechai/
Substack | LUNARTECH - https://lunartech.substack.com/

How to Get Promoted from Senior to Staff Engineer – Tips from My Experience

Shruti Kapoor — Mon, 28 Jul 2025 23:55:34 +0000

Navigating the journey from senior engineer to staff engineer can be daunting. Promotions are often confusing, and this particular leap can feel even more ambiguous.

As someone who has successfully transitioned to a Staff Engineer role, I want to share my insights and experiences to help you on this journey.

In this article, I’ll address key questions you might have, like:

What is a Staff Engineer and how does the role differ from a Senior Engineer?
What do you need to become a Staff Engineer?
How do you know you’re ready?
Who do you talk to for guidance?

I’ll also share my personal story and a structured six-month plan to help you achieve your goals. Let’s dive in!

If you’d also like to watch this as a video, check this out:

Here’s what we’ll cover:

My Path to Becoming a Staff Engineer
What is a Staff Engineer?
Story Time: Building Font Selection in Slack
The Key Differences between Staff Engineers and Senior Engineers
- Staff Engineer vs Manager Role
Why You Might Not Be Promoted Yet
How to Become a Staff Engineer
Evaluation Criteria
How to Build Your Staff Engineer Portfolio
The Promotion Process
A Six-Month Plan to Get Promoted
Wrapping Up

My Path to Becoming a Staff Engineer

Over the last decade, I’ve worked at companies like PayPal and Slack, progressing from a Junior web developer to a Staff Engineer. Here’s a quick timeline:

Junior Web Developer at Telus Communications in 2013
Web Developer at Pix System in 2017
Software Engineer 2 at PayPal in 2018
- Promoted to Senior Software Engineer in 2019 and later Staff Engineer in 2021.
Lead Member of Technical Staff at Slack in 2022
- Worked for three years before transitioning to full-time content creation.

One key lesson I’ve learned: It’s not just about what you do – it’s about who knows what you do.

What is a Staff Engineer?

A Staff Engineer is a technical leader who drives their team’s effectiveness. You’re judged not only by the features you deliver but also by how you enable and grow your team. It’s not just about code anymore. Staff engineering is a blend of technical expertise, leadership, and mentorship.

The Four Pillars of Staff Engineering:

Technical excellence: Staff Engineers build scalable systems, evaluate technologies, and make critical technical decisions while managing technical debt.
Organizational impact: They solve problems that extend beyond their team, acting as the communication link between various stakeholders.
Mentorship and team development: Staff Engineers grow their team by helping others succeed, onboarding new members, and leveling up their colleagues.
Project leadership: They ensure projects are delivered on time, breaking down complex problems into manageable pieces while leading cross-functional initiatives.

Story Time: Building Font Selection in Slack

As a Staff Engineer, I’ve had the privilege to work on some truly exciting projects. Not only have these opportunities helped me grow my technical expertise, but they’ve also taught me critical interpersonal skills – like team coordination, estimation, overcoming imposter syndrome, managing burnout, and effective delegation. These are essential qualities for any engineer looking to advance their career.

One standout project was the Font Selection feature in Slack. On the Appearance tab in Slack’s Preferences, users can now select a font – let’s say, Comic Sans – and instantly see the change reflected everywhere in the application.

This project was a classic example of a greenfield initiative: it had never been done before in Slack. That meant navigating technical debt, considering migrations to new systems, and sometimes deprecating outdated components. It also required collaboration with teams across the organization, because the blast radius of a feature like this was huge. The work impacted not just one area, but the entire user experience.

Being a technical leader on such a project was both challenging and rewarding. It stretched my skills and reinforced the importance of cross-functional teamwork and technical vision in delivering high-impact features.

The Key Differences between Staff Engineers and Senior Engineers

Senior Engineers are excellent executors who can take complex requirements and turn them into working software. They solve technical problems efficiently and mentor junior developers on their team.

Staff Engineers are strategic thinkers who identify what problems need to be solved in the first place. They look across the entire organization to find opportunities for impact and then create projects to address them.

Senior Engineers are typically assigned projects by their manager or team lead and work on priorities set by their leadership. They excel at executing defined work and solving complex problems within their domain.

Staff Engineers, however, operate on a bigger scale beyond their team. They proactively look for improvements in the engineering organization and code architecture. Rather than waiting for direction, they chart out a plan to improve the codebase – figuring out performance optimizations, analyzing customer feedback, enhancing developer productivity, reducing technical debt.

When Staff Engineers identify these improvement areas, they don’t just raise tickets or file requests. They take ownership of crafting solutions by defining a project scope, writing a tech spec, outlining deliverables and creating architectural designs. This shift from reactive to proactive work represents one of the key transitions in moving from senior to staff level.

Think of it this way: Senior Engineers are given projects, Staff Engineers make projects.

Staff Engineering isn't about being a better individual contributor. It's about multiplying the effectiveness of everyone around you and growing others.

Staff Engineer vs Manager Role

While Staff Engineers interview new candidates, mentor engineers, provide feedback, and chart out technical direction, their role significantly differs from a Manager-level position in most companies. Although some responsibilities overlap, there are key distinctions between these two career tracks.

Staff Engineers do not:

Participate in the hiring and firing decisions.
Have input in promotion process beyond providing feedback for the employee
Have direct reports
Hold formal management authority

The fundamental distinction:

Staff Engineer: Technical leader who influences through expertise and mentorship
Manager: Combines HR, technical, and project leadership with formal authority

In most organizations, Staff Engineering and Management represent two distinct career advancement paths, allowing technical professionals to grow in seniority and impact without transitioning into people management roles.

Why You Might Not Be Promoted Yet

Here are five common reasons holding people back:

Waiting to Be Told You’re Ready

Many people fall into the trap of waiting for their manager to tap them on the shoulder and say, "You're ready for promotion." This passive approach can keep you stuck for years. The reality is that nobody will come to you to hand you anything. Promotion committees don't promote people based on potential – they promote based on evidence of current performance at the next level.

Fix: Start performing like a Staff Engineer today. Take ownership of cross-team initiatives, mentor junior developers, and drive technical decisions. Share your accomplishments with the broader team.

When you act like a Staff Engineer consistently for 6-12 months and everyone knows how well you are performing, the promotion becomes a formality rather than a leap of faith.

Action Item: Identify one area where you can begin operating at Staff level this month. Whether it's leading a technical design review or mentoring a junior engineer, start demonstrating the behaviors you want to be recognized for. Share status updates on a regular cadence – daily or weekly depending on the team and project, and share challenges encountered and solutions developed. This will get you not only noticed, but also help to create a balance of self-promotion and knowledge sharing.

Believe in yourself – don’t wait for external validation.

Imposter Syndrome

That voice in your head saying "I'm not smart enough" or "I don't know enough" is imposter syndrome. Everyone feels like an imposter at some point. But successful people take action DESPITE feeling like an imposter.

If you let imposter syndrome dictate your actions or prevent you from taking action, you'll hold yourself back from achieving great things. The people you look up to online have also faced imposter syndrome. Most successful people encounter it weekly, if not daily. The difference is that they learn to set it aside and create a plan to move forward.

The Fix: Reframe your mindset. Instead of thinking "I don't know X," think "I don't know X yet." Focus on your growth trajectory, not your current knowledge gaps. Remember, you were hired for your potential, not because you already knew everything.

Action Item: Keep a "brag document" where you document your accomplishments weekly. When imposter syndrome strikes, review this evidence of your capabilities and impact. Over time, you'll notice that you overcame things that felt hard or impossible in the moment.

You can do hard things!

I made a detailed video highlighting strategies to overcome imposter syndrome. You can check it out here:

Waiting to Become a Deep Technical Expert

Many engineers think they need to know everything about every technology before they can get promoted. This "I must be perfect" thinking will hurt your career. Staff Engineers don't need to know everything. The most important skills you need to know is how to find answers and help others solve hard problems.

The Fix: Learn a little about many things instead of learning everything about one thing. A Staff Engineer who knows some things about databases, systems, and websites will be able to perform and help more effectively than someone who only knows one thing really well. You'll learn more when you work on big projects as a Staff Engineer.

Action Item: Pick 2-3 tech areas that you don't know much about but your company uses, or pick a new tech stack. Spend 30 minutes each week learning about each one. Focus on how they work with the systems you already know.

If you don’t believe in yourself, why should others?

Avoiding Politics

Many engineers think that talking about their accomplishments is "playing politics" and they don't do it. This will ruin their career. Telling people about your wins isn't boastful bragging – it’s necessary. Your boss and the promotion committee can't help you if they don't know what you did.

The Fix: Reframe self-advocacy as data sharing. You're providing factual information about your impact to help leadership make informed decisions. Create a monthly update template that includes: projects completed, problems solved, metrics improved, and people mentored. You are sharing facts, not making up stories.

Action Item: Start a bi-weekly email to your manager highlighting your key contributions. Include specific metrics when possible: "Reduced API response time by 40%, impacting 10,000 daily users" rather than "Improved API performance."

If you don’t toot your own horn, who will?

Struggling with Networking

Internal networking feels awkward to many engineers, but it's crucial for Senior-level and Staff-level promotion. Staff Engineers need to influence across the organization, which requires relationships with people beyond their immediate team.

The Fix: Approach networking as building genuine professional relationships rather than transactional interactions. Start with curiosity about others' work. Ask questions like "What are the biggest technical challenges your team is facing?" or "What would make your team more effective?"

Action Item: Set a goal to have one coffee chat monthly with someone outside your immediate team. Focus on learning about their challenges and how your expertise might help. Share what you are working on and what goals you have in mind. These relationships often lead to collaboration opportunities that demonstrate your Staff-level impact.

How to Become a Staff Engineer

Every company's promotion process is different, but here are some general steps that work at most places:

1. Evaluate Yourself

Create a requirements document using your company's engineering ladder (or a publicly available template like the Block Engineering Career Ladder). Go through each requirement honestly and mark them as either "Achieving" or "Needs Work."

How to do it well:

Be brutally honest with yourself. It's better to underestimate than overestimate your current level
If something feels unclear, ask for examples of each requirement from your manager or other Staff Engineers
Use concrete evidence from your work and links to PRs, not just your feelings about your abilities
Update this document every 1-2 months to track your progress

Common mistake: Being too generous with your self-assessment. If you're not 100% confident you're "Achieving" something, mark it as "Needs Work."

2. Discuss Gaps with Your Manager

First, schedule a one-on-one meeting with your manager to review your evaluation. Work together to identify specific projects that can help you grow in areas where you need improvement. This meeting can feel painful, but it is extremely helpful in level setting. Request feedback on your assessment to see if they agree or disagree.

After evaluation with manager:

Have 2-3 specific areas you want to focus on
Ask about upcoming projects that could help you develop these skills
Plan out a process for achieving those goals.

What to ask:

"What projects in the next 6 months could help me develop [specific skill]?"
"Who are the Staff Engineers I should learn from?"
"What does success look like for someone at my level trying to get promoted?"

Follow-up: Send a summary email after the meeting with your agreed-upon focus areas and next steps.

3. Build a Support Squad

Promotion committees want to hear from multiple people about your impact. Your manager's opinion alone isn't enough. So it’s a good idea to build a support squad for yourself. These are people other than your manager, such as mentors, mentees, leadership folks or inspiring colleagues, who can vouch for your skills and impact during promotion discussions. These are essentially your “sponsors”.

Who to include:

A skip-level manager (your manager’s manager) who can speak to your organizational impact
Engineers who have worked with you on cross-team projects
Engineers you've mentored who can talk about your leadership and teaching skills
Staff Engineers from other teams who can validate your technical decisions

How to build these relationships:

Volunteer for cross-team projects and initiatives
Offer to help other teams with technical challenges in your area of expertise
Attend company tech talks and ask thoughtful questions
Share your knowledge through internal presentations or documentation

These relationships should be genuine, not transactional. Focus on offering value first.

4. Perform Like a Staff Engineer

Promotion committees promote people who are already performing at the next level, not those who might be able to perform at that level. Start acting as a Staff Engineer now. Deliver impactful projects, mentor others, and take ownership of technical decisions.

How to get started:

Pick one area where you can start demonstrating Staff-level impact this month
Volunteer for the next cross-team project or technical initiative
Ask your manager if you can lead the technical design for an upcoming feature
Start mentoring a junior engineer and document your impact on their growth
Share knowledge through documentation, presentations, and code reviews
Identify and address technical debt that impacts multiple teams
Evaluate new technologies and make recommendations

Evaluation Criteria

To transition to Staff Engineer, you need to demonstrate consistent excellence across multiple dimensions. It helps to understand the evaluation criteria and take on projects that showcase the following -

Deep Technical Expertise

You’re expected to:

Write detailed technical specs for others to follow.
Define shared standards (e.g., caching layers, API contracts).
Improve system-wide architecture across teams (e.g., microservices communication that lowers latency by 40%).

Examples of projects that exhibit these skills:

Architecting scalable, fault-tolerant systems (e.g., disaster recovery for payment processing).
Redesigning systems like user login to support 50M+ daily users with 99.99% uptime.

Problem Solving & Innovation

You’re expected to:

Build reusable tools or frameworks widely adopted by others.
Lead new tech adoption (e.g., Kubernetes, chaos engineering).
Set engineering-wide standards through your innovations.

Examples of projects that exhibit these skills:

Fixing infinity mirroring problem in video recording software. (I did this and filed my first patent!)
Solving memory leaks or production crashes.
Creating innovative solutions that reduce cost or improve performance

Technology Leadership

You improve the team's quality standards and lead technical decisions, such as by-

Making thoughtful buid-vs-buy decisions
Reducing technical debt
Establishing best practices and ensure team follows them
Leading tech migrations while ensuring 0 downtime.
Giving helpful, educational code review feedback
Setting team-wide coding, testing, and review standards
Creating templates, guides, and documentation that improve efficiency.

Mentoring Engineers

You help other engineers grow by:

Guiding junior/mid-level engineers through complex systems.
Reviewing designs and code thoughtfully.
Teaching system design, architecture, and debugging skills.

You might:

Create learning plans (e.g., 6-month onboarding curriculum).
Host workshops (e.g., debugging production, distributed systems).
Help others think like system designers.
Help other engineers get promoted.

How to Build Your Staff Engineer Portfolio

Think of your portfolio as your promotion story. It should represent your accomplishments over the past 12–18 months, backed by clear proof. You’re not just showing what you’ve built – you’re showing how you think, how you lead, and the impact you’ve made.

What to Include

Your portfolio should be 10-15 pages that tell a compelling story about your technical leadership. Here’s what makes a strong Staff Engineer portfolio:

Executive summary: One-page overview highlighting your most significant contributions
Technical leadership projects: 3-4 major initiatives presented as detailed case studies
Metrics dashboard: Quantified impact across performance, cost savings, and adoption rates
Peer testimonials: External validation from engineers across different teams
Growth trajectory: A 12–18 month view of how your scope and influence have grown.

How to Structure Each Case Study

Each case study should follow a structured format that tells a complete story. First, you’ll want to discuss the problem statement: what was broken? Who was affected? Why did it matter?

Then cover the technical approach you took. You can discuss the high-level architecture, key decisions, and trade-offs you considered. After this, you can discuss how you implemented everything – the timeline, how you coordinated your team, and any technical challenges you faced.

Finally, you’ll share your results and discuss the business and technical impact with before-after metrics. It’s also a good idea to share any lessons learned and talk about what you'd do differently and how it contributed to your growth.

The Promotion Process

Most companies use a committee-based approach where your manager advocates for your promotion using a comprehensive packet of evidence. The committee asks one simple question: Do we know this person and do we trust their credibility?

Your manager submits a promotion packet to the committee and you are evaluated against everyone else who is up for promotion. To help your manager put your best foot forward in front of the committee, I recommend that you build a promotion packet with your manager. This packet includes a few key components.

1. Career Plan & Engineering Ladder Evaluation

This is the same document that you have hopefully been working on with your manager for a while. This document maps your current performance against your company's engineering ladder criteria. This isn't just a self-assessment – it's a strategic document with concrete evidence for each competency level.

Key elements to include:

Line-by-line evaluation against ladder criteria
Specific examples and proof points for each skill area
Clear demonstration of "Achieving" performance across all relevant categories
Identification of any gaps and plans to address them

2. Portfolio & Proof of Accomplishments

Make sure your portfolio explains not just what you accomplished but the impact you made along with proofs. It should include:

Code examples and design documents
Finished projects and improvements you made
Feedback from users and coworkers
Comments from code reviews and awards you got

3. Brag Document

We’ve talked about this a bit already, but this detailed log serves as your professional highlight reel, demonstrating both quantitative metrics and qualitative growth over your review period.

Quantitative metrics to track:

Pull requests created and reviewed
Features delivered and timeline performance
Code review discussions
Bug fix rates and system improvements
Performance optimizations achieved

Qualitative accomplishments to document:

Technical leadership moments
Cross-functional collaboration successes
Mentoring and knowledge sharing contributions
Innovation and creative solutions implemented

Growth documentation:

New technologies learned and applied
Leadership opportunities taken
Feedback incorporation and skill development

4. Testimonials

Get recommendations from teammates, other departments, people you helped, and customers. Ask them to give specific examples.

Your packet should be organized, based on facts, and show how you've grown. Make it simple for the committee to see your value and for your manager to speak up for you.

A Six-Month Plan to Get Promoted

Here’s a structured approach to help you navigate your promotion:

Month 1: Evaluate Yourself

Review your company’s engineering ladder and assess your gaps. If they don't have one, use a publicly available career ladder.

Month 2: Create a Plan

Discuss your evaluation with your manager. Identify projects to address your gaps.

Month 3: Work on Impactful Projects

Start networking with your skip-level manager and other stakeholders.

Month 4: Gather Proof

Document your work, seek feedback, and build evidence of your contributions.

Month 5: Review Progress

Update your career doc and review it with your manager.

Month 6+: Keep Going

The process may take longer than six months, but persistence and structure will help you get there. Don’t worry if it does take longer – that doesn’t mean you’re not qualified or won’t make it. Keep working at it.

Wrapping Up

Promotion isn’t always about skills – it’s also about timing and organizational factors. If you’re feeling burnt out or unsure about continuing in tech, it’s okay to pause or reconsider your path.

Take charge of your career. Advocate for yourself and share your accomplishments. And if you need personalized guidance, book a one-on-one session coaching session with me to create a personalized plan for you: https://topmate.io/shrutikapoor08.

If you enjoyed this article, share it with someone you know and spread the love.

Connect with Me:

YouTube: https://www.youtube.com/@shrutikapoor08
Sign up for articles like this: http://bit.ly/shruti-newsletter
Discord: bit.ly/shruti-discord
Twitter: @shrutikapoor08
Instagram: https://www.instagram.com/itsshrutikapoor/

How to Reduce Technical Debt in the Power Platform

Brandon Wozniewicz — Fri, 06 Jun 2025 10:37:39 +0000

Technical debt refers to the future cost – measured in terms of time, money, effort, or opportunity – of choosing expedient solutions today instead of more deliberate and scalable ones. And it's not just a pro-code concept.

It might be easier to understand if we compare it to financial debt.

Howard G. Cunningham – the creator of the first wiki – described technical debt this way:

Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable.

The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.

In this article, you'll learn why technical debt is just as much a concern in low-code projects as in traditional development – and why, in some ways, it can be even more prominent. We'll walk through eight common contributors to technical debt in Power Platform projects that, if left unchecked, can lead to future headaches.

Why Technical Debt Is Also a Low-Code Problem
Eight Examples of Technical Debt in a Low-Code Project
Wrapping Up

Why Technical Debt Is Also a Low-Code Problem

Technical debt builds when short-term decisions ignore long-term consequences. While it exists in any development, low-code platforms increase the risk for a simple reason: they remove much of the traditional friction that forces teams to slow down.

With fewer barriers to entry, it's easier for citizen developers – and even professional developers new to the platform – to start building without considering maintainability, scalability, and security.

Low-code platforms enable speed – but if you aren't intentional, the speed can create an environment that fosters the growth of technical debt.

Eight Examples of Technical Debt in a Low-Code Project

Hard-Coded or Static Values in Your Code

We've all seen code like this:

Office365Outlook.SendEmailV2(
  gblAuthenticatedUser.Email, 
  "Sign-Up Request Received!", 
  "A team member will reach out shortly with more information."
)

At first glance, this appears to be fine. But what happens if the subject or body of the email needs to change?

Hard-coded values are fragile. A better approach is to store your email templates in a data source, even if it only contains one record.

With({
  wthEmailTemplate: LookUp(EmailTemplates, TemplateType="new_signup")},
  Office365Outlook.SendEmailV2(
    gblAuthenticatedUser, 
    wthEmailTemplate.Subject, 
    wthEmailTemplate.Message
  )
)

Now, if the email needs to be changed, you update the data source, not the application logic.

Duplicated Code

While it can be trickier to avoid than in traditional code, duplicate logic is a future maintenance headache.

Imagine two different ways to create comments in an application:

//Main dialog in a stream of comments
With(
    {
        wthNewlyCreatedComment: Patch(
            Comments,
            Defaults(Comments),
            {Comment: txt_hs_comment.Value}
        )
    },
    Collect(
        colComments,
        wthNewlyCreatedComment
    );
    Set(
        gblCommentCount,
        CountRows(colComments)
    )
)

//A different dialog when replying to another's comment
With(
    {
        wthNewlyCreatedComment: Patch(
            Comments,
            Defaults(Comments),
            {Comment: txt_hs_dialogComment.Value}
        )
    },
    Collect(
        colComments,
        wthNewlyCreatedComment
    );
    Set(
        gblCommentCount,
        CountRows(colComments)
    )
)

These two blocks do the same thing. If the logic ever changes, you must remember to update it in both places.

A cleaner approach is to use a user-defined function (UDF) to encapsulate logic that gets reused across your app.

// App.Formulas
UpdateComments(comment: Text):Void = 
{
	With(
        {
            wthNewlyCreatedComment: Patch(
                Comments,
                Defaults(Comments),
                {Comment: comment}
            )
        },
        Collect(
            colComments,
            wthNewlyCreatedComment
        );
        Set(
            gblCommentCount,
            CountRows(colComments)
        )
    )
};

And then in each of the locations that need this formula:

//Main dialog in a stream of comments
UpdateComments(txt_hs_comment.Value)

//A different dialog when replying to another's comment
UpdateComments(txt_hs_dialogComment.Value)

As of this writing, user-defined functions in Canvas apps support side effects (such as modifying collections or setting variables), but the overall feature is still in preview.

If UDFs aren't an option for your current use case, a common workaround is to use a hidden button that encapsulates the logic and call it with Select(ButtonName). Just keep in mind: the control must be on the same screen where it's being invoked.

Poor Naming of Controls and Variables

Home Screen
  ButtonCanvas4
  TextCanvas2
  ButtonCanvas1

What's wrong with the scenario above? It is impossible to know what each control is responsible for.

Good naming isn't just a nice-to-have – it's one of the best ways to reduce confusion and improve maintainability, especially in collaborative environments.

Here is an improved version that allows any developer to understand what these controls do:

Home Screen
  txt_hs_userName
  btn_hs_submitForm
  btn_hs_cancelSubmission

In this example, we follow the pattern of:

[control_type]_[screen]_[control responsibility]

This helps make it easy to search for items quickly as well as identify what they do.

Another aspect that naturally lends itself to naming conventions is the use of variables. Canvas apps have various methods for storing data locally. They include:

Collections (ClearCollect/Collect)
Global Variables (Set)
Local Variables (UpdateContext)
Contextual Variables (With functions)

Each type of variable has a different scoping associated with it. Collections are tables and available throughout your entire application. Global variables are also available throughout the entire application. Variables set using UpdateContext are scoped to the screen on which they are declared. And variables contained within a With function are available only within that function.

It is a good idea to ensure that the variable name accurately reflects the type of variable it represents. For example:

// prefixed with "wth" for a with-function scoped variable
With({wthNewlyCreatedUser: Patch(AppUsers,...)},...)

// prefixed with "ctx" for a screen-scoped contextual variable
UpdateContext({ctxCurrentPostVotes: LookUp(colPostVotes, ....)})

// prefixed with "gbl" for global
Set(gblAuthenticatedUser, LookUp(AppUsers,....))

// prefixed with "col" for a collection
ClearCollect(colUserRoles, LookUp(AppRoles, ...))

Each data storage type is designated by a prefix that indicates its kind, which makes debugging an application easier.

Overloaded Screens and Applications

It can be tempting to keep everything on one screen for simple applications. But canvas apps can quickly become non-performant if too many controls or too much logic is on a single screen. The recommended limit is no more than 500 controls per app and 300 controls per screen. Using and editing the application can slow down significantly if these limits are exceeded.

One way to prevent this issue is to think more modularly. For example, you may have both administrative and non-administrative tasks within a single application. Instead, you can make two applications, one for admin users and the other for general users.

Another way to avoid these issues in the same application is to build using components. The controls that make up a component don't count individually towards the screen limits and are also a natural way to reduce duplication within and across your applications. Components can be created within an application or as a component library (if your component needs to be used in multiple applications – for example, loaders/spinners and confirmation dialogs).

For more information on components, refer to this article I wrote about building reusable components.

No Version Notes

As the Power Platform ecosystem grows, advanced versioning techniques are being introduced, including the integration of solutions with Git. But even if you don't have that git integration, there is something simple you can do.

When you save an application after any non-trivial change, use the built-in version notes.

This simple habit will make two things much easier:

If you ever need to roll back changes, it becomes much easier to identify the correct version to roll back to.
When using multiple environments (for example, Dev, Test, and Prod), this can help you identify which version is currently in each environment, as the built-in version numbers may not necessarily match.

To view version notes for a canvas app, select 'View Details' for the app and then select the versions tab.

Invisible Logic

Invisible logic is logic that supports a product, but it is not immediately recognizable. For example, custom APIs and cloud flows can quickly become forgotten if there is no documentation reminding developers that these critical components exist – and what they actually do.

One of the best ways to document a project is by using solutions. Solutions will typically include the majority of a project's assets – often more than 90% – but there are notable exceptions, such as SharePoint lists, Power BI reports, and certain external integrations.

Some things a solution often won't or can't include are assets that belong to core or base solutions – for example, generic cloud flows that serve multiple projects or products. Depending on your solution strategy, you may not want to add these to each solution, and they will only exist in a core or base solution.

Other things that fall under the umbrella of invisible logic include Power BI assets and Dataflows, along with their respective automation architectures (for example, how and when a Dataflow gets triggered).

As a best practice, utilize the self-documenting nature of solutions to provide references to all assets, logic, and dependencies that a project uses. Also, consider adopting a feature-based documentation practice, where each feature or user story implemented includes basic documentation, including high-level implementation details and any underlying logic. This could be a wiki-like document that allows developers, who may be troubleshooting or extending a feature, a simple way to get oriented before diving into an unfamiliar project.

Denormalized Data Models

Data normalization is a topic of its own, but you don't need to be an expert to get started building robust and scalable data models. In simple terms, data normalization involves grouping similar data elements and eliminating duplication.

Take a look at the following example of the employee table.

Employees Table (Denormalized)

| Employee ID | Name   | Department Name | Department Location |
|-------------|--------|------------------|----------------------|
| 1           | Alice  | HR               | Building A           |
| 2           | Bob    | IT               | Building B           |
| 3           | Carol  | HR               | Building A           |
| 4           | Dan    | IT               | Building B           |
| 5           | Eve    | Finance          | Building C           |

In the above table, we can see the EMPLOYEE table records contain information about the department. Conceptually, this is fine, but the main issue is that the attributes of each record not only describe the employee but also provide details about the department.

This type of data is referred to as denormalized data. Denormalized data makes the data model harder to scale and maintain. For example, if the Department Name changes, we must locate every record with that department name and update it accordingly.

Instead, let's examine a more normalized data model that consists of two tables now.

Employees Table (Normalized)

| Employee ID | Name   | Department ID |
|-------------|--------|----------------|
| 1           | Alice  | 1              |
| 2           | Bob    | 2              |
| 3           | Carol  | 1              |
| 4           | Dan    | 2              |
| 5           | Eve    | 3              |


Departments Table

| Department ID | Department Name | Department Location |
|---------------|------------------|----------------------|
| 1             | HR               | Building A           |
| 2             | IT               | Building B           |
| 3             | Finance          | Building C           |

This data model eliminates duplication and simplifies attribute updates for the department, requiring only a single record update to be made. And because each attribute of the EMPLOYEES and DEPARTMENTS tables only describes the primary key of the respective table, this is a normalized data model.

One common misconception among new developers is that more tables are a bad thing. Many believe that fewer data sources are easier to maintain, but that’s not always true.

In development, what makes things easier to maintain isn't less of something, but rather how atomic, modular, and dependent-free it is. For example, a few small, pure functions that do just one thing will be easier to maintain than a single side-effect-producing function that does many things.

Don't shy away from normalized data just because it creates more tables. Shy away from data models that won't scale.

One final note: Denormalized data has its place, too, and it's not a bad thing. For example, reporting data is often denormalized and is much more preferred as it makes reporting logic much easier.

Leading with Layout Before Logic

Low code makes it easy to jump in and start building, which is a significant benefit. But this model can also make it very easy to skip important aspects of development, such as requirement gathering, user interface design, and data modeling.

It's perfectly fine to prototype ideas. This is great for quickly determining if something may or may not be feasible. But you must have the discipline to stop before getting too far along, and take time to plan properly.

For example, consider employing a business logic-first approach. This means that the requirements and core business logic are decided on (and often implemented) before you even start building the user interface.

The core principle of this type of development is that, regardless of the interface a user chooses to interact with our data – and remember, a web application is nothing more than an interface to your data – the core business logic should function properly. In this light, a Canvas app becomes just an aesthetic wrapper that complements what is hopefully well-designed business logic.

Wrapping Up

Technical debt exists in both traditional and low-code development. Recognizing this debt early, before it begins to accumulate, is critical. Some tips that can reduce and keep technical debt at manageable levels are:

Avoid hard-coded or static data in your app logic
Eliminate duplicated logic with user-defined functions (UDFs)
Use consistent naming conventions for controls and variables
Break overloaded apps into multiple screens or multiple apps
Add version notes to track meaningful changes
Document invisible logic such as flows and APIs
Normalize your data to reduce duplication
Start with business logic—not layout or visuals

Found this helpful? I write about practical automation, productivity systems, and building smarter workflows — without the jargon. Visit me at brandonwoz.com.

What is Technical Debt and How Do You Manage it?

Manish Shivanandhan — Fri, 09 May 2025 16:48:38 +0000

You’ve probably heard someone say, “We’ll fix it later.”

Maybe you’ve said it yourself.

In the rush to launch a feature, meet a deadline, or impress a client, you take a shortcut. The code works – for now. The design passes – for now.

But over time, these choices pile up. They slow you down. They make every change harder. That’s technical debt.

Technical debt is a quiet, creeping cost. It doesn’t show up in metrics like churn or conversion rate. But it eats away at your product’s quality, your team’s velocity, and your ability to innovate.

You don’t notice it right away. Then, suddenly, everything feels slower. Nothing is simple anymore.

One bug fix breaks two new things. Engineers groan when they touch certain parts of the code. That’s when you realize you’re in debt.

Let’s talk about what technical debt really is, how it forms, and how you can deal with it before it kills your product.

What Is Technical Debt?

Think of technical debt like financial debt. When you borrow money, you get something now – at the cost of interest later.

In software, it’s the same. You make a quick decision to save time now, knowing it might cost you more effort down the line.

There’s nothing inherently wrong with that. Sometimes, taking on debt is smart. You might need to ship fast to test an idea or respond to the market.

But if you keep borrowing and never repay, the interest builds. And technical debt doesn’t just grow – it compounds. The longer you leave it, the worse it gets.

You don’t just end up paying more later. You slow down your whole team.

Every new feature takes longer. Bugs multiply. Morale drops. And eventually, your product feels fragile. That’s when the real damage begins.

Types of Technical Debt

Not all debt is the same. Some is like a short-term loan – you know you’re taking it and why. Other debt is like a bad mortgage – no one even knows it’s there until things break.

Here are the most common types:

Intentional debt — You cut corners to hit a deadline but log it and plan to fix it. This can be healthy if managed well.
Unintentional debt — You didn’t realise the shortcut was harmful. Often happens with new tech or unclear requirements.
Environmental debt — Your tools, libraries, or frameworks get outdated. Even if your code is clean, it sits on crumbling infrastructure.
Process debt — The way you build software becomes inefficient. Poor handoffs, unclear documentation, or weak testing pipelines all contribute.

Recognising which type you’re dealing with helps you prioritise. Not all debt needs immediate repayment. But all of it needs attention.

How Technical Debt Happens

As you can probably imagine, technical debt shows up in many ways.

Sometimes it’s deliberate. You make a tradeoff. You know it’s a shortcut, and you plan to clean it up later. That’s manageable – if you actually clean it up.

But most technical debt isn’t planned. It sneaks in through decisions that feel small in the moment. A rushed feature. A new hire not trained on the codebase. A spec that changes mid-sprint. Over time, these small cracks become deep fractures.

Here’s a simple example:

// Temporary workaround for product discounts
function applyDiscount(price, productType) {
  if (productType === 'electronics') {
    return price * 0.9; // 10% off
  } else if (productType === 'clothing') {
    return price * 0.8; // 20% off
  } else if (productType === 'books') {
    return price * 0.95; // 5% off
  } else {
    return price;
  }
}

This started as a quick fix. But over time, new product types get added with more exceptions.

Soon, you will have twenty if-else branches. It’s fragile. Every change risks breaking something. That’s technical debt.

The worst part? You might not even notice until a year later, when a bug in that logic takes hours to trace. You wonder, “How did this get so messy?” The answer: one shortcut at a time.

A better long-term approach in the above example would be a configuration-driven system or a discount rules engine.

// Config-driven discount logic
const discountRates = {
  electronics: 0.10,
  clothing: 0.20,
  books: 0.05
};

function applyDiscount(price, productType) {
  const discount = discountRates[productType] || 0;
  return price * (1 - discount);
}

Why Technical Debt Can Be Dangerous

Technical debt slows you down. That’s its most visible cost.

A feature that should take a day now takes a week. Simple changes break unrelated things. Your team spends more time fixing than building.

But the real danger goes deeper. Technical debt makes you afraid to touch your code.

Engineers stop refactoring because “it’s too risky.” You start saying no to new ideas because the system can’t handle them. The product becomes rigid. You stop innovating.

It also hurts your team. Developers don’t like working in messy codebases. It leads to burnout. New hires struggle to onboard.

Your best engineers spend their time firefighting instead of creating. Eventually, people leave. And your debt remains.

How to Manage Technical Debt

You can’t eliminate all technical debt. But you can manage it.

First, treat it like real debt. Track it. Prioritize it. Make regular payments.

Start by writing it down. Every time someone takes a shortcut, log it. You don’t need a fancy tool – a shared doc or Jira tag works fine. Just make it visible.

Next, build time into your workflow to pay it off. Use 10–20% of each sprint to refactor or improve the codebase. Don’t wait for a rewrite. Small, steady work adds up.

Code reviews help too. Encourage your team to ask: “Is this a shortcut?” If yes, make a conscious choice. Leave a clear comment. Note the tradeoff. Now it’s not a hidden cost – it’s a known one.

And when you do pay off debt, celebrate it. Make it part of your culture. The same way you’d celebrate shipping a feature, acknowledge when your team makes the codebase better. That builds pride and ownership.

Knowing When to Refactor

You can’t fix all the debt at once. So how do you choose?

Look for signs of pain. If a file breaks every sprint, fix it. If one part of the system takes days to test, improve it. If new hires always get stuck in one module, clean it up.

Focus on the code you touch often. There’s no point polishing a dead feature. But if something is part of your core flow, invest in it.

Also, listen to your team. Engineers know where the pain lives. If someone says, “This part scares me,” take that seriously. Fear in the codebase is a red flag.

When Debt Becomes Fatal

Sometimes, the debt gets so bad that small fixes won’t save you. The system collapses under its own weight. Everything feels slow. Nothing is safe to change. That’s when teams start talking about rewrites.

But rewrites are risky. They take time. They often miss hidden business logic. And they can carry old debt into new code if not done carefully.

If you must rewrite, do it incrementally. Replace modules one at a time. Add tests. Migrate data with care.

And don’t forget why the old system failed. If you don’t fix the culture, the new system will rot too.

Final Thoughts

Technical debt isn’t evil. It’s part of building software. But like financial debt, it needs discipline. You can’t just ignore it and hope it goes away.

Great products aren’t just well-designed. They’re well-maintained. The teams behind them care about quality — not just in what users see, but in what engineers live with every day.

So the next time you say, “We’ll fix it later,” ask yourself: will you? Or are you just borrowing against the future?

Hope you enjoyed this article. Join my newsletter for similar articles and connect with me on Linkedin.

How to Start Building Projects with LLMs

Harshit Tyagi — Mon, 30 Sep 2024 18:46:25 +0000

If you’re an aspiring AI professional, becoming an LLM engineer offers an exciting and promising career path.

But where should you start? What should your trajectory look like? How should you learn?

In one of my previous posts, I laid out the complete roadmap to become an AI / LLM Engineer. Reading this article will give you insights into the types of skills you’ll need to acquire and how to start learning.

The Best Way to Learn is to BUILD!

As Andrej Karpathy puts it:

Andrej emphasizes that you should build concrete projects, and explain everything you learn in your own words. (He also instructs us to only compare ourselves to a younger version of ourselves – never to others.)

And I agree – building projects is the best way to not just learn but really grok these concepts. It will further sharpen the skills you’re learning to think about cutting edge use cases.

But the main challenge with this learning philosophy is that good projects can be hard to find.

And that’s the problem I am trying to resolve. I want to help people, including myself, discover and build practical and real-world projects that help you develop skills that are worth showcasing in your portfolio.

Here’s What We’ll Cover:

What Should Be Your First Project?
Project #1: YouTube Video Summarizer
Project #2 preview: Multi-purpose Customer Service Bot
Project #3 preview: RAG-Powered Support Bot
Conclusion

What Should Be Your First Project?

If you’re a beginner who knows basic to intermediate programming, your initial projects should showcase that you can comfortably build applications with LLMs.

They should demonstrate that:

you know what APIs are
you know how to consume them
you know how to build products that people actually want to use

Building a chatbot provides a great starting point, but at this point everyone has developed one. And there are many solutions for easy Streamlit based prototypes. So, you need to develop something that’s actually usable and has the potential to reach a wider audience.

I’d suggest building a chatbot for WhatsApp or Discord or Telegram. Build a chatbot which solves a problem people struggle with, a problem that companies have started to build solutions for.

If I had to pick a good and, arguably, the most common AI project that every company has started to work on, it would be RAG-powered chatbots.

But before you get to building RAG-powered bots, you should start building something slightly more basic but practical with LLMs.

To kick things off, let’s start by building a YouTube Summariser.

Project #1: Summarise YouTube Videos

We’ll build the first part of this project in this tutorial: the core functionality of a YouTube video summariser tool.

Our bot will:

Receive the YouTube URL.
Validate if the URL is correct.
Retrieve the transcript of the video
Use an LLM to analyze and summarize the video’s content.
Return the summary to the user.

Setup and Requirements

For this project, we’ll code the core functionality in a Jupyter Notebook using the following Python packages:

langchain-together — for the LLM using the LangChain <> Together AI integration
langchain-community — for specific data loaders
langchain — for programming with LLMs
pytube — for fetching video info
youtube-transcript-api — for youtube video transcript

We’ll use the Llama 3.1 model offered as an API by Together AI.

Together AI is a cloud platform that offers the open source models as inference APIs. without worrying about the underlying infrastructure.

Let’s start by installing these:

!pip install — upgrade — quiet langchain
!pip install — quiet langchain-community
!pip install — upgrade — quiet langchain-together
!pip install youtube_transcript_api
!pip install pytube

Now let’s set up our LLM:

## setting up the language model
from langchain_together import ChatTogether
import api_key

llm = ChatTogether(api_key=api_key.api,temperature=0.0, 
                   model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo")

The next step is to process the YouTube videos as a data source. For this we’ll need to understand the concept of document loaders.

Introduction to Document Loaders

Document loaders provide a unified interface to load data from various sources into a standardized Document format.

They automatically extract and attach relevant metadata to the loaded content.
The metadata can include source information, timestamps, or other contextual data that can be valuable for downstream processing.
LangChain offers loaders for CSV, PDF, HTML, JSON, and even specialized loaders for sources like YouTube transcripts or GitHub repositories, as listed in their integrations page.

Categories of Document Loaders

Document loaders in LangChain can be broadly categorized into two types:

File Type-Based Loaders

Parse and load documents based on specific file formats
Examples include: CSV, PDF, HTML, Markdown

2. Data Source-Based Loaders

Retrieve data from various external sources
Load the data into Document objects
Examples include: YouTube, Wikipedia, GitHub

Integration Capabilities

LangChain’s document loaders can integrate with almost any file format you might need.
They also support many third-party data sources.

For our project, we’ll use the YoutubeLoader to get the transcripts in the required format.

YoutubeLoader from LangChain to Get Transcript:

## import the youtube documnent loader from LangChain
from langchain_community.document_loaders import YoutubeLoader

video_url = 'https://www.youtube.com/watch?v=gaWxyWwziwE'
loader = YoutubeLoader.from_youtube_url(video_url, add_video_info=False)
data = loader.load()

Process the YouTube Transcript

Display raw transcript content
Use the LLM to summarize and extract key points from the transcript:

# show the extracted page content
data[0].page_content

The page_content attribute contains the complete transcript as shown in the output below:

Now that we have the transcript, we simply need to pass this to the LLM we configured above along with the prompt to summarise.

First, let’s understand a simple method:

Langchain offers the invoke() method to which you need to pass the system message and the user or human message.

The system message is essentially the instructions for the LLM on how it is supposed to process the human request.

And the human message is simply what we want the LLM to do.

# This code creates a list of messages for the language model:
# 1. A system message with instructions on how to summarize the video transcript
# 2. A human message containing the actual video transcript

# The messages are then passed to the language model (llm) for processing
# The model's response is stored in the 'ai_msg' variable and returned

messages = [
    (
        "system", 
        """Read through the entire transcript carefully.
           Provide a concise summary of the video's main topic and purpose.
           Extract and list the five most interesting or important points from the transcript. For each point: State the key idea in a clear and concise manner.

        - Ensure your summary and key points capture the essence of the video without including unnecessary details.
        - Use clear, engaging language that is accessible to a general audience.
        - If the transcript includes any statistical data, expert opinions, or unique insights, prioritize including these in your summary or key points.""",
    ),
    ("human", data[0].page_content),
]
ai_msg = llm.invoke(messages)
ai_msg

But this method won’t work when you have more variables and when you want a more dynamic solution.

For this, LangChain offers PromptTemplate:

A PromptTemplate in LangChain is a powerful tool that helps in creating dynamic prompts for large language models (LLMs). It allows you to define a template with placeholders for variables that can be filled in with actual values at runtime.

This helps in managing and reusing prompts efficiently, ensuring consistency and reducing the likelihood of errors in prompt creation.

A PromptTemplate consists of:

Template String: The actual prompt text with placeholders for variables.
Input Variables: A list of variables that will be replaced in the template string at runtime.

# Set up a prompt template for summarizing a video transcript using LangChain

# Import necessary classes from LangChain
from langchain.prompts import PromptTemplate
from langchain import LLMChain

# Define a PromptTemplate for summarizing video transcripts
# The template includes instructions for the AI model on how to process the transcript
product_description_template = PromptTemplate(
    input_variables=["video_transcript"],
    template="""
    Read through the entire transcript carefully.
           Provide a concise summary of the video's main topic and purpose.
           Extract and list the five most interesting or important points from the transcript. 
           For each point: State the key idea in a clear and concise manner.

        - Ensure your summary and key points capture the essence of the video without including unnecessary details.
        - Use clear, engaging language that is accessible to a general audience.
        - If the transcript includes any statistical data, expert opinions, or unique insights, 
        prioritize including these in your summary or key points.

    Video transcript: {video_transcript}    """
)

How to Use LLMChain / LCEL for Summarization

A chain is a sequence of steps that consists of a language model, PromptTemplate, and an optional output parser.

Create an LLMChain with the custom prompt template
Generate a summary of the video transcript using the chain

Here, we are using LLMChain but you can also use LangChain Expression Language as well to do this:

## invoke the chain with the video transcript 
chain = LLMChain(llm=llm, prompt=product_description_template)

# Run the chain with the provided product details
summary = chain.invoke({
    "video_transcript": data[0].page_content
})

This will give you the summary object which has the text attribute that contains the response in markdown format.

summary['text']

The raw response will look like this:

To see the Markdown formatted response:

from IPython.display import Markdown, display

display(Markdown(summary['text']))

And there you go:

So, the core functionality of our YouTube summariser is now working.

But this is working in your Jupyter Notebook, to make it more accessible, we’d need to get this functionality deployed on WhatsApp.

How to serve the YT summariser on WhatsApp

For this, we’d need to serve our YT summarisation functionality as an API endpoint for which we are going to use Flask. You can also use FastAPI.

Now we’ll turn all the code in the Jupyter notebook into functions. So, add a function to check if it is a valid youtube URL, then define the summarise function that is basically a compilation of what we wrote in the Jupyter notebook.

You can configure our endpoint in the following manner:

@app.route('/summary', methods=['POST'])
def summary():
    url = request.form.get('Body')  # Get the JSON data from the request body
    print(url)
    if is_youtube_url(url):
        response = summarise(url)
    else:
        response = "please check if this is a correct youtube video url"
    print(response)
    resp = MessagingResponse()
    msg = resp.message()
    msg.body(response)
    return str(resp)

Once your app.py is ready with your Flask API, run the Python script, and you should have your server running locally on your system.

The next step is to make your local server connect with WhatsApp, and that’s where we’ll use Twilio.

Twilio allows us to implement this handshake by offering a WhatsApp sandbox to test your bot. You can follow the steps in this guide here to build this connection.

I got the connection established:

Now, we can start testing our WhatsApp bot:

Amazing!

I explain all the steps in detail in my project-based course on Building LLM-powered WhatsApp Chatbots.

It’s a 3-project course that contains two other more complex projects. I’ll give you a brief summary of those other projects here so you can try them out for yourselves. And if you’re interested, you can check out the course as well.

Project #2 — Build a Bot that Can Handle Different Types of User Queries

This bot acts as a customer service representative for an airline. It can answer questions related to flight status, baggage inquiries, ticket booking, and more. It uses Langchain’s Router and LLM models to dynamically generate responses based on the user’s input.

Different prompt templates are defined for various customer queries, such as flight status, baggage inquiries, and complaints.
Based on the query, the router selects the appropriate template and generates a response.
Twilio then sends the response back to the WhatsApp chat.

Project #3 — RAG-Powered Support Bot

This chatbot answers questions related to airline services using a document-based system. The document is converted into embeddings, which are then queried using Langchain’s RAG system to generate responses. Companies want developers these days who have these skills, so this is an especially practical project.

The guidelines/rules document is embedded using FAISS and HuggingFace models.
When a user submits a question, the RAG system retrieves relevant information from the document.
The system then generates a response using a pre-trained LLM and sends it back via Twilio.

These 3 projects will get you started so you can continue experimenting and learning more about AI engineering.

Customer Support is the most funded category in AI because it reduces the cost instantly if AI can handle communication with disgruntled users.

So, we build bots that can handle different types of queries, intelligent RAG powered bots which will have access to proprietary documents to provided up-to-date information to the users.

That’s why I created this project-based course to help you start building with LLMs.

Check out the course preview here:

And to thank you for reading this guide, you can use the code FREECODECAMP to get a 20% discount on my course.

I want to make this affordably accessible for all those who are sincere about building with AI, so I’ve priced it affordably at $14.99 USD.

Conclusion

In this tutorial, we focused on building a fun YouTube video summarizer tool that is served on WhatsApp.

The bot's core functionality includes:

Receiving a YouTube URL
Validating the URL
Retrieving the video transcript
Using an LLM to summarize the content
Returning the summary to the user

We used a number of Python packages including langchain-together, langchain-community, langchain, pytube, and youtube-transcript-api.

The project uses the Llama 3.1 model via Together AI's API.

We built the core summarisation functionality using

Using LangChain's invoke() method with system and human messages
Using PromptTemplate and LLMChain for more dynamic solutions

To make the tool accessible via WhatsApp:

The functionality is served as an API endpoint using Flask
Twilio is used to connect the local server with WhatsApp
A WhatsApp sandbox is used for testing the bot

To continue building further projects, check out the course.

It is a beginner track course where you start from learning to build with LLMs, then apply those skills to build 3 different types of LLM applications. Not just that – you learn to serve your applications as WA chatbots.

Test Engineering Anti-Patterns: Destroy Your Customer Satisfaction and Crater Your Quality By Using These 9 Easy Organizational Practices

freeCodeCamp — Mon, 30 Sep 2019 13:59:11 +0000

By Cristian Medina

A recent podcast episode reminded me that it's a good idea to examine things from different perspectives. Doing so can expose behaviors that appear purposeful as consequences of environmental factors. You won't succeed at fixing those behaviors unless you do something about the environment that encourages them.

Today the discussion is about Test engineering and organizational practices that tend to lower the quality of your deliverables.

When reading through, please keep in mind that we're emphasizing goals that are contrary to what you expect - lower quality and customer satisfaction. The suggestions may seem cynical and counter-intuitive, but that's the point of the exercise. So let's get started.

1. Make the Test teams solely responsible for quality

One of the best ways to lower quality is by shifting focus away from the product and towards internal politics.

You want to capitalize on default human behavioral responses that minimize reasoning and maximize conflict — what better way of doing so than encouraging tribalism.

Holding only one organization responsible for a goal (no matter what it is) will guarantee the divisive behavior you're trying to obtain.

That group will spend most of its time and resources proving their responsibilities were met, instead of facilitating product goals.

To maximize the impact of this principle, make sure that Test teams only own the quality objective. They should not be responsible for determining release content, that's the Marketing team. Nor should they be involved in setting release dates, that's a Sales function. They also don't determine which problems to fix, that's up to the Product team.

You want the Development team focused on delivering features by the time the Marketing and Sales orgs promised them. So avoid imposing any code commit or merge rules requested by Test, like requiring code reviews, passing linters, passing unit tests, and others. After all, quality is Test's responsibility, not Development's.

Breaking up objectives in this way, turns meeting agendas away from customer experience and towards the blame game. You know you're on the right track when you hear statements like:

"Why didn't you find this bug sooner!"
"The current tests failed, but we really need to merge this new feature so you can start testing it."
"Why didn't Test find this customer issue?"
"We promised the customer a release by this date, so let's defer all Test findings and give them a build today."
"We found the issue that the customer reported, but you deferred it last month!"
"I can't run tests because you gave me a broken build."

Spending time arguing about whose job it is to do the work is an excellent mechanism to reduce satisfaction with your product or service.

2. Require all tests to be automated before releasing

Another device for distracting from quality is requiring test automation for every feature. It guarantees that your Test organization spends most of its time doing software development instead of test engineering.

The goal should change from validating a product attribute, to counting how many automated tests ran to verify it.

Rather than thinking through the test breakdown that validates various code paths or fundamentals of a function, engineers will spend time and resources executing multiple iterations of the same test. It leads to higher counts.

It also promotes a fragile test infrastructure and increases maintenance costs.

Since there's no time to consider test strategy and abstractions, they're forced to write very specific - yet brittle - workflows. Whenever Development makes slight changes to the feature, the tests break.

The Test organization must focus on producing a large number of tests as fast as possible. Otherwise, they'll have time to push more value into the product with more meaningful tests.

Make sure to reward this behavior by promoting engineers that produce the highest number of automated tests that run frequently. But don't forget about those that save time by writing scripts that execute other scripts while hard-coding all the arguments so the user won't have to type them.

3. Require 100% code coverage

Since product quality and customer experience are only correlated to code coverage, forcing your Test and Dev teams to chase 100% coverage is another tactic.

Make sure you direct all execution conversations towards the coverage measurement and hold teams accountable for it. People will focus on building tests that call all code functions, regardless of their output.

A nice bonus is that measuring coverage tends to affect timing or performance. It makes it hard for anyone to understand the actual customer experience, thereby adding to your goal of decreased satisfaction.

Make sure that tool implementations are automated, and that every report includes the coverage numbers, so it's easy to bring up in conversations.

Avoid focusing on code branching coverage. It's essential to prove that you tested every function, not every code path in the function. After all, there's no point in planning for customers to hit failure cases. They only happen a small percentage of the time.

Just like the previous section, don't forget to reward the team for being as close as possible to the 100% coverage number. It goes a long way in showing your engineers the behavior that matters.

4. Isolate the Test organization from Development

One of the worst things in software development is when the Dev team is so in-sync with Test that they finish each other's sentences. It's a red flag! It means that you're on a path for producing a quality product or service.

Reduce all chances for collaboration, including physical proximity.

Make sure to separate the teams, so it takes physical effort for a Test engineer to walk over to a Developer and ask questions. Human nature will take hold and solve the rest of the problem for you.

When both of these organizations actively collaborate, they tend to help each other considerably. You'll find the Developer gives the Tester a heads-up about new changes. Alternatively, the opposite could happen, where a Developer will know about a bug before it's reported. Not only does this break tribalism, but it sidetracks well-defined responsibilities.

Collaboration leads to less process:

In some cases, issues are just fixed instead of reported.
Internal documentation and knowledge transfers receive less time and attention.
Test engineers become more knowledgeable about the product.
Release schedules start to reflect reality during planning.

All of these lead to higher quality and better customer satisfaction! The direct opposite of your goals.

Enforcing more process is a great way of discouraging this behavior. It helps highlight the requirements that an engineer doesn't meet when directly engaging with other teams. It can come in the form of extra checkpoints and meetings, but preferably, more documentation.

Another excellent tool for increasing isolation is access controls. Make sure that Test does not have access to source code and that Dev cannot see the procedures of a test case.

I find that separating tests from code is a must! If you have both in the same repository, then Developers will know ahead of time when they break a test. A process shortcut that leads to the very efficiencies we don't want.

5. Measure the success of the process, not the product

I can't tell you how many conversations I've had with coworkers that keep asking impertinent questions about customer adoption.

Things like: How many customers are using that feature? What's the potential impact to our install base? How many steps must the user go through to use this feature? What's the attach rate? How big is our potential customer pool for that requirement? How many users asked for this function?

These folks don't get it. It's imperative to discourage this behavior as it happens. It tends to spread to other team members, and pretty soon, you'll have an organization that only cares about quality.

The best thing to do is redirect that energy by asking questions about the process instead. The items below are some stats that help accomplish this.

Number of open issues and who opened them

It helps prove how well the Test team is doing and how much work they're putting in.

Don't forget to encourage the employees that opened the highest number of issues. Bring it up in meetings often, so the rest of the folks are not surprised during performance reviews.

Some will try to argue that too many open issues lead to significant overhead in managing them. Again, that's the whole point.

Refocus team effort away from quality and towards process. Plus, if there are too few issues, then the Test org looks like their not working.

Amount of time an issue spent waiting for Test response

It's trickier to measure, but when you can pull it off, it's a great way to show how well the Test org is doing when compared to Development. It encourages behaviors that help maximize the active issue count.

To help the situation, try to tie issue status changes to process requirements that enforce particular field values for specific situations.

It reduces the time an issue spends on the Test side whenever parameters don't match because the engineer will have to ask for the correction.

Development is usually too busy to care about this stuff. So it also leads to lots of political discussions in project management meetings and encourages more tribalism.

Percentage of issues in a closed state

Before a release, you want to encourage the team to close all issues. It's best to base the performance review of Development engineers on how many issues were open against their code at release time.

It guarantees fewer defects opened by the Development team - further boosting your Test performance. It also discourages communication with the Test organization and increases friction.

Percentage of tests executed compared to an execution plan

Don't forget to make an execution curve telling the team how to spend their time throughout the test cycle. Be careful not to base it on history, or you may find that the org meets the numbers and improves quality.

Check the graph in every status meeting and ask for progress regardless of the state of the product. It forces the inclusion of tests meant to always complete just to pad the numbers and meet expectations.

Discourage anyone that beats projections. Doing so makes it look like the Test org has free time.

Measure the pass rate

You'll want to track the percent of test execution that's passing vs. failing for all the tests put together.

Make sure you explain to the team the required pass rate to reach a release stage. It reminds them to design suites with enough tests that always pass so that you can meet process requirements.

6. Require granular projections from engineers

Getting a schedule from Dev is hard. Many times they try to throw reality at the problem as if complexity mattered. The truth is you'll release by the date designated by the Sales organization regardless, but Dev never seems to understand that.

Whenever discussing code completion dates, make sure to ask for the day, not the week or the month when they'll complete. If you disagree with that date, bring it up in a public forum of all their peers so they can argue it.

Doing so accomplishes several essential points in lowering quality:

Developers fight among themselves, reducing communication.
They stop thinking they have a say in how to spend their days or do their jobs, which increases turnover.
It makes it unlikely that they meet their deadlines, which lowers their performance review and also increases turnover.

Achieving high turnover is like the holy grail of lowering quality. It helps reduce the bottom line with fewer raises and promotions. It increases disinformation, emphasizing individualism. Finally, it removes any ownership that might build up in the team over time.

7. Reward quick patching instead of solving

Given enough time, engineers can solve almost every problem. But when you do the opposite, an interesting phenomenon occurs: instead of resolving the root issue, they fix the symptoms, sometimes ignoring the problem completely.

Here are some of the benefits you'll see when using this technique:

Development gets praise for being quick.
The base problem isn't solved, so there's a bunch of opportunities for writing future bugs, making the Test team and the process look better.
The customer will keep running into the same issues, which translates to lower satisfaction.
The patches for each symptom develop a spiderweb of dependencies, making future work harder and more brittle, which translates to lower quality.
You'll gain issues for everyone you fix at a viral pace. The more problems to fix, the more process you need to track the ones left behind. Win-win!

Rewards are fundamental here as well. Make sure to incentivize this type of work with awards, and broadcast them to the team so everyone is aware of the behavior you want.

8. Plan for today instead of tomorrow

There's always someone trying to foresee what tomorrow brings. These days some engineers will do statistical analysis or machine learning. Formulating an algorithm that explains how well execution is going, which issues are important, how much time is actually needed, the number of resources required to test, etc.

Then there are the folks that have been around long enough and keep bringing up past mistakes or wasted efforts you should avoid.

Ignore that advice and always plan for the best case today. It doesn't matter if technical debt is high, if the 3rd party vendor has low quality, or if everyone planned a vacation on the same day during the next release.

Deal with problems when they occur, not before. Otherwise, efficiency will increase.

9. Conclusions

While there are many more points to cover, it seems our fictional Sales team decided to ship before completing the article!

On a more serious note, test engineering and product validation gets exponentially harder with complex systems and large organizations. It's important to incentivize the right type of behaviors in order to succeed. However, those aren't always obvious and in some cases, they're even counter-intuitive.

I do hope the article helped you observe the test world from a different perspective. One that provides some helpful insight. I find this practice of aiming for bad outcomes quite illuminating sometimes. At the very least, it's loads of fun to think through.

If you liked the article and want to read more about development best practices from Cristian Medina and others, please visit tryexceptpass.org. Stay informed with their latest content by subscribing to the mailing list.

Good enough engineering to start an Internet company

freeCodeCamp — Fri, 02 Aug 2019 18:28:06 +0000

By Wenbin Fang

I gave a guest lecture in an undergraduate software engineering class (CSCE431) at Texas A&M University in March 2019. Now I’ve turned this lecture into a blog post here, and hopefully some people on the Internet will find this useful.

If you arrived from a Google search, here are some contexts:

I’m running a small Internet company — Listen Notes, Inc. — with only one full-time employee (me), as of August 2, 2019. We built a podcast search engine website ListenNotes.com and a podcast API.

I’ll share with you my experience about starting an internet company. Building an internet product is not like building an iPhone or a pyramid. Your product doesn’t need to be perfect at the beginning. If you are building something useful, other people will tell you what to do next. And you’ll figure out what’s next. Generally, you should be comfortable dealing with uncertainty, if you want to start your own company.

The first version of Facebook was launched in early February 2004, which was an undergraduate student’s mere four weeks worth of work. It was a good enough product with good enough engineering. Every Computer Science college graduate nowadays should be able to build the first version of Facebook over a weekend, using a modern web programming framework (e.g., Rails, Django…).

Companies such as Google, Snapchat, Spotify, Amazon, Twitter, and others are all great internet companies in our generation.

However, I can’t tell you how to become a company as successful as them. I’m not there yet. These companies are so successful and so big. For example, Google has 100,000 employees, as of March 2019. But there was a moment in the history of time when Google had only two employees. You have to start from somewhere or nowhere. I can talk about how to get started.

Hopefully after this lesson, you’ll feel that starting an internet company is not hard at all. And you’ll learn about some tools and services that you can use in your future projects.

You’ll hear me saying “there’s a tool” a lot. This is what I mean by “good enough engineering.” It’s 2019 now. It’s unlikely that you are the first person to encounter a fundamentally new problem. There must be tools and services out there that can help you solve problems — oftentimes, for free!

Before we delve further, let me provide a brief introduction of Listen Notes.

Listen Notes is a podcast search engine website. You type in a keyword and you search the whole internet’s podcasts. It’s as simple as Google :) But simple doesn’t mean easy. We also built a lot of tools on top of the core search engine to help people discover & enjoy podcasts.

https://www.listennotes.com/

Let’s take a step back to talk about podcasts. A podcast is a type of media format. Some people call it “on-demand radio.” We consume tons of media content everyday. We watch YouTube videos, watch TVs, read books, listen to music, and listen to podcasts. We consume media content because we want to get information, gain knowledge, and be entertained. Nowadays you can literally learn any topics by listening to podcasts. You can listen to podcasts while your eyes & hands are busy (e.g., driving, working out, walking, doing housework…).

In early 2017, I found myself consuming more information from podcasts than from other media formats (e.g., TVs, books…). I needed a podcast search engine that I could use to search & find individual episodes to binge listen to. In hindsight, a podcast search engine should be a very straightforward thing to exist. But I couldn’t find a good one. So I spent less than one week building a prototype of Listen Notes, the podcast search engine birthed out of my own wishes and necessities.

Very first version of Listen Notes. Credits: Lifehacker

I launched the prototype and used it a lot myself. But I didn’t touch the code for about nine months, until I decided to work on it full-time after I left my first failed startup. That was September 2017. Cut to 1.5 years later, I’m still having fun working on Listen Notes :)

Do you need to know where to find an office? Yes, there’s a service for that. I use WeWork. I’ve got a small office inside WeWork in San Francisco. The office is not cheap, but I think it’s a good investment for productivity.

I can choose to spend money being 200% more productive myself, or spend money hiring one more employee. I can choose to save money and waste more time, or to save time and make more money. If you were me, what would your choice be?

By starting an internet business, we need to have a formal company, a legal entity. There’s a service for that.

I used Stripe Atlas to incorporate Listen Notes, Inc. I paid $500, and I got a Delaware C Corp within 10 days. Stripe Atlas provides some nice perks, e.g., $5k AWS (Amazon Web Services) credits. But to keep the company around, I have to pay ~$2k/year for taxes, accounting, and other stuff. This gives you a basic idea of the minimum cost of running a company.

Because we are a company now, we have to deal with some legal stuff. There’s a service for that.

You can use Clerky to generate legal documents or use UpCounsel to get a lawyer. I’ve used both services. They aren’t perfect, but they worth the money. You can’t expect to get Ritz-Carlton-level services with a McDonald’s price, right? If you want to be happy, you’d better set the correct level of expectation.

To get started, you just need a good enough domain name — a $10/year .com domain. You can always buy a great domain name later.

For example, Dropbox used getdropbox.com for more than 2 years before they bought dropbox.com for $300k. How did I discover that kind of trivia? I listen to podcasts! There was a podcast interview of Drew Houston(Dropbox co-founder & CEO), where he talked about how they secured the dropbox.com domain name.

After you get a domain name, make sure you also create a bunch of company accounts on social networking sites such as Twitter, Facebook, Instagram, and (maybe) Snapchat.

And most companies just use G Suite for their company email, which is basically Gmail.

Internet companies build online services. Most software today is online services. If you can’t access the Internet, you can’t use most apps on your phone.

Online services typically follow the client/server model. The client side software sends requests to the server side and gets responses to render the UI or to perform certain tasks.

All websites are online services, obviously. We use web browsers to access websites. To some degree, mobile apps are specialized browsers.

On the server side, we run servers. You just need good enough servers to get started. By “servers” I mean VPS (Virtual Private Servers), which basically provides root access to an IP address. Once you ssh into an IP address (a VPS), you can do whatever you want, e.g., install software & put your code there. And you are now open for business.

For VPS, I recommend using something simple at the beginning, e.g., DigitalOcean or AWS Lightsail. At Listen Notes, we used DigitalOcean for about one year because it’s cheap & easy to set up, then we switched to AWS EC2 when our website got more traffic and we needed more flexibility & “production”-ish setup.

Such backend architecture is pretty common for running an online service.

The client side (e.g., browsers) sends requests. The load balancer distributes requests evenly to web servers. We typically run a lot of web servers, where the server side code is running (e.g., Rails, Django…). We need a datastore to store our data. Web servers interact with the datastore to read and write data.

On the left hand side, it’s synchronous processing. Here comes a request, and the web server processes it and returns a response right away. It’s synchronous.

We need asynchronous processing as well to handle non-urgent, long-running, or compute-intensive tasks. We don’t want to consume compute resources for such tasks on the web server. So we typically offload such tasks for the workers to process. Web servers place messages in the message queue, and workers pick up messages to process the tasks.

For example, we generate this kind of image for search results on Listen Notes (Example):

This type of image-generation task is kind of compute intensive and is not urgent. So we offload it for workers to process, instead of handling them on the web servers.

Finally, there must be a Scheduler for time-based scheduling jobs. Many companies just use cron jobs on Linux and they switch to something else when they become bigger (e.g., I built a Scheduler system for my former employer Nextdoor a few years ago to replace cron jobs). For Listen Notes, we have a lot of time-based scheduling jobs, e.g., sending Listen Alerts.

We use Nginx as a simple load balancer. The backend code is mostly in Python/Django. We use different datastores for different purposes. We use Postgres for the main datastore, which is our single source of truth. We are a search engine, so we use Elasticsearch. We use Redis for a lot of things, but mostly for caching & implementing some “fast” features like Listen Real-Time.

On the asynchronous processing land, we use Celery, Celery Beat, and RabbitMQ.

We have to keep the server-side processes up and running 24/7. We’d better use some kind of process manager to automatically restart processes if they crash. We use Supervisor a lot at Listen Notes.

Two recommendations here:

Learn tech stacks of real companies on stackshare.io
Listen to Software Engineering Daily. They interview engineers from real companies, so you can learn how companies do engineering.

As end users, we get tons of notifications from online services via email, SMS, and push notifications. When an Uber driver is approaching, we get push notifications. When we shop on Amazon, we get email notifications (typically with receipts). When our bank accounts experience problems, we get SMS notifications.

Let’s turn the table. When we build online services, how do we send notifications to users? There’s a service or API for each notification channel, e.g., SendGrid or Amazon SES for email, Twilio for SMS…

Next, we need some kind of triggers to initiate notifications.

One type of trigger is from user actions. For example, a user can invite people to contribute to the same playlist on Listen Later. When he or she clicks the invite button, it triggers an email notification that is sent to the potential contributor. So the web server places a message in the message queue, and one of the workers picks up the message and sends the invite email later.

Another type of trigger is from time-based schedules, e.g., send emails to these 500 people at 7 a.m. every morning.

Each of the blocks in the architecture diagram represents a process or multiple processes on the operating systems. We can run those processes on one server or multiple servers.

We typically create a provision of multiple sets of servers for different audiences & for different purposes.

Each set of servers runs in a separate environment.

Servers in the production environment serve live traffic. The audience is made up of real users.

Servers in the staging environment are primarily for testing. The audience is comprised of employees in the company. We need the staging environment to manually test product features before releasing the whole shebang to production.

And we need a dev environment for development purposes, which is typically used by a single developer.

For Listen Notes, we use Vagrant & VirtualBox to set up a virtual machine. And we run everything inside this virtual machine.

Since the backend code of Listen Notes is primarily written in Python/Django, I use PyCharm to write code. I know, it’s not VS Code or whatever cool text editors others use. But I’m 1000x more productive using PyCharm than using other text editors - though, I was a Vim user for 5 years and an Emacs user for another 6 years :) It’s like some people like spicy food, while others don’t. We can’t blame people who don’t like spicy food, right? Don’t get involved in the religious war of IDEs, languages, technologies… GETTING THINGS DONE™ is more important.

In terms of front-end engineering, I have very little to share here. Listen Notes has only a website. We don’t have native apps (except for an experimental Just Listen app).

For the web front-end, I use the conventional Reactjs & Bootstrap. Pretty standard nowadays.

If you just get started with your projects, I would suggest focusing on a single platform first. Don’t go cross-platform too early. We typically have very limited resources at the very beginning. Look at Instagram: When they were an independent company, they had only an iOS app initially. And they got acquired for $1B.

If you are building a website, make sure you make it responsive from day 1. You can easily use modern browsers’ developer tools to test different screen sizes (e.g., on Chrome).

You also want to test on multiple operating systems & browsers. There’s a service for that: Use BrowserStack.

If you have more than one server, you’d better NOT install software & do configuration manually. Infrastructure as code is a common practice in internet companies nowadays. Basically, we codify the specification of servers and automate the server configuration.

For Listen Notes, we use Ansible. We need to write a bunch of yml configuration files to specify what software to install & where to put config files. When we run ansible-playbook in command line, it’ll automatically configure multiple servers for us.

Nowadays, servers are elastic or even ephemeral, with on-demand configurations to suit the necessary workload. Servers come and go. You may run more servers during the daytime, when there’s more traffic; and run fewer servers at night, when there’s less traffic. Infrastructure as code makes this type of thing easy.

Internet companies deploy code pretty frequently nowadays, at least once per week or even many times a day. Some companies do continuous deployment, shipping code whenever there’s a new git commit.

For Listen Notes, I’ve got a quick script to deploy code, where I can specify the deployment environment, server type, and git commit SHA as parameters. So I can push a button and deploy a specific version of code (e.g., HEAD, or any git commit) to specific servers (e.g., web, API, worker…) in a specific environment (e.g., production, staging…).

We deploy new code, but we don’t necessarily release new features. Nowadays we do feature toggles in the code, which is basically some if-else statements. We can hide new code behind an if statement, and we use the feature toggle variable to control whether or not to execute the new code. We typically store the feature toggle variable in some kind of datastore, e.g., Redis. We can go very fancy here. We can turn on the new feature to 10% of users first, then 20%, then 50%, then 100%.

Online services need to be up 24/7. So it’s important to have tools for monitoring & alerting.

There’s a service for that!

Many companies use Datadog. We use Datadog at Listen Notes as well. We can easily build monitoring graphs to provide great visibility for servers and applications. We can also set up alerts when some metrics are abnormal (e.g., higher than a certain threshold).

If you want to learn the best practices of building & operating online services, read The Twelve-Factor App.

When it comes to internal tools, I see this iceberg image. Internal tools are a big chunk of code like an iceberg lurking under the water, which is invisible to outsiders.

If you have never operated a popular online service before, you won’t be aware that you need to build a lot of tools to use internally (by yourself, by your employees).

Different companies build different internal tools for various purposes. So far, I’ve built some internal tools to help development (e.g., preview email notification without actually sending emails), to provide God’s view (e.g., see search queries, quickly pull info for a particular user…), and to fight bad actors (e.g., content moderation, detect spam…).

So by this point, we know how to start a company with $500 and we know how to build an online service with good enough engineering.

Profit!

Oh, wait….

How do people find the thing you have built? How do you make money?

Pinterest CEO says key to success was marketing, not engineering. Well, this is so true.

Engineering is deterministic. Marketing and business are non-deterministic.If you want to build an online service, you can build it. But we live in a noisy world now. Tons of things are competing for our attention. Marketing is super hard.

I’m not an expert of marketing. I’m still figuring out how to do better marketing myself…

There are multiple channels that you can use to get your messages out. Try them all. Find the most effective one. Double down on that one. A recommendation here: The 19 Channels You Can Use to Get Traction

If you want to do SEO, you can find some good tutorials on the Internet. But generally, you want to make your website as fast as possible. Google prefers fast websites nowadays.

The best SEO document is from Google itself.

And if you are curious about a website’s traffic, just use SimilarWeb’s chrome extension. The number is not 100% accurate, but should be in the same order of magnitude :)

An Internet company makes money by selling eyeballs to advertisers or selling goods/services directly to users.

At Listen Notes, we do both. We run ads with a combination of Carbon Ads& direct sales (managed via Google Ad Manager). We also sell API to developers.

You may ask me why I don’t use XYZ technologies. Well, I’m a practical person. The goal is to get things done, instead of doing tech for the sake of doing tech.

In particular, I prefer boring technologies, which typically have existed for many years or even decades. Check out this blog post for Listen Notes tech stack.

Software engineering nowadays is mostly Google and StackOverflow-driven :) If you need help, you can find more information for old & mature technologies from Google & StackOverflow — but like many things in our lives, this is not always true.

I need to tell you bad news: It’s impossible for you to come up with a 100% original startup ideas nowadays. If you think your idea is unique and original, then it’s more likely that you don’t read enough books or don’t listen to enough podcasts :)

When I mentioned “there’s a tool/service for that,” it’s a startup itself. You can borrow their ideas, and build a better version. Or you can tackle similar problems from a different angle.

Oh, and here’s a video about Facebook’s tech in 2005: https://www.youtube.com/watch?v=xFFs9UgOAlE

Okay. You can start a company TODAY!

Eighty Percent of Success Is Showing Up

You can find the deck here: http://bit.ly/good-enough-tech

And you can always talk to me asynchronously via email :)

If you like podcasts, try Listen Notes.

If you want to build a podcast app, try Listen API.

Why career changers have an advantage in the tech world

Zubin Pratap — Wed, 03 Jul 2019 11:12:00 +0000

I am no stranger to changing careers. I started 16 years ago as a litigating lawyer, then moved to corporate law, and after 11 years of practice, left the law and went into middle management. Then I started my own startup. As my startup struggled to attain sustainability, I discovered a love for coding (largely out of necessity, in startup-land!). I set crazy learning goals, and switched my attention from my startup to code (mainly to become my own technical co-founder). And then I figured that since I love to code, and I love technology, I might as well become a dev.

Looking back I see how I've grown exponentially with each career change. But here's the unexpected professional advantage : each new 'job' benefited enormously from my previous experience, even though most people (especially non-career changers) thought it was a bad idea. As it turns out all the people who had strong views about it had never done it themselves.

Career changers have some very significant advantages that are often overlooked, and almost always underplayed

Right after I decided to try and land a dev job late last year, I also decided to document the strategies and plans that actually worked for me when I taught myself to code - by publishing a course on Udemy targeted at people (especially those that long to get into a tech role) who want to teach themselves this vital skill of the new world. As part of that process, and based on feedback from a number of colleagues and recruiters for dev roles, I realised that us career changers have some very significant advantages that are often overlooked, and almost always underplayed.

This is part 1 of 2. In this part I will cover the big advantages of having one or more previous careers offers when you switch to a career in dev. In part 2 I cover strategies that make you more compelling if you're worried about how your previous career will look when you interview for dev roles.

Experience

OK, so this is a no-brainer. But a shallow analysis of this will only mean that you don't actually leverage this asset properly. Why is experience in an unrelated, or semi-related field of any use? Most people would tell you that you need direct and "relevant" experience. And that is definitely the case when it comes to satisfying the specifics of the Job Description. But anyone that has actually reflected on their role will acknowledge that the real experience of the job is often quite different from the sanitized words in a JD. The difference is the same difference as a colouring in book before you colour it in and after. So here are the main reasons why experience really does matter, even if it is an unrelated field (potentially even more so then as it gives you depth and breadth of life experience).

Maturity. Dealing with teams teaches you soft skills, hard skills, and subconscious skills at dealing with personalities, temperaments, cultures, habits and mindsets.
Insight. other roles gives you insight into other functions in an organisation. For example, if you're implementing a billing system you will have a better awareness of critical aspects of the user's experience if you understand how a sales person uses billing data to manage the sales pipelines and increase customer acquisition or reduce churn. If you were previously in sales or marketing or had your own startup, you would really have valuable skills when it comes to designing the system and selecting SaaS products.
Context. Have worked in another team, in another role, in another organisation also helps you contextualise the drivers, habits, behaviours and motivations that influence how teams interact. Product, engineering, marketing, finance, they all have drivers and unique pressures and motives. Truly understanding the context of your co-workers and collaborators outside your narrow function makes you hugely valuable because it helps you cooperate better, which in turn encourages others to cooperate with you. No matter how skilled a dev you are, if you annoy the heck out of the folks who are driving revenue, you will struggle to be taken seriously, and that will impact the quality of your work life.
Cultural adaptiveness. The ability to understand context, have insight, and deal with coworkers maturely adds up to an overall skills that I call your cultural adaptability. This just isn't about different ethnic, regional or anthropological cultures. This is about team-cultures too, the unique dynamics that an organisation's culture produces in its teams. Being adaptive in this way makes you highly effective, and effective team players are more valuable as employees than talented but ineffective members. Being great at your role is much, much more than pure technical skill or intellectual horse power.
Communication. While a lot is written about improving communication in the engineering culture, the fact is that relatively poor communicators can still be highly effective if they have a broadened vocabulary in the context of their business. You may not have to know or care what Net Present Value or EBITDA is. But understanding these things will make you a better inter-team communicator. In exactly the same way that it helps if your finance person understands what a firewall, or API call is. Knowing some of the technical language spoken by other team members is flattering to them, useful to you, and beneficial for the organisation. And since these may be very boring to you (or not), the best way to pick up the vernacular is to have been in or close to those functions in a previous life, so you know not just what they mean but why they're important to your colleague.
Prioritization. Effective team members prioritise in a way that benefits the team and also the larger organisation. If you have previous experience in other domains, you will understand how something that is a low priority to you has an outsized impact on another function, and vice versa. This sensitivity to the co-worker's perspective and professional pressures is a huge asset when it comes to building rapport, trust, camaraderies, and very important - organisational influence.
Organisational Dynamics. This is the name for its healthy form. It's unhealthy twin is known as sh!tty politics. But it's a fact of life. You do not and should not have to participate. But it really helps to be able to detect it, foresee it, recognise it, sidestep it, and handle it. On rainy days it always helps to see a car speeding towards that puddle next to you, so you can avoid being splashed, right? In all its forms, the ability to navigate and deal constructively with Organisational Dynamics is a tremendous skill as it reduces stress, improves productivity, builds trust and credibility and delivers successful outcomes for the teams involved. If you've been only in one career or domain all your life you will become adept at recognising its forms in your specific function and in your context. But recognising it when it happens elsewhere but is coming soon to a colleague near you, is a huge strategic advantage as you can anticipate and prepare for it appropriately.

Thanks for reading!

If you would like to learn more about my journey into code, check out episode 53 of the freeCodeCamp podcast, where Quincy (founder of freeCodeCamp) and I share our experiences as career changers that may help you on your journey. You can also access the podcast on iTunes, Stitcher, and Spotify.

I will also hold a few AMAs and webinars in the coming months. If this is of interest to you please let me know by going here. And of course, you can also Tweet me at @ZubinPratap.

Do it like an engineer

freeCodeCamp — Thu, 20 Jun 2019 17:01:50 +0000

By Luca Florio

The work of a Software Engineer is to solve problems. Everything can be reduced to this activity. This is why it is important to have a solid methodology to tackle problems. We are engineers after all, and we are trained to solve problems. We have to do it like an engineer.

Understand the requirements

The first step is to understand the requirements. To solve a problem, you have to understand exactly what the problem is. It happened to me that a feature that would have been a 2/3 weeks work, became 3 useless months of “Oh, there is also that…”, “Oh, we didn’t think about that!”, and “Maybe it would be better if….”. My fault. Lesson learned. When you start solving a problem, be sure to understand the starting point, the end goal, and the obstacles in between. The worst possible thing is to produce a solution that actually doesn’t do what’s expected. As a final note, remember that you and only you decide how to solve the problem. It is your job, as it is the job of who gives you the requirements of the new feature to express them the best possible way. If someone that is not an engineer tries to tell you how to solve the problem, punch him in the face. You are perfectly justified. At least by me.

Understand the size

we all agree that serving 100000 requests per second is a bit different from serving 100 requests per minute. The approach to solving the problem is different. This is why it is important to understand what is the “size of the input” or, put another way, how big the problem is. Otherwise, there are two scenarios. The best case is that you spend 6 months designing and implementing a system that is used at 10%. The worst case is when you spend 1 month designing a system that is used at 180%. If the best case is a waste of time/resources, you don’t want to find yourself in the worst case. To avoid this situation, we have to ask the right questions.

How many requests the system should satisfy?
What is the expected response time?
How many resources do we have?
What about deadlines? The right questions depend on the context, but the target is one and only one: understand the size of the problem.

Stand on the shoulders of giants…

I’ll tell you a secret. The chance someone else already solved your problem is high. Very high. All you have to do is a search in the literature to find out if there is a solution for a problem matching your use case. Avoid home-made solution to well-known problems, they only bring other problems. There are lots of companies having “their Hibernate”, “their Kafka”, etc. because:

“We have a different use case” (I want to see it)
“The performance of technology X is not enough for us” (really?)
“We can do it better” (this is the funniest ?) Bottom line: once you know your requirements and the size of the problem, do a search in the literature. There is no point in reinventing the wheel.

…But remember you are not a giant

It’s ok to build-up on existing solutions, but avoid overshooting. Remember that you are not Google. In the ocean of cool technologies out there, the most famous/innovative/used not always is the best one for you. Deploy a Kafka cluster to process 5 messages a day probably is not a good idea. Choose the technology that will do the job with the minimum complexity. This decision will pay you in the long run.

Grandma-Driven Development

Implement your solution trying to make it understandable by your grandma. Avoid fancy and super complex implementations. Put them aside in favour of a simple and understandable one. This will make the code more maintainable. Leave optimization to the moment they are necessary. More formally, your implementation should follow the Rule of Least Power. The original rule refers to the choice of the programming language. In this context, we can read it as:

“Among the available solutions, choose the least powerful one that can solve your problem.”

I learned this rule when I started using functional programming. It allows you to implement solutions with an unpaired elegance. However, such solutions sometimes are too much complex. I prefer an implementation a little less elegant and efficient, but a lot more understandable and maintainable. You will not be the only one reading your code.

Conclusion

We are engineers, our work is to solve problems, in whatever form they appear. We need to apply our engineering skills and analyze the problem pragmatically to deliver the correct solution. We need to remember that it is not the solution for producing the desired result. It is the one that does it requiring the least effort, and with the least complexity. I hope the methodology I described in this story will help you make a step toward such an achievement.

See you! ?

Books that every engineering manager should read

freeCodeCamp — Fri, 19 Apr 2019 21:47:48 +0000

By Ovidiu Bute

It’s a rare occasion that companies provide leadership training before you become a manager. A few days or weeks after what was probably one of the happiest days in your recent memory, the day you were offered a position outside of the individual contributor track, you find yourself with a million questions. You feel that you were tricked into signing something without reading the fine print.

That feeling you’re experiencing isn’t new, it’s just that you’ve all but forgotten it. It’s not knowing what you’re supposed to do. It’s being clueless. Because if you think years of writing software trained you to become a manager, research states the contrary. But it’s not the end of the world. Even though your company most likely doesn’t understand the need for formal management training, there is a plethora of information available to you that will make your job easier, and maybe even enjoyable.

When I became a manager I did what I typically do when faced with a challenge I know almost nothing about: I started reading. I’ve read a lot of books, some were good, a few of them were amazing. All of them shaped the way I do my job, and so I thought I would share them for other aspiring or active managers out there.

I’ve curated this list based on several factors:

The books should cover a broad set of engineering management and leadership topics. It’s easy to find overlapping books. It is much harder to find a diversity of information when you’re very new to leadership.
They should be written in different eras. The software industry is constantly evolving. It doesn’t make sense to only read about what was happening in the 1980s or 90s.
Reading order matters a lot. Some books are more specialized than others. The information provided can be thought of as layers that stack on top of each other. If you’re inexperienced, you may start in the middle or the end, and that will basically ruin other books for you.
Finally, I put a hard limit of 7, just because I think this list is enough to build a foundation layer on top of which you can continue reading and maybe even doing your own research further on.

But enough introduction, let’s see the list :)

Peopleware: Productive Projects and Teams, by Tom DeMarco & Tim Lister

This should be mandatory reading for everyone. Period. No, not just everyone in software, everyone working in a private company should read this book. It’s amazing to me how pretty much all of the problems that people deal with it on a daily basis have already been solved. In the 1980s. If you read just one book from this list, let it be this one.

High Output Management, by Andrew S. Grove

Soon after you make the switch from the individual contributor track to the leadership track you will ask yourself a basic question.

How do I measure my own success?

You used to be able to answer that easily. Now that you’re writing code less and less, and dealing with team dynamics and people problems, how will you or your boss measure your progress and success? This book provides a now universally accepted answer: you are measured through the success of the people reporting to you. That is to say, if your team is successful, then you’re successful. I’m over-simplifying of course but that was my biggest take away from this book.

There are more insightful bits of information such as a walk-through of a production pipeline, how to run meetings, or how to conduct one on ones. This is a classic book that everyone should read even if they’re not interested in leadership roles. It’s that good.

I will say though that this book was written a long time ago. The impression I got was that leaders from that age were more authoritative than what you see in companies these days. This is not a critique, it’s a fair warning. You may read a few chapters and conclude that there’s no way anyone could get away with doing that in a company. There’s a certain nuance to the leadership style described in this book and it is important to understand it.

Influence: Science and Practice, by Robert B. Cialdini

An engineering manager’s job is to ensure their team has everything it needs to succeed. This means managing the interaction between multiple groups of people towards an agreeable outcome.

If you’ve ever tried to convince a friend to move from WhatsApp to Telegram and failed, you’ve made an attempt at exerting influence. You’ll need to do that basically every day, and in my experience, this is a very hard skill to learn. It takes a lot of practice, and there’s really no sandbox mode in which you can fail and it’ll be OK. You will try to talk to someone at some point into doing something, and you will fail, and you or your team will suffer for it.

This book is the definitive guide on how to approach the problem scientifically. A lot of managers seem to think they don’t need to learn how to influence others, particularly their direct reports, as rank is the ultimate influencer. Thinking that will keep you from ever becoming a great leader, in my opinion. Yes, you will probably overrule someone at some point and it will feel terrific while you do it. But if that person ends up hating you for it, you’ve just lost their trust and you will see the consequences of that later on.

Rapid Development: Taming Wild Software Schedules, by Steve McConnell

This is another classic textbook that I wish everyone in software would read. If you’re currently in a company who’s agile and are struggling with unproductive meetings, poor quality code, team members that can’t get along with each other, or stakeholders who push you to leave the office late at night, you will find at least one solution within this book. It’s also a good leadership book as it describes, in what is probably my favorite chapter, a list of things you definitely should not do if you want to have a great team. I still refer to that chapter from time to time, it’s just really good and insightful.

Managing Humans, by Michael Lopp

I loved this book for its humor and insights into some of the biggest tech companies’ cultures. It doesn’t feel like a textbook compared to the others on this list, but that’s what makes it a great fit.

You need to understand that leaders are still human, they’re going to screw up, and the end result may be tragic or hilarious. Or both. That’s a very important lesson to learn, one that took me the longest time, to be honest.

It’s very easy to get caught up in theory while trying to learn how to be a leader. But reality often is so insane that no book can offer you the solution you need. You will end up in situations where you will do everything by the book, and people will still be unhappy. And that’s OK. The stories from Managing Humans will help you understand that.

The Manager’s Path, by Camille Fournier

What makes this book awesome is that it offers a clear and simple description of your responsibilities and goals at every step in the leadership ladder. Most companies don’t even have that information for their own leadership track.

It starts by describing what an entry-level leader should focus on, such as mentoring junior developers. It moves on to engineering managers, senior managers, and even covers what a VP of software should do. I personally have always wondered what a day in a VP’s shoes looks like, and if you have as well, this book is for you.

Beyond just taming your curiosity of what the upper echelons look like, I love this book because it is so realistic. There’s no utopian view of teams and processes here. It’s all ground in the experience of the author. That makes the information within highly applicable in your work environment.

The one minute manager, by Kenneth H. Blanchard

I initially didn’t want to read this after I found it, but I’m glad I changed my mind about it. It’s basically a short story about a manager who is very good at his job, particularly in one area.

I won’t spoil it for you. The information you get out of this story isn’t particularly innovative. There are other books that dive into its topic in more detail. But there’s something really satisfying about it that I can’t really explain. It’s like a GitHub gist that solves a problem in under 100 lines of code.

You can be dismissive of it — there’s no way it could be that simple, right? Or you can read through it carefully and appreciate its simplicity, knowing that it’s not a generalized solution, but still good enough that it’s worth having in your toolbox. It is entirely possible you’ll read this book and think I’m exaggerating, but hey, it’s my list ;)

Radical Candor, by Kim Scott

I can count to 7 just fine, but I really wanted to offer this book as sort of a special addition to my list. I wouldn’t say it’s mandatory reading. Once you’ve gone through all the books above, you should pick this up for a very striking alternative view.

Basically, Radical Candor is a framework on how to relate to people. It’s not really related to software development. You can apply this anywhere, with your friends, with your family, it’s frankly very abstract in that regard.

The author, Kim Scott, has worked both at Google and Apple. She recalls some of her experiences while working there. I found those chapters to be extremely enjoyable, but do take them with a grain of salt. Some of the more recent reports in the media directly contradict some stories from this book. This is why I’m hesitant to recommend it to everyone.

The framework itself makes sense but it’s so hard to actually put in practice. I know that’s not the author’s fault. I wish the world would work in the way she describes, but I don’t think it ever will. Nonetheless striving to achieve even 50% of Radical Candor will make you a better leader. If you have the stomach for 100% then I’d like to meet you in person.

Is that it?

This list is a solid foundation for newbie managers or people who are thinking of becoming team leads or managers. There is so much more to learn. I would’ve appreciated a simple and short list like this myself when I made the step towards leadership. Hopefully you do as well.

How I navigated the job hunt and landed my dream job

freeCodeCamp — Fri, 05 Apr 2019 21:12:00 +0000

By Julius Zerwick

Photo by Unsplash

This article is about how I went through my job hunt for a full time position as a software engineer in New York City and ended up with my dream job. I had spent two years building my skills and had aspirations to work at a top tech company with a high bar for engineering excellence.

I knew this goal would not be easy to achieve, especially without a CS degree, barely having any network in New York City, and having to compete against engineers from top schools. It would be a long shot, and I was prepared for the long haul.

9 weeks later…

I’m ecstatic to say that I’ve accepted an offer from DigitalOcean to work as a Software Engineer II from their NYC HQ!

This success came after countless study sessions, mock interviews, hours of practice, and facing numerous challenges & rejections. But I kept my spirits up and came out the other end happier than I imagined possible.

How did you go from start to finish?

That question is what this article is all about! I’ll be detailing my preparation process for interviews, how I built my network, and managed myself during this job hunt.

That being said, everyone’s experiences are different and should pull from this article what they feel will benefit them. No two job hunts are the same and there are different factors in each one.

With that said, let’s dive in!

First things first: build a solid foundation

Photo by [Unsplash](https://unsplash.com/photos/vII7qKAk-9A?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText" rel="noopener">NESA by Makers on Launch School which offers an extensive curriculum and rigorous assessment process that ensured that I had deeply learned the fundamentals before progressing.

Build a project that impresses employers & engineers

SpaceCraft demo

In the current day, there are multitudes of bootcamps and online tutorials that provide aspiring software developers with projects to help them land a job.

However, there is now a saturation in the market of developers with simple CRUD apps or clones of popular apps like Instagram, Netflix, Reddit, etc. These projects no longer impress employers or other engineers as they once did.

That said, many engineers need some way to evaluate you before giving you a referral and there is no replacement for showing your skills off with a solid project.

That’s why Gooi, Nick, and I built SpaceCraft, a real-time collaborative REPL that allows developers to write and execute code in the browser for Ruby, JavaScript, and Python.

It was an incredible experience, and also a TON of work. You can read more about how we built SpaceCraft in this article or read our case study.

We spent hundreds of hours researching how to build a REPL, enable real-time collaboration, use Docker to create isolated user sessions, and handle user requests with a proxy server.

Market yourself effectively

https://livestream.com/accounts/686369/events/8535115/videos/186248334/player?autoPlay=true&height=360&mute=false&referrer=https:%2F%2Fmedium.freecodecamp.org%2Fmedia%2F6caa6cbadb228f40bd5697c72bc7aad1%3FpostId%3Db37bf3d0d630&width=640

While SpaceCraft may showcase our skills, I still needed to get people to look at it. Since I had relocated to NYC for my job hunt, my professional network was close to nil and I did my best to put myself in front of as many engineers as possible.

That’s why our team wrote our case study, an article on HackerNoon, and presented at Meetups. These efforts helped us stand out among the sea of other applicants and generated interest from employers & engineers.

Presenting at Meetups were a game changer for me as they got my name out there and procured referrals to several companies, along with giving me the chance to practice and refine my communication skills.

I also built a personal website and beefed up my LinkedIn & AngelList profiles in order to make it easier for prospective companies to find me and see my work.

Devoting time to market yourself and skill set through writing articles, presenting at events, and professional networking sites can lead to a big payoff in the interview process.

I had a big increase in the number of engineers and companies reaching out to me after seeing my updated profiles and hearing me present, and having a video of my presentation to include in my outreach emails resulted in a higher response rate and more interviews.

Reach out to engineers & companies

In the same vein, I made sure to consistently reach out to engineers at companies I was interested in. Using LinkedIn Connect requests, I had over a dozen coffee meetings with engineers from DigitalOcean, Cockroach Labs, Oscar Health, DataDog, Peloton, and more. These meetings helped me get a feel for the job market in NYC, hear an insider perspective on company culture and teams, and gain referrals.

My advice is to approach these meetings casually and with interest in your fellow engineer’s work, career, and company. Don’t go in asking for a referral at the beginning, many times they will offer to refer you if you’re nice and the conversation goes well.

Another form of outreach that I did was more directly emailing engineers. Whenever I found a job posting that I really liked, I would reach out to an engineer who works at the company through LinkedIn Inmail, using Clearbit Connect when needed to find contact info.

In the email, I would include an introduction on my background, my most impressive project, a link to one of my presentations, and ask if they were open to chatting about the position.

This form of outreach lead to my highest response rate out of any other avenue with nearly 1/3 of all my emails getting a response. Remember, the job hunt is entirely a numbers game and it’s best to try out all forms of outreach to see which nets you the highest returns.

Study, practice, repeat

Photo by Unsplash

That said, none of these efforts would go anywhere if I didn’t have the skills to succeed in the technical interviews. With a solid foundation in the fundamentals, I needed to spend my time solving coding challenges and studying topic relevant to the positions.

The types of questions and challenges you can face in interviews are VAST and often times you won’t know what to expect. Many of the top companies will focus on algorithms and data structures, while startups and mid-size companies may focus on more “practical” problems like refactoring an existing class or adding features to a basic project.

My advice is to study a variety of coding challenges and do your best to predict the questions they will ask you. For algorithms and data structures, my main resources for building a foundation and conceptual understanding were:

A Common-Sense Guide to Data Structures and Algorithms

Cracking the Coding Interview

BaseCS

I also solved 2–3 problems a day on LeetCode for 5–6 days a week.

Many companies will also have a system design interview to assess what level to hire you for. If you’re aiming for mid-to-senior level roles, you’ll need to devote more time to this area.

I found the best resources to be Grokking the System Design Interview and reading various blogs on specific topics. Here are a few blogs I’d recommend:

All Things Distributed

High Scalability

Gainlo

BaseDS

Another very common step is to complete a take home challenge. The variety of these challenges can make it hard to prepare for, but there are some common topics. Several that I saw were:

Build a RESTful API that handles several provided cURL requests

Build a CLI app that stores inputs (like albums, artists, and release year) and allows users to retrieve them

Build a game using React/Rails (ex: Tic-Tac-Toe, Blackjack, Minesweeper)

Given a spec and example of the finished product, create a web page that is as close to pixel perfect to the example as possible.

The upside is that you have more time to complete them in the comfort of your own home, and there are lots of opportunities to impress by writing a comprehensive test suite, identifying & addressing edge cases, and finding extra touches to add.

Over my job hunt, I typically spent Monday-Thursday applying to companies, doing outreach, and solving LeetCode challenges or studying system design. I reserved Friday-Saturday to practice building a take home project. Here are some to consider:

https://x-team.com/blog/how-to-create-a-ruby-api-with-sinatra/

https://www.discoverdev.io/blog/series/js30/

https://daveceddia.com/react-practice-projects/

https://medium.freecodecamp.org/how-to-build-a-react-js-chat-app-in-10-minutes-c9233794642b

https://codeburst.io/writing-a-crud-app-with-node-js-and-mongodb-e0827cbbdafb

https://codeburst.io/learning-react-js-by-building-a-minesweeper-game-ced9d41560ed

https://github.com/karan/Projects

To prepare for live coding challenges, I used Pramp for mock interviews 3–4 times a week, which improved my performances by leaps and bounds.

I’ve written before on this topic in Why You Need an Interview Script, but essentially when under pressure our mental abilities can suffer due to the feeling of being evaluated. The best way to combat this issue is to have mock interviews to recreate the experience until we’ve adjusted to it. Having a problem solving process also really helped to give me a leg up, and the best one that I’ve found is PEDAC.

Know what you’re looking for in a role & company

Photo by Unsplash

With all that said, it’s important to take the time and consider exactly what you’re looking. Do you have a strong preference between backend, frontend, or full stack positions? How big of a company do you want to join? Which industries interest you? And in which direction do you want to grow your career?

Your answers to these questions can have a dramatic effect on which companies you choose to interview with. As software developers, we’re lucky to have no shortage of jobs and companies looking to hire, but it’s important to remember these points:

There is a large variety of roles and areas of work to choose from, and you should consider which areas of software development you want to specialize in.

You should try to find a role and company that you’ll be happy to work in for several years. My personal opinion is that it’s important to build your experience, credibility, and professional network as an engineer and it’s best to work 1–2 years minimum in each role, rather than job hopping every 6 months — 1 year.

Just as there are a lot of good jobs in software, there are also a lot of bad jobs that you want to avoid. Not every job will have you work on mature, experienced teams building ambitious, challenging projects. There are jobs in which you could mainly handle tedious tasks, clean up legacy code, or simply minor UI components day-in & day-out.

So take the time to think about what you want and list the things that you look for in your dream job. For myself, I wanted a role that focused mainly on backend development with the opportunity to:

Learn a compiled language like Go.

Work on distributed systems and possibly microservices.

Join an experienced team of engineers to learn from.

Have the ability to work remotely from time-to-time.

Work at a company that values diversity and inclusion.

Have benefits toward continued education.

I aggressively sought out companies that offered most of them and didn’t spend time applying to those that didn’t. While not every company would have everything I was looking for, I knew that I would be happier with the end result than if I just mass applied everywhere.

A side benefit was that by targeting the specifics of my next role, I was able to focus my studies on relevant topics that allowed me to improve with every interview, which I feel gave me the needed edge to land my role at DigitalOcean.

Manage the interview cycle

Photo by Unsplash

The interview process for software development is grueling, and the potential for burn out is very high. Typically, the process can comprise of 3–5 steps, or even more, like so:

Initial phone screen with HR or a recruiter

Phone call with a company engineer or hiring manager

Live code challenge with a company engineer

Take home project

Onsite at the company

Potential offer extended

Exhausting, right? And every company is different, with some having only 1–2 interviews before the onsite and others taking months!

From my job hunt, I experienced a cycle that repeated itself. First, I would have initial phone screens with multiple companies. These would then lead to code challenges, take home projects, or technical phone screens. After several of these steps and opportunities getting filtered out, any onsites would get clustered into the same week or two.

Since the intensity of these stages increased the further I went in the process, I had less time for new applications due to increased studying, culminating with my onsites which had me studying full-time without sending fresh applications. If no suitable offers were extended, then I would start again with fresh applications and the cycle would start over.

Here are some tips when going through this cycle:

Try to keep your application activity high throughout the cycle. It’s important to keep applying and doing outreach in order to fill your interview funnel with new opportunities.

Have a daily goal of applications and outreach to complete. My daily goal for Monday-Thursday was to submit at least 5 applications (often with cover letters) and reach out to 5 engineers for coffee or to ask about open positions.

Think critically about what to expect in each interview. You may get some details in your initial phone screen about the interview process, but always ask for more info in order to prepare.

If you have a call with a CTO, know that it’s common for them to ask you technical questions regarding your resume as well as behavioral questions.

Don’t schedule onsites on back-to-back days. They are by far the most exhausting steps and need more preparation. Always give yourself at least one day of rest in-between onsites to recharge, review topics, and mentally prepare.

Avoiding burnout through the process

At this point, I want to address the importance of avoiding burn out during the job hunt. With the sheer number of interviews to complete and associated stress, it can easily feel overwhelming at times.

Don’t overwork yourself by studying & coding for 6+ hours a day on top of interviews, and dedicate time each week to disconnect and recharge. It’ll help you maintain your energy & focus as you steadily ramp up the intensity of later stage interviews.

At most, I studied 4 hours a day and prioritized deep work as described in Cal Newport’s Deep Work, which is where you focus intensely for a period of time on a task that is more difficult than what you’re used to without any distractions.

This meant solving algorithm problems or practicing projects that were harder than what I had already completed. This made sure that I kept improving my skills, while also identifying where I struggled and needed to study more.

And no matter how busy my week was, I always took at least one day off to relax and recharge for upcoming interviews.

In the end…

I’m extremely happy with the results of my job hunt and feel that the approaches I detailed above worked well for me. For those of you out job hunting now, I hope that this article proves helpful to you. I also recommend reading the following article, which I found very helpful for my job hunt:

What it took to land my dream software engineering job — Sun-Li Beatteay

Good luck to everyone out there job hunting, keep it up!

Please let me know what you think in the comments! I’d love to hear more about your job hunt experiences and what has worked for you.

How to make Google’s Work Manager work for you

freeCodeCamp — Mon, 01 Apr 2019 19:03:50 +0000

By Ayusch Jain

The article was originally posted here

Consider some use cases which we encounter in our daily Android development tasks:

Downloading data files from a remote server for a mobile game.

Downloading files from a server.

Synchronizing data collected on mobile with a back-end service, for example: uploading crash analytics, log files, etc…

Backing up device files to online storage.

How I went from n00b intern to engineering team lead

freeCodeCamp — Sat, 23 Feb 2019 17:01:32 +0000

By Kostis Maninakis

And all the things I learned, in retrospect.

When I see a rocket launching gracefully into the sky, I always think about all the hard work and engineering that led up to everything going according to plan.

First, a reality check:
If you are looking for the 10-minute-abs, zero-to-hero-with-this-one-weird-trick kind of article, you won’t find it here. Achieving anything meaningful is hard and needs persistent work over long time.

It doesn’t matter where you are starting from

I didn’t grow up in a good place. There, any kind of studying as a teenager was totally not cool between my peers. Statistically, if you didn’t grow up in this kind of environment, you have a head start already.

For more than 12 years I’ve been a Computer Science student. Even though I’m almost done, I still haven’t managed to get my CS degree after all this time.

Over the years I’ve done every other kind of job you could imagine. Some where pretty crappy ones. But how do I get into an industry where everyone asks for at least 3 years of experience?

Took me a long time to realize that the start is always the hardest and takes the longest to leave behind. Particularly while you are in it, it seems like it will be rough like this forever.

The truth, though, is that the engineering industry is today more diverse than ever. There’s a perfectly fitting niche for everyone (who knows themselves enough).

In fact, engineering demand is growing faster than the engineering supply can meet. This results in an ever-increasing deficit. Specifically the demand for good engineers is very high all over the world right now. So it’s only going to get easier for you to get a job, even if you just wait. Play your cards right and the potential opportunity skyrockets.

Here are some of the things I’ve learned along the way.

Act like the person you want to become

Always do your best. That means, whatever you do, always try to go the extra mile. Think of your best as it’s defined in The Four Agreements.

In order to get extraordinary results you need to make extraordinary efforts. Trying your best guarantees nothing, of course, but not settling for “good enough” is your only ticket to getting access to what the rest of the pack might never get.

Always be genuinely curious and never stop learning. I cannot stress this enough. To this day I still watch 1 - 3 technical conference talks on YouTube per day, and I have for years. I’m a very slow reader so this works around my problem, but for you it could be Medium articles or a technical book, or grinding a Udemy course or whatever else. If you don’t have this innate curiosity, then I strongly suggest you reconsider your career choice.

After you are done building something, always ask yourself how you could make it better. Then research your approach, then make it better! Handling edge cases, optimizing security and performance, refactoring into DRYer and cleaner code or adding features - it doesn’t matter. Just follow this guideline and you will learn loads!

Find a project that excites you and put your heart and soul into it. Let it lead you to the knowledge that you lack, embrace it and make it part of you and your every-day routine.

A while ago I almost started up a fintech startup. The project found its end prematurely, but that’s OK. What I’ve learned while working on it and the experience I’ve earned leading up to selling the tech were rewards, justifying my efforts in full plus interest.

Apply for an internship in a small company (no more than 50 employees). You will learn new things in real working conditions and with real consequences to drive you. Plus it’s a gateway to a junior position in the same place, bypassing the 3 years experience requirement.

Take care of yourself. Seriously. Think of yourself as a person in your life you love and care about. Treat yourself accordingly. Food, sleep, exercise, meditation, fun; try to persuade that person to do well in all those dimensions for their own good. Having this attitude will result in a functional and performant mind and body. You can road trip life on a slow, smelly, bus, or a sleek, fast cabriolet playing your favorite tunes on the speakers. Your choice.

You’ll never get something you didn’t ask for

You can’t always get what you want, but if you try, sometimes you get what you need.

Apply to as many interesting jobs as you can find (indeed.com, glassdoor.com, stackoveflow.com/jobs). This implies you have a good enough definition of what is interesting to you, beforehand.

Don’t be afraid to look abroad. In fact that’s what you should be prioritizing. It’s a guaranteed way to get you out of your comfort zone. What better way to better grow yourself in your craft and as a person! At this day and age, many companies offer benefits such as relocation or visa bonuses. Search for it, ask for it.

Absolutely do apply to interesting jobs you think you don’t deserve! There’s a high chance you underestimate yourself and overestimate those others out there. You don’t know how you really stack-up against the other resumes these hiring managers receive, and in the end, you just never know! Just give it a shot or 20, it’s free! Worst that can happen is you won’t be accepted.

Always be honest and hold your weaker points like a badge of honor. Still, do try to polish up those outliers if you can.

Marketing is key, as always

Marketing applies to everything: From the barista at the coffee shop that takes more care of you because he likes you, down to your personal relationships. And that’s OK. Marketing, in its core, is the art of communicating information in a manner that is interesting and valuable from the perspective of the recipient. Marketing, when inaccurate, can indeed be deceptive — but when practiced correctly, it’s the best way to get yourself understood.

You need to market yourself the way you would market your product or service to a customer. That means branding yourself, having a landing page, paying close attention to the wording/copy used to describe yourself. The end goal is to spike the recruiter’s interest, to stand out and make them want you to join their team or, at the very least, to want to know more about you. Trying to dazzle them, though, won’t work. Their BS detectors are pretty sensitive.

Diversify yourself from the pack however possible. An uncommon cover letter, a populated GitHub account, stack overflow activity, a personal website, a readable CV style, some forum presence, anything — do more!

Think long and hard about your cover letter. First impressions are paramount, and chances are that with a mediocre/generic cover letter, your chances to proceed in the hiring process have just tanked already.

Be you. During the interview have zero expectations of performance or outcome. Nada. Nil. Forget about all that! Just let it all go and just be you with the recruiter or whoever you’re talking with. Don’t try to look smart, but don’t be too introverted either (unless that’s who you actually are or how you are feeling that day). Just get out of your head and be in the interview in full.

Have a system

This goes back to acting the role.

Write things down. Where you have applied, when and what you asked for, what was talked about during interview, their replies to you, other notes of yours maybe as labels, whether you got an offer and what that was…document it all.

Have a central place where all your notes live. I used Trello but it could be anything really.

Calculate your projected expenses for each city/position before you get asked about your expected salary (to roughly account for taxes, multiply by 0.66). Numbeo can help you with that.

Don’t forget that, no matter your skill/experience level, you can - and should - negotiate (but only after receiving the offer).

In my experience, it’s better to be an underdog and then stand out, than trying to meet standards set too high, too early.

Be patient, be persevering

Nothing ever happens overnight. I practiced all the above at least 2 years before I even applied for my internship. I spent about a month setting up my GitHub, Stack Overflow, personal website, researching jobs, before I began to apply.

Anything meaningful is hard and takes time. 10-minute-abs tricks don’t exist.

Embrace failure. Expect that you will fail. You will quit. You will drop the ball. Then you’ll pick it up again, every time, and recommit.

Fail, recommit! Fail, recommit! Fail, recommit! Fail, recommit! (Ad infinitum)

Any time you have to make any choice, project it against the direction you want to end up, and opt for the option aligning best (hint: usually it’s the one you fear most to take). Keep applying this and success is unavoidable.

Stay on the road and keep chipping away… till you get there!

…Then you’ll eventually realize there was no definite “there” all along :). But now you’re still leagues beyond where you started, which is the only thing that matters anyway.

_Don’t let your dreams be dreams. [Just DO it!](https://www.youtube.com/watch?v=ZXsQAXx_ao0" rel="noopener" target="blank" title=")

Good luck, friend from the interwebs! ?

If you found this was worth your time, please click the ? below so that other people may see it here on Medium.

Thank you for your attention.
--maninak

How to make your future self happy by writing good docs

freeCodeCamp — Mon, 18 Feb 2019 17:04:52 +0000

By Gabriele Cimato

Or how to be less miserable when dusting off an old code base

This is a little overview of common problems faced when working on a new or old project. Sometimes making a little effort up front can save you time and energy down the line. Writing good docs is like getting ready for your future self to high-five you ✋! We’ll see a silly example and a few recommended practices to get started with a good README.md.

The Struggle

I’m almost certain that in your career, or in your everyday life, you faced a situation before that makes you think:

“This problem looks familiar, I’m pretty sure I solved it before. If only I could remember how I did it!”

Especially coming from an engineering perspective this happens quite a lot. Repeated patterns, functions or bugs we’ve met before that require us to go through the exact same past effort to overcome an issue. Sometimes we are willing to do it again, so we go ahead and figure it out once more. Some other times though…

An example

I lead the R&D department at Hutch and we often have to dig deep into new and unseen technologies, frameworks or languages. You try and experiment a lot and can’t expect to remember everything you interact with. You work on something for a couple months then, once you’re done, switch to something very different. Or maybe you just work on the next step in a pipeline.

Then, when you least expect it, it happens. You have to go back to that framework you used 3 months before to make a change. You give it a look, a puzzled one ?.

“Oh, actually I remember this. To make it behave this other way I go here…change this…and voilà!”

At that moment you feel pretty good about it. You were able to recollect how things worked. You’re even proud of yourself for leaving simple docs on many of the functions you wrote many moons before. Then with a light touch of your mouse, you run the project and…

⛔ ERROR! The main frame has some cowbells directed towards Mars, this is not allowed.

? Yikes! This looks very cryptic. You take a look at the code you changed and, well…you try to run it again. Maybe something will automatically change. Maybe looking at it again had some quantum effect of some sort. Nope.

⛔ ERROR! The main frame has some cowbells directed towards Mars, this is not allowed.

You then read through the comments or the docs. Nothing mentions this error, nothing points you in the right direction. This error is so distinctive that you’re sure you saw it before, and also solved it before. As daunting as it feels, you have to figure it out…again!

You start googling the problem and notice some previously visited links.

“Great! This link is probably the one that helped me with the error…phew. Crisis averted!”

You then scroll the page and, oh no! More…many more visited links. So now you have no idea which one led to a solution if any. And so the search continues until eventually, you figure it out — minutes, hours or even days later.

Good documentation covers more than just happy paths ?

I learned this the hard way. Multiple times. It’s often easy, although admirable, to add docs to your functions/methods/classes with the assumption that everything will work well.

I always try to make life easier for whoever will dig into my code. That includes future me! So I diligently add docs to almost all of my functions but the trivial ones. As many have said for decades:

Your comments should explain the why more than the what.

Your code should be “self-documenting” so that any added comment addressing the “what” would result redundant.

In all fairness, I tend to add comments even for the “what”, only when I know a function is either long or somehow intricate. Adding a few lines of comments would allow me to skip examining every line of code. This has been helpful countless times and I absolutely recommend it!

But documentation is not just lines of comments on a function. Good documentation is a well written README.md. In some scenarios even a full-fledged dedicated website for bigger projects (see React, Redux, Vue, Slate, …).

The projects mentioned are all open source. Teams are basically compelled to go in greater detail to help people start using their framework or library (and have been doing a great job in that regard!). But what about smaller private projects? What about those projects that will only live within the company (no matter what the size of the team is)? And what about all those issues that are not purely code related?

More often than not we tend to skip the README.md file or dismiss it with a few lines only. I’ve been following a simple yet powerful practice to make this task less intimidating and help go beyond the happy paths.

A basic approach to creating a good README

When mentioning “happy paths” I refer to the practice of assuming everything will run smoothly. This means we are only addressing each step of a process as if it will always succeed.

Unfortunately, that is not always the case so, how can we make our lives better? How do we make sure that even the not-so-happy paths are covered?

Here are a few suggestions:

Start by writing down a few lines about what the project is about and what problem you are trying to solve. This helps you, and whoever goes through it, understanding the intent of the project.

As you go about setting everything up, make sure you add each step to the README.md. It doesn’t have to be well formatted and phrased, it just needs to be there for now. This usually comes in the form of installation instructions.

If at any time you face an error of any sort, add a section at the bottom. You can call it Common Errors. There you add some details about the error you faced and how you solved it. One crucial thing I like to do here is add links to the source of the solution (or anything that helped me get there).

When I reach a stable point in the project I try to install it on a new machine (if it’s a possibility). The goal here is to ensure that the set-up steps we listed before are correct and work as expected.

Most importantly, you need to have a section answering the question: how do I use/run this project? This should be as clear as possible, so put some effort into it! Sometimes, though, you can’t answer this question until later on. You can wait until you are in a working/running state to update the README.md .

Put aside some time to review and clean up your README.md file. Most of the time you’ll probably need to update it.

This is often enough for small size projects. For mid to large-sized ones it can be a starting point to develop some good practices. Talk about it with the rest of the team and agree on a common approach that will make everyone happy. Keep in mind that maintaining the docs up to date is crucial! Hold each other accountable for it and after the initial effort, it’ll become natural, just like a simple refactoring!

Conclusion

Writing good docs involves maintaining a good README.md and documenting the whys in your code more than the what.

If you make a small effort and incrementally build up a good README.md it will feel less intimidating. Not only will it make your life better in the future, but it will help anyone else contributing.

Don’t cover only the happy paths expecting everything will work, also cover eventual errors that you face or any issue a newcomer might face. Keep a dedicated section for this.

Bonus: when estimating your work with a PM, take into account the effort required to write/update the docs. Don’t underestimate the fact that good docs require a good amount of time. But that time is very well spent!

? Hi, I’m Gabri! I love innovation and lead R&D at Hutch. I also love React, Javascript and Machine Learning (among a million other things). You can follow me on twitter @GabriOnTheRocks and on GitHub @Gabri3l . Leave a comment if you have any other recommendation that you’d like to share, or send a DM on Twitter if you have any question!

engineering - freeCodeCamp.org

Product-Led Research: A Practical Guide for R&D Leaders [Full Book]

What You'll Learn

Who Is This Book For?

Who Am I?

The Approach of This Book

Structure of This Book

Why Is This Book Publicly Available?

Feedback Is Welcome

Table of Contents

Part 1: Foundations

Part 2: Research Management Methods

Part 3: Ensuring Product Impact

Book Summary

Note

Part 1: Foundations

Chapter 1 - What is Research?

The Difference

Schoenfeld's Framework for Problem Solving

Why This Matters for R&D Leaders

Notes

References

Chapter 2 - Research and Development

Defining Research vs. Development

A Quick Test

Key Indicators You're Doing Research

Common Misconceptions

Your Role as Research Leader

Part 1 - Summary

Part 2: Research Management Methods

Chapter 3 - Why Methodology Matters: A True Story

The Lesson

What This Part Covers

Chapter 4 - The Research Tree

What Is a Research Tree?

Your First Research Tree

The Power of Stopping to Think

Choosing Your First Path

What If Your First Choice Doesn't Work?

Choosing When Everything Seems Equal

Heuristics Can Be Combined

How Answers Lead to New Questions

The Complete Picture

Color-Coding Status

Additional Tips

The Research Tree Prevents Common Pitfalls

Time to Practice

Using the Tree with Your Team

Tools

Pro Tips

Recap - The Research Tree

Chapter 5 - Time-Boxing Research Explorations

The Problem: Research Without Time Limits

Time-Boxing: Creating Decision Points

The Decision Point

Example: Detecting God Objects

How to Set Time Boxes

Using Time-Boxing with Your Team

Integration with the Research Tree

Recap

Part 2 Summary

Part 3: Ensuring Product Impact

Chapter 6 - How to Choose Research Initiatives

1. Starting From a Concrete Problem

2. Starting From a Technological Opportunity

Problem-Driven vs. Opportunity-Driven: A Comparison

Should You Pursue a Research Initiative?

Pre-Research Checks

How to Choose Research Initiatives - Summary

Chapter 7 - Drawing Backwards

The Spiral Game

How to Apply Drawing Backwards to Product-led Research

Case Study: Extracting Business Rules from COBOL

Why Drawing Backwards Is So Powerful

Practical Application: Your Research Tree

Summary: Drawing Backwards

Chapter 8 - End-to-End Iterations

Drawing Backwards + End-to-End: A Combined Approach

The Five Principles of End-to-End Iterations

Principle 1: Outline the End-to-End Process