Then I joined a real team.

Suddenly, I was introduced to this thing called Kanban – and I realized I’d been treating software like a solo art project, not a process.

If that sounds familiar, you’re in the right place.

This guide will walk you through how Kanban actually works, how developers use it to track and prioritize work, and how it can help you stay sane when juggling bugs, features, and real-world deadlines.

Without further delay, lets get into it.

Here’s what we’ll cover:

• So… What Is Kanban?

• The Classic Kanban Board: Three Simple Columns

• How Developers Use Kanban in Real Life

• Kanban vs Scrum: What’s the Difference?

• So which one should you use Scrum or Kanban?

• What Tools Do Teams Use for Kanban?

• How to Use Kanban to Manage Your Own Coding Projects

• Final Thoughts: Why Kanban Isn’t Just a Board

So… What Is Kanban?

At its core, Kanban is a visual way to manage work. It helps teams (or team members) see:

• What needs to get done

• What’s in progress

• What’s finished

• Where things are getting stuck

The concept comes from lean manufacturing, but in tech, it’s often used in Agile teams that need flexibility without the structure of Scrum sprints.

Think of Kanban like a whiteboard that tells a story. Not just what’s done, but how work flows.

The Classic Kanban Board: Three Simple Columns

So what exactly is a Kanban board? At its core, it’s a visual representation of your workflow – a board that shows all the work your team (or you, solo warrior) are juggling, and where each task stands.

It can be physical, like an actual whiteboard with sticky notes that move from one column to the next. Or digital, using tools like Trello, Jira, GitHub Projects, or Notion. The key is that it’s visual and up-to-date. You can walk into a room or open a tab and instantly understand: What’s being worked on? What’s ready to go? Where are things stuck?

It’s like having your brain on a wall, but organized. And slightly less chaotic.

The beauty of Kanban is how dead simple it is to get started. At minimum, your board has three columns:

Each task – or card – moves from left to right as it gets worked on.

Let’s say your team is building a blog platform. Your Kanban board might have cards like:

• “Create signup form”

• “Fix image upload bug”

• “Deploy staging build”

Now, while Kanban is flexible, it can absolutely be taken too far.

I’ve seen boards with more columns than a Greek temple: “Needs Review,” “Pending Client Feedback,” “QA Rework Round 2,” “Blocked but Still Hopeful,” “In Existential Limbo,” and so on. Every card had six tags, three owners, two checklists, and one migraine.

The lesson? Don’t turn your board into a bureaucratic jungle.

You don’t need to account for every edge case. Start simple: “To Do,” “In Progress,” “Review,” “Done.” These basic stages cover most workflows. If you discover a real need for something more – like a dedicated “QA” column or “Blocked” column – add it intentionally, not because you feel like your board needs to look fancy.

Remember: A Kanban board should be helpful, not overwhelming. If you spend more time managing the board than doing the work on it… it’s doing the opposite of what it’s meant to do.

How Developers Use Kanban in Real Life

Here’s how you might interact with a Kanban board on a dev team:

1. You pick up a card from “To Do” – let’s say, “Add dark mode toggle.”

2. You move it to “In Progress.”

3. When it’s ready for review, you might move it to a temporary “Review” or “Testing” column.

4. Once it’s merged, tested, and deployed, you move it to “Done.”

5. You smile, drink some coffee, and grab the next card.

That’s it. But over time, this process helps the whole team:

• Spot bottlenecks

• Prevent duplicate work

• Reduce context switching

• Keep everyone aligned

What’s a WIP Limit — And Why Should You Care?

WIP = Work In Progress. This is the most important concept to keep us in check.

One of Kanban’s key principles is limiting how many things you’re working on at once. Because guess what? Multitasking kills momentum.

A typical WIP limit might look like:

• No more than 2–3 cards per person in “In Progress” Again this is best practice, but folks do pick up a lot and then they end up being the bottleneck.

• No more than 5 tasks waiting on QA.

Why? Because when everything’s urgent, nothing gets done. WIP limits force you to finish one thing before you start more – and that’s how real velocity happens.

If there are more than 5 tasks in the “To Do” column, the team doesn’t take up new ones. Instead, everyone chips in to see how they can help unclog the bottleneck. A bottleneck is your worst enemy in Kanban, and you want to resolve it so items move smoothly on time and on target.

Here’s a video recapping key concepts.

Kanban vs Scrum: What’s the Difference?

You’ve probably heard Scrum and Kanban mentioned in the same breath – and both are popular Agile frameworks. But they’re not interchangeable.

Scrum is structured, with roles like Product Owner and Scrum Master, and work gets organized into time-boxed sprints. It’s perfect for teams that benefit from rhythm and rituals – like sprint planning, daily standups, and retrospectives.

Kanban, on the other hand, is a little looser. No official roles, no set sprint timelines. Work flows continuously, and change can happen anytime. It’s perfect for teams who need more flexibility and fewer ceremonies.

So how do they compare in practice? Let’s break it down:

Bottom line:

• Scrum is a scheduled loop. Kanban is a living flow.

• One’s a playbook. The other’s a status window.

Here’s a video on the main differences between Scrum and Kanban you can watch if you want more detail.

So which one should you use Scrum or Kanban?

So… which one should you use?

It really depends on your team, your product, and your pain points.

✔️ If you’re working on a brand-new product where requirements shift a lot, and your team thrives with structure and routines – Scrum is likely the better fit. Sprints give you a sense of pacing, and ceremonies help ensure alignment.

✔️ If you’re managing ongoing work like bug triage, tech debt, infrastructure tasks, or anything that’s more “whenever it comes in” than “we need to ship this in two weeks” – Kanban gives you flexibility and visibility without the overhead.

And yes, there’s such a thing as Scrumban – a hybrid approach where teams use visual boards and WIP limits from Kanban, but keep some of Scrum’s structure like standups and retros. It’s like Agile tapas: you get the flavors that work best for your appetite.

Here is a detailed video that’'ll teach you more about how Scrumban works in practice.

Watch the Scrumban video only when you are familiar and comfortable with both Scrum and Kanban – otherwise, you might get confused from the cross-pollination of ideas and frameworks.

I personally have never seen a Scrumban implementation thats scaled well – too many folks trying too many things and none of them work. But thats just based on my experience – it may work for you and your team. I’ll let you be the judge.

What Tools Do Teams Use for Kanban?

You’ve probably seen (or used) one already:

• Trello – Simple and great for solo or small teams

• Jira – Enterprise-level, customizable workflows

• GitHub Projects – Lightweight but powerful for devs

• ClickUp / Asana / Notion – Integrated with docs/tasks

Kanban isn’t tied to any one tool – you can use an app, a browser tab, or a whiteboard and a pack of sticky notes from the office supply closet. What matters is how you use it. But let’s walk through some of the most common tools and what they offer in a Kanban context:

🟩 Trello

Trello is probably the easiest way to start with Kanban. It gives you a simple digital board with columns and cards you can drag and drop. It’s great for devs or small teams who don’t need tons of automation – just a clean place to track work visually.

🟨 Jira

Jira is a heavyweight – and while it’s built for Scrum, it also supports robust Kanban boards. You can define custom workflows, use built-in reports like cumulative flow diagrams, enforce WIP limits, and manage team velocity. Ideal for large teams that need traceability, integrations, and permissions.

🟦 GitHub Projects

If your code lives in GitHub, GitHub Projects is a clean way to stay close to your codebase. It lets you create Kanban-style boards with issues and pull requests as cards, so you’re never toggling between tools just to track what’s in progress.

🟧 ClickUp / Asana / Notion

These are all-in-one productivity platforms. They combine Kanban boards with documentation, team chat, calendars, and reporting. If your team needs more than just “move card left to right,” these tools let you manage projects, meetings, notes, and workflows in one place.

🟪 Whiteboard + Sticky Notes

Don’t underestimate the analog approach. It’s fast. It’s visible. It’s tactile. Physically moving a task from “Doing” to “Done” gives you a sense of progress no digital tool can match. And when something’s blocked? Slap a red sticky on it and call it a day.

Bottom line: The best tool is the one your team will actually use. Fancy doesn’t beat consistent. And the actual tool doesn’t matter as much as the discipline your team has to actually use it.

How to Use Kanban to Manage Your Own Coding Projects

Even if you're not on a team yet, Kanban is great for your own workflow. Here’s how you can use it to help yourself out:

1. Create a basic 3-column board (To Do, In Progress, Done)

2. Write out every task, big or small

3. Set a WIP limit (for example, no more than 2 tasks at once)

4. Update it daily. Make it a ritual.

5. Review your flow weekly – What got stuck? What moved fast?

Example:

You’ll be shocked how much clearer your thinking gets when you can see your work. It’s simple but super powerful to visualize your work it in this way.

Final Thoughts: Why Kanban Isn’t Just a Board

Kanban isn’t just a tool – it’s a mindset.

It helps you focus. It helps your team collaborate. And it gives everyone – even non-technical folks – visibility into what’s going on.

If you’re learning to code and want to feel more confident working with others, learning Kanban is low-effort, high-impact.

So don’t wait until your first job. Start using it now – and show up to that standup with confidence.

I hope this small 101 Guide to Kanban was helpful to you all. My sole purpose to write this was to help beginner developers understand Kanban as a practical workflow system – especially for those transitioning from solo coding to collaborative, real-world development environments. It aims to demystify the methodology in a casual, beginner-friendly tone while still offering actionable guidance.

I hope you enjoyed my beginners guide to Kanban.

Until next time, keep Learning, Unlearning and Relearning, folks….

You’ve been grinding through tutorials. You've built a portfolio site, maybe deployed a few projects on GitHub. And now you're trying to land a job or join a team.

Then the interviews start.

Suddenly, people ask:

• "Are you familiar with Agile?"

• "Have you worked in a Scrum environment?"

• "What’s your experience with sprints?"

Cue the imposter syndrome. Because no one teaches this stuff in JavaScript 101.

This guide is for you.

I’ll help make the Scrum process – a very common way developers work together – make actual sense. I’ll walk you through the basics, but also tell you what developers actually do, how standups feel when you're new, and what’s expected of you when you’re no longer coding in a vacuum.

Let’s break it down.

Here’s what we’ll cover:

• What Even Is Scrum?

• The Three Roles in Scrum (and Who Does What)

• The Scrum Rhythm: What a Sprint Actually Looks Like

• Who attends the Ceremonies:

• Standups: Where You Talk Like a Human, Not a Robot

• Sprint Planning

• What’s a User Story and Why Does It Sound Like a Children’s Book?

• What Counts as “Done”? Definition of Done and Why It’s Important

• Demos, Retros, and Saying the Hard Things

• Tools You Might Encounter

• If You’re Preparing for a Job, Here’s What You Can Do

• Final Thoughts

What Even Is Scrum?

Scrum is not a tool. It’s not a software. It’s not some elite thing only PMs care about.

It’s a lightweight framework that helps software teams build things incrementally, together, in short focused cycles called sprints.

Scrum is used by everyone from FAANG teams to indie dev shops because it helps:

• Keep teams aligned

• Deliver working software fast

• Course-correct often

• Spot problems early (before they go nuclear)

It’s the opposite of the old-school “build for a year and pray it works” model.

The Three Roles in Scrum (and Who Does What)

Scrum officially defines three roles. Here's what that means in practice:

1. Product Owner (PO)

Think: Vision-holder. They decide what the team builds and why. A product owner:

• Writes user stories (think of these as feature requests written from a user’s point of view)

• Prioritizes the work

• Clarifies what success looks like

• Says “yes” or “not yet” to features

2. Scrum Master (SM)

Think: Air-traffic controller meets therapist. They make sure the process works. The are master Facilitators, like between Dev and PO’s. A Scrum Master:

• Facilitates meetings

• Removes blockers (“Your AWS access is stuck? I’ll escalate it.”)

• Coaches the team on Scrum practices

• Doesn’t manage people – manages flow

3. Developers (YOU!)

Think: Builders. You write code, test it, ship it, fix it, and improve it. You also:

• Break down stories into tasks

• Pick up work from the team board (like Jira or Trello)

• Communicate progress

• Demo what you’ve built at the end of the sprint

You might also work with designers, testers, or DevOps folks – but within Scrum, you’re all “developers” building a product together.

The Scrum Rhythm: What a Sprint Actually Looks Like

Image Source: https://www.invensislearning.com/blog/what-are-scrum-ceremonies/

Understanding the Scrum Cycle

So, what does it actually look like when a team uses Scrum to build software?

Let’s walk through a full sprint – not just the buzzwords, but what really happens when a group of humans tries to plan, build, review, and improve together. Think of this as your backstage pass to the rhythm of modern teamwork.

📦 Step 1: Build and Refine the Product Backlog

Before any coding starts, the team needs to agree on what they might build – not just this week, but in the near future too.

That’s where the Product Backlog comes in. This is a big, running list of everything the product might need – features, bug fixes, improvements, ideas, and maybe a few wild dreams. It’s like the wishlist for the product, but more organized (ideally).

The Product Owner is responsible for maintaining and prioritizing this list. They decide what’s most important to work on based on customer needs, business goals, and feedback.

But the PO doesn’t do this in isolation. Enter the Backlog Refinement meeting.

In these sessions, the Scrum Team – that’s the PO, the Scrum Master (SM), and the Developers – come together to:

• Review the most important upcoming items

• Clarify any vague or confusing parts of each task

• Break big items down into smaller, buildable pieces called user stories

• Estimate effort (how much time or complexity is involved for each story)

This meeting makes sure the team isn’t caught off guard in the sprint – that they understand the work ahead and can actually start sprinting when the time comes.

🧭 Step 2: Sprint Planning – What Are We Building This Time?

Now that we’ve got a solid backlog, it’s time to pick what to build right now.

At the start of each sprint (which typically lasts 1 to 4 weeks), the team holds a Sprint Planning meeting. This meeting sets the stage for the entire sprint – it’s like the huddle before the big game.

In Sprint Planning, the team:

• Reviews the top items from the backlog

• Discusses what can realistically be completed based on their availability and capacity

• Chooses a handful of these stories to commit to

• Defines a Sprint Goal – a simple statement that captures the purpose of this sprint

For example, the Sprint Goal might be:
🎯 “Allow users to reset their passwords.”

Every user story chosen should contribute to that goal. The collection of these stories becomes the Sprint Backlog – basically, the to-do list for the sprint.

So when we say:

“The team selects an ordered list of user stories to comprise the Sprint Backlog for the next sprint, which will be achievable to satisfy the Sprint Goal...”

We’re really just saying:
👉 “Pick a realistic number of important tasks that, if completed, will help us hit our target for the sprint.”

Not too vague. Not too ambitious. Just achievable and focused.

☀️ Step 3: Daily Standups – Stay in Sync

Now the sprint is underway! But how does everyone stay aligned and avoid working in silos?

That’s where the Daily Standup comes in. Every day – usually in the morning – the team has a quick check-in (about 15 minutes) where each person answers three questions:

1. What did I do yesterday?

2. What am I working on today?

3. Is anything blocking me? (that is, am I stuck?)

Example:

“Yesterday I set up the login API integration. Today I’ll work on the UI validation. I’m blocked on getting access to the staging database — may need help.”

These standups keep the team in sync and surface blockers early so they can be addressed quickly. They’re not about micromanaging or showing off. They’re about visibility and support.

📉 What’s a Sprint Burndown Chart?

You might hear your team mention a “burndown chart.” No, this isn’t about things going down in flames (hopefully).

A Sprint Burndown Chart is a graph that shows how much work is left in the sprint – day by day.

• The y-axis is the amount of work remaining (often measured in story points or tasks)

• The x-axis is the number of days left in the sprint

The line should ideally trend downward as work gets completed – hence “burning down.” If it flattens out or slopes up, that’s a red flag that the team might be stuck, behind schedule, or not updating the board.

Think of it as a visual heartbeat of the sprint. You can learn more via a practical example in this video.

🖥️ Step 4: Sprint Review – Show What You’ve Built

At the end of the sprint, the team holds a Sprint Review (also called a “demo”). This is where you show what was actually built during the sprint.

• The Developers demo working features – live, not just screenshots

• The Product Owner reviews whether the Sprint Goal was achieved

• Stakeholders may ask questions, give feedback, or suggest tweaks

This meeting isn’t just for show – it’s a feedback loop. It helps the team validate that what they built is useful, usable, and meets expectations. If changes are needed, those get added to the backlog for future sprints.

🔍 Step 5: Sprint Retrospective – Look Back to Move Forward

Once the review is done, the team shifts focus from what they built to how they worked together.

Enter the Sprint Retrospective – a meeting to reflect on the process, not the product.

The team discusses:

• ✅ What went well

• ❌ What didn’t go so well

• 🔁 What could be improved next time

This isn’t about pointing fingers. It’s about learning, adapting, and continuously improving how the team collaborates.

The Scrum Master often facilitates this meeting and helps turn feedback into action items for the next sprint. For example:

“We underestimated testing time. Next sprint, let’s budget for QA earlier.”

The best teams take retros seriously. Why? Because even if your code is perfect, your process needs tuning too – and small process changes often lead to big gains.

♻️ Scrum Is a Loop

Here’s the rhythm:

1. Plan the sprint

2. Check in daily

3. Build and demo the product

4. Reflect and improve

Then do it all over again – with slightly better coordination and slightly more trust each time.

It’s not about being fast. It’s about being intentional, consistent, and collaborative.

Example Sprint

Let’s say, for example, that your team does 4-week sprints. (Keep in mind that Sprints can differ by team, nature of product, release cycles, and so on.)

Here’s the rough beat:

Scrum works in loops. Every 2-4 weeks (depending on your cadence and sprint cycle), your team should have working, demo-able software to show for it – even if it’s small.

And no, it’s not about “speed.” It’s about consistency, communication, and collaboration.

Who attends the Ceremonies:

Now lets dive deeper and understand practically how each of these ceremonies work:

Standups: Where You Talk Like a Human, Not a Robot

So how does the team actually stay connected day to day? That’s where standups come in.

Every morning, your team meets briefly – usually on Zoom or in a circle – and you answer 3 questions:

1. What did I work on yesterday?

2. What will I work on today?

3. What’s blocking me? Any impediments?

Example:

"Yesterday I cleaned up the signup validation logic. Today I’m working on the email verification flow. I’m stuck on SendGrid config – might need help setting up credentials."

It’s not about impressing anyone. It’s about keeping everyone in sync. Some days you’ll say, “I spent the whole day debugging a CSS bug that turned out to be a semicolon.” That’s okay.

How does it work?

The Scrum Master gathers everyone in a huddle room, the PO and Dev Team included, and opens the the Standup. They are the facilitator of the ceremony. Everyone gets a chance to answer the 3 questions above (usually about 2-5 minutes each). It’s not a full report – it’s quick. When one person is done, they pass it on to someone else.

This ensures there is team cohesion and transparency.

Here is a video example of a standup.

Sprint Planning

The goal of the planning meeting is to answer the questions “What are we going to work on, and how are we going to do it?” It is critical for the team to have a shared goal and a shared commitment to this goal before beginning this ceremony.

Participants should:

• Measure growth

• Sync with the Scrum Master

• Sync with the Product Owner

Sprint planning happens just before the sprint starts, and usually lasts for an hour or two. In this meeting, the team goes over a collection of user stories and discuss, plan, measure, and prioritize. This is where they decide what is going to be in scope for their upcoming sprint cycle.

The Product Owner will have a prioritized view of things in the backlog. They work with the team on each object or customer experience. Together, as a group they go through and make calculations, deciding to what they can commit.

What’s a User Story and Why Does It Sound Like a Children’s Book?

So you might be wondering: how do you know what to work on? What to build? So much work, so little time? Thats where user stories come in.

In Scrum, teams don’t just write vague tasks like “code the login.” Instead, they write user stories – short, human-centered feature descriptions that describe what the user needs, why they need it, and what success looks like.

Here’s an example:

As a user, I want to be able to reset my password, so I can access my account if I forget it.

User stories are the scaffolding of teamwork. They’re written with empathy, not just efficiency. And each one comes with acceptance criteria – a checklist that clarifies what “done” actually means:

• A “Forgot Password” link is visible

• Clicking it shows a form

• An email gets sent with a reset link

Once a story is agreed upon, developers break it down into tasks, like “build form,” “hook into backend,” or “handle email validation.” It’s collaborative, not prescriptive. And user stories have priority so you know what’s the most important and what’s the least.

A helpful rule of thumb many teams use is the Gherkin-style "Given–When–Then" format:

• Given some initial context

• When an event occurs

• Then a specific outcome should happen

This ensures that everyone – devs, testers, and product owners – shares the same understanding of behavior and expectations.

Here is a great video example thats outlines how to draft effective and powerful user stories.

What Counts as “Done”? Definition of Done and Why It’s Important

Now you might be wondering – how do I know when a task is done and can be closed out?

The Definition of Done is a type of documentation in the form of a team agreement. The Definition of Done identifies the conditions that need to be achieved in order for the product to be considered done (as in potentially shippable).

This is how we know that we "did the thing right". Meaning, we built the correct level of quality into the product. The Definition of Done is not the same as the acceptance criteria, which are written by the product owner to help us know we did the "right thing".

Every team has a Definition of Done – it’s not just “I pushed code.” It could mean:

• Code is written

• Reviewed by a peer

• Merged into main

• Tested on staging

• Possibly deployed

This clarity keeps teams honest and accountable. No “it works on my machine” energy here. The DoD sets a quality bar. It prevents ambiguity, rework, and “it works on my machine” moments. When every card on the board passes the same finish line, teams move faster – and trust each other more.

Everyone should know what done is in a team. Either its Done as per DoD standards or its not.

Here is a beautiful video highlighting the impotence of DoD.

Demos, Retros, and Saying the Hard Things

Once you’ve built the product, then comes demos (showcasing your work) and retros (analysis as a team on what when well and what areas to improve on).

In the retro, everyone’s encouraged to speak up:

• What went well?

• What didn’t?

• What should we try next time?

Example:

“We missed a lot of stories because we didn’t account for testing time. Maybe we buffer next sprint with fewer tasks.”

The goal is not to blame – it’s to improve. Over time, this feedback loop becomes gold. The Scrum Master usually facilitates, collects feedback (via tools like Parabol, Miro, or sticky notes), and helps turn insights into actionable experiments for the next sprint.

Over time, retros become the heartbeat of team evolution.

Here is a video highlighting the importance of a Retro and Sprint Review.

🧠 Why Retrospection Matters More Than You Think

The Sprint Retrospective is more than just another meeting. It’s a mirror for your team – a safe, structured space to pause, reflect, and improve together.

You discuss:

✅ what went well

❌ what did not go well

🔁 what could we do better next time

Great teams don't just deliver great software, they continually deliver better ways of working.

This is why many experienced Scrum practitioners consider the retro to be the most important event in Scrum. Code is deployed once, but process improvements grow exponentially, sprint after sprint.

Tools You Might Encounter

Scrum doesn’t require software, but real-world teams use a variety of tools:

• Jira – Tracks sprints, issues, velocity

• Trello – Simple board, good for small teams

• Slack – Where standups often happen if async

• Notion / Confluence – Docs, retros, notes

• GitHub Projects – Lightweight planning for devs

Don’t worry if you’re not fluent in these yet. They’re tools – you’ll learn them on the job.

If You’re Preparing for a Job, Here’s What You Can Do

• ✍️ Practice writing user stories from your side projects

• 🧪 Run a mini-sprint: Plan your weekend project, set goals, and “review” it at the end

• 🤝 Contribute to an open-source project that uses Scrum or Agile workflows

• 🧾 Write about what you learned – maybe as a tutorial (hint hint)

Final Thoughts

So to recap, Scrum is a simple yet powerful way for teams to work together, stay organized, and deliver results quickly. It runs in short cycles called sprints, where the team plans what to do, checks in daily, shows their progress at the end, and reflects on how to improve.

The four key ceremonies – Sprint Planning, Daily Scrum, Sprint Review, and Sprint Retrospective – help keep everyone aligned and focused. With clear roles and regular feedback, Scrum makes it easier to handle changes, solve problems early, and continuously get better as a team.

But scrum isn’t a magic spell. It’s just a way for humans to build complex things – together – without falling apart.

You don’t need to be a Scrum Master. You don’t need a certification. But if you understand how sprints work, what’s expected of you, and how to show up to meetings with clarity and candor, you’re 10 steps ahead of most.

Scrum helps teams talk, plan, build, and learn. And now? You can too.

If you liked this, please do share. You never know who it might help out.

Until then…keep learning, unlearning, and relearning!!!

The Continuous Integration (CI) part of CI/CD means that developers can always integrate or merge their changes into the shared repository without breaking anything. Continuous Delivery, on the other hand, means that the code changes are automatically prepared for release after testing and validation.

CI/CD primarily involves various stages like building, testing, staging and deployment.

• Build phase: This is where the code and its dependencies are compiled into a single executable. This is the first phase of Continuous Integration, and is triggered by an event like pushing code to the repository.

• Test phase: Here, the built artifacts are tested to be sure that the code runs as expected.

• Staging: Here, the application is run in a production-like environment so as to be sure it is production ready.

• Deployment: Here, the application is automatically deployed to the end-users.

In this article, I’m going to explain how GitHub Actions works. I’ll also talk about basic GitHub Actions concepts, and then we’ll use it to build an example CI/CD pipeline.

What is GitHub Actions?

GitHub Actions is a service or feature of the GitHub platform that lets developers create their own CI/CD workflows directly on GitHub. It runs jobs on containers hosted by GitHub. The tasks are executed as defined in a YAML file called a workflow. This workflow file has to live on the .github/workflows folder on the repository for it to work.

Basic GitHub Actions Concepts

GitHub Actions consists of events, jobs, tasks, runners, workflows, and various other features. Here is a brief explanation of the main concepts:

Events: An event is basically something that happened. With GitHub, an event can be a push (when you push your code to the repository), a pull request, or even a cron job. These events trigger the CI/CD process.

Tasks: When you use CI/CD, you want to be able to trigger an activity that should be done automatically. That activity is known as a task or step in GitHub. It could be building your code or testing it or deploying it.

Each of those tasks has to be defined by commands. A GitHub Actions task usually consists of the name, and the instructions on what to do in the form of a command which starts with - run: or an Action which starts with - uses:.

steps:
  - name: Checkout code
    uses: actions/checkout@v3

  - name: Set up Node.js
    uses: actions/setup-node@v3
    with:
      node-version: 16

  - name: Install dependencies
    run: npm install

  - name: Run tests
    run: npm test

  - name: Build project
    run: npm run build

  - name: Deploy
    run: echo "Deploy step goes here"

Runner: A GitHub runner is a server that runs your tasks. It executes what is defined in your GitHub workflow. You can use your own runners or you can use the GitHub runners.

Job: A job is a collection of steps that are being executed on the same runner. Jobs are defined in a file called the workflow.

Workflow: The GitHub workflow is a series of jobs defined in a YAML file, that are triggered upon an event. The events do not trigger individual tasks. They can only trigger workflows. Then the tasks in the jobs of the workflow are executed.

Contexts: These provide a way to access information about workflows, jobs, and environments in GitHub. They are accessed with the expression $${{ <context> }}. Examples include github, env, vars, and secrets. The github context is used to access information about the workflow. For example:

$${{github.repository}} # should tell the name of the repository

$${{github.actor}}  # should tell the username of user that initially triggered the workflow

Secrets: This is used to store and access sensitive information that’s used by, and is available to, the workflow. Secrets are redacted when printed to the log. An example is $${{secrets.GITHUB_TOKEN}}.

How to Build a Simple CI/CD Pipeline

Here, we’re going to build an example workflow to deploy a simple HTML and CSS website to GitHub Pages. Follow the steps below:

1. Go to the sample code in my repository and fork it from here.

2. Go to the settings tab in the GitHub repository:

3. Go to the Pages settings:

   

4. Set the deployment source to the main branch:

   

5. Go to the General Actions settings and scroll down to the bottom:

6. 

   At the bottom, set the Workflow permissions to read and write:

7. In the GitHub repository, you can clone it to your PC or press the fullstop (.) on your keyboard to open GitHub Codespaces, the online version of VS Code.

8. Go to the sidebar and click on create a new file:

   

9. Create a workflows folder and file. You can call it deploy.yaml.

   

10. Copy this code into the file:

name: Deploy Static HTML and CSS to GitHub Pages

# Trigger the workflow on push to the main branch

on:
  push:
    branches:
      - main
# Define what operating system the job should run on
jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: write

    steps:
    # Step 1: Checkout the repository
    - name: Checkout Code
      uses: actions/checkout@v4

    # Step 2: Check the files that have been checked out
    - name: Display files
      run: ls

    # Step 3: Deploy to GitHub Pages
    - name: Deploy
      uses: peaceiris/actions-gh-pages@v4
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        publish_dir: ./ # The HTML and CSS files lie in the root directory, hence that should be the publish directory

11. Commit the code. You should see the job running when you go back to the repo:

When you’re done, go back to the home page of the repository and click on the Deployments section. There, you will see the GitHub Pages link to the deployment:

When you’re done, your repository should look like this.

Conclusion

In this article, you learned about how the CI/CD process works. We also covered the basic concepts of GitHub Actions. Finally, we created an example CI/CD pipeline with GitHub Actions. If you enjoyed this article, share it with others. You can also reach me on LinkedIn or X.

Also, writing quality and performant code helps your program or website work as expected – which should be every developer's goal.

SonarLint is a tool that helps you make sure your code is top-notch. It's like having a friendly guide who checks your code to see if it's well-written and doesn't have mistakes.

What is SonarLint?

SonarLint is an open-source code analysis tool that helps you find and resolve security and code quality problems in your source code as you're writing it. This plugin works with several integrated development environments (IDEs), including well-known ones like IntelliJ IDEA, Eclipse, and Visual Studio.

SonarLint's main purpose is to give you immediate feedback on possible problems with your code, including security flaws, bugs, and other recommended practices for programming. SonarLint analyzes code in the background while you create or change it in your IDE, giving you instant feedback and frequently exposing problems right in the code editor.

SonarLint is a component of the larger SonarQube ecosystem.

In this article, I'll teach you how to use SonarLint to help you write quality code.

Why is SonarLint Useful?

Let's imagine that building a website is like constructing a house. You want your house to be safe and well-designed, right? Well, SonarLint is like having a thorough inspector who checks your work as you build, making sure everything is just right.

Here's why SonarLint is important in web development:

1. Catch Mistakes Early (Code Quality): Assuming you were building a staircase, and accidentally put a step in the wrong place, SonarLint is like a smart friend who tells you immediately, "Hey, you might have made a mistake here!" It helps catch small errors in your code before they become big problems.

2. Follow the Blueprint (Coding Standards): When building a house, you follow a blueprint to make sure everything fits together. In coding, there are also rules (like a blueprint) on how to write good code. SonarLint helps you stick to these rules, making your code easy to read and work with.

3. Keep It Secure (Security): Just like you'd want your house to have good locks on the doors, you'd want your website to be secure. SonarLint checks your code for potential security issues, ensuring there are no "unlocked doors" that could let bad things happen.

4. Work Together Well (Collaboration): Imagine that each builder in your team used a different kind of tool. It would be chaos! SonarLint helps your team work together smoothly by making sure everyone follows the same coding standards. This way, everyone can understand and contribute to the project easily.

5. Save Time and Effort (Efficiency): Fixing mistakes after the whole house is built would take a lot of time and effort. SonarLint helps you fix issues as you go, saving you from returning and redoing things. It's like having a helpful friend who stops you from making mistakes in the first place.

6. Learn and Improve (Education): SonarLint does not only point out mistakes but also explains why they might be a problem. It's like having a coding teacher who helps you understand how to write better code. This way, you learn and become a better developer over time.

So, in the world of web development, SonarLint is your coding buddy, and it makes sure your JavaScript "house" is strong, secure, and well-organized from the ground up. It's a valuable tool in your workflow and helps you create high-quality websites that everyone can enjoy.

But you still may be wondering why you need this tool. You have a debugger already installed in your IDE and it can already track the errors in your environment.

Well, integrating SonarLint complements your debugger by focusing on code quality and security during development.

While a debugger helps to find and fix runtime issues, SonarLint analyzes code in real time, identifies bugs and potential vulnerabilities, and enforces coding standards.

You can also customize and configure coding rules based on your project's specific requirements and coding standards.

This proactive approach enhances overall code quality and ensures cleaner, more maintainable code before it reaches the debugging stage. This leads to fewer errors and smoother development workflows.

Before we get into the details of how to set up and use SonarLint, let's look at what makes code high-quality.

Code Quality Metrics

Be the best cookbook author! When writing code there are specific guidelines you must follow.

Just like when you're cooking up a new recipe or want to follow a traditional one, you should make sure anyone who reads it can follow along, and that the recipe you wrote down results in a good dish.

In the same sense, when writing code, it's always important to make your code readable so that other developers can understand it easily and so your code works as it's supposed to. Code quality metrics are like measuring how well you followed the recipe.

Here's a breakdown:

Readability (Clarity): This is similar to making sure your recipe instructions are clear. The code should be easy for others (or future you) to understand.

Maintainability (Ease of Changes): If you had to change ingredients in your recipe (code), it should be easy to switch things without chaos.

Performance (Speed): As you'd want your meal ready quickly, efficient code is expected to run fast. Code quality metrics check how fast your code is executed.

Reliability (Consistency): A good recipe always tastes the same. Similarly, reliable code consistently produces the correct results.

Security (Safety): Just like checking if ingredients are safe to eat, code quality metrics look for potential dangers in your code that could be exploited.

These should be your goals when writing quality code. And SonarLint can help you accomplish them.

How to Set Up SonarLint and Integrate with Your IDE

An IDE (Integrated Development Environment) is a software application that helps developers write and debug code more effectively. IDEs include a code editor and compiler or interpreter.

In this article, you'll see how to install SonarLint using VS Code extensions.

How to Install SonarLint with VS Code

First, install VS Code or open the application if you've already installed it.

Next, head over to the Extensions tab on VS Code and download the SonarLint extension.

To use the SonarLint extension for JavaScript, TypeScript, or CSS, you should have a minimum version of 14.17.0 of Node.js installed on your system (especially if you want to use Connected Mode with Sonar Cloud).

Install SonarLint in VS Code

Now that you've installed SonarLint, running an analysis on your code should be easy and it should start immediately after you open a new file.

This means SonarLint will start working and catching errors in your code as you write them on your IDE. Now let's look at an example in the next section.

How to Use SonarLint in Your IDE

Now let's look at how you can get the best out of SonarLint on your IDE. SonarLint is also a wonderful teacher, helping you better understand how to write clean code and giving you more information about why you have an error.

So instead of scouring the web to figure out what's wrong with your code, SonarLint explains why it has given you an error.

Here's an example of how to use it.

To be able to see the interface where we'll be working, open your terminal on VS Code – you can use Ctrl + backtick (`) to do this. I'm currently working on a project in React.js, and I didn't notice that I have duplicate border property names in my CSS class. Luckily, SonarLint caught this issue.

A CSS error caught by SonarLint

If you click on the light bulb, you'll see an option to fix the code by deactivating the CSS rule or to be blunt removing the extra border which I have in my CSS class in this instance. There's also another option for opening a description rule so you can understand why you're getting that error.

So SonarLint gives you two options:

1. It offers an option that gives you the ability and resources to understand why you have that error with the "Open description of rule".

2. It offers a solution to the error of the code which it has found.

Options offered by SonarLint to help fix and understand this problem

If you click on the open description rule, SonarLint opens another tab in VS Code to help you understand why it has thrown that error and how you can write cleaner/better code.

Tab opened by SonarLint in VSCode to help understand the error

As you can see, SonarLint is an excellent teacher and your best buddy as you continue to build and work on your IDE.

Another great thing about SonarLint is that it works with multiple programming languages. So whatever programming language you're using, chances are that SonarLint can handle it.

Conclusion

SonarLint is like having a coding buddy that helps you become a better programmer. As you write code, it's like having a teacher right there with you, pointing out ways to improve.

Imagine having a super-smart friend who does not just spot mistakes but also explains why they're wrong and shows you how to fix them.

Before your code gets to the testing stage, SonarLint checks for mistakes and makes sure your program runs smoothly. It's like having a superpower to catch issues early, saving you time and effort.

But that's not all! SonarLint is also like a security guard for your code. It watches out for any potential weak points that could let bad things happen. By learning from SonarLint, you can write better code and become more aware of keeping your code safe and secure.

I'll describe all the key elements of a good playbook in just a moment. But it's important to emphasize that a playbook on its own is more or less useless unless your team is able to read it and convert it into real-world results.

To do that you'll need to make sure every relevant member of your team is completely familiar with their roles and how they'll be expected to carry them out. That'll require you to distribute copies of the plan and ensure that everyone gets the training they'll need to perform perfectly when the time arrives.

This article comes from my Introduction to Cybersecurity course. If you'd like, you can follow the video version here:

How to Define Playbook Scope

At any rate, a good plan begins with clear definitions:

• Where can you find up-to-date and clean copies of the source code?
• Where should your production environment be hosted? In a public cloud like AWS? On-premises?
• What is the infrastructure supposed to accomplish?
• What's the scope of your operation: what scale of hardware resources will it require?

A playbook should also clearly define the policies that must be followed through the rebuilding process:

• How is organizational data to be protected?
• What decisions must be made only by senior company officers?
• Are there restrictions on what software and third party solutions can be used...or from which countries they can be acquired?
• Are there stack components that must remain local, or can everything live in the cloud?

How to Define the Tools

Perhaps the core of any playbook is the section addressing the software and deployment tools and procedures that you'll use at every stage of your workflow.

This section should include the complete code for the scripts handling moving resources from code to deployment, along with links to all the software code in use, and instructions for authenticating to the services you'll be using.

How to Define the Participants

IT deployments are performed by people. But which people?

• Who do you speak to who has access to a credit card so you can purchase needed resources?
• Who has access to the key codebases and online accounts you'll need?
• Who's responsible for testing and signing off before code is pushed to production?
• What if that person isn't available?

Each and every role relating to the project you're documenting needs to be defined, and the person responsible must be identified – along with current contact information.

Beyond operational contacts, the playbook should also include a complete company communications directory. If you're paying someone a paycheck each month, the odds are they'll be expected to perform some important function during a recovery. So you'll want a reliable source for contact information – preferably containing multiple contact endpoints for each person.

How to Document Your Recovery

Recovery operations can be chaotic. But it's nevertheless critically important that log records for every step – pre-, post-, and during recovery – should be kept. So log generation and storage should also be part of your playbook.

Even if you don't have the time to read them right now, they'll be invaluable later as you try to review events and figure out exactly what happened. The existence of accurate and reliable logs and other records might actually be legally mandated.

Any code review and application testing you would normally incorporate in your deployment lifecycles should be included in your recovery playbook. After all, bugs and failures aren't going to be any more fun after a crisis than they were before it. The actual code for all the scripts that would normally power your testing should be included here, too.

How to Keep Your Playbook Current

Finally, your playbook should be regularly updated to reflect changes to your application and its supporting environment. Naturally, you want to keep all details up-to-date, including changes to the personnel responsible for specific roles, along with their correct contact information.

A complete playbook created for a relatively complex operation can easily run into the hundreds of pages. When you add the task of coordinating the actions of all the many individuals who will be involved in your recovery, the whole thing might feel a bit hard to manage. Unfortunately, you just have to do this: there's really no alternative.

How to Automate Your Recovery

Well, there's almost no alternative. Remember how I told you that you should include complete operations scripts and links to your code base in the playbook? Do you think our playbook could be convinced to play itself? Why not?

Think about it. Orchestration tools like Ansible or Terraform – or cloud-specific tools like Amazon's CloudFormation – allow you to very closely define every layer of your infrastructure in a format that can be invoked and launched with a single command.

In theory at least, there's no reason why you couldn't build your playbook as an actual script, complete with commands to pull software repos, launch complex virtual networks and compute instances, and route DNS domains. That would be a fantastic example of the power of infrastructure as code.

How to Test Your Playbook

While we're still on the topic of plans and playbooks, I should add one more very important note. If you're going to go to all the trouble of researching and then writing a playbook, you don't want to discover in the middle of a crisis that your plans don't actually work.

The safe assumption is that nothing in technology will work unless it's been carefully and repeatedly tested in advance. That's true of recovery playbooks, and it's just as true of backups: until you've successfully restored a backup archive into a real environment, you should assume it'll fail.

With what you've now seen about the scope, tools, documentation, updates and automation for playbooks, you're now all set to get to work creating your own. Well don't let me get in your way!

This article comes from my Introduction to Cybersecurity course. And there's much more technology goodness available at bootstrap-it.com

Working as a Software Developer, there are always new tools and frameworks coming out that can completely change your workflows – for the better (or worse?).

Either way, there is always the possibility to optimize how you do things day-to-day.

This article contains some Linux utilities that have recently replaced my overused and under-productive daily programming workflow.

You will learn about those utilities and how they are a better alternative to their counterparts.

Let's begin.

How to Use Mcfly

Do you relentlessly hit up arrow on the terminal until you get the command you ran previously? I've been there. I didn't know I could optimize this, so I was using it religiously for quite some time.

Then I got introduced to ctrl + r. It allows you to search through your command history and has wildcard search. Wow.

Results? My finger tapping exercise was over from the first day. Again, I thought this must be the peak DX for such a small utility. I was so wrong.

So what? Well, there's a better ctrl-r for you. Introducing Mcfly 🦋.

In addition to the regular ctrl+r feature, it has some extra niceness:

1. The suggestions are customized using a neural network which considers your current working directory and recently executed commands.

2. It tracks exit status of commands (you probably don't want to run a failed command again), timestamp, and other useful information.

3. You can use % as a wildcard to match multiple characters.

Here are the suggestions I got on two different repositories, based on my shell history:

suggestions include a build script specific to this project

project has a different build script

You can install Mcfly from here.

How to Use Cheat.sh

Who loves reading man pages? I don’t. When I’m struggling with a command, the last thing I want to read is a man page. Not because it's not helpful, but it is overwhelming.

I often just need quick examples that I can grab on the go and use. When I found TLDR pages, I was the happiest person. Now with Cheat (sheets), I’m even happier.

Cheat gives you access to the cheatsheets for potentially every command you will ever need — only examples, without the encyclopedia.

If you do not want to install the utility, you can get the cheatsheet using CURL like this:

curl cheat.sh/uptime

So instead of installing the cheatsheets on your machine, you are fetching the information for only the command you need. You can visit cheat.sh and use it on your browser as well.

Here is how the output of the above command looks:

Cheatsheet output for uptime command

You will find a lot of examples in the codebase.

How to Use Git Open

I often need to open the GitHub repository of the project I'm working on in the browser. It may be to check for comment updates on the Pull Request I raised, change repository settings, or just about anything that will require the GitHub repository page.

Well, we have a utility even for this!

Running git open will open your current working repository on your browser. By default, it opens the remote page for the branch you are on. You can even go ahead and create an alias for the commands to avoid typing the whole thing.

Here are some of alias ideas for you:

alias go="git open"
alias blog="git open https://github.com/<username>/blog <branch>"

Check out Git Open on Github here.

How to Use Bat

We have all done cat, right? Bat is just that but has syntax highlighting, nice formatting and styling options, and git diff support. It is very versatile, integrates easily with other tools, and provides custom theming options as well.

Let's a take a look. This is our express server file using cat:

output of cat command

The above output has no syntax highlighting which decreases the readability of the code. Let's do the same using bat:

output of bat command

This is clearly more readable. It has the appropriate syntax highlighting automatically applied (without any config), provides the file name, and line numbers.

Feel free to start using it here.

How to Use Jq

Jq is a command-line processor for JSON. You can slice and dice your JSON, perform projection to only show certain fields, and extract only the required information from a (huge) JSON. No more overwhelming the terminal output.

[
  {"value": 1, "rating": 2 },
  {"value": 2, "rating": 4 },
  {"value": 3, "rating": 5 }
]

Accessing a key from an array looks like this:

jq '.[0] | { value }'

We are asking for the first element in the array and projecting only the value field:

{
  "value": 1
}

To learn more, head over to their official tutorial.

They also have a handy online playground. I have created a snippet here. Feel free to tweak it and play around.

You can even take it a step further with jid. It is an interactive JSON digger that leverages Jq. It provides you very convenient suggestions and an auto-complete feature.

Conclusion

These were some of the utilities that expanded the horizon for me and made me realise that there is always a better way to do things. You just need to keep Googling. Starting with "how to do X" and "better alternatives to X".

I use these utilities quite often in my daily programming workflow. I hope at least one of them will be useful to you.

I would love to know what utilities are crucial to your daily workflow – do you use any of the ones mentioned in this article?

Liked the article? Get biweekly improvement pills on backend web development 💌.

A workflow as a sequence of operations, from start to finish. The workflows in Airflow are authored as Directed Acyclic Graphs (DAG) using standard Python programming.

You can configure when a DAG should start execution and when it should finish. You can also set up workflow monitoring through the very intuitive Airflow UI.

You can be up and running on Airflow in no time – it’s easy to use and you only need some basic Python knowledge. It's also completely open source.

Apache Airflow also has a helpful collection of operators that work easily with the Google Cloud, Azure, and AWS platforms.

In this article we are going to cover

• What are Directed Acyclic Graphs (DAGs)?

• What are Operators?

• How to Create your First DAG

• A Use-Case for DAGs

• How to Set Up Cloud Composer

• How to Run the Pipeline on Composer

What are Directed Acyclic Graphs, or DAGs?

DAGs, or Directed Acyclic Graphs, have nodes and edges. DAGs should not contain any loops and their edges should always be directed.

In short, a DAG is a data pipeline and each node in a DAG is a task. Some examples of nodes are downloading a file from GCS (Google Cloud Storage) to Local, applying business logic on a file using Pandas, querying the database, making a rest call, or uploading a file again to a GCS bucket.

Visualizing DAGs

Correct DAG with no loops

Incorrect DAG with Loop

You can schedule DAGs in Airflow using the schedule_interval attribute. By default it’s "None" which means that the DAG can be run only using the Airflow UI.

You can schedule the DAG to run once every hour, every day, once a week, monthly, yearly or whatever you wish using the cron presets options (@hour, @daily, @weekly, @hourly, @monthly, @yearly).

If you need to run the DAG every 5 mins, every 10 mins, every day at 14:00, or once on a specific day like every Thursday at 10:00am, then you should use these cron-based expressions.

*/5 * * * * = Every 5 minutes

0 14 * * * = Every day at 14:00

What are Operators?

A DAG consists of multiple tasks. You can create tasks in a DAG using operators which are nodes in the graph.

There are various ready to use operators available in Airflow, such as:

• LocalFilesystemToGCSOperator – use it to upload a file from Local to GCS bucket.

• PythonOperator – use it to execute Python callables.

• functionEmailOperator – use it to send email.

• SimpleHTTPOperator – use it to make an HTTP Request.

How to Create Your First DAG

The example DAG we are going to create consists of only one operator (the Python operator) which executes a Python function.

from airflow import DAG
from datetime import datetime
from airflow.operators.python_operator import PythonOperator

def message():
    print("First DAG executed Successfully!!")

with DAG(dag_id="FirstDAG", start_date=datetime(2022,1,23), schedule_interval="@hourly",
         catchup=False) as dag:

    task = PythonOperator(
        task_id="task",
        python_callable=message)

task

The first step is to import the necessary modules required for DAG development. The line with DAG is the DAG which is a data pipeline that has basic parameters like dag_id, start_date, and schedule_interval.

The schedule_interval is configured as @hourly which indicates that the DAG will run every hour.

The task in the DAG is to print a message in the logs. We have used the PythonOperator here. This operator is used to execute any callable Python function.

Once the execution is complete, we should see the message “First DAG executed Successfully” in the logs. We are going to execute all our DAGs on GCP Cloud Composer.

Airflow UI

After successful execution, the message is printed on the logs:

Logs

A Use-Case for DAGs

The use-case we are going to cover in this article involves a three-step process.

In step one, we will upload a .csv file in some input GCS bucket. This file should be processed by PythonOperator in the DAG. The function which will be executed by the PythonOperator consists of Pandas code, which represents how users can use Pandas code for transforming the data in the Airflow Data Pipeline.

In step two, we'll upload the transformed .csv file to another GCS bucket. This task will be handled by the GCSToGCSOperator.

Step three is to send the status email indicating the that the pipeline execution is completed which will be handled by the EmailOperator.

In this use-case we will also cover how to notify the team via email in case any step of the execution failed.

How to Install Cloud Composer

In GCP, Cloud Composer is a managed service built on Apache Airflow. Cloud Composer has default integration with other GCP Services such as GCS, BigQuery, Cloud Dataflow and so on.

First, we need to create the Cloud Composer Environment. So search for Cloud Composer on the search bar and click on "Create Environment" as shown below:

Create Environment

In the Environments option, I am selecting the "Composer 1" option as we don’t need auto-scaling.

Once we select the type of composer we need, we'll need to do some basic configuration just like in any GCP managed service ("Instance Name", "Location", and so on).

The node count here should always be 3 as GCP will setup the 3 services needed for Airflow.

Once we're done with that, it'll set up an Airflow instance for us. To upload a DAG, we need to open the DAGs folder shown in ‘DAGs folder’ section.

Airflow Instance

If you go to the "Kubernetes Engine" section on GCP, we can see 3 services up and running:

Kubernetes Engine

All DAGs will reside in a bucket created by Airflow.

Airflow Instance bucket for DAGs

How to Create and Run the Pipeline on Composer

In the Pipeline, we have two buckets. input_csv will contain the csv which requires some transformation, and the transformed_csv bucket will be the location where the file will be uploaded once the transformation is done.

The entire pipeline code is the following:

from airflow import DAG
from datetime import datetime
import pandas as pd

from airflow.utils.email import send_email
from airflow.operators.python_operator import PythonOperator
from airflow.operators.email_operator import EmailOperator
from airflow.providers.google.cloud.transfers.gcs_to_gcs import GCSToGCSOperator


def transformation():
    trainDetailsDF = pd.read_csv('gs://input_csv/Event_File_03_16_2022.csv')
    print(trainDetailsDF.head())


with DAG(
        dag_id="pipeline_demo",
        schedule_interval="@hourly",
        start_date=datetime(2022, 1, 23),
        catchup=False
) as dag:
    buisness_logic_task = PythonOperator(
        task_id='ApplyBusinessLogic',
        python_callable=transformation,
        dag=dag)

    upload_task = GCSToGCSOperator(
        task_id='upload_task',
        source_bucket='input_csv',
        destination_bucket='transformed_csv',
        source_object='Event_File_03_16_2022.csv',
        move_object=True,
        dag=dag
    )

    email_task = EmailOperator(
        task_id="SendStatusEmail",
        depends_on_past=True,
        to='youremail',
        subject='Pipeline Status!',
        html_content='<p>Hi Everyone, Process completed Successfully! <p>',
        dag=dag)

    buisness_logic_task >> upload_task >> email_task

In the first task, all we are doing is creating a DataFrame from the input file and printing the head elements. In the logs it looks like this:

DataFrame Head

In the second task, GCSToGCSOperator, we have used the attribute move_object=True which will delete the file from the Source bucket.

Once we upload the file to the bucket, we can see that the DAG is being scheduled. The name of the DAG is "pipeline_demo".

DAGs

Note that in case if you encounter any "import errors" after uploading or executing a DAG, something like this:

You can upload these missing packages through the "PYPI Packages" option in GCP. This will update the environment after few minutes.

Updating environment with missing Packages

To open an Airflow UI, Click on the "Airflow" link under Airflow webserver.

Airflow Instance, click Airflow link to Open UI

The Airflow UI looks like this:

Upon successful execution of Pipeline, here's what you should see:

In order to send email if a task fails, you can use the on_failure_callback like this:

def notify_email(contextDict, **kwargs):
    title = "Airflow alert: {task_name} Failed".format(**contextDict)
    body = """
    Task Name :{task_name} Failed.<br>
    """.format(**contextDict)
    send_email('youremail’, title, body)



buisness_logic_task = PythonOperator(
    task_id='ApplyBusinessLogic',
    python_callable=transformation,
    on_failure_callback=notify_email,
    dag=dag)

We're doing the notification email configuration on composer through Sendgrid. Also, once you are done with Cloud Composer, don't forget to delete the instance as it cannot be stopped.

Conclusion

Apache Airflow is a fairly easy-to-use tool. There's also a lot of help now available on the internet and the community is growing.

GCP simplified working with Airflow a lot by creating a separate managed service for it.

My decade-long journey as a web developer may not be very different from many of your journeys. But lately, I have been more focused on getting things done quickly with quality. It is not only about what and how we code. It is also about how we test, debug, and finally ship/deploy the application.

In this article, you will learn a few web development workflows that can help you develop, test, build, and deploy faster.

Browser Console – A Web Dev's Best Friend

In my early days of development, the browser's console used to be my best friend (and it still is today).

You can try out all your JavaScript logic, code snippets, and code ideas directly in the console. It works like a charm when you want to try out proofs-of-concept of your logic in the console before putting the code in the app.

JavaScript Program in the Browser's Console

You can launch the DevTools (for Chrome, and available in other browsers, too) using the F12 key from your favorite browser. Next, you can navigate to the console tab to start writing and experimenting with the code.

That's for JavaScript logic, though...

The browser's console is a great way to try out the logic part, but it is not sufficient to build and test a web application on the go.

If you are getting started with any popular framework or library like Angular, React, Vue, or Svelte, they've got you covered. You will have a complete infrastructure to develop, test, run, and see the changes almost in realtime.

However, suppose you are doing a project with plain JavaScript, HTML5, and CSS (I'm in love with this combo). In that case, you may not need any particular infrastructure.

Here are a couple of options to get you started.

VS Code + Live Server or Anything Equivalent

Suppose you are using Visual Studio Code as your preferred editor/IDE for web development. In that case, you can install an extension called Live Server. It launches a local development Server with a live reload feature for static and dynamic pages.

Once installed, you can right-click on the entry point HTML file (usually an index.html file) of the app and open it with Live Server.

Launch the App using the Live Server

Alternatively, you can try the Go Live option from VS Code's status bar.

Alternate way to Go Live

This workflow works really well to develop and test your web app incrementally. It provides hot reloading, so you do not need to refresh the browser to see the latest changes. It happens automatically.

In case you are not using VS Code, there might be other alternatives for your favorite IDE.

With the npx serve Command

The npx stands for Node Package Execute. It comes with npm to execute any package you want from the npm registry without installing it locally. serve is a utility to help you serve a static site, single-page app locally.

To use it, open a command prompt at the root of the project folder and type this:

npx serve

You will get a URL to access the app locally.

Using npx serve

Both of these methodologies are very helpful to run and test the app locally. But you may need a bit more sophisticated workflow to build and deploy the app to test the production version simultaneously.

A More Sophisticated Workflow

The last thing you want is for your application to not behave the same way in the production as it does locally. It could be a nightmare if you discover it close to deadline.

A better approach would be to develop functionality, test it locally, and then deploy and preview it on a production-like infrastructure. If all goes well, then you can launch it to production. You can do all that in a fully automated way within a few minutes.

I have started following a workflow that helps me to achieve these tasks:

Some fantastic cloud-based hosting services are available that help you remove many complexities with building, deploying, and hosting your web apps. Here are a few you can check out (my favorites are Vercel and Netlify):

• Heroku
• Netlify
• Vercel
• Surge
• Firebase
• Fly

It is standard practice to manage and track your source code changes using a Version Control System like Git and an app like GitHub. If you do not have an account on GitHub, you can create one now.

If you are looking for a GitHub repo guide to increase engagement on your public repositories, you can read this article I wrote:

Alright, coming back to our topic. The hosting services mentioned above have integrations with GitHub. It means you can connect a particular repository in GitHub to build the source code from it and deploy it to a CDN by the hosting services.

The best part is, all these happen automatically every time you push your commits to a repository branch. For each pull request, you get a preview URL to test your changes in a production-like infrastructure. Isn't that cool?

Once you're satisfied with testing, you just merge the pull request to the master (or main) branch to build and deploy your app/site from there.

So, what does the workflow look like at a high level? Take a look at the image below:

An Automated Workflow

• We, as developers, create a GitHub repository for our source code and commit, push the initial code to it.
• We perform a few easy steps with the hosting provider (like Vercel, Netlify) to integrate the GitHub repository.
• Next, we create branches in the repository to implement features or fix bugs. Netlify, Vercel, or other services will create a deployable build, deploy it on a CDN, and provide us a preview URL for each branch.
• We then use this URL to test the feature. We can also share this URL publicly with our stakeholders.
• Once we're done, we merge the PR, and the changes in the master or main branch are also built and deployed automatically.

This workflow helps me the most with my web development. I just focus on delivering my code changes to GitHub, and the rest all happens automatically.

Let's now go through the quick steps of deploying a simple web app on Vercel. It's pretty similar to other platforms which I'll leave for you to explore.

How to Deploy a Web App on Vercel

• Sign up with Vercel and login. Then click on the New Project button:

Select a New Project

• Next, configure and import a GitHub repository:

Configure GitHub Repo

• Now install Vercel on your GitHub organization where this repo exists.

Install Vercel on GitHub

• You'll need to provide access to all the repositories or select the ones you want Vercel to manage. Then save the changes.

Select Repository

• Now control comes back to the Vercel interface and you can import the repository from there.

Import the Selected Repo to Deploy

• You can select the entry point of the project root directory and deploy.

A Basic Configuration

• The deployment may take a while depending on the type of your web project. Once it's done, you will get a screen like the below confirming it.

Done!

Great, I hope that was easy. Please explore it further, and I'm sure you will love it.

Before We End...

I hope you found the article insightful and informative. My DMs are open on Twitter if you want to discuss further.

Let's connect. I share my learnings on JavaScript, Web Development, and Blogging on these platforms as well:

• Follow me on Twitter
• Subscribe to my YouTube Channel
• Side projects on GitHub

A workflow, or a pattern of the work, gives you a standardized way to do something.

It is extremely helpful in a team, and can benefit you even if you work individually. A good workflow enables you to streamline a process, organize it, and make it less error-prone.

This article summaries several approaches when working with Terraform, both individually and in a team. I tried to gather the most common ones, but you might also want to develop your own.

The common requirement for all of them is a version control system (such as Git). This is how you ensure nothing is lost and all your code changes are properly versioned tracked.

Table of contents:

• Basic Concepts
• Core individual workflow
• Core team workflow
• Team workflow with automation
• Import workflow

Basic Concepts

Let’s define the basic actions first.

All described workflows are built on top of three key steps: Write, Plan, and Apply. Nevertheless, their details and actions vary between workflows.

Write – this is where you make changes to the code.

Plan – this is where you review changes and decide whether to accept them.

Apply – this is where you accept changes and apply them against real infrastructure.

It's a simple idea with a variety of possible implementations.

Core individual workflow

This is the most simple workflow if you work alone on a relatively small TF project. This workflow suits both local and remote backends well.

Write

You clone the remote code repo or pull the latest changes, edit the configuration code, then run the terraform validate and terraform fmt commands to make sure your code works well.

Plan

This is where you run the terraform plan command to make sure that your changes do what you need. This is a good time to commit your code changes changes (or you can do it in the next step).

Apply

This is when you run terraform apply and introduce the changes to real infrastructure objects. Also, this is when you push committed changes to the remote repository.

Core team workflow

This workflow is good for when you work with configuration code in a team and want to use feature branches to manage the changes accurately.

Write

Start by checking out a new branch, make your changes, and run the terraform validate and terraform fmt commands to make sure your code works well.

Running terraform plan at this step will help ensure that you'll get what you expect.

Plan

This is where code and plan reviews happen.

Add the output of the terraform plan command to the Pull Request with your changes. It would be a good idea to add only the changed parts of the common output, which is the part that starts with "Terraform will perform the following actions" string.

Apply

Once the PR is reviewed and merged to the upstream branch, it is safe to finally pull the upstream branch locally and apply the configuration with terraform apply.

Team workflow with automation

In a nutshell, this workflow allows you to introduce a kind of smoke test for your infrastructure code (using plan) and also to automate the feedback in the CI process.

The automated part of this workflow consists of a speculative plan on commit and/or Pull Request (PR ), along with adding the output of plan to the comment of the PR. A speculative plan mean just to show the changes, and not apply them afterward.

‌‌

Write

This step is the same as in the previous workflow.

Plan

This is where your CI tool does its job.

Let’s review this step by step:

1. You create a PR with the code changes you wish to implement.
2. The CI pipeline is triggered by an event from your code repository (such as webhook push) and it runs a speculative plan against your code.
3. The list of changes (a so-called "plan diff") is added to PR for review by the CI.
4. Once merged, the CI pipeline runs again and you get the final plan that's ready to be applied to the infrastructure.

Apply

Now that you have a branch (i.e. main) with the fresh code to apply, you need to pull it locally and run terraform apply.

You can also add the automated apply here – step 5 in the picture below. This may be very useful for disposable environments such as testing, staging, development, and so on.

The exact CI tool to be used here is up to you: Jenkins, GitHub Actions, and Travis CI all work well.

An important thing to note is that the CI pipeline must be configured in a bi-directional way with your repository to get the code from it and report back with comments to PR.

As an option, you may consider using Terraform Cloud which has a lot of functionality, including the above mentioned repo integration, even with the free subscription.

If you have never worked with Terraform Cloud before and want to advice to get started, I'll provide the links at the end of this article.

Import workflow

‌‌This workflow refers to a situation when you have some objects already created (i.e., up and running), and you need to manage them with Terraform.

Suppose we already have an S3 bucket in AWS called "someassetsbucket" and we want to include it into our configuration code.‌‌

Prepare

You should create a resource block to be used later for the real object you’re going to import.

You don’t need to fill the arguments in it at the start, so it may be just a blank resource block, for example:

resource "aws_s3_bucket" "assets" {
‌‌
}

Import

Now you need to import the information about the real object into your existing Terraform state file.

This can be done with the terraform import command, for example:

terraform import aws_s3_bucket.assets "someassetsbucket"

Write

Now you need to write the corresponding Terraform code for this bucket.

To avoid modifying your real object on the terraform apply action, you should specify all needed arguments with the exact values from the import phase.

You can see the details by running the terraform state show command, for example:

terraform state show aws_s3_bucket.assets

The output of this command will be very similar to the configuration code. But it contains both arguments and attributes of the resource, so you need to clean it up before applying it.

You can use one of the following tactics:

• either copy/paste it, and then run terraform validate and terraform plan several times to make sure there are no errors like "argument is not expected here" or "this field cannot be set"
• or you can pick and write only the necessary arguments

In any case, be sure to refer to the documentation of the resource during this process.

Plan

The goal is to have a terraform plan output showing "~ update in-place" changes only.

However, it is not always clear whether the real object will be modified or only the state file will be updated. This is why you should understand how a real object works and know its life cycle to make sure it is safe to apply the plan.

Apply

This is usual the terraform apply action.

Once applied, your configuration and state file will correspond to the real object configuration.

Wrapping up

Here is an overview of Terraform Cloud for those who never worked with it before: ‌‌Overview of Terraform Cloud Features

And here is a nice tutorial to start with: Get Started - Terraform Cloud

Also, here is an overview of workflows at scale from the HashiCorp CTO which might be useful for more experienced Terraform users: Terraform Workflow Best Practices at Scale

Thank you for reading. I hope you will try one of these workflows, or develop your own!‌‌

‌‌

Can a Makefile improve your DevOps and keep developers happy? How awesome would it be if a new developer working on your project didn’t start out by copying and pasting commands from your README? What if instead of:

pip3 install pipenv
pipenv shell --python 3.8
pipenv install --dev
npm install
pre-commit install --install-hooks
# look up how to install Framework X...
# copy and paste from README...
npm run serve

… you could just type:

make start

…and then start working?

Making a difference

I use make every day to take the tedium out of common development activities like updating programs, installing dependencies, and testing.

To do all this with a Makefile (GNU make), we use Makefile rules and recipes. Similar parallels exist for POSIX flavor make, like Target Rules. Here’s a great article on POSIX-compatible Makefiles.

Here’s some examples of things we can make easier (sorry):

update: ## Do apt upgrade and autoremove
    sudo apt update && sudo apt upgrade -y
    sudo apt autoremove -y

env:
    pip3 install pipenv
    pipenv shell --python 3.8

install: ## Install or update dependencies
    pipenv install --dev
    npm install
    pre-commit install --install-hooks

serve: ## Run the local development server
    hugo serve --enableGitInfo --disableFastRender --environment development

initial: update env install serve ## Install tools and start development server

Now we have some command-line aliases that you can check in. Great idea! If you’re wondering what’s up with that weird ## comment syntax, it gets better.

A self-documenting Makefile

Aliases are great, if you remember what they all are and what they do without constantly typing cat Makefile. Naturally, you need a help command:

.PHONY: help
help: ## Show this help
    @egrep -h '\s##\s' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

With a little command-line magic, this egrep command takes the output of MAKEFILE_LIST, sorts it, and uses awk to find strings that follow the ## pattern. It then prints a helpful formatted version of the comments.

We’ll put it at the top of the file so it’s the default target. Now to see all our handy shortcuts and what they do, we just run make, or make help:

help                 Show this help
initial              Install tools and start development server
install              Install or update dependencies
serve                Run the local development server
update               Do apt upgrade and autoremove

Now we have our very own personalized and project-specific CLI tool!

The possibilities for improving your DevOps flow with a self-documenting Makefile are almost endless. You can use one to simplify any workflow and produce some very happy developers.

But we’re technologists and it’s 2020. How can we use tools like Zapier to make robots do these things for us?

• What is Zapier?
• What can we do with Zapier Zaps?
• Zap 1: Get a text if it’s going to rain with Zapier
• Zap 2: Print a test every week with Google Cloud Print
• Zap 3: Smashing Magazine Job Alerts with Gmail

What is Zapier?

Zapier is an automation tool that connects all of the apps you love and builds powerful fully automated workflows. Whether it’s automating sending an email or making sure that new blog post gets a tweet, we can remove the manual steps of mundane tasks to focus on other things that art important.

Each time you create a new workflow, you’re creating a “Zap”. It’s essentially Zapier’s way to give a name to the workflow you create.

What can we do with Zapier Zaps?

The brilliant part about Zapier is each app integration makes its API available via Zapier to other app integrations giving you a ton of options to connect and build powerful workflows.

Particularly, we’re going to learn how to do a few things:

• Sending a text every morning if it’s going to rain
• Set up a weekly print job to keep your ink fresh
• Receive emails for new jobs on Smashing Magazine’s job board

While each of these tasks are small, they end up saving you a lot of time. And if you’re creative, you can build upon these workflows to customize a whole lot more.

Getting started with a Zapier account

Before we get into setting up workflows, you’ll need an account.

Signing up for Zapier is free and you get 5 free Zaps to start, so we don’t have to worry about cost here.

Now let’s get into the Zaps.

Zap 1: Get a text if it’s going to rain with Zapier

To get an idea of how this works, we’ll start with something simple. We’re going to set up a Zap that will send us a text message if the weather predicts rain.

To get started, click the big Make a Zap button on the top left of the page when you’re logged into your account.

Making a new Zap

Here, Zapier wants to know the first app we want to connect. Since we’re going to base our Zap on the weather, search for “weather” and select Weather by Zapier.

Selecting the Weather by Zapier integration

It will then ask you to choose a Trigger Event, where you’ll select “Will It Rain Today?”, then you can click the Continue button.

Choosing the Will it Rain Today? event

When choosing Weather as an event, it requires a little bit of information to give us a personalized prediction. Particularly, it requires your Latitude and Longitude, which we can look up using latlong.net.

Finding latitude and longitude with latlong.net

You can then enter your Latitude and Longitude into the Customize Forecast screen of Zapier, select your Units which defaults to Fahrenheit, and then click the big Continue button.

Configuring forecast with latitude and longitude

At this point, you can click Test Trigger, which simply makes sure it’s working, and click Continue again.

Now we’re going to tell Zapier what to do with the information once it knows it’s going to rain.

In the "Do this..." panel, search for “sms” and select SMS by Zapier.

Selecting SMS by Zapier

We’re going to leave the App and Event as the defaults, so the next screen you can just click Continue.

SMS App and Event

Now, for Zapier to send you a text, it needs to verify that your phone number belongs to you or that the phone number is knowingly signing up for these texts. To do that, it sends you a one-time PIN that you’ll enter.

So click Sign in to SMS by Zapier, which will open a popup window.

Here, enter your phone number, and choose Sms or Call as the verification method at which point it will contact you with a PIN.

Sign in to SMS by Zapier and send a PIN

With that PIN, enter it into the field and click Continue.

Entering the SMS verification PING

At this point the window will close and move you back to the original flow. Here, click Continue again.

Now we get to customize the text that we receive.

In the From Number field, Zapier gives a bunch of phone numbers you can use. You can either select one number to always send from, which you can set up as a contact so you know it’s Zapier, or you can select Random, which will use a random number every time.

Then, click inside of Message, and it will bring up some options. I want to know everything possible if it’s going to rain, including the probability, max temperature, and summary, so we can select all or as much as we want and again click Continue.

Configuring Weather message

Finally we get to test if our Zap worked. At this point everything should be configured, so click the Test & Review button and you should receive a sample text!

Note: If you choose a single From Number, you might be limited in how frequent you can receive texts, so if you don’t get it right away, that might be why. Setting random helps prevent that issue, but the the number isn’t consistent.

And once you’re happy with the configuration, you can click Turn On Zap.

Turning on the SMS Zap

You’ll now get texts in the morning if the weather is predicting rain!

Zap 2: Print a test every week with Google Cloud Print

This one doesn’t sound exciting, but have you ever gone through a long period of time where you didn’t print something, only to end up with dried out printer heads or worse yet, a now unsalvageable printer?

We can avoid this by simply running a weekly print job that keeps our printer ink nice and fresh.

For this, we’ll use Google Cloud Print. To make this work, you’ll need to already have this configured with your Google account.

Let’s create a new Zap and this time for our “When this happens…” search for and select Schedule by Zapier.

Selecting Schedule by Zapier

We can then select a Trigger Event of Every Week and click Continue.

Setting Trigger Event as Every Week

Next, you can choose the Day Of The Week and Time Of Day you’d like to print. I personally run this job weekly at 8pm on Sundays right before the start of a new week. Once configured, click Continue.

Configuring Zapier schedule

At this point, we can click Test trigger, which just like before makes sure it’s working properly, and then we can click Continue.

Now, for our “Do this…” we want to print, so search for and select Google Cloud Print.

Selecting Google Cloud Print

And for the action, select Submit Print Job.

Setting Submit Print Job as Action Event

At this point, you’ll need to sign into Google Cloud Print. This will open a window and have you log in through Google so that Zapier can interface with the service.

Once connected, click Continue.

Now we can configure out print job. Here we’ll want to define what we print.

Configuring Print Job in Zapier

In the above, we’re configuring:

• Which Printer: the printer we want to print to connected to Google Cloud Print
• Content: this can be a URL to a document, HTML, or plain text. I’m using a URL which is a simple test page I made that has some color in it
• Content Type: you’ll want to set this depending on what you set in Content. If you set a URL like I did, it should be URL
• Title Of Print Job: the name of the job so you can see it in the print logs
• Number of Copies: probably just want 1 so it doesn’t waste ink and paper
• Color or Monochrome: you’ll want to explicitly set this if you want color. The idea is to refresh all of the ink cartridges, so only printing black won’t help the color ink, so in my case, I selected color

The rest of the fields are optional, feel free to customize to your liking.

With our configuration set, click Continue, and similar to before, we can click Test to see our print job in action and if we’re happy, we can click Turn on Zap!

Test print from Zapier print job

If you want to use the same document, you can find it here: https://fay.io/printer-test.pdf

Zap 3: Smashing Magazine Job Alerts with Gmail

If we’re looking for a job, it can be a pain to have to visit every job board every day (or every hour, am I right?). But we can automate this process when the job board supports it.

Luckily, job boards like Smashing Magazine and a whole lot of others provide RSS feeds which we can hook right in to Zapier to automate getting an email whenever a new job is posted.

To get started, let’s create a new Zap, and this time, search for RSS and select RSS by Zapier.

Selecting RSS by Zapier

For our Trigger Event, select New Item in Feed, then click Continue.

Setting New Item in Feed as Trigger Event

At this point, we want to enter a feed URL. This will be the URL to the XML RSS feed that websites make available. For Smashing Magazine, you can find it here:

https://www.smashingmagazine.com/jobs/feed

So enter that URL above into Feed URL (you can leave Username and Password blank), and keep Different GUID selected for What Triggers A New Feed Item. Then click Continue.

Setting RSS Feed URL

Same as usual, now you can test the trigger to make sure it works. If the RSS feed is valid, this will be smooth, otherwise you might see an error. The above URL should be valid!

Testing the RSS feed

Next, we need to choose what we want to do with the new item. Since we want it emailed, we can choose our email service, which in my case is Gmail.

Selecting Gmail in Zapier

For our action, we want to Send Email.

Setting Action Event as Send Email

Next, you want to sign in to your account, similar to what we did with Google Cloud Print. This should be your Google account that you use Gmail with.

Sign in to Gmail

Now when we customize our email, we want to include the following:

• To: wherever you want these sent to, probably the same account you signed into Gmail with
• From: select your Gmail account
• From Name: can be anything you’ll recognize, like I’ll use Colbybot
• Subject: can be whatever you’d like, but a helpful idea could be “New Job Alert:" followed by selecting the title from the dropdown selection
• Body Type: you can leave as Plain
• Body: I would recommend including all tokens you’ll find helpful including the Title, Description, and Link

Configuring job notification email

Once you’re done configuring, you can hit continue. Then, similar to before, click Test & Review, and you should receive your test email.

Test job alert email

Finally if you’re happy with the configuration, turn on the Zap, and enjoy your job search!

What else can you do?

More Ideas

Here’s more ideas to get you moving in the right direction:

• Post Tweets to Slack: every time an account tweets or someone from a list of accounts tweets, post that tweet to Slack
• File Github Bugs to Jira: whenever someone tags a Github Issue with a label of “bug”, create a new ticket in Jira with that Issue details
• Post RSS Items to Twitter: do you write your own content? Set up an RSS feed to automatically post a tweet with your new blog post

Not Google Assistant

The only thing that it’s missing for me right now is Google Assistant, otherwise I would have included some Zap ideas for that. IFTTT supports Google Assistant for simpler flows, but Zapier can get more powerful.

Webhooks

Zapier supports web hooks — meaning you can really do whatever you want. Though this is a premium feature, you can set up some custom workflows based off of new data Zapier sees or accept requests that you post to trigger other automations.

What's your favorite Zap?

Love hearing new creative ways to use Zapier. Share with me on Twitter!

If you want to get more technical with your product management skills, being able to work with flowchart or diagram creator apps is surely on your list.

The following guide aims to save you hours of time you’d otherwise spend researching the best diagramming solution for your project while at the same time perfecting the design of your flowcharts.

As a general rule, a flowchart or diagram is used to communicate requirements, processes, and workflows in a logical way so complex concepts will be a little less complicated.

In product management, workflows are commonly used to describe:

• User flows
• Processes and other systems involved in them
• Dependencies between systems and inputs, outputs, or other conditions

I’ve tested and reviewed several tools for creating flowcharts and workflows over the past few years. Here, I've picked 5 that you must try out to describe your product’s funnels and flows.

But first…

Why are diagrams so powerful in product management?

While the whole idea of using flowcharts and diagrams might be overlooked or seem like it’s an unnecessary step, there are several benefits to using diagrams in PM:

• It simplifies all processes
• It helps you spot issues or weak points at a glance
• It allows you to find any duplicate parts of a process
• It establishes clear funnels
• It supports your Quality Assurance team during the creation of test scenarios
• It acts as a roadmap for software architecture

So how do you choose the best diagramming software?

Creating any kind of diagram is a fairly easy process. Building a successful one, though, takes a bit more research and time.

That’s why you need to choose a tool that will help you save hours of work while also giving you the predefined requirements you’ll need to build accurate diagrams and keep the process fun.

The key aspects to pay attention to when choosing a flowchart creator software solution are:

• Availability of ready-made templates
• Lots of choices for shapes and arrows to work with
• Extra tools or integrations
• Exporting options
• Strong collaboration features
• Ease of use from the beginning

To make it easier for you to select your next tool for building diagrams, I’ve highlighted all of these aspects for every individual solution across the following reviews.

Next up are my favorite free flowchart creator and workflow diagram app picks you need to take into consideration.

5 free flowchart creator and workflow diagram tools

Lucidchart

Lucidchart should be your top-of-mind solution for creating customer experience maps and systems diagrams in particular. From the very beginning, the tool makes it easy for product managers to select the type of flowcharts they want to create. Matter of fact, the onboarding process and resources available online make it a tool with a fast learning curve.

From the very beginning, you’ll get all possible templates needed for you to create product strategies, create roadmaps and product flows, and even pitch your ideas to stakeholders. With ready-made templates and literally thousands of shapes and arrows to choose from, there are endless opportunities for product managers to create their materials in no time.

Also, if you’re like me and like to add a few too many colors, this diagram app comes with a bunch of theme options to help you nail the design of your creations. Bonus points for customization here from non-designers.

The reason I’ve placed Lucidchart first on this list is that it’s a complete flowchart creator tool. Collaboration options are some of the strongest with comments that can be left on charts just like you would in Google Docs, slides that can be used to present your diagrams, and even sharing and exporting options.

If you’d like to take it all one step further, you can choose from dozens of integrations with tools you might already be using like Slack, GitHub, or Salesforce.

You can also check out my video tutorial on using the tool so you won’t have to find your way around Lucidchart yourself.

Pros

• Free plan available
• Easy to learn
• Lots of templates and elements to choose from
• Strong collaboration features

Cons

• Limited space for free accounts
• No desktop app

Draw.io

Draw.io is a top choice for many PMs since the tool’s free, so you might have already been using it for several other business processes. Since the tool belongs to Google the interface is similar to other Google Apps like Docs and Slides that you’re probably working with anyway.

Compared to Lucidchart, Draw.io might be harder to use for non-designers since there are no theme colors to choose from if you want to make your diagrams eye-appealing.

Whenever you want to create a new diagram, you’ll get to choose from a series of templates ordered by their structure. There are no specifics for product management.

The flowchart tool does save itself through lots of elements you can use on your workspace. I advise you to use the search bar to find a specific item or element faster.

Since the tool is so simple that even a child could use it, Draw.io is missing the advanced functionalities and collaboration features that would otherwise allow you to communicate with your team directly on the document. To do this, you’ll need to integrate it with a tool like Confluence or Quip.

Pros

• Free to use
• Clear interface which makes it easy to get accustomed to

Cons

• Limited templates
• No collaboration options
• No mobile apps
• Few integrations

Creately

Creately is only a free workflow diagram app if you can accept the fact that all of your design will be public and you’re limited to 5 documents. Team plans start at $12 which is a fairly decent price considering you’re getting all the features with it.

I specifically found their onboarding process handy since it introduces you to the tool so you won’t start and have no idea what to do next.

In many ways, the tool is similar to Lucidchart. You’ve got lots of templates to choose from, all of which are ordered by their purpose. Then you’ve got your elements which are some of the most beautiful I’ve seen, along with other styles and theme options to help product managers who aren’t that skilled at design too.

One thing people struggle with while using Lucidchart is file organization, a feature that Creately nails by turning their tool into an alternate file storage app. The tool also eases the process of working with several elements by integrating Google search so you can find images to use without leaving the app.

In terms of collaboration, Creately works similarly to a prototyping tool, allowing people to leave comments on the exact element they have feedback to offer on. Collaboration is done in real-time via the online or desktop version of the app.

Pros

• Lots of templates to pick from
• Theme options
• Communication options
• File organization option
• Works offline too
• Easy to learn

Cons

• The free option only allows you to create public content
• Some features are still under development but will be released soon

Whimsical

Whimsical is a user-friendly visual communication tool that’s commonly used to create wireframes but has separate options for putting together flowcharts, mind maps, and sticky notes.

While the tool is highly visual, there aren’t that many template options to choose from. This is because the app is centered around creating basic wireframes and flowcharts for internal use only. Shapes and connectors are also limited, but there are hundreds of icons to go for if you want to add some extra color to a mind map or flowchart.

On to the benefits of using the tool: collaborating on each project takes fewer than two steps, with comments being placed on every single element you want to leave immediate feedback on. All work is easy to share, export, and even embed.

Ever been annoyed by losing your past versions for a product development flowchart you were supposed to show to your stakeholders? Whimsical is one of the few tools that has a fully-featured file versioning option that records every single change you’re making on a chart so you can restore better versions.

Pros

• Strong design options
• Easy to learn
• Sharing and exporting options
• Lots of icons to add to your creation
• File versioning
• Free version available

Cons

• Limited template options

Canva

Canva is not really a common option since it’s usually used by marketing and social media teams who need highly-visual content to display publicly. If you’re also looking to create a flowchart to present on your social networks, in your pitch deck, or to any other stakeholders, it’s definitely worth giving it a try.

Not only is it easy to use, but the fact that it’s a general app instead of being tailored only to building diagrams also means it has thousands of elements and icons you can choose from. Illustrations, arrows of all kinds, customizable charts, you name it. Even videos.

As for the sharing options, they’re top-notch. You can export your content into any file type you want, resize your workspace to share it on multiple platforms, and use the presentation mode, complete with notes for presenters.

Once you invite your team into the tool, you’ll be able to leave comments on your designs to keep everyone on the same page.

Pros

• Free option
• Extremely easy to use
• Versatile
• Focused on great design
• Lots and lots of options for elements
• Collaboration options

Cons

• Limited template options for product management

Haven’t decided yet?

Start by creating a list of your own requirements. Are you in need of strong templates so you don’t have to build everything from scratch? Or maybe you can give up on lots of elements as long as collaboration options are on point.

Based on your own needs and goals, go over the above list to pick an option that works best for you and give it a try. Each option has its own unique features that distinguish it from the others:

• Lucidchart - lots of template options for PM
• Draw.io - super easy to use
• Creately - most features in one tool
• Whimsical - clear interface for internal use
• Canva - focused on design

If you have any other tips on helping product managers choose their next flowchart creator workflow diagram app, let me know.

I learned by doing projects (often freeCodeCamp ones!). But I needed to make sure I didn't lose my hard work, and that others could see the hard work I was putting in, so every project had to go on Github. The more complete projects I had on Github, the easier it would be for recruiters. But the steps required to set up a project, initialise a repo, and sync with Github were really annoying and repetitive, so I decided to solve the problem.

Bad news: this isn't going to be a big, fancy, detailed and technically sexy post. It's going to be very un-sexy.

Good news: you don't need to be a shell scripting god(dess) to do it.

So my typical project setup work flow usually goes like this:

1. Go to my ../projects folder and run mkdir project-of-some-name to create a folder with the name project-of-some-name.

2. cd into that project folder and do git init to initialise a local git repo in there.

3. run touch README.MD to create the README file, open it and add some basic descriptions, including links to the resources / tutorials I was implementing in that project. Save the file.

4. run git add . and then git commit -m ' ...some initial commit message...

5. open a browser, go to Github, login, create a new (remote) repository, copy the url, return to my terminal, make sure I was in the correct project folder project-of-some-name...then run the git scripts needed to set up the remote repo as the 'upstream' repo and connect my local repo to it. Then, finally, I can run a git push and my local commit would get pushed up

6. lie down and take a nap, exhausted from this repetitive process.

Admittedly, this was my process, but I liked to stay organised and always be able to access my projects so I can refer to them.

Since automation is a great way to practice your coding skills, I decided to write a small shell script that automates these horrible and repetitive steps. The script is at the bottom of this post, and be warned - it's not sophisticated or fancy. But it sure gets the job done, and I don't need to log in to Github and fool around will all those steps!

Before you copy the script, you need to know how to run it on your Mac. So below are the steps you need to implement to be able to use the script to automate your setup workflow.

1. I keep my scripts in my root/home folder, in a sub-folder called scripts. I suggest you do the same or similar. To get to the root/home folder, in your terminal type cd ~ because the tilda ( ~ )is the symbol for the home folder. In your Mac Finder app it shows up as the one with a house icon. So all my scripts are stored in ~/scripts

2. This matters because to run a shell script from any directory in the terminal, you have to type out the full path. In my case I have to type out ~/scripts/git-script.sh to run the script. But we're getting ahead of ourselves.

3. copy the code chunk at the bottom of this post and then open a text editor, paste it in and then save it as [filename].sh. The .sh is the extension for shell scripts. Save the file in the directory you want to save it at - again I recommend ~/scripts as the folder to save your scripts in.

4. Navigate to that folder in your terminal. To be safe run ls in the terminal to check that you can see the script is there. If it's not you're in the wrong folder or step 3 didn't successfully complete.

5. Make the shell script executable. To do that you type the following in the terminal: chmod +x <<the-correct-filename.sh>>. This is the unix way to make a shell script "executable". I'm not confident I fully understand what that means, other than it's needed to make any shell scripts you write executable, so don't ask me and I won't lie to you.

6. navigate to your projects folder and make a new folder that you intend to house your project. Effectively, you've got to do this: mkdir - create a project-of-some-name inside the folder where you keep all your projects. So your project will eventually be placed inside my-computer/my-projects/project-of-some-name. cd into this folder and then type pwd to get the full path. Copy that - you will need to paste it shortly. It should look like my-computer/my-projects/project-of-some-name

7. open your terminal again, and then type ~/scripts/<<the-correct-filename.sh>>``. The script runs! You will be guided through some input... The main steps are:

   what do you want to call your Github repo (don't use spaces- 'my-awesome-project' is good. Don't use 'my awesome project' as the repo name.

   Enter a description that shows up in the Github repo's description. For this it's safe to use spaces.

Enter the project path you got in step 6, the one that you get after typing pwd in the terminal and getting something like my-computer/my-projects/project-of-some-name

enter your Github username (not email address) and then your Github password. Be careful as you type as these values don't show up on the screen.

....that's it. The script will set up your git repo locally inside my-computer/my-projects/project-of-some-name and then create a README.MD (blank) and then commit it locally, then set up a remote repo in Github (log you in via API) etc and then push everything up!

finally, you will see that the terminal you were interacting with has changed the currently active directory to your project folder. It will now be at my-computer/my-projects/project-of-some-name and you can type in ls and see the README.MD file. If you then type git status you will see your local repo's status (the state of your local project) and if you type in git remote it will show you the Github url for your project!

Done! Happy Coding!

Annnd.....finally......here is the script! I've commented each step so you can reason your way through it.

# Make executable with chmod +x <<filename.sh>>

CURRENTDIR=${pwd}

# step 1: name of the remote repo. Enter a SINGLE WORD ..or...separate with hyphens
echo "What name do you want to give your remote repo?"
read REPO_NAME

echo "Enter a repo description: "
read DESCRIPTION


# step 2:  the local project folder path
echo "what is the absolute path to your local project directory?"
read PROJECT_PATH

echo "What is your github username?"
read USERNAME

# step 3 : go to path 
cd "$PROJECT_PATH"


# step 4: initialise the repo locally, create blank README, add and commit
git init
touch README.MD
git add README.MD
git commit -m 'initial commit -setup with .sh script'


# step 5 use github API to log the user in
curl -u ${USERNAME} https://api.github.com/user/repos -d "{\"name\": \"${REPO_NAME}\", \"description\": \"${DESCRIPTION}\"}"

#  step 6 add the remote github repo to local repo and push
git remote add origin https://github.com/${USERNAME}/${REPO_NAME}.git
git push --set-upstream origin master

# step 7 change to your project's root directory.
cd "$PROJECT_PATH"

echo "Done. Go to https://github.com/$USERNAME/$REPO_NAME to see." 
echo " *** You're now in your project root. ***"

Thanks for reading!

If you would like to learn more about my journey into code, check out episode 53 of the freeCodeCamp podcast, where Quincy (founder of freeCodeCamp) and I share our experiences as career changers that may help you on your journey. You can also access the podcast on iTunes, Stitcher, and Spotify.

I will also hold a few AMAs and webinars in the coming months. If this is of interest to you please let me know by going here. And of course, you can also Tweet me at @ZubinPratap.

Nearly every developer works with different tools on a daily basis to help them achieve their goals or complete specific tasks. In this post, I’ll outline the tools I use for my day to day activities. First thing I’ll like to do is define a workflow.

What is a Workflow?

A workflow is a series of events or tasks that processes a set of data. Workflows occur across every kind of business and industry. They are the paths that describe how something goes from being a work in progress to complete, or raw to processed.

Here’s an example: I have a workflow for writing articles. I start by looking for ideas for a post. Next, I create an outline. Then writing the actual content of the post to publishing all these are series of events to achieving a task.

As a developer, I tend to make use of certain tools that help speed up the process of completing a specific task. Even though a lot of these processes might be different for everyone, I still feel the need to share this with you. In the end, no knowledge is lost, right?

Tools I use to help speed up productivity

This is going to be a broad list so I’ll try and split them into different categories, I’ll be highlighting the tools I use when coding, writing, and other tools I think you will find helpful in general.

Coding Tools

GitKraken

GitKraken Homepage

I found out about this tool recently from a friend and I really love using it. I’m a big fan of using the command line over using GUI but this tool is much different. GitKraken is a GitHub client application that works just like the command line you are already used to but with intuitive UI/UX that makes for a great experience when you are using this over the command line.

Insomnia

Insomnia Homepage

This tool is another one I really like using. It is similar to Postman but also allows your query GraphQL endpoints which are cool right. Insomnia is a Powerful HTTP and GraphQL tool belt that is available on Mac, Linux and Windows platforms. Its features include GraphQL support, SVG and image previews, AWS authentication, etc.

Visual Studio Code

Visual Studio Code Homepage

Visual Studio Code is a code editor that I really love using. I have been using for over a couple of years now and I’ve not found any reason to stop. With constant improvements and new features added regularly, this is my go-to choice for a code editor.

Dash

Dash website

Dash is a Mac application for developers that give you access to over 200+ API documentation offline for free. You can create your own docsets or even request for docs that are not included. For Windows users, an alternative for this app is Zeal App.

SourceTree

Sourcetree Homepage

Ever wanted to use GUI instead of the command line for Git? Then SourceTree is your best bet. I already mentioned GitKraken and they both do the same job — I usually alternate between both apps occasionally. So I’ll say use what works best for you they are both great apps.

Postico

Postico Homepage

Postico is one client app for Postgres. It has a beautiful UI and it’s easily accessible. I prefer using this to other apps I’ve found or even using the Postgres command line.

Hyper

Hyper Homepage

Hyper is one tool I love using. It is, in fact, my default terminal for coding. Hyper is easily customizable with a lot of plugins to integrate into it making it a great choice for me.

Writing Tools

Notion

Notion website

Notion is a tool for almost everything you need. I’ve yet to come in contact with anyone with negative thoughts about this tool. It is super helpful. You can write, plan and collaborate with Notion.

Dropbox Paper

Dropbox paper website

Paper by Dropbox is a great tool for writing. I frequently use this in creating notes, writing a post, and a lot of other things. It supports markdown formatting.

Bear App

Bear Website

I like variety, so pardon me If I have shared a lot of apps that perform similar tasks. Bear App is an app for crafting and writing anything. It can be used for writing articles, jotting down notes, and also as a todo list app.

Caret

Caret is a markdown editor for Mac, Windows and Linux users. It enables you to write content in Markdown and also ships with quite a number of features that are really useful like dark mode which I really love.

Deploying

Netlify

Netlify website

Netlify is a tool I love using for deploying my apps. It is a unified platform that automates your code to create high-performant, easily maintainable sites, and web apps.

GitHub Pages

GitHub Pages Website

GitHub Pages is quite useful when you need to spin up a deployed version of your website for free. With a few steps of setup from your git repository, you have a deployed website. I use this a lot.

Zapier

Zapier website

Zapier is an automation tool for automating virtually anything you can think of. I use it for automating tweets sent out when I publish a new article by creating a zap with my twitter account and my RSS feed.

Miscellaneous

Rocket

Rocket website

Rocket is a slack-style emoji for your Mac. It works just like the emoji finder in Slack. Enter a colon when you want to use an emoji, type its name, and hit enter again to insert. It’s much more convenient than using Mac’s built-in emoji keyboard.

Numi

Numi Homepage

Numi is a calculator app for your Mac. This is not just a calculator, I must say, because its features are much more than that. It blew my mind when I discovered I could convert virtually anything with it, i.e convert dollars to naira, my local currency! ?

Taskade

Taskade website

Taskade is an application with a lot of features. I use this for todo lists, my weekly planner, and also for creating notes. Meanwhile, there are a whole lot of things that can be done with this app. No wonder it is tagged as an instant collaboration tool for Agile teams.

RecordIt

Recordit website

RecordIt is an application for recording screencasts. I find this really helpful for when I need to create a quick demo or show how a particular thing works in gif format.

Aware

Aware website

Aware is a Mac application that helps you keep track of how long you’ve been actively using your computer. I use this keep track of the time I spend working or completing a particular task.

Conclusion

I hope you find all these tools listed useful and start using them to make your job a lot easier. The idea is to share with you tools I found helpful, and really I’d like it if you can also share with me tools you have found useful that are not listed here. Let’s all stay productive!

Working as a front-end developer for nearly two years, I’ve got helpful experience from being part of several web development projects of design/digital agencies.

One obvious but valuable lesson I’ve learnt is that collaborating between each groups with one goal but distinct responsibilities and purposes is not easy. There’re different aspects and levels of difficulties in terms of collaboration, and the specific part of which I’d like to address here is workflow process.

Based on my experience, and with the help from my designer and developer friends, I built a website development workflow designed for small team (5–15 people). The system is composed of Confluence, Jira, Airtable and Abstract. In this article, I’ll share the why and how of the this workflow.

Motivation for building a new workflow

To deliver a customized website without using templates provided by website builders, the minimum talent requirements includes a designer, developer and project manager. After participating in a couple of cases, I had a sense that there was something wrong with the workflow we had: important information was always not aligned both internally between different roles and externally to the client. This inefficient communication was clearly slowing down the development cycle and hurting the team.

So I started to solve this problem.

_Google search workflow great resources: [workflow definition](https://medium.com/eightshapes-llc/system-features-step-by-step-e69c90982630" rel="noopener" target="_blank" title="">Design systems features, style guide resources and <a href="https://www.projectmanager.com/training/define-workflow-process" rel="noopener" target="blank" title=")

I Google searched resources about establishing and improving a workflow. Though I learned a lot from all the great resources, I found nearly none of which was for web development projects in a design/digital agency. It was either a design system or coding guidelines that scoped in design or front-end roles, or a workflow that was built for a team with its own product.

As a result, I decided to cherry pick the parts I needed to solve our problems, and formed a customized workflow for website development.

Problems and goals

Following are the problems I inspected from our existing workflow, and the corresponding improvement goals:

1. Waterfall methodology

waterfall model abstract demo

Problem: Based on my experience, website projects adopt a waterfall approach because clients don’t have a concept of a minimum viable product (MVP). Instead of splitting functionalities from views and modulization, clients tend to think about the site in a traditional page-by-page way, which forces both designers and developers to work page by page in sequence. This causes them to lose a universal perspective across the project. This situation results in lots of back-and-forth redundant revisions between pages.

Goal: Changing the mindset of clients is both arrogant and unrealistic. The goal is to find a way to separate requirements from views as soon as possible and develop in as as modulized a way as possible internally based on page-by-page model.

2. Universal design tokens and components managed by both designers and developers

_design tokens from [Salesforce](https://www.lightningdesignsystem.com/design-tokens/" rel="noopener" target="blank" title=")

Problem: This is a common issue that a lot of articles have shared great solutions to, which mostly propose building a design system that’s managed by style guide/library generators. Though it is great solution, managing an extra site that barely provided edit permission to designers was not appropriate in our situation.

Goal: Except for creating universal design tokens and languages that designers, developers and managers can all understand, build a system that allows everyone to manage the assets in a synchronous way.

3. Accurate, updated progress dashboard

we need an editable and accessible progress dashboard

Problem: Though issue trackers, kanban, and more project management models are useful and practical, most of them failed to act as a straightforward, flexible and friendly progress dashboard. This kind of dashboard would save the team a lot of time because it would prevent team members from actively reporting or asking about the current situation of specific tasks. It also makes managers’ lives easier if they have a clear knowledge of the entire project without too much effort.

Goal: Build a dashboard system that provides edit permission for individuals in charge of specific tasks.

Workflow diagram

Before we dive into the detail introduction of the management tools stack, let’s take a look at the abstract simplified workflow I organized. It’s pretty much just a visualization of a normal workflow that most agencies have, but there’s two points to be noted here.

workflow diagram I designed

1. Developer evaluation

First, when requirements or issues coming from the client are approved and documented by manager, with the exception of sending the task to a designer, they also go to a developer for evaluation. In this process, the developer reviews the specification of the task, checking if there are any rather complicated functions or features included. If it’s positive, the developer could start working on it or notify the designer about the potential problems beforehand.

2. Single source of truth

Also notice that after design deliverable is approved by the client, and before handing the task over to the developer’s hand, it goes through a process of register/modify/delete over design store conducted by the designer. This is because the developer should always be exposed to one and only one source of design store, which contains constantly maintained and updated assets ready for development.

Now we can dive into the management tools stack I prepared and see how the tools help us solve our problems.

The tools stack

After experimenting with various options on the market, the stack I’m proposing here is composed of Confluence, Jira, Airtable and Abstract. In addition to basic introduction and few key application examples, I’ll not cover all the details of using the tools.

_[ABEM](http://atomicdesign.bradfrost.com/" rel="noopener" target="_blank" title="">atomic design and <a href="https://css-tricks.com/abem-useful-adaptation-bem/" rel="noopener" target="blank" title=")

Note: the system assumes that the development team adopts the atomic design methodology and ABEM naming system.

1. Confluence

Role: information and resource center

Though it’s intimidating at first, Confluence provides a powerful workspace that’s easy to organize, and it has tons of features, integration of apps, and customized templates. It’s definitely not a universal solution to all problems, but it’s perfect for documentation of specifications, requirements, meeting notes and more.

Therefore, Confluence in this stack works as an information and resource center, which means every related link and detail about this project should be documented properly in here.

My favorite advantage of Confluence is the ability to customize document templates. This feature makes it really convenience to standardize the workflow.

developer evaluation stage

Example: Component functionality review

I mentioned the developer evaluation process above, which is actually a complicated job. This is because this process includes basic information of the component, a developer’s FSM review (if necessary), FAQ space and more. But the flexibility of the template and tools Confluence provides makes this super easy. Just build a template in configuration settings and you’re good to go.

custom template for component review in Confluence

2. Jira

Role: issue tracking and action type management

Also a member of the Atlassian family, Jira is a super powerful issue tracking and project planning software. My favorite part about it is making customized issue workflows. Since there are tons of great tutorials on how to utilize power of Jira, the only thing I want to point out here is using issue type as mentioned below.

designer update design store

Example: Update developer on changes of design store by issue type

To ensure that developers are building the components based on correct design views, they need to be notified whenever something in the design store is being updated, which includes actions like register, modify and delete. So, as a component is updated, the designer should open an issue with the responsible developer assigned and the correct issue/action type selected.

_[Jira issue types function](https://confluence.atlassian.com/jiraportfolioserver024/configuring-initiatives-and-other-hierarchy-levels-934716034.html" rel="noopener" target="blank" title=")

3. Airtable

Role: component management and progress dashboard

Airtable, a mixture of spreadsheet and database, is the thing that makes this stack work. There’s two amazing features that support my workflow: four types of view transition in single table and related content linking. I’ll showcase two examples of using these two features here.

developer starts working on the task

Example 1: Component management

How do you manage your component library? We chose not to use a style guide generator, because it’s not accessible for designers to edit. Using the Sketch component library wasn’t appropriate either, because it’s has too many limitations if we tried to use it outside the scope of the software itself.

I wouldn’t say Airtable is a perfect solution, but it’s the easiest and most flexible option I could think of. Take a look at the demo template of the component management table here:

component table

Once a newly registered design view that’s ready to be developed programmatically is submitted to the developer, they would asses the view based on the ABEM system, and register it into the table. There are 9 columns in the table, including:

1. Name: naming of the component in ABEM principle

2. Preview: screenshot or exported image of component

3. Linked page: link to the page contains this component

4. Children component: link to children components contains this one

5. Modifier: checks if there’s style variations (ex: — active, — red)

6. Component category: a general category classification (ex: text, hero, sidebar)

7. Development status: status of development progress (pending, assigned, in progress, complete, in revision)

8. Assignee: developer responsible for this component

9. Atomic level: atomic category of this component (atom, molecule, organism)

The best thing here is that you can reference data in both the same and other tables. This connection of dots prevents things from getting messier as the scale grows. Also notice that you can filter, sort and change views easily.

Example 2: Page development status

Since the assumption here is that we’ll inevitably asses development progress page by page, a table template designed for this purpose is necessary. This table can be a progress dashboard for both internal teams and shared with client at the same time.

page list table

Any information about the page, including deadline, InVision prototype link, assignee, and children component can be organized here. Note that it’s very convenient to document and update design, front-end, and back-end development status at the same time.

4. Abstract

Role: single source of truth and design assets version control

Abstract is GitHub for Sketch assets that saves designers from the hell of copying and pasting files. It’s out of this article’s scope to demonstrate details of managing version control flow. The key takeaway here is that Abstract is the design store that acts as the single source of truth. Designers should keep updating master branch to the latest version of confirmed design and then notify developers. On the other hand, developers should only take design assets in the master branch as reference.

_[Abstract branch template](https://www.abstract.com/how-it-works/" rel="noopener" target="blank" title=")

More work to be done

From my own experience, development of the entire project after adopting this new workflow has been at least two times faster than before. It’s not a perfect solution, because it still requires lots of manual labor to update and maintain.

But I think it could be aa helpful reference to website development teams searching for aa better workflow, and hopefully more people can share their workflows in the future!

?中文版連結 (Chinese version) / Originally posted on vinceshao.com