But there’s a quiet shift happening inside the browser environment that a lot of engineers are completely missing out on.

The modern browser isn’t just a glorified engine for rendering HTML and CSS anymore. It’s turning into a full-blown runtime for local intelligence. We’ve reached a point where you can ship raw machine learning models straight to a user's device and run inference completely client-side. No server trips, no API keys to protect, and once those initial assets load, zero dependency on an internet connection.

This is the reality of Web AI. If you're building for the web today, understanding this paradigm shift is easily one of the most valuable skills you can add to your stack.

In this guide, we’re going to pull back the curtain on how Web AI actually operates under the hood, break down the browser technology stack making it possible, and build a real, working image classifier using Teachable Machine and TensorFlow.js. Along the way, we’ll also set up a live benchmark so you can watch exactly how WebGL and WebGPU stack up against each other in real-time execution speeds.

Prerequisites

To follow along with this tutorial, you should have:

• A working knowledge of JavaScript

• Basic familiarity with HTML and how the browser works

• Google Chrome installed (required for WebGPU support and Chrome's built-in AI APIs)

• A code editor like VS Code with the Live Server extension installed (recommended for running the demo locally)

No prior machine learning experience is required.

Table of Contents

• What is Web AI?

• Browser AI vs Cloud AI

• The Technology Stack

• How to Build AI in the Browser

• Chrome's Built-in AI APIs

• Where Web AI Is Headed

• What You Learned

• Resources

What is Web AI?

Instead of sending data off to a distant cloud server, Web AI lets you run machine learning models directly on the user’s device inside their browser. It uses standard web tech like JavaScript, WebAssembly, and WebGPU to handle all the heavy lifting right then and there.

The simplest definition: intelligence that runs in the browser, without sending your data anywhere.

Most of us already interact with on-device AI every day without realizing it. Think about unlocking an iPhone. The second you lift it, Face ID maps out roughly 30,000 infrared points, feeds that data through a neural network living on Apple's local silicon, matches it against an encrypted embedding, and opens the phone. The whole process takes milliseconds and happens entirely offline.

Browser-based AI works on that exact same core architecture. The only real difference is that we're building on top of shared web standards rather than native hardware APIs. When you spin up a face-tracking model using TensorFlow.js or MediaPipe in Chrome, you're running that exact same pipeline:

Camera input → Local ML model → Local decision

No round trip. No server. The browser is your Neural Engine.

Browser AI vs Cloud AI

There’s no right or wrong answer here. It just depends on what you’re trying to build. Both approaches have their pros and cons, so it’s just a matter of picking the tool that fits your specific use case.

Use browser AI when:

• You need split-second speed for things like tracking gestures or detecting objects live on a webcam

• The app has to work offline (whether it's a PWA or just needs to survive spotty internet)

• Privacy is a hard requirement to keep sensitive data like medical inputs, biometrics, or financial information strictly local

• You want to reduce or eliminate API costs on high-frequency, lightweight predictions

Use cloud AI when:

• You need large models like GPT-4, Gemini Pro, or Stable Diffusion

• You need centralized model updates, A/B testing, or user analytics

• You require serious GPU or TPU compute power

Most production systems actually use a mix of both. Take Google Photos: it handles face detection right on your device so it’s fast and private, but leaves the heavier categorization work for the cloud. Or think of a modern web app that might use TensorFlow.js locally to classify images instantly, but calls the Gemini API when it needs deeper language processing.

This hybrid setup, keeping lightweight intelligence at the edge and heavy compute in the cloud, is usually the sweet spot for most apps.

The Technology Stack

Browser AI isn’t just a single tool – it’s a stacked layer of technologies. Knowing how these layers fit together makes it a lot easier to choose your setup and navigate the trade-offs.

Tensors

Before jumping into any ML framework, you need to understand tensors. Not deeply, just enough of a handle on them so you don't get blindsided by tensor shape errors, because they will happen and they can be tricky to debug.

Think of a tensor as a multi-dimensional grid of numbers. Whether your model is processing images, audio, or text, everything gets converted into this format first. Models only speak numbers, and tensors are the containers that hold them.

A single number       → 0D tensor (scalar):  42
A list of numbers     → 1D tensor (vector):  [0.2, 0.8, 0.5]
A table of numbers    → 2D tensor (matrix):  [[1,2,3],[4,5,6]]
An image              → 3D tensor:           shape [224, 224, 3]
A batch of images     → 4D tensor:           shape [32, 224, 224, 3]

Models accept inputs in specific shapes. If your tensor shape doesn't match the model's expected input, your code breaks. That's why understanding dimensions is practical, not just theoretical.

TensorFlow is literally named after this concept. Tensor + Flow = tensors flowing through neural networks.

Here's how you create tensors in TensorFlow.js:

// 1D tensor — a list of values
const scores = tf.tensor([0.1, 0.7, 0.2]);

// 3D tensor — a single image (height x width x RGB channels)
const image = tf.tensor([
  [[255, 0, 0], [0, 255, 0]],
  [[0, 0, 255], [255, 255, 0]]
]);

// 4D tensor — a batch of 32 images
const batch = tf.zeros([32, 224, 224, 3]);

TensorFlow.js

TensorFlow.js is Google's JavaScript version of TensorFlow. It lets you run pre-trained models right in the browser and, if you really want to, train new ones completely client-side.

The most important concept in TensorFlow.js is the backend, the hardware your model actually runs on. You can switch between backends depending on what the user's device supports, and it makes a significant difference to performance.

await tf.setBackend('webgpu');  // fastest — true GPU compute
await tf.setBackend('webgl');   // very fast — GPU via graphics shaders
await tf.setBackend('wasm');    // fast — near-native CPU speed
await tf.setBackend('cpu');     // slowest — plain JavaScript on CPU

await tf.ready();
console.log('Running on:', tf.getBackend());

In practice, you want to try the fastest available backend and fall back gracefully if a user's browser doesn't support it:

const backends = ['webgpu', 'webgl', 'wasm', 'cpu'];

for (const backend of backends) {
  try {
    await tf.setBackend(backend);
    await tf.ready();
    console.log('Using backend:', backend);
    break;
  } catch {
    continue;
  }
}

WebAssembly

WebAssembly (WASM) basically lets code written in C++ or Rust run inside the browser at near-native speeds. When it comes to AI, this is a big deal because heavy math operations like tensor calculations, data preprocessing, and running compressed models happen way faster in WASM than they ever could in standard JavaScript.

Under the hood, TensorFlow.js's WASM backend is using a compiled C++ runtime. If you're running compressed models on a device's CPU, switching to the WASM backend can make your app anywhere from 2 to 10 times faster than just sticking with regular JavaScript.

await tf.setBackend('wasm');
await tf.ready();

WebGL and WebGPU

This is where browser AI performance gets interesting.

WebGL was originally built for 3D graphics. But developers discovered that the parallel computation that GPUs use for rendering is exactly the kind of parallel computation neural networks need.

TensorFlow.js's WebGL backend encodes tensor operations as graphics shader programs and runs them on the GPU. It works well, but it's a workaround, as WebGL was never designed for this kind of work.

WebGPU is what was actually designed for the job. It launched in Chrome back in April 2023 after six years of collaboration between Apple, Google, Mozilla, Intel, and Microsoft.

Instead of just handling graphics, it's a modern API built from the ground up for general-purpose computing. When it comes to running AI models, it can be 2 to 3 times faster than WebGL, which means you can actually run significantly larger models right in the browser.

Here's how to check for WebGPU support and use it:

if ('gpu' in navigator) {
  console.log('WebGPU is supported');
  await tf.setBackend('webgpu');
} else {
  console.warn('WebGPU not available, falling back to WebGL');
  await tf.setBackend('webgl');
}

await tf.ready();

To enable WebGPU in Chrome for development, go to:

chrome://flags/#enable-unsafe-webgpu → Enable → Restart Chrome

The performance progression across backends looks like this:

MediaPipe

MediaPipe is Google's framework for real-time perception tasks like hand tracking, face mesh detection, pose estimation, and object detection. Think of it as plug-and-play AI for anything that involves a camera.

You don't build these models yourself – you just import them and use them. MediaPipe is what actually powers the background blur in Google Meet and the visual filters in YouTube. Under the hood, it runs on TensorFlow.js and WebAssembly to keep everything moving fast.

You can try all MediaPipe models interactively before writing any code at MediaPipe Studio.

How to Build AI in the Browser

Step 1: Train a Model with Teachable Machine

Teachable Machine is Google's no-code tool for building models. It lets you create custom images, audio, or pose classifiers right from your webcam without needing any machine learning experience. Once you're done, you can export them as TensorFlow.js models that are completely ready to drop straight into your app.

Here's how to get started:

1. Go to teachablemachine.withgoogle.com

2. Choose Image Project, standard image model.

3. Create two or more classes. "Thumbs Up" and "Thumbs Down" is a simple starting point

4. Record examples for each class using your webcam

5. Click Train Model — training happens entirely in your browser

6. Click Export Model and choose TensorFlow.js

When you export, you get three files:

• model.json: The model architecture: layers, input/output shapes, and paths to the weights

• weights.bin: The trained weights stored as binary data

• metadata.json: Class labels, input size, and inference configuration

A note on training data quality

Teachable Machine relies on supervised learning. You give the model labeled examples, and it figures out the underlying patterns. When you're gathering your data, two things matter way more than the sheer number of pictures you take:

• Balance: If one class has significantly more examples than another, the model will be biased toward it. Keep the data roughly equal across classes.

  Variety: Fifty photos from different angles, distances, and lighting conditions will easily outperform two hundred near-identical shots from the same spot. The model needs to understand the concept of a "thumbs up", not memorise one specific photo of your specific thumb.

Keep in mind that the actual machine learning model is usually just a tiny fraction of your overall codebase. The vast majority of what you write is going to be standard JavaScript. At the end of the day, it's just another asset in your stack.

Step 2: Setting up and Writing the Code

Now that you have your model files, set up your project structure like this and create an index.html file:

your-project/
├── index.html
├── model.json
├── weights.bin
└── metadata.json

The model.json, weights.bin, and metadata.json files all go in the same folder as your index.html. The demo loads them from the same directory using const URL = "./".

To run it locally, open the folder in VS Code or your preferred IDE and use the Live Server extension. Just right-click index.html and select Open with Live Server. Opening the file directly in the browser without a server will cause CORS errors when loading the model files.

Step 3: Load the Model and Run Predictions

Paste the following in your index.html file. This demo loads your Teachable Machine model, starts your webcam, and runs continuous predictions in a loop:

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Teachable Machine - Webcam + Backend Switch Demo</title>
    <style>
        body {
            font-family: Arial;
            text-align: center;
            margin: 20px;
        }

        #webcam-container {
            margin-top: 20px;
        }

        #label-container {
            margin-top: 10px;
            font-size: 18px;
            font-weight: bold;
        }

        button.backend-btn {
            margin: 5px;
            padding: 8px 16px;
            font-size: 16px;
            cursor: pointer;
        }

        #status {
            margin-top: 10px;
            font-weight: bold;
            color: #0078ff;
        }

        table {
            margin: 20px auto;
            border-collapse: collapse;
            width: 80%;
            max-width: 600px;
        }

        th,
        td {
            border: 1px solid #ccc;
            padding: 10px;
        }

        th {
            background: #0078ff;
            color: white;
        }
    </style>
</head>

<body>
    <h2>AI in the web Demo</h2>

    <div>
        <button class="backend-btn" onclick="switchBackend('cpu')">CPU</button>
        <button class="backend-btn" onclick="switchBackend('webgl')">WebGL</button>
        <button class="backend-btn" onclick="switchBackend('webgpu')">WebGPU</button>
    </div>

    <p id="status">Click a backend to start</p>

    <table>
        <thead>
            <tr>
                <th>Backend</th>
                <th>Load Time (s)</th>
                <th>Inference Time (ms)</th>
                <th>Status</th>
            </tr>
        </thead>
        <tbody id="results"></tbody>
    </table>

    <div id="webcam-container"></div>
    <div id="label-container"></div>

    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest/dist/tf.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgpu"></script>
    <script
        src="https://cdn.jsdelivr.net/npm/@teachablemachine/image@latest/dist/teachablemachine-image.min.js"></script>

    <script>
        const URL = "./";
        const resultsTable = document.getElementById("results");
        const statusEl = document.getElementById("status");
        const backends = ["cpu", "webgl", "webgpu"];

        let model, webcam, maxPredictions;
        const backendResults = {};

        // Initialize webcam
        async function initWebcam() {
            if (!webcam) {
                webcam = new tmImage.Webcam(200, 200, true);
                await webcam.setup();
                await webcam.play();
                document.getElementById("webcam-container").appendChild(webcam.canvas);

                const labelContainer = document.getElementById("label-container");
                labelContainer.innerHTML = "";
                for (let i = 0; i < 2; i++) labelContainer.appendChild(document.createElement("div"));
            }
        }

        async function switchBackend(backend) {
            statusEl.innerText = `Switching to ${backend.toUpperCase()}...`;

            await initWebcam();

            try {
                const startLoad = performance.now();
                await tf.setBackend(backend);
                await tf.ready();
                model = await tmImage.load(URL + "model.json", URL + "metadata.json");
                maxPredictions = model.getTotalClasses();
                const endLoad = performance.now();
                const loadTime = ((endLoad - startLoad) / 1000).toFixed(2);

                // Single inference to measure time
                const startInference = performance.now();
                await model.predict(webcam.canvas);
                const endInference = performance.now();
                const inferenceTime = (endInference - startInference).toFixed(1);

                // Store results
                backendResults[backend] = { loadTime, inferenceTime };

                updateTable();

                statusEl.innerText = `${backend.toUpperCase()} ready`;
            } catch (err) {
                console.error(`${backend} not supported:`, err);
                statusEl.innerText = `${backend.toUpperCase()} not supported`;
            }
        }


        function updateTable() {
            resultsTable.innerHTML = "";
            for (let backend of backends) {
                const row = document.createElement("tr");
                const backendCell = document.createElement("td");
                const loadCell = document.createElement("td");
                const inferenceCell = document.createElement("td");
                const statusCell = document.createElement("td");

                backendCell.textContent = backend.toUpperCase();

                if (backendResults[backend]) {
                    loadCell.textContent = backendResults[backend].loadTime;
                    inferenceCell.textContent = backendResults[backend].inferenceTime;
                    statusCell.textContent = "✓";
                } else {
                    loadCell.textContent = "-";
                    inferenceCell.textContent = "-";
                    statusCell.textContent = "-";
                }

                row.appendChild(backendCell);
                row.appendChild(loadCell);
                row.appendChild(inferenceCell);
                row.appendChild(statusCell);
                resultsTable.appendChild(row);
            }
        }

        // Continuous prediction loop
        async function loop() {
            if (webcam && model) {
                webcam.update();
                const prediction = await model.predict(webcam.canvas);
                const labelContainer = document.getElementById("label-container");
                labelContainer.innerHTML = "";
                for (let i = 0; i < maxPredictions; i++) {
                    const p = document.createElement("div");
                    p.textContent = `\({prediction[i].className}: \){(prediction[i].probability * 100).toFixed(1)}%`;
                    labelContainer.appendChild(p);
                }
            }
            requestAnimationFrame(loop);
        }

        loop();
    </script>
</body>

</html>

A few things worth understanding about what this code is doing:

The switchBackend function does more than just swap the backend. Each time you click a backend button, it records how long the model takes to load on that backend and how long a single inference takes. Those numbers go straight into the comparison table so you can see the difference without having to look at console logs.

The loop function runs continuously using requestAnimationFrame. Every frame, it grabs the current webcam image, passes it to the model, and updates the prediction labels on screen. This is what makes the detection feel real-time.

Notice that initWebcam only runs once. It checks if webcam already exists before setting up. Switching backends reloads the model but keeps the same webcam stream running.

Open Chrome DevTools and go to the Network tab while the demo runs. After the model files finish loading, you'll see zero outbound requests. Every prediction is happening entirely in the browser.

Step 4: Switch Backends and Compare Performance

Once the demo is running, click each backend button one at a time: CPU, then WebGL, then WebGPU. The table updates after each switch and shows you the load time in seconds and inference time in milliseconds for each backend side by side.

Here's what you should expect to see:

• CPU will be the slowest with everything running in plain JavaScript

• WebGL will be noticeably faster as the GPU is now handling the tensor operations

• WebGPU will be the fastest with true GPU compute and less overhead than WebGL. The exact numbers depend on your machine, but the gap between CPU and WebGPU is usually significant enough to see immediately in the table.

Note: WebGPU requires Chrome with the flag enabled. If the WebGPU button shows "not supported", go to chrome://flags/#enable-unsafe-webgpu, enable it, and restart Chrome.

Chrome's Built-in AI APIs

Beyond loading your own models, Chrome is rolling out native AI capabilities that you can hook into directly through browser APIs. This means no managing bulky model files, no importing TensorFlow.js, and zero manual setup.

The powerhouse here is Gemini Nano, a lightweight version of Google's Gemini model built to run completely on-device inside Chrome. It handles tasks like smart replies and page summarization right in the browser without ever making a cloud call.

If you want to build with it, you can tap into these experimental APIs that Chrome exposes to developers:

chrome://flags → search "Prompt API for Gemini Nano" → Enable → Restart Chrome

These are still experimental and behind flags. But they show clearly where the platform is heading.

For the full prerequisites and setup guide for Chrome's built-in AI, see the official Chrome AI getting started documentation.

Where Web AI Is Headed

The browser is evolving into something that doesn't really have a clean name yet. It's no longer just a document viewer, and it's not quite a native app runtime either. Instead, it's becoming an intelligent edge node – a piece of infrastructure that can perceive, process, and act all on its own, without constantly phoning home for permission.

A few massive shifts are already well underway:

• Native AI built directly into the platform: AI capabilities are turning into standard browser APIs. Because they're cached and shared across the entire ecosystem, you won't have to re-download massive models for every single domain you visit.

  Browsers designed with AI as their core foundation are already popping up. OpenAI's Atlas browser is a perfect early signal of this trend. Every year, the idea of the browser acting as an intelligent agent platform rather than a simple content renderer gets more concrete.

• The developer shift: For developers, the immediate future is clear: a significant chunk of AI features that currently live on expensive servers will migrate straight to the client side. It won't be everything, but the lightweight, high-frequency, and privacy-sensitive tasks will absolutely make the jump.

WebGPU isn't just a flashy demo technology, and browser inference is definitely not a toy. These are serious production tools, and they're only getting more capable as AI models shrink and user hardware gets more powerful.

If you're currently building an interactive, AI-powered feature, it's well worth pausing to ask yourself: does this actually need a server?

Sometimes the answer is still yes. But more and more often, the answer is a definitive no.

What You Learned

In this tutorial, we covered:

• What Web AI is and how it differs from cloud-based AI

• When to use browser AI versus cloud AI and how a hybrid approach works

• The technology stack behind browser AI: tensors, TensorFlow.js, WebAssembly, WebGL, WebGPU, and MediaPipe

• How to train a custom model with Teachable Machine and export it for the browser

• How to load that model, run it against live webcam input, and manage GPU memory correctly

• How to benchmark WebGL vs WebGPU inference times to measure real performance differences

• How to access Chrome's built-in AI APIs including Gemini Nano

If you found this useful or want to connect, you can find me on Twitter/X or LinkedIn.

Resources

• TensorFlow.js Documentation

• Teachable Machine

• MediaPipe Studio

• WebGPU in Chrome

• Chrome Built-in AI — Getting Started

• Chrome AI Translator API

• Google Web AI Demos on GitHub

• Hugging Face Transformers.js

• WebLLM — Run LLMs in the Browser

• Chrome AI Features — Google Blog

If you are getting started with deep learning, the available tools and frameworks will be overwhelming. Industry experts may recommend TensorFlow while hardcore ML engineers may prefer PyTorch.

Both these frameworks are powerful deep-learning tools. While TensorFlow is used in Google search and by Uber, Pytorch powers OpenAI’s ChatGPT and Tesla's autopilot.

Choosing between these two frameworks is a common challenge for developers. If you're in this position, in this article we’ll compare TensorFlow and PyTorch to help you make an informed choice.

Understanding PyTorch and TensorFlow

Let’s start by getting to know our contenders better.

PyTorch, created by Facebook’s AI Research lab, has gained recognition for its simplicity and user-friendliness. Pytorch can efficiently handle dynamic computational graphs.

A computation graph is a visual representation of mathematical operations and their relationships. It’s like a flowchart that shows how data flow through the deep learning model.

Training neural networks involves a lot of computations. So computation graphs help computers organize and execute calculations efficiently when training neural networks.

PyTorch is easy to use, making it a favoured choice among developers and researchers alike. For people who appreciate a straightforward framework for their projects, PyTorch is a perfect choice.

TensorFlow, Google’s brainchild, has robust production capabilities and support for distributed training. TensorFlow excels in scenarios where you need large-scale machine learning models in real-world applications.

Distributed training is a technique used in deep learning to train large and complex models. By spreading the training process across multiple machines or devices, it is useful when dealing with massive datasets.

Tensorflow is the go-to choice for companies that need scalability and reliability in their deep learning models.

So as you may be able to see, the choice between PyTorch and TensorFlow often depends on the specific needs of a project.

PyTorch vs TensorFlow – Which One's Right for You?

Ease of Learning and Use

When you’re starting a new project, it's helpful to have an easier learning curve. It helps both in building the project as well as hiring / training engineers for your project.

PyTorch is simpler and has a “Pythonic” way of doing things. It's a favourite for beginners and researchers. And its dynamic computation graph means you can change things on the fly, which is great for experimentation.

TensorFlow offers a more structured approach. Its static computation graph requires a bit more planning ahead. TensorFlow also comes with a steep learning curve. But this can lead to more optimized and high-performance models.

TensorFlow 2.0 has also made strides in simplicity. It has incorporated more of PyTorch’s dynamic nature through its Eager Execution feature.

But when it comes to simplicity and ease of learning, PyTorch is a clear winner.

Performance and Scalability

When it comes to performance and scalability, TensorFlow shines. Its can handle large-scale, distributed training with ease. So TensorFlow is a go-to choice for production environments.

TensorFlow’s integrated tool, TensorBoard, is also a powerful tool for visualization and debugging.

PyTorch is catching up, with recent updates improving its scalability.

PyTorch has made improvements to support distributed training and scalability. It provides tools to help you train deep learning models on multiple GPUs and even across multiple machines.

But TensorFlow still holds the lead in deploying large-scale models in production.

Community and Support

The strength of a framework is also partly defined by its community. As these are open-source frameworks, there is no customer support. So you have to depend on the community for help if you get stuck while building a project using these frameworks.

TensorFlow, being older, has a larger community. It also has a vast array of tutorials, courses, and books.

PyTorch, while younger, has seen rapid growth in its community. PyTorch is a favourite, especially among researchers since it's easy to use Pytorch for experimenting with datasets.

Both frameworks have strong support, but TensorFlow’s maturity gives it a slight edge in this area.

Flexibility and Innovation

If you’re working on cutting-edge research or need more flexibility, PyTorch is your best bet. Its dynamic computation graph allows for more creative and complex model architectures.

As I said before, this flexibility makes PyTorch a beloved tool in the research community. Where rapid prototyping and experimentation are key, PyTorch is your best option.

TensorFlow has been working towards adding more flexibility. But it's a difficult battle to win since PyTorch is built for simplicity from the ground up.

Industry Adoption

PyTorch (blue) vs TensorFlow (red)

TensorFlow has tpyically had the upper hand, particularly in large companies and production environments. Its robustness and scalability make it a safe choice for businesses.

But PyTorch is quickly gaining ground. As you can see in the trends chart, PyTorch has already taken over TensorFlow as the most searched deep learning library. You can find the live chart here.

Multiple industries are starting to adopt PyTorch for research and development due to its user-friendliness and flexibility. Pytorch has also proved its capability as a production-grade tool after the release of models like ChatGPT.

Here is a list of companies using TensorFlow and PyTorch.

Products Using Tensorflow

• Google Search and Recommendations: Google uses TensorFlow to enhance its search engine and recommendation systems. It helps improve search accuracy and provides personalized recommendations based on user behaviour and preferences.
• NVIDIA Deep Learning Accelerator (NVDLA): NVDLA is a hardware accelerator for deep learning applications. It uses TensorFlow to optimize and deploy models on this hardware.
• Uber’s Michelangelo: Uber uses TensorFlow in its Michelangelo platform for machine learning. It assists in various tasks, including ETA predictions, fraud detection, and dynamic pricing.

Products Using PyTorch

• Facebook: Since PyTorch is from Facebook, Facebook uses PyTorch for various internal AI research and applications, including content recommendations and language translation.
• Tesla Autopilot: Tesla’s Autopilot system relies on PyTorch for its deep learning components, such as object detection and navigation.
• OpenAI’s GPT Models: Many of OpenAI’s language models, including GPT-2 and GPT-3, are built using PyTorch. These models are used for a wide range of natural language processing tasks, including text generation and language translation.

Conclusion

Choosing between PyTorch and TensorFlow depends on your project’s needs.

For those who need ease of use and flexibility, PyTorch is a great choice. If you prefer scalability from the ground up, production deployment, and a mature ecosystem, TensorFlow might be the way to go.

Both frameworks are evolving, so keep an eye on their development. Your choice today might not be your choice tomorrow. Remember, the best tool is the one that suits your project’s needs and not the popular one.

Thanks for coming this far. If you want weekly machine learning tutorials delivered to your inbox, join my newsletter. To get in touch with me, you can connect with me on LinkedIn.

Binary classification is used in a wide range of applications, such as spam email detection, medical diagnosis, sentiment analysis, fraud detection, and many more.

In this article, we'll explore binary classification using TensorFlow, one of the most popular deep learning libraries.

Before getting into the Binary Classification, let's discuss a little about classification problem in Machine Learning.

What is Classification problem?

A Classification problem is a type of machine learning or statistical problem in which the goal is to assign a category or label to a set of input data based on their characteristics or features. The objective is to learn a mapping between input data and predefined classes or categories, and then use this mapping to predict the class labels of new, unseen data points.

Sample Multi Classification

The above diagram represents a multi-classification problem in which the data will be classified into more than two (three here) types of classes.

Sample Binary Classification

This diagram defines Binary Classification, where data is classified into two type of classes.

This simple concept is enough to understand classification problems. Let's explore this with a real-life example.

Heart Attack Analytics Prediction Using Binary Classification

In this article, we will embark on the journey of constructing a predictive model for heart attack analysis utilizing straightforward deep learning libraries.

The model that we'll be building, while being a relatively simple neural network, is capable of achieving an accuracy level of approximately 80%.

Solving real-world problems through the lens of machine learning entails a series of essential steps:

1. Data Collection and Analytics
2. Data preprocessing
3. Building ML Model
4. Train the Model
5. Prediction and Evaluation

Data Collection and Analytics

It's worth noting that for this project, I obtained the dataset from Kaggle, a popular platform for data science competitions and datasets.

I encourage you to take a closer look at its contents. Understanding the dataset is crucial as it allows you to grasp the nuances and intricacies of the data, which can help you make informed decisions throughout the machine learning pipeline.

This dataset is well-structured, and there's no immediate need for further analysis. However, if you are collecting the dataset on your own, you will need to perform data analytics and visualization independently to achieve better accuracy.

Let's put on our coding shoes.

Here I am using Google Colab. You can use your own machine (in which case you will need to create a .ipynb file) or Google Colab on your account to run the notebook. You can find my source code here.

As the first step, let's import the required libraries.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
import sklearn
import pandas as pd
import keras
from keras.models import Sequential
from keras.layers import Dense
import tensorflow as tf
from sklearn.metrics import confusion_matrix,ConfusionMatrixDisplay
from sklearn.preprocessing import MinMaxScaler

I have the dataset in my drive and I'm reading it from my drive. You can download the same dataset here.

Remember the replace the path of your file in the read_csv method:

df = pd.read_csv("/content/drive/MyDrive/Datasets/heart.csv")
df.head()

Sample 5 record in the dataset

The dataset contains thirteen input columns (age, sex, cp, and so on) and one output column (output), which will contain the data as either 0 or 1.

Considering the input readings, 0 in the output represents the person will not get heart attack, while the 1 represents the person will be affected by heart attack.

Let's split our input and output from the above dataset to train our model:

target_column = "output"
numerical_column = df.columns.drop(target_column)
output_rows = df[target_column]
df.drop(target_column,axis=1,inplace=True)

Since our objective is to predict the likelihood of a heart attack (0 or 1), represented by the target column, we split that into a separate dataset.

Data preprocessing

Data preprocessing is a crucial step in the machine learning pipeline, and binary classification is no exception. It involves the cleaning, transformation, and organization of raw data into a format that is suitable for training machine learning models.

A dataset will contain multiple type of data such as Numerical Data, Categorical Data, Timestamp Data, and so on.

But most of the Machine Learning algorithms are designed to work with numerical data. They require input data to be in a numeric format for mathematical operations, optimization, and model training.

In this dataset, all the columns contain numerical data, so we don't need to encode the data. We can proceed with simple normalization.

Remember if you have any non-numerical columns in your dataset, you may have to convert it into numerical by performing one-hot encoding or using other encoding algorithms.

There are lot of normalization strategies. Here I am using Min-Max Normalization:

Min-Max Scaling Formula

Don't worry – we don't need to apply this formula manually. We have some machine learning libraries to do this. Here I am using MinMaxScaler from sklearn:

scaler = MinMaxScaler()
scaler.fit(df)
t_df = scaler.transform(df)

scaler.fit(df) computes the mean and standard deviation (or other scaling parameters) necessary to perform the scaling operation. The fit method essentially learns these parameters from the data.

t_df = scaler.transform(df): After fitting the scaler, we need to transform the dataset. The transformation typically scales the features to have a mean of 0 and a standard deviation of 1 (standardization) or scales them to a specific range (for example, [0, 1] with Min-Max scaling) depending on the scaler used.

We have completed the preprocessing. The next crucial step is to split the dataset into training and testing sets.

To accomplish this, I will utilize the train_test_split function from scikit-learn.

X_train and X_test are the variables that hold the independent variables.

y_train and y_test are the variables that hold the dependent variable, which represents the output we are aiming to predict.

X_train, X_test, y_train, y_test = train_test_split(t_df, output_rows, test_size=0.25, random_state=0)

print('X_train:',np.shape(X_train))
print('y_train:',np.shape(y_train))
print('X_test:',np.shape(X_test))
print('y_test:',np.shape(y_test))

Sample training and testing dataset size

We split the dataset by 75% and 25%, where 75% goes for training our model and 25% goes for testing our model.

Building ML Model

A machine learning model is a computational representation of a problem or a system that is designed to learn patterns, relationships, and associations from data. It serves as a mathematical and algorithmic framework capable of making predictions, classifications, or decisions based on input data.

In essence, a model encapsulates the knowledge extracted from data, allowing it to generalize and make informed responses to new, previously unseen data.

Here, I am building a simple sequential model with one input layer and one output layer. Being a simple model, I am not using any hidden layer as it might increase the complexity of the concept.

Initialize Sequential Model

basic_model = Sequential()

Sequential is a type of model in Keras that allows you to create neural networks layer by layer in a sequential manner. Each layer is added on top of the previous one.

Input Layer

basic_model.add(Dense(units=16, activation='relu', input_shape=(13,)))

Dense is a type of layer in Keras, representing a fully connected layer. It has 16 units, which means it has 16 neurons.

activation='relu' specifies the Rectified Linear Unit (ReLU) activation function, which is commonly used in input or hidden layers of neural networks.

input_shape=(13,) indicates the shape of the input data for this layer. In this case, we are using 13 input features (columns).

Output Layer

basic_model.add(Dense(1, activation='sigmoid'))

This line adds the output layer to the model.

It's a single neuron (1 unit) because this appears to be a binary classification problem, where you're predicting one of two classes (0 or 1).

The activation function used here is 'sigmoid', which is commonly used for binary classification tasks. It squashes the output to a range between 0 and 1, representing the probability of belonging to one of the classes.

Optimizer

adam = keras.optimizers.Adam(learning_rate=0.001)

This line initializes the Adam optimizer with a learning rate of 0.001. The optimizer is responsible for updating the model's weights during training to minimize the defined loss function.

Compile Model

basic_model.compile(loss='binary_crossentropy', optimizer=adam, metrics=["accuracy"])

Here, we'll compile the model.

loss='binary_crossentropy' is the loss function used for binary classification. It measures the difference between the predicted and actual values and is minimized during training.

metrics=["accuracy"]: During training, we want to monitor the accuracy metric, which tells you how well the model is performing in terms of correct predictions.

Train model with dataset

Hurray, we built the model. Now it's time to train the model with our training dataset.

basic_model.fit(X_train, y_train, epochs=100)

X_train represents the training data, which consists of the independent variables (features). The model will learn from these features to make predictions or classifications.

y_train are the corresponding target labels or dependent variables for the training data. The model will use this information to learn the patterns and relationships between the features and the target variable.

epochs=100: The epochs parameter specifies the number of times the model will iterate over the entire training dataset. Each pass through in the dataset is called an epoch. In this case, we have 100 epochs, meaning the model will see the entire training dataset 100 times during training.

loss_and_metrics = basic_model.evaluate(X_test, y_test)
print(loss_and_metrics)
print('Loss = ',loss_and_metrics[0])
print('Accuracy = ',loss_and_metrics[1])

The evaluate method is used to assess how well the trained model performs on the test dataset. It computes the loss (often the same loss function used during training) and any specified metrics (for example, accuracy) for the model's predictions on the test data.

Sample output to find the Loss and Accuracy

Here we got around 82% accuracy.

Prediction and Evaluation

predicted = basic_model.predict(X_test)

The predict method is used to generate predictions from the model based on the input data (X_test in this case). The output (predicted) will contain the model's predictions for each data point in the training dataset.

Since I have only minimum dataset I am using the test dataset for prediction. However, it is a recommend practice to split a part of dataset (say 10%) to use as a validation dataset.

Evaluation

Evaluating predictions in machine learning is a crucial step to assess the performance of a model.

One commonly tool used for evaluating classification models is the confusion matrix. Let's explore what a confusion matrix is and how it's used for model evaluation:

In a binary classification problem (two classes, for example, "positive" and "negative"), a confusion matrix typically looks like this:

Here's the code to plot the confusion matrix from the predicted data of our model:

predicted = tf.squeeze(predicted)
predicted = np.array([1 if x >= 0.5 else 0 for x in predicted])
actual = np.array(y_test)
conf_mat = confusion_matrix(actual, predicted)
displ = ConfusionMatrixDisplay(confusion_matrix=conf_mat)
displ.plot()

Confusion matrix for the predicted output

Bravo! We've made significant progress toward obtaining the required output, with approximately 84% of the data appearing to be correct.

It's worth noting that we can further optimize this model by leveraging a larger dataset and fine-tuning the hyper-parameters. However, for a foundational understanding, what we've accomplished so far is quite impressive.

Given that this dataset and the corresponding machine learning models are at a very basic level, it's important to acknowledge that real-world scenarios often involve much more complex datasets and machine learning tasks.

While this model may perform adequately for simple problems, it may not be suitable for tackling more intricate challenges.

In real-world applications, datasets can be vast and diverse, containing a multitude of features, intricate relationships, and hidden patterns. Consequently, addressing such complexities often demands a more sophisticated approach.

Here are some key factors to consider when working with complex datasets.

1. Complex Data Preprocessing
2. Advanced Data Encoding
3. Understanding Data Correlation
4. Multiple Neural Network Layers
5. Feature Engineering
6. Regularization

If you're already familiar with building a basic neural network, I highly recommend delving into these concepts to excel in the world of Machine Learning.

Conclusion

In this article, we embarked on a journey into the fascinating world of machine learning, starting with the basics.

We explored the fundamentals of binary classification—a fundamental machine learning task. From understanding the problem to building a simple model, we've gained insights into the foundational concepts that underpin this powerful field.

So, whether you're just starting or already well along the path, keep exploring, experimenting, and pushing the boundaries of what's possible with machine learning. I'll see you in another exciting article!

If you wish to learn more about artificial intelligence / machine learning / deep learning, subscribe to my article by visiting my site, which has a consolidated list of all my articles.

We just posted a course on the freeCodeCamp.org YouTube channel that will teach you how to build and evaluate medical AI models with TensorFlow.

Dr. Jason Adleberg teaches this course. He is a radiologist in New York City and a skilled programmer, making him the perfect instructor to guide you through this course.

You will use TensorFlow to evaluate chest x-rays.

In this hands-on course, you will learn how to build and evaluate AI models using TensorFlow, one of the most popular and powerful machine learning frameworks. The course is structured into two parts, offering both theoretical knowledge and practical application.

Part 1: Building and Training TensorFlow Models

This section starts with the basics, guiding you step-by-step to build and train a simple yet effective TensorFlow model. You will learn the fundamental concepts of TensorFlow, gain insights into model architecture, and discover various techniques to optimize model performance. Dr. Adleberg's expertise will help you grasp the essentials of medical AI model development.

Part 2: Evaluating Medical AI Models

Once you have mastered model building, you'll lean how to evaluate. In this part of the course, you'll explore key metrics like AUC (Area Under the Curve), sensitivity, and specificity. These metrics play a vital role in assessing model accuracy and reliability, particularly in clinical settings.

Here are the sections in the course, covering the two parts above.

• Getting started with Google Colab
• Facts about Chest X-Rays
• Defining a Problem
• Preparing the Data
• Training the Model
• Running the Model
• Evaluating Performance
• Stats: Histogram, Sensitivity & Specificity
• Stats: AUC Curve
• Saving our Model

Watch the full course on the freeCodeCamp.org YouTube channel (1-hour watch).

Course Transcript (autogenerated)

Machine learning is being used to save lives in the medical industry.

In this course, you will learn how to build and evaluate AI models with TensorFlow.

This is a great real-world project for improving your machine learning skills.

Dr.

Jason Adelberg teaches this course.

He is a radiologist in New York City and also a programmer.

So let's start learning.

Hey everyone, my name is Jason.

I'm a doctor and computer programmer in New York City.

And today we're going to talk about how to build and evaluate medical AI models with TensorFlow.

This tutorial today will have two parts.

The first part is going to be building and training a really simple TensorFlow model.

And the second part is going to be going through the statistics, the evaluation of our model, and we'll talk about metrics like AUC, sensitivity, specificity.

This stuff will be really useful, especially if we're interested in deploying this in the clinical space.

I wanted to give a big shout out to Dr.

Walter Wiggins for inspiring this tutorial.

Here's his Twitter and here's mine.

And with that, let's get started.

All right, so today we're going to be using Google Colab.

Google Colab is a really cool website through which you can run different parts of Python code.

If you've never used it before, it's relatively easy.

We'll start by clicking connect up here.

And then basically in this tutorial, all these different blocks of code here, these are known as cells, and we can click the play button here to the left of it to get everything up and running.

This first cell will just be us downloading a whole bunch of things, so it's a good one to get started with.

All right, so today we're going to be working with chest x-rays.

And chest x-rays are the most common imaging study performed in hospitals, in emergency departments, in outpatient settings.

It's usually one of the very first things that doctors want to know about if you're not feeling well.

Now, there are a number of things and structures you can see on a chest x-ray, so let's just go over them real quickly.

Here's a normal chest x-ray.

You can see the lungs.

You can see the heart here in the middle.

You can see the aorta coming off of the heart and supplying blood to the rest of the body.

You can see a number of skeletal structures.

So, for example, here's your collarbone or your clavicle.

You can see all the ribs here on both sides.

You can see the vertebra here in the middle.

This line here is called your diaphragm.

It goes around like that.

And this separates your chest, your thorax, from your abdomen.

Underneath this line is your liver.

Your spleen is over here.

And this little kind of air bubble sitting in here, this is your stomach, which in this case just has a little bit of air in it.

And again, this is a normal chest x-ray.

Today, we're going to be using a pretty big data set of chest x-rays called the N-ray.

And this is an open source data set of a few thousand chest x-rays, which happens to have eight different labels.

So, here's the eight different labels available to us in this data set.

And these represent eight pretty commonly seen things on x-rays.

This is not everything that you can see that can go wrong in a chest x-ray, but this is some of the more common things in the world.

And let's just go through them real quickly.

Here we have atelectasis.

This is when a little piece of the lung kind of deflates a little bit.

So, that is up here.

Basically, this wedge-shaped thing up here in the right upper lobe.

Here we have cardiomegaly.

Cardiomegaly is when you have a really big heart.

Specifically, it's when like this length of the heart is more than 50% the length of rib-to-rib.

And this is a sign of heart disease.

Here we have a pleural effusion.

That's when you have something that's basically sitting right outside the lung in what's called the pleural space, which is not really supposed to be full of anything.

This one here, this is an infiltrate.

And that's when you have something sitting inside the alveoli of the lungs.

It's not supposed to be there.

There's a few different things that can do that, but most of the time this means that you have a pneumonia.

In this x-ray, we have a mass.

It's this rounded density up here in the left upper lobe apex or on top of the lung.

A mass is something that's more than three centimeters in diameter.

And with the mass, you know, again, it depends on the context, but generally we're sort of worried here about some sort of tumor.

This is a nodule, and a nodule is a mass that's smaller than three centimeters.

So this is a little bit harder to see.

Here's pneumonia.

A pneumonia is an infection inside of your alveoli.

Again, this is not like mutually exclusive with infiltrate.

And then finally, last but not least, this is a pneumothorax.

And you know how in this one we said you can have sometimes some fluid sitting in the plural space.

Well, here you have air in the plural space.

And this is also known as a collapsed lung.

I bring all this up because some of these conditions are a little bit easier to see and some are a little bit harder to see.

For example, you can bet that you can have some of these conditions For example, you can bet that a mass, which again is more than three centimeters, you can bet that that will be easier for us to see than a nodule.

And so it should be easier for a computer to see this than to see this, to see a nodule.

You know, basically how good our model is is going to depend partly on the technology that we're using, but partly on the data itself, generally speaking.

You know, if you have more data, that's better.

If you have a higher diversity of data, like different types of nodules, et cetera, the model will work a little bit better.

But the quality of the labels really matters as well.

So now that we've looked at the eight different findings, the e-label is available to us in our data set, let's choose one for the AI model.

So I'm going to choose cardiomegaly.

And let's take a look at the data to actually see kind of a little bit more about the different images that we have.

So I'm going to click here on the folder button and click here on the medical AI folder and here on the images.

And here's just a random one we'll open up.

You can see that this is an example of a chest x-ray up here.

Also in the data that we downloaded, so there's all the images there.

And then here is a giant spreadsheet of all of the images that we have available to us with all the labels on them.

So for example, here are, you know, 100 or so images with the labels right there in that column.

I'm going to grab the path for that file.

And then put that in here.

And then here's just another way that we can make sure that the python that our google collab can see everything.

This is showing just the first five rows which happen to be atelectasis.

Okay, next I'm going to look for the rows of this column where the label equals our finding.

And then we'll do the same thing for where the finding is no finding at all.

These are going to serve as negatives to us.

We want to make sure that we have enough images to train a model with here.

So for this line here, I'm just going to go ahead and make sure that we have enough examples of positive cases.

So this is showing us that we have 146 examples of cardiomegaly that we can use today.

That's pretty good.

Now one concept for building AI models is that you want to separate the data into a training set and a testing set.

Usually you do about 80% for training and 20% for testing.

What does that mean? So, you know, the AI model is going to get taught, is going to learn what's what from our training data set.

But in order to tell if it's working or not, we want to show it images that's never seen before.

And that's going to be our testing data set, which is the other 20% of all those images.

So I'm just going to manually define that right now, just like this.

And then I'm going to go ahead and spell out exactly the number of images that we're going to use for our training data set.

So I'm just going to say that that number is 80% of what we have to work with today.

And the same thing for our test data set.

Let's print those out just to make sure that they make sense.

And you can see that we're going to go, and as you can see, we'll use 116 positive cases for our training data set and 29 positive cases for our testing data set.

Here, you can see we have quite a lot, we have a lot of negative examples.

So our limiting factor today is going to be the number of positive cases.

For this example, today, we're going to do a 50-50 split where we want to have an equal number of positive and negative cases for our training and for our testing.

So we're just going to spell that out right here.

Right here, I'm just going to put together the rows that are going to go in our training data set.

That will look like this.

So right here, I'm just going to put together the rows that are going to go in our training data set.

And the same thing for our testing data set.

And the same thing for our testing data set.

Cool.

Okay, great.

All right, so now that we know how many images we have to work with, now we'll just move them to different folders.

And we'll see a few examples of the images with Python to make sure we're ready to go.

So now we're just going to make some directories.

Here is our root directory with all the images in it.

And then this is how I'm just choosing to make some new directories.

So we'll make one like this for our finding.

We'll make one for our test data set.

We'll make one for our train data set.

And then we're going to do the same thing, but just make negative folders instead of the positive folders too.

And negative just means that there's no finding at all.

Like that.

Now we're going to go ahead and just move those files over.

And we're going to do this basically by iterating through like the rows in our data frame that we care about.

So we pay a little bit of attention to exactly how many images we have to move over.

But here's where, for example, the training positive examples happen to be located.

They are like this, as it's defined in our CSV file.

Here's where we're going to move it to.

And this is, again, the way that we defined it in the code block above.

And then this line here, this actually does the moving.

We'll try it real quick just to make sure it works.

If it does, we can double check by clicking here.

We're going into images.

Waiting a minute.

Going to cardiomegaly.

And you can see that now we've made some test folders, some train folders, and this should be a whole bunch of the images here, right? Now I'm just going to copy and paste this.

And instead of doing the positives in our training data set, I'm going to do the positives in our testing data set.

Because our training data set is 80% of all of this, this is more or less like the 80th percent row of all of the positive things that we have.

And then again, we're going to move this to the testing part.

Just going to tweak that right there.

And then we're going to do the same thing for our negatives.

So now I'm just going to again copy and paste those lines.

I'm just going to change positives to negatives.

Positive to negative.

Here and here.

Done.

I'll go ahead and just copy this here.

Cool.

Now that we've moved everything over, let's just use Python to show everything directly in this notebook, just to make sure that what we're doing makes sense and is appropriate.

So I'm going to define two arrays.

And then this is something that we declared in the first blocker code when we started today.

But let's just like, for emphasis, describe that.

Right here, we're just going to show like smaller versions of the pictures that we're loading up.

And then we'll just show like six examples.

And this is the way that we're going to load it.

And this is the way that we're going to load it into Python itself.

image dot open image path dot resize image width image height image height like that positive images append.

And this is basically like a helper function that we declared up at the very top again.

Image just like that.

And I'll be sure that I spelled it correctly.

Just going to do the same thing again for the negative images.

So again, those are the images that have no findings at all.

Like that.

Now that we've actually loaded everything, we'll go ahead and actually just like show everything with matplotlib.

And so we're going to go ahead and show just six images.

So these will be our six images that have cardiomegaly.

That will go there.

And then we're gonna do the same thing.

I'm copy and pasting this for negative images.

So I'm just going to tweak that.

I'm going to tweak this.

And we're not using, again, like the whole point with the negative images is to show ones that have no finding at all.

So let's go ahead and click play.

Here you can see that it's showing us six examples of cardiomegaly from like the folder that it's created.

And to me, these all look like definite cases of cardiomegaly.

You can see that the heart here is for sure it's enlarged.

It's more than half of the chest.

And here these look like cases with no findings.

So here the heart is normal size and there's not really much else going on as well.

Okay, so now we have visualized our data.

We've moved everything around and we know that we want to build an AI model for cardiomegaly.

In this part, we're going to actually build the TensorFlow model.

Now there are lots of different ways to approach model building and we can spend an hour on this topic alone.

But basically here, I want to talk about two different concepts for this part.

So the first is called transfer learning.

And the second concept is called data augmentation.

We're going to use both of those things today.

All right, so this first line here, this is going to have us load our model directly through TensorFlow itself.

And we're just going to define the size of the image that it's going to be working with, which we kind of talked about earlier, but these are going to be kind of smaller, basically scaled down versions of our chest x-ray.

This three here is saying that basically it's a three channel image, so like a color image, right, like red, green, and blue.

Now it's true that, you know, our chest x-rays are actually just all shades of gray, but we're just going to put that in there because it's a little bit easier.

And then this one here, include top equals false.

We'll come back to this, but this is basically going to let us customize our model to do what we want to do here.

This should be weights plural.

And now we're using again the ImageNet network here.

But what we're basically going to do is, on top of all this, for the last layer of our model, we're going to have it spit out whether it thinks there's cardiomegaly or not.

So we're going to just manually define that here.

This is basically talking about exactly what type of output we want our model to put out.

So this is just getting that last layer, and here we're going to go ahead and define this by saying that we want to say either basically yes or no, positive or negative.

So this is the way that we can do that.

This is just basically some stuff to, again, help us with our big task, which is saying yes or no, cardiomegaly or not cardiomegaly.

I'm going to say yes or no, and then I'm going to say yes or no.

So this is just basically some stuff to, again, help us with our big task, which is saying yes or no, cardiomegaly or not cardiomegaly.

All these decisions here, we could talk about this for such a long time, but I'm going to skip over exactly some of the specifics here.

And if there's something to put an x in there, right.

Okay, as for the model, so now we have this model with a slightly customized last layer.

And now we're just going to basically go ahead and compile it.

Here we're just going to say that we're interested in the accuracy, basically, so how accurate is our model in between, saying whether something is positive or negative, and that's sort of the metric that we're going to use to help us figure out if our model is working or not.

Oh, and then one other thing is that we should put an equal sign right in there, and that would help us as well.

Okay, so now that this is all done, now we're going to go ahead and just define a bunch of things that are basically just kind of helping point our model to the right information.

So if you remember from earlier, all of our images are hiding out right there.

The directory where everything is located for the imaging for the training stuff is like this.

For testing, it's just about the same, but like this, change this here.

And then this kind of keeps going.

So the way we have our data structured is that we have like sub-folders in our training directory that are called positive, one called negative, which we're going to put in right here.

That looks like this.

Like this.

And then the same thing also exists just for our test data.

So I'm going to make these changes here, here, here, and there.

I'll click play right there.

Okay, so now the next concept I want to talk about is something called data augmentation.

So as we discussed earlier, we have a bunch of images available to us with cardiomegaly, but it's not like a ton.

There are some ways that we can basically kind of cleverly create more training data for ourselves, and that's called data augmentation.

So what we're going to use here is basically a really cool concept called an image data generator.

And this is basically something that's going to look at all the images that we have and kind of tweak them a little bit so that we're kind of generating like more data from the data we already have.

Now there's a lot of different ways that you can augment data, that you can kind of tweak data around, but these are the ones we'll just happen to use today.

And, you know, if you're playing around with this sort of on your own, you're welcome to, you know, kind of experiment with different ideas here.

So what are we doing? So first of all, we're just kind of, all these methods here basically are generating extra images that are slightly tweaked from our images.

So specifically, this is going to generate images that are slightly rotated one way or the other, that are slightly shifted, like stretched out, that are sheared, and that are zoomed in as well.

One thing that's important with data augmentation is that it's not just going to be zoomed in.

One thing that's important with augmentation for medical data is that, you know, there's kind of some different thoughts on this, but when it comes to flipping images, you know, a chest x-ray is never really going to be flipped around, right? Like, you know, your heart is always going to be like, you know, you're, you know, when you take a chest x-ray, like, right side up, right? Like your stomach's always going to be below your lungs.

So if we were to flip the data upside down, that wouldn't necessarily really be helpful for our AI model.

In regard to horizontal flipping, you know, I think there's not really like a consensus on this when it comes to medical imaging or specifically chest x-rays.

It's true that our bodies are not completely symmetrical, even though we like to pretend there are, so your heart's a little bit more on your left side.

There is a condition where your heart can be more on the right side, but it's not super common.

So I'm just going to say that we're not going to flip things horizontally so that we don't kind of confuse our model, but that's a choice that you could make if you wanted to.

For our test data, we definitely don't want to augment, don't want to mess with our test data at all.

So this is just a line where, you know, we are creating an image generator, but all that we're doing here is just like redefining the numbers that are inside of our, of each pixel, basically.

So this is, it's not really actually doing anything, right? This is basically saying that instead of each pixel going from like zero to 255, it's going to be between zero and one.

All right, let's go ahead and click there.

That works.

So that's image augmentation.

And now this here is something that we need to use in order to get our model to train, called a train and test generator.

And this is basically the way that we do it here.

So from directory, which we already have, target size and then we're just going to do the same thing for testing right below.

And then, you know, of course, we just need to kind of tweak some of these things here for this particular part.

We'll go ahead and just define the number of steps that we're going to use for this, which is basically because we have a batch size of one.

It's happens to be the same, happens to be just like all the images that we're using.

So, I mean, this is one way of doing it, like the way that our data is structured.

Like the way that our data is structured.

So again, train steps equals the length of that folder times two.

And then for testing, we're going to go ahead and say that it's like this.

And then we actually typed this in wrong here.

So let's go ahead and just fix this real quick.

That goes like that.

And then this goes like this.

And you can see here that when we click, so it's found 232 images for the training part and 60 images for the testing part.

And again, that is pretty close to 80% and 20% split.

All right, so we have everything set up.

We have our model ready to go.

We have our data ready to go.

This is the cool part.

Now we're finally going to run our model and let it train.

So in order to do that, this is the method that we're going to use.

We're going to use what's called model.fit.

We're going to point it to the train generator that we talked about earlier.

We're just going to mention exactly the number of steps per epoch.

And epoch is basically one scan through all the data.

So in this particular example, we're just going to ask it to look at all the pictures 20 times in order to do the training.

And then we're also pointing it to the validation set.

That's our test set, the 20% that we sectioned off that it has not seen before.

And when we click play, you can see that you can see that I spelled epochs wrong.

And then finally, it's going to actually start to do its thing.

Now, the way that we formatted it for this tutorial today, it shouldn't really take too long to go through all the data.

But this will take about a few minutes or so.

So we're just going to click play and then basically step back.

We'll come back in a few minutes.

All right, so we're back.

It's been three minutes of training, which is really not that much time at all.

But that's OK.

That's enough for us to understand some of the concepts that we're working with today.

So now what we're going to do is we're going to see exactly how good of a job it did over time.

So basically, we're just going to plot how the accuracy changed.

And this is basically the accuracy for the training set, which should go up over time.

And then this is the accuracy for the validation set, the testing data set, which is basically the accuracy for the , you know, it hasn't seen before.

So we hope that this goes up.

But, you know, this is why we had to keep it separate, because this will actually show us like if it's doing a good job or not.

I do the same thing for loss.

And basically, loss is sort of another kind of abstract way of thinking about how good a job it's doing.

So loss is basically, you know, you want your accuracy to be going up over time, and you want your loss to be going down over time.

And loss is basically sort of a mathematical way of talking about, you know, the way that the network should look and the way that the network does look as it currently is.

So I'm just going to plot using matplotlib, some different stuff related to the loss and related to the accuracy as it changes over time.

We'll do the same thing here, but for the validation data set.

And again, this is going to be the training and test accuracy as it changes over time.

I'll just put this in here like this.

Let's click play.

And this is actually pretty encouraging.

So, you know, as this thing ran for a few minutes, only after a few minutes, it started to figure out what was what, what was cartomegaly and what wasn't.

So the training data set, we expect that to go up.

We expect that to get more accurate over time.

But the testing data set we hope gets more accurate.

And as you can see, it did.

So when it first started, it had like a 50-50 chance.

But after just a few minutes of training, it got to be roughly 70% or so accurate.

And, you know, that's better than flipping a coin.

I'm also going to go ahead and just do the same thing for loss, which again, that's another way of thinking about how good a job our model is doing.

And so instead of accuracy, we're just going to plot loss.

And I'm just going to like change this here so that we have a slightly better idea of what's going on.

So here's our loss.

And as you can see, this goes down quite a lot over time.

We could still, you know, we could zoom in on this to see a little bit more, but that's kind of the basic idea here.

All right, so now that we've trained a model, let's see how well it does.

And this section will basically have two parts.

So first, we'll just play around with a few images to see like what the model thinks.

And then we'll systematically look through all of the images with statistics, thinking about things like sensitivity, specificity, and AUC, or area under the curve.

You know, I think that kind of what distinguishes applications of AI in medicine is attention to all these details, because I think that, you know, these metrics are really important when it comes to thinking about whether, you know, this is really something that's going to ultimately help people.

All right, so let's start off with just like two different helper methods.

This first one here is just going to be a little helper method to load up an image like this.

We're going to resize it to fit the size that our model requires, which looks like that.

And we have to do an actual parentheses here.

From here, it's going to load it into a numpy array.

This is something we have to use just again to convert it from like zero to 255.

That is each pixel is going to go from a value between zero to 255 to zero and one.

This is just making sure that the array has three different values for like red, green, and blue.

Even though it's true, we aren't really using that, because x-rays are like usually all hopefully all in gray.

Okay, and then this line here.

This is what's going to return.

So this line model.predict.

This is how we're going to interact with our model to actually return a value between zero and one, where zero means that the model thinks there's no finding, and one means it thinks there is a finding, which here is cardiomegaly.

We're going to do another helper method here, and this is sort of like a sanity check for us.

So what this is ultimately going to do is this is going to show us an image.

It's going to show us like the actual file path that's associated with it.

Let's go ahead and load this up here.

And then, because our model returns us a value between zero and one, what we're going to do is we're going to define some cutoff point, which here I'm going to say is 0.5, where if the prediction is above 0.5, then the model's going to think that it's positive, right? So in this case, the model's going to think that it has cardiomegaly.

And this will make a little bit more sense in a minute once we actually like use this method, but this is just basically showing us all of the different information about the image as we're doing the prediction on it.

So this is the way that I'm going to just access all of that information.

Guess, plus, score, okay.

And then once we're down here, this will basically just use matplotlib to go ahead and get all of that up and running.

Let's go ahead and click play there.

Let's make sure that we added all of the plus signs.

And again, you know, one thing that's really important is that we want to be systematic in the way that we evaluate all of our data.

So what we're going to do here is we're going to iterate through all of the pictures, all the images in our test set, and basically try to just systematically, like in an organized way, figure out what I thought about everything.

This is going to be really important when it comes to getting things like sensitivity and specificity, which I'll come back to in one second.

But basically what we're going to do here is we're going to go through all of the negative images, so all the images in which there is no finding.

It was labeled, that is, as no finding.

And we're just going to do predictions on every single one of them.

So we have this array called results array.

And basically what we're going to do is we're just going to create this kind of like results array with all of the information that we care about.

So each row in the results array is going to be the file name, like the image itself, whether or not it had a label of being positive or negative.

These are all the negative images here, so it's going to have a negative label.

The guess, so what it happened to think.

And then the confidence, which again is that number between 0 and 1, where we're using like a 0.5 cutoff for positive versus negative, or cardiomegaly versus not cardiomegaly.

That first part was looking at all the negative images.

And now we're going to do the same thing with all the positive images.

I'm just going to change this down here, because again, these are like positive labels.

So at this point, we'll have some array with a whole bunch of stuff in it, with a whole bunch of predictions for all of the positive and negative test images.

What I'm going to do here is I'm just going to sort this array on basically that last column, which is like the confidence column.

So basically it's going to show us in order what it really thought was cardiomegaly, and then it's going to show us what it really thought was not cardiomegaly, what was no finding at all.

We're going to create a data frame from all these results here.

And then so that we can be kind of organized, we are going to create an actual list of column names too, that would be helpful.

File path, file name, label, guess, confidence.

And then once we click this here, it's going to go through and basically just make a guess on all the different images in our test set.

Okay, let's scroll.

Again, this array that we created, or this data frame that we created, let's just take a sneak peek to make sure that this makes sense.

Cool.

And here's the first five rows, right? So here's just five images in our data set where these are the five images that our model thought had the most cardiomegaly.

All right, so that helper method that we did earlier, where it was like a sanity check to see if our model was actually working, let's go ahead and actually make a call to that.

So first, we're just going to grab a random number, like from the test set.

And then we're going to grab a random row from our data frame, which is like all the predictions on the test set.

And this is that helper method that we used earlier, right? So here's a random picture.

You can see that it was labeled as having cardiomegaly, and the model guessed there was cardiomegaly.

Here's an example where this was labeled as not having cardiomegaly.

I agree, I think that's a normal heart.

And our model also thought there was no cardiomegaly.

Here's an example where it was labeled as having cardiomegaly, and our model got it wrong.

Here's an example where the model said that there was cardiomegaly, and it was labeled also as cardiomegaly.

And just like checking it out, I think that's a pretty big heart, so I have to agree with that.

We can click this button a whole bunch of different times to see whether or not it was accurate or not.

We can also use some numbers, like again, sensitivity and specificity, AUC, to help us also get a better feel for if it's doing a good job or not.

So we'll get to that in one second, but let's just go ahead and show the entire data frame right now.

Or rather, let's show every fifth row in the data frame, just to get another feel for what it had.

So this is going to show the name of the file, whether or not it was labeled as positive or negative, whether or not it was labeled as positive or negative, and basically the confidence that it had.

So just looking at this really quick kind of overview, it looks like it did a pretty decent job, but some of the ones here in the middle I wasn't so sure about.

So in other words, you know, when it was really confident that there was cardiomegaly, it got it right.

When it was really confident that there was not cardiomegaly, it got it right.

But some of these ones in the middle, it kind of was a little bit iffy.

So now I want to show the same information in a histogram format, and this is good because this is going to kind of start to get us to think about some of those statistics that we mentioned earlier.

So this is the way that I'm going to build the histogram.

I'm going to grab the same thing here for the negative labels.

And then this is going to use the map plot lib histogram function.

That was kind of a mouthful there, but just some more stuff setting up this histogram chart.

X-axis, title, confidence, scores for different images.

And make a legend.

Okay.

Plot.show.

And here you can start to see exactly how good of a job it did.

We'll go ahead and just scoot this legend a little bit out of the way.

And this is a little bit better, right? So you can see that when the model was pretty confident, it got it right on both sides.

So every example close to a one, it got it right.

Every example close to a zero, it got it right.

But in the middle, it was kind of iffy.

Overall, this is actually pretty decent, though, in my opinion.

So now let's go ahead and look at a confidence score.

So now let's think about, you know, whether...

So earlier we had mentioned that, you know, again, the model was returning a value between zero and one.

And we were using like 50 or 0.5 as the cutoff.

So if it was above 0.5, we would say that it was positive, that the model thought that there was card immediately.

And if the model returned a number between zero and 0.5, we would think that it had no finding.

Let's see if maybe that was the best number for it to use.

So here we're going to create a helper function.

Called createWithCutoff that basically is going to kind of just redraw our histogram, but now we're going to talk about false negatives, false positives, true negatives, and true positives.

So this line here, this is saying, let's find everything where it was labeled as positive.

And the model thought that it was positive, or it thought that the confidence value was more than that.

This is going to give us an array of like exactly all of the confidence values that are above our cutoff.

Let's do the same thing with false positive, true negative, and false negative.

So for false positive, that means that the real label was actually negative, but we thought it was positive.

For true negative, that means that the real label was negative, and our model said that it was lower than the cutoff.

And I'm going to go ahead and just fix this right here before I forget.

And then again for false negatives, finally, that's going to mean that, you know, this was labeled as positive, but the model accidentally thought it was less than our cutoff value.

Here we're just going to make another histogram, but we're just going to kind of tweak it a little bit.

So now instead of having two different colors, we're going to have four different colors, one for each of those four categories we talked about.

It's going to be pretty similar otherwise.

And then all this stuff here is going to be basically just the same as it was last time.

So I won't go through basically all this again.

Just going to type another title in, so confidence scores for different images.

This line here, this is going to draw like a vertical line that helps us kind of differentiate where that cutoff value is, and I'll come back to that in a second as to why we care about that.

We're going to put this in the upper right.

All right, and now this part here is pretty important.

So now we're going to calculate the sensitivity.

So sensitivity basically is something that we use a mess and a lot to see if we should roll something in.

So like a screening test, generally, you want to have high sensitivity for that.

And the formula for sensitivity is true positives over true positives plus false negatives.

So another way of saying this is that something that's really sensitive has a low number of false negatives.

It might have a lot of false positives, so it might just think that like everything is positive, but at least we're avoiding false negatives that way.

And then, you know, the general scheme of things is that once we have a confirmatory test, then we want high specificity.

So for that, it's kind of the other way around where we want there to be basically a low false positive rate.

So, you know, something that's like an example of this, like if you remember, you know, in 2020, when we wanted to know more about whether someone had COVID, you could use like the screening test.

Sometimes that would be like the self swab test or a spit test.

I guess that was a thing.

But ultimately, the PCR test was the most specific.

So that was like the more confirmatory test.

A lot of the screening tests would have high false positives, but the PCR test, that was like a lot more accurate.

But the PCR test was like a little bit more specific.

Here, I'm just going to actually like spell it out on the thing itself.

All right.

And here, we're going to go ahead and basically say that we want a cutoff of 0.5.

So let's try this out.

Okay.

And as you can see, now that we've defined a cutoff of 0.5, now we can start to think about true positives, false positives, true negatives and false negatives.

So here you can see that if we use a confidence level of 0.5, we have a relatively low rate of false positives, but higher amount of false negatives.

Let's see about if we change this value to be something like that.

How does that change our sensitivity and specificity? Well, if we lower that, then we have higher sensitivity, but we have worse specificity.

And basically, the big picture here is that a lot of tests in medicine kind of struggle with this.

Like, you know, how do we define a cutoff value for whether or not someone has a disease? For example, if you have diabetes, you can get a test called an A1C that basically looks at the amount of sugar in your blood over time.

And, you know, what should the cutoff be for someone who has diabetes versus someone who doesn't have diabetes? Now, the answer to that is 6.5.

And a lot of testing went in to figure out what's the best value for that particular test.

But here we're kind of thinking about a similar thing.

I mean, what is the cutoff for whether or not our model thinks you have cardiomegaly or doesn't have cardiomegaly? And the kind of important concept here is that, you know, we just arbitrarily chose 0.5, but that might not actually be the very best value.

And a lot of issues that might not be the very best value.

And, you know, ultimately, if our goal is to build something that can look at a chest x-ray and diagnose a disease, we want to be totally sure that whatever cutoff value it has for saying yes or no, we want to be sure that it's the very, very best cutoff value.

So for this next part, we're going to talk about ROC, or receiver operator curve, which is related to AUC, or area under the curve.

And this concept is basically how we can kind of figure out what is the very best cutoff point for this particular test, for this particular AI model, which again is going to say yes or no.

So let's build a method to create an AUC curve.

And basically what an AUC curve is going to do is it's going to do every single possible cutoff between 0 and 1.

It's going to calculate the sensitivity and specificity for all those possible values and classifications.

And that's going to give us a lot more information through which we can then figure out what's the best cutoff value.

So what we're going to do here is we're going to feed this thing all of the guesses that it made earlier.

So we're going to have it go through each one and basically now create that AUC curve.

So this is basically just going through all of the guesses that it had made.

And we'll take it that way.

Alright, so now we're just going to return basically all of the numbers of true positives, false positives, true negatives, and false negatives.

We're going to manually calculate the sensitivity, which we talked about earlier.

So we'll go ahead and do it this way, like this.

Specificity is going to be like this.

Oops.

Alright, so this is sort of the first step.

And then now that we've created this function, AUC curve, sorted results.

Now basically from here, we're just going to create a line that basically strings together like all of these different sensitivities and specificities for all these different possible cutoff values.

And now that we have all these different values as well, we're going to make sure that we calculate what's called the area under the curve, which basically like numerically represents how good of a model this is.

Or in other words, it's basically how close is our AUC curve to being perfect.

So here's just some code to create a line plot.

Alright, there's the rest of the code there.

And now once we go ahead and run, you can see that our AUC curve is actually pretty good.

It has a pretty high area of 0.923.

One thing that's kind of funky about this curve, about this whole graph, is that you notice the x-axis here is kind of like backwards, like one minus specificity.

The reason for that is basically, you know, as sensitivity goes up, as a test becomes more sensitive, it gets less specific.

So as the number of false negatives goes down, the number of false positives is going to go up.

So here, you know, like this point on the curve up here, this test is like super sensitive, which really just means that it doesn't really have that many false negatives.

But you can see there's going to be a lot of false positives, so it's not going to be as specific as it could be.

Overall, this is a pretty good ROC.

And you know, given that we only trained this model for like three minutes, and we only fed it like 100 images instead of like thousands of images, this is pretty encouraging.

You know, if we were to redo this experiment with like a way more powerful model, with like way more images, and with a lot more training time, so three minutes, something like maybe, you know, overnight or whatever.

There's no reason why this ROC curve couldn't get way closer to one.

But again, 0.923 is already like pretty good as it is.

If you want to go ahead and save your model, that's pretty easy to do with the code base that we're using.

You can just do model.save.

We'll save it to the content part.

And we'll just go ahead and put it like this so that you can find it maybe a little bit more easily.

You also might want to zip all this stuff up.

And what's really cool about Colab is that you can do just exactly that by hitting the exclamation point when you start in Colab that allows us to do like shell shell commands.

So if you want to go ahead and try all this here, we'll click play.

And in a minute, you'll see that the model is basically saved to here and zipped up.

I took like two minutes or so for it to zip everything up there but anyway, just click on this icon here.

And then here's our model which is about a gigabyte or so big.

Click on those three dots, download, and then that'll save it to your computer in a minute or so.

Alright, well that just about wraps up this presentation.

If you have any questions, feel free to reach me at that Twitter account above or comment on this video below.

And I hope you enjoyed that and thanks for your attention.

Thanks for your time.

We just published a 37-hour course on the freeCodeCamp.org YouTube channel that will teach you about deep learning for computer vision using TensorFlow. The course was expertly created by Folefac Martins from Neuralearn.ai.

A Sneak Peek into the Course

This course is meticulously designed to cover a broad range of topics, starting from the basics of tensors and variables to the implementation of advanced deep learning models for complex tasks such as human emotion detection and image generation.

After introducing the prerequisites and discussing what learners can expect from the course, the first segment focuses on the foundational aspects of tensors and variables. You'll understand the basics, initialization and casting, indexing, and common TensorFlow functions. The topics extend to cover the intriguing concepts of ragged, sparse, and string tensors, laying the groundwork for building neural networks.

As you venture into the world of neural networks, you'll start by predicting car prices. This practical project involves steps from data preparation to measuring model performance, and it'll provide an understanding of linear regression models, error sanctioning, and training and optimization techniques.

The course then delves into convolutional neural networks (ConvNets), which are particularly useful for image data. You will use ConvNets to diagnose malaria, a task that includes data preparation, visualization, and processing, and learn how to build ConvNets with TensorFlow. Along the way, you'll explore binary cross-entropy loss, model training and evaluation, and saving and loading models on Google Drive.

Advanced topics in TensorFlow, such as custom loss and metrics, eager and graph modes, and custom training loops, are also thoroughly discussed. A significant portion of the course is devoted to improving model performance, evaluating classification models, and using data augmentation techniques to enhance the quality and diversity of data.

The course proceeds to explore modern Convolutional Neural Networks like AlexNet, VGGNet, ResNet, MobileNet, and EfficientNet, applied to a human emotions detection project. Additionally, the course illustrates the black box of these models by visualizing intermediate layers and using the Gradcam method.

There's a great section dedicated to Transformers in Vision, understanding and building Vision Transformers (ViTs) from scratch, and fine-tuning Huggingface ViT. This section includes practical training with the Weights and Biases tool for experiment tracking, hyperparameter tuning, dataset and model versioning, known as MLOps.

Finally, the course closes with important topics in model deployment, including converting TensorFlow models to Onnx format, understanding and implementing quantization, building and deploying an API with FastAPI, and load testing with Locust.

The course concludes with a module on object detection using the YOLO algorithm and image generation using Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs).

The Learning Experience

What sets this course apart is the combination of theoretical understanding and practical applications. It is a guided journey through the intricacies of TensorFlow, deep learning, and computer vision, using real-world projects such as car price prediction, malaria diagnosis, human emotion detection, and image generation.

The course is perfect for anyone passionate about machine learning and AI, regardless of their current expertise level. So whether you're a complete beginner, a data scientist looking to update your skills, or an AI enthusiast, this course promises a thorough and practical understanding of computer vision and deep learning with TensorFlow.

Watch the full course on the freeCodeCamp.org YouTube channel (37-hour course, with subtitles).

We can use TensorFlow to train simple to complex neural networks using large sets of data.

TensorFlow is used in a variety of applications, from image and speech recognition to natural language processing and robotics. TensorFlow enables us to quickly and easily build powerful AI models with high accuracy and performance.

TensorFlow also works with GPUs and TPUs, which are types of computer chips built to extend TensorFlow’s capabilities. These chips make TensorFlow run faster, which is helpful when you have a lot of data to work with.

In this article, we will learn about tensors and how to work with tensors using TensorFlow. Let’s dive right in.

What is a Tensor?

A simple explanation would be that a tensor is a multi-dimensional array.

Scalar, Vector, Matrix and Tensor

A scalar is a single number. A vector is an array of numbers. A matrix is a 2-dimensional array. A tensor is an n-dimensional array.

In TensorFlow, everything can be considered a tensor including a scalar. A scalar would be a tensor of dimension 0, a vector of dimension 1, and a matrix of dimension 2.

Now, this is useful because we are not limited to working with complex datasets in TensorFlow. TensorFlow can handle any type of data and feed it to machine learning models.

What is TensorFlow?

TensorFlow is an open-source software library for building neural networks. Google Brain team was the one who built it and it is the most popular deep learning library in the market today.

You can use TensorFlow to build AI models including image and speech recognition, natural language processing, and predictive modeling.

Classification neural network

TensorFlow uses a dataflow graph to represent computations. To put it simply, TensorFlow has made it easy to build complex machine learning models.

TensorFlow takes care of a lot of work behind the scenes which makes it useful while building and training any type of machine learning model. TensorFlow also manages the computation, including parallelization and optimization, on the user’s behalf.

TensorFlow and Keras

Tensorflow and Keras

TensorFlow has a high-level API called Keras. Keras was a standalone project which is now available within the TensorFlow library. Keras makes it easy to define and train models while TensorFlow provides more control over the computation.

TensorFlow supports a wide range of hardware, including CPUs, GPUs, and TPUs. TPUs are Tensor processing Unites, built specifically to work with Tensors and TensorFlow.

We can also run TensorFlow on mobile devices and IoT devices using TensorFlow Lite. TensorFlow also has a large community of developers, and it is updated with new features and capabilities.

How to Build Tensors with TensorFlow

Let’s start writing some code. If you don't have TensorFlow installed, you can use a Google colab notebook to follow along.

Let’s start by importing TensorFlow and printing out the version.

import tensorflow as tf
print(tf.__version__)

OUTPUT:
2.9.2

Let’s first create a scalar using tf.constant. We use tf.constant to create a new constant value. We can also use tf.variable to create a variable value. We will then print the value and also check the dimension of the scalar using the ndim property. Its dimension will be zero because it is a single value.

scalar = tf.constant(7)
print(scalar)
print(scalar.ndim)

OUTPUT:
tf.Tensor(7, shape=(), dtype=int32)
0

Now let’s create a vector and print its dimensions. You can see that the dimension is 1.

vector = tf.constant([10,10])
print(vector)
print(vector.ndim)

OUTPUT:
tf.Tensor([10 10], shape=(2,), dtype=int32)
1

Now let’s try creating a matrix and printing its dimensions.

matrix = tf.constant([
    [10,11],
    [12,13]
])
print(matrix)
print(matrix.ndim)

OUTPUT:
tf.Tensor(
[[10 11]
 [12 13]], shape=(2, 2), dtype=int32)
2

You will see that the dimension is now 2. You can also see that the shape of the matrix is 2 by 2.

Shapes and dimensions are useful when working with TensorFlow because we will often change them while using these data to train neural networks.

We have seen that these tensors have a default datatype of int32. What if we want to create a dataset with a custom datatype?

tf.constant provides us with the dtype argument. Let’s create the same matrix again with float16 as the data type.

tensor_1 = tf.constant([
    [
        [1,2,3]
    ],
    [
        [4,5,6]
    ],
    [
        [7,8,9]
    ]
],dtype='float32')
print(tensor_1)

OUTPUT:
tf.Tensor(
[[[1. 2. 3.]]

 [[4. 5. 6.]]

 [[7. 8. 9.]]], shape=(3, 1, 3), dtype=float32)

Now let’s create a tensor. We will input a 3-dimensional array to tf.constant. We will also print its dimensions.

tensor = tf.constant([
    [
        [1,2,3]
    ],
    [
        [4,5,6]
    ],
    [
        [7,8,9]
    ]
])
print(tensor)
print(tensor.ndim)

OUTPUT:
tf.Tensor(
[[[1 2 3]]
 [[4 5 6]]
 [[7 8 9]]], shape=(3, 1, 3), dtype=int32)
3

Now we have a tensor of dimension 3 and shape 3 by 1 by 3. This is the simplest tensor you can create. In real-world scenarios, we will be dealing with tensors of higher dimensions and bigger shapes.

Now let’s look at how to create a variable tensor. We won’t be using variable tensors very often compared to constant tensors, but it is good to know that we have an option.

We will use tf.Variable to create a variable tensor. The difference between the constant tensor and variable tensor is that you can change the data in a variable tensor, but you can’t change the values in a constant tensor. Let’s create a variable tensor and print the dimensions.

var_tensor = tf.Variable([
    [
        [1,2,3]
    ],
    [
        [4,5,6]
    ],
    [
        [7,8,9]
    ]
])
print(var_tensor)

OUTPUT:
<tf.Variable 'Variable:0' shape=(3, 1, 3) dtype=int32, numpy=
array([[[1, 2, 3]],
       [[4, 5, 6]],
       [[7, 8, 9]]], dtype=int32)>

How to Generate and Load Tensors

Let’s look at how to generate tensors. In most cases, you won’t be creating tensors from scratch. You will either load a dataset, convert other datasets like NumPy arrays to tensors, or generate tensors. First, let’s look at how to generate tensors.

Let’s create a tensor with random values. There are two common ways you can do this: generate a normal distribution of data or a uniform distribution of data.

Normal distribution

The normal distribution is a bell-shaped curve that represents the distribution of data. Most of the data will be close to the average and fewer data will be away from the average. This means the probability of getting a value near the average is higher.

Uniform distribution

The uniform distribution is a straight line that represents the distribution of data. All the values in a uniform distribution will have an equal probability of occurring within a given range.

Before we generate random values, you must understand what a seed is. If we use a seed value, we can regenerate the same set of data multiple times. This will be useful when we want to test our machine-learning model against the same data after we tweak its performance.

Let’s create two arrays of random tensors. We will first set a seed and generate the random values using that seed.

seed = tf.random.Generator.from_seed(42)

Now we will create a normal and uniform distribution with the shape of 3 by 2.

normal_tensor = seed.normal(shape=(3,2))
print(normal_tensor)
uniform_tensor = seed.uniform(shape=(3,2))
print(uniform_tensor)

OUTPUT:
tf.Tensor( [[-0.7565803  -0.06854702]  [ 0.07595026 -1.2573844 ]  [-0.23193765 -1.8107855 ]], shape=(3, 2), dtype=float32)
tf.Tensor( [[0.7647915  0.03845465]  [0.8506975  0.20781887]  [0.711869   0.8843919 ]], shape=(3, 2), dtype=float32)

We have two tensors created, one with a normal distribution of random numbers and the other with a uniform distribution of random numbers.

Next, we will create a tensor with zeros and ones. In TensorFlow, tensors filled with zeros or ones are often used as a starting point for creating other tensors. They can also be placeholders for inputs in a computational graph.

To create a tensor of zeroes, use the tf.zeros function with a shape as the input argument. To create a tensor with ones, we use tf.ones with the shape as input argument.

zeros = tf.zeros(shape=(3,2))
print(zeros)
ones = tf.ones(shape=(3,2))
print(ones)

OUTPUT:
tf.Tensor(
[[0. 0.]
 [0. 0.]
 [0. 0.]], shape=(3, 2), dtype=float32)
tf.Tensor(
[[1. 1.]
 [1. 1.]
 [1. 1.]], shape=(3, 2), dtype=float32)

Now, let’s look at converting NumPy arrays into tensors. If you don’t know what NumPy is, it is a Python library for numerical computing. It helps us handle large datasets and perform a variety of computations on them.

Let’s import NumPy and create a NumPy array using NumPy’s arrange function.

import numpy as np
numpy_arr = np.arange(1,25,dtype=np.int32)

Now, we can create a tensor using the tf.constant function with the NumPy array as input. TensorFlow has built-in support to handle NumPy arrays, so it is just a matter of importing a NumPy array and setting a shape.

print(numpy_arr)
numpy_tensor = tf.constant(numpy_arr,shape=[2,4,3])
print(numpy_tensor)

OUTPUT:
[ 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]
tf.Tensor(
[[[ 1  2  3]
  [ 4  5  6]
  [ 7  8  9]
  [10 11 12]]
 [[13 14 15]
  [16 17 18]
  [19 20 21]
  [22 23 24]]], shape=(2, 4, 3), dtype=int32)

You can see both the NumPy array as well as our tensor. The original NumPy array was 1x12 but our tensor is 2x4x3. This is called re-shaping a tensor which we will often do while training deep neural networks.

Basic Operations using Tensorflow

We have learned how tensors are created in TensorFlow. Now let’s look at some basic operations using tensors.

We will start by getting some information on our tensors. Let’s create a 4D tensor with 0 values with the shape 2x3x4x5.

rank4_tensor = tf.zeros([2,3,4,5])
print(rank4_tensor)

OUTPUT:
tf.Tensor(
[[[[0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]]
  [[0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]]
  [[0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]]]
 [[[0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]]
  [[0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]]
  [[0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]
   [0. 0. 0. 0. 0.]]]], shape=(2, 3, 4, 5), dtype=float32)

We have created our rank 4 tensor. Now let's get some information about the size, shape (number of values), and the dimension of the tensor.

We will use tf.size function to get the size. The shape and ndim properties will give us the shape and dimensions of the tensor.

print("Size",tf.size(rank4_tensor))
print("shape",rank4_tensor.shape)
print("Dimension",rank4_tensor.ndim)

OUTPUT: 

Size tf.Tensor(120, shape=(), dtype=int32)
shape (2, 3, 4, 5)
Dimension 4

Let’s look at some simple calculations using the tensor. I will create a new basic tensor.

basic_tensor = tf.constant([[10,11],[12,13]])
print(basic_tensor)

OUTPUT: 

tf.Tensor(
[[10 11]
 [12 13]], shape=(2, 2), dtype=int32)

Let’s try some simple operations. We can add, subtract, multiply, and divide every value in a tensor using the basic operators.

print(basic_tensor + 10)
print(basic_tensor - 10)
print(basic_tensor * 10)
print(basic_tensor / 10)

OUTPUT:
tf.Tensor(
[[20 21]
 [22 23]], shape=(2, 2), dtype=int32)
tf.Tensor(
[[0 1]
 [2 3]], shape=(2, 2), dtype=int32)
tf.Tensor(
[[100 110]
 [120 130]], shape=(2, 2), dtype=int32)
tf.Tensor(
[[1.  1.1]
 [1.2 1.3]], shape=(2, 2), dtype=float64)

Now let’s try matrix multiplication. I will create two simple tensors tensor_011 and tensor_012.

tensor_011 = tf.constant([[2,2],[4,4]])
tensor_012 = tf.constant([[2,3],[4,5]])

Keep in mind that in matrix multiplication, the inner dimensions should match. For example, a (3, 5) (3, 5) multiplication won’t work but (3, 5) (5, 3) will work.

The final shape of the resulting matrix will be its outer dimension. so, a 3x5 tensor multiplied by a 5x3 tensor will give us a 5x5 tensor. We will use the tf.matmul function to perform matrix multiplication.

print(tf.matmul(tensor_011,tensor_012))

OUTPUT:
tf.Tensor(
[[12 16]
 [24 32]], shape=(2, 2), dtype=int32)

Next, let’s look at reshaping and transposing a matrix. As we saw before, we will often use reshaping to change our matrix structure while training neural networks.

For example, an image pixel matrix of 28x28 will be converted into a 1-dimensional 784-pixel array for an image classification neural network.

To reshape, we use the tf.reshape function. To transpose, we use the tf.transpose function. If you don't know what a transpose is, it's converting rows into columns and columns into rows.

print(tf.reshape(tensor_011,[4,1]))
print(tf.transpose(tensor_011))

OUTPUT:
tf.Tensor(
[[2]
 [2]
 [4]
 [4]], shape=(4, 1), dtype=int32)
tf.Tensor(
[[2 4]
 [2 4]], shape=(2, 2), dtype=int32)

Finally, let’s look at some aggregate operations like min, max, standard deviation, square and square root.

To find the minimum and maximum values, we use the tf.reduce_min and tf.reduce_max functions. And to find the sum of the array, we use the tf.reduce_sum function.

tensor_013 = tf.constant([
    [1,2,3],
    [4,5,6],
    [7,8,9]
],dtype='float32')
print(tf.reduce_min(tensor_013))
print(tf.reduce_max(tensor_013))
print(tf.reduce_sum(tensor_013))

OUTPUT:
tf.Tensor(1.0, shape=(), dtype=float32)
tf.Tensor(9.0, shape=(), dtype=float32)
tf.Tensor(45.0, shape=(), dtype=float32)

Now for the standard deviation and variance, we use the tf.math.reduce_std function and tf.math.reduce_variance function.

print(tf.math.reduce_std(tensor_013))
print(tf.math.reduce_variance(tensor_013))

OUTPUT:
tf.Tensor(2.5819888, shape=(), dtype=float32)
tf.Tensor(6.6666665, shape=(), dtype=float32)

Let’s find the square, square root, and log of each value in a tensor.

print(tf.sqrt(tensor_013))
print(tf.square(tensor_013))
print(tf.math.log(tensor_013))

OUTPUT:
tf.Tensor(
[[1.        1.4142135 1.7320508]
 [2.        2.236068  2.4494898]
 [2.6457512 2.828427  3.       ]], shape=(3, 3), dtype=float32)
tf.Tensor(
[[ 1.  4.  9.]
 [16. 25. 36.]
 [49. 64. 81.]], shape=(3, 3), dtype=float32)
tf.Tensor(
[[0.        0.6931472 1.0986123]
 [1.3862944 1.609438  1.7917595]
 [1.9459102 2.0794415 2.1972246]], shape=(3, 3), dtype=float32)

We have learned the basics of TensorFlow in this article. You are now equipped to work with TensorFlow and use it to model data.

If you want to start using this knowledge and build a project, you can check out my course on building a handwriting recognition neural network using TensorFlow. You can also learn advanced TensorFlow concepts using the official documentation.

Conclusion

Tensorflow is a powerful library to build deep-learning models. It has all the tools we need to construct neural networks to solve problems like image classification, sentiment analysis, stock market predictions, etc.

With the advent of technologies like ChatGPT, learning TensorFlow will give you a head start in the current job market.

Hope you liked this article. You can learn more about me and my articles/videos at manishmshiva.com.

But the excitement soon disappeared when I received feedback from users. Simply put, the users felt the model was bad.

I was saddened by this, but looking back, they were correct. The model may have performed well in terms of top-level metrics. But from the perspective of the consumer, if a machine learning model provides a poor forecast, that person's experience with the model will be bad.

The issue was that specific model features, or slices of data, were causing the model to perform poorly.

In short, before deploying any machine learning model, the onus is on machine learning engineers to assess it, make sure it satisfies strict quality standards, and acts as predicted for all pertinent slices of data.

What is TensorFlow Model Analysis?

To enable Machine Learning engineers to look at the performance of their models at a deeper level, Google created TensorFlow Model Analysis (TFMA). According to the docs, "TFMA performs its computations in a distributed manner over large amounts of data using Apache Beam."

TFMA, as a tool, enables you to really dig into the model's performance and understand how it varies on different slices of data. It provides support for calculating metrics that were used at training time (that is built-in metrics) as well as metrics defined after the model was saved as part of the TFMA configuration settings.

In this tutorial, you will analyze and evaluate results on a previously trained machine learning model. The model you will use is trained for a Chicago Taxi Example, which uses the Taxi Trips dataset released by the city of Chicago. You can check out the full dataset here.

When you are done with this tutorial, you will be able to use Apache Beam to do a full pass over the specified evaluation dataset. Also, you will not only have a more accurate calculation of metrics, but you'll be able to scale up to massive evaluation datasets, since Beam pipelines can be run using distributed processing back-ends.

Prerequisites

• Fundamental knowledge of Apache Beam. The Beam Programming Guide is a great place to start.
• Fundamental understanding of the workings of machine learning models.
• A new Google Colab notebook to run the Python code in your Google Drive. You can set this up by following this tutorial.

Step 1 – How to Install TensorFlow Model Analysis (TFMA)

With your Google Colab notebook ready, the first thing to do is to pull in all the dependencies. This will take a while.

A blank (new) notebook in dark mode

Rename the file from Untitled.ipynb to TFMA.ipynb.

!pip install -U pip
!pip install tensorflow-model-analysis`

The first line upgrades pip to the latest version. pip is the package management system used to install and manage software packages written in Python. It stands for “preferred installer program”. The second line will install TensorFlow Model Analysis, TFMA.

Now, after that is done, restart the runtime before running the cells below. It is important to restart the runtime before running the cells.

import sys
assert sys.version_info.major==3 
import tensorflow as tf
import apache_beam as beam
import tensorflow_model_analysis as tfma

This block of code imports the needed libraries – sys, tensorflow, apache_beam and tensorflow_model_analysis. You use the assert sys.version_info.major==3 command to verify that the notebook is being run using Python 3.

Step 2 – How to Load the dataset

You will download the tar file and extract it.

import io, os, tempfile
TAR_NAME = 'saved_models-2.2'
BASE_DIR = tempfile.mkdtemp()
DATA_DIR = os.path.join(BASE_DIR, TAR_NAME, 'data')
MODELS_DIR = os.path.join(BASE_DIR, TAR_NAME, 'models')
SCHEMA = os.path.join(BASE_DIR, TAR_NAME, 'schema.pbtxt')
OUTPUT_DIR = os.path.join(BASE_DIR, 'output')

!curl -O https://storage.googleapis.com/artifacts.tfx-oss-public.appspot.com/datasets/{TAR_NAME}.tar
!tar xf {TAR_NAME}.tar
!mv {TAR_NAME} {BASE_DIR}
!rm {TAR_NAME}.tar

The dataset downloaded is in the tar file format. It includes the training datasets, evaluation datasets, the data schema and the training and serving saved models along with eval saved models. You will need all of them in this tutorial.

Step 3 – How to Parse the Schema

You need to parse the downloaded schema so that you can use it with TFMA.

import tensorflow as tf
from google.protobuf import text_format
from tensorflow.python.lib.io import file_io
from tensorflow_metadata.proto.v0 import schema_pb2
from tensorflow.core.example import example_pb2

schema = schema_pb2.Schema()
contents = file_io.read_file_to_string(SCHEMA)
schema = text_format.Parse(contents, schema)

You will parse the schema using the text_format method of the google.protobuf library to convert the protobuf message to text format and TensorFlow's schema_pb2.

Step 4 – How to Use the Schema to Create TFRecords

The next course of action would be to give TFMA access to our dataset. For this, we need to create a TFRecords file. We used our schema to create it, since it gives us the correct type for each feature.

import csv
datafile = os.path.join(DATA_DIR, 'eval', 'data.csv')
reader = csv.DictReader(open(datafile, 'r'))
examples = []
for line in reader:
  example = example_pb2.Example()
  for feature in schema.feature:
    key = feature.name
    if feature.type == schema_pb2.FLOAT:
      example.features.feature[key].float_list.value[:] = (
          [float(line[key])] if len(line[key]) > 0 else [])
    elif feature.type == schema_pb2.INT:
      example.features.feature[key].int64_list.value[:] = (
          [int(line[key])] if len(line[key]) > 0 else [])
    elif feature.type == schema_pb2.BYTES:
      example.features.feature[key].bytes_list.value[:] = (
          [line[key].encode('utf8')] if len(line[key]) > 0 else [])
  # Add a new column 'big_tipper' that indicates if the tip was > 20% of the fare. 
  # TODO(b/157064428): Remove after label transformation is supported for Keras.
  big_tipper = float(line['tips']) > float(line['fare']) * 0.2
  example.features.feature['big_tipper'].float_list.value[:] = [big_tipper]
  examples.append(example)
tfrecord_file = os.path.join(BASE_DIR, 'train_data.rio')
with tf.io.TFRecordWriter(tfrecord_file) as writer:
  for example in examples:
    writer.write(example.SerializeToString())
!ls {tfrecord_file}

It is worthy of note that TFMA supports a number of different model types including TF Keras models, models based on generic TF2 signature APIs, as well TF estimator-based models. However, for this tutorial, you will configure a Keras-based model.

In your Keras setup, you will add your metrics and plots manually as part of the configuration (see the metrics guide for information on the metrics and plots that are supported).

Step 5 – How to Set Up and Run TFMA using Keras

import tensorflow_model_analysis as tfma

You'll finally call and use the instance of tfma that you previously imported at this point.

# You will setup tfma.EvalConfig settings
keras_eval_config = text_format.Parse("""
  ## Model information
  model_specs {
    # For keras (and serving models) we need to add a `label_key`.
    label_key: "big_tipper"
  }

  ## You will post training metric information. These will be merged with any built-in
  ## metrics from training.
  metrics_specs {
    metrics { class_name: "ExampleCount" }
    metrics { class_name: "BinaryAccuracy" }
    metrics { class_name: "BinaryCrossentropy" }
    metrics { class_name: "AUC" }
    metrics { class_name: "AUCPrecisionRecall" }
    metrics { class_name: "Precision" }
    metrics { class_name: "Recall" }
    metrics { class_name: "MeanLabel" }
    metrics { class_name: "MeanPrediction" }
    metrics { class_name: "Calibration" }
    metrics { class_name: "CalibrationPlot" }
    metrics { class_name: "ConfusionMatrixPlot" }
    # ... add additional metrics and plots ...
  }

  ## You will slice the information
  slicing_specs {}  # overall slice
  slicing_specs {
    feature_keys: ["trip_start_hour"]
  }
  slicing_specs {
    feature_keys: ["trip_start_day"]
  }
  slicing_specs {
    feature_values: {
      key: "trip_start_month"
      value: "1"
    }
  }
  slicing_specs {
    feature_keys: ["trip_start_hour", "trip_start_day"]
  }
""", tfma.EvalConfig())

It's also important that you create a tfma.EvalSharedModel that points at the Keras model.

keras_model_path = os.path.join(MODELS_DIR, 'keras', '2')
keras_eval_shared_model = tfma.default_eval_shared_model(
    eval_saved_model_path=keras_model_path,
    eval_config=keras_eval_config)

keras_output_path = os.path.join(OUTPUT_DIR, 'keras')

And then you finally run TFMA, ending this step.

keras_eval_result = tfma.run_model_analysis(
    eval_shared_model=keras_eval_shared_model,
    eval_config=keras_eval_config,
    data_location=tfrecord_file,
    output_path=keras_output_path)

Now that you have run the evaluation, look at the visualizations using TFMA. For the following examples, you can visualize the results from running the evaluation on the Keras model.

To view metrics, you will use [tfma.view.render_slicing_metrics](https://www.tensorflow.org/tfx/model_analysis/api_docs/python/tfma/view/render_slicing_metrics). By default, the views will display the Overall slice. To view a particular slice, you can either use the name of the column (by setting slicing_column) or provide a tfma.SlicingSpec.

Step 6 – How to Visualize the Metrics and Plots

At this point, it is important that you note that the columns used in the dataset are as follows:

• pickup_community_area
• fare
• trip_start_month
• trip_start_hour
• trip_start_day
• trip_start_timestamp
• pickup_latitude
• pickup_longitude
• dropoff_latitude
• dropoff_longitude
• trip_miles
• pickup_census_tract
• dropoff_census_tract
• payment_type
• company
• trip_seconds
• dropoff_community_area, and
• tips

For a first trial and as an example, you can set slicing_column to look at the trip_start_hour feature from our previous slicing_specs. You are then able to visualize the column.

tfma.view.render_slicing_metrics(keras_eval_result, slicing_column='trip_start_hour')

On running this, you will see that the metrics visualization supports the following interactions:

• Click and drag to pan
• Scroll to zoom
• Right click to reset the view
• Hover over the desired data point to see more details.
• Select from four different types of views using the selections at the bottom.

Note that your initial tfma.EvalConfig has created a whole list of slicing_specs, which you can visualize by updating slice information passed to tfma.view.render_slicing_metrics. Here you can select the trip_start_day slice (days of the week).

tfma.view.render_slicing_metrics(keras_eval_result, slicing_column='trip_start_day')

TFMA also supports creating feature crosses to analyze combinations of features. To test this, you will create a cross between trip_start_hour and trip_start_day.

tfma.view.render_slicing_metrics(
    keras_eval_result,
    slicing_spec=tfma.SlicingSpec(
        feature_keys=['trip_start_hour', 'trip_start_day']))

Now, crossing the two columns creates a lot of combinations! But you will narrow down your cross to only look at trips that start at 1pm. Then, you will select binary_accuracy from the visualization as shown below.

tfma.view.render_slicing_metrics(
    keras_eval_result,
    slicing_spec=tfma.SlicingSpec(
        feature_keys=['trip_start_day'], feature_values={'trip_start_hour': '13'}))

Step 7 – How to Track Your Model's Performance Over Time

You'll use your training dataset for training your model. It will hopefully be representative of your test dataset and the data that will be sent to your model in production.

But while the data in inference requests may remain the same as your training data, in many cases it will start to change enough so that the performance of your model will change.

That means that you need to monitor and measure your model's performance on an ongoing basis, so that you can be aware of and react to changes.

Let's look at how TFMA can help.

output_paths = []
for i in range(3):
  # Create a tfma.EvalSharedModel that points to our saved model.
  eval_shared_model = tfma.default_eval_shared_model(
      eval_saved_model_path=os.path.join(MODELS_DIR, 'keras', str(i)),
      eval_config=keras_eval_config)

  output_path = os.path.join(OUTPUT_DIR, 'time_series', str(i))
  output_paths.append(output_path)

  # Run TFMA
  tfma.run_model_analysis(eval_shared_model=eval_shared_model,
                          eval_config=keras_eval_config,
                          data_location=tfrecord_file,
                          output_path=output_path)

  eval_results_from_disk = tfma.load_eval_results(output_paths[:2])

tfma.view.render_time_series(eval_results_from_disk)

Using the tfma, you can validate and evaluate your machine learning models across different slices of data.

You can see from the image above that you can evaluate the auc (area under the curve), auc_precision_recall, binary_accuracy, binary_crossentropy, calibration, example_count, mean_label, mean_prediction, precision, and recall metrics of the machine learning model.

Conclusion

Finally, it is important that TFMA can be configured to evaluate multiple models at the same time. Typically, you do this to compare a new model against a baseline (such as the currently serving model) to determine what the performance differences in metrics (for example AUC) are relative to the baseline.

When thresholds are configured, TFMA will produce a tfma.ValidationResult record indicating whether the performance matches expectations.

If at this point, you have questions about the difference between evaluating machine learning models using TensorBoard and TensorFlow Metrics Analysis (TFMA), this is a valid concern. Both are tools for providing the measurements and visualizations needed during the Machine Learning workflow.

But it is important to note that you use them in different stages of the development process. At a high level, you use TensorBoard to analyze the training process itself while TFMA is concerned with the deep analysis of the 'finished' trained model.

Thank you for reading!

By virtue of their very nature, Machine Learning models improve iteratively. There is hardly any machine learning model that is trained perfectly on the first try. Usuaully, several iterations are required.

As you would imagine, these models have to be evaluated to make them better. In other words, a machine learning model needs to be assessed before it can be improved on.

TensorBoard was developed to give machine learning engineers a more in-depth look at the performance of their models.

What is TensorBoard?

TensorBoard's basic functionality is to deliver the metrics and visualizations you need for your Machine Learning workflow. It allows you to monitor loss and accuracy, view and assess error graphs, and perform many other tasks.

TensorBoard uses graph concepts to represent the data flow and model actions whilst allowing you to see the graph topologies and parameters of complex, huge models. It also has a very user-friendly and basic UI.

In this tutorial, you will analyze and evaluate results on a trained machine learning model. The model you will use will be trained for a MNIST handwritten digits dataset. It uses the MNIST (Modified National Institute of Standards and Technology) database, which contains an ample collection of handwritten digits. This dataset is commonly used for training various image processing systems.

Prerequisites

To complete this tutorial, you will need:

1. Fundamental understanding of the workings of Machine Learning models.
2. A new Google Colab notebook to run the Python code in your Google Drive. You can set this up by following this tutorial.

Step 1 – How to Set Up TensorBoard

Since TensorBoard comes automatically with TensorFlow, you don't need to install it using pip in this setup. Also, since TensorFlow comes pre-installed when you create a new notebook on Google Colab, TensorBoard comes pre-installed as well. So, when setting TensorBoard up, you only need to import tensorflow .

A blank (new) notebook in dark mode

• Load the tensorboard extension using the %load_ext magic in your notebook.
• After doing this, import the necessary libraries (that is, tensorflow and datetime) as shown below:

%load_ext tensorboard

import tensorflow as tf
import datetime

At this point, you have successfully imported an instance of TensorBoard and set it up. You can now get started.

Step 2 – How to Create and Train the Model

In this tutorial, you will use the MNIST dataset, which includes tiny 28 x 28-pixel handwritten single-digit greyscale images. The dataset, which is one of the pre-installed datasets offered by Keras is frequently used to develop Machine Learning models for digit recognition.

• Create an instance of the dataset and name it mnist.
• Split the data into train sets and test sets. A train set is a subset of the original data that is used to train the machine learning model while a test set is the subset that is used to check the accuracy of the model.
• Standardize all the values of your train and test sets. This implies normalizing the image to the [0,1] range.
• Define a function that will be used to train the machine learning model on your dataset. The Sequential Keras model will be used.

mnist = tf.keras.datasets.mnist

(x_train, y_train),(x_test, y_test) = mnist.load_data()

x_train, x_test = x_train/255.0, x_test/255.0

def create_model():
  return tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(512, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
  ])

You will use the Sequential Keras model. At its core, it groups a linear stack of layers into tf.keras.Model whilst providing training and inference features on this model.

The .Flatten() layer flattens the input without affecting the batch size. The input shape in this example is 28 x 28 since the images from the dataset are 28×28-pixel grayscale images of handwritten single-digits. The first .Dense() layer is a regular densely connected NN layer.

The activation function used is 'relu' and the dimensionality of its output space is 512. The .Dropout() layer drops some of the input with the fraction of the input units dropped in this tutorial given as 0.2.

Finally, like the first one, the second. Dense layer is also your regular densely connected NN layer. The activation function we're using is 'softmax' and the dimensionality of its output space is ten.

• Call the defined function for the model like this:
• With the defined function called, train the model with suitable parameters.
• Using the datatime library you previously imported, place the logs in a timestamped subdirectory to allow easy selection of different training runs.

model = create_model()

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

The logs are important because the TensorBoard will read from the logs to display the various visualizations with respect to the time at the point.

log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)

• Finally, train (or fit) the machine learning model on three epochs (iterations).

model.fit(x=x_train, 
          y=y_train, 
          epochs=3, 
          validation_data=(x_test, y_test), 
          callbacks=[tensorboard_callback])

Step 3 – How to Evaluate the Model

To start TensorBoard within your notebook, run the code below:

%tensorboard --logdir logs/fit

You can now view the dashboards showing the metrics for the model on tabs at the top and evaluate and improve your machine learning models accordingly.

Step 4 – How to Improve the Model

Since the point of evaluating your Machine Learning models is to gain better insight to improve the algorithm, it is imperative that we enhance our model. With these visuals, you can now see the in-depth performance of the model.

• The Scalars dashboard can be used to observe other scalar values such as training efficiency and learning rate. It demonstrates how the metrics and loss fluctuate with each epoch.
• As the name implies, the Graphs dashboard is used to visualize your model.

The Graph with the tensorboard

To improve this model, you will adjust the number of epochs from 3 to 6 and see how the model performs.

In general, the number of epochs is the number of iterations over the entire training dataset the machine learning model is trained on.

Intuitively, increasing this number almost always improves the performance of your machine learning model. To do this, you will run the code as follows:

model.fit(x=x_train, 
          y=y_train, 
          epochs=6, 
          validation_data=(x_test, y_test), 
          callbacks=[tensorboard_callback])

With the change we made, you can then generate another TensorBoard like this:

%tensorboard --logdir logs/fit

From the newly generated visuals, you can see that there is a remarkable improvement in the model's performance.

Conclusion

In this article, you learned how you can use TensorBoard to assess and improve your Machine Learning model's performance.

If at this point you have questions about the difference between TensorBoard and TensorFlow Metrics Analysis (TFMA), this is a valid concern. After all, both are tools for providing the measurements and visualizations needed during the Machine Learning workflow.

But it is important to note that you use each of these tools in distinct stages of the development process. At its core, TensorBoard is used to analyze the training process itself, while TFMA is concerned with the analysis of the 'finished' trained model.

Finally, I share my writings on Twitter if you enjoyed this article and want to see more.

Thank you for reading :)

We just published a course on the freeCodeCamp.org YouTube channel that will teach you how to classify text using TensorFlow.

This course will give you an introduction to machine learning concepts and neural network implementation using TensorFlow.

Kylie Ying developed this course. Kylie is a current computer science grad student at MIT, working on research in the domain of machine learning and particle physics. She has a YouTube channel focused on programming tutorials and projects, and is passionate about teaching code and inspiring people to pursue STEM.

Kylie explains basic concepts, such as classification, regression, training/validation/test datasets, loss functions, neural networks, and model training. She then demonstrates how to implement a feedforward neural network to predict whether someone has diabetes, as well as two different neural net architectures to classify wine reviews.

Here are all the sections covered in this course.

• Introduction
• Colab intro (importing wine dataset)
• What is machine learning?
• Features (inputs)
• Outputs (predictions)
• Anatomy of a dataset
• Assessing performance
• Neural nets
• Tensorflow
• Colab (feedforward network using diabetes dataset)
• Recurrent neural networks
• Colab (text classification networks using wine dataset)

Watch the full course below or on the freeCodeCamp.org YouTube channel (2-hour watch).

If you're like I am, then you've probably watched and read a number of tutorials on creating machine learning models with TensorFlow, PyTorch, Scikit-Learn or any other framework out there.

But there is one thing that these tutorials tend to miss out on, and that's model deployment.

In this tutorial, I'll discuss on how to deploy a CNN TensorFlow model that classifies food images to Heroku using FastAPI and Docker.

Tech We'll Be Using

If you're unfamiliar, FastAPI is a Python web framework for creating fast API applications. And in my opinion, it is the easiest to learn out of all the Python web frameworks out there.

FastAPI also has default integration with swagger documentation and makes it easy to configure and update.

Docker, on the other hand, is an industry staple in software engineering, as it is one of the most popular containerization softwares out there. Docker is used for developing, deploying, and managing applications in virtualized environments called containers.

The main selling point of using Docker is that it solves the problem "it works on my machine, why not in yours?". Coincidentally, I actually faced this exact issue working on this very project, ultimately fixing it when I decided to use Docker.

Heroku, lastly, is a cloud platform where you can deploy, manage, and scale web applications. It works with back-end applications, front-end applications, or full-stack applications.

Prerequisuites

Before we begin, you'll first need the following:

1. A Docker account
2. A Heroku account, and the Heroku CLI
3. A Python installation

The Application We're Building

We're going to be building a RESTful API service for a TensorFlow CNN model that classifies food images.

After building the API service, I'll show you how to dockerize the application, and then deploy it to Heroku.

How to Download the Necessities

You'll first need to clone the GitHub repository at this link.

git clone https://github.com/eRuaro/food-vision-api.git

There are two branches in this repository – you'll use the start-here branch as main is the completed branch.

Once you've gotten the cloned repository, you'll need to download Docker to your local system, and the Heroku CLI as well.

You must also install the following packages on pip:

1. FastAPI
2. TensorFlow
3. Numpy
4. Uvicorn
5. Image

To do so, create a requirements.txt file on the start-here branch, and put in the following. Note that you can use any other version of the listed packages below, as long as they still work together.

fastapi==0.73.0
numpy==1.19.5
uvicorn==0.15.0
image==1.5.33
tensorflow-cpu==2.7.0

After which you can install the packages using the command
pip install -r requirements.txt.

Currently our start-here branch has the saved model file, as well as the Jupyter notebook used in creating the model. The notebook also has the code that implements our API feature. That is, it implements predicting the food class of an image based on its URL link.

Brief introduction to FastAPI

With that in mind, let's start writing the code! In the root directory, create a main.py file. In that file, add the following lines of code:

from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from uvicorn import run
import os

app = FastAPI()

origins = ["*"]
methods = ["*"]
headers = ["*"]

app.add_middleware(
    CORSMiddleware, 
    allow_origins = origins,
    allow_credentials = True,
    allow_methods = methods,
    allow_headers = headers    
)

@app.get("/")
async def root():
    return {"message": "Welcome to the Food Vision API!"}

if __name == "__main__":
    port = int(os.environ.get('PORT', 5000))
    run(app, host="0.0.0.0", port=port)

Running the command python -m uvicorn main:app --reload will run the app, and will listen to changes we make on the server.

Alternatively, you can use python main.py and it will run the app on port 5000, courtesy of the last 3 lines of code. However, this won't let the app listen to changes we make, so you'll have to re-run the app every time you want to see your changes.

We also added the CORSMiddleware which essentially allows us to access the API in a different host. That is, we can extend the app further by creating a front-end interface for it. We won't cover that in this article but I put it here just in case you want to create a front-end to interact with the API as well.

Going to the port where the app is running, you'll get this.

{
    "message": "Welcome to the Food Vision API!"
}

The command python -m uvicorn main:app --reload refers to the following:

main -> The file main.py
app -> The object created inside of main.py with the line app = FastAPI()
--reload -> Make the server restart after code changes

Let's dissect the code we've written so far.

@app.get("/")
async def root():
    return {"message": "Welcome to the Food Vision API!"}

@app is needed for FastAPI commands. The get is an HTTP method, while the "/" is the URL path of that specific API request. Below that we call a function that will return something. Here we just return a simple json message.

That is, we have a template for writing API endpoints with FastAPI.

@app.http_method("url_path")
async def functionName():
    return something

How to Write the API Functionality

Let's write the main API functionality, that is, taking a food image URL from the internet, and predicting the name of that food.

First, let's extend the code that we wrote earlier, import all the required functions that we'll use, and load the model itself.

from fastapi import FastAPI
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file 
from tensorflow.keras.utils import load_img 
from tensorflow.keras.utils import img_to_array
from tensorflow import expand_dims
from tensorflow.nn import softmax
from numpy import argmax
from numpy import max
from numpy import array
from json import dumps
from uvicorn import run
import os

app = FastAPI()
model_dir = "food-vision-model.h5"
model = load_model(model_dir)

...
...
...

if __name == "__main__":
    port = int(os.environ.get('PORT', 5000))
    run(app, host="0.0.0.0", port=port)

After loading in the model, let's add in the food classes that we have, which are based on the Food 101 dataset.

class_predictions = array([
    'apple pie',
    'baby back ribs',
    'baklava',
    'beef carpaccio',
    'beef tartare',
    'beet salad',
    'beignets',
    'bibimbap',
    'bread pudding',
    'breakfast burrito',
    'bruschetta',
    'caesar salad',
    'cannoli',
    'caprese salad',
    'carrot cake',
    'ceviche',
    'cheesecake',
    'cheese plate',
    'chicken curry',
    'chicken quesadilla',
    'chicken wings',
    'chocolate cake',
    'chocolate mousse',
    'churros',
    'clam chowder',
    'club sandwich',
    'crab cakes',
    'creme brulee',
    'croque madame',
    'cup cakes',
    'deviled eggs',
    'donuts',
    'dumplings',
    'edamame',
    'eggs benedict',
    'escargots',
    'falafel',
    'filet mignon',
    'fish and chips',
    'foie gras',
    'french fries',
    'french onion soup',
    'french toast',
    'fried calamari',
    'fried rice',
    'frozen yogurt',
    'garlic bread',
    'gnocchi',
    'greek salad',
    'grilled cheese sandwich',
    'grilled salmon',
    'guacamole',
    'gyoza',
    'hamburger',
    'hot and sour soup',
    'hot dog',
    'huevos rancheros',
    'hummus',
    'ice cream',
    'lasagna',
    'lobster bisque',
    'lobster roll sandwich',
    'macaroni and cheese',
    'macarons',
    'miso soup',
    'mussels',
    'nachos',
    'omelette',
    'onion rings',
    'oysters',
    'pad thai',
    'paella',
    'pancakes',
    'panna cotta',
    'peking duck',
    'pho',
    'pizza',
    'pork chop',
    'poutine',
    'prime rib',
    'pulled pork sandwich',
    'ramen',
    'ravioli',
    'red velvet cake',
    'risotto',
    'samosa',
    'sashimi',
    'scallops',
    'seaweed salad',
    'shrimp and grits',
    'spaghetti bolognese',
    'spaghetti carbonara',
    'spring rolls',
    'steak',
    'strawberry shortcake',
    'sushi',
    'tacos',
    'takoyaki',
    'tiramisu',
    'tuna tartare',
    'waffles'
])

Now that we have the food classes, let's write the main API functionality.

@app.post("/net/image/prediction/")
async def get_net_image_prediction(image_link: str = ""):
    if image_link == "":
        return {"message": "No image link provided"}

    img_path = get_file(
        origin = image_link
    )
    img = load_img(
        img_path, 
        target_size = (224, 224)
    )

    img_array = img_to_array(img)
    img_array = expand_dims(img_array, 0)

    pred = model.predict(img_array)
    score = softmax(pred[0])

    class_prediction = class_predictions[argmax(score)]
    model_score = round(max(score) * 100, 2)

    return {
        "model-prediction": class_prediction,
        "model-prediction-confidence-score": model_score
    }

Here, we make a post request to the endpoint /net/image/prediction/ and provide the image_url as a query parameter. That is, the full endpoint when posting an image URL link would be /net/image/prediction/image_url=image-url.

For simplicity's sake, we give the image_link a default value of "" and when there's no link passed to the endpoint, we simply return a message saying that there's no image link provided.

get_file() downloads the image through the provided URL link, while load_img() loads the image in PIL format, and turns it into the appropriate image size that the model wants.

img_to_array() converts the loaded image to a NumPy array. expand_dims() expands the dimensions of the array by one at the zero'th index.

We then use model.predict() to get the model prediction on the loaded image, and get the model's confidence score on said prediction using softmax(). I used softmax here as that's the activation function used in creating the model.

We finally then get the food type by using argmax() on the model's confidence score. We'll use that as the index that we'll use in searching through the class_predictions array which contains the various food classes we have.

Lastly, we multiply the model's confidence score by 100 so that the range of the score would be from 1 to 100.

We then return the model's prediction, and the model's confidence score.

Why We Need to Use Docker to Deploy this App

You can actually deploy this app as is on Heroku, using the usual method of defining a Procfile. But when I tried this method, I kept on getting a ValueError: Out of range float values are not JSON compliant error. I also get this error when running the app on Windows Subsystem for Linux (WSL). When I run on Windows, however, the error disappears.

You can actually avoid this error by adding this line of code, after the initial assignment of the model_score variable:

model_score = dumps(model_score.tolist())

This lets the app run on both Heroku and WSL, but it will only return these values when making the POST request.

{
    "model-prediction": "apple pie",
    "model-prediction-confidence-score": NaN,
}

So, it works on my machine (Windows), but not on Heroku (using Procfile), nor on WSL. This is the kind of problem that Docker solves!

How to Dockerize the Application

Let's start dockerizing the application. Create a Dockerfile in the project's root directory and put in the following content:

FROM python:3.7.3-stretch

# Maintainer info
LABEL maintainer="your-email-address"

# Make working directories
RUN  mkdir -p  /food-vision-api
WORKDIR  /food-vision-api

# Upgrade pip with no cache
RUN pip install --no-cache-dir -U pip

# Copy application requirements file to the created working directory
COPY requirements.txt .

# Install application dependencies from the requirements file
RUN pip install -r requirements.txt

# Copy every file in the source folder to the created working directory
COPY  . .

# Run the python application
CMD ["python", "main.py"]

This pulls the Python 3.7.3 image, and installs all the necessary packages defined in the requirements.txt file. Then it runs the application by using the command python main.py as defined in the last line of the file.

You can then build and run the application using the following CLI commands:

$ docker image build -t <app-name> .
$ docker run -p 5000:5000 -d <app-name>

Then you can stop the app, and free up system resources by running the following:

$ docker container stop <container-id>
$ docker system prune

container-id is returned when running the docker run command above.

How to Deploy to Heroku

With the app now dockerized, we can deploy it to Heroku. I'm assuming you already have the Heroku CLI installed, and have already logged the CLI into your Heroku account.

Let's first create the app in Heroku through the CLI:

$ heroku create <app-name>

Then we can push and release the app through the Docker container we made earlier with the following commands:

$ heroku container:push web --app <app-name>
$ heroku container:release web --app <app-name>

After this, you can go to your Heroku dashboard and open the app. You should be greeted with the JSON message we have in the "/" directory of the application.

JSON message greeting on "/" directory

When you navigate to the /docs you'll be greeted with the Swagger documentation of the application. Here you can play around with the POST request we created and see if the model predictions are correct. Note that you must upload image links with the jpeg or png in its URL.

Swagger documentation of the application on /docs

Let's try this out by using a picture of a chocolate cake, its URL link is this.

Image from tallypress.com

Paste the link to the text box in the /docs as so, then press Execute.

Demonstration of the app

After pressing the Execute button, it will take a few seconds until we get the model prediction. That's because we're using tensorflow-cpu because we're limited with the RAM and the slug size of our application when using the free tier of Heroku.

After the execution is finished, you should be greeted with this response:

Response of the API after usage

As you can see, the model predicted it correctly, with a confidence score of 2.65%. This confidence score is alright as we're not dealing with model accuracy (which requires the truth value beforehand), and we're dealing with data the model hasn't seen before.

Conclusion

In this article, you learned how to deploy a TensorFlow CNN model to Heroku by serving it as a RESTful API, and by using Docker.

If you find this article helpful, feel free to share it on social media. Let's connect on Twitter! You can also support me by buying me a coffee.

We just published a TensorFlow Lite course on the freeCodeCamp.org YouTube channel.

Bhavesh Bhatt created this course. Bhavesh has created many courses on his own channel and is a great teacher.

TensorFlow Lite is developed by Google and is used to train Machine Learning models on mobile, IoT (Interned of Things), and embedded devices.

When you use TensorFlow Lite the machine learning all happens within the device. This can avoid sending data back and forth with a server.

Here are the topics covered in this course:

• Why do we need TensorFlow Lite?
• What is Edge Computing?
• Why is Edge Computing gaining popularity?
• Challenges in deploying models on Edge devices
• What is TensorFlow Lite or TFLite?
• TensorFlow Lite Workflow
• Creating a TensorFlow or Keras model
• Converting a TensorFlow or Keras model to TFLite
• Validating the TFLite model performance
• What is Quantization?
• Compressing the TFLite model further
• Compressing the TFLite model even further
• Validating the most compressed TFLite model performance

Watch the full course below or on the freeCodeCamp.org YouTube channel (1-hour watch).

Transcript

(autogenerated)

TensorFlow light allows you to do machine learning on small devices.

bhavesh is an experienced instructor and He will teach you all about TensorFlow light in this course.

Hello, everyone.

In this tutorial, you will learn the basics of TensorFlow light, and how TensorFlow light can help you create really efficient models that you can deploy on edge devices.

So without wasting any further time, let's kick start the tutorial.

Let us kick start today's discussion about the flight with a small story.

I have a friend whose name is john.

JOHN really likes traveling to different places.

One of his favorite applications is Google lense.

Whenever he visits a new country, he removes his cell phone takes a photograph of the monument that is in front of.

And Google essentially tells him which monument it is.

So now, just by the sheer fascination of the tool, john goes forward and creates his own neural network for detecting landmarks.

He goes through a rigorous process of collecting data, labeling data, cleaning data.

And finally he creates a machine learning model that can tell john, which monument it is.

So everything looks good, he is able to reach a very high accuracy score as well.

Now the only challenge that he has is where should he deploy the model that is created.

So technically, he has two options.

The option that he explores first is cloud computing.

He takes his train model, and he deploys it on Cloud.

He exposes the API that he is created.

And essentially, he creates an Android application that kind of queries the API and fetches the result once he's kind of passed in an image.

Now one of the major challenges that he saw when he created the solution is the network latency.

So the images that are captured generally by cell phones today, range anywhere between three to 10 Mb.

So transporting such huge files take up a lot of time in the entire process of making a prediction.

The second piece that adds complexity to the entire solution is the cost.

The neural network that john has created requires to resources, a storage resource wherein he can save the model weights, and a compute resource for making an inference, the overall project becomes a costly affair for john.

What does he do next, he creates an Android application.

And he finds a way to make an inference on Android using this huge model that he is created.

So john now faces a new issue.

The issue is the model is really huge.

And the cell phone is not very capable enough of storing and processing such a big model.

So now john, is really confused.

He's tried out different techniques to make this work.

But nothing is solving this issue.

Well, it is here that TF light comes into picture.

Before we jump and discuss about the flight, I wanted to throw some light in terms of what edge devices are.

So edge devices are your normal cell phones that you use.

So if you're planning to create some amazing TensorFlow based applications, then essentially one of the main platforms that you can utilize is your cell phones, bat or Android cell phone, or an iOS powered cell phone, anything works.

The other pieces of hardware devices that you can classify into edge devices or microcontrollers.

So there are some amazing applications that have been built using very small compute power.

And all of it is thanks to microcontrollers.

Given how recently, the variable devices that you were essentially have increased their computation power by using a more or faster CPU, you can also put your wearable devices such as your smartwatches into the edge computing bracket.

Now let me go forward and give you a formal definition of edge computing.

So edge computing is basically the practice of moving compute and storage resources closer to the location at which it is needed.

So that is where your edge devices would come into picture.

Now, if you recall it, john had two options to deploy the model.

The first option was to deploy the model on Cloud.

Now many of you would have the impression that a machine learning model running on the server using a large GPU is much more better as compared to running it on the device itself.

Well, the truth is edge devices have become an important platform for machine learning.

Why is it gaining so much of popularity that you essentially have to run your entire machine learning models on the edge devices? Well, let me share details one at a time.

The first and foremost reason why edge computing as a whole is gaining popularity is because of latency.

The use cases that require real time speed, definitely require models to run on the device.

For example, you might be able to reduce the inference latency of resnet 50, from 30 milliseconds to 20 milliseconds.

But the network latency can go up to seconds.

So essentially, it also depends on where your model is deployed.

Where are you waiting the API from.

So if you take into account all of these factors, then essentially deploying a model on server would not be the best possible situation when you want inferences in near real time, or exactly real time as well.

The second reason why creating machine learning models on edge devices is important is because of network and activity.

So if you go back to our earlier example, wherein john wanted to create his own version of Google lense for detecting landmarks, if he happens to visit a country where there is little to no connectivity, it is here that creating a model that sits on the device would be much more better as compared to deploying it on server, because there is this additional dependency on the network that comes into picture.

The third reason why it's very important for you to create machine learning models that run on edge devices, is user privacy.

Putting your machine learning models on the edge devices is also appealing when you are handling sensitive user data.

Machine learning on the cloud means that your systems might have to send user data over networks, making it susceptible to being intercepted.

Cloud computing also means that storing data of many users in the same place or location, which means a data breach can affect many people at once.

So it becomes really important to create machine learning models that can run on edge devices, which can preserve the user privacy data.

Let me now go forward and show you some examples of on device machine learning use cases.

The first one that I'm showing you right now is the feature to try out various cosmetics using AR on YouTube.

The entire computation piece that you're seeing here, is essentially happening on the device itself.

Now the second example that I want to show you is something that you might be already aware of, which is Google Translate.

Google Translate has a feature that allows you to capture text with your phone camera, and translate them in real time without any internet connection.

All of this is essentially possible using edge computing, and more specifically, TF light.

Now that you've seen the amazing new applications that you can create at your end as well, you might be wondering, the second alternative that john had taken initially, that is to create an entire huge TensorFlow model, and then run the inferences from the device.

Why did that fail? Well, to answer that question, here are some of the challenges that you might face when you create chaos or TensorFlow models, and deploy them directly onto edge devices.

Edge devices, not only restricted to mobile phones, but your microcontrollers as well have limited compute power.

Limited memory.

battery consumption is also a factor that you have to account for, as well as the application size.

If I consider a simple microcontroller is well, the processing power isn't so much that you can essentially run inferences on a three or four GB model.

If you consider the storage capacity of majority of the edge devices, well, then ideally, you wouldn't have a lot of storage that you can utilize for just one model.

So these are the challenges that john faced when he took the second approach as well.

So what is the solution? Well, the solution is TensorFlow light.

And so flow light is a production ready cross platform framework for deploying machine learning models on mobile devices and embedded systems.

And flow light at this point of time supports Android, iOS, and any IoT device that can run Linux.

So essentially, if you have any of these hardware devices handy with you Then you can quickly create a TensorFlow model, convert it to an equivalent TF light model, and start using the amazing TF light model.

Now, you might wonder, what exactly is the workflow to create a TF light model? Well, let's look at that as well.

So the workflow is fairly simple, you start by creating a TensorFlow slash karass model.

So in the entire process, you would have to collect data, you would have to clean data, pre processed data, then create models, iterate over multiple models, and based on the metric that you're chasing, if you are chasing for a higher accuracy score, then you would choose a model that would give you the best possible accuracy.

And that's about it, you have your TensorFlow model ready.

Now from that TensorFlow model, you convert it to a TensorFlow light model.

So there is a format change that happens.

I'll talk more about it as we go along.

Once you've converted your model from TensorFlow, to TensorFlow light, then you go forward and deploy the entire TF light model and run your inferences on the edge device.

When I mean inferences, I mean, predictions.

Okay.

Let me now explain this using a block diagram.

So this essentially is the workflow that I've just mentioned, you start off with your high level curiosity is create a model.

Once you have the model ready, then you can essentially use a TF light converter.

And the converter basically takes your save TensorFlow format file, and converts it into a flat buffer file.

I'll give you an idea in terms of what I mean by flat buffer file.

So let's move forward.

So TensorFlow lite represents your model in a flat buffer format.

Now flat buffer format is an efficient cross platform serialization library for c++, C sharp, go Java, kotlin, JavaScript, Python, and so on and so forth.

It was originally created at Google for game development and other performance critical applications.

But slowly, Google realized that you can use the flat buffer format in deploying models on edge devices.

Now you might have an obvious question, why not use our old tried and tested protocol buffers.

And why shift to something that is as newest flatbuffers protocol buffers just to give you context, all your kiraz models that you create all your TensorFlow models that you create are essentially protocol buffer format.

protocol buffers are essentially very similar to the flat buffer format that Google has created.

The major difference is flatbuffers do not need a parsing or an unpacking step to a secondary representation, before you can access the data.

And the code essentially is also larger in case of protocol buffers.

It is your while using TF lite, we make use a flat buffer format, and not protocol buffer format.

So now let's move forward.

So far, we've looked at various aspects of edge computing, we've looked at how deploying models on edge devices is better as compared to deploying it on Cloud.

We've also looked at what TensorFlow light is, and what all it can support at this point of time.

Now is where things would get interesting when I show you through code, how TF light can actually compress your model size without compromising on the accuracy piece.

So now let's go forward and witness the magic of TF light.

Now that we've understood the basics of TensorFlow light, and edge computing, let me show you the power of TensorFlow light using Python.

So for this example, I'm using Google collab.

For those of you who don't know, Google lab is an online environment wherein you can write Python code, you can create machine learning and deep learning models, you can also make use of deep learning models that Google gives you access to for good amount of time.

So this is the interface that I'll be using.

I'll be attaching the link to the GitHub repository in the description section of the video, feel free to access the code from there.

Also inside the GitHub repository.

I'll also give you a link that can open a Google collab notebook directly.

With all the groundwork done, let me now go forward and show you the magic of TensorFlow light.

So the process that I'll follow in this particular tutorial, is I'll create a deep learning model using TensorFlow slash Eros.

I will scale it down to a TF light equivalent model.

Once that is accomplished, I will show you the size difference between the original model as well as the compression model.

I will show you techniques how you can keep compressing the model even further without having to compromise on the accuracy piece.

With that, let me create an instance on Google collab by pressing Connect.

So currently, Google is allocating some space for my computations.

If you're planning to replicate the entire thing that I show in today's video in your local machine, then you will require some set of installations as well.

Given that I'm working with Google collab, all the dependencies that I require for this example are already met.

So now let me go through the different things that are required for this entire tutorial.

First things first, I require the voice module to essentially read my files.

Next up, I'll import NumPy as NP I will require this particular library for mathematical operations.

I will also require h5 p y library.

The h5 p vi library is a pythonic interface to the HD f5 binary data format.

So technically, whatever models I create in chaos, I will basically save it into the h5 format.

Next up, I require matplotlib.

This is again used for visualization.

I require TensorFlow, I'll import kiraz.

From TensorFlow.

These are some of the layers that will require when we come to the deep learning model creation aspect.

If I want to calculate how good my model is performing in terms of accuracy score, this is where SK learn dot matrix module will give me the accuracy score functionality.

And from the system library, I'll also require the function get size of.

So these are some of the things that I require in order to create a TF like model.

So now, let me go forward and run the cell.

So when I run the cell, what essentially happens is Python essentially runs that piece of code.

So let me now go forward and run the cell.

So I don't see any error.

That means all of our imports are in place.

Let me go forward and show you the TensorFlow version that I'm using for this particular tutorial.

I'm currently using TensorFlow 2.6 point zero.

If there are changes that creep up with respect to the API.

Feel free to refer to the TensorFlow documentation.

There are two functions that have created the first function name is get underscore file underscore size.

Essentially, I'm passing in the file location using the OAS library and specifically the function get size, I'm able to get the size of a particular file that I pass in, in byte so let me now go forward and run this cell.

In the previous function, that is get underscore file underscore size, the value returned would be in bytes.

So now rather than comprehending the values of a file size in byte, I've created a helper function called as convert underscore bytes, which essentially takes in the input size in bytes and converts it in either KB MB.

So this is something that I've created.

I've not included DBS because so this is more of an explainer video wherein I intend to create a smaller model or rather a proof of concept rather than like a huge model.

So that is why I have restricted my units to cavies or MDS.

So let me go forward and run the sale.

Now for this particular example, I'll be basically using a very famous data set and deep learning, Carla's fashion m&s data set, so let me unhide the cell so fashion emnes data set contains 70,000 grayscale images, which belong to 10 different categories.

So categories would include the shirt, trouser, pullover, dress, code, sandal, shirt, sneaker bike, and ankle boot.

So these are the different categories that are part of this data set.

The entire activity is a supervised learning task, I will have a set of images, and every image will have a label associated with it.

And I'm trying to train a deep learning model.

So now let me go forward.

So you don't have to worry about the download pieces.

Well, if you have installed TensorFlow correctly, then essentially you just have to call the chaos dot data set dot fashion amnesty function, save the entire data set into a variable called as fashion underscore m nest.

So that is the first step that I've done here.

Once you've done that, then essentially what you have to do next is you have to split your data set into training and testing.

The way you can achieve that is ideally by calling a function called as load underscore data.

So this is what you have in terms of the function.

Once you call this function from fashion underscore m nished, the variable that you just created, you would be able to split your data into training images, training labels, st images, and test labels.

So that's how simple it is.

So let me go forward and run the cell.

So we've downloaded the data files, we've split our data into training and testing.

I've also created a variable, a list variable, called as class underscore names, which contains all the names of the classes that are part of this entire activity.

So let me go forward and run the cell.

So if you recollect, we had 70,000 samples in our data set, we've already split that entire data set into training and testing.

So let me go forward and show you how many images are part of the training data set.

So let me run this cell.

So the shape of my training data set is 60,000 comma 28, comma 28.

So I have 60,000 images.

Each image has a size of 28 cross 28.

So 28 rows, 28 columns represent each image, and I have 60,000 such images.

Given this is a supervised learning task, I would also require 60,000 labels.

So let me check if the total number of labels in my training data set are 60,000.

So let me run this cell.

So as you can clearly see, I have 60,000 labels as well.

Now given that, I've already mentioned that there are 10 unique classes, let me verify that as well.

So as you can clearly see I have class numbers ranging from zero to nine.

And the mapping is what I've created here, which is contained in the variable class underscore names.

Let's now go forward and explore that testing data set as well.

So let me quickly unhide this let me show you the total number of images in the testing data set.

So that come out to be 10,000 60,000 for training in 1000.

For testing, each image is again of the size 28 cross 28.

Similarly, if I look at test underscore labels, I'll have 10,000 samples.

What I intend to show you next is I want to show you a sample image.

So let me show that to you.

So this is a sample image that is part of this data set.

This is clearly an ankle boot.

The size of the image is 28 cross 28.

So 28 rows 28 columns is what you see here.

Before we go forward and train a neural network, a good practice is to scale In the intensity values of the images, which range between zero to 255, zero to one.

So that is what I have done in this piece of code.

So let me run this.

So now my train images will have values ranging from zero to one, and not zero to 255.

So far, we've downloaded the data set, we've split our data set into training and testing.

And we've done some sort of pre processing as well, now is the time when we'll create a simple neural network that 10 classify the entire images into one of the 10 categories that are there.

So in this piece of code, I'm calling the sequential class from the chaos library.

I'm passing in the first layer as a flattened layer.

Now, if you recollect, the images were 28 cross 28.

If I have to pass it through a layer, then I have to basically flatten it first, I am not creating a convolutional neural network given that the data set is fairly simple, I will stick to a normal deep neural network.

So the first layer that I add is a flattened layer, wherein I pass the input shape, which is 28, comma 28.

The second layer is the dense layer.

And the activations supplied to this dense layer is relu.

The final layer is again a dense layer, given that I have 10 different classes to classify between.

So that is what I have here.

So let me quickly create an instance of the model.

So let me run this.

Before we go forward and compile the model, I'll show you the structure of the model as well.

So I'll say model dot summary.

So this essentially is the model summary.

So for the given architecture, we have close to 100k trainable parameters.

Now the next step is to compile the model.

I'm passing in the optimizer, I am passing in this past categorical cross entropy loss.

Given that our classes are mutually exclusive, I'm using the sparse categorical cross entropy loss as compared to the normal categorical cross entropy loss.

And the matrix attend casing for his accuracy.

So I want to create a model that is fairly accurate.

So let me now go forward and run the cell.

So I've created an instance of the model.

I've also compiled the model, now is the time when I'll pass in the training images as well as the training labels to train the entire trainable parameters.

So let me now call the model dot fit function, wherein I'll be passing in the train images, train labels, and I kind of run the entire exercise for 10 epochs.

So let me run the cell.

So with every epoch, you can see that the accuracy is increasing.

So we've successfully trained our model and we've reached a training accuracy score of around 91%, which is Something that's reasonably good given that I've trained the model for only 10 epochs.

So let's go forward.

And remember one thing, the objective of the video is not to train the most accurate classifier at this point of time, but to show you the power of TF light, so that is why I've kind of stopped at 10 a box.

Now the next thing that I do is I create a variable called SK Ross underscore model underscore name.

This is something that will be used as reference, or this will be the baseline model performance that I evaluate later on with the TF light models as well.

So the name of this particular variable is pf underscore model, underscore fashion underscore m NIST dot h phi.

So let me quickly run the cell.

Now let me go forward and call the model dot save function and pass in the filename that I just created.

So let me run the cell.

So as soon as you run the cell, you would have a file that would be created in your Google collab session or in your local directory, which is essentially your saved model file.

So let me show that is well.

So this is our saved model file that has been created.

So let me quickly and I the cell again.

Now I'll go forward, and I'll show you the size of this particular file that we've created.

So let me call the two functions that I've created, that is convert underscore bytes, and get underscore file underscore size, I pass in the same file name, and I want the file size to be in MB.

So let me run the cell.

So currently, I have a model that occupies 1.2 Mb.

So I'll go forward and I'll create a variable called SK Ross underscore model, underscore size, and save the byte equivalent size into this particular variable.

So let me run the cell.

We know for a fact that the model is performing really well on the training data set.

But then essential litmus test is to check how well the model is performing on unseen data that is my testing data set.

So let me go forward and evaluate how good our model performances on their testing data set.

So I call the function model dot evaluate, I pass in the test images, test labels.

And I save the results into two variables called test underscore loss, and test underscore accuracy.

So let me quickly run the cell.

So as you can clearly see, the loss is at a very small value, which is around point three, seven.

And I've reached a testing accuracy score of around 88%.

So we've completed the first part, now it's time to move on to the next part that is creating a TF light equivalent of the same model.

So let's go forward.

So I start the activity by creating a variable called as TF underscore light underscore model underscore file underscore name.

And I pass in an equivalent name to this particular TF light model, which in our case, currently is TF underscore light underscore model.tf light.

So let me quickly run the cell.

Now the process of converting a TensorFlow model or a karass model into a TF light model, essentially requires just a couple of steps.

So this is what I'll highlight right now.

So the first step is to call tf.light.tf light converter dot from Kara's underscore model, I pass in the model that I created.

So if you remember the name of the model variable was essentially model so that is what I'm passing in here in the first line.

Once I've created an instance of the TF light converter from Eros MADI, I save the entire piece into a variable called as TF underscore light underscore converter.

And finally, what I do next is I call the Convert function.

Once the conversion happens, I want the result to be saved into a variable called as TF light underscore model.

So let me quickly run the cell.

So if you look at the output, it says that assets are written into a particular temporary file.

So from that temporary file, I basically have to retrieve the model weights and save it into a TF light equivalent file.

So that is what I've done using this piece of code.

I've created the first variable which is TF light underscore module underscore name.

And I'm passing in the initial name that I've created in the first line of this particular section.

I open the file name with write access, and I write this particular temporary file into this file that I've created.

So that's how simple it is.

So let me quickly run this.

So there is a particular output that is displayed.

This tells me the total number of bytes that have been returned to this particular file.

Now let me go forward and show you the exact size of this TF lite model in kilobytes.

So let me run this.

So the overall file size is close to 400 kilobytes.

So we started off with a model which was occupying around 1.2 Mb.

And after just running a couple of lines of code, we have brought down the file size to around 400 kb.

Now, let me go forward and save this file size into a variable called s TF light underscore file underscore size.

This is something that will make a lot of sense once we go forward.

So let me quickly run this L.

Now we've already converted a model from kiraz to tf light.

But one thing that we've not validated currently is how good the model is in terms of performance.

Is it actually good on unseen data? Or has it dropped in terms of the accuracy score.

So that is what I want to check next, that after compressing the model using TF light, are we losing out on accuracy or not.

So in this section, I'll go over how you can validate the results in terms of how good your TF light model is performing.

So now, let me quickly unhide the cell.

Now, don't get scared by looking at this piece of code, I'll help you understand what I'm trying to achieve.

You're now loading a model, or TensorFlow or a karass model into a TensorFlow session is fairly easy.

But here, what we have done is we've kind of created a TF light model.

If you go back to the discussion that we had, TF led models are essentially flat buffer format files and not your normal usual protocol buffer files.

So in order for us to actually make an inference out of TF light files on our TensorFlow or a Python session, we require something called as an interpreter.

So it is your that will be basically making use of tensor flows interpreter to load the TF light file, and then make inferences or predictions.

So let me now take you through each and every line of code.

So in the first line, I create an instance of the interpreter class, I pass in the TF light model name that we've just created.

So if you look at this particular section, you will also have a TF lite file.

This is what I'm passing in here.

Now once we've created the interpreter object, the interpreter object saves details about the model.

It will have details about the input that it expects the value that type of values it expects, and in terms of the output, it will tell you what the shape of the output should be.

In terms of the output, it will tell you the output shape as well as the output the type that is the output values, it will predict what are the values and what are the type of values.

So all of that is what this particular interpreter will actually have details about.

The details it is fetching is again from the interpreter object that we created and we passed in the TF lite file.

So all of the details would be captured in this particular TF lite file, which is what is read by this interpreter object.

And that is what we are trying to accumulate from input underscore details and output underscore details.

Once we have the input and output details, I also am interested in the shape of the input that is expected and that type of inputs are the variable nature of the inputs that are there.

So let me quickly run this cell to make more sense.

So if you look closely, the input shape is 128 28.

The input type that it expects is NumPy, float 32 the output shape is one comma 10 that is one so basically has one row and 10 columns and the output type is again NumPy float 32.

So this is essentially what the TF lite file contains.

Now if you look at this particular one, this denotes that the A flight is expecting one input at a time.

Now I want to check how good it is performing for 10,000 inputs.

That is where I'll have to reshape the input shape to a particular value, which is what I'll be achieving in this piece of code.

Just to reiterate, again, the input shape is 128 28.

So ideally, I have to pass in just one image sample.

And essentially, I would get a corresponding output for it.

But essentially, in my use case, I want to validate how good the TF light model is performing for the testing data set that I have, which essentially contains 10,000 images.

So if this idea is clear to you, let's go forward.

So now I want to validate how good my TF light model is performing on my testing data set.

So I call the resize underscore tensor underscore input function.

I pass in the details that I want to resize this particular index value, and I pass in how I have to resize it.

So currently, I have 10,000 samples.

So that is what I have entered here, that is 10,000 comma 28, comma 28.

Similar resize operation is what I'm doing at the output side.

So you can see your 10,000 comma 10, from the initial one comma 10.

So that is what I've done here.

Now, once the resize operation has happened, I want to call the allocate tensors.

to actually change the entire structure of the interpreter.

This is what it's read using the TF light file.

And now when I print the input details and output details, I should be able to see that the entire TF light input output values have changed.

So let me quickly run this piece of code.

So as you can clearly see, the input shape has changed from 128 28 to 10,028 28.

So this essentially will help me to validate how good my TF light model is performing.

The other thing that I want to highlight right now is s underscore images dot d type is float 64.

So if you look at the input shape that the model expects is NumPy dot float 32.

So now the only other change that I have to make in order to validate my TF light model is I have to create a new array called as test underscore images underscore NumPy.

Pass in the original array, and change the D type.

I can do it in the same area as well.

But I'm essentially choosing to create two different arrays.

So let me quickly run the cell.

So now if I show you the D type of test underscore images dot NumPy, it will be NumPy float 32.

Now that we have the entire interpreter object set up correctly for our set of inputs, that is the testing data set.

All I have to do right now is firstly call the set underscore tensor function, passing the test underscore images underscore NumPy array that I just created and call the invoke function.

What the invoke function would essentially do is pass in the inputs, get the output.

And once you have the output ready, you call the get underscore Insert Function, which will kind of have the output ready for you and save it into a variable called as TF slide underscore model underscore predictions.

So let me quickly run the cell.

Now the output that you see here, which is prediction results shape is 10,000 rows and 10 columns.

So every column would essentially contain a probability score.

So what I have to do next is I have to pick out the value or the index between zero to nine that has the maximum probability, which is what I've done using this function called as NP dot arg max.

So this will help me get numbers directly that is zero to nine rather than having 10 different columns with probability scores.

Now let me calculate the accuracy score.

And let me print it out for you.

So the testing accuracy of the TF light model is exactly the same that you see when you compare it with your normal karass model.

Now how much of space Have you saved in this entire process? says, Mel let me calculate a ratio between TF lite file size and karass model file size.

So overall, the TF lite model occupies close to 32% of the overall file size that my normal karass model occupies.

But the uniqueness is that I'm not losing out on any accuracy.

So this is the power of TF Lite.

I've been able to compress my entire model from 1.2 Mb to around 400 kb.

And I haven't yet compromised a bit on the accuracy pieces.

Well, Isn't this amazing? Well, if you think the story is ended here, hang on for a second, there is more to go.

So far, what we've done is I've taken a TensorFlow model.

And without any optimization, I've basically converted that into an equivalent TF light model.

Now I'll show you how you can compress your model even further.

without losing out on accuracy as such.

So now let me introduce you to a new concept called les quantization.

So what exactly is this term that I've just mentioned, that is quantization.

So for a given weight value that can be represented in float 32 or float 64 format? Wouldn't it be great if we can bring down the size of those particular values and see very little change in accuracy? Well, this essentially is the concept of quantization, I'm reducing the total number of bits for every weight value, so that the overall size of the entire array reduces.

Just to be more clear, if I have a neural network something like this, where this particular weight value is 5.31345, this particular weight value is 3.8958.

And you have the other way values is well, what if I can change these representations that occupies so many bits to something like this, there will be a small hit in the accuracy.

But overall, I'll be able to compress my model even further.

How when whatever questions you have in mind, just wait for some time.

If this entire idea is clear to you, let us go back to the coding section.

And I'll show you how you can compress your TF light model even further.

So by default, in the previous example, wherein we took a karass model, and we converted that to a TF light model, every weight value is essentially float 32 format.

Wouldn't it be great if I compress it from float 32 to float 16.

This is the activity that I'll be performing next.

So I create a variable with an MTF underscore light underscore model underscore float underscore 16 underscore file underscore name.

And I basically give it a TF lightning which essentially represents that the entire weights inside it would be fluid 16.

So that is what I have here.

So let me quickly run the cell.

If you look at the previous section in terms of how we created a TF light model, the first line of code is something that is pretty much familiar to you, you pass in your karass model, you call the TF light converter dot from kiraz model, and you save it into a variable.

Even the last piece of code is also something that you've already looked at.

What you haven't seen so far is the optimization.

So when you create an instance of the TF light converter, there is a flag called as optimizations.

So you have the optimizations flag here.

I set it to tf dot light dot optimizer default, so I want the default optimizations to take place.

And one other things that I do here.

So I'll speak more about the optimizations in the next section.

So hold on to that thought as well.

Now here there is one more flag called as target underscore spec, and supported underscore types.

Here is where I set every weight value from floor 32 to float 16.

So that is what I'm doing here.

So let me quickly run this piece of code.

So now I have a TF light model, wherein every weight value would be a float 64 Automat, I follow the same process again, wherein I'm fetching data from the temporary file and saving it into a TF lite model.

So let me run this.

I don't know if you've guessed it already or not.

This essentially is a file size in bytes for the newly converted a flight model.

So if I now show you the size of this newly converted TF flight model, then my size has drastically reduced from 400 kilobytes to 200 kilobytes.

The only thing that I changed here was I changed the individual representation of every made value that's about it.

Isn't this amazing, I'm able to save so much of memory, just by changing few values here and there.

Say again, save the file size into a variable called as TF lite underscore float underscore 16 underscore file underscore size.

So let me run this.

Now if I compare it to the original karass model, this particular model occupies 16% of the size that the original model occupied.

And if I compare it to the previously created version, then I can see almost 50% compression that I'm able to achieve by changing the weights from floor 32 to floor 16.

So this is the power of optimizations and TF light.

If you think this is it weird for the next section, wherein I compress the model even further.

I'm not showing you the accuracy piece right now you might be wondering, why isn't he showing us the accuracy has accuracy taken a toss? Well, the answer is no.

I show you the accuracy of an even compress model.

So that will give you a fair sense in terms of how much compression is changing the overall accuracy values as well.

Okay.

So now we have reached the final section wherein I will compress the model even further.

Okay, so here I've created a variable called as TF underscore light, underscore size, underscore current underscore model, underscore file underscore name.

And here I want to see what the eflite file with this particular name.

So I'll quickly run the cell.

In the previous example, I changed every weight value from floor 32, to floor 16.

Rather than you deciding what is good for your model, I would rather let TF light decide that for me.

So if you've been following along so far, then this piece of code is something that we've already covered.

This piece of code is also something that we've already covered.

This is something that is unique.

So here, I set the optimizations fly.

And here I just mentioned, optimize for size, there are different values that you can kind of go through in the documentation.

So based on your needs, you can kind of optimize for size, and the other optimization that are also available for TF light.

So I'm not specifying what type of data type I want, I just want the most optimized version wherein the size that is occupied by this particular TF light model is the most compressed version.

Okay, so I'll quickly run the cell.

So I catch all of the file that is saved into a temporary variable, I save it into this particular variable that I created.

And if you've guessed by now, this is the new file size in bytes.

If I go to the kilobyte section, then my file occupies around 100 kb.

So if you remember, we started from 1.4 Mb, and we have brought down the file size of a deep neural network.

200 kilobytes.

Isn't this amazing.

Just to give you some numbers again, if I compare this particular file size with my original file size, then my current file is almost 8% the size of my original file, that is my karass model that I created.

If I compare it to the previous model as well, I'm basically able to achieve 50% compression, all because of optimizing for size.

This essentially is a power of TF light.

Now I'm really happy with the compression, I have a 1.4 Mb file that I've compressed down to around 100 kb.

But is the accuracy still the same? Well, we'll again follow the same process, wherein I load the newly quantized model into the interpreter object.

I'll get details of those objects, and I'll again reshape the values and pass it testing data set entirely through the interpreter object.

So I'll quickly run the cell.

So as you can clearly see, 128 28 is the interpreter object input values that it expects, I have the output as one comma 10.

The input and output values that are expected are NumPy, float 32.

So that is all good as well.

Coming to the final section, I follow the same process.

Again, no change in the process.

I have a testing data set of 10,000 images, which is what I pass in here.

I allocate tensors, I get the details, and I'll show the details to you as well.

So let me quickly run this 10,028 28 128 28.

So we've resized the tensor input values that we are expecting to validate our testing data set.

You essentially don't need the step again, but as kind of copy it from the initial part.

So I'm running it again.

I pass in the values again.

Now I calculate the accuracy score.

Now is the litmus test for this highly compressed TF light model.

So let me quickly run the cell.

So the accuracy of a TF lite model that occupies almost 8% of the size of the original karass model is equivalent to the original karass model.

If I go up, I'm recording this video in one go.

So if I go up where did this value go? here the value was at 7.66%.

Here it is at 87.59%.

So this is what you can achieve using TF Lite.

I started off with a very simple neural network, the entire model occupied around 1.2 Mb.

You might also argue that 1.2 Mb is kind of small.

But the problem statement was fairly simple.

If you have like a really complex example, wherein you have to classify images into 1000 or 10,000 categories, the model size would eventually increase.

So the objective then becomes can we compress the model size? And the answer is yes, TF lite will help you compress the model size without having to compromise any bit on the accuracy front.

If you've reached this point, then I'm assuming you've seen the entire video or if randomly reached at this point, whichever way you've reached this point.

I hope you enjoyed today's video, I keep creating such amazing videos on data science, machine learning, and Python.

So feel free to check out my channel in the description section of the video as well.

Thank you so much for watching this video.

We just published a full course on the freeCodeCamp.org YouTube channel that will teach you how to use TensorFlow 2 for computer vision applications.

Nour Islam Mokhtari created this course. Nour is a Machine Learning Engineer and experienced teacher.

The course shows you how to create two computer vision projects. The first involves an image classification model with a prepared dataset. The second is a more real-world problem where you will have to clean and prepare a dataset before using it.

MNIST Dataset with labels

Here are the topics covered in this course:

• Why learn Tensorflow
• We will be using an IDE and not notebooks
• Visual Studio Code (how to download and install it)
• Miniconda - how to install it
• Miniconda - why we need it
• How are we going to use conda virtual environments in VS Code?
• Installing Tensorflow 2 (CPU version)
• Installing Tensorflow 2 (GPU version)
• What do we want to achieve?
• Exploring MNIST dataset
• Tensorflow layers
• Building a neural network the sequential way
• Compiling the model and fitting the data
• Building a neural network the functional way
• Building a neural network the Model Class way
• Things we should add
• Restructuring our code for better readability
• First part summary
• What we want to achieve
• Downloading and exploring the dataset
• Preparing train and validation sets
• Preparing the test set
• Building a neural network the functional way
• Creating data generators
• Instantiating the generators
• Compiling the model and fitting the data
• Adding callbacks
• Evaluating the model
• Potential improvements
• Running prediction on single images

Watch the full course below or on the freeCodeCamp.org YouTube channel (4.5-hour watch).

Deep Learning is a branch of Machine Learning. Though machine learning has various algorithms, the most powerful are neural networks.

Deep learning is the technique of building complex multi-layered neural networks. This helps us solve tough problems like image recognition, language translation, self-driving car technology, and more.

There are tons of real-world applications of deep learning from self-driving Tesla cars to AI assistants like Siri. To build these neural networks, we use different frameworks like Tensorflow, CNTK, and MxNet.

If you are new to deep learning, start here for a good overview.

Frameworks

Without the right framework, constructing quality neural networks can be hard. With the right framework, you only have to worry about getting your hands on the right data.

That doesn’t imply that knowledge of the deep learning frameworks alone is enough to make you a successful data scientist.

You need a strong foundation of the fundamental concepts to be a successful deep learning engineer. But the right framework will make your life easier.

Also, not all programming languages have their own machine learning / deep learning frameworks. This is because not all programming languages have the capacity to handle machine learning problems.

Languages like Python stand out among others due to their complex data processing capability.

Let's go through some of the popular deep learning frameworks in use today. Each one comes with its own set of advantages and limitations. It is important to have at least a basic understanding of these frameworks so you can choose the right one for your organization or project.

TensorFlow

TensorFlow is the most famous deep learning library around. If you are a data scientist, you probably started with Tensorflow. It is one of the most efficient open-source libraries to work with.

Google built TensorFlow to use as an internal deep learning tool before open-sourcing it. TensorFlow powers a lot of useful applications including Uber, Dropbox, and Airbnb.

Advantages of Tensorflow

• User Friendly. Easy to learn if you are familiar with Python.
• Tensorboard for monitoring and visualization. It is a great tool if you want to see your deep learning models in action.
• Community support. Experts engineers from Google and other companies improve TensorFlow almost on a daily basis.
• You can use TensorFlow Lite to run TensorFlow models on mobile devices.
• Tensorflow.js lets you to run real-time deep learning models in the browser using JavaScript.

Limitations of Tensorflow

• TensorFlow is a bit slow compared to frameworks like MxNet and CNTK.
• Debugging can be challenging.
• No support for OpenCL.

Apache MXNet

MXNet is another popular Deep Learning framework. Founded by the Apache Software Foundation, MXNet supports a wide range of languages like JavaScript, Python, and C++. MXNet is also supported by Amazon Web Services to build deep learning models.

MXNet is a computationally efficient framework used in business as well as in academia.

Advantages of Apache MXNet

• Efficient, scalable, and fast.
• Supported by all major platforms.
• Provides GPU support, along with multi-GPU mode.
• Support for programming languages like Scala, R, Python, C++, and JavaScript.
• Easy model serving and high-performance API.

Disadvantages of Apache MXNet

• Compared to TensorFlow, MXNet has a smaller open source community.
• Improvements, bug fixes, and other features take longer due to a lack of major community support.
• Despite being widely used by many organizations in the tech industry, MxNet is not as popular as Tensorflow.

Microsoft CNTK

Large companies usually use Microsoft Cognitive Toolkit (CNTK) to build deep learning models.

Though created by Microsoft, CNTK is an open-source framework. It illustrates neural networks in the form of directed graphs by using a sequence of computational steps.

CNTK is written using C++, but it supports various languages like C#, Python, C++, and Java.

Microsoft’s backing is an advantage for CNTK since Windows is the preferred operating system for enterprises. CNTK is also heavily used in the Microsoft ecosystem.

Popular products that use CNTK are Xbox, Cortana, and Skype.

Advantages of Microsoft CNTK

• Offers reliable and excellent performance.
• The scalability of CNTK has made it a popular choice in many enterprises.
• Has numerous optimized components.
• Easy to integrate with Apache Spark, an analytics engine for data processing.
• Works well with Azure Cloud, both being backed by Microsoft.
• Resource usage and management are efficient.

Disadvantages of Microsoft CNTK

• Minimal community support compared to Tensorflow, but has a dedicated team of Microsoft engineers working full time on it.
• Significant learning curve.

PyTorch

PyTorch is another popular deep learning framework. Facebook developed Pytorch in its AI research lab (FAIR). Pytorch has been giving tough competition to Google’s Tensorflow.

Pytorch supports both Python and C++ to build deep learning models. Released three years ago, it's already being used by companies like Salesforce, Facebook, and Twitter.

Image Recognition, Natural Language Processing, and Reinforcement Learning are some of the many areas in which PyTorch shines. It is also used in research by universities like Oxford and organizations like IBM.

PyTorch is also a great choice for creating computational graphs. It also supports cloud software development and offers useful features, tools, and libraries. And it works well with cloud platforms like AWS and Azure.

Advantages of PyTorch

• User-friendly design and structure that makes constructing deep learning models transparent.
• Has useful debugging tools like PyCharm debugger.
• Contains many pre-trained models and supports distributed training.

Disadvantages of PyTorch

• Does not have interfaces for monitoring and visualization like TensorFlow.
• Comparatively, PyTorch is a new deep learning framework and currently has less community support.

DeepLearning4j

DeepLearning4j is an excellent framework if your main programming language is Java. It is a commercial-grade, open-source, distributed deep-learning library.

Deeplearning4j supports all major types of neural network architectures like RNNs and CNNs.

Deeplearning4j is written for Java and Scala. It also integrates well with Hadoop and Apache Spark. Deeplearning4j also has support for GPUs, making it a great choice for Java-based deep learning solutions.

Advantages of DeepLearning4j

• Scalable and can easily process large amounts of data.
• Easy integration with Apache Spark.
• Excellent community support and documentation.

Disadvantages of DeepLearning4j

• Limited to the Java programming language.
• Relatively less popular compared to Tensorflow and PyTorch.

Conclusion

Each framework comes with its list of pros and cons. But choosing the right framework is crucial to the success of a project.

You have to consider various factors like security, scalability, and performance. For enterprise-grade solutions, reliability becomes another primary contributing factor.

If you are just getting started, begin with Tensorflow. If you are building a Windows-based enterprise product, choose CNTK. If you prefer Java, choose DL4J.

I hope this article helps you choose the right deep learning framework for your next project. If you have any questions, reach out to me.

Loved this article? Join my Newsletter and get a summary of my articles and videos every Monday.

Cut to June 13, and I am TensorFlow Developer Certified. ✅

So what happened in this 3-month long gap?

After honoring all my business and personal commitments, I managed to take off one month to prepare for the exam. After studying all the details of the exam, I created a learning plan to get myself exam-ready in 14 days.*

That’s all cool – but what is TensorFlow?

The gist: TensorFlow is an end-to-end open-source machine learning platform. It has a comprehensive ecosystem of libraries, tools, and community resources that lets ML/AI Engineers, Scientists/Analysts build and deploy ML-powered applications.

Google, Airbnb, DeepMind, intel, Twitter, and many others are currently powered by TensorFlow and it helps them solve a wide gamut of problems.

Now, I am not a certification evangelist. But since I was already using and following TensorFlow so closely as a Data Science Enthusiast it got my attention.

It has been an amazing learning streak and I am here to share all the nitty-gritty details of what the program is, how I did it, and how you can do it too!

What is this certificate program about?

The certificate is an official validation confirming your proficiency with TensorFlow with respect to solving deep learning and ML problems in the AI-driven job market.

If you’re someone who has got the skills to develop those Deep Neural Networks and solve problems with it, you can take the exam to differentiate yourself with the certificate.

Oh, snap! Not another Certification Program…?

Why should you take the exam?

Firstly, this is not like the certification where you watch a few 2–3 minute-long video lectures and take a quiz of multiple-choice questions and get yourself certified. This will require you to code and solve a class of problems that you'll need to prepare for.

Secondly, how many times have you thought of mastering a new library or technique, and then abandoned your plans midway? If you're anything like me, 99% of the time.

For me, the certification worked as the destination for my learning journey. I had some experience using TensorFlow but this came in as a challenge to work on problems that I hadn't actually solved myself.

Thirdly, you should keep monitoring the technology space in your field at least. So here is a trend from StackOverflow that shows how TensorFlow is being used by a huge number of users accounting for nearly 1 out of every 100 questions on the platform:

Lastly, I feel that Google always provides value to its users/developers. I believe the way they have structured the exam makes it worth trying, as it validates your skillsets and adds weight to your profile.

OKAY! I’m sold, can you tell me what am I supposed to do in this exam?

Exam Walkthrough

The exam is an online performance-based test where you are provided with questions to solve by building TensorFlow models within a dedicated PyCharm environment.

You can take this exam from your computer that supports the PyCharm IDE requirements. You'll need a reliable internet connection, and you can take the exam at whatever time suits you (I started mine at midnight).

The exam tests your ability to solve problems like Image classification from real-world images, Natural Language Processing, and time series forecasting using Tensorflow 2.x.

You can take up to 5 hours for the exam. If you exceed the time limit, the exam will be auto-submitted and you will only be graded for the questions for which you have submitted and tested your model.

You are allowed to use whatever learning resources you would normally use during your ML development work.

Exam Cost: Each attempt costs you $100 USD.

Ah-hah_! so how did you prepare for this scary long exam?_

How I started preparing for the Exam

The first thing I did was spend a good amount of time studying the exam itself. The TensorFlow team provides you with this comprehensive handbook that tells you every detail about the exam and what skills you should master before taking it:

After studying the exam, I designed a curriculum for myself to cover every skillset that is mentioned in this handbook.

Next, I set myself up with a schedule so that my work engagements didn't push me off track and I prioritized learning for those ~20 days.

And that’s all – I started preparing for the exam using this curriculum comprised of these recommended and useful resources:

Link to my compilation of resources: https://www.notion.so/15049893501f4387893a5de0059ef8a5?v=9154c52a61494668b12802f157bce0d4

[Imp]: Learning Curriculum — Review of all the resources I used to pass the exam

For someone new to Tensorflow or Machine learning, the handbook might portray a terrifying picture. But having a plan and setting up a schedule will get you through. Here’s the curriculum that will prepare you well for the exam.

The Tensorflow team again did an amazing job of suggesting the resources based on your familiarity with Machine Learning. On top of that, I had been following a few books and playlists that helped me a great deal to cement the fundamentals in my brain and helped me go beyond the exam requirements themselves.

I have also reviewed all these resources that I used with a scoring scale of 5, based off the following qualities:

• Usefulness — to pass the exam

• Learning Value — might not have a direct effect on the exam results but will help you build a strong foundation and work on more complex problems.

Here’s the list of resources along with the time and cost that each will incur:

1. Coursera’s TensorFlow in Practice Specialization

Usefulness: 5/5 — This is absolutely needed to score well (or even pass) on the exam. It will help you cover every skill mentioned on the skills checklist in the Handbook. This is the recommended course on the Certification home page.

If you carefully study the skills checklist and then compare it with the course outline, you’ll be able to figure out the direct mapping of each skill. It looks like either the course was created with the certification exam in mind or vice versa.

The entire specialization contains 4 courses:

• Introduction to Machine Learning and Deep Learning.

• Convolutional Neural Networks in TensorFlow

• Natural Language Processing in TensorFlow

• Sequence, Time series, and Prediction

Learning Value: 4/5 — The course itself depends on other resources to help you get an in-depth understanding of the fundamental concepts and topics that it uses. This is more of a Hands-On course.

Time: It should take you 4–8 weeks depending on the amount of time you dedicate. I had prior experience with Image classification problems, and it took me 14 days to watch the entire specialization series and practice all the exercises they provide.

Cost: This comes at a cost of $59 per month after a 7-day free trial. Totally worth it if you have to pay. The other resources provide a free alternative.

2. YouTube Playlists on Machine Learning Foundation by Laurence Moroney

Usefulness: 4/5 — This is an alternative to the starting 2 courses in the TensorFlow specialization on the Google Developers YouTube channel.

There is a dedicated NLP zero to hero playlist by the same author — Laurence Moroney.

Learning Value: 3/5 —Same as above but relies on other videos and resources in case you’re a beginner in Machine Learning.

Time: 1-2 weeks per playlist if you’re dedicating like 3–4 hours daily to your preparation.

Cost: Free

3. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Usefulness: 3/5 — The score is because of its relevance to the exam. For beginners, this will be a foundational resource to understanding Machine Learning and then gradually diving into the depths of Deep Learning, TensorFlow, Computer Vision, CNNs, RNNs, and much more.

Following are the most useful chapters from the book:

• Chapter 10 — Introduction to Artificial Neural Networks with Keras

• Chapter 11 — Training Deep Neural Networks

• Chapter 12 — Custom Models and Training with TensorFlow

• Chapter 13 — Loading and Preprocessing Data with TensorFlow

• Chapter 14 — Deep Computer Vision Using Convolutional Neural Networks

• Chapter 15 — Processing Sequences Using RNNs and CNNs

• Chapter 16 — Natural Language Processing with RNNs and Attention

I have been reading this book since before the exam and the author Aurelion has created a gem of a book for aspiring Data Scientists, ML/AI engineers.

It elucidates the foundational concepts, explains the mathematics behind each algorithm, and then explains the hands-on code to solve problems along with the best practices, covering everything. A MUST-read for all Machine Learning aspirants.

Learning Value: 5/5 — This is by far the best book to get started with Machine Learning.

Time: 3–4 Months — I would recommend that you read each chapter slowly and then practice the exercise given at the end of each chapter.

Cost: If you can afford it, I’d recommend getting an O’Reilly Media subscription for $50 a month where you not only get this book but all the publications and video/live lectures. Alternatively, you can buy the paperback on Amazon for the price it is available in your region (around $60).

I am an O’Reilly Instructor, so I have the resources available in my portal.

4. Other Useful YouTube Playlists

These are a few playlists that I went over to get a good grip over each of the required concepts:

• MIT 6.S191: Introduction to Deep Learning:
  Usefulness 3/5 — It will help you get familiar with Deep learning and developing neural networks using TensorFlow. You should cover the first 3 videos in the playlist — Intro to DL, Recurrent Neural Network and Convolutional Neural Networks.
  Learning Value 4/5 — Gives you a good refresher on the basics and I used it as a good video to watch when I was just in the mood to watch and not actually do much hands-on.
  Cost: Free
  Time: 3 hours

• Convolutional Neural Networks by Andrew NG
  Just like the above playlist but with Andrew NG’s method of explaining Deep learning. I watched this series last year, very helpful.
  I watched the videos that Laurence recommended in his course.
  Usefulness: 3/5 — More on the basics.
  Learning Value: 4/5
  Time: 8–10 hours to understand the concepts in each video.

• Sequence Models by Andrew NG
  Usefulness: 3/5 — More on the basics.
  Learning Value: 4/5
  Time: 8–10 hours to understand concepts in each video.

5. PyCharm Tutorial Series and Environment Set up guidelines

In case you have never worked in an IDE before, getting familiar with the exam environment is highly recommended.

Usefulness: 5/5 (required) — This is a getting started series for PyCharm beginners that’ll help you get up to speed with how to use PyCharm efficiently.

Learning Value: NA

Make sure you read the environment set up guidelines to take the TensorFlow Developer Certificate exam.

_[https://www.tensorflow.org/site-assets/downloads/marketing/cert/Setting_Up_TF_Developer_Certificate_Exam.pdf?authuser=4](https://www.tensorflow.org/site-assets/downloads/marketing/cert/Setting_Up_TF_Developer_Certificate_Exam.pdf?authuser=4" class="by cv lk ll lm ln" target="blank" rel="noopener nofollow" style="box-sizing: inherit; color: inherit; text-decoration: none; -webkit-tap-highlight-color: transparent; background-repeat: repeat-x; background-image: url("data:image/svg+xml;utf8,<svg preserveAspectRatio="none" viewBox="0 0 1 1" xmlns="http://www.w3.org/2000/svg"><line x1="0" y1="0" x2="1" y2="1" stroke="rgba(41, 41, 41, 1)" />"); background-size: 1px 1px; background-position: 0px calc(1em + 1px);)

Follow the instructions mentioned in the PDF because the certification team can’t be held responsible for your negligence.

Whoa! That is a long list of resources, how did you manage to study?

My Schedule for Preparation

By the end of April, I was sure to check this off my list. I’d take it up just like any other project and was determined to see it through.

So, I used to plan every night what I was about to do the next morning. The pink-colored time slots are blocked for studying for the course. These 3–4 hours in the morning were my most productive where I could grasp the most.

I had a fairly consistent routine throughout the 2 weeks and I raised the intensity when I got close to exam day with more than 5–6 hours of practice each day.

Ok, so w_hat was your process of studying?_

How I studied

I used to first watch the lessons of each week, then practice the code in the colab provided following the video lessons.

At the end of each week, I would complete the assignment designed by Laurence in his course.
NOTE: I used to write the entire code myself rather than just completing the placeholder code.

I would also revisit the chapters in the Hands-on ML book later at night before sleeping or at the end of my time slot just to make everything crystal clear. Then I would learn about the next steps that were beyond the exam curriculum.

TL;DR: WATCH. CODE. PRACTICE. READ. REPEAT.

All prepared to take the Exam — What’s next?

If you think that you have covered all the skills mentioned in the Handbook and feel like you’re ready to take the exam, that's great.

Now you're ready to purchase your exam. It's served by a third party platform called TrueAbility. You are required to submit your government issued ID (passport would work) for authentication.

Pay $100 for the exam. You are now good to go, you can start the exam as and when you feel ready.

They provide you detailed instructions on how to set up your PyCharm for the exam. Here’s what I recommend doing before starting your exam:

• Make sure that you have a good reliable internet connection.

• Make sure that you have gone through the PyCharm beginner tutorial if you’re new to the IDE.

• I tested my PyCharm by running a few TensorFlow tutorials. They worked fine and I was ready to install the exam plugin to get started.

• I read the exam instructions thoroughly before hitting the start exam button. It will be provided to you after signing up for the exam.

HIT the Start Exam button!

During the Exam

Your exam environment will be created and you’ll be directed to the questions you'll have to solve. I won’t be sharing the details of the exam as that’d be unethical.

In my experience, it all went smoothly, and I was fairly confident I'd complete the exam after looking at the questions. And sure enough I completed the exam within 3 hours.

Tips and Tricks

• Make sure you practice a few exercises on PyCharm 1–2 days before the exam rather than just working on Colab notebooks.

• For the models that took time on my local machine, I trained them on Google Colab and then uploaded the trained model in the project folder.

• Keep working on other questions while your model is training; I had 3 models under training — 1 on my machine and 2 on Google colab and I was working on the 4th while I was trying to tune the hyperparameters.

• If you have enough time, keep trying to get the best results for each model.

Post-Exam Rituals

When you're finished, hit the Submit and End Exam button. When I was done, I received an email from TrueAbility congratulating me on passing the exam:

There is no detailed analysis or report on how you did on the exam. They simply mention whether or not you’ve passed the exam.

After passing the exam, you are requested to join the TensorFlow Certificate Network that tells you the Certificate holders in different regions:

Where is the Certificate?

It takes a week or so to actually get your hands onto the certificate. I got mine 3 days after the exam.

My Certificate

Once you received your certificate, you can flash that badge on your social media profiles and mark it as an achievement in your resume.

Exam FAQs

Is it really that important to take the exam, can’t I just work on an equivalent project based on each section?

I’d say you can definitely do that and in fact, that is probably the better approach when you’re developing a new skill.

But the Exam helps you get recognized and, since it is coming from Google, it is nice to have. It's not a be-all-end-all solution to learning Deep learning or TensorFlow.

I want to start from scratch, what resources should I be looking at?

Learn by doing things. Many blogs talk about learning deep mathematics first but you’ll soon loose interest using that approach.

Start by learning programming (Python or any other language) and then gradually dive into Machine Learning. You can also look at this course by Andrew NG.

I always need a mentor or someone to push me to do things and solve my doubts and problems, can you propose a solution?

A mentor does indeed help in many cases. If you’re someone who wants someone to help you with theses details apart from these resources, you can look at Codementor where you’ll find ML and AI experts who can help you resolve all your queries.

This is a little expensive for me, is there a free or less expensive approach?

Yes, the Tensorflow team is offering a few stipends to people who might have some trouble affording the exam. Visit this link for more details.

If your question is not addressed here, feel free to respond to this post and I’ll get back to you. :)

What’s next?

Just like with any other skill, start building things and working on real-world projects. Start looking into open-source projects like TensorFlow. Apply for jobs with this badge and share your story with others.

I’m working on a complete Deep Learning Foundation series that’ll be useful for ML/DL aspirants. You can watch me teach on to my Youtube channel in the meanwhile.

Here is a video based on this blog where you can watch me share my journey:

I’ll be rolling out a complete series on TensorFlow soon. Subscribe to my channel for interesting data science content.

Data Science with Harshit

With this channel, I am planning to roll out a couple of series covering the entire data science space. Here is why you should be subscribing to the channel:

• These series would cover all the required/demanded quality tutorials on each of the topics and subtopics like Python fundamentals for Data Science.

• Explained Mathematics and derivations of why we do what we do in ML and Deep Learning.

• Podcasts with Data Scientists and Engineers at Google, Microsoft, Amazon, etc, and CEOs of big data-driven companies.

• Projects and instructions to implement the topics learned so far.

If this tutorial was helpful, you should check out my data science and machine learning courses on Wiplane Academy. They are comprehensive yet compact and helps you build a solid foundation of work to showcase.

We have released a 7-hour TensorFlow 2.0 course on the freeCodeCamp.org YouTube channel. The course is designed for Python programmers looking to enhance their knowledge and skills in machine learning and artificial intelligence.

Not only will this course teach you how to use TensorFlow, it will also give you a great overview of machine learning and artificial intelligence.

The creator of this course is Tim Ruscica, who is known for his popular “Tech With Tim” YouTube channel. Throughout eight modules, Tim covers the fundamental concepts and methods in machine learning and artificial intelligence like:

• core learning algorithms,
• deep learning with neural networks,
• computer vision with convolutional neural networks,
• natural language processing with recurrent neural networks,
• and reinforcement learning.

To go along with the video portion of this course, there are six information-packed Jupyter notebook files. These files contain extensive notes, instructions, and diagrams. They also include all the code used in the course so you can easily try out the code yourself. And you can access the files on Google Colaboratory, allowing you to run all the code in your browser.

After completing this course you will have a thorough knowledge of the core techniques in machine learning and AI and have the skills necessary to apply these techniques to your own datasets.

Here is a break-down of each module.

Module 1: Machine Learning Fundamentals

The first module covers the difference between artificial intelligence, neural networks, and machine learning. The machine learning fundamentals laid out in this module will provide the foundation for the rest of the course.

Module 2: Introduction to TensorFlow

This module provides a general introduction to TensorFlow. You will learn what a Tensor is and learn about shapes and data representation. You will also learn how TensorFlow works on a lower level.

While you can create machine learning models without knowing how everything works, a more in-depth understanding makes it easier to tweak models and get the best results.

Module 3: Core Learning Algorithms

You will learn four of the fundamental machine learning algorithms. Each of the algorithms will be applied to unique problems and datasets.

The algorithms covered are:

• Linear regression
• Classification
• Clustering
• Hidden Markov models

Module 4: Neural Networks with TensorFlow

In this module you will learn how neural networks work and the math behind them. You will learn about gradient descent, backpropagation, and how information flows through a neural network.

In the second part of the module you will see how to create a neural network with Karas to classify articles of clothing.

Module 5: Deep Computer Vision - Convolutional Neural Networks

This module will teach how to use a convolutional neural network to perform image classification and object detection/recognition.

You will learn about the following concepts:

• Image data
• Convolutional layers
• Pooling layers
• CNN architectures

Module 6: Natural Language Processing with RNNs

Natural Language Processing (NLP for sort) is a discipline in computing that deals with the communication between natural (human) languages and computer languages. A common example of NLP is something like spellcheck or autocomplete.

This module introduce a new kind of neural network called a recurrent neural network (RNN for short). These networks are often used for NLP.

You will learn how to use an RNN for sentiment analysis and character generation.

Module 7: Reinforcement Learning with Q-Learning

In this module you will learn about Reinforcement Learning.

This technique is different than many of the other machine learning techniques covered earlier in the course. Rather than feeding our machine learning model millions of examples we let our model come up with its own examples by exploring an environment.

You will learn how to create a machine learning model using reinforcement learning.

Module 8: Conclusion and Next Steps

In the final module, you will learn about next steps to learn more about TensorFlow and machine learning

Time to Watch!

If you are ready to start learning about TensorFlow and machine learning, watch the course below or on the freeCodeCamp.org YouTube channel.