First, we’ll turn your website into a Progressive Web App (PWA). Then we'll use a free command-line tool from Google called Bubblewrap to package that PWA into an Android app. Let’s get started.

Prerequisites

If you follow along with this tutorial, there are some prerequisites:

• Basic knowledge of web development

• Your site should be live to the public, and you’ll need to have access to its source code.

• We'll use npm to install the necessary tools, so make sure you have Node.js installed.

Note: This tutorial is based on a Vite project, but the final steps with Bubblewrap are the same for any web framework.

Table of Contents

1. What is a PWA?

2. What is Bubblewrap?

   • What is a TWA (Trusted Web Activity)?

   • How TWA Verifies Trust

3. Step 1 – Configure Your PWA in Vite

   • Create Your App Icons

   • Install the Vite PWA plugin.

   • Configure the Plugin

4. Step 2 – Create the Android App

   • Create a Build Folder

   • Install the Bubblewrap CLI

   • Initialize the Project

   • Let’s troubleshoot the init command.

5. Step 3 – Answer Bubblewrap Questions

6. Step 4 – Build the App

7. Step 5 – Setting Up TWA Validation

   • What is the .well-known folder?

   • What is delegate_permission/common.handle_all_urls?

8. Step 6 (Optional) – Customize the In-App Experience

9. Wrapping Up

What is a PWA?

PWA stands for Progressive Web Application, and its goal is to make your website look and feel just like a native app. If you’ve visited a website in your browser and seen an install icon that lets you download it to your phone or laptop, you've used a PWA.

But it’s not just about the look and feel. A PWA also has app-like features, such as working offline, sending push notifications, and more.

There are two main components of a PWA.

• The manifest file describes your app, such as its name, icons, start URL, and so on.

• A service worker is a background JavaScript file that acts as a proxy. The caching and push notifications are handled by a service file, which runs as a different thread apart from the main thread.

Without these two components, browsers won’t let users download the app locally.

The manifest file and the service worker are like a checklist for the browser. When you visit a website, the browser looks for both of these components. If they are present and correctly configured, the browser knows it's a true PWA and will show the "install" icon, allowing users to download the app locally. Without them, the browser just sees a regular website, and the option to install won't be available.

What is Bubblewrap?

Bubblewrap is a command-line tool made by Google that takes your PWA and turns it into an Android App using a Trusted Web Activity (TWA).

Bubblewrap simplifies the process of creating a TWA, turning a PWA's manifest file into an Android app package (APK or AAB).

What is a TWA (Trusted Web Activity)?

A Trusted Web Activity (TWA) is a modern Android feature that lets you display your live website full-screen inside an Android app. Basically, it runs the website on the browser, but it doesn’t show the browser address bar on the App. This helps it feel like a native app.

To unlock this full-screen feature, your app needs to be “Trusted“.

This is where the "secret handshake" comes in. Android needs to be sure that the person who built the app and the person who owns the website are the same. Without this proof of ownership, the TWA will run in a fallback mode and show the browser address bar at the top, ruining the native app feel.

How TWA Verifies Trust

This trust is verified using a system called Digital Asset Links. You place a special file on your website (we'll do this in the implementation part) that contains your app's unique digital fingerprint. When a user opens your app, the Android OS checks this file. If the fingerprints match, it grants your app "trusted" status, removes the address bar, and enables other features like deep linking.

You can check this relationship yourself using Google's official testing tool: Digital Asset Links Verifier.

Now that you understand the project and tools, let’s start building.

Step 1 – Configure Your PWA in Vite

The first step is to add the two main components for a PWA: the manifest file and service worker. This is what will allow the browser to recognize it as "installable."

This guide is based on a project built with Vite, which makes this process easy with a special plugin. If you're using a different tool, the concepts are the same, but you'll need to look up different resources about the specific steps for your environment.

Create Your App Icons

Before we touch any code, we need the icons for our app. Android requires specific sizes for the app's launcher icon (what you see on your home screen) and the splash screen (what you see when the app starts).

You'll need two main sizes: 192x192 pixels and 512x512 pixels. You can use this Favicon Generator to generate your logo in the respective sizes. You can upload your main logo, and it will generate all the necessary sizes for you.

Then just download the generated files and place the 192x192 and 512x512 files into the public folder of your project.

Install the Vite PWA plugin.

A PWA requires a manifest file and a service worker. We can create these manually, but this plugin automates that entire process. It will automatically generate a manifest.json and service-worker.js for you every time you build your project.

npm install vite-plugin-pwa -D

Configure the Plugin

In this step, we’ll use this plugin and configure our app's manifest. Edit the vite.config.ts file. This configuration will tell the plugin what to name your app, which icons to use, and so on.

In vite.config.ts:

export default defineConfig({
  plugins: [
    VitePWA({
      registerType: "autoUpdate",   
      manifest: {
        name: "your app name",
        short_name: "your app short name",
        description: "write any description",
        theme_color: "#0d1117",
        background_color: "#ffffff",
        display: "standalone",
        start_url: "/",
        icons: [
          {
            src: "/web-app-manifest-192x192.png",
            sizes: "192x192",
            type: "image/png",
          },
          {
            src: "/web-app-manifest-512x512.png",
            sizes: "512x512",
            type: "image/png",
          },
        ],
      },
    }),
  ]

Now, when you run npm run build, the plugin will automatically generate the manifest and service worker files for you. With that done, deploy the changes. Now your website is a PWA.

Step 2 – Create the Android App

Now that your website is a PWA, let’s use Bubblewrap to package it into an Android app.

Create a Build Folder

Create a dedicated folder for your Android project files. In your project's root, create a new folder. I'll call mine android.

project/
├── client/
├── server/
└── android/

Now navigate to the new folder that you created.

Install the Bubblewrap CLI

npm install -g @bubblewrap/cli

Initialize the Project

Next, run the init command. Bubblewrap will connect to your live website, read the manifest.webmanifest file that Vite created, and use that information to generate a basic Android project.

bubblewrap init --manifest=https://your-website-domain/manifest.webmanifest

Run the command, replacing your-website-domain with your actual URL:

Let’s troubleshoot the init command

As you run the init command, Bubblewrap will need two key software packages: the Java Development Kit (JDK) and the Android SDK. It will offer to install them for you.

JDK setup:

? Do you want Bubblewrap to install the JDK (recommended)?
  (Enter "No" to use your own JDK 17 installation) (Y/n)

In my case, when I let Bubblewrap install the JDK, the process downloaded the files but then failed at the "decompressing" step. If you face this same problem, don't worry! The fix is to install it manually.

• Say No to the prompt.

• Download the recommended version (usually JDK 17) from a source like Adoptium.

• Install it and set up your system's environment variables to include the JDK's bin path. If you’re not sure how to set environment variables, you can check out this site: Set Environment Variables.

• When Bubblewrap asks for the path, provide it directly, such as C:\java\jdk-17.0.16.8-hotspot.

Android SDK setup:

Once the JDK is set up successfully, the next step is to configure the Android SDK.

? Do you want Bubblewrap to install the Android SDK (recommended)?
  (Enter "No" to use your own Android SDK installation) (Y/n)

Since I didn't have the Android SDK, I let Bubblewrap handle this by selecting Yes. I didn't face any problems here.

If you face any problem in setting up on Android SDK, just set it up manually and give the path, just like the JDK setup.

Step 3 – Answer Bubblewrap Questions

After the SDK is set up, Bubblewrap will ask a bunch of questions to configure your app. This information is used to create the twa-manifest.json file, which is the blueprint for your App.

Domain: Press Enter (auto-filled from your manifest)

Application name: Your full app name

Application ID: (e.g, chat.yourapp.twa)

Display mode: standalone

Orientation: portrait

Status bar color: Press Enter (accepts default)

Splash screen color: Press Enter (accepts default)

Icon URL: Press Enter (accepts default)

Include support for Play Billing?: Type Y if your app uses Google Play in-app purchases. Otherwise, N

Request geolocation permission?: Type Y if your app needs location access. Otherwise, N

In these questions, the important part is the key store and the key.

First and Last names: Your full name

Organizational Unit: Developer or anything

Organization: Your organization name

Country (2-letter code): Your country code

Password for key store: Enter a new password

Password for key: Re-enter the same password

Note: These passwords for both the key store and key should be the same, or else it will throw an error. Refer to this issue: Bubblewrap Issue.

Step 4 – Build the App

bubblewrap build --universalApk

This command starts building your application. Here, the flag universalApk will produce the .apk and .abb. If you’re going to publish your application in the Play Store, upload the .abb file to the Play Store. For our testing, we need an APK file, so this flag universalApk will produce both files. If we didn't give this flag, it would only give us .abb.

Step 5 – Setting Up TWA Validation

Once the build is done, you’ll get the APK. Transfer it to your phone and test it. When you open the app, you’ll see the browser address bar. This is because we haven't set up the "trust" between your app and your website yet. Let's fix that now.

In your frontend project, go to the public folder, create a new folder called .well-known, and inside that, create a file called assetlinks.json.

frontend/
├── public/
    ├── .well-known/
        └── assetlinks.json

What is the .well-known folder?

A well-known folder is used to store files that define configurations for protocols, as it’s used for external sources to find the validation for your website. In our case, our app checks the well-known folder from our website and verifies the validation.

Paste the following into assetlinks.json:

[
  {
    "relation": ["delegate_permission/common.handle_all_urls"],
    "target": {
      "namespace": "android_app",
      "package_name": "chat.yourapp.twa",
      "sha256_cert_fingerprints": [
       "your_sha256_fingerprint"
      ]
    }
  }
]

What is delegate_permission/common.handle_all_urls?

This is a special flag that opens all the links from the app instead of the domain. Simply put, it acts as a deeplink. After you install the app, if you click your website link from WhatsApp or from somewhere, it will open your app instead of opening in a browser, acting as a deeplink.

The package_name field should be the packageId, which you can get from your Android build folder in twa-manifest.json.

Now, get your fingerprints. Run the following command to do so:

keytool -list -v -keystore android.keystore -alias android

The alias name should be the value that you created. Once you enter this command, it’ll ask for the key store password. Enter that, and you’ll get your SHA256 fingerprint. Copy that and paste it into the assetslinks.json file in the sha256_cert_fingerprints array. Now push these changes to production. You can verify the validation in Digital Asset Links

That’s it! Now you can install the app and test it.

Step 6 (Optional) – Customize the In-App Experience

Now, additionally, there will be some cases where we want to show different content to users on the website vs the mobile app. Can we do that? Yes!

In your Android build folder, in twa-manifest.json, there will be a field called startUrl. If not, add it and add the value "startUrl": "/?twa=true". The startUrl is the entry point. I have a query parameter of value twa=true.

Run the build again with bubblewrap build --universalApk.

Now, if you open your app, it will open the app with the entry URL as yourwebsitedomain.com/?twa=true.

In your frontend:

const twaParam = queryParams.get("twa");

const [isTwa, setIsTwa] = useState<boolean>(() => {
   return localStorage.getItem("isTwa") === "true";
});

useEffect(() => {
  if (twaParam === "true") {
    localStorage.setItem("isTwa", "true"); // set the value to local storage
    setIsTwa(true);
  }
}, [twaParam]);

 {isTwa? (
    <Link to="/contact" className="underline hover:text-primary">
       Contact
    </Link> 
  ) : (
     <Link to="/download" className="underline hover:text-primary">
       Download App
      </Link>
  )}

In the code above, we check for the twa=true query parameter in the URL. If it's present, we save that information to local storage, and then we conditionally render the content for the user.

That's it. We have created an App.

If you want to change any name, colour, or splash screen, you can change it in twa-manifest.json and run the build again.

Wrapping Up

Bubblewrap is only for Android. If you want the app to support cross-platform, there are some other platforms, like Capacitor, which I’ll write about in another article.

By the way, you can check out the App that I made using Bubblewrap here: Stranger Talk.

If there are any mistakes or you have any questions, contact me on LinkedIn or Instagram.

Thank you for reading!

Table of Contents

• How to Get Your API Key

• Project Setup

• Front End Setup

• Server Setup

• Update the Front End

• Summary

• Conclusion

Prerequisites

To build this project, you should know the basics of React and Express.

What is the Gemini API?

Google's Gemini API is a powerful tool that lets you integrate advanced AI capabilities into your applications. Gemini is a multimodal model, which means you can use various types of input, like text, images, audio, and video.

It’s good at analyzing and processing large amounts of text as well as pulling information from videos – which makes it great for our use case of a subtitle generator.

How to Get Your API Key

An API key acts as a unique identifier and authenticates your requests to the service. It's essential for accessing and using Gemini AI’s capabilities. This key will allow our application to communicate with Gemini and help us build our project.

Go to Google AI Studio, then click “Get API Key”:

After you are redirected to the API KEY page, click “Create API Key“:

A new API KEY will be created. Then make sure you copy the key.

This is your API key. This key is used to authenticate your application's requests to the Gemini API. Each time your application sends a request to Gemini, this key must be included. Gemini uses this key to verify that the request is coming from an authorized source. Without this API key, your requests will be rejected, and you won't be able to access Gemini's services.

Project Setup

Start by creating a new folder for your project. Let's call it ai-subtitle-generator.

Inside the ai-subtitle-generator folder, create two subfolders: client and server. The client folder will contain the React frontend, and the server folder will contain the Express backend.

Front End Setup

First, we will focus on the front end and set up a basic React application.

Navigate to the client folder:

cd client

Then create a new React project using Vite. To do that, run the following command:

npm create vite@latest .

When prompted, choose “React“. Select “React + TS” or “React + JS”. In this tutorial, I will use React + TS. You can also follow along with JS.

Next, install the dependencies with this command:

npm install

Then start the development server:

npm run dev

How to Handle File Uploads in the Frontend

Now in client/src/App.tsx, add the following code:

//  client/src/App.tsx

const App = () => {
    const handleSubmit = async (e: React.FormEvent<HTMLFormElement>): Promise<void> => {
    e.preventDefault();
    try {
      const formData = new FormData(e.currentTarget);
      console.log(formData)
    } catch (error) {
      console.log(error);
    }
  };

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <input type="file" accept="video/*,.mkv" name="video" />
        <input type="submit" />
      </form>
    </div>
  );
};

export default App;

In the above code, we have used an input tag that will accept the video and name it as video. This name will be appended to the FormData object.

While sending the video to the server, we need to send it as a key-value pair, where the key is a video and the value is the file data.

Why key-value pairs? Because when the server receives the request, it needs to parse the incoming chunks. After parsing, the video data will be available in req.files[key], where the key is the name we have assigned in the frontend (video in this case).

This is why we are using the FormData object. When we create a new FormData instance and pass e.target to it, all the form fields and their names will automatically be available as key-value pairs.

Server Setup

Now that we have our API key, let's set up the backend server. This server will handle video uploads from the frontend and communicate with the Gemini API for subtitle generation.

Navigate to server folder:

cd server

And initialize the project:

npm init -y

Then install the necessary packages:

npm install express dotenv cors @google/generative-ai express-fileupload nodemon

These are the back-end dependencies we’re using in this project:

• express: The web framework for creating the backend API.

• dotenv: Loads environment variables from a .env file.

• cors: Enables Cross-Origin Resource Sharing, allowing your frontend to communicate with your backend.

• @google/generative-ai: The Google AI library for interacting with the Gemini API.

• express-fileupload: Handles file uploads, making it easy to access uploaded files on the server.

• nodemon: Automatically restarts the server when you make changes to your code.

Set Up the Environment Variables

Now, create a file called .env. This is where you’ll manage your API keys.

//.env
API_KEY = YOUR_API_API
PORT = 3000

Update the package.json

For this project, we are using ES6 modules instead of CommonJS. To enable this, update your package.json file with the following code:

{
  "name": "server",
  "version": "1.0.0",
  "main": "index.js",
  "type": "module",       //Add "type": "module" to enable ES6 modules
  "scripts": {
    "start": "node server.js",
    "dev": "nodemon server.js"    //configure nodemon
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": "",
  "dependencies": {
    "@google/generative-ai": "^0.21.0",
    "cors": "^2.8.5",
    "dotenv": "^16.4.7",
    "express": "^4.21.1",
    "express-fileupload": "^1.5.1",
    "nodemon": "^3.1.7"
  }
}

Basic Setup of Express

Create a file server.js. Now, let’s set up a basic Express application.

//  server/server.js

import express from "express";
import { configDotenv } from "dotenv";
import fileUpload from "express-fileupload";
import cors from "cors"

const app = express();

configDotenv();           //configure the env
app.use(fileUpload());    //it will parse the mutipart data
app.use(express.json());  // Enable JSON parsing for request bodies
app.use(cors())           //configure cors

app.use("/api/subs",subRoutes);  // Use routes for the "/api/subs" endpoint

app.listen(process.env.PORT, () => {   //access the PORT from the .env
  console.log("server started");         
});

In this code, we create an Express app instance and then load our environment variables. This is where we keep sensitive data like API keys secure. Next, we apply middleware functions: fileUpload prepares the server to receive uploaded videos, express.json allows us to receive JSON data, and cors enables communication between our frontend and backend.

We define a route (/api/subs) that will handle all requests related to subtitle generation. The specific logic for these routes will be defined in subs.routes.js. Finally, we start the server, telling it to listen for requests on the port specified in our .env file.

Now we need to create some folders to manage the code. You can also manage the entire code in a single file, but structuring it into separate folders and managing them all that way will be easier.

This is the final folder structure for the server:

server/
├── server.js
├── controller/
│   └── subs.controller.js
├── gemini/
│   ├── gemini.config.js
├── routes/
│   └── subs.routes.js
├── uploads/
├── utils/
│   ├── fileUpload.js
│   └── genContent.js
└── .env

Note: Don’t worry about creating this folder structure now. This is just for reference. Follow along with me step by step, and we will build this structure together.

Create the Routes

Now create a routes folder and then create subs.routes.js:

// server/routes/sub.routes.js

import express from "express"
import { uploadFile } from "../controller/subs.controller.js"    // import the uploadFile function from the controller folder

const router = express.Router()

router.post("/",uploadFile)    // define a POST route that calls the uploadFile function

export default router     // export the router to use in the main server.js file

This code defines the routes for our server, specifically the route that handles video uploads and subtitle generation.

We create a new router instance using express.Router(). This allows us to define routes separate from our main server file, improving code organization. We define a POST route at the root path ("/") of our API endpoint. When a POST request is made to this route (which will happen when a user submits the video upload form on the frontend), the uploadFile function is called. This function will handle the actual upload and subtitle generation.

Finally, we export the router so that it can be used in our main server file (server.js) to connect this route to the main application.

Configure Gemini

Now, let's configure how our application will interact with Gemini.

Create a gemini folder and then create a new file called gemini.config.js:

//  server/gemini/gemini.config.js

import {
  GoogleGenerativeAI,
  HarmBlockThreshold,
  HarmCategory,
} from "@google/generative-ai";
import { configDotenv } from "dotenv";
configDotenv();

const genAI = new GoogleGenerativeAI(process.env.API_KEY);  // Initialize Google Generative AI with the API key

const safetySettings = [
  {
    category: HarmCategory.HARM_CATEGORY_HARASSMENT,
    threshold: HarmBlockThreshold.BLOCK_NONE,
  },
  {
    category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
    threshold: HarmBlockThreshold.BLOCK_NONE,
  },
  {
    category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
    threshold: HarmBlockThreshold.BLOCK_NONE,
  },
  {
    category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
    threshold: HarmBlockThreshold.BLOCK_NONE,
  },
];

const model = genAI.getGenerativeModel({
  model: "gemini-1.5-flash-001",    //choose the model
  safetySettings: safetySettings,   //optional safety settings
});

export default model;    //export the model

In the code above, the safetySettings are optional. These settings allow you to define thresholds for potentially harmful content (like hate speech, violence, or explicit material) in Gemini's output.

You can read more about Gemini’s safety settings here.

Create a Controller to Handle Endpoint Logic

Now, create a controller folder, and inside it create a file named subs.controller.js. In this file, you'll handle the endpoint logic for interacting with the Gemini model.

In server/controller/subs.controller.js, add this code:

// server/controller/subs.controller.js

import { fileURLToPath } from "url";
import path from "path";
import fs from "fs";

const __filename = fileURLToPath(import.meta.url);  //converts the module URL to a file path
const __dirname = path.dirname(__filename);   //get the current file directory

export const uploadFile = async (req, res) => {
  try {
    if (!req.files || !req.files.video) {   //if there is no file available, return error to the client
      return res.status(400).json({ error: "No video uploaded" });
    }

    const videoFile = req.files.video;   //access the video
    const uploadDir = path.join(__dirname, "..", "uploads");   //path to upload the video temporarily

    if (!fs.existsSync(uploadDir)) {   //check if the directory exists
      fs.mkdirSync(uploadDir);      //if not create a new one
    }

    const uploadPath = path.join(uploadDir, videoFile.name);  

    await videoFile.mv(uploadPath);  //it moves the video from the buffer to the "upload" folder

    return res.status(200).json({ message:"file uploaded sucessfully" });
  } catch (error) {
    return res
      .status(500)
      .json({ error: "Internal server error: " + error.message });
  }
};

Since we are using an ES6 module, the __dirname is not available by default. The file handling mechanism is different compared to CommonJS. Because of this, we’ll use fileURLToPath to handle file paths.

We moved the file from the default temporary location which is the buffer to the uploads folder.

But the file upload process is not yet complete. We still need to send the file to Google AI File Manager, and after uploading, it will return a URI. This URI will then be passed to the model for video analysis.

How to Upload a File to the Google AI File Manager

Create a folder utils and create a file fileUpload.js. You can refer to the folder structure provided above.

//  server/utils/fileUpload.js

import { GoogleAIFileManager, FileState } from "@google/generative-ai/server";
import { configDotenv } from "dotenv";
configDotenv();

export const fileManager = new GoogleAIFileManager(process.env.API_KEY);  //create a new GoogleAIFileManager instance

export async function fileUpload(path, videoData) {  
  try {
    const uploadResponse = await fileManager.uploadFile(path, {   //give the path as an argument
      mimeType: videoData.mimetype,  
      displayName: videoData.name,
    });
    const name = uploadResponse.file.name;
    let file = await fileManager.getFile(name);    
    while (file.state === FileState.PROCESSING) {     //check the state of the file
      process.stdout.write(".");
      await new Promise((res) => setTimeout(res, 10000));   //check every 10 second
      file = await fileManager.getFile(name);
    }
    if (file.state === FileState.FAILED) {   
      throw new Error("Video processing failed");
    }
    return file;   // return the file object, containing the upload file information and the uri
  } catch (error) {
    throw error;
  }
}

In the code above, we created a function called fileUpload that takes two arguments. These arguments will be passed from the controller function, which we'll set up later.

The fileUpload function uses the fileManager.uploadFile method to send the video to Google's servers. This method needs two arguments: the file path and an object containing metadata about the file (its MIME type and display name).

Because video processing on Google's servers takes time, we need to check the file's status. We do this using a loop that checks the file's state every 10 seconds using fileManager.getFile(). The loop continues as long as the file's state is PROCESSING. Once the state changes to either SUCCESS or FAILED, the loop stops.

The function then checks if the processing was successful. If so, it returns the file object, which contains information about the uploaded and processed video, including its URI. Otherwise, if the state is FAILED, the function throws an error.

Pass the URI to the Gemini Model

Now in the utils folder, create a file called genContent.js:

// server/utils/genContent.js

import model from "../gemini/gemini.config.js";
import { configDotenv } from "dotenv";
configDotenv();

export async function getContent(file) {
  try {
    const result = await model.generateContent([
      {
        fileData: {
          mimeType: file.mimeType,
          fileUri: file.uri,
        },
      },
      {
        text: "You need to write a subtitle for this full video, write the subtitle in the SRT format, don't write anything else other than a subtitle in the response, create accurate subtitle.",
      },
    ]);
    return result.response.text();
  } catch (error) {
    throw error;
  }
}

Import the model that we configured earlier. Create a function called getContent. The getContent function takes the file object (returned from the fileUpload function).

Pass the file URI and the mimi to the model. Then we’ll provide a prompt instructing the model to generate subtitles for the entire video in SRT format. You can also add your prompt if you want. Then return the response.

Update the subs.controller.js File

Finally, we need to update the controller file. We've created the fileUpload and getContent functions, and now we'll use them in the controller and provide the required arguments.

In the server/controller/subs.controller.js:

//  server/controller/subs.controller.js

import { fileURLToPath } from "url";
import path from "path";
import fs from "fs";
import { fileUpload } from "../utils/fileUpload.js";
import { getContent } from "../utils/genContent.js";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

export const uploadFile = async (req, res) => {
  try {
    if (!req.files || !req.files.video) {
      return res.status(400).json({ error: "No video uploaded" });
    }

    const videoFile = req.files.video;
    const uploadDir = path.join(__dirname, "..", "uploads");

    if (!fs.existsSync(uploadDir)) {
      fs.mkdirSync(uploadDir);
    }

    const uploadPath = path.join(uploadDir, videoFile.name);

    await videoFile.mv(uploadPath);

    const response = await fileUpload(uploadPath, req.files.video);  //we pass 'uploadPath' and the video file data to 'fileUpload'
    const genContent = await getContent(response);   //the 'response' (containing the file URI) is passed to 'getContent'

    return res.status(200).json({ subs: genContent });   //// return the generated subtitles to the client
  } catch (error) {
    console.error("Error uploading video:", error);
    return res
      .status(500)
      .json({ error: "Internal server error: " + error.message });
  }
};

With this, the backend API is complete. Now, we'll move on to updating the front end.

Update the Front End

Our frontend currently only allows users to select a video. In this section, we'll update it to send the video data to our backend for processing. The frontend will then receive the generated subtitles from the backend and initiate a download of the .srt file.

Navigate to the client folder:

cd client

Install axios. We’ll use it to handle HTTP requests.

npm install axios

In the client/src/App.tsx:

//   client/src/App.tsx

import axios from "axios";

const App = () => {
  const handleSubmit = async (e: React.FormEvent<HTMLFormElement>): Promise<void> => {
    e.preventDefault();
    try {
      const formData = new FormData(e.currentTarget);
      // sending a POST request with form data
      const response = await axios.post(
        "http://localhost:3000/api/subs/",   
        formData
      );
// creating a Blob from the server response and triggering the file download
      const blob = new Blob([response.data.subs], { type: "text/plain" }); 
      const link = document.createElement("a");
      link.href = URL.createObjectURL(blob);
      link.download = "subtitle.srt";
      link.click();
      link.remove();
    } catch (error) {
      console.log(error);
    }
  };

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <input type="file" accept="video/*,.mkv" name="video" />
        <input type="submit" />
      </form>
    </div>
  );
};

export default App;

axios makes the POST request to your backend API endpoint (/api/subs). The server will process the video, and this might take some time.

After the server sends the generated subtitles, the frontend receives them as a response. To handle this response and allow users to download the subtitles, we'll use a Blob. A Blob (Binary Large Object) is a web API object that represents raw binary data, essentially acting like a file. In our case, the subtitles returned from the server will be converted into a Blob, which will then allow us to trigger a download in the user's browser.

Summary

In this tutorial, you learned how to build an AI-powered subtitle generator using Google's Gemini API, React, and Express. You can upload videos, send them to the Gemini API for subtitle generation, and provide the generated subtitles for download.

Conclusion

That's it! You've successfully built an AI-powered subtitle generator using the Gemini API. For quicker testing, start with shorter video clips (3-5 minutes). Longer videos might take more time to process.

Want to create a customizable video prompting application? Just add an input field to let users enter their prompts, send that prompt to the server, and use it in place of the hardcoded prompt. That's all it takes.

For more information about the Gemini API, refer to the official Gemini API Docs

You can find the full code here: AI-Subtitle-Generator

If there are any mistakes or you have any questions, contact me on LinkedIn or Instagram.

Thank you for reading!

In this tutorial, you’ll learn how to build a CRUD application using React and Convex. We’ll cover these operations by building a project called Book Collections. In this project, users will be able to add books and update their status once they complete a book.

Table of Contents

• What is Convex?

• How to Set Up Your Project

• How to Create the Schema

• How to Create the UI

• How to Create CRUD Functions

• Styling

• Summary

What is Convex?

Convex is the Baas Platform that simplifies backend development. Convex comes with a real-time database, and you do not need to worry about writing server-side logic separately because it provides methods for querying and mutating the database.

Prerequisites

In order to follow this tutorial, you must know the fundamentals of React. I will be using TypeScript in this project, but it is optional, so you can also follow along with JavaScript.

How to Set Up Your Project

Create a separate folder for the project and name it as you wish – I will name mine Books. We’ll set up Convex and React in that folder.

You can create a React app using this command:

npm create vite@latest my-app -- --template react-ts

If you want to work with JavaScript, then remove the ts at the end. That is:

npm create vite@latest my-app -- --template react

How to Setup Convex

We have to install Convex in the same folder. You can do that using this command:

npm install convex

Next, run npx convex dev. If you’re doing this for the first time, it should ask you for authentication. Otherwise, it should ask for the project name.

You can visit the Convex dashboard to see the project that you have created.

Now that we have the Convex and React App set up, we need to connect the Convex backend to the React app.

In the src/main.tsx, wrap your App component with the ConvexReactClient:

import { createRoot } from "react-dom/client";
import App from "./App.tsx";
import { ConvexProvider, ConvexReactClient } from "convex/react";
import "./index.css"

const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL as string);

createRoot(document.getElementById("root")!).render(
  <ConvexProvider client={convex}>
    <App />
  </ConvexProvider>
);

When you set up Convex, a .env.local was created. You can see your backend URL in that file.

In the line below, we instantiated the React Convex Client with the URL.

const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL as string);

How to Create the Schema

In your main project directory, you should see the convex directory. We’ll handle the database queries and mutations here.

Create a schema.ts file in the convex folder:

import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  books: defineTable({
    title: v.string(),
    author: v.string(),
    isCompleted: v.boolean(),
  }),
});

You can define a Schema for your document with defineSchema and create a table with defineTable. Convex provides these functions for defining a schema and creating a table.

v is the type validator, it is used to provide types for each data we add to the table.

For this project, as it is a book collection application, the structure will have title, author, and isCompleted. You can add more fields.

Now that you have defined your schema, let’s set up the basic UI in React.

How to Create the UI

In the src folder, create a folder called component and a file Home.tsx. Here, you can define the UI.

import { useState } from "react";
import "../styles/home.css";

const Home = () => {
  const [title, setTitle] = useState("");
  const [author, setAuthor] = useState("");
  return (
    <div className="main-container">
      <h1>Book Collections</h1>
      <form onSubmit={handleSubmit}>
        <input
          type="text"
          name="title"
          value={title}
          onChange={(e) => setTitle(e.target.value)}
          placeholder="book title"
        />
        <br />
        <input
          type="text"
          name="author"
          value={author}
          onChange={(e) => setAuthor(e.target.value)}
          placeholder="book author"
        />
        <br />
        <input type="submit" />
      </form>
      {books ? <Books books={books} /> : "Loading..."}
    </div>
  );
};

export default Home;

You can create your component as you wish. I added two input fields title, author, and submit button. This is the basic structure. Now we can create CRUD methods in the backend.

How to Create CRUD Functions

In the convex folder, you can create a separate queries.ts file for the CRUD functions.

Create Function

In convex/queries.ts:

You can define a function createBooks. You can use the mutation function from Convex to create, update, and delete data. Reading the data will come under query.

The mutation function expects these arguments:

• agrs: the data we need to store in the database.

• handler: handles the logic to store date in the database. The handler is an async function, and it has two arguments: ctx and args. Here, ctx is the context object that we’ll use to handle the database operations.

You’ll use the insert method to insert new data. The first parameter in the insert is the table name and the second is the data that needs to be inserted.

Lastly, you can return the data from the database.

Here’s the code:

import { mutation} from "./_generated/server";
import { v } from "convex/values";

export const createBooks = mutation({
  args: { title: v.string(), author: v.string() },
  handler: async (ctx, args) => {
    const newBookId = await ctx.db.insert("books", {
      title: args.title,
      author: args.author,
      isCompleted: false,
    });
    return newBookId;
  },
});

Read Function

In convex/queries.ts:

import { query } from "./_generated/server";
import { v } from "convex/values";

//read
export const getBooks = query({
  args: {},
  handler: async (ctx) => {
    return await ctx.db.query("books").collect();
  },
});

In this read operation, we used the built-in query function from Convex. Here, args will be empty since we are not getting any data from the user. Similarly, the handler function is async and uses the ctx object to query the database and return the data.

Update Function

In convex/queries.ts:

Create a updateStatus function. We are only going to update the isCompleted status.

Here, you need to get the document ID and the status from the user. In the args, we’ll define id and the isCompleted, which will come from the user.

In the handler, we’ll use the patch method to update the data. The patch method expects two arguments: the first argument is the id of the document and the second is the updated data.

import { mutation } from "./_generated/server";
import { v } from "convex/values";

//update
export const updateStatus = mutation({
  args: { id: v.id("books"), isCompleted: v.boolean() },
  handler: async (ctx, args) => {
    const { id } = args;
    await ctx.db.patch(id, { isCompleted: args.isCompleted });
    return "updated"
  },
});

Delete Function

In convex/queries.ts:

Create a deleteBooks function and use the mutation function. We’ll need the ID of the document to be deleted. In the args, define an ID. In the handler, use the ctx object delete method, and pass the ID. This will delete the document.

import { mutation } from "./_generated/server";
import { v } from "convex/values";

//delete
export const deleteBooks = mutation({
  args: { id: v.id("books") },
  handler: async (ctx, args) => {
    await ctx.db.delete(args.id);
    return "deleted";
  },
});

As of now, you have completed the CRUD functions in the backend. Now we need to make it work in the UI. Let’s jump back to React.

Update the UI

You’ve already created some basic UI in the React app, along with some input fields. Let’s update it.

In src/component/Home.tsx:

import { useQuery, useMutation } from "convex/react";
import { api } from "../../convex/_generated/api";
import { Books } from "./Books";
import { useState } from "react";
import "../styles/home.css";

const Home = () => {
  const [title, setTitle] = useState("");
  const [author, setAuthor] = useState("");
  const books = useQuery(api.queries.getBooks);
  const createBooks = useMutation(api.queries.createBooks);

  const handleSubmit = (e: React.FormEvent<HTMLFormElement>): void => {
    e.preventDefault();
    createBooks({ title, author })
      .then(() => {
        console.log("created");
        setTitle("");
        setAuthor("");
      })
      .catch((err) => console.log(err));
  };
  return (
    <div className="main-container">
      <h1>Book Collections</h1>
      <form onSubmit={handleSubmit}>
        <input
          type="text"
          name="title"
          value={title}
          onChange={(e) => setTitle(e.target.value)}
          placeholder="book title"
        />
        <br />
        <input
          type="text"
          name="author"
          value={author}
          onChange={(e) => setAuthor(e.target.value)}
          placeholder="book author"
        />
        <br />
        <input type="submit" />
      </form>
      {books ? <Books books={books} /> : "Loading..."}
    </div>
  );
};

export default Home;

We can now use the backend API functions by using api from Convex. As you can see, we called two API functions: you can use useQuery if you’re going to read data and useMutation if you want to change data. Now in this file, we are doing, two operations that are create and read.

We got all the data by using this method:

 const books = useQuery(api.queries.getBooks);

The array of objects will be stored in the books variable.

We got the create function from the backend with this line of code:

const createBooks = useMutation(api.queries.createBooks);

How to Use the Create Function in the UI

Let’s use the create function in the UI.

Since input fields are in the form tag, we’ll use the onSubmit attribute to handle the form submission.

//In the Home.tsx

const handleSubmit = (e: React.FormEvent<HTMLFormElement>): void => {
    e.preventDefault();
    createBooks({ title, author })
      .then(() => {
        console.log("created");
        setTitle("");
        setAuthor("");
      })
      .catch((err) => console.log(err));
  };

When you click submit, it triggers the handleSubmit function.

We used the createBooks to pass the title and author from the state. The endpoint function is async, so we can use the handleSubmit as async or use .then. I used the .then method to handle the asynchronous data.

You can create a separate component to display the data fetched from the database. The returned data is in the Home.tsx, so we will pass the data to the Books.tsx component as props.

In Books.tsx:

import { useState } from "react";
import { book } from "../types/book.type";
import { useMutation } from "convex/react";
import { api } from "../../convex/_generated/api";
import { Id } from "../../convex/_generated/dataModel";
import "../styles/book.css";

export const Books = ({ books }: { books: book[] }) => {
  const [update, setUpdate] = useState(false);
  const [id, setId] = useState("");

  const deleteBooks = useMutation(api.queries.deleteBooks);
  const updateStatus = useMutation(api.queries.updateStatus);

  const handleClick = (id: string) => {
    setId(id);
    setUpdate(!update);
  };

  const handleDelete = (id: string) => {
    deleteBooks({ id: id as Id<"books"> })
      .then((mess) => console.log(mess))
      .catch((err) => console.log(err));
  };

  const handleUpdate = (e: React.FormEvent<HTMLFormElement>, id: string) => {
    e.preventDefault();
    const formdata = new FormData(e.currentTarget);
    const isCompleted: boolean =
      (formdata.get("completed") as string) === "true";
    updateStatus({ id: id as Id<"books">, isCompleted })
      .then((mess) => console.log(mess))
      .catch((err) => console.log(err));
    setUpdate(false);
  };

  return (
    <div>
      {books.map((data: book, index: number) => {
        return (
          <div
            key={data._id}
            className={`book-container ${data.isCompleted ? "completed" : "not-completed"}`}
          >
            <h3>Book no: {index + 1}</h3>
            <p>Book title: {data.title}</p>
            <p>Book Author: {data.author}</p>
            <p>
              Completed Status:{" "}
              {data.isCompleted ? "Completed" : "Not Completed"}
            </p>
            <button onClick={() => handleClick(data._id)}>Update</button>
            {id === data._id && update && (
              <>
                <form onSubmit={(e) => handleUpdate(e, data._id)}>
                  <select name="completed">
                    <option value="true">Completed</option>
                    <option value="false">Not Completed</option>
                  </select>
                  <input type="submit" />
                </form>
              </>
            )}
            <button onClick={() => handleDelete(data._id)}>delete</button>
          </div>
        );
      })}
    </div>
  );
};

In the Books.jsx component, you can display data from the database and handle the functionality for updating and deleting records.

Let’s walk through each of these features step by step.

How to Display the Data

You can get the data passed as a prop in the Home.tsx component. If you are using TypeScript, I have defined a type for the object that is returned from the query. You can ignore this if you are using JavaScript.

Create `books.types.ts:

export type book = {
    _id: string,
    title: string,
    author: string,
    isCompleted: boolean
}

You can use the map function to display the data.

import { useState } from "react";
import { book } from "../types/book.type";
import { useMutation } from "convex/react";
import { api } from "../../convex/_generated/api";
import { Id } from "../../convex/_generated/dataModel";
import "../styles/book.css";

export const Books = ({ books }: { books: book[] }) => {
  const [update, setUpdate] = useState(false);

  return (
    <div>
      {books.map((data: book, index: number) => {
        return (
          <div
            key={data._id}
            className={`book-container ${data.isCompleted ? "completed" : "not-completed"}`}
          >
            <h3>Book no: {index + 1}</h3>
            <p>Book title: {data.title}</p>
            <p>Book Author: {data.author}</p>
            <p>
              Completed Status:{" "}
              {data.isCompleted ? "Completed" : "Not Completed"}
            </p>
            <button onClick={() => handleClick(data._id)}>Update</button>
            {id === data._id && update && (
              <>
                <form onSubmit={(e) => handleUpdate(e, data._id)}>
                  <select name="completed">
                    <option value="true">Completed</option>
                    <option value="false">Not Completed</option>
                  </select>
                  <input type="submit" />
                </form>
              </>
            )}
            <button onClick={() => handleDelete(data._id)}>delete</button>
          </div>
        );
      })}
    </div>
  );
};

This is the basic structure. We displayed the title, author, and status, along with an update and delete button.

Now, let’s add the functionalities.

import { useState } from "react";
import { book } from "../types/book.type";
import { useMutation } from "convex/react";
import { api } from "../../convex/_generated/api";
import { Id } from "../../convex/_generated/dataModel";
import "../styles/book.css";

export const Books = ({ books }: { books: book[] }) => {
  const [update, setUpdate] = useState(false);
  const [id, setId] = useState("");

  const deleteBooks = useMutation(api.queries.deleteBooks);
  const updateStatus = useMutation(api.queries.updateStatus);

  const handleClick = (id: string) => {
    setId(id);
    setUpdate(!update);
  };

  const handleDelete = (id: string) => {
    deleteBooks({ id: id as Id<"books"> })
      .then((mess) => console.log(mess))
      .catch((err) => console.log(err));
  };

  const handleUpdate = (e: React.FormEvent<HTMLFormElement>, id: string) => {
    e.preventDefault();
    const formdata = new FormData(e.currentTarget);
    const isCompleted: boolean =
      (formdata.get("completed") as string) === "true";
    updateStatus({ id: id as Id<"books">, isCompleted })
      .then((mess) => console.log(mess))
      .catch((err) => console.log(err));
    setUpdate(false);
  };

  return (
    <div>
      {books.map((data: book, index: number) => {
        return (
          <div
            key={data._id}
            className={`book-container ${data.isCompleted ? "completed" : "not-completed"}`}
          >
            <h3>Book no: {index + 1}</h3>
            <p>Book title: {data.title}</p>
            <p>Book Author: {data.author}</p>
            <p>
              Completed Status:{" "}
              {data.isCompleted ? "Completed" : "Not Completed"}
            </p>
            <button onClick={() => handleClick(data._id)}>Update</button>
            {id === data._id && update && (
              <>
                <form onSubmit={(e) => handleUpdate(e, data._id)}>
                  <select name="completed">
                    <option value="true">Completed</option>
                    <option value="false">Not Completed</option>
                  </select>
                  <input type="submit" />
                </form>
              </>
            )}
            <button onClick={() => handleDelete(data._id)}>delete</button>
          </div>
        );
      })}
    </div>
  );
};

This is the entire component code. Let me explain what we did.

First, we need to toggle the update, so we defined the handleClick function, and passed a document ID to it.

//handleClick
 const handleClick = (id: string) => {
    setId(id);
    setUpdate(!update);
  };

In the handleClick you can update the ID state and toggle the update state so that it will toggle the update input when clicked, and on another click, it will close.

Next, we have handleUpdate. We need the document ID to update the data, so we passed the event object as well as the document ID. To get the input, we can use FormData.

const updateStatus = useMutation(api.queries.updateStatus);

const handleUpdate = (e: React.FormEvent<HTMLFormElement>, id: string) => {
    e.preventDefault();
    const formdata = new FormData(e.currentTarget);
    const isCompleted: boolean =
      (formdata.get("completed") as string) === "true";
    updateStatus({ id: id as Id<"books">, isCompleted })
      .then((mess) => console.log(mess))
      .catch((err) => console.log(err));
    setUpdate(false);
  };

We need to use the useMutation to get the updateStatus function. Pass the ID and the completed status to the function, and handle the asynchronous part using .then

For the delete function, the document ID is enough. Just like the previous one, call the delete function using the useMutation and pass the ID to it.

Then pass the document ID and handle the promise.

const deleteBooks = useMutation(api.queries.deleteBooks);

const handleDelete = (id: string) => {
    deleteBooks({ id: id as Id<"books"> })
      .then((mess) => console.log(mess))
      .catch((err) => console.log(err));
 };

Styling

Finally, what’s left is to add some styling. I added some basic styling. If the book has not been completed, it will be in red, and if the book has been completed, it will be in green.

Here’s the screenshot:

This is it guys!!

You can check my repository for the full code: convex-curd

Summary

In this article, we implemented the CRUD (Create, Read, Update, and Delete) operations by building a book collections app. We begin by setting up Convex and React, and writing CRUD logic.

This tutorial covered both the frontend and the backend, demonstrating how to build a serverless application.

You can find the full code here: convex-curd

If there are any mistakes or any doubt contact me on LinkedIn, Instagram.

Thank you for reading!