Creating accessible video tools

Introduction

The goal with this project is to create a suite of tools to aid with basic media work, such as converting video formats, extracting audio tracks or creating gifs from videos. There are two main factors to how we approach this problem. The main factors are cost and performance, for this project I want to keep the website free without it costing me any significant amount of money. I also don't want to run ads on the website as I think it is distasteful as it makes the site look "cheap" and therefor not professional. Because of this I have to keep the site costs at a minimum.

Designing the site architecture

With the requirements figured out it is time to start thinking about how to go about building the site. The way these kind of sites are usually built is by having the user upload the file they want to convert some server. On the server we can then run the operations on the file and finally give the user a download link to the new file. This solution has the benefit of having control of the server ensuring that the speed to convert files will be pretty consistent no matter the power of the users device. The bottleneck here lies in the user's internet speed, as the user has to upload and download files from the server. Another issue here is queue times if the available servers don't possess the adequate processing power to handle all incoming requests we would have to add a que which could easily back up if the usage out grows our servers. This approach also isn't viable for this site as servers cost a decent bit of money especially if it needs to handle heavy traffic. Therefore another solution has to be chosen.

Webassembly is a pretty recent addition to the web development space but a great fit for this project. Webassembly isn't yet very widely used but is a great solution for websites that require high levels of performance. It is used by some major sites like Figma to power their graphics editor. For this project WASM is a great fit, using WASM the main load can be of the processing can be moved to the users device. This comes with the advantage of not being bottlenecked by network or queues and removing the need for servers for media processing making it very cheap. The drawback being that the speed of processing is now totally dependent on the user device processing speed. This is great for times when you have limited internet access but worse when users don't have access to high end hardware. But as the site is aimed more towards the professional crowd bad hardware should be less of an issue. Another benefit is that it could be made completely offline compatible as a PWA.

Creating the frontend

For the frontend the goal was to create a sleek modern looking website with good UX. To do this NEXT JS was chosen as it offers good fullstack capabilities with server rendering. There are also a lot of great existing tools to work with as the frontend is React. To allow more focus to be directed towards building out the actual site function and not creating a good looking UI from scratch shadcn ui was chosen as the component library as it provides pre styled accessible components with without abstracting anything away from the developer.

Website structure

The website consists of five pages. A landing page that explains the site and links to the tools and one page for each tool. On the tool pages the user can select a file by clicking the file input which opens the explorer or simply dragging and dropping the file. The user is then prompted to choose a file format and finally the convert button is enabled allowing the user to convert the file.

Creating the converter

The converter uses FFmpeg compiled to WASM via the ffmpeg.wasm NPM library. The FFmpeg WASM loading is encapsulated in to its own custom hook

"use client"

import { FFmpeg } from "@ffmpeg/ffmpeg";
import { toBlobURL } from "@ffmpeg/util";
import { useEffect, useRef, useState } from "react"

export function useFFmpeg({
    onProgress,
}: {
    onProgress?: (progress: number) => void,
}) {
    const [loading, setLoading] = useState(true);
    const ffmpeg = useRef(new FFmpeg());

    async function load(onProgress?: (progress: number) => void) {
        const baseURL = 'https://unpkg.com/@ffmpeg/core@0.12.6/dist/umd'

        ffmpeg.current.on("progress", ({ progress }) => {
            onProgress && onProgress(progress);
        });

        await ffmpeg.current.load({
            coreURL: await toBlobURL(`${baseURL}/ffmpeg-core.js`, "text/javascript"),
            wasmURL: await toBlobURL(`${baseURL}/ffmpeg-core.wasm`, "application/wasm"),
        })

        setLoading(false);
    }

    useEffect(() => {
        load(onProgress);
    }, []);

    return {
        ffmpeg,
        loading
    }
}

The FFmpeg instance can then be accessed from anywhere on the client via the hook.

const { loading, ffmpeg } = useFFmpeg({
    onProgress: setProgress
})

Then we can run commands with exec function from FFmpeg.

await ffmpeg.current.exec(['-i', audio.name, `out${conversionFormat}`]);

Then we create link to the in memory converted file and set it as the HREF value of an anchor tag so that the user can save it.

const url = URL.createObjectURL(new Blob([data.buffer], { type: mimeType }));
setAudioUrl(url);

/*
 * Other code
*/

<Button asChild className="mt-3 ml-auto">
    <a href={audioUrl} download={true}>
        <DownloadIcon
        className="mr-2 size-4"
        />
        Download
    </a>
</Button>

Conclusion

That's it, no more magic needed. Hook everything up with a drag and drop file input, some dropdowns for format selection and a progress bar and it's all done. You can see the finished product at video-tools-nine.vercel.app or if you want to take a better look at the code the project is open source on my github https://github.com/JoarM/video-tools.