Why the arrows on this video are drawn by the browser and only placed by ffmpeg
ffmpeg can draw on video by itself. The drawtext filter renders a string with a font file and drawbox paints a rectangle, and the engine this site runs in the browser is built with both. They are still the wrong tools for annotating, for a reason that has nothing to do with capability. An annotation tool needs a live preview, and the preview is drawn by the browser. If the export were drawn by a different renderer, with a different font file, different antialiasing and different rules for where a stroke sits relative to a coordinate, the text you placed carefully in the preview would land a few pixels off in the file, in a slightly different typeface, and wrap or clip where the preview did not. So nothing here asks ffmpeg to draw. Every annotation is painted by one function in the page, and ffmpeg only lays the result over the frames.
That function paints in the pixels of the original video, not the pixels of the screen. The preview is a canvas the exact size of the video frame, positioned over the player and shrunk by CSS to fit the page, so a 1080p clip is previewed on a 1920 by 1080 canvas. For the export, the same function paints each annotation into its own transparent canvas cropped to just the area it covers, which is then saved as a PNG and handed to the engine. The two points you drag are stored as fractions of the frame, and the stroke width and text size are fractions of the frame height, so an arrow looks the same on a phone clip and a 4K recording. Text height is taken from the font box rather than the ink, so a callout does not change size as you type a capital letter.
Working out the area each annotation covers is where the small details live. A rectangle is not just its corners, because half of its stroke sits outside them. An arrow is not just the line between two points, because its head spreads sideways past the shaft. Text needs room for the outline drawn around it, and a callout needs the union of its box and the dot at the point it calls out. Each of those is padded, then aligned outward to even pixel coordinates, because the output video stores colour at half resolution and an overlay on an odd pixel puts its colour half a sample away from its brightness. Keeping every layer to exactly the area it paints matters for speed as well: the engine blends each layer on every frame it is shown, and a full-frame transparent image would be blended pixel by pixel across the whole picture for nothing.
The engine command is short once that is done. The video is the first input and each PNG is another, and the filter graph overlays them one after another at their pixel positions. Each image is a single frame, and the overlay filter holds an input on its last frame once it runs out, so one still picture covers any length of time without being turned into a video first. Whether it is shown is decided by an enable expression on the timestamp, between the start and end you set, and an annotation that covers the whole clip gets no expression at all, because it would be evaluated on every frame to always say yes. Audio is checked for before the graph is built, since mapping a soundtrack that does not exist fails the whole export, and is encoded to AAC so a WebM source with Opus audio can go into the MP4.
Colour is the one place the page makes a judgement for you. The six colours are bright on purpose, because annotations sit on footage that can be any colour at all, and each stroke is drawn over a translucent dark edge so a yellow line still reads against a sky. Text needs more than that. Free text gets an outline in black or white, and callout text is black or white on its box, chosen by comparing the contrast of each against the fill the way the accessibility guidelines define it. The result is not always what habit expects: on the red offered here, black text has almost twice the contrast of white, so the page uses black. The export is burned into the pixels, so an annotation cannot be switched off in the file afterwards, and like the other video tools it takes at least as long as the clip plays.