KickTrack
I built a tool that follows every player in football match footage and measures what they do: how far they run, where they spend their time, and when the ball goes from one teammate to another. It runs in real time on my MacBook.
How it works
Detection
Calibration
Playback
The model is slower than a 50 fps video, so it runs on its own thread and always takes the newest frame. Playback holds each frame back a little and places the markers from detections of that exact frame, so the video stays smooth.
Problems I ran into
The camera wouldn't sit still
My first tracker, ByteTrack, matched boxes by overlap alone. The footage pans and zooms a little, and that was enough to give players a new ID almost every frame. BoT-SORT corrects for camera motion, which fixed most of it. For the rest, when a new ID shows up where a player was just lost, I hand it back to that player.
Too slow for 50 fps
On the GPU with PyTorch, one frame took about 70 ms. A 50 fps clip gives you 20. Smaller models and lower resolutions were faster, but they missed players, and missing players is the one thing this tool can't do. So I kept full resolution and moved the model to the Neural Engine with CoreML. Then I changed the input from a square to the footage's 16:9 shape, so it stopped spending time on padding.
Frames skipped on the 50 fps clip: 64% → about 1%
Two players, one box
When two players overlap, the detector sometimes draws one box around both. The tracker keeps its ID on that box, and when they split up the ID can leave with the wrong player. Nothing looks lost, so the usual recovery never kicks in. I tied each identity to a jersey colour. A track only changes hands once the original player is seen on their own again. Switching to YOLO26s helped as well, because it separates overlapping players much more often.
Keeper and defender overlap: both kept in 19 of 19 frames (was 3)
Pitch lines that drift
To measure in metres I need to know where the camera is pointing. A full calibration takes up to 1.8 s, so between calibrations I follow the camera frame by frame. Chaining those small steps drifted, partly because the scoreboard and logos stay put while the camera pans. Now every frame is measured against one anchor frame, using points spread across the whole picture.
Passes credited to the wrong team
A player's team can read wrong for a few seconds, for example while they stand next to someone in a different kit. So a pass is counted 2 s after it happens, with the team the player showed over the ten seconds around it. On the Man City v Tottenham clip I labelled every pass by hand to check it.
9 real passes: 8 found, none invented (earlier versions invented 2 to 4)
What's still hard
- Footage from a camera in the stands doesn't calibrate. The pitch model I use was trained on TV broadcast views.
- A quick one-touch pass usually gets merged into the next one.
- If the ball can't be seen at either end of a pass, I miss the pass.
Built with
Python, Ultralytics YOLO26s exported to CoreML, BoT-SORT, OpenCV, PnLCalib for pitch calibration, and ffmpeg. Everything runs locally on an M3 Pro MacBook.