With the jerseys, it should be doable. There are teams already using this kind of software. Maybe improve on their code?
I also know that Mi-Pal uses some "histogram of gradients" method to find a robot, recognize its shirt color (without it being preset), and so discover which team has which jersey (or something). Maybe interesting to use as well?