You could sort the pixels of the color you want then use a multi dimensional array to sort groupings of those pixels. You could then determine distance/angle by the amount of pixels and their movement relative to previous frames, or other context clues on environment such as luminosity.
This is a very basic solution to the problem, but your example would be (relatively) trivial to do.