It's a distance thing. Focal point distance to the sensor. I can show you how I imagine it personally. It's probably not 100% correct.
that's why smaller sensor seems zoomed in and larger sensor starts to see way too much, even the round sides of the lens.
and here is another example on why smaller sensor needs a longer lens to get the same image