Why are sound effects done so different from reality?

Largely because, while those sounds may be picked up by microphones during the recording of the scene, the sound mixers and boom operators will likely focus the mics on the people speaking, not on their footsteps, etc. So not all the surrounding noise that's captured during the shoot of the actual scene will come through for the audience in the way it would if you were just listening to it in real life, especially since microphones don't exactly pick up sound in the same way that real ears do. Background noise recorded during the actual shoot might come out sounding a little off in the recording, or just not be as loud or distinct as if you were actually in the room, especially if the actor creating the noise is farther away from the mic. So things like footsteps will need to be beefed up with Foley sound effects.

It's also quite possible that something won't make the sound that the audience would expect to hear from it (e.g. if a piece of silverware clatters to the floor but doesn't make the sound you'd expect to hear from a metal utensil, or if a horse's hooves aren't as noisy as you'd expect for the kind of surface the horse is walking on - sure, that's the way it may sound in real life, but since the audience isn't watching the scene being shot in real life, it may end up coming across as weird to the audience unless - again - you add some Foley effects to beef up the sound a bit).

You might even re-record somebody's dialogue and dub it over after the fact in the event that background noise in a shot made it difficult to hear the person while you were actually recording. (In my experience, people will generally try to re-shoot the scene whenever possible in that scenario, but sometimes that just isn't feasible, so you end up dubbing.) Or you may need to do partial dubbing because of language issues, as Fellini famously had to do for La Strada. But I digress - you were mostly asking about sound effects, rather than actors' voices, I believe.

Alternatively, the actual place/environment where a scene is shot may differ in some ways from the environment where the scene is supposed to take place. (This is especially true for scenes shot almost entirely in front of green screen, where the environment that the actors are in is likely very different from what will be inserted with CGI and other such effects later.) Same goes for props that may not actually be made out of the kind of material that they're supposed to look like.

Then there's also using sound levels as a way to bring attention to sounds that you particularly want the audience to notice - kind of like putting a word in italics while writing. Yes, you might be just fine not putting the word in italics, but putting it in italics can affect the tone in a way that it wouldn't otherwise. Similarly, making certain sounds louder/more audible (and for longer) than they would be in real life brings focus to those sounds, and can change the tone of a scene to something more suspenseful, scary, frustrating - whatever tone the scene is going for. The short film "The Last Farm" is a good example of making certain sounds louder in order to emphasize/draw attention to them. I would be entirely unsurprised if some of those sounds are Foley. (Indeed, I'd be more surprised if somebody told me that there wasn't any Foley in there.)

This info is largely based on my personal experience, but if you want to read/hear more, I'd encourage you to watch the video on Foley sound that I linked to earlier and/or read the Wikipedia page for Foley sound is actually a decent resource.

Because it is quite difficult to record production quality natural sounds ("nats") at the same time as shooting the video/film. It is typically so fiddly and time-consuming that it is simply not economical to try to set up the scene so that you can both shoot production-quality images and to record production-quality sound concurrently. It is simply too expensive to have the entire camera, lighting, grip, etc. crews standing around just to get the sound recording perfect.

And even if you could, getting the various sounds (dialog, ambient sounds, sound effects, etc, etc. etc.) in the proper proportions is nearly impossible unless you have independent control over them to make the proper adjustments in the post-production soundtrack final mix.

Most of what we hear in film and television sound tracks are "cheated" and "re-created" to make the viewer perceive what the producer and editor and sound-director want.