Why didn't Elio and Oliver end up together in "Call Me By Your Name"?

Well, to me the movie left a few loose endings (like the ones you asked about) and are open to multiple interpretations.

We do not know much about either Elio's past or Oliver's past. It is clear that Elio becomes aware of his sexuality after meeting Oliver, but what Oliver is, that we don't know.

1. Oliver may be bisexual, but hides his true self because of the society.
2. Oliver may have been straight all his life and just like Elio, he discovers his other self after meeting Elio. And now he lacks the courage to come out, because of the society. He hates to come out of his comfort zone.
3. There might be a third possibility which I don't want to throw away. Oliver IS straight. When Elio first confessed his feeling to Oliver, Oliver ignored. But then after realising that he is in a much more liberal family (and also what harm may it do, he is going to go away in few weeks !!), he takes the chance and plays with Elio.

So, the reason for leaving Elio, is either societal fear or Oliver's straight orientation.

I found it really strange that while both Elio and Oliver were so into each other, they never raised a question on the imminent parting; they accepted it like it was a dogma.

If they started asking questions about those things, they would simply lose the time they had, which obviously they didn't want to. It is like either you take what you get or regret for the rest of your life.

...however, in one particular scene he asks Elio to come towards him, shows his affection (doing something) and then shuts the door! What was that about?

Oliver just checked in broad daylight if Elio really had any feeling for him or if the previous night was a mistake.

Now, at the end I would like to add: it doesn't matter why Oliver didn't stay with Elio. The point is to show the struggles of a teenager with various emotions related to first love and loss, and grief is a part of that, which couldn't be shown if the story had a happy ending!

It's because of Oliver's fears. Oliver is from America, this poses two obstacles for their relationship as Oliver is 24 while Elio is 17. Oliver by American laws is committing statutory rape, and around the time the movie takes place homophobia is very prevalent on top of that homosexuality is considered a mental illness at this time. This can be backed by Oliver telling Elio how lucky he is to have his parents as his own father would've carted him off to a correctional facility.

Throughout the movie Oliver shows he's hesitant to indulge in his attraction to Elio because of his fear of Elio's age and society. Oliver is the first to show his affection towards Elio at a volleyball game. Elio's uncomfortableness causes Oliver to believe he's molesting Elio. He's always very cautious asking Elio if he's ok with things and worrying about how he feels because of his age. As far as society goes when Elio confuses his feelings for Oliver in public Oliver rejects him and says they can't talk about such things in public as they are wrong. He's constantly worried about not doing the wrong thing.

Oliver is the older of the two, so of course he can come as the more dominant one, but Elio has just as much control over Oliver as Oliver has over Elio. Oliver was only hesitant at first because of Elio's initial reaction on the volleyball field he's the one who caught feelings first. The blowjob scene was vital to Oliver and he did it because he wasn't sure if Elio was actually attracted to him or not. That's why when he made Elio hard he stopped he had confirmed Elio was into him and it wasn't a fluke.