Whose voice was on the phone in I'm Thinking Of Ending Things?

What we can safely say is that the voice is definitely the janitor's (old Jake). And it does seem that the very title of the movie "I'm thinking of ending things" is pointing to Jake's struggle on whether to end his life or not. It seems that the fantasy is the only thing keeping him alive and if Lucy ends things, he ends his life.

This article speaks about the question in length. Here is a section from it that talks about Jake's struggle:

Nearly every time, the voicemails begin and end the same. Jake knows he needs to answer the question the young woman has debated for the entirety of the film, and that is whether or not they are going to end things. His awareness of his inability to think clearly establishes the frail state he is currently in, and Kaufman furthers that by making his fantasy begin to fall apart around him. As the story progresses, the more confusing it gets; the characters change, and Jake gets increasingly closer to ending things. The calls are coming from inside of Jake’s head and interrupting the projection of his desires because he cannot escape who he is and what he is actually thinking about doing.

I also found this article that dissects in detail every call, it's really quite thorough. I suggest you read it. :)

According to Vanity Fair:

The Calls Are Coming From Inside the House

One clue that the young woman is a fantasy created by older janitor Jake is a series of mysterious phone calls that she receives throughout the story. In the book, these creepy calls, which are from an unidentified older man, are coming from her own number. For the film, Kaufman modifies the clue somewhat: The woman’s constantly ringing cell phone displays calls coming from “Lucy” or “Lucia” or “Louisa” or even “Yvonne.” The caller ID switches as the woman’s name switches. In both the book and film, the voicemails are almost always the same. An older man says: “There’s only one question to resolve. I’m scared. I feel a little crazy. I’m not lucid.” These are older Jake’s real, suicidal thoughts intruding on his fantasy.