It appears that the Wilhelm scream started as an inside joke, an easter egg maybe. However, over the years it has become a snowball, spawning more and more movies that incorporate the bit of audio.

Distant Drums

The scream was first used in the 1951 movie Distant Drums, as a sound effect for someone being eating by a crocodile:

Pretend an alligator just bit off your arm. Now scream. That's what a voice actor did to overdub a shriek for the 1951 film Distant Drums.
_{Cue the Scream: Meet Hollywood’s Go-To Shriek}

Voice actor of the scream

About the suspected voice actor of the scream, wikipedia reports the following:

Research by Burtt suggests that Sheb Wooley, best known for his novelty song "The Purple People Eater" in 1958 and as scout Pete Nolan on the television series Rawhide, is likely to have been the voice actor who originally performed the scream. This has been supported by an interview in 2005 with Linda Dotson, Wooley's widow. Burtt discovered records at Warner Brothers from the editor of Distant Drums including a short list of names of actors scheduled to record lines of dialogue for miscellaneous roles in the movie. Wooley played the uncredited role of Private Jessup in Distant Drums, and was one of the few actors assembled for the recording of additional vocal elements for the film. Wooley performed additional vocal elements, including the screams for a man being bitten by an alligator. Dotson confirmed Wooley's scream had been in many Westerns, adding, "He always used to joke about how he was so great about screaming and dying in films." Despite the usage of the sound, no royalties are paid.
_{Wikipedia - Wilhelm Scream}

Star wars

That was it for a long while, until sound designer Ben Burtt stumbled on the original recording. He then used it in a scene in Star Wars, in which Luke Skywalker shoots a Stormtrooper off of a ledge.

The start of the easter eggs

Reportedly, Ben Burtt then continued to use this sound effect as some sort of small red line throughout his career. As taken from his wikipedia page:

Burtt has a reputation for including a sound effect dubbed "the Wilhelm scream" in many of the movies he's worked on. Taken from a character named "Wilhelm" in the film The Charge at Feather River, the sound can be heard in countless films: for instance, in Star Wars Episode IV: A New Hope when a stormtrooper falls into a chasm and in Raiders of the Lost Ark when a Nazi soldier falls off the back of a moving car.

One of such examples might actually be a reference back to the original recording, as it is used in a scene from Indiana Jones and the Temple of Doom, when someone is actually being eaten by a crocodile once more.

Inside joke

Using that original piece of audio ended up becoming an inside joke, used by 372 movies and counting.

In the following radio broadcast, an interview with Stephen Altobello, sound editor at Spin Cycle Post in New York, it is claimed that using the piece of audio is also a way "of marking your film, of autographing the sound on it."

STEPHEN ALTOBELLO: I don't want to say it's a stupid sound, but it's ridiculous. It's certainly extremist. You watch it once, and you don't know it's there. You're like, okay this scene's fine. But when you watch it knowing it's there, it really does leap out of the sound track too. You're like oh, my God, that's a big - that's a huge ridiculous scream there, but it works. They always find the right spot, the right frame.

DAVID SERCHUK: Sound editors like Anderson and Altobello say that often when directors notice The Wilhelm they demand it be pulled. But it seems it's become almost an obsession for Altobello, and frequently the first thing he looks for in a new project is "The Wilhelm Moment."

STEPHEN ALTOBELLO: I've even tried to mix it in, like mix it into a track so that it can't be removed. Like if you want this car sound on that TV set, you gotta have the scream. I can't even turn - you know - and I act stupid, like, "Well I don't know! That's just part of it! You know?" I tried to get it into an HBO after school special about not using drugs but the filmmaker pulled it out. I tried to get it into a film called Chicago Cab, and they were like, "You've got to be kidding me."

They go on explaining that specifically the use in A Star is Born the reason is for their homages, not as much the Ben Burtt films:

STEPHEN ALTOBELLO: Whoever put it in the movie in the background for one scene, that's fine; that was probably expected. But whoever found a way to weasel it into the arrangement of a Judy Garland song, that's somebody who really pulled off the ultimate, I think, because the movie stops and it's the only thing that's happening. I'll never be able to pull that off.