Was the owl that woke up Cousin Vinny real or a prop?

Yes, the owl was real and trained, according to several sources. From IMDb:

According to director Jonathan Lynn the screech owl in the scene in the woods was a real owl that had a little prior training so it wouldn't be scared away by the gunfire. The crew got it to open its mouth by giving it little pieces of beef, and artificially induced screeches were added to the film in post production. The owl's reaction to Vinny shooting the gun was authentic and needed only one take. The director states on the DVD commentary, "we got amazingly lucky with that screech owl".

MentalFloss elaborates:

One of the film’s running gags is the fact that Vinny is always awakened by something—a steam whistle, noisy pigs, and, finally, a screech owl. Lynn and his team used an actual owl for the scene, “which was probably a ridiculous chance to take,” he said in DVD commentary. “People … think it’s a Muppet because its behavior was so perfect. It screeched, it looked back at Vinny, and then it looked back at the camera and screeched again. We got amazingly lucky with that screech owl.”

The owl’s screeches were added later. To get the bird to open his mouth at the right time, they used a trick: “We discovered that if you put a little bit of meat into its beak, it half swallows [it] and then, approximately three seconds later, opens its beak as the meat goes down,” Lynn said. “So we fed it a little bit of beef just before the camera starting turning so that for its first screech, which is added afterwards, his beak opened at the right moment. Everything else he did in that scene was pure luck, and we couldn’t believe our eyes when he reacted so perfectly, and of course we never shot it again.” The owl was basically a wild animal, Lynn said, though it had been trained a little bit: “He had heard a lot of gunfire in the previous weeks so that he wouldn’t get frightened by it.”