In Mystic River, How did listening to the 911 tape solve the case?

This break through might look a little vague but still there was a shot in the dark for Sean and Whitey. The only connection they could make from the case was that the shot fired to kill Katie was fired using the gun of 'Just Ray'. So there was a possibility that someone from the Harris household would have committed the crime. However the elder Harris had passed his polygraph with flying colors as mentioned by Sean. When they went to question Katie's boyfriend Brendan for the first time at his house they had met the Harris junior with his friend John O'shea too.

After listening to the 911 tape they found out that the boys knew more about the murder. Hence they might have taken a wild guess that mute Harris junior might have access to that gun and he was always with his friend John. Although it looks to be exaggerated but for investigative purpose they went to meet him.

One more feasible explanation can be that after listening to the tape Sean and Whitey had gone to question Brendan about those two boys whom Katie might have known or have grudge against. But at the time they entered John had already drawn a gun over Brendan. Once police got them they confessed everything.

Sean and Whitey knew enough facts about the case to at least make a weak connection to Silent Ray and John O'Shea after listening to the tape.

1. They assumed that Katie knew her killer because the old lady witness told them that she heard a "Hi" followed by the shot.
2. They knew that the gun used to kill Katie was the gun (not just caliber or type, but THE gun) that Just Ray used in a liquor store robbery years ago based on the ballistic analysis.
3. Katie was killed in the Harris' neighborhood, assumedly on her way to Brendan's house (they were leaving for Vegas the next day.)
4. You can tell from the call that they were kids and they knew that the victim was a girl but were obviously trying to hide what they knew.

So all of this adds up to (in Sean and Whitey's mind): How can all the evidence point to Brendan and yet, based on the polygraph, he is not the killer? Answer: They were looking at the wrong brother. That at least warrants a conversation with the boys. The rest is explained by Qrious above.

As I stated in another post, from a writer's perspective, the call is simply the mechanism that allows the characters to break the case. Whether the call is a plausible case-breaker or whether the call itself is realistic seems to be the question. On the former, yes, on the latter, that remains open for debate.