Parallel Universe in Fringe

Well it entirely depends on which theory of the "multiverse" is adopted. They are not all the same and/or implemented the same way. You might want to give the different theories a read to better understand how it is represented based on point of view:

The structure of the multiverse, the nature of each universe within it and the relationship between the various constituent universes, depend on the specific multiverse hypothesis considered. Multiple universes have been hypothesized in cosmology, physics, astronomy, religion, philosophy, transpersonal psychology and fiction, particularly in science fiction and fantasy. In these contexts, parallel universes are also called "alternative universes", "quantum universes", "interpenetrating dimensions", "parallel dimensions", "parallel worlds", "alternative realities", "alternative timelines", and "dimensional planes," among others. The term 'multiverse' was coined in 1895 by the American philosopher and psychologist William James in a different context.

The multiverse hypothesis is a source of disagreement within the physics community. Physicists disagree about whether the multiverse exists, and whether the multiverse is a proper subject of scientific inquiry. Supporters of one of the multiverse hypotheses include Stephen Hawking, Steven Weinberg, Brian Greene, and Max Tegmark. In contrast, critics such as David Gross, Paul Steinhardt, and Paul Davies have argued that the multiverse question is philosophical rather than scientific, or even that the multiverse hypothesis is harmful or pseudoscientific. [Source]

Within Fringe, the prime and parallel worlds are inexorably linked, hypothesized by the characters as the result of a divergent event in the past that formed the two universes. Quantum entanglement of objects between the two universes is significant, having been shown as part of the function of the doomsday device and an electric typewriter used by shapeshifters to communicate with the parallel universe

I would also like to add in Fringe's specific case, it is the idea that the red & blue universes (and potentially infinite iterations of them, as expressed in The Day We Died and tie-in comic extending that episode, Peter and the Machine) that both universes derive out of "mitosis" AND have become inextricably linked by a pair of quantum entangled devices...

The "last great storm" is a concept attributed to William Bell by Nina Sharp in Momentum Deferred with regard to the two parallel universes and the likelihood that one, or the other of them, would be destroyed once the doorway between them was opened. The conflict, a product of the "Pauli Exclusion Principle" means that no two objects can occupy the same space at the same time, which arises as a direct result of Walter's initial abduction of Peter from the parallel universe...

According to Nina, William had warned her that the "great storm" would cause one of the two universes to be completely destroyed by the other. Such events are shown to come to pass in the future Peter witnesses while using the Machine in "The Day We Died", in which the parallel universe had been fully obliterated by the singularities. To try to stave off this occurrence, Peter uses the Machine to create a bridge between the two universes to allow both sides to work together to avoid this result.

This is also furthered by "The Last Great Storm" and the Pauli Exclusion Principal, relating to the specific event that blue Walter crossed universes, in an attempt to save his counterpart's son, as his own had just died, by bringing him a solution to his disease, but the flasks breaks, and the grieving Walter in selfish act, takes Peter back to his universe and raised him as his own son....

To better explain, when one gets underneath Fringe's bigger-picture plot, it's not only revealed that the wave sync "doomsday" devices are one major reason that these parallel universes are inextricably linked (and both continue die without out the other), but it is also revealed that many previously existing Peter's and/or "The First People" kept sending the wave-syn "doomsday" device(s) back in time AND though out time (on a blank strip with no machine parts yet taken), in which the machine parts get unearthed over and over again, leading to new iterations of the timelines being made, and in which we fixate on iterations where one Peter lives and one Peter dies AND that the previous Peter(s) who lived, tended to choose one universe over the other...

As a theory to help support this, viewers may notice how there are only ever one set of Observers, with no "counterparts".

We learn that their creation date is 2167 and if The Day We Died and season Five's time period is any consolation, then we may assume that both universes kept dying at around the 2020-2030's mark and that the season 4 & 5 blue timeline is the original time line of the Observers' creation (Micheal was taken from this timeline and hidden temporarily in the season 1-3 blue timeline),

BUT it gets "printed over" with the characters from the blue season 1-3 timeline with Peter's materialization, Olivia's conscience transference, and later, Walter's memories given to him by Micheal, but yet causing a new kind of war-like situation that is no longer about either the wave sync device/universes being linked, saving child-aged Peter, or the parallel universes, but rather a fight with William Bell at first, then shifting with a fight with the Observers themselves...

"Now my question is, How come the same versions of people exist in both the universes, when every single choice made, affect the course of the future. If every choice affect the future, then why not some characters did not exist in either of the universe. I know the father of Astrid died in one of the universe, but my question is some of the characters must not have born in either of the universe."

So setting aside the specific mythology, the reason it is done this way is because it acts a storytelling choice.

Fringe is more intimately about one man's (Blue Walter Bishop's) redemption story and giving back something he took from the other two characters, which includes Peter's and Olivia's forgiveness, but also a time line where Olivia gets to fall in love and become a mother, where she doesn't have to worry as much about the state of reality of her daughter, as the season 5 timeline gets reset or branched off in the final scenes of series, where there is no initial Observer threat and where Walter and Micheal exist about a 150+ years into the future (aka variable exchange).

"Both the universes are shown with the same set of characters with a little difference in their lifestyle."

Yes and No. The red characters tend to be more extroverted and self-centered characters, where the blue characters are introverted, compassionate, but also broken.

By having the characters face their counterparts and in which a Red Peter and Blue Olivia fall in love is way to symbolize the "balance needed to fix the dying universes, get underneath the real original problem, as the main character heal and become a family together, and Walter is ultimately redeemed. In other words like almost every Bad Robot TV series, this was about spiritual progress, but this time through "established" parallel universes and iterations of those universes.

Comics are an integral part of the Fringe multiverse. To date, three series of comic books have been created that take place within the Fringe multiverse. The first series of comic books were released to promote the series in 2008. The second series of comics, "Tales From the Fringe," were released in early 2011 and the most recent comics were released later that year.

And lastly, to address the possibility of more "variation", there were several comics released along the side the TV Series. The first couple of sets don't really go outside the general premise of the characters in the TV series, however the last series published, Beyond the Fringe was willing to put the characters in different roles and within in very different settings, genres, or tones. From Peter and Olivia being Batman-like Superheroes, to Astrid being a spy, or Walter leading a space-expedition, we did get to see a great expansion of Fringe's multiverse...