Is Bjorn not Ragnar's son?

From the Vikings Wiki:

The Seer had once prophesized that Ragnar will have many sons. Bjorn is the first of these.

As time goes on, he becomes more and more frustrated that he hasn't been graced with more sons.

Remember, given the era as well, multiple children would have been common - especially boys. This historical page about Viking life states:

A typical woman probably bore 7 infants during her lifetime, 29 months apart on average. During pregnancy, women were expected to continue working. After the child's birth, the mother typically returned to work with little delay. Evidence suggests that mothers nursed their children until the age of 2 years, which may have dictated the interval between the births of a couple's children. A typical couple probably had 2 or 3 living children at any one time. Few parents lived to see their children marry. And fewer lived to see their first grandchild. Three generation families were rare.

So to answer your questions: Bjorn is indeed the true son of Ragnar. And Ragnar is indeed extremely frustrated that the prophecy of his many sons doesn't seem to be coming to fruition with Lagertha.

History says that Bjorn is not Lagerthas son, but the show Vikings has rewritten history yet again. Lagertha did have a son with Ragnar but it was not Bjorn. Check the wiki on them. The show is really loosely based on history...key word is loosely. Lol!

According to historical story/myth, Tale of Ragnar's sons (Old Norse: Ragnarssona þáttr), Bjorn's mother was Aslaug not Lagertha. The saga as translated by Peter Tunstall as the following:

Ragnar married Aslaug, whom some call Randalin, the daughter of Sigurd Fafnir's Bane and Brynhild Budli's daughter. They had four sons. Ivar Boneless was the eldest, then Bjorn Ironside, then Hvitserk, then Sigurd. There was a mark is his eye, as if a snake lay around the pupil, and that's why he was called Sigurd Snake-in-Eye.

I'm not sure why they changed that for the tv show.

In this scene (1:02) from S05E11 Rollo says "and that Björn is my son".

However, it contradicts the scene (0:51) from S04E16 when Hvitserk and Björn hears Ragnar's spirit (?) speaking about piglets.

Im convinced he's Rollo's in the show. No matter show writing or historical facts. He's just like young Rollo with the exception of raping women on raids. He's a brute. He's twice as big as everyone else. He's narcissistic and cares about no one but himself. Only doing what's right out of obligation. Cares nothing for any of his wives or children. Changes from woman to woman for no reason, abandons his children left and right. He's basically pre Paris Rollo. And I had read about the theory of Rollo and Lagertha and it all makes sense with tensions in season 1.

I am pretty sure Bjorn is Rollo's son. Well explains tension between Lagherta and Rollo from earlier seasons. Possibly due to rape