Are all Ricks immortal?

Given the fact that there are infinite realities, this means that our Rick is essentially immortal.

There are infinite realities, but not necessarily infinite realities in which the Phoenix protocol was both developed and never axed.
When Rick and Morty jumped after Love Potion #9, Rick specifically stated that there were limited dimensions to which they could bail in the future - so the general "everything's infinite" approach to alternate realities simply doesn't apply in the show.

And then there's still general malfunctions and a simple lack of clone bodies that could lead to "dud" realities where there is a developed and non-axed Phoenix protocol yet no possibility for Rick to be resurrected in.

There may also be other limiting factors that simply were never revealed, e.g. if only a certain subset of Phoenix protocols are able to communicate cross-dimension.

My question is, is our Rick the only one whose mind can be rerouted into alternate realities, or do all the Ricks have this ability?

It doesn't make sense that Rick would want this kind of immortality if he ended up axing his own Phoenix protocol.

The other Ricks (in whose reality our Rick resurrects) seem to understand the cross-dimensional possibility fairly quickly. Our Rick also only realized it when it first happened. So it seems like this was not a planned ability, definitely not by our Rick.

It doesn't seem like this was the intended behavior of the system, but possibly an unintended consequence from designing the Phoenix protocol to retrieve its Rick from different dimensions (i.e. when portals into another one and dies); which inadvertently leads to the Phoenix protocol also retrieving different Ricks from different dimensions.

It's possible that our Rick just happens to be the first one to stumble on it, but it's statistically more likely than some Ricks already realized this - regardless of whether they intended to develop this or not.

Possibly.

According to rickandmorty.fandom

In Edge of Tomorty: Rick Die Rickpeat, the purpose of Operation Phoenix was revealed. When Rick dies in any dimension, his mind is automatically uploaded into a clone body.

Due to differences in realities / dimensions, it's possible that the Phoenix Protocol is deactivated for good in some of them.

Since the experiment started during the Big Trouble in Little Sanchez episode (season 2 episode 7) it's possible that deaths were final before that. For example, in Rick Potion #9 (season 1 episode 6)

Just as Rick twists the screw the third time, the device violently explodes, instantly (and messily) killing Rick and Morty.

So those are dead-dead, real dead. Later, another Rick first tells Morty about the infinite realities, and

Rick further explains the he just had to find one of those realities in which they died soon afterwards so that they could come in and take their place.

Unless the writers decide to resurrect them later for plot purposes. I wouldn't expect much from absolute rules in this series.