Why did the original Star Trek change the Enterprise captain?

Out of universe

In the original pilot for Star Trek that aired in 1965...

It didn't. The general viewing public didn't see it and didn't know anything about Captain Pike when Star Trek first aired in 1966. The original pilot, The Cage, was only shown to network executives (and a few other insiders), who didn't like it. But they decided to commission a second pilot that was "less cerebral." (Whether that second pilot was less cerebral is a matter of...some debate, but it did the job and the show was commissioned.) Jeffrey Hunter wasn't available to do the second pilot (he was pursuing his film career, and some claim his then-wife convinced him science fiction was beneath him; they divorced a couple of years later), so the role was recast with William Shatner and along the way renamed (Christopher Pike became James Kirk). There were some concerns about his portrayal of Pike (see various notes on The Cage) so it may not have been guaranteed he'd have gotten it anyway (though he said he was offered it, and there's a nice note from Gene Roddenberry about he [Hunter] having decided not to continue with Star Trek).

(Side note: Sadly, Jeffrey Hunter died in 1969, possibly partially as as result of an accident on a film set in 1968.)

In universe

The events of the first pilot occurred 11 years prior to James T. Kirk assuming command of the Enterprise, under its previous captain, Christopher Pike, who commanded the Enterprise from 2250 to 2265 (when Kirk took over). We first see those events 13 years after they occurred, in The Menagerie, Part 1. Kirk, a now-injured Captain Pike, Spock, Commodore Mendez, and others see the events "of 13 years ago" played for them by the Talosians.

So the change was simply that after a long and successful command of the Enterprise, Captain Pike was moved up/on to Fleet Captain and a new captain was assigned to Enterprise in 2265. Not too long after that (in 2266), Captain Pike was caught up in a catastrophic accident, which ultimately lead to the events of The Menagerie.

The pilot, titled "The Cage", was never broadcast until 1988, therefore no explanation was required.

Elements of The Cage were used in the episode "The Menagerie", presented as historical events prior to Kirk becoming captain. The footage presented in The Menagerie is not necessarily consistent with the plot of The Cage, but again this presented no inconsistency to the audience as they had never seen The Cage.

The Cage was first seen by the public in its original form in a 1986 VHS release long after the original series was completed. As the audience already knew it was a pilot and not part of the original series, no particular on-screen explanation was needed for any discrepancy between events in The Cage and those seen in the broadcast series.

As for an out-of-universe reason for the change, according to Wikipedia, Jeffrey Hunter, who played Pike, declined to appear in the second pilot (which became the 3rd broadcast episode, "Where No Man Has Gone Before") and William Shatner was offered the role of Captain. As for why the rest of the crew was re-cast, the details may be lost to history. In general, the casting was not solidly determined even by the filming of the second pilot. For example, George Takei appeared in "Where No Man..." as the ship's physicist Lt. Sulu, but in later episodes became the helmsman).

Presumably, a great many changes between The Cage and the second pilot (and the other broadcast episodes) were made by the producers to better convince network executives the show could attract an audience.

When the second pilot was to be filmed Jeffery Hunter the original Captain Pike was not available because he was making a movie. This is according to the book The Making Of Star Trek pages 135-136.

The Star Trek Star Fleet is modeled after the modern navy. Seldomly does a person in the military retain the same rank and duty station more than a few years before they are promoted or transferred to a position with higher responsibilities (making them promotable). This also opens up a person’s current position for it to be filled by the next promotable person. The expectation is that all service members progress in their duties and become promotable. If a person does not earn/warrant a promotion after a certain amount of time, that indicates lack of progression. Eventually, if more time is spent in the same position, the military will seek to separate them from service as unpromotable.

Warrant officer positions are a little different than other officer and enlisted positions. The warrant officer is a valuable subject matter expert. They may stay in the same position as long as the military and the person come to an “agreement” as such. This still does not rule out frequent transfers to the same position elsewhere. Just not as often in most cases.

Based on observation alone, the Star Fleet Table of Organization and Equipment (TOE) precludes having an officer above the rank of Captain command the Enterprise. Under a Modified Table of Organization and Equipment (MTOE), a higher ranking person can be added to the crew under specific or special circumstances, as the situation dictates. This higher ranking person would command the captain of the ship due to their outranking the captain.

Therefore, Pike’s departure from the Enterprise was inevitable and expected. No explanation would be necessary. Spock’s retention as science officer is less clear. He might have been classified as the preeminent subject matter expert and exempted from the normal line of progression even though he was not a warrant officer. Or, his lack of people skills might have limited others’ perception of his leadership skills until they were proven under (and attested by) Kirk, his superior.

Coincidentally, the US military has toyed with the idea of exempting pilots from the progression structure in order to make the position more appealing to the best candidates. Instead of frequent duty position changes and an eventual promotion out of the cockpit to fly a desk, pilots will be retained in their positions for a longer period of time. That way, their family lives are more stable. And, they have the opportunity to do the thing they love most, flying, while still receiving promotions and pay increases. It will be a voluntary exemption. Pilots that chose to continue to climb the military political ladder may still choose to do so.