Why didn't Sam come back to the White House?

His fate is addressed towards the end of the final season of the show. From Wikipedia:

Although Sam is mentioned occasionally following his departure — most notably calling Josh to tell him to "roll with the punches" after the latter unwittingly caused the defection of a Democratic Senator — he is not seen in the series until the last episodes of the seventh and final season, following the election of Congressman Matt Santos as President. Resolving the debate over the result of the California 47th's special election, it is implied that Sam was defeated by Congressman Webb and declined the promotion to Senior Counselor to the President that had been suggested by Toby. After summarily quitting politics, Sam remained in his home state of California and joined an unnamed law firm in Los Angeles which pays him a salary that would "make [Josh] puke".

The "real world reason" for Rob Lowe's exit from the show was apparently money:

Lowe's salary is at the heart of the dispute. He has reportedly been earning the same $75,000-an-episode paycheck since the series began in 1999 and he's upset the show's producer, Warner Brothers, wouldn't even consider giving him a raise, even though other cast members have received salary increases.

Sheen's salary recently tripled, reaching $300,000 an episode. Cast members Bradley Whitford, Richard Schiff, John Spencer and Allison Janney joined forces to successfully pressure producers for pay hikes, although they reportedly still make less than Lowe.