Why did Dr. Manhattan choose the hydrogen atom for the symbol on his forehead?

He [Dr. Manhattan / Jon] is also provided with a costume which he grudgingly accepts, though he refuses to accept the icon design which is provided for him (this being a stylized orbital model of the atom). Instead, Jon chooses as his emblem a representation of a hydrogen atom, whose simplicity he declares to be something that kindles his respect; accordingly, he painlessly burns the mark into his forehead. This preference for material mechanisms marks the beginning of Jon's declining humanity, which is progressively mirrored by his gradual shedding of the uniform - by the end of the 1970s, he refuses to wear anything at all except for mandatory public appearances.

^{Source: "Jonathan Osterman (Watchmen)", DC Comics Database Image source: Watchmen, Chapter IV}

Out-of-universe

Out-of-universe, this scene is an influence of or carried over from Charlton Comics' Captain Atom.

Watchmen originated from a story proposal Moore submitted to DC featuring superhero characters that the company had acquired from Charlton Comics. As Moore's proposed story would have left many of the characters unusable for future stories, managing editor Dick Giordano convinced Moore to create original characters instead.

^{Source: "Watchmen", Wikipedia}

Alan Moore: ...So, we started to reshape the concept—using the Charlton characters as the jumping-off point, because those were the ones we submitted to Dick—and that's what the plot involved. We started to mutate the characters, and I began to realize the changes allowed me so much more freedom. The only idea of Captain Atom as a nuclear super-hero—that had the shadow of the atom bomb hung around him—had been part of the original proposal, but with Dr. Manhattan, by making him kind of a quantum super-hero, it took it into a whole new dimension, it wasn't just the shadow of the nuclear threat around him. The things that we could do with Dr. Manhattan's consciousness and the way he saw time wouldn't have been appropriate for Captain Atom...

Comic Book Artist: Just to map this out: The prototype for Rorshach was The Question, right?

Alan Moore: The Question was Rorschach, yep. Dr. Manhattan and Captain Atom were obviously equivalent.

^{Source: "Alan Moore discusses the Charlton-Watchmen Connection", Comic Book Artist #9}

^{Image source: Who's Who: The Definitive Directory of the DC Universe Vol. 1 #4 as uploaded in DC Comics Database}

The Charlton Comics universe is currently known as 'Earth-4' in DC Comics. For comparison, here's the current Earth-4 Captain Atom (whose design is influenced by the Watchmen's Dr. Manhattan).
Image source: The Multiversity - Guidebook: Maps and Legends

From the comic, Dr. Manhattan chooses the Hydrogen atom because he wants a symbol he can "respect":

The Wikipedia article for his character simply states:

Instead, Jon chooses as his emblem a representation of a hydrogen atom, whose simplicity he declares to be something that kindles his respect; accordingly, he painlessly burns the mark into his forehead.

I couldn't find much other sure info but, there's a lot of conjecture out there, including relating his name to the fusion bomb, which is a hydrogen bomb and the creation part of the Manhattan Project; and inferring that, as hydrogen is the simplest element, and most abundant, it was appropriate for him.