They don't know. They don't care.

It doesn't matter what were the original colors. As the movie was filmed in black and white, the set, the costumes, even the makeup was designed to look good in black and white and not to have correct color.

A famous example is the sets of the Addams Family which were built in pink, but the filming rendered them in grim colors. The emulsion of early films (about 1920s and earlier) was almost completely insensitive to red (reds were rendered as blacks) but on the other hand, it was overly sensitive to UV. One can say that the movies were filmed in ultraviolet, so not only the colors were "lost", but even the luminance we see on the film is absolutely nothing like the actors and set looked like to a naked eye. Here's an excellent article about peculiarities of early-film makeup, (credit to user151841 and 1006a for finding it) which explains how the actors were painted in colors resembling zombies in order to get a lively look on the film. So when/if black-and-white movies are colorized, it's counterproductive to restore the way they really looked. The new color has to reflect the way their creators wanted them to look, and there usually is no way to know that but guess.

Coloring process is not about "conversion" at all. It's an artistic process in which color is added by coloring artist as they see fit and to match to their taste.

There are lots of ways they can determine the colors.

If those movies props are still available, they can take a look at it. Usually, their descriptions are also available, they can guess the colors using them. If there is not, then they will guess or see which colors will fit best. It would depend on the producer or the person deciding on the colors to be original or better for TV.

Furthermore , if any of the production team or actors is available, they can simply ask them. If they don't know, they can come up with a color that suits that frame.

We know the obvious colors of some things, like a green tree, blue sky, blue water, red blood, white marble (as in your example) etc.

However, besides those methods, colors are guessed and it's an art to select the most suitable color.

They usually do that frame by frame. And once you know which one to use, you can copy it in other frames. For example, the color of dresses that characters wear in the subsequent shots.

If you want to know how do they do that, take a look at this question at Signal Processing SE

There are three general situations:

1. For material which was deliberately shot in black and white, there might not be any "true" intended color, so whoever is doing the colorizing may have to invent it.

2. For material that was shot in color, but for which no color copy survives, someone who is attempting to recreate the original may use any existing references as a basis.

3. Sometimes black and white films shot from color PAL video sources may end up with a "texture" superimposed upon it from the chroma signal. Alternating scan lines will be shifted by amount which depends upon the exact hue. If one merely examined a single character without knowing what color it represented, there would be four combinations of chroma and line phase that could yield it [and each possibility would yield different hues] but looking at a picture as a whole it would usually be possible to figure out which combination "makes sense".

The first situation above represents what's most commonly referred to as "colorization", but it's not the only way of adding color to images captured on black and white film.

First of all, remember that The Wizard of Oz was shot in color (except for the Kansas scenes).

But, for the general case, where the color isn't immediately obvious (sky, water, grass, trees, etc.), you have several resources:

• For costumes, you may have access to period clothing, or color photographs of such clothing (such as these amazing color photographs from Imperial Russia);
• For building exteriors and sets, you may have photographs, notes, or descriptions;
• Skin tones would be relatively straightforward;

But, yeah, sometimes you may not have an historical resource available, and go with something that a) makes sense, and b) fits in the overall aesthetic. For example, the actor's tunic in the first image may not have originally been that shade of green, but it's lighter than the vest and yet not pure white. Same thing with the actress' dress. The chosen colors may represent something that's known to have been common at the time, and they look good together on the screen.

In the old days of colorizing film was done BY HAND. Literally painting the negatives and reshooting it to make a new frame. While color film was expensive, it wasn't for still images and records of the colors used in fabrics, and set pieces allowed them to pic the color they needed.

Now it's digital, software analyzes the gray in the movie and assigns a pixel a certain degree based on light level and darkness. Thus a color hex pixel is assigned.