Why did Major Hellstrom get suspicious?

I think the reason he got suspicious was because there were many reasons to be suspicious.

When the Major first sits down, he inquires about them and where they are from, as they are officers and he does not recognize their names, despite knowing all the names of officers in the region (or something similar, I can't quite remember). They give him a reasonable explanation for their presence and his ignorance of them, and he lets it go.

Then he asks about the accent, which is again explained away by von Hammersmark, and they go on with their conversation.

Finally when the three fingers are lifted up, this is the straw that breaks the camels back. This was one irregularity too many.

Had this been the first, or possibly second, suspicious thing that they did, the Major may not have been convinced that they were spies. However, the fact that he keeps asking about them shows that his suspicion was roused from the very beginning, and he possibly sat with them in order to make sure they were not spies before letting them go about their business.

So the fact that he raised the three fingers was not the only thing that gave them away as spies, it was simply the thing that eventually convinced him.

Well I think it was the fingers; One of the things people forget sometimes is that every culture/country has its own way of moving, gestures and mannerisms.

The way that people count on their fingers is just one of the many bits of body language unique in every country.

Finger-counting varies between cultures and over time, and is studied by ethnomathematics. Cultural differences in counting are sometimes used as a shibboleth, particularly to distinguish nationalities in war time. These form a plot point in the film Inglourious Basterds, by Quentin Tarantino, and in the novel Pi in the Sky, by John D. Barrow.

A person indicating a numeral to another will hold up their fingers to signal the specific number. For example, a North American will raise their index, middle, and ring fingers vertically to signal the number 3.

For Continental Europeans, the thumb represents the first digit to be counted (number 1), as opposed to the index finger in North America. The index finger is number 2 through to the little finger as number 5. Fingers are generally extended while counting, beginning at the thumb and finishing at the little finger. For example, Europeans would use their thumb, and index, middle and ring fingers to express the number 4, whereas in North America they would use their index, middle, ring, and little finger.

Finger-counting systems in use in many regions of Asia allow the counting to 12 by using a single hand. The thumb acts as a pointer touching the three finger bones of each finger in turn, starting with the outermost bone of the little finger. One hand is used to count numbers up to 12. The other hand is used to display the number of completed base-12s. This continues until twelve dozen is reached, therefore 144 is counted.

source 1

When someone from North America or the UK count numbers on their hand it starts with the extension of the index finger (number 1) and continues to the little finger (number 4). The ‘number 5’ is then represented by extending all the fingers and thumb. The exact same process is repeated on the other hand if you want to count up to 10.

Western Europe

(i.e. Germans, Italians, Spanish, French)

When Western Europeans count numbers on their hands the thumb represents the ‘number 1’ and the index finger is ‘number 2’ and so on and so forth… with the little finger representing the ‘number 5’. Fingers are generally extended while counting, beginning at the thumb and finishing at the little finger.

source 2

So it is entirely possible that by using just one gesture he could have betrayed himself.

I am German and using fingers without thumb to indicate 3 is really a dead giveaway that you are an American. It's like drinking booze openly on the street in public, in Germany nobody would bat an eye, in the USA everybody would know I am foreign.

A common problem which caught uncounted escaped prisoners is the fork-flipping: When knifes are necessary, Germans eat with both knife and fork staying in both hands; US citizens use the knife, put it away, switch the fork, eat, switch the fork, use the knife etc. This extremely unusual way means that you could put blinking letters on your forehead: I am American!

Language was another dead giveaway: Any words with umlauts or "ch" could not be spoken correctly. So if someone was acting suspicious, the person would be kindly asked to say Eichhörnchen (squirrel) or Streichholzschächtelchen (little box of matches).

First of all, having slight accent differences does not change the fundamentals of communication. Accepting that something is pronounced slightly differently would not be a cover for someone saying, in English, "plant protection area/reserve" instead of "pflanzenschutzgebiet."

I disagree that this is a minor thing. I am a US-American, born and raised, never have lived anywhere else. I also referee a sport where we use international hand signals to indicate numbers, and I was on a school ski trip to Austria in high school where members of our group were constantly being brought two beverages instead of one because they indicated with an index finger.

As an American, as soon as he did put up two fingers for the drinks, I immediately thought "that idiot, they're screwed." - before the major ever reacted. What was more unbelievable to me, than the major keying in on that, was that this guy who is supposedly fluent in German would make that error in the first place.