What advantage does the hexadecimal numeric system have over the decimal system in The Martian?

First, the degrees being too narrow is a non-starter. The stick he uses for the 22 degree markers provides the accuracy, and a wider circle would compensate for the smaller, 13 degree sections. And the Pathfinder Lander (renamed the Carl Sagan Memorial Station) IMP (Imager for Mars Pathfinder) would absolutely be able to accurately match any arbitrary degree of rotation. A quality Servo can have a position accuracy in the sub-degree range!

That said, the main reason that Mark uses Hexadecimal is that he needs a way of communicating with NASA that someone will know, without having the luxury of being able to communicate with them how. It's a Chicken or Egg problem. There is no standard base-10 language set. But Hexadecimal is used throughout computers everywhere, and by luck one of the other astronauts had a table on their laptop that Mark could use. Without that table, he would have had to create something from thin air, and hope that they could reply back in a reasonable manner. The ASCII to Hexadecimal was an already established protocol that they could both easily use without having to make something up from scratch, and the Table provided it.

Mark to Camera: I figured one of you guys kept an ASCII table lying around. And I was right. Ladies and gentlemen, I give you super-nerd Beth Johanssen.

Followed by a nameless Engineer immediately understanding it.

Engineer: I know where he's going with this.

Any nonsense about not being able to send numbers or punctuation using a decimal notation is ridiculous, as both numbers and punctuation can be spelled out, and Mark and NASA engineers could work out short hand to prevent needing to spell out longer words.

Of course the Movie Portrayal is fairly non-realistic. Why would any engineer bother with a relatively hard to read multiplication table instead of the much more common linear table, which tend to have Hex, Octo, Decimal, and AlphaNumeric side by side. Why bother with a Question Mark when Hexadecimal has a Question Mark in it (0x63) AND they don't even bother to use punctuation when they send him anything. They also didn't use a space, leaving Mark to figure out what they mean by context.

After reading an answer to the exact question on Science Fiction & Fantasy Stack Exchange (as pointed by Richard), it became very clear why hexadecimal is the better system to communicate with.

The reason has little to do with the number of slices in the circle, but mainly with the pre-agreed upon protocol of communication.

You see, using a decimal system means that each character in Ascii would require 3 digits to represent. While hexadecimal can represent any character in 2 digits.

Also, the hexadecimal will make it easier to distinct and to decipher than decimal.

Consider the following sequence of decimals: 072101108108111

The above stand for:

072 -> H
101 -> e
108 -> l
108 -> l
111 -> o

but it could easily be mistaken by:

007 -> ?
210 -> "
110 -> n
810 -> ?
811 -> ?
1   -> ?

Whereas in hexadecimal, that sequence would be: 48656c6c6f

First of all, the number of digits a down by a third, and second, the margin of error is reduced by a large extent.

The hexadecimal system would only use 2 "bytes" per character. With the decimal system there could be letters or symbols requiring 3 "bytes" per character. If you want to avoid confusion and add a 0 in front of characters requiring 2 "bytes", you effectively add 50% to the communication compared to the hexadecimal system.

Plus, since Johanssen has an ASCII table laying around, chances are it contains both the decimal, octal and hexadecimal value for each character. This makes it easy to use an established system of communication.

Errors in the stream are also far more obvious with hex than with decimal. For example, if you get the following stream of characters, you know something is wrong:

141424344...

Numbers fall into the hex range 30-39, uppercase letters are 41-5A, lowercase are 61-7A. Space is 20. If you get a two-character sequence that does not start with 3, 4, 5, 6, or 7, or is not "20," then you know the sequence has an error.

But if you shift that sequence forward one character (ignore the leading "1"), it makes sense again: "ABC." You can assume the first character got garbled in transmission and is lost, but the rest of the transmission is readable. It would be far more difficult to do that with decimal.

16 positions is much easier for a human to read at a glance than 10 would be. A compass rose is a 16-position system (N, NNE NE, ENE...). Now try to think of a commonly used circle divided into tenths. There really aren't many, and we aren't as used to reading them.

There's a subtler advantage to hex ascii: the highest bit will never be set. If you see a hex digit greater than 7, it must be tge second digit of the chsracter. That gives you a quick way to realize you dropped a digit while transcribing, since half the second-nybble digits will be 7 or greater.

But, yeah, it's mostly a matter of "us geeks already know a suitable code; many of us could recreate most of it from memory., it'll be really easy for the machines to handle... why not."

As was said in most of the other answers, hexadecimal (and, more generally, base-2^n) numbers are THE standard in all things computer.

But there is another practical reason: when designing a system of communication involving humans, one wants to maximize throughput while minimizing the possibility of human error. And in terms of human error, hexadecimal notation is a lot more convenient when converting to ASCII, because since each character is one byte long, in is represented by precisely 2 hexadecimal digits. For comparison, in decimal a byte would be about 2.408 digits per byte in the case of continuous streaming (which would vastly increase human error), or 3 when encoding each character separately (which decreases throughput while still increasing probability of error, as one has to then convert the digits to a number afterwards and match the numbers to a linear table instead of using the more natural 16x16 ASCII table where the first digit is one coordinate and the second digit the other). Additionally, this perfect match eliminates the question of which mode of transmission to use (continuous or per-character).

All that is well and good but please tell me how this wouldn't be simpler to transcribe. It's what i thought of when he first started to work it out in the movie: 1. Using the numbers 5,4,3,2,1... create your own "ASCII" style table with 52 letters, 20 digits, required punctuation and space. You have 120 combinations right there if I am not mistaken. So 11111 could be A. 11112 could be B. Point to a number and pause 2 seconds then and spin camera 360 degrees and then point to next number to ensure repeating digits are recognized. If the camera has vertical panning capabilities, have it go up then down for all spaces. This would reset every word at least to help with missed letters.

So I only need 5 signs to point to which is way easier to follow than 16 or whatever. Plus I have more "compression" so quicker transmissions.

Write it on a sign and let them take a picture of your instructions. What am I missing?