What's the criteria/reasoning for Rick's three questions in The Walking Dead?

Ankit has given the authoritative answer, from an official source, and I fully agree with that reasoning. For the sake of furthering in-universe theorycrafting, and because I (at least) find it interesting, I'll provide my own personal insight on this one.

As Ankit's answer reveals, the primary purpose of the three questions is to evaluate newcomers and act as a first step to root out undesirables. There is no specific "right answer" or "wrong answer" - it's just about hearing what they have to say. Also important to note that getting "a pass" on the three questions does not mean you're free and clear. Your actions within the group will continue to be observed until trust has been established.

How many walkers have you killed?

This question is trying to determine if the newcomer has what it takes to survive. Killing walkers is something that needs to be done in this world, so if the answer is "zero" (or particularly low) that might indicate an unwillingness to act, to stop walkers, which could put others in the group at risk.

Also, not without merit, there have been a number of issues Rick's group has run into as a direct result of people refusing to kill walkers for one reason or another:

Hershall's insistence that walkers are merely "sick" and might someday be "cured," leading to a barn full of them in season 2; those two girls from season 4-ish who were feeding the walkers like pets; Morgan's inability to kill his zombie wife (seen mainly in season 1), which ultimately lead to the death of his son and himself going "off the rails" for a while. Etc.

How many people have you killed?

By extension of the above logic, killing people is also something that needs to be done to survive sometimes - however - they don't want some psycho who likes killing too much either. That could be a threat to the group either directly (he starts killing people in the group) or indirectly (he kills too many outsiders, provoking conflict with other groups.) As I said above, there's no specific "right answer," but if the number is too high, it would be a cause for concern. As time goes on in this world, it would also become increasingly probable that an answer of "zero" is likely a falsehood.

Why?

This one I generally interpret to be referencing question #2 more than #1. Basically trying to determine if there was a good reason for killing what people they did. If they did it to survive, fair enough. If they did it just for giggles or because they feel the need to kill everyone "before everyone kills me," hmm.... maybe not.

The answer was given by Andrew Lincoln, himself. Quoted from comicbook.com:

During the Talking Dead after-show, actor Andrew Lincoln explained the reasons behind the three questions. "If they give any sort of answer we think is suspect or the way they answer it is suspect, then they don’t come back," said Lincoln. "So it’s basically just a buffer, a first shield, and then it’s just a matter of proving yourself."

In a later segment on the Talking Dead, showrunner Scott Gimple explained that it's not just Rick that asks potential new group members the three questions. The three questions were implemented by the governing council and even Daryl has to ask them.

I think that it should be mentioned that while at the prison, Rick hallucinated a phone call with his dead wife, and she asked him the three questions. So he didn't come up with the questions through a logical process, his subconscious decided that these were questions to be asked. Now, we can still ask what reasons the group as a whole adopted them, but the initial coming up with them wasn't governed by rational reasoning.