Wouldn't both glasses of brandy have "tasted a bit shit"?

My thought is that the GLASS was poisoned, not the alcohol. Perhaps in the bottom of the glass intended for Eggsy, there was a small bit of powder, or the inside of the glass was coated with a clear poison. When the alcohol touched it, the poison would liquefy in the alcohol and be consumed.

I rewatched the scene and agree: There's little chance Arthur could slip the poison in. We see it unfold in real time and his hands and the glass are always visible either to us or to Eggsy, except for the time that Eggsy distracts him and switches the glasses. And since Arthur wouldn't want to poison himself, there's also little chance that the poison was in the bottle. So that really leaves us with the option that the glass was already laced with poison, like the other answer says, which isn't farfetched because this is a snifter with a concave, opaque bottom. A tiny dose of poison would probably go undetected.

So the most sensible scenario is that Arthur knew Eggsy was coming (either Eggsy called ahead or Arthur was notified he's in the building) since we see that he's already laid out 2 glasses in advance, and one of them was already poisoned somehow because Arthur wanted to control Eggsy and kill him if he won't go along with Valentine's plan. In fact, it's possible Arthur did it to the other Kingsmen too just before this scene (since he says they all drank a toast to Harry just now) and then decided to deal with Eggsy in the same manner.

I always thought he said this because he just finds whisky a bit shit. Considering he's from a pretty poor walk of life, he probably hadn't tasted whisky, let alone good whisky before.

I always perceived his reaction as more of a Oh, whisky tastes shit rather than Oh, this specific brand of whisky tastes funny, that's unsual.

"A bit shit" in this case just means it tasted terrible to him. Arthur may very well enjoy/appreciate good whisky.