Why does Frank Castle place a red pump beside a car?

That is a fake fire hydrant. It is not allowed to park your car next to a fire hydrant, because in case of a fire the fire brigade cannot have a clear access to the hydrant. If you park your car next to one, you will get a ticket, you will be fined, and that is what happens with Howard Saint's wife. It seems she parked next to a hydrant a she gets a ticket. Later, Howard Saints finds this ticket and this way he knows that his wife was at the hotel at the same time when his trusted associate was also there. This all is part of Castle's plan to manipulate Saint into thinking that his wife and his associate have an affair.

He is using a fake fire hydrant to make people think they can't park in that space. Blocking a hydrant is a big fine and possible towing. When he wants to park, he puts the hydrant away. When he wants someone to get a fine, he puts the hydrant there.

Here, he's doing it to get her ticketed, to further his plan.

It's all part of the punishment... He places the fire hydrant at the location he steals Mrs. Saint's car from to simply save that exact parking spot... Once Frank steals the car (while Mrs. Saint is watching movies or whatever) he goes and illegally parks it at the country club in effort to accumulate a bunch of parking tickets which are time and date stamped (without her knowing it). Frank then returns Mrs. Saint's car, collects the fire hydrant which saved the exact parking spot, and prevents any suspicion from Mrs. Saint. One can only assume Frank does this for weeks on end due to the huge glob of tickets Mr. Saint throws in his consigliere's face upon of accusing him of cheating with his wife... Combine this with Frank's phony blackmail photo scam (which baited the gay consigliere to the country club at the same time the tickets were handed out), the consigliere was forced to lie to Mr. Saint twice about his wearabouts creating the very suspicion which was comfirmed by the parking tickets (time, date, and location thereof) and the earring on his bed... Frank manufactured certainty in Howard Saint's mind that his beloved wife was cheating on him with his gay consigliere and reveals it at the end before killing him. All part of the punishment...