Is there a reason for the appearance of the kid with the toy gun in the supermarket?

The scene is, according to audio commentary on the DVD, an homage to the Diane Arbus photograph Boy with a Grenade:

The usage of this homage is actually another homage, to Stanley Kubrick and, in particular, The Shining. Both stories deal with self-destructive writers, as well as involving scenes of madness and delusion. It can also be argued that the branches on trees in Capote mirrors the maze in the Shining (referring to the writer's mental state).

I call the photograph above a homage as The Shining also contained references to a set of Diane Arbus photographs: The Wade Sisters:

These were of course the inspiration behind The Shining's sisters:

There are plenty of other similarities in the film, but ultimately, to answer your main question - the usage of the boy with the toy gun was an (indirect) homage to Stanley Kubrick, as confirmed on the DVD.

s for the actual purpose of the scene? I like to think that its juxtaposition - between Capote seeing Perry lifeless in his cell and Capote spoon feeding him - suggests Capote is contemplating when innocent youngsters become hardened criminals. He clearly is moved by what he sees in the prison and this creepy child, with his toy gun, captivates him. Why the interest in guns? Will it last his whole life? Will he turn to real guns? Will he become a killer? All these thoughts possibly come to mind, leaving Capote disturbed as he leaves the store.