Rory and the engagement ring

From near the end of the episode "Flesh and Stone" earlier in the same season:

DOCTOR: Neither did you. I kept saying. The Angels all fell into the Time Field. The Angel in your memory never existed. It can't harm you now.
AMY: Then why do I remember it at all? Those guys on the ship didn't remember each other.
DOCTOR: You're a time traveller now. Amy. It changes the way you see the universe, forever. Good, isn't it?
AMY: And the crack, is that gone too?

Being a time traveller increases the ability to remember things that were erased from history. That's how the Doctor, and to some extent his Companions, can remember a lot of past events which, after goodness knows how much changing the past and a couple of times rebooting the universe, probably never actually happened in the current timeline. And that's how Amy can remember the soldier-clerics who were sucked into the Crack, even though they can't remember each other.

And, just after Rory was erased from time:

DOCTOR: Keep him in your mind. Don't forget him. If you forget him, you'll lose him forever.
AMY: When we were on the Byzantium, I still remembered the Clerics because I am a time traveller now, you said.
DOCTOR: They weren't part of your world. This is different. This is your own history changing.
AMY: Don't tell me it's going to be okay. You have to make it okay.
DOCTOR: It's going to be hard, but you can do it, Amy. Tell me about Rory, eh? Fantastic Rory. Funny Rory. Gorgeous Rory. Amy, listen to me. Do exactly as I say. Amy, please. Keep concentrating. You can do this.
AMY: I can't.
DOCTOR: You can. You can do it. I can't help you unless you do. Come on. We can still save his memory. Come on, Amy. Please. Come on, Amy, come on. Amy, please. Don't let anything distract you. Remember Rory. Keep remembering. Rory's only alive in your memory. You must keep hold of him. Don't let anything distract you. Rory still lives in your mind.
(The Tardis comes to a sudden halt. They are thrown to the floor. The ring box lands in front of the Doctor.)
AMY: What were you saying?

It's harder to remember something that's been erased if you were close to it. This may also contribute to the soldier-clerics' inability to remember each other, if they'd served together and known each other for years. And this is why Amy can't remember Rory for long but the Doctor can.

As for why the ring remains, my best guess is that the TARDIS protected it? You're right that it doesn't make sense for Rory's engagement ring to remain when Rory himself never existed ... however, it does make some sense, especially given (spoilers!) the origin of the Crack, that its influence can't penetrate inside the TARDIS. It's often implied in the show that the TARDIS is everything-proof, that inside it you're basically as absolutely safe as you can be. Safety from time-altering influences is a reasonable thing for a time capsule to give you. So even when Rory was sucked into the Crack, and presumably other things he'd created then disappeared from history, the ring remained because it was inside the TARDIS.