If you need to use a coarse abrasive for stubborn rust on mirror-finished armor you need to remove the rust and then return the armor to a mirror finish using the same procedure that got it there in the first place, that is using finer and finer grits until finally using a polishing compound to match the finish.
Because I have absolutely 0 desire to ever go through that much work to clean my armor I shy away from anything with a mirror finish
Your best bet is to prevent rust in the first place by keeping your armor oiled or waxed. I prefer the use of Breakfree CLP firearm cleaner, lubricant, and protection (hence the CLP
) It forms a film on the surface of the armor and has always prevented surface rust from forming even after long-term storage with no re-application.
When I wear my armor for long periods (as I did this past weekend), it got some dirt and some minor rust spots on the surface. I use gray scotch-brite pads with CLP to remove it and then wipe it clean with a sock or rag, and the oil will stay on there and then I'm done until the next armor-wearing marathon.
If you're not really attached to a mirror-finish, I would recommend biting the bullet and removing the finish with a gray or green scotch-bride pad and bringing your armor to a satin finish so you don't have to worry as much since every time you remove rust you'll match the finish perfectly.