I would soak in oil and scrub at it with an old piece of denim before going to more abrasive material. Denim can work pretty well for scrubbing things without doing damage.
Use a pre 1982 penny. Rub only the rust spots with the penny edge. This allows you to scrape the rust from the surface with out removing the bluing. Use gun oil to remove the copper marks. I found this to be the most effective for removing light surface without damaging the bluing. On case harden bluing its invisible all other there will small blemish but not rust. I'd recommend this before going too invasive or else you won't be happy until a complete reblue job.