It is an easement. You own the dirt under the sidewalk, but the sidewalk itself is city propert. BGE runs lines through your property and they get an easement. Easement says the property owner is limited in what they can do with their property (easy now ).
I would not want to try to fight something in court of you doing something "on county property" that is on your real estate.
Not necessarily, very often sidewalks are built in the fee simple or dedicated right of way which means you don't "own the dirt"/property on which the sidewalk is built.