Depends on how much interaction they'll be doing with the city. Friends is filmed in L.A., but takes place in NYC, the X-Files were filmed in L.A., and managed to cover many places all over the U.S. and get the atmosphere right.
Plus, if all these things were a big deal, they really aren't to me (I won't know if they seem miscast until i watch the first episode), you shouldn't have to boycott it for it to go away.
Shan