There was a recent story published on jpgmag.com by a member that suggested photography of transportation systems and/or certain buildings was illegal. The story was meant to provide Ten Tips for Street Photography; and it's not a bad list of tips (http://jpgmag.com/stories/15194)
However, there was one comment I wanted to expand upon; in the United States, there are very few restrictions to shooting in a public space (However if you intend the photographs for commercial use, you need a model release of individuals but not in all cases. The rule of thumb is if the individual is the central theme or a very large part of the photograph one needs a model release; if its a structure that is not a government building one would need a property release).
Bert Krages at http://www.krages.com/phoright.htm
Provides a great deal of information on the topic and the challenges photographers face in the post 9/11 world we live in. However, the author of the Ten Tips, suggested the best defense as the theme of his article. That of Kindness. Being kind to someone that is upset is often the best way to defuse the individual or group. However, your safety can be compromised, so while it's important to understand your rights and you have a right to defend those rights, a broken camera is not easily replaced and a broken skull is often harder to replace.
The bottom line; In Street Photography it is very likely one will face a challenging moment. Kindness is your best first defense; and in some cases the security guard or individual will react positively to Mr. Krages pamphlet (though that usually just fans the flames, if you're at that point; and while pointing at the security camera might work; it is usually well past logic at that point.) But through all this be kind; do not get overly defensive, do not turn over your gear, film or memory card! You can welcome a call to the police; because at this point, it might be the safest way to get you and your gear out of harms way. And the police are likely to react better to a kind photographer with a pamphlet explaining the photographers rights; then the 'outraged' individual or security guard.
By all means; know your rights. Understand how to protect your rights; but most importantly, keep Safe and keep your equipment safe.