some people take it as wanting to deport all non-white people

Did someone actually advance that suggestion? Tons of US citizens are non-white. They don't get deported.

Having said that, I'd be in favor of changing the rule to strip citizenship and deport communists and supporters of other anti-American, anti-Constitution, totalitarian systems (hi islam, how's it hangin') as a threat to our democratic representative republic.