Often people who oppose illegal immigration do seem to also be racists.

I think if someone says they oppose illegal immigration and also genuinely feels they have done serious introspection and feel they are not racist, they might benefit from asking themselves what they dislike about illegal immigration and see if those things actually have the negative impacts they fear or if the negative impacts they see are but drops in the bucket compared to other sources of similar impacts.