It seems to me the preferential phrase would be “white privilege” instead. It takes the conversation to the right place and avoids the connotations of the white supremacy thing.