One in 10 Americans believe small business owners should be free to refuse to serve or do business with African-Americans on religious grounds, according to a new poll.

A survey released this week by the Public Religion Research Institute finds that strong majorities of Americans reject the idea that businesses should be legally allowed to refuse to serve either African-American, Jewish, gays and lesbians, or atheists, but a small portion of the country still believes you should be able to.

Among the four groups, the survey found the least support for refusing service to African-Americans, but the most for discriminating against gay or lesbian individuals.

Asked if it should be legal to refuse to do business with members of the LGBT community on religious grounds, 16% said yes and 80% said no. Similarly, 15% said it should be legal to refuse service to atheists, with 81% saying it shouldn’t be. The polling found slightly less support for religious beliefs being sufficient to allow a small business owner to refuse to business with Jewish people, 12% said yes and 85% said no. And when it came to African-Americans, 10% said they supported the legal right to refuse service and 87% said they did not.

More: 1 in 10 Still Support Discrimination Against African-Americans on Religious Grounds