From the Wikipedia article

There is also some evidence that the community charge had a lasting effect of people not registering themselves on the electoral register to evade collection attempts. This may have had an effect on the results of the 1992 general election, which ended in a fourth successive Conservative victory, despite most opinion polls pointing to a hung parliament or narrow Labour majority.[3]

So you’re not entirely right.