Note too the precedence of the Confederates.
There were a few rounds of pardons.
But the big one gave pardons so broadly that nowhere could all of them have already been tried, much less convicted, of taking up arms against the US.
By given the Confederates pardons, the entire criminal possibility was simply nullified.