Similar to your first idea, each team picks 11 players, and they have to play O and D throughout OT.
You can only sub in a player if someone gets injured.
If a player is penalized, the other team can take the yards or have the offending player sit out on the next play, leaving that team short handed, like a power play in hockey.