These two reasons for submission summarize 13:1–4. in reverse order. The reason to avoid punishment connects to 13:3–4. The matter of conscience connects to 13:1–2.
The word conscience (Greek suneidēsis) refers here to the painful awareness of doing wrong. Christians know that they have a duty to submit to governing authorities. If they fail to do so, they would experience the pain of a guilty conscience.