Paul did not want to punish the Corinthians harshly (12:21). He aimed to use his apostolic authority positively. However, his role required him to both strengthen and tear down (compare Jeremiah 1:10).
for building you up: This is Paul's favorite way to describe a stable Christian community (see Ephesians 4:12), where the Lord lives in and among the believers (2 Corinthians 13:5).