Surely we are being punished: The brothers believed that God was punishing them. They thought this because they now had to bring Benjamin to Egypt, even though their father did not want this. They connected this trouble with their past sin of selling Joseph to traders.
They began to see this as divine punishment (God judging wrong actions). The fear of divine punishment began to cause feelings of guilt that Joseph's cries for mercy and their father's tears had failed to produce (37:34–35).