The Chatam Sofer explains why we sing the confessional prayers on Yom Kippur.

When you confess, you are involved in the mitzvah of teshuva, repenting, and when one does mitzvot, he has to be happy! The opportunity that G-d gives us to get close to Him – what a great opportunity! That is a mitzvah, and that calls for a celebration!

…

When one is involved in teshuva m’ahavah, if one repents out of love for G-d, his averot (sins) transform to good deeds, to merits!…

But more than that, when one repents, he is, in essence, cleaning up the dwelling place, the palace of the King where the King resides because G-d resides in every one of us. If that is the case, now you are allowing G-d to stay in a much cleaner environment.”