Most of the time when Muppets perform musical numbers, the performers prerecord the vocal tracks, just because it's difficult to concentrate on puppetry and also singing in key, etc. The reason I bring this up is because I'm pretty sure this song was recorded live, which makes it all the more special. Sometimes true art is in the imperfections.
By the way, on Thursday Ernie's drum set was red and now it's blue. How many drum sets does he have, anyway?