Problem with google sitemaps
Off Topic
I've done the same, everyone knows impatience. Especially google - I think they play on that a bit with their secret alg-rhythms and strange black cube ships.
Nobody can tell you if you are being penalised apart from google, and google won't tell you.
Why? Because they want you to make the sitemap, use their stuff, and rely on them. They don't care if you don't, they have plaenty of users. So a marketing strategy, a bit like apple, playing on people being impatient, applying attention to the things they are impatient about, encouraging insecurity as a means of gaining mindshare. Getting onboard with that is optional, but a sitemap is not.
2014 you need a sitemap. Just in case a sitemap becomes essential in 2015.