Obama started out popular but his approval rating dropped to the low 40s fairly quickly, just in time for both of his midterm elections.
Being moderate had nothing to do with it. In fact, Gallup polling in 2012 only found about 10% of the population that said Obama was too conservative, compared to 35% that pegged him as "just right" and 51% as too liberal:
https://news.gallup.com/poll/4729/presidency.aspx
The dip in Obama's popularity was mainly due to the sluggish pace the economy recovered at. In 2014 it was a coordinated hatchet job between the GOP and the media that sought to blame him for ISIS and the ebola crisis, which they conveniently stopped talking about the day after the GOP swept the Congressional elections in 2014.
Ideology does not matter to most voters, at least not as much as you think it does.