Don’t worry, there’s a solution: modding the hell out of your game.
Whether it be not enough followers or certain companions not being good enough, room for improvement will always exist. However, as TES gamers we all know there’s always something to complain about. It’s fair to say the Skyrim follower system works much better than its pretty poor Oblivion counterpart.