It's a tough coaching job, but there are others that have their own challenges. Imagine taking over State Penn after the Sandusky thing.
Your question, though, demonstrates one of the challenges of coaching at Texas. We're convinced that we're the ultimate in whatever it is we're talking about.
Expectations are too high on an annual basis, and BMDs interfere. The spotlight is harsh. On the flip side, money's not a problem. Recruiting a kid to Austin has to be easier than recruiting them to Little Rock or Stillwater. Texas will get the benefit of the doubt in rankings over TCU, for example.
If there's one thing that, IMHO, is the toughest thing about coaching at Texas, it's that the program has become more of a reality TV show than a football team. Sometimes it seems like football is just icing on the cake of the "game day experience." If Herman, or whoever, is going to make the team successful, they're going to have to isolate the players from the carnival atmosphere that has taken over pretty much since Colt got knocked out of the championship game.