I personally think Texas belongs, but I think Aggy very clearly not being as strong as their ranking indicated is also going to hurt the discussion internally. I haven't bought them all season, similar to ND, and I would be willing to bet some of the committee members agree.
Texas being very inconsistent over the season despite some of the impressive performances they have turned in hurts as well. I'm not saying I think that should be a factor, but I think the committee will take that into account and maybe even say something about it publicly. That's just a gut feeling.
It's funny, because I had a long conversation with this with my best friend a couple days ago and I was firmly in the "Texas beats Aggy and they're 100% in" camp, but I woke up this morning with a lot of the same doubts I saw many people (including yourself) posting on this board over the last week.
Ultimately, I just don't think the committee will give you enough leeway. I think they probably should, but I don't think they will.
And yeah, that's a strong case against scheduling marquee OOC matchups and that kinda sucks.