If you're an outdoor kinda person, it's looking like an awesome fall Kentucky weekend. I highly recommend the Red River Gorge (30 mins from Lexington) for hiking, zip lines, and world class rock climbing. Follow it up with Miguel's Pizza. Alternatively, Shaker Village is a beautiful and very unique place for hiking and preserved history (look up the Shaker people).
For other history-oriented activities, check out Ashland- the Henry Clay Estate, Mary Todd Lincoln's House, and the Waveland Museum (built by a close Daniel Boone relative). The Lexington Cemetery is also beautiful and perfect for a stroll. In addition to Perryville (mentioned up-thread), Camp Nelson is a Civil War Battlefield and National Monument- another place for rolling bluegrass hills and easy "hiking".
For nicer food and fancier atmosphere, our go-to staples are Carson's, Dudley's, Coles 735 and the Merrick Inn. Other local favorites of lower or slightly lower scale are Ramsey's Diner, DV8, Malone's/ Drake's, and Bourbon n' Toulouse. There are plenty of other options as well.
We'll be there with my kids so I might stop by if y'all are around. In case it helps, Kroger has a good sized gameday setup near the stadium with free food. Not great food (think hot dogs and a variety of sides) and usually some activities for kids.
Enjoy your time here, it's the best time of the year in Kentucky.