I've made many trips there, all unguided and planned personally. Frankly, I'm not a strong cyclist. Sure, I want to climb the Tourmalet, but then I want a duck confit and a glass of rose at the top. I'm not some savage. I have hotel and route recommendations if you want.
I have history with Ventoux. It was the first climb I did in France, first trip, in 2011. Was very concerned my Northern California mountain training was insufficient, so we did Ventoux from Sault.
For a base of operations, the amazing Hotel Les Florets in Gigondas would do nicely. The restaurant is reason enough to visit - I recall having a foie gras three ways set menu for 50EUR in 2011. Accommodations are not luxurious, but the location makes up for it. Gorgeous courtyard in which you can have a bottle of rose and house-cured olives while entertaining the many friendly stray cats pre-dinner. You look out at the Montmiral hills, with the amazing Chateau St. Cosme at the base of the hotel.