Yes, a possibility would be to get a driver to take you there and bring you back. Cars with drivers / chauffeurs are called NCC in Italian, so if you look for "NCC in Florence" you'll find many options. I would suggest you contact several to ask for availability and price quotes for the dinner out.
If you're planning of going at lunch time, you could take the bus -- the 365 line takes you to Panzano (not all buses do, so check carefully).
Here's the bus schedule: http://www.acvbus.it/orari/365.pdf
Share and Bookmark!