I know OP has already completed his trip, but for those looking for a great Key West restaurant I recommend Latitudes on Sunset Key.
It’s just off of Key West near Mallory Square and you’ll need to take a water taxi to get there. Advance reservations are an absolute must!