We spend time there in November - December as well as some time in January-February. The time frame mentioned of 4 pm to 7 pm is correct. Most days traffic is heavy at that time. However it's not always backed up. If we are planing to head out for dinner we check on a couple of the live Web cameras to see if things are contested. The Island Camera that shows the causeway round about will tell you if things are moving or not. It can change quickly.
We stay on Simpson Bay Beach so if traffic is too bad we'll just walk to one of the many great restaurants nearby.