For a more local experience for fresh fish, one could always buy from the locals at the corner in the French Quarter next to the fresh produce stands there.

As far as HH, I fully support Seth and the Dinghy Dock crew. For a nice view at sunset you could also try Chesterfields for some drinks too.