To buy fresh local fish, the best thing is to get up early in the morning and go to the fish market near the entry to Gustavia, across from the pharmacy, between 6am and 8:30am (Mon - Sat). The island’s fishermen sell their fish to individuals here and it doesn’t come any fresher than this. According to the season, they might have lobsters, mahi-mahi, wahoo, or tuna, which you can purchase whole or in slices or filets, with or without the skin.
In Lorient, just before the large hill that goes toward Camaruche, fishermen have a small stand where they sell lobster and fresh local fish of all sorts for a few hours after they come back from their day’s fishing. This stand is not always open and does not have set hours as it depends on when the fisherman come in.