I think we'll see Dorry and Broggy again. It's possible all bounties were on the lower end of the scale in their time, all the current powerful figures in the One Piece world seem to have been born later on with figures like Gold Roger and the Yonko shifting the paradigm. Kinda like inflation.
Both of them had weakened each other in the Little Garden arc, which was why a scrub like Mr. 3 could subdue them. I think they'd be pretty damn strong in their prime if they return.