Turns out that you can just bruteforce that in a day or two. And the results are pretty good.

One drawback is that depending on if you're looking at an older website, a lot of the links are dead. The deduplication isn't great either.