This change allows more complex scope boundaries, including loosening edges a bit to include related resources of HTML pages (which makes for more complete archives if desired).
The queuing code now performs proper lease accounting, and it will not return a URL twice if the page load is slow.