- 12 Jul, 2021 2 commits
-
-
renovate authored
-
- 11 Jul, 2021 1 commit
-
-
renovate authored
-
- 19 Jun, 2021 10 commits
- 26 Aug, 2020 2 commits
- 24 Aug, 2020 2 commits
- 23 Aug, 2020 4 commits
- 20 Aug, 2020 1 commit
-
-
ale authored
-
- 30 Jul, 2020 2 commits
- 17 Feb, 2020 2 commits
- 04 Dec, 2019 1 commit
-
-
ale authored
-
- 13 Nov, 2019 2 commits
- 07 Oct, 2019 2 commits
- 26 Sep, 2019 3 commits
- 20 Jan, 2019 1 commit
-
-
ale authored
Introduce an interface to decouple the Enqueue functionality from the Crawler implementation.
-
- 19 Jan, 2019 1 commit
-
-
ale authored
The whole URLInfo structure, while neat, is unused except for the purpose of verifying if we have already seen a specific URL. The presence check is also now limited to Enqueue().
-
- 02 Jan, 2019 1 commit
-
-
ale authored
The output stage can now write to size-limited, rotating WARC files using a user-specified pattern, so that output files are always unique.
-
- 28 Dec, 2018 1 commit
-
-
ale authored
-
- 27 Dec, 2018 2 commits