Desktop website crawlers for a small site: FreeCrawl, Screaming Frog and alternatives
FreeCrawl is the crawler to start with: free, open source and cross-platform, with no URL cap and crawl comparisons included, narrowly ahead of Screaming Frog. Screaming Frog is the established choice with a free version for small sites; Scrawly, our Rising Star, explains each finding; SEODisias is another free newcomer, less fully documented.
FreeCrawl
Free, open-source crawler for Windows, macOS and Linux that finds broken links, redirects and indexing problems, with saved projects, crawl comparison and scheduled crawls included.
It is less than six months old and mainly one developer’s work, and its installers are not code-signed, so your computer asks you to approve the first launch.
Why FreeCrawl is our PickTeam behind FreeCrawl? Get your award badge
Scrawly
A free, open-source crawler first released on October 6, 2026, that explains each finding and how to fix it, with crawl comparison and scheduled audits listed as included.
You install it by pasting a command into Terminal or PowerShell, it updates itself from GitHub each time it opens, and it has only about five weeks of public history.
Why Scrawly earned Rising StarWho is this article for? People who own or maintain a small website, up to a few thousand URLs, and want a desktop crawler to find broken links, redirects and indexing problems, see which pages to edit and check again after changes.
About 29 min read
What Preferred Pick, Rising Star and Ready to Rock mean
The quick list
Three free crawlers, two of them open source, and one with a free version and an annual licence. Compare what each finds, what a year costs and what is left when you stop paying.
Prices for one user, checked November 12, 2026. FreeCrawl and Scrawly are free under the MIT licence, with no paid tier. SEODisias is free, and its maker plans optional paid plans for extras. Screaming Frog’s licence is $279 a year (€245); its free version stops at 500 URLs per crawl. Other crawlers we priced, such as Sitebulb Lite at $180 a year and Scrutiny at $149 once for Mac, are covered in our research notes. Taxes and checkout totals were not checked.
The comparison at a glance
Compare what each crawler finds, what it costs for one person, how repeat checks work and what remains when you stop paying.
Order follows our verdict for a small website owner, not measured speed or accuracy. All tools remain selectable.
Hides matching answers for the selected tools. Unknowns and unverified claims stay visible.
| Feature | FreeCrawl | Scrawly | Screaming Frog SEO Spider | SEODisias |
|---|---|---|---|---|
| Best fit | Preferred Pick: free, unlimited checks with saved projects, crawl comparison and scheduling | Rising Star: free audits that explain each finding and how to fix it | Also consider: an established crawler with issue priorities and a free version for small sites | A free crawler with packaged installers on Windows, macOS and Linux; newer and less fully documented |
| Price for one user, checked November 12, 2026 | Free (MIT licence); no paid tier | Free (MIT licence); no paid tier | Free up to 500 URLs per crawl; licence $279 (€245) a year | Free; optional paid plans planned for extras |
| Crawl allowance | No cap; the maker says it is built for more than 1 million URLs on one machine | No cap; you set a limit for each audit | Free: 500 URLs per crawl. Licence: limited by your memory and storage | No URL limit, per the maker |
| Broken links and the pages that contain them | Broken Links tab exports every broken link; the list of pages linking to any URL exports separately | Broken links among the internal-linking checks; each finding lists the pages affected, per the maker | Bulk export of every page linking to 4xx or 5xx URLs | Response codes, external link status and inbound links with anchor text; export of linking pages not documented |
| Comparing two checks | Compare With Project: added and removed URLs, and changes to status, title, meta description, H1 and canonical; scheduled crawls compare automatically since November 2026 | Crawl-to-crawl comparison, listed by the maker without further documentation | Licence: crawl comparison; scheduled crawls can compare automatically and email changes (since May 2026) | No crawl-to-crawl comparison documented |
| When payment stops | Nothing to pay; projects stay usable | Nothing to pay; audits stay on your computer | Returns to the free limits; saved crawls cannot be opened | Nothing to pay today; optional paid plans are planned; the free licence is revocable |
Desktop crawlers for a small website you own. Public-source research checked October 19, 2026, with prices, releases and the newer crawlers rechecked on November 12: no installs, scans, accounts or purchases by us, so no speed, accuracy or completeness results. Findings show what your server returned to the crawler, not Google’s index status or rankings. Export findings you want to keep before a licence ends. Next substantive review: January 17, 2027.
All features and sources 17 criteria
Open the groups that matter to you. They compare the same selected tools. “Planned” means future work, not an available feature. Sources include the published conditions.
Price, platforms and allowances
Show 5 criteria
| Feature | FreeCrawl | Scrawly | Screaming Frog SEO Spider | SEODisias |
|---|---|---|---|---|
| Price for one user, checked November 12, 2026 | Free (MIT licence); no paid tier | Free (MIT licence); no paid tier | Free up to 500 URLs per crawl; licence $279 (€245) a year | Free; optional paid plans planned for extras |
| Crawl allowance | No cap; the maker says it is built for more than 1 million URLs on one machine | No cap; you set a limit for each audit | Free: 500 URLs per crawl. Licence: limited by your memory and storage | No URL limit, per the maker |
| Runs on | Windows 10 or later, macOS 12 or later (Apple Silicon and Intel), Linux; installers not code-signed or notarized | macOS, Windows and Linux; installed by pasting a command into Terminal or PowerShell | Windows, macOS (Apple Silicon and Intel), Ubuntu and Fedora Linux | Windows, macOS (Apple Silicon and Intel) and Linux; macOS builds signed and notarized, Windows and Linux builds not code-signed |
| Free use or trial | Everything free; no account | Everything free; no account | Free version with no time limit; no saving, settings or rendering | Everything free; no account |
| Refunds and renewal | No purchase; MIT licence | No purchase; MIT licence; updates itself from GitHub each time it opens unless you turn this off | Full refund within 14 days; no pro-rata refunds; auto-renew off unless chosen | No purchase; free for personal and commercial use under a revocable licence |
These selected tools have matching answers in this group. Turn off “Show differences only” to see them.
Sources for price, platforms and allowances
Checked November 12, 2026: FreeCrawl README and downloads and website; Scrawly README and install and update guide; Screaming Frog pricing and feature table, FAQ and overview; SEODisias website and terms. Prices are one-user USD amounts before tax; taxes and checkout totals were not checked.
Findings and repeat checks
Show 9 criteria
| Feature | FreeCrawl | Scrawly | Screaming Frog SEO Spider | SEODisias |
|---|---|---|---|---|
| Best fit | Preferred Pick: free, unlimited checks with saved projects, crawl comparison and scheduling | Rising Star: free audits that explain each finding and how to fix it | Also consider: an established crawler with issue priorities and a free version for small sites | A free crawler with packaged installers on Windows, macOS and Linux; newer and less fully documented |
| JavaScript rendering | Included; Chromium bundled with the installer | Included (headless Chromium) | Licence only (Chromium) | Optional, per the maker |
| Broken links and the pages that contain them | Broken Links tab exports every broken link; the list of pages linking to any URL exports separately | Broken links among the internal-linking checks; each finding lists the pages affected, per the maker | Bulk export of every page linking to 4xx or 5xx URLs | Response codes, external link status and inbound links with anchor text; export of linking pages not documented |
| Redirects | Redirects tab; redirect chains in bulk export | Redirect chains and loops; temporary and permanent redirects | Redirect chains | Temporary and permanent redirects; redirect chains listed in its FAQ, per the maker |
| Canonical and robots checks | Canonicals and directives tabs; indexability for each URL; conflicting and cross-domain canonical checks | robots.txt, meta robots and X-Robots-Tag; noindex conflicts; missing, conflicting or cross-domain canonicals | Canonicals in HTML and HTTP headers; meta robots and other directives | Canonical tags that are missing or mismatched; meta robots directives; robots.txt |
| How findings are presented | More than 200 checks in 34 tabs, with severity tiers (Critical, Warning, Info); no overall score, so you decide what to fix first | 250 checks, each with why it matters and how to fix it, ranked by impact, per the maker | Issues with a type and an estimated priority, which you can change since version 25.0 | More than 30 types of issue; PDF reports |
| Saved work | Each project saved as one file you can reopen | Audits kept on your computer; saved crawl profiles | Licence only; crawls saved automatically in database storage mode | A projects dashboard and crawl history on your computer |
| Comparing two checks | Compare With Project: added and removed URLs, and changes to status, title, meta description, H1 and canonical; scheduled crawls compare automatically since November 2026 | Crawl-to-crawl comparison, listed by the maker without further documentation | Licence: crawl comparison; scheduled crawls can compare automatically and email changes (since May 2026) | No crawl-to-crawl comparison documented |
| Scheduling | Included; exports reports and notifies Slack, Discord, Teams or a webhook; a command-line mode runs without the window; whether an in-app schedule runs with the app closed is not documented | Scheduled audits with regression alerts, listed by the maker without further documentation | Licence only | Planned, per its roadmap; not available yet |
These selected tools have matching answers in this group. Turn off “Show differences only” to see them.
Sources for findings and repeat checks
FreeCrawl README, changelog and issue on priorities; Scrawly README and check list; Screaming Frog overview, quick-start guide, paid features and version 25.0 notes; SEODisias website, changelog and roadmap. These are documented features, not results from scanning a site. FreeCrawl’s README counts more than 200 checks and its notes for version 1.0.0 count 189; we use the lower figure in prose. We did not establish how scheduled jobs behave while a computer is asleep, offline or, for FreeCrawl, with the app closed. Where a cell says not documented, the maker’s website we read does not say; we did not open SEODisias’s public repository.
Retention, data and maker
Show 3 criteria
| Feature | FreeCrawl | Scrawly | Screaming Frog SEO Spider | SEODisias |
|---|---|---|---|---|
| When payment stops | Nothing to pay; projects stay usable | Nothing to pay; audits stay on your computer | Returns to the free limits; saved crawls cannot be opened | Nothing to pay today; optional paid plans are planned; the free licence is revocable |
| What the vendor collects | No telemetry, per the maker; crawls stay on your computer | No accounts or analytics, per the maker; checks GitHub for updates | Licence checks, usage statistics and crash data; support logs can include crawled sites | Crawl data stays on your device, per the maker; checks its server for updates (app version, system, processor, language, and an anonymous installation identifier in newer versions) and loads a news feed; optional account |
| Maker and latest release | Kemal Acar, open source; first release May 2026; 1.1.5, November 7, 2026; built on Electron 41, out of security support since August 23, 2026 | Ilias Sami, open source; 1.0.0, October 6, 2026, its first public release | Screaming Frog Ltd (UK, since 2010); 25.1, November 4, 2026 | A Sparkzone product (a Projekod brand; company not verified); version 0.1.9, June 27, 2026; source code not shown to us |
These selected tools have matching answers in this group. Turn off “Show differences only” to see them.
Sources for retention, data and maker
FreeCrawl README, releases, desktop package file and security advisories; Scrawly releases, privacy and update notes and security advisories; Electron release schedule; Screaming Frog expiry and licensing, privacy policy and release history; Companies House record for Screaming Frog Limited; SEODisias website, terms, privacy policy and releases repository. No privacy, credential-storage or security audit.
What matters when choosing
FreeCrawl is our Practical Pick for someone who owns a small website and wants to find broken links, redirects and indexing problems, see which pages need editing, and check again after making changes. It is free and open source under the MIT licence, runs on Windows, macOS and Linux, needs no account and has no URL cap. JavaScript rendering, saved projects, comparison between two crawls and scheduled crawls, which Screaming Frog keeps for its $279-a-year licence, are all included, and nothing locks when you stop using it. Its Broken Links tab exports every broken link, and the list of pages that link to any URL exports on its own, so you can hand an editor the pages to fix. Under the criteria we fixed before scoring, FreeCrawl scored 4.15 out of 5 and Screaming Frog 3.87. That lead is narrow: it is smaller than a one-point change on our most heavily weighted criterion, it shrinks to about a tenth of a point when price counts for less, and in that case half a point less on FreeCrawl’s findings score would put Screaming Frog first. FreeCrawl still leads when price counts for more. FreeCrawl README, changelog, releases
The compromises come with its age. FreeCrawl’s first release was in May 2026 and version 1.0 in September, and it is mainly one developer’s work. Its changelog shows fixes to core results, such as robots.txt, canonical and indexability checks, as recently as November 5, so check a sample of findings by hand before changing many pages. Its installers are not code-signed, so Windows and macOS warn you and ask you to approve the first launch, and it is built on Electron 41, a version of its application framework whose security support ended on August 23, 2026. Disclosure: Practical Picks is researched and written with Claude, an AI model made by Anthropic. FreeCrawl, Scrawly, Screaming Frog and Sitebulb all advertise connections to Claude, through MCP servers or built-in AI features, and SEODisias’s website quotes users who pass its reports to Claude. We gave AI features no weight in this comparison, and we have no relationship with Anthropic or with any of these makers. Electron release schedule, Sitebulb plans
Scrawly is our Rising Star for someone who wants each finding explained, at no cost. Its first public release, version 1.0.0, came on October 6, 2026. Its documentation lists 250 checks, including broken links, redirect chains, robots directives and canonicals, and says each finding comes with why it matters, how to fix it and the pages affected; crawl comparison and scheduled audits are listed as included. It scored 3.73, third overall at our usual weighting, 0.13 points ahead of SEODisias, a narrow lead that rests on two judgments: credit for crawl comparison and scheduled audits listed in Scrawly’s README, which we could not check, and half a point off SEODisias for recent fixes to its core results. The limits are real: you install it by pasting a command into Terminal or PowerShell, it updates its own code from GitHub each time it opens unless you turn that off, and its public record is about five weeks long. Scrawly README, install and update guide
Consider Screaming Frog SEO Spider if you would rather rely on an established product with issue priorities. It comes from a UK company trading since 2010 and runs on Windows, macOS and Linux. Its free version, with no time limit, crawls up to 500 URLs and reports broken links, server errors, redirects and robots directives, and its bulk export lists every page that links to a URL returning an error. A licence costs $279 a year per user (€245 on the same page) and adds saved crawls, JavaScript rendering, scheduling and comparison between two crawls; a scheduled crawl can compare itself with the previous one and email you what changed. Version 25.0, released November 3, 2026, lets you change each issue’s type and priority. When the licence ends, the application returns to the free limits and cannot open the crawls you saved, so export the findings you want to keep before it lapses. It scored 3.87, second overall at our usual weighting. Pricing and free-versus-paid table, licensing FAQ, version 25.0 notes
Consider SEODisias if you want a free crawler with a packaged installer on Windows, macOS or Linux. Its website lists more than 30 checks, including broken links, redirects and redirect chains, canonical tags and robots directives, with no URL cap and no account. It is a Sparkzone product, and Sparkzone is a Projekod brand; we have not verified the company. It scored 3.60, fourth, because it documents no way to compare two crawls, lists scheduled crawls as planned, does not say how to export the pages that link to each broken URL, and its changelog shows recent fixes to core results. SEODisias website, changelog, roadmap
These are researched judgments from the vendors’ documentation, prices, licences and policies and from public reports, checked October 19, 2026, rechecked on November 12 for prices, releases and the newer crawlers, and on October 20 for FreeCrawl and Scrawly. We did not install any crawler or scan a site, so we make no claim about speed, completeness or accuracy. A crawler reports what your server returned to it at that moment; it cannot tell you whether Google has indexed a page or how it ranks, and Google says even its own live test cannot guarantee indexing. Treat warnings as questions to check: Screaming Frog describes its issue priority as an estimate, and a redirect can be deliberate. Three conditions would change our recommendation: if you cannot approve unsigned software on your computer, or want a long track record behind the results, choose Screaming Frog; a Mac user who wants to pay once and does not need to compare checks can look at Scrutiny, $149 once, which scored below the free crawlers; and someone who checks now and then and wants each finding explained by an established product can pay for one month of Sitebulb Lite at $18. Google URL Inspection help
FreeCrawl
Preferred Pick · free, unlimited checks on a small site
Choose FreeCrawl to find broken links, redirects and indexing problems for free, with saved projects, crawl comparison and scheduled crawls included. Approve its unsigned installer on the first launch, and check a sample of findings by hand before fixing many pages.
Reasons to choose it
- Free with no URL cap and no account
- Saved projects, crawl comparison and scheduled crawls at no cost
- Windows, macOS and Linux; JavaScript rendering included
What to watch
- First released in May 2026; mainly one developer’s work
- Installers are not code-signed; the first launch needs your approval
- Built on Electron 41, out of security support since August 23, 2026
FreeCrawl is a free, open-source desktop crawler, released under the MIT licence by developer Kemal Acar. Its first GitHub release came on May 25, 2026, version 1.0.0, which the maker calls the full release, on September 21, and 1.1.5 on November 7. Installers are offered for Windows 10 or later, macOS 12 or later on Apple Silicon and Intel, and Linux (AppImage, .deb and .rpm). There is no paid tier, no account and no URL cap; the maker says it is built for more than a million URLs on one machine. Its README counts more than 200 issue checks in 34 tabs, and its own notes for version 1.0.0 count 189 (we use the lower figure). For this job it documents a Broken Links tab that exports every broken link, a list of the pages linking to any URL that exports on its own, response codes and redirects, redirect and canonical chains in its bulk export, canonical and directive checks, an indexability status for each URL, and a soft-404 check. Pages behind bot protection are marked as blocked rather than broken. JavaScript rendering uses a Chromium browser bundled with the installer. README, changelog, releases
Repeat checks cost nothing. Each project saves as one file you can reopen. Compare With Project lists the URLs added and removed between two crawls and the changes to status, title, meta description, H1 and canonical. Since version 1.1.4 on November 5, scheduled and command-line crawls also compare themselves with the previous crawl, and a summary can go to Slack, Discord, Microsoft Teams or a webhook; the README documents a command-line mode with GitHub Actions and GitLab examples that crawl on a schedule. We could not establish whether a schedule set inside the app runs while the app is closed. Issues carry severity tiers (Critical, Warning and Info), but there is no overall score: when a user asked in September for clearer priorities, the maintainer replied that the tool is designed for “user-led evaluation and prioritization”. Changelog, issue on priorities
Read the full FreeCrawl analysis and sources
The limits come with a young project. It is mainly one developer’s work: four contributors are listed, and the release notes credit the others with a handful of changes. Its issue tracker and changelog show fixes to core results through the summer and autumn: noindex pages reported as indexable (fixed in August), false errors on redirected pages (August), relative canonicals and robots.txt rules read wrongly (August 20 to 23), links to pages that time out not counted as broken (October 29) and robots.txt rules not applied while the file was still loading (November 5). Check a sample of findings by hand before acting on many. Issues, version 1.1.4
Its installers are not code-signed or notarized. Windows SmartScreen and macOS Gatekeeper warn on the first launch; on macOS 15 or later you approve it under System Settings, Privacy & Security, and the README’s fix for a “damaged” warning is a Terminal command. On Ubuntu 24.04 and later, where the system blocks it, the AppImage runs its Chromium without the sandbox; the .deb and .rpm packages keep it. The app is built on Electron 41, whose security support ended on August 23, 2026, according to Electron’s schedule, and it had not moved to a supported version by November 12. If you crawl pages behind a login, use version 1.1.5 or later: version 1.1.3 on November 1 stopped web-login details and custom headers being sent to other sites during a crawl, and 1.1.5 on November 7 stopped saved proxy passwords and login values being stored unencrypted. GitHub listed no security advisories for the project on November 12. README, desktop package file, Electron release schedule, security advisories
The maker says FreeCrawl keeps no telemetry and that crawls stay on your computer; connections to Google Search Console, Analytics or an AI provider are optional and use your own accounts. Its source is public, so these statements can be checked, but we have not audited them. The app’s network settings and its README carry a sponsor’s card for a proxy service. Its website compares it with Screaming Frog and describes Screaming Frog’s free version as feature-complete up to 500 URLs, which Screaming Frog’s own feature table contradicts, so we used each maker’s documentation only for its own product. Choose Screaming Frog if you want an established product with a long record, or Scrawly if you want each finding explained. Website, README
Used FreeCrawl? Share your experience to help other readers choose.
Scrawly
Rising Star · a newcomer worth considering
Why Rising Star? First released on October 6, 2026, Scrawly is the highest-scoring newer crawler after our Pick, 0.13 points ahead of SEODisias, a narrow lead resting on credit for comparison and scheduling listed in its README, which we could not check, and half a point off SEODisias for recent core-result fixes: free, with no page cap, and it explains each finding and how to fix it. A terminal install, code that updates itself from GitHub and a public record of about five weeks are the limits. This is not our broader Ready to Rock endorsement.
Consider Scrawly if you want each finding explained with how to fix it, at no cost, and you are comfortable installing it with one Terminal or PowerShell command. It is about five weeks old, so treat its findings as questions to check.
Reasons to choose it
- Free with no page cap and no account
- Each finding explained, with the pages affected, per the maker
- Crawl comparison and scheduled audits listed as included
What to watch
- Installed by a Terminal or PowerShell command; no signed installer
- Updates itself from GitHub each time it opens unless you turn this off
- First public release October 6, 2026; one developer, nine commits
Scrawly is a free, open-source desktop crawler, released under the MIT licence by SEO consultant Ilias Sami. Version 1.0.0, its first public release, came on October 6, 2026, and it was still the only release on November 12. It runs on macOS, Windows and Linux with no account and no page cap. Its README lists 250 checks in 23 areas. For this job they include 4xx and 5xx pages, redirect chains and loops, robots.txt, meta robots and X-Robots-Tag rules, noindex conflicts, missing, conflicting or cross-domain canonicals, and broken links among its internal-linking checks. It says every finding comes with why it matters, how to fix it and the exact pages affected, ranked by impact, and that reports export to PDF, Excel or CSV. JavaScript rendering is included. These are the maker’s descriptions. README and check list, changelog
Choose it if you want plain explanations of each finding without paying, and can follow a short installation in Terminal or PowerShell. It scored 3.73 under our criteria, third overall at our usual weighting, behind FreeCrawl and Screaming Frog and ahead of SEODisias, because it is free and documents the whole job. Crawl-to-crawl comparison and scheduled audits with alerts are listed as included, but only in one line of the README, with no documentation of how they work, so we scored repeat checks lower than FreeCrawl’s. README
Read the full Scrawly analysis and sources
Installation is the main hurdle. You paste one command into Terminal on a Mac or Linux, or PowerShell on Windows, which downloads a script from GitHub and runs it. The script installs git and Python 3.11 or later where it can, creates a private Python environment, downloads a crawler browser of about 150 MB and adds Scrawly to your apps. There is no signed installer. Each time Scrawly opens, it checks GitHub and updates itself; the maker says only versions whose automated tests passed are installed, and you can turn this off with a setting. Install and update guide
Its record is short: one developer, nine commits and one issue on GitHub, the last code change on October 10, and no published security advisories on November 12, 2026. The maker says Scrawly has no accounts and no analytics and that audits stay on your computer; it connects to the sites you audit, to GitHub for updates and to services you set up yourself. An optional WordPress plugin can change titles, meta descriptions, canonicals and redirects on your live site after you confirm, keeping a snapshot to undo each change. Choose FreeCrawl if you want a packaged installer and a documented comparison, or Screaming Frog for an established product. README, security advisories, releases
Used Scrawly? Share your experience to help other readers choose.
Screaming Frog SEO Spider
Also consider · an established crawler with a free version for small sites
Consider Screaming Frog if you would rather rely on an established product with issue priorities. Start free on a site under 500 URLs; buy the licence when you need saved crawls, rendering or comparisons, and export findings before it lapses.
Reasons to choose it
- Free version with no time limit, up to 500 URLs per crawl
- Bulk export of every page that links to an error URL
- Established since 2010; scheduled crawls can compare and email changes
What to watch
- Saving, rendering, scheduling and comparison need the $279 annual licence
- Saved crawls cannot be opened after the licence expires
- The free allowance counts URLs, not pages
Screaming Frog SEO Spider is a desktop crawler from Screaming Frog Ltd, a UK company incorporated in 2010. It runs on Windows, on macOS for Apple Silicon and Intel, and on Ubuntu and Fedora Linux. It documents broken links, server errors and redirect chains, page titles and descriptions, meta robots and other directives, canonicals in HTML and HTTP headers, and XML sitemap generation. Each issue gets a type and an estimated priority, and since version 25.0 you can change both for each crawl. For fixing links, the bulk export does the work: it lists every page that links to URLs returning 4xx or 5xx responses, so you can hand an editor a list of pages and links rather than a list of errors. Overview, quick-start guide, Companies House record
We verified that the free version has no time limit but stops at 500 URLs per crawl and cannot save crawls, change the crawl settings, render JavaScript, schedule crawls or compare them (feature table checked October 19, price November 12). That is enough for a one-off check of a small site whose links are in the HTML the server sends. A licence, $279 a year per user (€245 on the same page), removes the URL cap, subject to your computer’s memory and storage, and unlocks those features. Licences do not renew automatically unless you choose auto-renew, and discounts start at five licences. Pricing, FAQ
Read the full Screaming Frog SEO Spider analysis and sources
With a licence, crawls are saved automatically in the default database storage mode and can be reopened later. Crawl comparison shows what changed between two crawls. A scheduled crawl can compare the last two crawls in a project automatically and put the changes in the notification email (since version 24.0 in May 2026). Version 25.0, released November 3, 2026 with a bug-fix release 25.1 the next day, added a custom dashboard tab, an auto-throttle that slows requests when a site pushes back, and tooling for its MCP server. Release history, version 25.0 notes, FAQ
The licensed version can render pages in Chromium, which matters when links or content are added by scripts. Rendering in a crawler shows what that crawler saw; it does not reproduce every part of Google’s processing. If your site is plain HTML, you may not need rendering at all, and the free version covers the same link checks. Overview
The licence is annual. When it expires, the application returns to the free version: the settings are locked, crawls stop at 500 URLs and saved crawls cannot be opened until you buy a new licence. Export the findings you want to keep as spreadsheets before then. Each licence is for one person, who may install it on several of their own devices; a team of five needs five licences. Screaming Frog offers a full refund within 14 days of purchase, but no pro-rata refund for unused time and no tax refund. Renewal costs the same as a new licence, and existing users are not kept on old prices after an increase. FAQ
Its privacy policy, last updated March 16, 2026, says the software may check your username, licence key, IP address and an anonymised machine ID against its records, and may gather statistics about your system, settings and use, plus crash data. Logs you send to support can include details of sites you crawled. Connections you choose to third-party services, such as Google Search Console, involve separate data transfers. Choose FreeCrawl if you want saved crawls, comparisons and rendering without a licence, or Scrawly if you want each finding explained. Privacy policy
Used Screaming Frog SEO Spider? Share your experience to help other readers choose.
SEODisias
A free crawler with packaged installers, newer and less fully documented
Consider SEODisias, a Sparkzone product (a Projekod brand; company not verified), if you want a free crawler with installers for Windows, macOS and Linux and are happy to check what it finds. It documents no way to compare two crawls, lists scheduled crawls as planned, and does not say how to export the pages that link to each broken URL.
Reasons to choose it
- Free with no URL limit and no account, per the maker
- Packaged installers for Windows, macOS and Linux; macOS builds notarized
- Covers broken links, redirects and redirect chains, canonical tags and robots directives
What to watch
- Version 0.1.9; no crawl comparison, scheduling planned, linking-page export not documented
- Maker is a Sparkzone product (a Projekod brand), not verified; the free licence is revocable
- Windows and Linux builds are not code-signed
SEODisias is a free desktop crawler for Windows, macOS (Apple Silicon and Intel) and Linux. Its website says everything that crawls and audits a site is free and will stay free, with no URL limit, no account and no time limit, and that optional paid plans for extras such as log file analysis, Search Console integration and team collaboration are planned. It lists more than 30 types of issue. For this job they include response codes and 4xx and 5xx errors, temporary and permanent redirects with redirect chains, the status of every external link, inbound links with their anchor text, canonical tags that are missing or mismatched, meta robots directives, the robots.txt file, an XML sitemap check and optional JavaScript rendering. Reports export as PDF, and a Markdown export groups issues by severity. These are the maker’s descriptions. Website, changelog
It is a newcomer, and a pre-1.0 one. Its releases repository on GitHub was started on February 15, 2026 and holds ten releases, the latest version 0.1.9 on June 27, 2026; its changelog gives earlier dates, from 2025, that conflict with the repository, and either way it is recent. The macOS builds are signed and notarized by Apple; the Windows and Linux builds are not code-signed yet, and the maker asks you to check them against published checksums. The changelog records fixes to core results: false 404 errors on excluded paths (March), false “excessively deep URL” errors and pages lost under heavy load (June), and empty reports after an update (June). It keeps a history of your crawls, but it documents no way to compare two crawls and its roadmap lists scheduled crawls as planned, so we scored repeat checks lower than FreeCrawl’s and Scrawly’s. Its website also does not say how to export the pages that link to each broken URL. Releases repository, changelog, roadmap
Read the full SEODisias analysis and sources
SEODisias says it is a Sparkzone product and that Sparkzone is a Projekod brand; its terms and privacy policy name Projekod Yazılım ve Bilişim Hizmetleri Ali Gündoğdu, a sole proprietorship in Türkiye, as the operator, which we have not verified. The maker says crawl data and reports stay on your device. The app asks the maker’s server about updates, sending its version, your operating system, processor type and language (newer versions add an anonymous installation identifier), and loads a news feed; an account is optional. The free licence is described as revocable. The maker’s GitHub organisation lists one public repository that we did not open, and its terms reserve the source code and bar reverse engineering, so we could not check these statements against code. We found no independent review. Choose FreeCrawl if you want a documented comparison and public source code, Scrawly if you want each finding explained, or Screaming Frog for an established product. Terms, privacy policy, about page
Used SEODisias? Share your experience to help other readers choose.
Which product are we missing?
Know a product that belongs in this article? Tell us what it does differently and why it deserves a closer look. Newcomers welcome.
Readers and product teams can suggest candidates. Suggestions inform our research; they do not guarantee inclusion or a recommendation.
Opens an email draft with this article attached as a link. Nothing is sent until you send it.
No email app? Send the product name, website, and this article’s title to admin@practicalpicks.guide.
What people are saying
Public discussion is a check on vendor claims, not a measured reliability score. A February 2025 Reddit discussion about whether Screaming Frog is still the recommended crawler was mostly positive; one participant warned against reading the free allowance as 500 HTML pages, which matches the vendor’s own unit of URLs. The Sitebulb complaint we found, from January 2023, gave too little detail to establish a current defect, and replies asked for specifics. We read both threads on September 6, 2026; Reddit blocked our automated recheck on October 19. Scrutiny’s MacUpdate page showed version 12.13.1 and an average of 4.3 from 29 user ratings on October 19, 2026; its vendor’s $149 price differs from MacUpdate’s listed price, and we use the vendor’s. Screaming Frog discussion, Sitebulb discussion, MacUpdate page
FreeCrawl’s GitHub issue tracker listed 22 issues on November 12, 2026, all closed. Several concerned core results, such as noindex pages reported as indexable and false errors on redirected pages, both fixed in August. One user wrote in September that it was hard to tell which issues mattered; the maker replied that the tool leaves prioritization to the user. Scrawly’s tracker had one issue, an offer of a case study. SEODisias’s website shows three testimonials chosen by its maker. We found no independent review of any of these three tools. FreeCrawl issues, Scrawly issues, SEODisias website
Our October 19 search for vulnerabilities, breaches, lawsuits, acquisitions and outages involving Screaming Frog, Sitebulb and Scrutiny found nothing that changed our conclusions. The US National Vulnerability Database returned no records for any of the three names, and Companies House lists both UK companies as active with current filings. On October 20 we searched for FreeCrawl and Scrawly in the same way and found only directory listings; on November 12 GitHub still listed no security advisories for either. A search that finds nothing is not proof that nothing exists.
How we researched
This is public-source research. We did not download, install or run any crawler, scan a website, create an account, start a trial or buy a licence. No vendor supplied access, payment or input, and we use no affiliate links; Scrutiny’s maker runs an affiliate program that we are not part of. We first compared Screaming Frog, Sitebulb and Scrutiny on September 6, 2026 and rechecked every material claim on October 19, 2026: prices, plan limits, platforms, licence and refund terms, what happens when payment stops, privacy policies, release records and company filings. On October 20 we read FreeCrawl’s and Scrawly’s repositories, documentation, release records, changelogs, issue trackers and security advisory pages. On November 12 we rechecked prices for every crawler we scored, read FreeCrawl’s and Scrawly’s releases, issues and advisory pages again, read Screaming Frog’s release history and Companies House record, and read SEODisias’s website, changelog, roadmap, terms, privacy policy and release pages. Screaming Frog’s free and paid features come from the feature table on its pricing page. Sitebulb’s pricing page loads its yearly prices with a script that did not display them on November 12, so we use the yearly figures we read from its price data on October 19, which a 2026 roundup also reports.
We wrote this for someone who owns or maintains a small website, up to a few thousand URLs, and wants to find broken links, redirects and indexing problems, see which pages to edit and check again after changes. We fixed five criteria and their weights before scoring: findings for that job (30%), cost (20%), repeat checks and what remains after payment stops (20%), platforms and access (15%), and data, terms and continuity (15%). We scored cost for the same job at every product: a 3,000-URL site, one user, saved work and exports, over three years, against the typical price of the crawlers we priced. FreeCrawl scored 4.15, Screaming Frog 3.87, Scrawly 3.73, SEODisias 3.60, Scrutiny 3.42, WebSite Auditor 3.39, Sitebulb Desktop (Lite plan, $180 a year) 3.34 and Netpeak Spider 3.25. FreeCrawl stayed first when we made price count for less or for more, and Screaming Frog stayed second except when price counted for more, where Scrawly passed it. The gap between the top two is narrow, and it closes if FreeCrawl’s findings score is half a point lower and price counts for less, so the order of the top two rests on judgments we state here rather than on a clear margin. FreeCrawl lost half a point on data and continuity because Electron 41 is out of security support; we asked the same runtime question of every product, and Screaming Frog’s Java runtime is current, while the browser versions inside the others are not published. We did not compare hosted crawlers such as Sitebulb Cloud, rank trackers or Google Search Console. AI features, which FreeCrawl, Scrawly, Screaming Frog, Sitebulb and SEODisias all advertise, carried no weight.
Sitebulb, Scrutiny, WebSite Auditor and Netpeak Spider scored below the four crawlers above, so they do not have their own sections. Sitebulb Desktop explains findings in plain language and runs on Windows and Mac; its Lite plan, the one we priced, has no audit comparison or scheduling, and its Pro plan costs $425 a year. Scrutiny runs on Mac only, costs $149 once, keeps working after payment stops and runs scheduled scans with the app closed, but documents no way to compare two scans. WebSite Auditor, from SEO PowerSuite, crawls free up to 500 URLs; its pricing page puts data export in the $359-a-year Enterprise plan, while its product page says Professional users can export, and we priced the less favourable one. Netpeak Spider costs $193 a year or $20 a month for Windows and macOS. The scores for WebSite Auditor and Netpeak Spider come from the vendors’ pages and are low-confidence. We could not read the websites of Visual SEO Studio or Beam Us Up, so we did not score them. Sitebulb plans, Scrutiny, SEO PowerSuite pricing, Netpeak Spider plans
We also search for new entrants. On November 12, 2026 we repeated that search through recent alternatives lists, Hacker News, GitHub and developer posts, and re-tested the crawlers we had screened before. Scrawly, first released on October 6, 2026, still meets every condition and is our Rising Star. SEODisias, whose releases repository began in February 2026, also meets them and ranked just behind Scrawly. Scrawly’s code sits in a folder named sentinelseo, possibly an earlier working name; a Hacker News search found no earlier post under that name. The others failed a condition: crawler.sh has no Windows desktop app, Scouter and LibreCrawl run as self-hosted web applications, SiteOne Crawler’s desktop app has had no release since August 2024, InterroBot has been available since 2018, LinkChecker is long established and checks links only, RustySEO’s repository was no longer available, and CrawlyCat has no packaged release. Crawly and crawlie have no Windows desktop app. Xenu’s Link Sleuth runs only on Windows, and its current version dates from 2010.
| Tool | Public sources | What remains uncertain |
|---|---|---|
| FreeCrawl (Preferred Pick) | Repository and README, releases, changelog, issue tracker, security advisories, desktop and core package files, website, Electron release schedule. | Accuracy on your site, whether scheduled crawls set inside the app run with the app closed, the version of the bundled Chromium, continuity with one main developer. |
| Scrawly (Rising Star) | README and check list, install and update guide, changelog, releases, issue tracker, security advisories. | How comparison and scheduling work, accuracy on your site, runtime versions, continuity with one developer and a record of about five weeks. |
| Screaming Frog SEO Spider | Pricing page and feature table, FAQ, quick-start guide, overview, release history, version 25.0 notes, privacy policy, Companies House record, a 2025 user discussion. | Accuracy and completeness of crawls on your site, exactly which resources count toward the 500-URL allowance, minimum operating system versions. |
| SEODisias | Website, frequently asked questions, changelog, roadmap, about page, terms, privacy policy, GitHub organisation and releases pages. | The company behind it, the contents of its public repository, whether it will add crawl comparison and scheduling, whether it exports linking pages, accuracy on your site, continuity of a pre-1.0 product. |
None of this establishes crawl speed, accuracy or security on your site. Next substantive review is due January 17, 2027, or sooner after a price change or major release. Found a changed price or feature? Contact the editors.
Sources and evidence
Sources were checked October 19, 2026 unless another date is given; FreeCrawl, Scrawly and Electron sources were checked October 20 and again on November 12, 2026; prices, Screaming Frog’s release history and Companies House record, and SEODisias, were read on November 12, 2026. Vendor material establishes documented features, prices and terms, not behavior on your site. Community posts are attributed opinions, not representative samples.
- FreeCrawl: repository and README; releases; changelog; issues, including the request for priorities, September 27, 2026; security advisories; desktop package file; website.
- Scrawly: repository, README and check list; install, update and data guide; changelog; releases; issues; security advisories.
- SEODisias: website, download table and frequently asked questions; changelog; roadmap; about page; terms; privacy policy; GitHub organisation; releases repository.
- Runtime support: Electron release schedule.
- Screaming Frog: overview; pricing and free-versus-paid feature table; FAQ, including licensing, expiry, refunds and storage mode; quick-start guide; release history; version 25.0 notes, November 3, 2026; privacy policy, updated March 16, 2026; Companies House record for Screaming Frog Limited.
- Other crawlers we scored but did not feature: Sitebulb: Desktop pricing; FAQ; product and reports; audit comparison; licensing and network troubleshooting. Scrutiny: product page; specification; version history; licence agreement, version 1.7, December 18, 2020. SEO PowerSuite: pricing; WebSite Auditor. Netpeak: plans; Netpeak Spider.
- Search-status boundary: Google URL Inspection help.
- Community evidence: Screaming Frog Reddit discussion, February 2025 and Sitebulb Reddit discussion, January 2023, both last read September 6, 2026; Scrutiny on MacUpdate. Vulnerability searches used the National Vulnerability Database.