How to Export All URLs from Ahrefs or Search Console (and Clean Them)
Every SEO tool exports beautifully and cleans up nothing. Pull a list of URLs from Ahrefs, Semrush, or Search Console and you get exactly what the tool stored: duplicates, full URLs when you wanted domains, tracking parameters, mixed protocols, and the occasional stray row. Before you can analyze that list, or open it, or compare it to another, it needs cleaning.
This guide covers both halves of the job: getting the export out of the tool in the first place, then turning that raw file into a clean, analysis-ready list using free browser-based tools. The cleaning takes under a minute once you know the order.
How to Export All URLs from Ahrefs
There is no single “all URLs” button, because “all URLs” means three different things depending on what you are doing. Pick the report that matches your question:
- Every page on the site. Run a crawl in Site Audit, then export from the page listing. This is the closest thing to a true inventory, because it is built from an actual crawl rather than from link data.
- Pages that have backlinks. In Site Explorer, open the Pages reports (Best by links / Top pages) and export. This lists pages Ahrefs knows about because something links to them, so orphaned and unlinked pages will be missing.
- Pages linking to you. In Site Explorer, open Backlinks and export. This gives referring URLs, not your own pages, which is the list you want for disavow work.
All of them export as CSV, and the number of rows you can export depends on your plan, so on large sites check whether the file is complete before treating it as one. Ahrefs’ own help documentation is the reliable reference here, since report names move around between interface updates.
One thing worth knowing: none of these reports is genuinely exhaustive. Link-based reports only contain what Ahrefs has discovered links to. If you need every URL that actually exists, combine an export with the site’s own sitemap — the methods in how to get all page URLs from a website cover that in full.
Exporting from Google Search Console
For a site you own, Search Console is often the better source. Open Indexing > Pages to export indexed and excluded URLs with the reason for each, or use Performance with the Pages dimension to export URLs that have actually earned impressions. The interface export caps at 1,000 rows; beyond that, use the Search Console API.
The Four Cleaning Passes
A messy export usually needs some combination of four passes. Run only the ones your list needs:
- Extract — get the URLs out of a report or HTML blob into a clean column.
- Strip tracking — remove UTM and click-ID parameters.
- Trim — reduce to root domains when you care about sites, not pages.
- Dedupe — collapse the inevitable repeats into a unique list.
Order matters a little: extract first, then strip/trim, then dedupe last (so it catches the duplicates that trimming and stripping create).
Pass 1: Extract the URLs
If your export is a clean CSV, just copy the URL column, it pastes as one URL per line, ready to go.
If the URLs are embedded in a report, a PDF copy-paste, or a block of HTML, they won’t come out as a tidy column. Paste the whole blob into a URL extractor and it isolates every valid link, dropping the surrounding text. Now you have a real list.
Pass 2: Strip Tracking Parameters
Exports of landing pages, campaign URLs, or referral data are often littered with ?utm_source=…, fbclid, and gclid. Those tags fragment your data, the same page shows up as several “different” URLs, and add noise to any analysis.
Remove the UTM and tracking parameters in one pass. The page’s real address stays intact; only the tracking junk is removed. This single step often reveals that your “200 unique landing pages” are really 60.
Pass 3: Trim to Root Domain (When You Want Sites)
For backlink analysis, disavow prep, or any “how many sites are here?” question, you want domains, not pages. Trim the list to root domain to strip the protocol, path, and query from every line at once. https://blog.example.com/post?ref=1 becomes example.com.
Skip this pass when you genuinely need page-level URLs (a content audit, a redirect check). It’s only for when the site is the unit of analysis.
Pass 4: Deduplicate
This is the last pass, because the earlier ones create new duplicates: stripping parameters and trimming to domain both collapse once-distinct URLs into identical lines. Remove the duplicates to get your final, unique list, and an accurate count you can actually trust.
If you want the result in a predictable order before you review it, sort the URLs as a final step so related paths sit together.
Putting It Together: Two Common Recipes
Cleaning a backlink export for domain-level analysis:
Extract (if needed) → Trim to root domain → Dedupe. Result: a clean list of unique referring domains.
Cleaning a landing-page / campaign export for reporting:
Extract (if needed) → Strip UTM parameters → Dedupe. Result: real page URLs with the tracking noise gone and repeats collapsed.
In both cases, once the list is clean you can open it in bulk to spot-check the pages, or feed it straight into your analysis.
Why Bother Cleaning First?
A dirty list quietly corrupts everything downstream:
- Counts are wrong. Duplicates and parameter variants inflate every total.
- Analysis is skewed. The same domain weighted five times distorts your view of a backlink profile.
- Time is wasted. Opening or checking duplicate URLs means doing the same review twice.
Sixty seconds of cleaning prevents an hour of chasing numbers that were never real.
Frequently Asked Questions
How do I export all URLs from Ahrefs?
Use Site Audit and export the crawled page listing if you want every page on a site, or Site Explorer’s Pages reports if you want pages that have backlinks. For links pointing at you rather than your own pages, export from Backlinks. All three export as CSV, and row limits depend on your plan.
Why does my export have the same URL multiple times?
Tools store a row per data point, a backlink export lists every linking page, a crawl lists every internal reference. The same destination naturally repeats. Deduping collapses them.
Should I dedupe before or after trimming?
After. Trimming to root domain turns many distinct page URLs into identical domain lines, so dedupe last to catch them all.
Are these tools safe for client exports?
Yes. Extract, trim, dedupe, and the UTM remover all run locally in your browser, nothing is uploaded or stored — so client URLs stay private.
Do I need to install anything?
No. Every step here is a free, web-based tool. Paste, click, copy.
Clean your next export in under a minute. Start with the extract, trim, and dedupe tools, all free, no login, nothing stored.