The Dormant URL Crawl Loop – Shawn Huber Presenting (Triangle Startup Collective)
The Dormant URL Crawl Loop: A Programmatic SEO Framework for Scaling Sites with Millions of Pages
One mechanism took a legal data platform from roughly 30,000 to nearly 4 million monthly organic clicks. Shawn Huber found it by watching what happens when a dormant, unindexed page suddenly gets a spike in human traffic, often from an external news event. Googlebot follows within 24 to 48 hours to recrawl the page.
That's a predictable, exploitable signal. Shawn built a framework around it: detect these traffic-spike crawl windows as they open, then inject internal links while Googlebot is paying attention, routing crawl attention toward the orphaned, deep-tail inventory that would otherwise never get discovered.
Every large programmatic site hits the same wall eventually. Legal, real estate, e-commerce, job boards, travel. Millions of deep-tail pages and no realistic way to link them all into relevance by hand. If your site has more pages than internal links, this framework was built for exactly that problem.
Speaker
Shawn Huber has spent nearly two decades in technical SEO, driving organic growth for brands like T-Mobile, Alaska Airlines, and Trellis Law. He was a founding member of T-Mobile's in-house SEO program, where his findings were significant enough that the company built a dedicated team around them. Shawn now leads technical SEO strategy at Lastmile Retail and speaks regularly at industry events including BrightonSEO and Advanced Search.
Venue
FM:Systems office, 1101 E Whitaker Mill Rd Suite 220, Raleigh, NC 27604
We'll be gathering at the FM:Systems (now part of Johnson Controls) office at Raleigh Iron Works, 1101 E Whitaker Mill Rd Suite 220, Raleigh, NC 27604. The office is located on the second story of the building, stairs and an elevator are available to reach the second story.
Quick first person video walkthrough of how to get up to the second story and to the entrance of the office: https://www.youtube.com/shorts/i6VHRPiwwPg
Parking at Raleigh Iron Works is free and can be found on-street surrounding the building, in a nearby parking deck (behind the Ponysaurus brewery), or in the handful of adjacent parking lots.
Thank you to Brian McDonald and team for graciously hosting us in their beautiful office!
Schedule
- 6:00pm - Doors open, drinks, mingling, networking
- 6:30pm - Presentation, Q&A
- 7:15pm - Drinks, mingling, networking
- 8:00pm - Doors close