The web is a graph. A crawler decides which URL to visit next with a limited request budget. BFS sweeps by levels, Shark-Search chases a topic (best-first with inherited scores), OPIC computes page importance online like a live PageRank. Same loop, different ordering — run all three on a site and compare.
BFS
Classic level-order sweep: every link gets the score
score = −(depth + 1)
so the closest pages to the seed are visited first. Uniform coverage, but blind to the topic: it spends requests on /login just like on /blog.
Shark-Search
Topical best-first. Each link blends what its parent passed down with its own anchor + URL words:
score = γ·inherited + (1−γ)·local
Children of a relevant parent inherit δ·relevance; barren branches decay as δⁿ and die off on their own. The crawler swarms the relevant region of the graph.
OPIC
Online importance, no query needed. The seed starts with cash = 1.0; on each visit the page banks its cash into its history and splits it evenly among its links:
cash(child) += cash(page) / out_links
Heavily linked pages accumulate cash from many parents and jump the queue — a live PageRank, computed while crawling.