Mining the hyperlinks of the web graph

Traditional link analysis treats all hyperlinks equally and makes the assumption that links confer endorsement, so that a web page author will create a link and thus have authority propagated through the link if and only if the target is valuable. Unfortunately, this assumption does not hold in today's World-Wide Web. Hyperlinks are not homogeneous, they may be created in different contexts and for different purposes. These factors will skew the web graph greatly and thus influence link-based authority calculation. This dissertation investigates novel characteristics of hyperlinks to help a search engine focus on relevant, trustworthy, and high quality content. Two important hyperlink features—topicality and trust—are proposed and studied. We present various models to incorporate these features into authority estimation mechanisms. Through retrieval experiments on multiple datasets, we demonstrate that such models can provide strengthened measures for web page reputation that result in improved web search quality.