the internet is ruined

Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is “The Effect of Cellular Phone Use Upon Driver Attention”, a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98]. It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.

Since it is very difficult even for experts to evaluate search engines, search engine bias is particularly insidious. A good example was OpenText, which was reported to be selling companies the right to be listed at the top of the search results for particular queries [Marchiori 97]. This type of bias is much more insidious than advertising, because it is not clear who “deserves” to be there, and who is willing to pay money to be listed. This business model resulted in an uproar, and OpenText has ceased to be a viable search engine. But less blatant bias are likely to be tolerated by the market. For example, a search engine could add a small factor to search results from “friendly” companies, and subtract a factor from results from competitors. This type of bias is very difficult to detect but could still have a significant effect on the market. Furthermore, advertising income often provides an incentive to provide poor quality search results. For example, we noticed a major search engine would not return a large airline’s homepage when the airline’s name was given as a query. It so happened that the airline had placed an expensive ad, linked to the query that was its name. A better search engine would not have required this ad, and possibly resulted in the loss of the revenue from the airline to the search engine. In general, it could be argued from the consumer point of view that the better the search engine is, the fewer advertisements will thebe needed for the consumer to find what they want. This of course erodes the advertising supported business model of the existing search engines. However, there will always be money from advertisers who want a customer to switch products, or have something that is genuinely new. But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.

This is an extract from The Anatomy of a Large-Scale Hypertextual Web Search Engine by Sergey Brin and Lawrence Page - founders of Google.

Google has come a long way since then.

seems like the frontier of AI being concentrated in the hands of a tiny number of companies will naturally lead to massive power concentration as we reach general and superintelligence.

Recommender systems are not on your side, and the better they get, the more true this becomes. The goal of most recommendation systems is to a) maximise your engagement to whatever content it is feeding you and b) extract money from you by upweighting whoever pays the most for ads. Your goals are presumably different from this. And the more powerful the system becomes, the more that your interests and the interests of the platform decouple.

When a platform starts out, it is often just trying to accumulate users (and creators), and maximising your engagement often just looks like showing you what you want to see. However, as these systems get more powerful, they can start to capitalise more and more on your base emotions, showing you content that might not be satisfying or informative in any way, but keep you on the platform. They can start to shape your desires to pull you down a rabbit hole that keeps you watching more and more. They start to shorten your attention span, creating a psychological dependence on your next fix of reels. Spotify have baked in paid placement into their algorithm to upweight artists in exchange for royalties. Half of the first page of an amazon search will be paid promos. Ads only work when they show you something you would not have bought all else equal. Ads directly make recommendation systems worse by obscuring the thing that is most relevant to you.

The internet used to be a tapestry of interconnected websites, but now more and more content is being subsumed by monopolising platforms. Alphabet, meta and netflix generate nearly half of all mobile internet traffic worldwide. These platforms lock you in and mediate your interactions with the online world. This is an enormous amount of power we are giving these companies.

Anyway, that’s why I want to build open source recommender systems that i can own and control to act in my interests instead of those of big tech companies, to aggregate across many different platforms and independent sites rather than locking me into a platform, to help me deal with the overwhelm of information that can be found and must be kept up to date with on the internet. If you’re interested in working on something like this, please get in touch.