How search engines work.
How search engines work and understanding their purpose is key to improving your search rankings.
Many bloggers, internet marketers, and webmasters struggle with search engine optimisation (SEO) because they fail to understand why search engines exist or what they want from websites.
For example, if you believe that search engines exist for the benefit of content creators and website owners, then you don’t really understand search engines at all. Additionally, your digital marketing tactics are likely to be flawed – which will hurt your site’s organic SEO rankings.
In this article, we will share with you the role of search engines and what they want from websites.
What are search engines?
Search engines like Google, Bing, and Duck Duck Go, exist to connect users to the most relevant, useful content about a given search query.
For example, let’s say that a user searches Google for “the best barbecue restaurants in Sydney.” Google’s algorithm will then return search results that are related to barbecue restaurants in Sydney – with the most relevant, informative, and user-friendly sites ranked at the top of the search results.
Therefore, search engines primary tasks are to discover, understand, and organize the internet’s content so that they can serve the most relevant results to the questions searchers are asking.
To do this, there are two main parts to every search engine.
- Search index – A data structure containing key information about discovered URLs and the content contained on those web pages.
- Search algorithms – A step by step process or procedure to find information within the search index in order to rank matching results to search queries.
What are search engine results pages (SERPS)?
“SERPS” or Search Engine Results Pages are exactly as the name suggests. They are a list of results or URLs that the search engine returns for a given search query.
SERPS generally contain three or four sections of grouped URLs. These include;
- Paid results: Also known as Google ads. These URLs are included at the top of the SERPs because they have paid to be there.
- Orangic results: These appear at the bottom or after paid results and other results. When we discuss page rankings we are generally referring to the organic results.
- Other results: These include image packs, featured snippets, FAQs, local business listings etc. This is dependant to the type of search query.
Following is a Google SERP for a query on SERPs. In this example, there are no paid results. That’s because no website has bought google ads for the keywords searched.
How do search engines work?
But how search engines work is a bit more complicated. How do they know what’s on your website and whether to include you in SERPs for customer queries?
Search engines follow three primary functions that enable them to discover, understand and organize the internet’s content.
Crawling:
This is where search engines discover content online. They do this by sending out “crawlers” or “spiders”, which are robots that scour over the code and content for each URL they find online. Hence the name “crawling”.
Content is anything from a webpage or media file – an image, video, pdf etc.
The way in which crawlers find the new content online is via links from pages that they already know about. The crawlers select a bunch of pages (billions of them) and follow the links on those pages to find new URLs. When a new URL is found, it is added to the search engine “index”. An index is essentially a large database of URLs.
Indexing:
Once a webpage has been discovered, information about the page is then added into a data structure called an “index”.
An index is, essentially, a large database of information that includes a number of relevant signals about the content that has been discovered on each URL. This information helps the search engine understand the content on a page and determine what content is “good enough” to serve users for given search queries.
The information added to a search engine index includes the following.
Keywords contained on the page indicate the key topics discussed.
Content types that are included on the page are indicated by schema/microdata. This includes different media as well as structured data items like FAQ blocks, recipes etc.
Published/edited dates or how fresh the content is on the page. Search engines prioritise newer content.
User engagement with your page and your domain including click-through rate, bounce rates and time of page. This helps search engines determine the relevance and quality of your content.
Ranking:
When a user performs a search query, the search engine will serve the most relevant pieces of content that it has in its index in response. The order in which the search engine serves the content is called search engine ranking. In general, you can assume that the higher a website is ranked, the more relevant the search engine believes that site is to the query.
To determine website rankings, search engines use algorithms to retrieve and serve content from information stored in its index. Search engines determine which URLs are the most relevant for particular queries based on what is called “ranking factors”.
Every search engine has its own ranking factors to determine exactly which pages to serve and in which order. Unfortunately, nobody really knows all of them. However, we do know about many of the most important ones.
Let’s use Google as an example as it is the most popular search engine by far. Google uses over 200 ranking factors. While we don’t know what all of them are exactly, we do know some of the key ranking factors which have the greatest impact on search results. These include backlinks, relevance, authority, freshness, page speed, mobile-friendliness, optimised content, and technical SEO.
If you need help getting better search rankings, get in touch with us on (02) 7906 8405 or send us an email at info@foundonline.com.au.
We are an SEO agency in Sydney with a team of website developers, content writers and SEO experts.
Sources:
How Search Works, Google https://www.google.com/search/howsearchworks/