Search Quality Evaluator Guideline Topics 2021
Introduction to Search Quality Rating
0.0 The Search Experience
0.1 The Purpose of Search Quality Rating
0.2 Raters Must Represent People in their Rating Locale
0.3 Browser Requirements
0.4 Ad Blocking Extensions
0.5 Internet Safety Information
Part 1: Page Quality Rating Guideline
=============================================
1.0 Introduction to Page Quality Rating
2.0 Understanding Webpages and Websites
2.1 Important Definitions
2.2 What is the Purpose of a Webpage?
2.3 Your Money or Your Life (YMYL) Pages
2.4 Understanding Webpage Content
2.4.1 Identifying the Main Content (MC)
2.4.2 Identifying the Supplementary Content (SC)
2.4.3 Identifying Advertisements/Monetization (Ads)
2.4.4 Summary of the Parts of the Page
2.5 Understanding the Website
2.5.1 Finding the Homepage
2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
2.5.3 Finding About Us, Contact Information, and Customer Service Information
2.6 Reputation of the Website and Creator of the Main Content
2.6.1 Research on the Reputation of the Website and Creator of the Main Content
2.6.2 Sources of Reputation Information
2.6.3 Customer Reviews of Stores/Businesses
2.6.4 How to Search for Reputation Information
2.6.5 What to Do When You Find No Reputation Information
3.0 Overall Page Quality Rating
3.1 Page Quality Rating: Most Important Factors
3.2 Expertise, Authoritativeness, and Trustworthiness (E-A-T)
4.0 High Quality Pages
4.1 Characteristics of High Quality Pages
4.2 A Satisfying Amount of High Quality Main Content
4.3 Clear and Satisfying Website Information: Who is Responsible and Customer Service
4.4 Positive Reputation
4.5 A High Level of Expertise/Authoritativeness/Trustworthiness (E-A-T)
5.1 Very High Quality MC
5.2 Very Positive Reputation
5.3 Very High Level of E-A-T
6.0 Low Quality Pages
6.1 Lacking Expertise, Authoritativeness, or Trustworthiness (E-A-T)
6.2 Low Quality Main Content
6.3 Unsatisfying Amount of Main Content
6.4 Distracting Ads/SC
6.5 Mixed or Mildly Negative Reputation of the Website or Creator of the Main Content
6.6 Unsatisfying Amount of Information about the Website or Creator of the Main Content
7.0 Lowest Quality Pages
7.1 Harmful to Self and Other Individuals
7.2 Harmful to Specified Groups
7.3 Harmfully Misleading Information
7.4 Untrustworthy Webpages or Websites
7.4.1 Inadequate Information about the Website or Creator of the MC for the Purpose of the Page
7.4.2 Lowest E-A-T and Lowest Reputation of the Website or Creator of the MC
7.4.3 Deceptive Purpose and Deceptive Page Design
7.4.4 Deliberately Obstructed or Obscured MC
7.4.5 Suspected Malicious Behavior
7.5 Spammy Webpages
7.5.1 Cannot Determine a Purpose, No MC, Little MC, or Lowest Quality MC
7.5.2 Hacked, Defaced, or Spammed Pages
7.5.3 Auto-generated MC
7.5.4 Copied MC
7.5.5 How to Determine if Content is Copied
8.0 Medium Quality Pages
8.1 Examples of Medium Quality Pages
Part 2: Understanding Mobile User Needs
=============================================
9.0 Understanding Mobile Users, Mobile Queries, and Mobile Results
9.1 Important Rating Definitions and Ideas
9.2 Understanding the Query
9.3 Locale and User Location
9.4 Queries with an Explicit Location
9.5 Queries with Multiple Meanings
9.6 Query Meanings Can Change Over Time
9.7 Understanding User Intent
9.7.1 Know and Know Simple Queries
9.7.2 Do and Device Action Queries
9.7.3 Website Queries
9.7.4 Visit-in-Person Queries and User Location
9.7.5 Queries with Multiple User Intents
9.8 Understanding Result Blocks
9.8.1 Web Search Result Block Examples
9.8.2 Special Content Result Block Examples
9.8.3 Device Action Result Block Examples
9.8.4 How Device Action Results are Displayed in Rating Tasks
Part 3: Needs Met Rating Guideline
========================================
10.0 Rating Using the Needs Met Scale
10.1 Rating Result Blocks: Block Content and Landing Pages
10.2 Fully Meets (FullyM)
10.2.1 Examples of Fully Meets (FullyM) Result Blocks
10.2.2 Examples of Queries that Cannot Have Fully Meets Results
10.3 Highly Meets (HM)
10.3.1 Examples of Highly Meets (HM) Result Blocks
10.4 Moderately Meets (MM)
10.4.1 Examples of Moderately Meets (MM) Result Blocks
10.5 Slightly Meets (SM)
10.5.1 Examples of Slightly Meets (SM) Result Blocks
10.6 Fails to Meet (FailsM)
10.6.1 Examples of Fails to Meet (FailsM) Result Blocks
11.0 The Relationship between Page Quality and Needs Met
12.0 Rating Porn, Foreign Language, and Did Not Load Result
12.1 Porn Flag
12.2 Needs Met Rating for Porn Results
12.2.1 Needs Met Rating for Clear Non-Porn Intent Queries
12.2.2 Needs Met Rating for Possible Porn Intent Queries
12.2.3 Needs Met Rating for Clear Porn Intent Queries
12.3 Reporting Illegal Images
12.4 Foreign Language Flag
12.4.1 Using the Foreign Language Flag
12.4.2 Needs Met Rating for Foreign Language Results
12.5 Did Not Load Flag
12.5.1 Using the Did Not Load Flag
12.5.2 Needs Met Rating and the Did Not Load Flag
12.6 Additional Flags in Some Rating Tasks
13.0 Rating Queries with Multiple Interpretations and Intents
13.1 Rating Queries with Both Website and Visit-in-Person Intent
14.0 Specificity of Queries and Landing Pages
15.0 Needs Met Rating and Freshness
16.0 Misspelled and Mistyped Queries and Results
16.1 Misspelled and Mistyped Queries
16.2 Name Queries
17.0 Non-Fully Meets Results for URL Queries
18.0 Product Queries: Importance of Browsing and Researching
19.0 Rating Visit-in-Person Intent Queries
19.1 Examples Where User Location Does (and Does Not) Matter
20.0 Rating English Language Results in Non-English Locales
20.1 Examples of English (and Non-English) Results in Non-English Locales
21.0 Rating Dictionary and Encyclopedia Results for Different Queries
