Search quality rater guidelines - 2022

Here is a screenshot of a result block that shows when the Foreign Language flag would be used.

Query and User Intent Result Block and LP, Needs Met Rating, Flag

Explanation

This LP is the homepage of the Baidu website. The result is the target page of the query, but is in a foreign language (Chinese). This result merits a FullyM rating and the Foreign Language flag.

Query: [baidu] Locale: English (US) User Intent: Go to the Baidu website.

Important : ● Please assign the Foreign Language flag even if you personally understand the language, but most users in your locale do not. ● Please assign the Foreign Language flag based on the language of the landing page, not the appearance of the result block. ● Please remember to flag all foreign pages with the Foreign Language flag, even if most users in your locale would expect or want a foreign language page for the query. ● Sometimes it is difficult to determine what language the landing page is in. The LP may have multiple languages or no words at all. In these cases, try to represent users in your locale. Does it feel like a foreign language page? When in doubt, don’t use the Foreign Language flag. 15.4.2 Needs Met Rating for Foreign Language Results You must assign a Needs Met rating for all result blocks in your task, even if the result blocks have a foreign language landing page. In most cases, pages you flag as Foreign Language should be rated FailsM , because they cannot be understood by most users in your locale and are therefore useless. Remember that if users in your locale can read the language, then you shouldn’t be using the Foreign Language flag. Occasionally, you will encounter helpful Foreign Language pages. If the query clearly indicates that most users would expect or want a foreign language result, then the Needs Met rating of the foreign language page should not be FailsM . For example, please assign the FullyM rating and Foreign Language flag for baidu.com if the query is [baidu.com], English (US). Videos are often an example where foreign language pages are helpful and desired. Think about user intent and what pages are good for users. If the query <asks= for a foreign language song, band, film, sporting event, etc., then a video of the song, band, film, sporting event, etc. is helpful since it can probably be understood or enjoyed even though it is in a foreign language. For these types of queries, foreign language results are often expected. If the video is someone talking about the song, band, film, or event, the result probably cannot be understood or enjoyed and should be assigned a FailsM rating and the Foreign Language flag. Note: If you are unable to evaluate the Page Quality rating of a Foreign Language result, you do not need to assign a Page Quality rating and can leave the slider at N/A .

Copyright 2022

129

Made with FlippingBook - Online magazine maker