To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[47]
If you use the wrong keywords, you’ll never get the visitors you want or need, because your text doesn’t match what your potential audience is searching for. But if you do use the keywords people are searching for, your business can thrive. So if you see it like that, your keywords should reflect what your audience is searching for. With the wrong keywords, you’ll end up with the wrong audience, or none at all. That’s why having the right keywords is really important.
This relationship between rankings and clicks (and traffic) is strongest amongst the top 3 search results. However, changing layout of the search results pages is constantly changing, with the inclusion of Google’s Knowledge Graph data and the integration of Universal Search elements (SERP Features) like videos, maps and Google Shopping ads. These developments can mean that the top 3 organic rankings are no longer the 3 best positions on the SERP. This has been demonstrated in heatmap and eye-tracking tests.
Search engines may penalize sites they discover using black or grey hat methods, either by reducing their rankings or eliminating their listings from their databases altogether. Such penalties can be applied either automatically by the search engines' algorithms, or by a manual site review. One example was the February 2006 Google removal of both BMW Germany and Ricoh Germany for use of deceptive practices.[54] Both companies, however, quickly apologized, fixed the offending pages, and were restored to Google's search engine results page.[55]
QUOTE: “So there’s three things that you really want to do well if you want to be the world’s best search engine you want to crawl the web comprehensively and deeply you want to index those pages and then you want to rank or serve those pages and return the most relevant ones first….. we basically take PageRank as the primary determinant and the more PageRank you have that is the more people who link to you and the more reputable those people are the more likely it is we’re going to discover your page…. we use page rank as well as over 200 other factors in our rankings to try to say okay maybe this document is really authoritative it has a lot of reputation because it has a lot of PageRank … and that’s kind of the secret sauce trying to figure out a way to combine those 200 different ranking signals in order to find the most relevant document.” Matt Cutts, Google
Websites that have extremely negative or malicious reputations. Also use the Lowest rating for violations of the Google Webmaster Quality Guidelines. Finally, Lowest+ may be used both for pages with many low-quality characteristics and for pages whose lack of a single Page Quality characteristic makes you question the true purpose of the page. Important: Negative reputation is sufficient reason to give a page a Low quality rating. Evidence of truly malicious or fraudulent behavior warrants the Lowest rating.
A variety of methods can increase the prominence of a webpage within the search results. Cross linking between pages of the same website to provide more links to important pages may improve its visibility.[48] Writing content that includes frequently searched keyword phrase, so as to be relevant to a wide variety of search queries will tend to increase traffic.[48] Updating content so as to keep search engines crawling back frequently can give additional weight to a site. Adding relevant keywords to a web page's metadata, including the title tag and meta description, will tend to improve the relevancy of a site's search listings, thus increasing traffic. URL canonicalization of web pages accessible via multiple URLs, using the canonical link element[49] or via 301 redirects can help make sure links to different versions of the URL all count towards the page's link popularity score.
Keyword analysis. From nomination, further identify a targeted list of key­words and phrases. Review competitive lists and other pertinent industry sources. Use your preliminary list to determine an indicative number of recent search engine queries and how many websites are competing for each key­word. Prioritize keywords and phrases, plurals, singulars and misspellings. (If search users commonly misspell a keyword, you should identify and use it). Please note that Google will try to correct the term when searching, so use this with care.
“Easy Internet Services have up to this point been an amazing company to work with. The mark of truly good customer service comes from someone not just going above and beyond their required line of duties, but them being happy to do so. In my eyes this description perfectly encapsulates how I feel about SEO Rankings. I have engaged them to redesign my website, which they have done an amazing job of. It is beautiful and functional. In addition to this I have asked them to do my SEO (Search Engine Optimisation) and have seen rapid results. I have total faith that in the near future they will have me at the top of my desired search terms. they are fast, efficient, friendly, very well priced and the proprietors of functional, effective and beautiful work. You will not go wrong putting the desires and dreams for your business in their hands.”
It's important to check that you have a mix of head terms and long-tail terms because it'll give you a keyword strategy that's well balanced with long-term goals and short-term wins. That's because head terms are generally searched more frequently, making them often (not always, but often) much more competitive and harder to rank for than long-tail terms. Think about it: Without even looking up search volume or difficulty, which of the following terms do you think would be harder to rank for?
Sometimes we refer to those root keywords as "vanity keywords," because if you do just one search to see who seems to be winning the space, you are likely to pick the single broadest keyword and see who comes up ranked highly. In nearly every case, however, we have found it to be more successful and deliver a significantly better return on your SEM investment by focusing on the hundreds or even thousands of more specific keywords that more closely match the services, products, brands, and locations that you sell or serve.
QUOTE: “Duplicated content is often not manipulative and is commonplace on many websites and often free from malicious intent. Copied content can often be penalised algorithmically or manually. Duplicate content is not penalised, but this is often not an optimal set-up for pages, either. Be VERY careful ‘spinning’ ‘copied’ text to make it unique!” Shaun Anderson, Hobo, 2018

Now, some buckets are worth more than others, and the three main buckets that you need to be aware of for search rankings are quality, trust and authority. So quality: what Google is trying to measure when they’re trying to figure out what sites should rank is offering something valuable or unique or interesting to googles searchers. For example: good content—if you are selling t-shirts and you are using the same description that every other t-shirt seller is using on their website then you are not offering anything unique to Google’s searchers.
Simple navigation reigns and quality content is king – A user-friendly website, with interesting and easy-to-find information is what will boost your traffic. Each page needs to be built around keyword themes, with unique content, so search engines can easily index yours and rank you higher. Positive behaviors from site visitors is your best bet for a better ranking, so keep the content natural and focused; avoid jargon and keyword stuffing to keep users from leaving the site unhappy and hurting its ranking.
QUOTE: “To summarize, a lack of helpful SC may be a reason for a Low quality rating, depending on the purpose of the page and the type of website. We have different standards for small websites which exist to serve their communities versus large websites with a large volume of webpages and content. For some types of “webpages,” such as PDFs and JPEG files, we expect no SC at all.” Google Search Quality Evaluator Guidelines 2017
Moz Keyword Explorer - Input a keyword in Keyword Explorer and get information like monthly search volume and SERP features (like local packs or featured snippets) that are ranking for that term. The tool extracts accurate search volume data by using live clickstream data. To learn more about how we're producing our keyword data, check out Announcing Keyword Explorer.
QUOTE: “Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.”
You likely have a few keywords in mind that you would like to rank for. These will be things like your products, services, or other topics your website addresses, and they are great seed keywords for your research, so start there! You can enter those keywords into a keyword research tool to discover average monthly search volume and similar keywords. We’ll get into search volume in greater depth in the next section, but during the discovery phase, it can help you determine which variations of your keywords are most popular amongst searchers.
This is why developing a list of keywords is one of the first and most important steps in any search engine optimization initiative. Keywords and SEO are directly connected when it comes to running a winning search marketing campaign. Because keywords are foundational for all your other SEO efforts, it's well worth the time and investment to ensure your SEO keywords are highly relevant to your audience and effectively organized for action.
×