Finding Keyword Opportunities Without Historical Data

Finding Keyword Opportunities Without Historical Data

Finding Keyword Opportunities Without Historical Data 1920 1281 Imogen Davies

At Google’s Search On event in October last year, Prabhakar Raghavan explained that 15% of daily queries are ones that have never been searched before. If we take the latest figures from Internet Live Stats, which state 3.5 billion queries are searched every day, that means that 525 million of those queries are brand new.

That is a huge number of opportunities waiting to be identified and worked into strategies, optimization, and content plans. The trouble is, all of the usual keyword research tools are, at best, a month behind with the data they can provide. Even then, the volumes they report need to be taken with a grain of salt – you’re telling me there are only 140 searches per month for “women’s discount designer clothing”? – and if you work in B2B industries, those searches are generally much smaller volumes to begin with.

So, we know there are huge amounts of searches available, with more and more being added every day, but without the data to see volumes, how do we know what we should be working into strategies? And how do we find these opportunities in the first place?

Finding the opportunities

The usual tools we turn to aren’t going to be much use for keywords and topics that haven’t been searched in volume previously. So, we need to get a little creative — both in where we look, and in how we identify the potential of queries in order to start prioritizing and working them into strategies. This means doing things like:

  1. Mining People Also Ask
  2. Scraping autosuggest
  3. Drilling into related keyword themes

Mining People Also Ask

People Also Ask is a great place to start looking for new keywords, and tends to be more up to date than the various tools you would normally use for research. The trap most marketers fall into is looking at this data on a small scale, realizing that (being longer-tail terms) they don’t have much volume, and discounting them from approaches. But when you follow a larger-scale process, you can get much more information about the themes and topics that users are searching for and can start plotting this over time to see emerging topics faster than you would from standard tools.

To mine PAA features, you need to:

1. Start with a seed list of keywords.

2. Use SerpAPI to run your keywords through the API call – you can see their demo interface below and try it yourself:

3. Export the “related questions” features returned in the API call and map them to overall topics using a spreadsheet:

4. Export the “related search boxes” and map these to overall topics as well:

5. Look for consistent themes in the topics being returned across related questions and searches.

6. Add these overall themes to your preferred research tool to identify additional related opportunities. For example, we can see coffee + health is a consistent topic area, so you can add that as an overall theme to explore further through advanced search parameters and modifiers.

7. Add these as seed terms to your preferred research tool to pull out related queries, like using broad match (+coffee health) and phrase match (“coffee health”) modifiers to return more relevant queries:

This then gives you a set of additional “suggested queries” to broaden your search (e.g. coffee benefits) as well as related keyword ideas you can explore further.

This is also a great place to start for identifying differences in search queries by location, like if you want to see different topics people are searching for in the UK vs. the US, then SerpAPI allows you to do that at a larger scale.

If you’re looking to do this on a smaller scale, or without the need to set up an API, you can also use this really handy tool from Candour – Also Asked – which pulls out the related questions for a broad topic and allows you to save the data as a .csv or an image for quick review:

Once you’ve identified all of the topics people are searching for, you can start drilling into new keyword opportunities around them and assess how they change over time. Many of these opportunities don’t have swathes of historical data reported in the usual research tools, but we know that people are searching for them and can use them to inform future content topics as well as immediate keyword opportunities.

You can also track these People Also Ask features to identify when your competitors are appearing in them, and get a better idea of how they’re changing their strategies over time and what kind of content and keywords they might also be targeting. At Found, we use our bespoke SERP Real Estate tool to do just that (and much more) so we can spot these opportunities quickly and work them into our approaches.

Scraping autosuggest

This one doesn’t need an API, but you’ll need to be careful with how frequently you use it, so you don’t start triggering the dreaded captchas.

Similar to People Also Ask, you can scrape the autosuggest queries from Google to quickly identify related searches people are entering. This tends to work better on a small scale, just because of the manual process behind it. You can try setting up a crawl with various parameters entered and a custom extraction, but Google will be pretty quick to pick up on what you’re doing.

To scrape autosuggest, you use a very simple URL query string:

https://suggestqueries.google.com/complete/search?output=toolbar&hl=&gl=uk&q=

Okay, it doesn’t look that simple, but it’s essentially a search query that outputs all of the suggested queries for your seed query.

So, if you were to enter “cyber security” after the “q=”, you would get:

This gives you the most common suggested queries for your seed term. Not only is this a goldmine for identifying additional queries, but it can show some of the newer queries that have started trending, as well as information related to those queries that the usual tools won’t provide data for.

For example, if you want to know what people are searching for related to COVID-19, you can’t get that data in Keyword Planner or most tools that utilize the platform, because of the advertising restrictions around it. But if you add it to the suggest queries string, you can see:

This can give you a starting point for new queries to cover without relying on historical volume. And it doesn’t just give you suggestions for broad topics – you can add whatever query you want and see what related suggestions are returned.

If you want to take this to another level, you can change the location settings in the query string, so instead of “gl=uk” you can add “=us” and see the suggested queries from the US. This then opens up another opportunity to look for differences in search behavior across different locations, and start identifying differences in the type of content you should be focusing on in different regions — particularly if you’re working on international websites or targeting international audiences.

Refining topic research

Although the usual tools won’t give you that much information on brand new queries, they can be a goldmine for identifying additional opportunities around a topic. So, if you have mined the PAA feature, scraped autosuggest, and grouped all of your new opportunities into topics and themes, you can enter these identified “topics” as seed terms to most keyword tools.

Google Ads Keyword Planner

Currently in beta, Google Ads now offers a “Refine keywords” feature as part of their Keyword Ideas tool, which is great for identifying keywords related to an overarching topic.

Below is an example of the types of keywords returned for a “coffee” search:

 Here we can see the keyword ideas have been grouped into:

  • Brand or Non-Brand – keywords relating to specific companies
  • Drink – types of coffee, e.g. espresso, iced coffee, brewed coffee
  • Product – capsules, pods, instant, ground
  • Method – e.g. cold brew, French press, drip coffee

These topic groupings are fantastic for finding additional areas to explore. You can either:

  1. Start here with an overarching topic to identify related terms and then go through the PAA/autosuggest identification process.
  2. Start with the PAA / autosuggest identification process and put your new topics into Keyword Planner

Whichever way you go about it, I’d recommend doing a few runs so you can get as many new ideas as possible. Once you’ve identified the topics, run them through the refine keywords beta to pull out more related topics, then run them through the PAA/autosuggest process to get more topics, and repeat a few times depending how many areas you want to explore or how in-depth you need your research to be.

Google Trends

Trends data is one of the most up-to-date sets you can look at for topics and specific queries. However, it is worth noting that for some topics, it doesn’t hold any data, so you might run into problems with more niche areas.

Using “travel ban” as an example, we can see the trends in searches as well as related topics and specific related queries:

Now, for new opportunities, you aren’t going to find a huge amount of data, but if you’ve grouped your opportunities into overarching topics and themes, you’ll be able to find some additional opportunities from the “Related topics” and “Related queries” sections.

In the example above we see these sections include specific locations and specific mentions of coronavirus – something that Keyword Planner won’t provide data on as you can’t bid on it.

Drilling into the different related topics and queries here will give you a bit more insight into additional areas to explore that you may not have otherwise been able to identify (or validate) through other Google platforms.

Moz Keyword Explorer

The Moz interface is a great starting point for validating keyword opportunities, as well as identifying what’s currently appearing in the SERPs for those terms. For example, a search for “london theatre” returns the following breakdown:

From here, you can drill into the keyword suggestions and start grouping them into themes as well, as well as being able to review the current SERP and see what kind of content is appearing. This is particularly useful when it comes to understanding the intent behind the terms to make sure you’re looking at the opportunities from the right angle – if a lot more ticket sellers are showing than news and guides, for example, then you want to be focusing these opportunities on more commercial pages than informational content.

Other tools

There are a variety of other tools you can use to further refine your keyword topics and identify new related ideas, including the likes of SEMRush, AHREFS, Answer The Public, Ubersuggest, and Sistrix, all offering relatively similar methods of refinement.

The key is identifying the opportunities you want to explore further, looking through the PAA and autosuggest queries, grouping them into themes, and then drilling into those themes.

Keyword research is an ever-evolving process, and the ways in which you can find opportunities are always changing, so how do you then start planning these new opportunities into strategies?

Forming a plan

Once you’ve got all of the data, you need to be able to formalize it into a plan to know when to start creating content, when to optimize pages, and when to put them on the back burner for a later date.

A quick (and consistent) way you can easily plot these new opportunities into your existing plans and strategies is to follow this process:

  1. Identify new searches and group into themes
  2. Monitor changes in new searches. Run the exercise once a month to see how much they change over time
  3. Plot trends in changes alongside industry developments. Was there an event that changed what people were searching for?
  4. Group the opportunities into actions: create, update, optimize.
  5. Group the opportunities into time-based categories: topical, interest, evergreen, growing, etc.
  6. Plot timeframes around the content pieces. Anything topical gets moved to the top of the list, growing themes can be plotted in around them, interest-based can be slotted in throughout the year, and evergreen pieces can be turned into more hero-style content.

Then you end up with a plan that covers:

  • All of your planned content.
  • All of your existing content and any updates you might want to make to include the new opportunities.
  • A revised optimization approach to work in new keywords on existing landing pages.
  • A revised FAQ structure to answer queries people are searching for (before your competitors do).
  • Developing themes of content for hubs and category page expansion.

Conclusion

Finding new keyword opportunities is imperative to staying ahead of the competition. New keywords mean new ways of searching, new information your audience needs, and new requirements to meet. With the processes outlined above, you’ll be able to keep on top of these emerging topics to plan your strategies and priorities around them. The world of search will always change, but the needs of your audience — and what they are searching for — should always be at the center of your plans.

* Checkbox GDPR is required

*

I agree

Will you like to book a consultation today?

We promise you’ll be glad to have us as the only premium website developer you’ve ever had!

Will you like to book a consultation today?

We promise you’ll be glad to have us as the only premium website developer you’ve ever had!

Bear Design - WordPress Development

Bear Design provides website development and design, creating content uploaded websites and improving web page placements and web traffic. Bear Design websites are unique, easy to use and responsive. Site owners can easily edit the content, or can trust the Bear Design & Communications to keep them up to date and supply quality content regularly.


GET IN TOUCH
160 City Road, EC1V 2NX London, United Kingdom
Monday – Thursday: 9:00 AM – 5:00 PM
Friday: 9:00 AM – 2:00 PM

WE ARE IN LONDON

Bear Design - WordPress Development

Bear Design provides website development and design, creating content uploaded websites and improving web page placements and web traffic. Bear Design websites are unique, easy to use and responsive. Site owners can easily edit the content, or can trust the Bear Design & Communications to keep them up to date and supply quality content regularly.


WE ARE IN LONDON

GET IN TOUCH
160 City Road, EC1V 2NX London, United Kingdom
Monday – Thursday: 9:00 AM – 5:00 PM
Friday: 9:00 AM – 2:00 PM

Bear Design - WordPress Development

Bear Design provides website development and design, creating content uploaded websites and improving web page placements and web traffic. Bear Design websites are unique, easy to use and responsive. Site owners can easily edit the content, or can trust the Bear Design & Communications to keep them up to date and supply quality content regularly.


GET IN TOUCH
160 City Road, EC1V 2NX London, United Kingdom
Monday – Thursday: 9:00 AM – 5:00 PM
Friday: 9:00 AM – 2:00 PM

WE ARE IN LONDON

© Made with by Bear Design

© Made with by Bear Design

    We are Bear Design

    WE DESIGN

    YOUR WORLD

    Bear Design & Communications Ltd.

    Address : 160 City Road, EC1V 2NX London, United Kingdom
    Phone : +36 702 448 100
    Email : [email protected]

    Opening hours :
    Monday – Thursday: 9:00 AM – 5:00 PM
    Friday: 9:00 AM – 2:00 PM

    Are you sure?
    You must approve our cookie policy to use our site. I you refuse it you will redirect to the Google.
    Refuse
    Approve Cookies
    Cookie Policy
    Cookie Policy
    This Bear Design Cookie Policy (“Policy”) outlines the general policy, practices, and types of cookies that Bear Design And Communications Ltd.. (“Bear Design”, “we”, “us” or “our”) may use to improve our services and your experience when visiting our websites.Cookies are small pieces of text used to store information on web browsers. They’re used by many websites to store and receive identifiers and other information on devices, such as a handheld phone or computer. Our site and services use cookies and other similar technologies (collectively in this Policy, “cookies”), in order to provide a better service to you and to generally improve our sites and services. For example, we may use cookies to help direct you to the appropriate part of our websites, by indicating that you are a repeat visitor. We may also use information to present you with services that are matched to your preferences.Some portions of our websites are functional without cookies, and you may generally choose whether to accept cookies. Most web browsers are set to accept cookies by default, however, you may be able to delete cookies yourself through your browser’s cookie manager. To do so, please follow the instructions provided by your web browser. Please note that disabling cookies will reset your session, disable auto-login, and may adversely the availability and functionality of our websites and the services we can provide to you.As part of our services, we may also place cookies on the computers of visitors to websites protected by Bear Design. We do this in order to identify malicious visitors, reduce the chance of blocking legitimate users, and to provide customized services.Our websites use first party cookies (i.e., cookies set directly by Bear Design) as well as third party cookies, as detailed in the table below.
    Type of CookieWhy we use these cookiesWho serves them and where can you find out more information?
    Analytics and research of usersThese are used to understand, improve, and research users visiting //beardesign.me and their needs for our product offerings. For example, we may use cookies to understand what pages a user browses before submitting a sales request form. We do not share information about this analysis with any third parties.Selected third parties listed and defined as follows:
    • Google Analytics – Web traffic tracking – //www.google.com/policies/privacy/
    • Bing – Conversion tracking from Bing ads – https://advertise.bingads.microsoft.com/en-us/resources/policies/microsoft-bing-adsprivacy-policy
    • Doubleclick – Google advertising platform that analyzes browsing activity across website to establish user profile – //www.google.com/policies/technologies/ads/
    • Twitter – Analyzes browsing activity across website to establish user profile – https://support.twitter.com/articles/20170514
    • Facebook – Analyzes browsing activity across website to establish user profile – https://www.facebook.com/policies/cookies/
    A user can delete these cookies through browser settings.
    Improving Website experienceThese provide functionality to help us deliver a better user experience for our website. For example, cookies help facilitate chats with our sales representatives, allow you to search the website, and deliver the user quickly to their intended website location.1st party and selected third parties as defined below:
    • __cfduid 3rd party cookie – This cookie is strictly necessary for Cloudflare’s security features
    • __hssc Cookie for keeping track of sessions. This is used to determine if we should increment the session number and timestamps in the __hstc cookie. It contains: the domain, viewCount (increments each pageView in a session), session start timestamp. (Expires: 30 min)
    • __hssrc Whenever HubSpot changes the session cookie, this cookie is also set. We set it simply to the value “1”, and use it to determine if the user has restarted their browser. If this cookie does not exist when we manage cookies, we assume it is a new session. (Expires: None. Session cookie)
    • __hstc The main cookie for tracking visitors. It contains: the domain, utk (see below), initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session) (Expires: 2 years)
    • hsfirstvisit This cookie used to keep track of a user’s first visit. (Expires: 10 years)
    • hubspotutk This cookie is used for to keep track of a visitor’s identity. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. (Expires: 10 years)
    • wordpress_ WordPress cookie for a logged in user.
    • wordpress_logged_in_ WordPress cookie for a logged in user.
    • wp-settings- WordPress also sets a few wp-settings-[UID] cookies. The number on the end is your individual user ID from the users database table. This is used to customize your view of admin interface, and possibly also the main site interface.
    • wp-settings-time- WordPress also sets a few wp-settings-{time}-[UID] cookies. The number on the end is your individual user ID from the users database table. This is used to customize your view of admin interface, and possibly also the main site interface.
    • __cfduid 3rd party cookie – This cookie is strictly necessary for Cloudflare’s security features
    A user can delete these cookies through browser settings.
    LAST UPDATE: 24.01.2018, LONDON
    Approve
    Refuse
    Cookie Policy
    This Bear Design Cookie Policy (“Policy”) outlines the general policy, practices, and types of cookies that Bear Design And Communications Ltd.. (“Bear Design”, “we”, “us” or “our”) may use to improve our services and your experience when visiting our websites.Cookies are small pieces of text used to store information on web browsers. They’re used by many websites to store and receive identifiers and other information on devices, such as a handheld phone or computer. Our site and services use cookies and other similar technologies (collectively in this Policy, “cookies”), in order to provide a better service to you and to generally improve our sites and services. For example, we may use cookies to help direct you to the appropriate part of our websites, by indicating that you are a repeat visitor. We may also use information to present you with services that are matched to your preferences.Some portions of our websites are functional without cookies, and you may generally choose whether to accept cookies. Most web browsers are set to accept cookies by default, however, you may be able to delete cookies yourself through your browser’s cookie manager. To do so, please follow the instructions provided by your web browser. Please note that disabling cookies will reset your session, disable auto-login, and may adversely the availability and functionality of our websites and the services we can provide to you.As part of our services, we may also place cookies on the computers of visitors to websites protected by Bear Design. We do this in order to identify malicious visitors, reduce the chance of blocking legitimate users, and to provide customized services.Our websites use first party cookies (i.e., cookies set directly by Bear Design) as well as third party cookies, as detailed in the table below.
    Type of CookieWhy we use these cookiesWho serves them and where can you find out more information?
    Analytics and research of usersThese are used to understand, improve, and research users visiting //beardesign.me and their needs for our product offerings. For example, we may use cookies to understand what pages a user browses before submitting a sales request form. We do not share information about this analysis with any third parties.Selected third parties listed and defined as follows:
    • Google Analytics – Web traffic tracking – //www.google.com/policies/privacy/
    • Bing – Conversion tracking from Bing ads – https://advertise.bingads.microsoft.com/en-us/resources/policies/microsoft-bing-adsprivacy-policy
    • Doubleclick – Google advertising platform that analyzes browsing activity across website to establish user profile – //www.google.com/policies/technologies/ads/
    • Twitter – Analyzes browsing activity across website to establish user profile – https://support.twitter.com/articles/20170514
    • Facebook – Analyzes browsing activity across website to establish user profile – https://www.facebook.com/policies/cookies/
    A user can delete these cookies through browser settings.
    Improving Website experienceThese provide functionality to help us deliver a better user experience for our website. For example, cookies help facilitate chats with our sales representatives, allow you to search the website, and deliver the user quickly to their intended website location.1st party and selected third parties as defined below:
    • __cfduid 3rd party cookie – This cookie is strictly necessary for Cloudflare’s security features
    • __hssc Cookie for keeping track of sessions. This is used to determine if we should increment the session number and timestamps in the __hstc cookie. It contains: the domain, viewCount (increments each pageView in a session), session start timestamp. (Expires: 30 min)
    • __hssrc Whenever HubSpot changes the session cookie, this cookie is also set. We set it simply to the value “1”, and use it to determine if the user has restarted their browser. If this cookie does not exist when we manage cookies, we assume it is a new session. (Expires: None. Session cookie)
    • __hstc The main cookie for tracking visitors. It contains: the domain, utk (see below), initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session) (Expires: 2 years)
    • hsfirstvisit This cookie used to keep track of a user’s first visit. (Expires: 10 years)
    • hubspotutk This cookie is used for to keep track of a visitor’s identity. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. (Expires: 10 years)
    • wordpress_ WordPress cookie for a logged in user.
    • wordpress_logged_in_ WordPress cookie for a logged in user.
    • wp-settings- WordPress also sets a few wp-settings-[UID] cookies. The number on the end is your individual user ID from the users database table. This is used to customize your view of admin interface, and possibly also the main site interface.
    • wp-settings-time- WordPress also sets a few wp-settings-{time}-[UID] cookies. The number on the end is your individual user ID from the users database table. This is used to customize your view of admin interface, and possibly also the main site interface.
    • __cfduid 3rd party cookie – This cookie is strictly necessary for Cloudflare’s security features
    A user can delete these cookies through browser settings.
    LAST UPDATE: 24.01.2018, LONDON
    Approve
    Refuse
    Welcome
    We use cookies to ensure that we give you the best experience on our website. Before you continue browsing you must approve or refuse our cookie policy.
    Approve
    Refuse
    Cookie Policy