To find the keywords your competitors rank for, identify the three to five sites that actually outrank you for your money terms, pull the list of queries each one ranks for, subtract everything your own site already covers, then filter what remains by buying intent and difficulty. The raw list takes ten minutes with any competitor research tool. The filtering is the part that decides whether you get a publishing queue or a spreadsheet nobody opens again.
Pick the right competitors, not your business rivals
This is where most competitor keyword research goes wrong in the first five minutes. People type in the names of the companies their sales team loses deals to. Those are commercial competitors. Your search competitors are whoever occupies the results page for the terms you want, and the two lists overlap less than you would expect.
Search your five most valuable keywords and write down who is in the top five for each. In most US B2B and ecommerce categories you will find a review or roundup site, a marketplace, one or two genuine rivals, and often a large publisher that covers your topic as one section of a much bigger site. Those are your search competitors. Studying a direct rival with no organic traffic teaches you nothing; studying the review site that ranks first for your category term teaches you what the market is actually asking.
A useful filter: a site is worth studying if it ranks in the top ten for at least three of your target terms, and if its audience overlaps yours enough that its traffic could plausibly buy from you. A giant publisher that outranks you on a definition query fails the second test even though it passes the first.
Pull their ranking keywords
Every serious tool has a version of this report. Ahrefs calls it Organic Keywords, Semrush calls it Organic Research, and both will show you every query where the domain appears in the top hundred results along with position, estimated volume and the ranking URL. Export all of them for each competitor.
Two settings matter more than people realize. Restrict to the United States if that is your market, because a global export mixes in demand you cannot serve and inflates every volume figure. And keep the ranking URL column, because knowing which of their pages ranks is half the intelligence. It tells you whether they won a term with a dedicated landing page, a blog post or a product page, which tells you what you need to build to compete.
If you do not have a paid tool, there is a slower free version. Read their sitemap, list every URL, and check the ones whose titles suggest commercial intent. You will not get volume data, but you will see the shape of their coverage, which is often enough to spot a section of the market you have ignored entirely. Pulling structured page data across a few hundred competitor URLs is also something a scraping API that returns clean structured data handles in a few minutes if you would rather not click through them by hand.
Subtract your own coverage
Now combine the exports, deduplicate, and remove every keyword your site already ranks for in the top twenty. What remains is the gap list.
Pull your own ranking terms from Search Console rather than a third-party tool for this step. Search Console reports what actually happened instead of estimating, so your side of the subtraction is accurate even when the competitor side is modeled. The Search Console keyword research workflow covers how to get a clean export of that.
Expect the gap list to be long. Two to four hundred rows is normal for a mid-size site, and seeing that number is the moment most people feel productive and then stop. The list is not the deliverable. It is the input to the part that takes judgment.
Filter by intent before you filter by volume
Sort your gap list by search volume and the top of it will be almost entirely useless: broad definitional queries from people who will never buy anything, which your competitors rank for because they have the authority to pick up that traffic incidentally.
Sort by intent instead. Four bands, in descending order of what they are worth:
- Comparison and alternative terms. "X vs Y", "alternative to X", "best X for Y". The searcher is choosing a vendor right now. If a competitor owns these and you have no page, that is the most expensive gap on your list.
- Use-case and segment terms. "X for accountants", "X for small manufacturers". Moderate volume, unmistakable buyer, and usually defended by one generic page you can beat with a specific one.
- Problem-aware terms. Someone describing the symptom your product fixes without knowing a product category exists. Slower to convert, and often the least competitive band on the list.
- Definition and how-to terms. Worth covering for topical coverage and for the answer boxes, but do not confuse the traffic with demand. Put them behind the first three.
Cut anything that fails the basic question: would a person typing this ever pay us. That single filter usually removes half the list and all of the disappointment that would have followed.
Score winnability on what survives
A gap is only an opportunity if you can take it. For each survivor, check difficulty, and then actually open the results page for your top ten or fifteen candidates.
What you are looking for is weakness, and it is visible in about fifteen seconds per query. Old dates on the top results. Pages that mention the query in one paragraph rather than answering it. Forum threads and aggregator pages holding top-five positions, which usually means nobody has built a proper page for the term. Any of those beats a low difficulty score as evidence, because the score is a model and the results page is the thing itself.
The opposite signal is worth respecting too. Three strong, current, thorough pages plus an AI overview means the term is spoken for this year. Note it, skip it, and spend the quarter on something open. A keyword difficulty checker gets you to the shortlist fast; your own eyes make the final call.
Group the survivors into pages
Twenty to forty keywords will get through the filters. They do not become twenty to forty pages. Group the ones a single page could satisfy, because Google already treats many of them as the same intent, and assign one page per group with a primary term and its variants attached.
Then decide per group: new page or revision. If you already have a page ranking anywhere in the top thirty for the group, revising it beats writing something new almost every time, because the ranking signal exists and only the content needs work. If you have nothing, it is a new page, and the group tells you what shape it should take. A cluster full of comparison queries wants a comparison page, not a blog post.
Write the assignment down as a map of URL to primary keyword to variants. Without it, large sites quietly build two pages targeting the same intent six months apart and split their own rankings, which is keyword cannibalization and is entirely self-inflicted.
How often to redo it
Quarterly is right for most sites. Competitors do not add coverage fast enough to justify monthly checks, and a full year is long enough for a rival to build out a section of the market while you were not looking. Each pass takes an afternoon once the competitor list is stable, and most of that afternoon is filtering rather than gathering.
The recurring version is worth structuring the same way every time: same competitors, same intent bands, same output format, so you can see what changed rather than starting fresh. That whole loop, from competitor set to assigned pages, is what the content gap analysis instrument is built to run, and the broader competitor method sits on the competitor keyword analysis page.
If you want a starting point in the next five minutes, drop your main category term into the free demo and look at the clusters that come back. The ones you have no page for are your gap list, before you have opened a single competitor export.
Put it into practice
The fastest way to apply this article: run your own niche through the free Keywordpilot demo. One seed keyword, twenty seconds, a clustered mini content plan. No account.