How to Use Embeddings for Paid Search

You can turn a large keyword list into a topic tree, then see which topics fit your products, which you already target and where to look for new keywords. Add search volume, rankings and your own performance data, and the same tree becomes a way to report on the market around your business.

Embeddings help group phrases by meaning. You can use them for many other jobs too: finding related searches, comparing keywords or matching a search to a product. First, we’ll build the topic map. Then we’ll put it inside a Python pipeline that judges relevance, joins performance data and proposes keywords to review.

The example is an imagined running shoe brand. Road shoes and trail shoes fit its products. Marathon plans and energy gels belong on the wider map too, even if the brand decides they are outside its paid search scope.

#1 Market inputs

Your Google Ads search terms show searches that triggered your ads. Search Console adds queries where your pages appeared organically. Both give you evidence about demand you already reach.

Competitor data expands the map beyond that footprint. Collect organic keywords from relevant brands, retailers, publishers and review sites. A publisher can reveal useful topics even when it sells nothing that competes with you.

The Semrush API can supply organic keywords and observed paid keywords for a domain. Competitor paid positions are another input you could add to this approach. They give you evidence that an advertiser appeared for a search, with the observation’s date and country. They do not reveal the advertiser’s complete keyword setup, match types or profitability.

Exploding Topics adds emerging topics and growth signals. Pass them through the same grouping and relevance checks: a rising topic still needs to fit your offer.

Keep the source, country, language and observation date with each phrase. Combine evidence from several domains, but count a keyword’s demand once for the chosen market and period. Keep global trend estimates separate from country-level keyword estimates. The map covers the sources you collected.

#2 Embedding basics

An embedding represents text as a list of numbers, called a vector. Think of it as a position on a map where similar meanings sit close together.

On its own, apple could be the fruit or the company selling iPhones. The surrounding words help distinguish them. Below you can see two recorded comparisons from Gemini Embedding 2, with 3,072 numbers per phrase and plain-text inputs:

A pear you eat sits closer to an apple you eat than the Apple you work on does, even though the laptop phrase shares the word apple.

Cosine similarity compares the directions of the two vectors. It runs from -1 to 1; higher means more similar. Read it as a similarity score: 0.853 does not mean “85.3% the same meaning”. A useful cutoff depends on the model, the data and the decision.

You can use the same idea in your business. Compare searches with each other to find related topics, or with product descriptions and existing keywords to find likely matches. Clustering builds the groups; an LLM with your business context judges which searches fit your offer.

#3 Topic hierarchy

A flat set of clusters gives you themes. A tree also keeps broader and narrower topics together, so you can move from running shoes to trail shoes to waterproof trail shoes.

One way to build it combines word rules with embeddings:

1. Combine word variants: fold agreed spelling variants and abbreviations, and group phrases with the same content words. Keep review and price when they change the intent.
2. Find parents: place a phrase beneath the most specific broader phrase whose words it contains. Waterproof trail running shoes belongs beneath trail running shoes.
3. Group related branches: take each branch’s average vector and rescale it to length 1. Ward’s method then joins sibling branches using Euclidean distance, choosing the join that adds the least variation within groups. Cluster within branches to keep the problem manageable at scale.
4. Choose useful levels: make small groups that a reviewer can read, then broader subtopics and topics above them. Check that each parent describes its children.

Word containment gives you a reproducible starting structure; embeddings connect related meanings it misses. Inspect the result: a size limit can produce a tidy branch with unrelated searches. Brand names and ambiguous acronyms need separate checks too.

You can also use k-means for flat groups, or another hierarchical method. The test is whether the phrases belong together.

Keep each phrase in one reporting group within a version of the tree. Product labels can then be attached separately, including more than one product when appropriate.

#4 Python architecture

A Python pipeline connects the parts: collect keywords and search terms, embed and group them, judge relevance, check coverage and produce recommendations. SQL reads the data, prompts carry the business context, and Python calls the APIs and checks their replies.

Clean and deduplicate the inputs before embedding. Each stage saves results you can inspect and reuse, so a change to the product rules can rerun the relevance judge without collecting or embedding every phrase again.

The embedding request belongs in the stage that builds or updates the map. With Google’s google-genai SDK and Vertex AI authentication configured:

from google import genai client = genai.Client(vertexai=True, project="your-project", location="global")result = client.models.embed_content(    model="gemini-embedding-2",    contents="trail running shoes",)vector = result.embeddings[0].values

Gemini Embedding 2 returns 3,072 numbers per phrase by default. Store the phrase, vector, model and configuration together, and reuse compatible vectors when the phrase is unchanged.

At a judgement step, Python loads the prompt, product context and topic rows, then requests a structured LLM reply. Validate the schema and business rules before saving it, with the topic ID, tree version, prompt and model version. An unreadable reply stays an error to retry. Run logs, rate limits and spending caps keep scheduled runs inspectable and bounded.

#5 Product relevance

Once the groups exist, give an LLM your product descriptions, exclusions and account rules. Ask it to judge each small group using the phrases inside it, rather than relying on the group’s name.

Ask for relevance, product category, commercial intent and a reason for the verdict as separate fields.

For the imagined brand, waterproof trail shoes might map to its trail range. Energy gels might be outside its offer. A marathon plan could describe a relevant audience while still being unsuitable for a campaign selling shoes. Those are business decisions the context must explain.

Relevance and product fit are separate fields. A group can be relevant to the business and routed to the wrong product, so a reviewer needs to be able to correct its destination without rejecting the topic itself.

Keep out-of-scope branches visible with their verdicts. For opportunity review, show the relevant groups first.

An approved group supplies candidates. Check each final keyword’s product fit and intent individually, and judge a changed group again when a rebuild changes what it contains.

#6 Coverage gaps

With the relevant topics identified, compare them with your account. Three measures answer different questions:

A phrase can be absent from your keyword list while an existing broad keyword already reaches it. It can also be present in your setup and receive no impressions during the period. Keep those states separate in the report.

Start with word checks, then compare the remaining candidates with the closest live keywords by meaning. A simple distance score is:

newness = 1 - closest_similarity

With cosine similarity, this score runs from 0 to 2; higher means further from the nearest keyword. Calibrate the cutoff on covered searches and topics you deliberately leave out. Show the nearest keyword and its match type beside the score. Score each phrase before summarising a topic: an average vector can hide members already covered.

Include exact match in the design deliberately. Google’s exact match also reaches searches with the same meaning or intent. A comparison using only phrase and broad keywords can therefore call a candidate new even when an exact keyword reaches it.

Check negatives and past decisions too. A negative’s effect depends on its match type and where it applies. A rejected destination should remain distinguishable from a rejected keyword, so the next run can suggest a better place.

Google withholds some low-volume queries from the search terms report. Label an empty result as “no reported ads” for that period and account scope. It is a candidate to investigate; the report cannot establish that your ads never reached it.

#7 Topic reporting

The same groups organise performance reporting. Map reported search terms to topic IDs, then join impressions, clicks, cost and conversions. Implemented keywords can retain topic IDs for keyword-level reporting too.

Search-term reporting shows which topics triggered your ads. Keyword reporting follows the assigned keywords, including searches they matched beyond that topic’s wording. Label the view and keep its data separate.

For trail shoes, inspect demand, organic rankings by domain and targeting beside cost and conversions. A relevant branch with little reported ad activity becomes a research candidate. One already spending money needs a performance decision too.

Keep demand estimates and account metrics as separate columns. A reported impression does not mean you captured a phrase’s estimated search volume. The share of mapped demand attached to phrases with reported ads is not impression share.

Aggregate each source to the intended reporting grain before joining. If the report needs one row per phrase, date and market, sum account rows to that level first. Several competitor pages or product labels must not multiply spend. Parent totals must match their children and the source data. Unassigned terms keep their cost visible.

Calculate CPA and ROAS from summed components, rather than averaging child ratios. Show reporting dates and data freshness, especially when conversions arrive later than clicks.

Keep the monthly trend beside the annual average: a short burst can inflate it. Compare like periods. Volume helps order the research; relevance and measured performance decide what happens next.

#8 Keyword recommendations

The map now gives you a shortlist of relevant topics with potential gaps. For each one, ask an LLM to select keywords from the phrases you collected, with a reason for each choice and refusal.

For broad match, ask for a small set representing the useful searches. Show those phrases beside each pick so a reviewer can see when it has become too broad. Test the number of picks and intent cutoff against your own decisions.

Route each pick to a product, campaign and ad group. Use an existing ad group when it fits, or propose a new one for a distinct job. Recheck live keywords, applicable negatives and review decisions at that destination before showing the recommendation.

A review card carries the topic path, product fit, represented searches, nearest live keywords and performance evidence. A human approves, rejects, defers or corrects the placement. Store those as different outcomes.

Retain source and topic IDs when approved keywords go live, so the report can follow their performance across accounts. An incremental-lift claim needs a separate test.

#9 Tests and feedback

Test the map at each level. Read small keyword groups and the broader branches above them. Sample rejected groups as well as accepted ones, because a false rejection otherwise disappears before a human sees it.

Test relevance, keyword selection and placement separately. “Wrong product”, “too broad” and “wrong ad group” point to different fixes. Build an eval system around those errors: save corrected examples with expected answers, then check each prompt change against them.

Keep a fixed answer set for coverage checks. Rewordings of covered searches should stay covered. A topic removed from the setup, including any broader keyword still covering it, should become new. Repeat LLM comparisons across several runs: agreement measures consistency, and a consistent answer can still be wrong.

The reporting checks need no model: every phrase belongs once, parent totals add up, joins preserve source totals and unknown assignments stay visible. Version the tree and its mappings so a regrouping cannot silently rewrite a historical comparison.

Negative Keyword Automation with AI covers the feedback loop in detail. Agentic Coding for PPC covers building and testing it with a coding agent.

Start with one market and a sample you can inspect. Automate collection, grouping, classification and the report once those checks hold. As new sources add phrases, the next review can show what changed: new topics, new searches within known topics and candidates for the account.