Find Keyword Cannibalization Using OpenAI’s Text Embeddings with Examples

This new collection of articles focuses on working with LLMs to scale your website positioning duties. We hope that can assist you combine AI into website positioning so you may degree up your expertise.

We hope you loved the previous article and perceive what vectors, vector distance, and text embeddings are.

Following this, it’s time to flex your “AI information muscle mass” by studying the way to use textual content embeddings to seek out keyword cannibalization.

We’ll begin with OpenAI’s textual content embeddings and evaluate them.

(*tokens will be thought of as phrases phrases.)

However earlier than we begin, it is advisable to set up Python and Jupyter in your pc.

Jupyter is a web-based software for professionals and researchers. It permits you to carry out advanced knowledge evaluation and machine studying mannequin improvement utilizing any programming language.

Don’t fear – it’s very easy and takes little time to complete the installations. And bear in mind, ChatGPT is your friend relating to programming.

In a nutshell:

• Obtain and install Python.
• Open your Home windows command line or terminal on Mac.
• Kind this instructions pip set up jupyterlab and pip set up pocket book
• Run Jupiter by this command: jupyter lab

We’ll use Jupyter to experiment with textual content embeddings; you’ll see how enjoyable it’s to work with!

However earlier than we begin, it’s essential to join OpenAI’s API and arrange billing by filling your steadiness.

When you’ve performed that, arrange electronic mail notifications to tell you when your spending exceeds a specific amount beneath Utilization limits.

Then, get hold of API keys beneath Dashboard > API keys, which you must preserve personal and by no means share publicly.

OpenAI API keys

Now, you may have all the required instruments to begin taking part in with embeddings.

• Open your pc command terminal and sort jupyter lab.
• You need to see one thing just like the under picture pop up in your browser.
• Click on on Python 3 beneath Pocket book.

jupyter lab

Within the opened window, you’ll write your code.

As a small job, let’s group related URLs from a CSV. The sample CSV has two columns: URL and Title. Our script’s job can be to group URLs with related semantic meanings primarily based on the title so we will consolidate these pages into one and repair key phrase cannibalization points.

Listed here are the steps it is advisable to do:

Set up required Python libraries with the next instructions in your PC’s terminal (or in Jupyter pocket book)

pip set up pandas openai scikit-learn numpy unidecode

The ‘openai’ library is required to work together with the OpenAI API to get embeddings, and ‘pandas’ is used for knowledge manipulation and dealing with CSV file operations.

The ‘scikit-learn’ library is critical for calculating cosine similarity, and ‘numpy’ is important for numerical operations and dealing with arrays. Lastly, unidecode is used to scrub textual content.

Then, obtain the sample sheet as a CSV, rename the file to pages.csv, and add it to your Jupyter folder the place your script is situated.

Set your OpenAI API key to the important thing you obtained within the step above, and copy-paste the code under into the pocket book.

Run the code by clicking the play triangle icon on the high of the pocket book.

import pandas as pd
import openai
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
import csv
from unidecode import unidecode

# Perform to scrub textual content
def clean_text(textual content: str) -> str:
    # First, substitute identified problematic characters with their appropriate equivalents
    replacements = {
        '–': '–',   # en sprint
        '’': '’',   # proper single citation mark
        '“': '“',   # left double citation mark
        '”': '”',   # proper double citation mark
        '‘': '‘',   # left single citation mark
        'â€': '—'     # em sprint
    }
    for outdated, new in replacements.gadgets():
        textual content = textual content.substitute(outdated, new)
    # Then, use unidecode to transliterate any remaining problematic Unicode characters
    textual content = unidecode(textual content)
    return textual content

# Load the CSV file with UTF-8 encoding from root folder of Jupiter venture folder
df = pd.read_csv('pages.csv', encoding='utf-8')

# Clear the 'Title' column to take away undesirable symbols
df['Title'] = df['Title'].apply(clean_text)

# Set your OpenAI API key
openai.api_key = 'your-api-key-goes-here'

# Perform to get embeddings
def get_embedding(textual content):
    response = openai.Embedding.create(enter=[text], engine="text-embedding-ada-002")
    return response['data'][0]['embedding']

# Generate embeddings for all titles
df['embedding'] = df['Title'].apply(get_embedding)

# Create a matrix of embeddings
embedding_matrix = np.vstack(df['embedding'].values)

# Compute cosine similarity matrix
similarity_matrix = cosine_similarity(embedding_matrix)

# Outline similarity threshold
similarity_threshold = 0.9  # since threshold is 0.1 for dissimilarity

# Create a listing to retailer teams
teams = []

# Hold observe of visited indices
visited = set()

# Group related titles primarily based on the similarity matrix
for i in vary(len(similarity_matrix)):
    if i not in visited:
        # Discover all related titles
        similar_indices = np.the place(similarity_matrix[i] >= similarity_threshold)[0]
        
        # Log comparisons
        print(f"nChecking similarity for '{df.iloc[i]['Title']}' (Index {i}):")
        print("-" * 50)
        for j in vary(len(similarity_matrix)):
            if i != j:  # Be certain that a title shouldn't be in contrast with itself
                similarity_value = similarity_matrix[i, j]
                comparison_result="larger" if similarity_value >= similarity_threshold else 'much less'
                print(f"In contrast with '{df.iloc[j]['Title']}' (Index {j}): similarity = {similarity_value:.4f} ({comparison_result} than threshold)")

        # Add these indices to visited
        visited.replace(similar_indices)
        # Add the group to the listing
        group = df.iloc[similar_indices][['URL', 'Title']].to_dict('information')
        teams.append(group)
        print(f"nFormed Group {len(teams)}:")
        for merchandise in group:
            print(f"  - URL: {merchandise['URL']}, Title: {merchandise['Title']}")

# Verify if teams had been created
if not teams:
    print("No teams had been created.")

# Outline the output CSV file
output_file="grouped_pages.csv"

# Write the outcomes to the CSV file with UTF-8 encoding
with open(output_file, 'w', newline="", encoding='utf-8') as csvfile:
    fieldnames = ['Group', 'URL', 'Title']
    author = csv.DictWriter(csvfile, fieldnames=fieldnames)
    
    author.writeheader()
    for group_index, group in enumerate(teams, begin=1):
        for web page in group:
            cleaned_title = clean_text(web page['Title'])  # Guarantee no undesirable symbols within the output
            author.writerow({'Group': group_index, 'URL': web page['URL'], 'Title': cleaned_title})
            print(f"Writing Group {group_index}, URL: {web page['URL']}, Title: {cleaned_title}")

print(f"Output written to {output_file}")

This code reads a CSV file, ‘pages.csv,’ containing titles and URLs, which you’ll be able to simply export out of your CMS or get by crawling a shopper web site utilizing Screaming Frog.

Then, it cleans the titles from non-UTF characters, generates embedding vectors for every title utilizing OpenAI’s API, calculates the similarity between the titles, teams related titles collectively, and writes the grouped outcomes to a brand new CSV file, ‘grouped_pages.csv.’

Within the key phrase cannibalization job, we use a similarity threshold of 0.9, which implies if cosine similarity is lower than 0.9, we’ll take into account articles as totally different. To visualise this in a simplified two-dimensional house, it’s going to seem as two vectors with an angle of roughly 25 levels between them.

In your case, you might wish to use a unique threshold, like 0.85 (roughly 31 levels between them), and run it on a pattern of your knowledge to judge the outcomes and the general high quality of matches. Whether it is unsatisfactory, you may enhance the brink to make it extra strict for higher precision.

You may set up ‘matplotlib’ by way of terminal.

And use the Python code under in a separate Jupyter pocket book to visualise cosine similarities in two-dimensional house by yourself. Strive it; it’s enjoyable!

import matplotlib.pyplot as plt
import numpy as np

# Outline the angle for cosine similarity of 0.9. Change right here to your required worth. 
theta = np.arccos(0.9)

# Outline the vectors
u = np.array([1, 0])
v = np.array([np.cos(theta), np.sin(theta)])

# Outline the 45 diploma rotation matrix
rotation_matrix = np.array([
    [np.cos(np.pi/4), -np.sin(np.pi/4)],
    [np.sin(np.pi/4), np.cos(np.pi/4)]
])

# Apply the rotation to each vectors
u_rotated = np.dot(rotation_matrix, u)
v_rotated = np.dot(rotation_matrix, v)

# Plotting the vectors
plt.determine()
plt.quiver(0, 0, u_rotated[0], u_rotated[1], angles="xy", scale_units="xy", scale=1, coloration="r")
plt.quiver(0, 0, v_rotated[0], v_rotated[1], angles="xy", scale_units="xy", scale=1, coloration="b")

# Setting the plot limits to solely optimistic ranges
plt.xlim(0, 1.5)
plt.ylim(0, 1.5)

# Including labels and grid
plt.xlabel('X-axis')
plt.ylabel('Y-axis')
plt.grid(True)
plt.title('Visualization of Vectors with Cosine Similarity of 0.9')

# Present the plot
plt.present()

I normally use 0.9 and better for figuring out key phrase cannibalization points, however you might must set it to 0.5 when coping with outdated article redirects, as outdated articles might not have almost equivalent articles which might be brisker however partially shut.

It could even be higher to have the meta description concatenated with the title in case of redirects, along with the title.

So, it will depend on the duty you’re performing. We’ll assessment the way to implement redirects in a separate article later on this collection.

Now, let’s assessment the outcomes with the three fashions talked about above and see how they had been in a position to establish shut articles from our knowledge pattern from Search Engine Journal’s articles.

Information Pattern

From the listing, we already see that the 2nd and 4th articles cowl the identical matter on ‘meta tags.’ The articles within the fifth and seventh rows are just about the identical – discussing the significance of H1 tags in website positioning – and will be merged.

The article within the third row doesn’t have any similarities with any of the articles within the listing however has widespread phrases like “Tag” or “website positioning.”

The article within the sixth row is once more about H1, however not precisely the identical as H1’s significance to website positioning. As an alternative, it represents Google’s opinion on whether or not they need to match.

Articles on the eighth and ninth rows are fairly shut however nonetheless totally different; they are often mixed.

text-embedding-ada-002

Through the use of ‘text-embedding-ada-002,’ we exactly discovered the 2nd and 4th articles with a cosine similarity of 0.92 and the fifth and seventh articles with a similarity of 0.91.

Screenshot from Jupyter log displaying cosine similarities

And it generated output with grouped URLs through the use of the identical group quantity for related articles. (colours are utilized manually for visualization functions).

Output sheet with grouped URLs

For the 2nd and third articles, which have widespread phrases “Tag” and “website positioning” however are unrelated, the cosine similarity was 0.86. This reveals why a excessive similarity threshold of 0.9 or larger is critical. If we set it to 0.85, it might be stuffed with false positives and will recommend merging unrelated articles.

text-embedding-3-small

Through the use of ‘text-embedding-3-small,’ fairly surprisingly, it didn’t discover any matches per our similarity threshold of 0.9 or larger.

For the 2nd and 4th articles, cosine similarity was 0.76, and for the fifth and seventh articles, with similarity 0.77.

To higher perceive this mannequin by experimentation, I’ve added a barely modified model of the first row with ’15’ vs. ’14’ to the pattern.

1. “14 Most Necessary Meta And HTML Tags You Want To Know For website positioning”
2. “15 Most Necessary Meta And HTML Tags You Want To Know For website positioning”

An instance which reveals text-embedding-3-small outcomes

Quite the opposite, ‘text-embedding-ada-002’ gave 0.98 cosine similarity between these variations.

Right here, we see that this mannequin shouldn’t be fairly match for evaluating titles.

text-embedding-3-large

This mannequin’s dimensionality is 3072, which is 2 instances larger than that of ‘text-embedding-3-small’ and ‘text-embedding-ada-002′, with 1536 dimensionality.

Because it has extra dimensions than the opposite fashions, we may anticipate it to seize semantic that means with larger precision.

Nonetheless, it gave the 2nd and 4th articles cosine similarity of 0.70 and the fifth and seventh articles similarity of 0.75.

I’ve examined it once more with barely modified variations of the primary article with ’15’ vs. ’14’ and with out ‘Most Necessary’ within the title.

1. “14 Most Necessary Meta And HTML Tags You Want To Know For website positioning”
2. “15 Most Necessary Meta And HTML Tags You Want To Know For website positioning”
3. “14 Meta And HTML Tags You Want To Know For website positioning”

So we will see that ‘text-embedding-3-large’ is underperforming in comparison with ‘text-embedding-ada-002’ once we calculate cosine similarities between titles.

I wish to be aware that the accuracy of ‘text-embedding-3-large’ will increase with the size of the textual content, however ‘text-embedding-ada-002’ nonetheless performs higher total.

One other strategy could possibly be to strip away cease phrases from the textual content. Eradicating these can generally assist focus the embeddings on extra significant phrases, probably enhancing the accuracy of duties like similarity calculations.

One of the best ways to find out whether or not eradicating cease phrases improves accuracy in your particular job and dataset is to empirically check each approaches and evaluate the outcomes.

Conclusion

With these examples, you may have realized the way to work with OpenAI’s embedding fashions and may already carry out a variety of duties.

For similarity thresholds, it is advisable to experiment with your personal datasets and see which thresholds make sense in your particular job by operating it on smaller samples of knowledge and performing a human assessment of the output.

Please be aware that the code we have now on this article shouldn’t be optimum for big datasets since it is advisable to create textual content embeddings of articles each time there’s a change in your dataset to judge in opposition to different rows.

To make it environment friendly, we should use vector databases and retailer embedding info there as soon as generated. We’ll cowl the way to use vector databases very quickly and alter the code pattern right here to make use of a vector database.

Extra assets: 

Featured Picture: BestForBest/Shutterstock