Before you start: if you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we suggest you start here or request a demo of SplitSignal.
First, we asked our Twitter followers to vote:
Before we jump into the result, let’s see what our SEO experts from the Semrush All-Stars Facebook group have to share:
Holly Weidman, SEO Specialist at SEO is Sweet
The click-through rate for the review rating with a star will be higher because of its psychology.
A review stands for actual people having the experience already and sharing it. We as humans like to reduce our risk, and someone else’s opinion of the experience is helpful to know we are reducing that risk.
Edward Roberts, SEO Manager at Hovia
I think the variant will increase clicks as review ratings can be a strong trust signal. Plenty of research has shown ratings can increase CTR by as much as 20-30%. Although I don’t think we’ll see a big uplift in this case.
Another thing to consider is when you have SERPs with lots of similar titles, standing out and catching a user’s eye with an emoji can make a difference. This could be an effective tactic if you currently rank just outside those top positions where users naturally tend to click.
Is this better than having a review schema on the page? Probably not, but I think it’s certainly worth testing and one I might try myself!
If you want to appear in the next blog post, follow us on Twitter and share your comment. Hate Twitter? No problem! Find us on Reddit.
The Case Study
We all love a good emoji, but can emojis really help your SEO efforts? Back in 2015, Google decided to stop showing Emoji in SERPs, mostly at least. However, in 2017 they reversed this decision but stated that they might filter them out if they deem them not to be helpful to searchers. You can often still get your Emojis to show up in SERPs.
So what does that mean for your SEO strategy? Should you be using Emojis in your title and meta? In this case study, we looked at the impact of adding a relevant Emoji into a title tag in a test group of pages on a vacation property rental website. The test included adding a star emoji accompanied by the number and average rating for a given property detail page in both the title tag and H1.
The Hypothesis
Our hypothesis for this test was that by adding a star emoji and review rating we’d capture more clicks from the SERPs by standing out and appearing like a better choice based on high ratings. This change also called attention to the reviews on the page, which leverages the psychological influence principle of social proof. The hope here is that similar to how star ratings from aggregate rating schema tend to increase CTR that this would also increase CTR.
We run another emoji SEO test previously (ð¸), read this short case study to find out the result.
The Test
We used SpitSignal to identify 56,730 pages that would be part of our test. We utilized 28,381 of those pages as the control group and then executed a change on 28,349 variation pages. To make sure changes are seen in SERPs, SplitSignal also measures whether Googlebot has seen the variant pages since launch. In this case, Googlebot visited 78% of the variation URLs. With that said, we do not know how many of these titles with emojis Google did or did not filter or rewrite but we did run some spot checks to ensure that the new titles were showing up.
Our test optimization consisted of adding the word “Reviews” and a “⭐” to both the title tag and H1. See below for an example of how this looked.
The Results Explained by Stella Rising
The test ran for 40 days from January 28th through March 8th. This title tag change had a negative outcome, resulting in 2,0957 (−5.1%) fewer clicks for the variant group compared to the control group. This test result has a 96% confidence level, indicating a strong degree of confidence in these results.
Why We Think This Test Was a Loser
The average user takes around 2 seconds to parse out the information in a SERP before making a decision on what to click first. It's possible that the presence of star emojis in the SERPs actually worked against us and led to fewer people wanting to click on the listings. When it comes to “cheap tricks” searchers are becoming more and more savvy so gimmicky things like emojis may make the page seem spammy and less desirable or less trustworthy to searchers, and thus get fewer clicks.
Due to how close this test was in terms of % change as well as other metrics we looked like CTR, we’re not ready to call this a conclusive test however it likely indicates that emojis in titles should be used with caution.
While we expected to see shifts in CTR, that wasn’t the case and both groups had a fairly similar CTR. We also did not see any major shifts in average position. This leads us to believe that the shift was really minimal and as such the results are not conclusive. This test ran on a very broad set of pages with very few visits per page. The total amount of traffic these pages received was also well below the suggested minimums for a statistically sound test. Lastly, in this case, 2 elements were simultaneously tested which also contributes to making this a bit of a toss-up.
Have your next test analyzed by SEO experts at Stella Rising.