Continuously staying on top of the competition is important in any industry and web scraping publicly available data plays a huge role in getting the right data for analysis and comparison to your own strategy and position in the marketplace. This is especially true for ecommerce companies, but the following techniques can apply to any industry.
Despite massive growth and an ultra-competitive landscape, there's still huge opportunity in the ecommerce marketplace. Total ecommerce spend per customer last year was estimated at $1,800 in the U.S. and $1,600 in the UK. And it's not just consumers driving growth in the online marketplace: the B2B online retail market is expected to grow to double the size of the B2C online market, reaching $6.7 trillion USD in revenue by 2020.
But steadily building up your ecommerce business, attracting a loyal audience and customer base, and diligently measuring your own data aren't going to be enough to drive success. Without knowing what your competitors are up to, you won't be able to gain the deeper insights into the marketplace that you need to compete.
A robust competitor analysis is a necessity for any successful business, and ecommerce is no different. In fact, the necessity actually increases because you're literally competing with other portals in your city, state, country, and the world at large.
But understanding the importance of performing an ecommerce competitor analysis and knowing how to do one are two different things. Fortunately, web scraping makes it easy to get the data you need for a comprehensive look at your competitors - and discover what you need to do to measure up.
What is Web Scraping?
You probably don't need statistics to know that there's a massive amount of data out there, but here's one that's still shocking: research group IDC predicts that the world will be creating 163 zettabytes (one trillion gigabytes) of data a year by 2025.
Having so much data available about your competitors is obviously a huge advantage, but that kind of volume also makes it harder to determine what's actually important. That's where web scraping comes in. Web scraping, which is the process of extracting data from websites, is one of the easiest ways to immediately start auditing your ecommerce competition, letting you quickly and affordably collect data on your rivals.
You could go website to website and copy and paste data or you could automate that function. Here's how to use web scraping to perform an ecommerce competitor analysis.
Audit Their SEO Strategy
Instead of envying the popularity of a competitor's ecommerce business and wondering how they do so well in the marketplace, audit their SEO strategy. Use SEO crawling software like BeamUsUp to uncover popular keywords and audit landing pages to get more insight into which keywords and links your competitors are using.
In addition to uncovering what your competitors are doing right, your SEO crawl could also shed some light on what your competitors are doing wrong. Look at the keywords they're ranking poorly for on Google or which keywords they're largely ignoring. That's your chance to create an SEO campaign built around dominating where your ecommerce competitors are failing.
Review Their PPC Spend
Although you can't find out your competitors' actual budget spend for PPC, you can get a good idea of how much they're likely budgeting, and discover their most profitable keywords and paid ads with SpyFu. The biggest advantage of this technique is that you'll never have to guess which keywords will work best - and which ones to avoid - in your own PPC spend.
SpyFu can show you every keyword your competitors have bought in AdWords, as well as ad variations over the last 10+ years. Once you have more insight into the keyword and PPC AdWord strategies your competitors are using, you can enter your own website into SpyFu and get feedback. It takes a look at your competitors' bidding history and gives you recommendations on the top profitable keywords you should be buying.
Dig Into Product Pricing
Customers have instant access to tons of data. Comparing prices and products takes them mere seconds, so you'd better make sure that you measure up to your competition. Sure, your team could simply click on a competitor's site and see what they're charging for a similar product, but that method doesn't offer very deep insights into their real-time pricing strategies.
But when you use a web scraping tool, you automatically receive updated information on what your competitors are charging, and when. You may discover your competitors launch flash sales on certain days of the week, or bundle their products to drive more sales. As you scrape together pricing data, you can make timely decisions on how to run your own sales, and which products to introduce or bundle that could potentially drive more sales and keep pace with your competition.
Web scraping also helps you keep your finger on the pulse of the platforms your competitors are selling on, like Amazon. The ecommerce juggernaut now accounts for about half of all online retail sales, so monitoring Amazon data could serve as a key competitive differentiator for your business. Knowing exactly where your competitors stand on pricing ensures your own strategy yields maximum sales and profits. It also allows you to keep an eye on your competitors' rankings so you can better understand what potential customers are looking for.
Audit Their Product Line and Categories
Have you ever wondered how your competitors seem to have the best products and categories while you scramble to source vendors and wholesalers? Gain invaluable insights by scraping through your competitors' product lines and categories to see what they're using, benefits and features, and how they're bundled together.
As you scrape through multiple competitors' products, you'll also be able to see similarities and trends in the marketplace. You may discover that a specific style or color is more popular than another and can adjust your own products accordingly. Here's where you can scrape a second time. Once you identify which vendors and wholesalers your competitors are using, you can scrape through their pages to get real-time pricing. This ensures that your own product pricing is as competitive as possible to attract budget-savvy customers.
Collect Reviews and Comments From Third-Party Sites
Some 88% of people say they trust online reviews as much as personal recommendations, and 39% read reviews on a regular basis. You're probably already paying attention to what's going on with your own social media channels, but wouldn't you also like to see what your competitors' customers are thinking, and find out what they like and don't like about your competitors' products - and then use that information to dominate the market?
Fortunately, there's an easy way to eavesdrop on your competitors' customers. Go straight to the source by scraping third-party sites, review sites, social media channels, and community forums to collect feedback on what customers love - and hate - about your competitors.
Look for commonalities among comments. Do customers like your competitors' service, quality, or speed of delivery? Whatever it is that they like, focus on those same things in your own business to deliver more of what customers want the most.
At the same time, it's perhaps even more imperative to pay attention to negative reviews. Customers who complain about packaging or damaged products would likely respond well to your company's careful attention to detail. Your business could also weave relevant statements into your copy, such as, “We carefully pack each item so it arrives in pristine condition in a keepsake box you'll love.” Offering a guarantee or warranty or small item as a free upgrade can also go a long way towards helping you outshine your competitors.
Deep Dive Into Their Social Media Strategy
We already talked about how important social media reviews are to the buying process and how much they can impact the marketplace. But there's more to glean from social media than just reviews and word-of-mouth referrals. How often your competitors engage with their audience, what types of sales they offer, and how well they communicate with their customers can all give clues and insights into how to create an effective social media strategy.
Scrape through your competitors' followers to target and engage with them. Or, collect insights on how many times a day your competitors post, what they're offering, and any visuals or graphics they're using to foster engagement. The more information you can gather about their social media strategy, the more you can incorporate those winning techniques into your own campaign.
There's another way your competitors are likely using social media that reaches beyond customer engagement: the Facebook, Instagram, LinkedIn, and Twitter ads they're using are likely attracting a broader audience. When you can see which ads your competitors are using on a regular basis, you can mimic their strategies to convert a new group of customers for yourself.
Study Their Funding
If you're wondering if your competitors are profitable on their own merits or have funding and venture capital behind them, there's a simple way to find out. Scrape through sites like Crunchbase and AngelList to find out exactly what type of funding your ecommerce competitors have received, how much, and what stage they're in. As you study the information, you may also be able to identify where this funding came from so you can pitch your own products or services to the same or similar investors.
Look for other ways your competitors may have received some kind of funding. Securing a business grant or an award for a business plan contest also gives an indication of how financially sound your competitors are.
There's a goldmine of information out there that can reveal exactly what your competition is doing; you just have to tap into it. You can easily automate the entire process with web scraping tools like Import.io to maximize your results in a fraction of the time it would take you to slog through it manually. And best of all, performing an ecommerce competitor analysis can help you stay competitive and better shape your business strategy - without spending a fortune.