Eighty-nine percent of smartphone users start with Google when they want to find something on the Internet. And yet, an average user who spends three and a half hours on their phone each day is inside apps for all but 20 minutes of that time. This is a huge disconnect, and despite the best efforts of Apple and Google, searching app content on mobile devices is often unproductive.
As a mobile app developer, this is a major problem. Fortunately, we at Branch have discovered a simple solution that uses well-established standards already supported by Google. We’re calling it App Content Sitemaps.
What are App Content Sitemaps, and why would I want one?
So why would you want to bring Google into your closed application? The answer is simple: because you want traffic, and Google search results provide that traffic. Your product doesn’t exist if a user doesn’t see your application — or content from your application — when they’re searching for a solution. In most cases, that’s why we leverage Search Engine Optimization (SEO). But SEO in apps is totally different — and increasingly important.
Doesn’t Google already do this?
No, Google does not index your app’s content automatically. Yes, the tech giant would REALLY love to do exactly that, but it’s difficult to search content inside apps. Difficult enough that the world’s leading search engine is still puzzling over this problem. Why? It has to do with how Google discovers content on the web.
Right now, Google leverages two main approaches to how it discovers content on public websites:
- Organic crawling — when Googlebot (Google’s web crawling bot) happens to stumble upon a link to your content from somewhere else.
- Sitemaps — when you intentionally submit webpages for Googlebot to index.
Both methods have something in common: they rely on Googlebot’s access to the public web. That’s a problem for app content because app content is trapped inside an app; it’s not on the public web.
Google is working to help Googlebot crawl app content through a collection of initiatives dubbed “App Indexing.” Google isn’t there yet, but it’s on its way. For now, the only real solution is to tie your pieces of content on the web to equivalent pieces of content inside your application. When you do this, you can automatically route users into your app, but only after they have it installed. In other words: Google trusts that your website, and the references leading into your app on your website, are what you promise them to be.
So what does this mean? This means you can adapt the two traditional approaches to content discovery to app content. All you have to do is to define the relationship between website and app. This is easier said than done. Indexing your app for Google is confusing and labor intensive. To top it off, Google gives you little to no feedback on the success of your efforts. Regardless, this is probably the best you can do if you want incoming traffic.
Okay, how do I create an App Content Sitemap?
If you use Branch for your deep linking, skip down about two pages. All of this work has already been done for you. Otherwise, keep reading:
Step 1: Configure your app to support deep link routing
Make sure your deep links work. If a visitor clicks on a link for a particular piece of content, then that piece of content must be delivered. Otherwise, your users will assume your app is broken.
Step 2: Create a web version of your app content
Assume every piece of content in your app needs to have a corresponding web version. Why? So Googlebot can access it. Remember: your website content (not the content inside your app) is what Googlebot indexes, so make sure it’s accurate and reflects best practices for mobile web SEO.
Step 3: Verify your site with Google
No matter what you do your website will need to be verified. This helps Google confirm your ownership of the domain and its legitimacy. The steps are easy:
- Use Google Search Console to configure a website property.
- Use one of the verification methods on this page. Important: please note the instructions marked “Verify a mobile app” on this page are not necessary for app content sitemaps in most situations. We know, counterintuitive.
Step 4: Enable App Indexing
Here’s where the magic happens: when properly configured, search result links will immediately launch your application (assuming the user has installed your app) instead of pushing them to your website. It’s a wonderful, fantastic, grand and immersive experience when it works.
You could skip this step by redirecting a user after they arrive on the web version of your content. But there are two problems with this approach: first, it’s a less user-friendly experience; second, Google won’t give you the “App Results” treatment they give to webpages that tie themselves with apps. So let’s take a look at how to do that with two of the biggest ecosystems out there: iOS and Android.
iOS configuration
When it comes to iOS, you need to familiarize yourself with Universal Links (something we recommend for all iOS apps). Universal Links can be complicated. Complicated enough that we can’t explain it here. So, if you’re not already familiar with Universal Links, you can check out our blog post explaining the whole scheme.
You can optionally also add a <link> tag in the header of each page of content on your website to indicate deep link information:
<link rel="alternate" href="ios-app://app_store_app_id/your_app_scheme/blog/post12345">
Replace app_store_app_id with the ID of your app in the Apple App Store (you can find this by going to the App Store web page for your app, and looking for the digits following id at the end of the URL. For example: https://itunes.apple.com/us/app/your_app_name/id000000000).
Android configuration
Android, by comparison, is simple. All you need to do is configure your app to respond to a custom URI scheme. We’ve already done a good deal of the work for you. Simply paste the following intent filter inside the <activity></activity> tags for the Activity you want to launch when the search result is opened:
<intent-filter> <data android_scheme="your_app_scheme"/> <action android_name="android.intent.action.VIEW" /> <category android_name="android.intent.category.DEFAULT" /> <category android_name="android.intent.category.BROWSABLE" /> </intent-filter>
Replace your_app_scheme with something of your own selection (it should be unique to your app).
Next, use a <link> tag in the header of each page of content on your website to indicate deep link information:
<link rel="alternate" href="android-app://your.package.name/your_app_scheme/blog/post12345">
In this example, we assume the following:
- Package name:
your.package.name - URI scheme:
your_app_scheme - Deep link path:
blog/post12345
Note: you may also use HTTP links instead of custom URI scheme links, as described in these steps from Google. This approach requires the extra instructions marked “Verify a mobile app” mentioned above, and may result in your users being asked to select between using your app and any installed web browsers when opening your links.
Once your website is up and App Indexing is configured, Google will (in theory) begin associating your web content with your app content in search results. However, this only works if Google can find your content in the first place. Unlike traditional websites, which have always lived in the land of easily-sharable URLs, apps are a closed system. Googlebot doesn’t easily index closed systems. But that’s fixable with App Content Sitemaps.
An App Content Sitemap allows you to manually submit an index of your pages to Google. Please note: when you do this Google will know that your app content exists, but it may not actually index your content. It’s still a valuable thing to do.
Step 5: Generate a sitemap of your site
So how do you generate a sitemap? For our purposes, it’s actually pretty difficult to do. Not only do you need to generate a large XML document, but you also need to keep it up-to-date and in sync with your app’s deep links. How you do this will depend on your system configuration (unless Branch is doing this for you), but the end result needs to look something like this:
<?xml version="1.0" encoding="UTF-8"?> <urlset > <url> <loc>https://your-website.com/path/to/awesome/content</loc> </url> <url> <loc>https://your-website.com/path/to/awesome/content-2</loc> </url> <url> <loc>https://your-website.com/path/to/awesome/content-3</loc> </url> </urlset>
We have built a handly App Indexing Validator based on what we have seen working in the wild. You can use this tool to help verify whether your configuration is working correctly.
Step 6: Upload this sitemap to Google Search Console
Now all you have to do is tell Google that you’ve a sitemap for them to use. You can upload the file directly to the Search Console, or simply add these lines to your robots.txt:
User-agent: *
Sitemap: https://your-website.com/sitemap.xml
Step 7: Wait
Last, but not least, you’ll need to wait. How Google’s search algorithm implements app content is nothing short of opaque, even at the best of times. We know that app content does appear in search results when submitted using the above process, but many of the variables at work are still unclear.
What is Branch doing to make this easier?
For Branch users, we already know all about your content and where it lives inside your app. We automatically create web placeholders for each piece of in-app content with every link your users generate. We handle the standards, the sitemaps, and the content needing to sync with Google. We even know when different links point to the same piece of content, so you don’t have to worry about multiple copies. All you need to do is check a box and Branch handles the rest.
We can help you see what’s working
Most importantly, we give you quantifiable insights into what’s happening behind the scenes. We measure sitemap generation for application content, how much unique content was included, and how many pieces of content Google actually crawled.
Our links analytics tool can help you track traffic and installs through your sitemap-submitted links with just one filter. These results are already showing up in search results and confirm that install and open analytics work as expected:
How does this work if I already have a website of my own?
Highly-ranked websites may not want bnc.it to appear in links next to their content. We’re working on it. Currently, we’re moving to integrate with custom domains and look forward to sharing an update with you soon.
The search field is changing
We mentioned eighty-nine percent of smartphone users start with Google when they want to find something on the Internet. What you probably don’t realize is that mobile device searches greatly outnumber searches from computers. That means content discovery will need to adjust, regardless of the ecosystem. We feel in-app content in search results is the next big issue Google needs to solve. While we know submitting app content via sitemaps works, something needs to change before it becomes equal to content from traditional websites with high domain scores. No matter what happens next, you can count on Branch to help.
Additional Resources
https://developers.google.com/app-indexing/?hl=en
https://support.google.com/webmasters/answer/183668
https://support.google.com/webmasters/answer/156184?hl=en&ref_topic=4581190
https://support.google.com/googleplay/android-developer/answer/6041489
https://support.google.com/webmasters/answer/139066
https://developer.android.com/intl/in/training/app-indexing/enabling-app-indexing.html