Make Your Academy Content Discoverable by AI
This article helps admins make their academy readable by AI assistants and control AI crawler access in Skilljar.
Overview
AI Discoverability makes your public academy content available to AI assistants and search engines in a machine-readable format. It also lets you control whether AI crawlers can index your content.
AI assistants rely on content they can access and interpret when generating responses. Enabling AI Discoverability helps AI assistants understand your public catalog, including your courses, learning paths, and course series. This increases the possibility that your academy content can be used as a source in AI-generated responses, with links back to your academy.
You control AI Discoverability at the domain level and can turn on or off AI crawler indexing based on your organization's requirements.
Enabling AI Discoverability does the following:
- Makes course metadata machine-readable, helping AI assistants identify that a page is a course, who offers it, what it teaches, and whether enrollment is required.
- Lets you control AI crawler indexing for your academy, rather than relying on a default setting.
- Makes your public academy content a potential source forAI-generated answers, with links back to your academy.
Prerequisites
Before you enable AI Discoverability, confirm the following:
- Your academy must be public. Private academies return a 404 error and are not available to AI tools.
- Each course you want AI tools to find is published, set to allow search engine indexing, and visible on your catalog.
How AI Discoverability Works
AI Discoverability controls the following three files that determine how crawlers and AI tools read your academy:
| File | What It Does |
|---|---|
| robots.txt | Tells crawlers where they can and cannot go. |
| llms.txt | Lists your course catalog in a format built for AI tools. |
| sitemap.xml | Provides a standard list of pages for search engines. |
robots.txt
Every academy already has a robots.txt file that Skilljar generates automatically. By default, the file restricts crawling of login, checkout, and other specified paths while allowing access to other public content:
User-agent: *
Disallow: /accounts/
Disallow: /auth/
Disallow: /checkout/
Disallow: /invite/
Disallow: /signup
Disallow: /login
When AI Discoverability is turned off, Skilljar adds a Disallow: / block for 14 known AI crawlers, such as GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and Bytespider. Each informs the crawler to stop reaching for any content on the academy.
When AI Discoverability is turned on, Skilljar removes these AI crawler restrictions from the robots.txt file.
llms.txt
Skilljar generates an llms.txt file. You do not create or maintain this file. Skilljar refreshes it when you publish or update a course.
The llms.txt file contains your academy name and the title, description, and link for each public course, learning path, and course series. It does not include lesson content, video transcripts, quiz questions, or SCORM files, so AI tools learn what training you offer, not what is inside it. For private Academies, requests to llms.txt return a 404 error..
A course appears in the file when it is live, set to allow search engine indexing, and visible on your catalog. For courses, the llms.txt file uses the course short description. To improve the information available to AI tools, review and update this field as needed.
Placeholder and divider courses are published courses, and therefore appear in llms.txt alongside your real courses. To exclude them, unpublish them or remove them from your catalog.
sitemap.xml
Your academy may already serve a sitemap available at /sitemap.xml. Unlike llms.txt, Skilljar does not generate the sitemap from your catalog, so it does not update when you publish or update a course and can become outdated.
While AI Discoverability is turned off, /sitemap.xml returns a 404 error for all requests, including Google searches. If you rely on your sitemap for SEO, enable AI Discoverability for the relevant domains.
Enable AI Discoverability
Enable AI Discoverability for each domain you want AI tools to index. Turning it on removes the AI-specific restrictions from robots.txt for that domain.
To enable AI Discoverability:
- Open your Skilljar Dashboard.
- Expand the Domain & Publishing header on the left, and select Domains.
- Select Domain Settings for the domain you want to enable.
- Navigate to the AI Discoverability section.
- Select the Allow AI crawlers to index this academy checkbox.
AI Discoverability is configured separately for each domain.
Once the Allow AI crawlers to index this academy checkbox is selected, AI crawlers can index the domain, and Skilljar removes the AI-specific restrictions from robots.txt.
Verify Your Configuration
After you enable AI Discoverability, verify that the relevant files are available and that your course pages expose data correctly.
Your academy sits behind a Content Delivery Network (CDN) that blocks bots that cannot prove their identity. A crawler that claims to be GPTBot from an unrecognized address is refused. Real crawlers from Google, Bing, and the major AI companies verify themselves by IP address, which is why your academy already appears in Google.
Some tests return a 403 Forbidden error even when the configuration is correct. To get reliable results, use the following recommended testing methods:
To verify your configuration:
- Open the following URLs directly in your browser. If the files load successfully, they are being served correctly.
- <your-academy-domain>/robots.txt
- <your-academy-domain>/llms.txt
- <your-academy-domain>/sitemap.xml
- Enter a public course URL in the Google Rich Results Test to verify its structured data. The tool accesses the page as a verified crawler.
Do not test with curl or a browser extension that spoofs a crawler user agent. The CDN blocks unverified bots and returns a 403 error. This result is expected and does not indicate a problem with your configuration.
What to Expect
After you enable AI Discoverability, consider the following :
- Content discovery can take time. AI crawlers may take several days or weeks to discover and process your academy content. Skilljar cannot control or accelerate this process.
- Citation is not guaranteed. AI Discoverability makes your content available to AI tools, but each AI provider determines whether to use or cite it in generated responses.
- Keep your catalog content up to date. AI tools favor recently updated content, so an actively maintained catalog performs better than a stale one.
Frequently Asked Questions
The following questions address common questions about AI Discoverability.
Will my training appear in ChatGPT?
Possibly. AI Discoverability makes your content available to AI crawlers but does not guarantee that AI assistants will use or cite it.
Do I need to change anything about your courses?
No. However, it is recommended to keep your course titles and short descriptions clear, accurate, and up to date to produce better AI answers.
Can you exclude one specific course?
There is no per-course AI setting. To exclude a course from llms.txt, unpublish it or remove it from your catalog.
Can you see which AI bots visited?
No. Skilljar does not report which AI crawlers visited your academy.