Training Resources for AI Agents

Last updated: August 27, 2025

AI agents can either follow a runbook or try to answer the question in deflection mode. For deflection mode (most requests/issues that the ai agent resolves will likely fall into this bucket) a built out knowledge base / documentation is crucial. Without a KB, there is agent has "no brain" and its ability to answer questions adequately suffers.

What is training data?

Think of training data as your ai agents "brain". All the data that you provide will be used to answer customer inquiries. You can define your training data under AI Controls > Training data. The concept of training data in Pylon expands beyond AI agents (for instance also relevant for resource tracking or AI assist)

image.png

Careful: Do not include customer specific knowledge in your training data! AI Agents quote the used sources and sources are being used across all customers. Instead: use our AI enabled KCS flow to easily create KB articles.

Does the AI agent use ticket data to formulate answers?

No, we do not use ticket specific data for anything where humans might not be in the loop anymore.

Can I have training data that lives outside of Pylon?
Yes, you can add files, external KBs or docs (as long as they are public), public GitHub repos and more.

How much information is too much information?

The idea for your documentation is to have concise ideally mutual exclusive articles where the header indicates clearly what the article is about. A Q/A style structure helps as well to identify relevant content.

Best Practices

  • Use the Gaps feature in our knowledge base to create new articles. Let the AI take a first stab at this by using Generate article and review & published the article.

    • You can also define the visibility conditions of these articles. Example: You can have articles that are only visible for the AI.

image.png

  • Make sure your knowledge base is MECE

    • ME: Mutually Exclusive (makes it easier to identify the right article)

    • CE: Collectively Exhaustive (make sure there are no gaps in your KB)

  • Keep the articles short and concise and try to use wording that your customers would use.

  • Make sure to only include documentation that is actually relevant

    • Analogy: It is easier to find the right article in an pool of 100 articles than 5000 articles - the AI works just like a human brain