Are Knowledge Graphs the Next Step After RAG? Why Agents Need Ontology
RAG retrieves evidence, knowledge graphs connect facts, and ontology defines meaning. Explore BeSir’s ontology-first approach to connecting enterprise data and actions.
On this page
Agents need to understand business concepts, data and available actions to move from answering questions to performing work. We examine the distinct roles of RAG, knowledge graphs and ontology, and the ontology-first approach BeSir aims to bring to enterprise AI.
Why ontology matters in the age of agents
“We built RAG, but the answers are not as accurate as we expected. Would a knowledge graph solve the problem? And how is an ontology different?”
These questions come up frequently when enterprises adopt AI. Because the three concepts often appear together, they can sound like successive stages of technological progress: from RAG to knowledge graphs, then to ontology.
They serve different purposes, however. As AI expands from answering questions to performing work as an agent, how these technologies work together becomes more important than choosing a single technology.
This is BeSir’s starting point. Our approach is an ontology-first architecture: define the organization’s business concepts and relationships, then connect the data and actions needed for the work. Building a knowledge graph of all enterprise knowledge is not a prerequisite. Graphs can be used where they help solve a particular problem.
Three concepts, three distinct roles
| Concept | Role | Example in project management |
|---|---|---|
| RAG | Generate answers grounded in retrieved information | Find the reason for a delay in meeting notes and explain it with supporting evidence |
| Knowledge graph | Represent entities and the relationships between them | Connect Customer A to Project B and its owner, Employee C |
| Ontology | Define entity types, relationships and their business meaning | Define what customers, projects and owners are, and how they relate |
RAG is a way to answer using retrieved evidence. A knowledge graph is a map of facts and relationships. An ontology is a model defining what the concepts and relationships in that map mean.
These capabilities can work together. A knowledge graph can supply evidence for RAG, and an ontology can guide the structure of a knowledge graph. Approaches such as GraphRAG use graphs in retrieval and answer generation. RAG should not be treated as a synonym for vector search alone. Original RAG paper, Microsoft GraphRAG documentation
When one request spans multiple systems
Consider this request:
Find overdue projects involving key customers, and tell me who owns them and which unresolved issues were discussed in recent meetings.
Answering it requires several definitions and facts:
- What qualifies as a “key customer” in this organization?
- Which project states count as “overdue”?
- Who owns each project?
- Which meeting notes belong to the project, and which issues remain unresolved?
Customer tiers may be in the CRM, project schedules and statuses in an operational database, and meeting notes in a document repository. An issue recorded as unresolved in a meeting may since have been marked complete in the business system.
That is why finding relevant documents is not enough. Beyond similarity, the agent needs to establish whether the information concerns the same customer and project, which point in time it describes, and which system is authoritative.
Knowledge graphs are useful for representing and exploring these relationships. An agent might follow a customer-to-project relationship, continue to the project’s owner and meetings, and then examine the issues linked to those meetings. Traversing several relationships in sequence is commonly called multi-hop traversal. W3C RDF Primer
Being able to traverse relationships does not by itself ensure a sound business conclusion. If “key customer” or “unresolved” has inconsistent meanings, connected data can still lead to an incorrect judgment.
A knowledge graph introduces operational responsibilities
When data is extracted from several systems and stored in a separate knowledge graph, maintenance needs to be designed alongside the connections.
The system must recognize that “ServerKit” in the CRM and “ServerKit Co., Ltd.” in the ERP refer to the same customer. Project names in documents must be linked to actual project identifiers. Changes to owners or statuses in source systems must be reflected in the graph. The origin and update time of each relationship also need to be traceable.
A graph built this way can become a reusable asset. Extraction, entity resolution, updates and quality management also need to be included in the operating plan. Microsoft’s GraphRAG documentation provides an example of extracting a graph from documents. Microsoft GraphRAG documentation
Not every knowledge graph requires copying data into a separate repository. Virtual knowledge graphs can access source data through mappings. The design question therefore includes where data resides and how it remains current, as well as whether to use a graph. Ontop virtual knowledge graph
This leads to a question BeSir asks:
“Must an enterprise first build a single knowledge graph of all its data before it can start using agents?”
From BeSir’s perspective, that is not always necessary.
Ontology: start by defining what the data means
An ontology explicitly models the concepts and relationships that appear in business work. It describes what a customer, project, contract or employee is, and how a project relates to a customer. This is a different role from recording the individual facts connecting particular customers and projects. W3C OWL 2 Primer
Designing an ontology and the business rules connected to it can address questions such as these:
| What to define | Example |
|---|---|
| Business concepts | A project is a unit of work associated with a customer |
| Properties and relationships | A project has a due date and status, and is linked to an owner and meetings |
| Connected business rules | An incomplete project past its due date is classified as overdue |
| Authoritative information | Customer tiers come from the CRM; project status comes from the project database |
One distinction matters: the concept of an ontology does not automatically include database connectivity or API execution. Source mappings, query tools, executable actions and access controls must be implemented and connected by the system using it.
BeSir places ontology at the center of business meaning and aims to connect that meaning to real data access and actions. The goal is for agents to use these definitions in their work, beyond simply defining terms and relationships.
BeSir’s approach: define meaning, then connect the appropriate tools
Enterprises already have systems of record. Orders and revenue may live in the ERP, customer information in the CRM, and progress in a project database. Meeting notes and manuals reside in document repositories.
BeSir’s ontology-first approach focuses on using these existing systems while connecting agents to the information and functions they need through shared business concepts.
For the earlier request, responsibilities could be divided as follows:
| Required task | Possible means |
|---|---|
| Check key customer tiers and identifiers | CRM API or database query |
| Find incomplete projects past their due dates | SQL or a project system API |
| Search recent meeting discussions | Document search scoped by project identifier |
| Explore complex relationships across projects and issues | Knowledge graph queries where useful |
| Explain the findings with evidence | Answer generation using retrieved and queried results |
| Perform a requested follow-up action | Business API execution after checking permissions and approval requirements |
SQL is a query language. An API is an interface to a system. A knowledge graph is a data representation model. RAG is a way to use retrieved evidence in answer generation. Their different roles allow them to be combined.
BeSir’s design perspective can be summarized conceptually as follows:
Business request
↓
Agent
↓
Ontology-based business meaning
+ definitions connecting data and functions
↓
Query and execution plan
↓
SQL · APIs · document search
· knowledge graph queries where needed
↓
Existing enterprise systems
Query results → evidence-based answer
Action request → permissions, validation
and required approval → execution and logging
This illustrates a design approach. The connections and actions provided in practice must be defined for the target systems and the scope of work.
From “What is a project?” to “What can we do with it?”
When a request changes from “Tell me who owns this project” to “Notify the owner about the delay,” the agent must move from retrieving information to taking action.
BeSir therefore considers the executable functions connected to a business object alongside its meaning. A project could be modeled like this:
| Area | Examples connected to a project |
|---|---|
| Properties | Status, due date, budget |
| Relationships | Customer, owner, contract, meetings, issues |
| Data connections | Project database, CRM, document repository |
| Executable functions | Query status, calculate delay risk, change the owner, send a notification |
| Execution conditions | Read and change permissions, input validation, required approval, execution logs |
These are examples of connecting business objects and functions, rather than specific API names or a list of capabilities available in every environment.
An agent’s awareness that a “change owner” function exists is separate from its permission to perform that action. Ontology provides a basis for connecting meaning and functions; the execution layer must enforce permissions and conditions.
In this sense, BeSir views ontology as central to a Semantic & Action Layer that interprets business meaning and connects it to permitted actions.
BeSir’s value must continue as the number of agents grows
The distinction BeSir emphasizes goes beyond the name of a technology. It involves jointly designing how business knowledge is structured, how it connects to existing systems and actions, and how internal teams continue to operate it.
First, turn practitioners’ explanations into business structures that agents can use.
Knowledge such as “we manage important customers separately” or “we must not move to the next stage in this state” may remain in employees’ experience rather than in documents. BeSir focuses on making this knowledge explicit as concepts and rules, drawing on domain interviews and system information. About BeSir
Second, connect understanding to execution through existing systems.
Connecting data and functions to shared business concepts can reduce repeated work when new agents are added. For example, a delay-reporting agent and a customer-response agent could use the same project definition and query function. This is the value of reuse that the ontology-first approach seeks to enable.
Third, build the organization’s capability to create and operate the next agent.
With only a few agents, external specialists may be able to build and update each one individually. Scaling to dozens or hundreds also requires deciding who will update definitions when the business changes, assess the impact and validate the results.
This is why BeSir emphasizes internal development and operational capabilities. The goal is for people who understand the work to respond to change and build subsequent agents using shared definitions and functions. BeSir’s public introduction describes support for customer teams to build follow-on agents after jointly developing the first one. About BeSir
Shared definitions do not automatically eliminate operational costs. When multiple agents use the same definitions, teams must manage change impact, versions, permissions and validation. Scaling depends on being able to verify that agents still behave as intended after a change, as well as reducing development time.
How should an agent understand an enterprise?
RAG retrieves evidence to support answers. Knowledge graphs represent and help explore relationships between entities. Ontologies define what those entities and relationships mean.
For an agent to perform work, another connection is required: link that meaning to real data, and clearly specify the functions it may perform and the conditions that govern them.
BeSir places ontology at the center of this connection. Depending on the task, the design can involve document search, database queries, graph traversal and API execution within permitted boundaries.
The question BeSir wants to start with is more specific than “Should we adopt a knowledge graph?”
“What work should the agent perform, and how are the concepts, data and actions needed to understand that work connected?”
A structure that answers this question can carry the knowledge and functions developed for the first agent into the next. Through its ontology-first approach, BeSir aims to provide a foundation that enables enterprises to expand and operate agents themselves.