If you’ve spent any time in thinking or reading about AI in the last two years, you’re probably absolutely sick of the word “agent” — everything has become an agent, even if it’s a single LLM call. The word has become overused to the point where it’s lost all meaning, just like mobile-native, web 3, and cloud before it. (In the pre-LLM days, we once had a customer ask us about an “on-premise cloud deployment!”) Unfortunately, it turns out that describing your product as an agent is actually a very useful thing to do as well, and we’re introducing the concept of an agent in RunLLM v2. Since we’re giving into the hype, we thought we should defend our choice. What is an agent anyway, why have we decided to adopt it now, and what does this mean for you?
What is an agent, anyway?
Putting aside all the hype, let’s start with a dictionary (ChatGPT, actually) definition of an agent:
agent (noun, AI context):
A software entity or system capable of perceiving its environment, processing information, and autonomously taking actions—often to achieve specific goals or tasks—based on rules, learned behaviors, or decision-making algorithms.
A few things are important in this definition. First, an agent is responsive to the context (”environment”) surrounding it; LLMs excel at responding to their inputs but are also reliant on user input to provide sufficient context. Second, it can process the information its given in order to inform it’s action — this is where LLMs are good without any reservations. Finally, and most importantly, the system has to be capable of autonomously taking actions to solve a problem — simply generating an output with an LLM call based on an input is not sufficient.
Practically, what that means is that most LLM applications that you use today are not agentic. That’s okay — it’s not a criticism in the slightest. There are plenty of extremely useful things you can build by taking the right kind of context and asking an LLM to process it. Even an LLM call that takes a piece of text and translates it into an API call that’s executed by a Python function just barely qualifies as an agent.
At the same time, it also means that there are systems that may seem to be similar to traditional chatbots that are truly agentic. o3 is a great example — between it’s ability to understand your question, search the web, run code, and analyze what it finds, it gives you answers to questions that feel a distinct step above anything else out there. (Here’s a random example of something we tried recently, and we were quite impressed by the quality of the results.)
All of this is to say that building agentic systems is extremely valuable when done well — but there’s plenty of value in regular LLM usage, and tossing an LLM Into your app simply qualify as building an agent.
Why we’re introducing Agents in RunLLM
RunLLM’s been on the border of this definition for a while now. For 18+ months, we’ve had a multi-LLM system that makes decisions about whether to answer a question, which data sources to use, when to search the web, and how to modulate its tone based on its confidence. But we refrained from calling it an agent because we weren’t really taking actions on behalf of the user. For uninteresting reasons of product evolution, we also weren’t presenting any of the complexity under the hood to our customers either.
Our general inclination is to stay way from buzzwords, so why are we adding the concept of an agent now? The short answer is that it’s what customers have been asking for — not the buzzwords, but the substance.
There’s a few key initiatives we’re working on that will make RunLLM a truly agentic system. First, we’re giving our customers more control over where and how RunLLM interacts with their support stack — everything from automatically tagging tickets based on their category to taking specific actions based on the type of question we get. Second, we’ve been incrementally loosening the constraints on our interaction modes (while still enforcing strong guardrails on outputs), which means that we already can or will be able to ask questions, send suggestions, execute & validate code, and update tickets with relevant metadata.
Finally — what we’re the most excited about — is that we’re opening up the ability for our system to make dynamic decisions about what data it needs. We always start with documentation and APIs because that allows us to learn the basics of a product, but we’ll now be able to pull information in from logging & telemetry systems, databases, and CRMs to aid us in understanding the user’s issue holistically and maximize our chances of giving them a successful solution.
While we don’t love throwing more buzzwords into the product, this does genuinely change the way that our system is architected, and it gives users a clearer understanding of what we do. The reality is that an “agent” sounds like it does a lot more than an “assistant” or a “chatbot” — and we think we’re finally earning that title.
What this means for you
The short answer is that it probably doesn’t really mean anything for you. This is, first and foremost, a semantic argument for what should be called an agent and what shouldn’t — but we have academic backgrounds, so we love engaging in somewhat tedious semantic debates!
At the end of the day, a useful product will be useful for you and snake oil will be snake oil — whether one’s an agent and the other’s a chatbot doesn’t really matter. However, for anyone who’s building an AI product, you should be thoughtful about how you’re setting expectations with customers and — maybe most importantly — careful to make sure that you don’t lose customer trust.
