Skip to main content

Quickstart

To best understand the agent framework, let’s build an agent that has two tools: one to look things up online, and one to look up specific data that we’ve loaded into a index.

This will assume knowledge of LLMs and retrieval so if you haven’t already explored those sections, it is recommended you do so.

Setup: LangSmith​

By definition, agents take a self-determined, input-dependent sequence of steps before returning a user-facing output. This makes debugging these systems particularly tricky, and observability particularly important. LangSmith is especially useful for such cases.

When building with LangChain, all steps will automatically be traced in LangSmith. To set up LangSmith we just need set the following environment variables:

export LANGCHAIN_TRACING_V2="true"
export LANGCHAIN_API_KEY="<your-api-key>"

Define tools​

We first need to create the tools we want to use. We will use two tools: Tavily (to search online) and then a retriever over a local index we will create

Tavily​

We have a built-in tool in LangChain to easily use Tavily search engine as tool. Note that this requires an API key - they have a free tier, but if you don’t have one or don’t want to create one, you can always ignore this step.

Once you create your API key, you will need to export that as:

export TAVILY_API_KEY="..."
from langchain_community.tools.tavily_search import TavilySearchResults
search = TavilySearchResults()
search.run("what is the weather in SF")
[{'url': 'https://weather.com/weather/tenday/l/San Francisco CA USCA0987:1:US',
  'content': 'recents Specialty Forecasts 10 Day Weather-San Francisco, CA Today Mon 18 | Day  Fri 22 Fri 22 | Day Foggy early, then partly cloudy later in the day. High around 60F. Winds W at 10 to 15 mph.  High 59F. Winds SSW at 10 to 15 mph. Chance of rain 60%.  Considerable cloudiness with occasional rain showers. High 59F. Winds SSE at 5 to 10 mph. Chance of rain 50%.San Francisco, CA 10-Day Weather Forecast - The Weather Channel | Weather.com 10 Day Weather - San Francisco, CA As of 12:09 pm PST Today 60°/ 54° 23% Tue 19 | Day 60° 23% S 12 mph More...'}]

Retriever​

We will also create a retriever over some data of our own. For a deeper explanation of each step here, see this section

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import DocArrayInMemorySearch
from langchain_openai import OpenAIEmbeddings

loader = WebBaseLoader("https://docs.smith.langchain.com/overview")
docs = loader.load()
documents = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200
).split_documents(docs)
vector = DocArrayInMemorySearch.from_documents(documents, OpenAIEmbeddings())
retriever = vector.as_retriever()
retriever.get_relevant_documents("how to upload a dataset")[0]
Document(page_content="dataset uploading.Once we have a dataset, how can we use it to test changes to a prompt or chain? The most basic approach is to run the chain over the data points and visualize the outputs. Despite technological advancements, there still is no substitute for looking at outputs by eye. Currently, running the chain over the data points needs to be done client-side. The LangSmith client makes it easy to pull down a dataset and then run a chain over them, logging the results to a new project associated with the dataset. From there, you can review them. We've made it easy to assign feedback to runs and mark them as correct or incorrect directly in the web app, displaying aggregate statistics for each test project.We also make it easier to evaluate these runs. To that end, we've added a set of evaluators to the open-source LangChain library. These evaluators can be specified when initiating a test run and will evaluate the results once the test run completes. If we‚Äôre being honest, most", metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | \uf8ffü¶úÔ∏è\uf8ffüõ†Ô∏è LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'})

Now that we have populated our index that we will do doing retrieval over, we can easily turn it into a tool (the format needed for an agent to properly use it)

from langchain.tools.retriever import create_retriever_tool
retriever_tool = create_retriever_tool(
    retriever,
    "langsmith_search",
    "Search for information about LangSmith. For any questions about LangSmith, you must use this tool!",
)

Tools​

Now that we have created both, we can create a list of tools that we will use downstream.

tools = [search, retriever_tool]

Create the agent​

Now that we have defined the tools, we can create the agent. We will be using an OpenAI Functions agent - for more information on this type of agent, as well as other options, see this guide

First, we choose the LLM we want to be guiding the agent.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

Next, we choose the prompt we want to use to guide the agent.

If you want to see the contents of this prompt and have access to LangSmith, you can go to:

https://smith.langchain.com/hub/hwchase17/openai-functions-agent

from langchain import hub

# Get the prompt to use - you can modify this!
prompt = hub.pull("hwchase17/openai-functions-agent")

Now, we can initalize the agent with the LLM, the prompt, and the tools. The agent is responsible for taking in input and deciding what actions to take. Crucially, the Agent does not execute those actions - that is done by the AgentExecutor (next step). For more information about how to think about these components, see our conceptual guide

from langchain.agents import create_openai_functions_agent

agent = create_openai_functions_agent(llm, tools, prompt)

Finally, we combine the agent (the brains) with the tools inside the AgentExecutor (which will repeatedly call the agent and execute tools). For more information about how to think about these components, see our conceptual guide

from langchain.agents import AgentExecutor

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

Run the agent​

We can now run the agent on a few queries! Note that for now, these are all stateless queries (it won’t remember previous interactions).

agent_executor.invoke({"input": "hi!"})


> Entering new AgentExecutor chain...
Hello! How can I assist you today?

> Finished chain.
{'input': 'hi!', 'output': 'Hello! How can I assist you today?'}
agent_executor.invoke({"input": "how can langsmith help with testing?"})


> Entering new AgentExecutor chain...

Invoking: `langsmith_search` with `{'query': 'LangSmith testing'}`


[Document(page_content='LangSmith Overview and User Guide | \uf8ffü¶úÔ∏è\uf8ffüõ†Ô∏è LangSmith', metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | \uf8ffü¶úÔ∏è\uf8ffüõ†Ô∏è LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'}), Document(page_content='Skip to main content\uf8ffü¶úÔ∏è\uf8ffüõ†Ô∏è LangSmith DocsPython DocsJS/TS DocsSearchGo to AppLangSmithOverviewTracingTesting & EvaluationOrganizationsHubLangSmith CookbookOverviewOn this pageLangSmith Overview and User GuideBuilding reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.Over the past two months, we at LangChain have been building and using LangSmith with the goal of bridging this gap. This is our tactical user guide to outline effective ways to use LangSmith and maximize its benefits.On by default‚ÄãAt LangChain, all of us have LangSmith‚Äôs tracing running in the background by default. On the Python side, this is achieved by setting environment variables, which we establish whenever we launch a virtual environment or open our bash shell and leave them set. The same principle applies to most', metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | \uf8ffü¶úÔ∏è\uf8ffüõ†Ô∏è LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'}), Document(page_content="applications can be expensive. LangSmith tracks the total token usage for a chain and the token usage of each step. This makes it easy to identify potentially costly parts of the chain.Collaborative debugging‚ÄãIn the past, sharing a faulty chain with a colleague for debugging was challenging when performed locally. With LangSmith, we've added a ‚ÄúShare‚Äù button that makes the chain and LLM runs accessible to anyone with the shared link.Collecting examples‚ÄãMost of the time we go to debug, it's because something bad or unexpected outcome has happened in our application. These failures are valuable data points! By identifying how our chain can fail and monitoring these failures, we can test future chain versions against these known issues.Why is this so impactful? When building LLM applications, it‚Äôs often common to start without a dataset of any kind. This is part of the power of LLMs! They are amazing zero-shot learners, making it possible to get started as easily as possible.", metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | \uf8ffü¶úÔ∏è\uf8ffüõ†Ô∏è LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'}), Document(page_content='You can also quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs.Monitoring‚ÄãAfter all this, your app might finally ready to go in production. LangSmith can also be used to monitor your application in much the same way that you used for debugging. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can also be assigned string tags or key-value metadata, allowing you to attach correlation ids or AB test variants, and filter runs accordingly.We‚Äôve also made it possible to associate feedback programmatically with runs. This means that if your application has a thumbs up/down button on it, you can use that to log feedback back to LangSmith. This can be used to track performance over time and pinpoint under performing data points, which you can subsequently add to a dataset for future testing ‚Äî mirroring', metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | \uf8ffü¶úÔ∏è\uf8ffüõ†Ô∏è LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'})]LangSmith can help with testing in several ways:

1. **Tracing**: LangSmith provides tracing capabilities that allow you to track the total token usage for a chain and the token usage of each step. This makes it easy to identify potentially costly parts of the chain during testing.

2. **Collaborative debugging**: LangSmith simplifies the process of sharing a faulty chain with a colleague for debugging. It has a "Share" button that makes the chain and language model runs accessible to anyone with the shared link, making collaboration and debugging more efficient.

3. **Collecting examples**: When testing LLM (Language Model) applications, failures and unexpected outcomes are valuable data points. LangSmith helps in identifying how a chain can fail and monitoring these failures. By testing future chain versions against known issues, you can improve the reliability and performance of your application.

4. **Editing examples and expanding evaluation sets**: LangSmith allows you to quickly edit examples and add them to datasets. This helps in expanding the surface area of your evaluation sets or fine-tuning a model for improved quality or reduced costs during testing.

5. **Monitoring**: LangSmith can be used to monitor your application in production. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can be assigned string tags or key-value metadata, allowing you to attach correlation IDs or AB test variants and filter runs accordingly. You can also associate feedback programmatically with runs, track performance over time, and pinpoint underperforming data points.

These features of LangSmith make it a valuable tool for testing and evaluating the performance of LLM applications.

> Finished chain.
{'input': 'how can langsmith help with testing?',
 'output': 'LangSmith can help with testing in several ways:\n\n1. **Tracing**: LangSmith provides tracing capabilities that allow you to track the total token usage for a chain and the token usage of each step. This makes it easy to identify potentially costly parts of the chain during testing.\n\n2. **Collaborative debugging**: LangSmith simplifies the process of sharing a faulty chain with a colleague for debugging. It has a "Share" button that makes the chain and language model runs accessible to anyone with the shared link, making collaboration and debugging more efficient.\n\n3. **Collecting examples**: When testing LLM (Language Model) applications, failures and unexpected outcomes are valuable data points. LangSmith helps in identifying how a chain can fail and monitoring these failures. By testing future chain versions against known issues, you can improve the reliability and performance of your application.\n\n4. **Editing examples and expanding evaluation sets**: LangSmith allows you to quickly edit examples and add them to datasets. This helps in expanding the surface area of your evaluation sets or fine-tuning a model for improved quality or reduced costs during testing.\n\n5. **Monitoring**: LangSmith can be used to monitor your application in production. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can be assigned string tags or key-value metadata, allowing you to attach correlation IDs or AB test variants and filter runs accordingly. You can also associate feedback programmatically with runs, track performance over time, and pinpoint underperforming data points.\n\nThese features of LangSmith make it a valuable tool for testing and evaluating the performance of LLM applications.'}
agent_executor.invoke({"input": "whats the weather in sf?"})


> Entering new AgentExecutor chain...

Invoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`


[{'url': 'https://weather.com/weather/tenday/l/San Francisco CA USCA0987:1:US', 'content': 'recents Specialty Forecasts 10 Day Weather-San Francisco, CA Today Mon 18 | Day  Fri 22 Fri 22 | Day Foggy early, then partly cloudy later in the day. High around 60F. Winds W at 10 to 15 mph.  Considerable cloudiness with occasional rain showers. High 59F. Winds SSE at 5 to 10 mph. Chance of rain 50%.  Thu 28 | Night Cloudy with showers. Low 46F. Winds S at 5 to 10 mph. Chance of rain 40%. Fri 29 Fri 29 | DaySan Francisco, CA 10-Day Weather Forecast - The Weather Channel | Weather.com 10 Day Weather - San Francisco, CA As of 12:09 pm PST Today 60°/ 54° 23% Tue 19 | Day 60° 23% S 12 mph...'}, {'url': 'https://www.sfchronicle.com/weather/article/us-forecast-18572409.php', 'content': 'San Diego, CA;64;53;65;48;Mist in the morning;N;6;72%;41%;3 San Francisco, CA;58;45;56;43;Partly sunny;ESE;6;79%;1%;2  Juneau, AK;40;36;41;36;Breezy with rain;SSE;15;90%;99%;0 Kansas City, MO;61;57;60;38;Periods of rain;E;13;83%;100%;1  St. Louis, MO;61;52;67;54;A little p.m. rain;SE;11;73%;90%;1 Tampa, FL;78;60;77;67;Rather cloudy;E;9;76%;22%;2  Salt Lake City, UT;38;22;35;22;Low clouds;SE;6;74%;0%;1 San Antonio, TX;72;64;76;47;Rain and a t-storm;NW;9;71%;94%;2US Forecast for Sunday, December 24, 2023'}]The weather in San Francisco is currently partly cloudy with a high around 60°F. The wind is coming from the west at 10 to 15 mph. There is a chance of rain showers later in the day. The temperature is expected to drop to a low of 46°F tonight with cloudy skies and showers.

> Finished chain.
{'input': 'whats the weather in sf?',
 'output': 'The weather in San Francisco is currently partly cloudy with a high around 60°F. The wind is coming from the west at 10 to 15 mph. There is a chance of rain showers later in the day. The temperature is expected to drop to a low of 46°F tonight with cloudy skies and showers.'}

Adding in memory​

As mentioned earlier, this agent is stateless. This means it does not remember previous interactions. To give it memory we need to pass in previous chat_history. Note: it needs to be called chat_history because of the prompt we are using. If we use a different prompt, we could change the variable name

# Here we pass in an empty list of messages for chat_history because it is the first message in the chat
agent_executor.invoke({"input": "hi! my name is bob", "chat_history": []})


> Entering new AgentExecutor chain...
Hello Bob! How can I assist you today?

> Finished chain.
{'input': 'hi! my name is bob',
 'chat_history': [],
 'output': 'Hello Bob! How can I assist you today?'}
from langchain_core.messages import AIMessage, HumanMessage
agent_executor.invoke(
    {
        "input": "what's my name?",
        "chat_history": [
            HumanMessage(content="hi! my name is bob"),
            AIMessage(content="Hello Bob! How can I assist you today?"),
        ],
    }
)


> Entering new AgentExecutor chain...
Your name is Bob.

> Finished chain.
{'input': "what's my name?",
 'chat_history': [HumanMessage(content='hi! my name is bob'),
  AIMessage(content='Hello Bob! How can I assist you today?')],
 'output': 'Your name is Bob.'}

If we want to keep track of these messages automatically, we can wrap this in a RunnableWithMessageHistory. For more information on how to use this, see this guide

from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
message_history = ChatMessageHistory()
agent_with_chat_history = RunnableWithMessageHistory(
    agent_executor,
    # This is needed because in most real world scenarios, a session id is needed
    # It isn't really used here because we are using a simple in memory ChatMessageHistory
    lambda session_id: message_history,
    input_messages_key="input",
    history_messages_key="chat_history",
)
agent_with_chat_history.invoke(
    {"input": "hi! I'm bob"},
    # This is needed because in most real world scenarios, a session id is needed
    # It isn't really used here because we are using a simple in memory ChatMessageHistory
    config={"configurable": {"session_id": "<foo>"}},
)


> Entering new AgentExecutor chain...
Hello Bob! How can I assist you today?

> Finished chain.
{'input': "hi! I'm bob",
 'chat_history': [],
 'output': 'Hello Bob! How can I assist you today?'}
agent_with_chat_history.invoke(
    {"input": "what's my name?"},
    # This is needed because in most real world scenarios, a session id is needed
    # It isn't really used here because we are using a simple in memory ChatMessageHistory
    config={"configurable": {"session_id": "<foo>"}},
)


> Entering new AgentExecutor chain...
Your name is Bob.

> Finished chain.
{'input': "what's my name?",
 'chat_history': [HumanMessage(content="hi! I'm bob"),
  AIMessage(content='Hello Bob! How can I assist you today?')],
 'output': 'Your name is Bob.'}

Conclusion​

That’s a wrap! In this quick start we covered how to create a simple agent. Agents are a complex topic, and there’s lot to learn! Head back to the main agent page to find more resources on conceptual guides, different types of agents, how to create custom tools, and more!