Sessions

Many interactions with LLM applications span multiple traces. Sessions in Langfuse are a way to group these traces together and see a simple session replay of the entire interaction. Get started by adding a sessionId when creating a trace.

Add a sessionId when creating/updating a trace. This can be any string that you use to identify the session. All traces with the same sessionId will be grouped together.

When using the @observe() decorator:

from langfuse.decorators import langfuse_context, observe
 
@observe()
def fn():
    langfuse_context.update_current_trace(
        session_id="your-session-id"
    )
 
fn()

When using the low-level SDK:

from langfuse import Langfuse
langfuse = Langfuse()
 
trace = langfuse.trace(
    session_id="your-session-id"
)

import { Langfuse } from "langfuse";
const langfuse = new Langfuse();
 
const trace = langfuse.trace({
  sessionId: "your-session-id",
});

See JS/TS SDK docs for more details.

When using the OpenAI SDK Integration, pass the session_id as an additional argument:

from langfuse.openai import openai
 
completion = openai.chat.completions.create(
  name="test-chat",
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a calculator."},
    {"role": "user", "content": "1 + 1 = "}],
  temperature=0,
 
  # add session_id as additional argument
  session_id="your-session-id"
)

When using the integration with the @observe() decorator (see interop docs), set the session_id via the langfuse_context:

from langfuse.decorators import langfuse_context, observe
from langfuse.openai import openai
 
@observe()
def fn():
    langfuse_context.update_current_trace(
        session_id="your-session-id"
    )
 
    completion = openai.chat.completions.create(
      name="test-chat",
      model="gpt-3.5-turbo",
      messages=[
        {"role": "system", "content": "You are a calculator."},
        {"role": "user", "content": "1 + 1 = "}],
      temperature=0,
    )
 
fn()

When using the CallbackHandler, you can pass the session_id as a keyword argument:

handler = CallbackHandler(
  session_id="your-session-id"
)

You can also set the session_id dynamically via the runnable configuration in the chain invocation:

from langfuse.callback import CallbackHandler
 
handler = CallbackHandler()
 
# Your existing Langchain code to create the chain
...
 
# Pass langfuse_session_id as metadata to the chain invocation to be parsed as the Langfuse session_id
chain.invoke(
    {"animal": "dog"},
    config={
        "callbacks": [handler],
        "metadata": {
            "langfuse_session_id": "your-session-id",
        },
    },
)

When using the integration with the @observe() decorator (see interop docs), set the session_id via the langfuse_context:

from langfuse.decorators import langfuse_context, observe
 
@observe()
def fn():
    langfuse_context.update_current_trace(
        session_id="your-session-id"
    )
 
    langfuse_handler = langfuse_context.get_current_langchain_handler()
 
    # Pass handler to invoke of your langchain chain/agent
    chain.invoke({"person": person}, config={"callbacks":[langfuse_handler]})
 
fn()

When using the CallbackHandler, you can pass the sessionId to the constructor:

const handler = new CallbackHandler({
  sessionId: "your-session-id",
});

You can also set the session_id dynamically via the runnable configuration in the chain invocation:

import { CallbackHandler } from "langfuse-langchain";
 
const langfuseHandler = new CallbackHandler();
 
// Your existing Langchain code to create the chain
...
 
// Pass langfuseSessionId as metadata to the chain invocation to be parsed as the Langfuse session_id
await chain.invoke(
  { input: "<user_input>" },
  { callbacks: [langfuseHandler], metadata: { langfuseSessionId: "your-session-id" } }
);

When using the integration with the JS SDK (see interop docs), set the sessionId via langfuse.trace():

import { CallbackHandler, Langfuse } from "langfuse-langchain";
const langfuse = new Langfuse();
 
const trace = langfuse.trace({
  sessionId: "your-session-id",
});
const langfuseHandler = new CallbackHandler({ root: trace });
 
// Add Langfuse handler as callback to your langchain chain/agent
await chain.invoke({ input: "<user_input>" }, { callbacks: [langfuseHandler] });

When using the LlamaIndex Integration, set the session_id via the instrumentor.observe() context manager:

from langfuse.llama_index import LlamaIndexInstrumentor
 
instrumentor = LlamaIndexInstrumentor()
 
with instrumentor.observe(session_id="my-session"):
    # ... your LlamaIndex index creation ...
 
    index.as_query_engine().query("What is the capital of France?")
 
instrumentor.flush()

When using the integration with the @observe() decorator (see interop docs), set the session_id via the langfuse_context:

from langfuse.decorators import langfuse_context, observe
from langfuse.llama_index import LlamaIndexInstrumentor
 
instrumentor = LlamaIndexInstrumentor()
 
@observe()
def llama_index_fn(question: str):
    # Update context
    langfuse_context.update_current_trace(session_id="your-session-id")
 
    # Get IDs
    current_trace_id = langfuse_context.get_current_trace_id()
    current_observation_id = langfuse_context.get_current_observation_id()
 
    # Pass to instrumentor
    with instrumentor.observe(
        trace_id=current_trace_id,
        parent_observation_id=current_observation_id,
        update_parent=False
    ) as trace:
        # ... your LlamaIndex index creation ...
 
        index.as_query_engine().query("What is the capital of France?")
 
        # Run application
        index = VectorStoreIndex.from_documents([doc1, doc2])
        response = index.as_query_engine().query(question)
 
        return response

When using the (deprecated) LlamaIndex Callback Integration, set the session_id via set_trace_params. All LlamaIndex traces created after set_trace_params will include the session_id. Learn more about set_trace_params here.

from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager
from langfuse import langfuse
 
# Instantiate a new LlamaIndexCallbackHandler and register it in the LlamaIndex Settings
langfuse_callback_handler = LlamaIndexCallbackHandler()
Settings.callback_manager = CallbackManager([langfuse_callback_handler])
 
langfuse_callback_handler.set_trace_params(
  session_id="session-abc",
)

When using the integration with the @observe() decorator (see interop docs), set the session_id via the langfuse_context:

from langfuse.decorators import langfuse_context, observe
from llama_index.core import Document, VectorStoreIndex
from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager
 
@observe()
def llama_index_fn(question: str):
    langfuse_context.update_current_trace(
        session_id="your-session-id"
    )
 
    # Set callback manager for LlamaIndex, will apply to all LlamaIndex executions in this function
    langfuse_handler = langfuse_context.get_current_llama_index_handler()
    Settings.callback_manager = CallbackManager([langfuse_handler])
 
    # Run application
    index = VectorStoreIndex.from_documents([doc1,doc2])
    response = index.as_query_engine().query(question)
    return response

Example

Try this feature using the public example project.

Example session spanning multiple traces

Session view

Other features

Publish a session to share with others as a public link (example)
Bookmark a session to easily find it later
Annotate sessions by adding scores via the Langfuse UI to record human-in-the-loop evaluations

Sessions

Example

Other features

GitHub Discussions

Was this page useful?

Questions? We're here to help

Subscribe to updates