Building a Mini Data Analyst with LLMs and Langgraph

[susmitpy]

This talk demonstrates how to create a powerful data analysis agent capable of answering questions about datasets using Large Language Models. We will explore the process of building this "Mini Data Analyst" by leveraging the capabilities of Langgraph to orchestrate the workflow. The session will cover the integration of tools that allow the LLM agent to execute Python code for data manipulation and analysis. Additionally, we will delve into implementing observability using Langtrace to monitor and understand the agent's behavior. A key focus will be on the flexibility of using both locally hosted models like Llama3 8B and cloud-based solutions such as AWS Bedrock with Llama3 70B.

---

Agenda

1. The Rise of AI Agents in Data Analytics

• Concept: Moving beyond simple Q&A. The shift from one-off prompts to conversational, tool-using AI agents.
• Problem: How can we empower LLMs to not just talk about data, but to actively analyze it?
• Our Goal: Introduce the concept of a "Mini Data Analyst" agent that can reason, plan, and execute code to answer data-driven questions.

---

2. Anatomy of a Code-Executing LLM Agent

• Concept 1: The Reasoning Engine (The LLM "Brain").

  • Discuss the role of the LLM in interpreting a user's question, formulating a plan, and generating Python code.
  • Explain the trade-offs between local models (e.g., Llama 3 8B for privacy/cost) and powerful cloud models (e.g., Llama 3 70B for complex reasoning).

• Concept 2: Empowering LLMs with Tools.

  • Explain the core principle of "Tool Use" in modern AI agents.
  • Focus on our key tool: A Python code executor that gives the agent access to the pandas library.
  • Discuss the importance of creating a safe and constrained execution environment (e.g., preventing arbitrary imports).

---

3. Orchestrating Agentic Workflows with LangGraph

• Concept 1: Beyond Linear Chains.
  Why do simple "prompt → response" chains fail for multi-step tasks? Introduce the need for cyclical, stateful workflows (Reason → Act → Observe → Repeat).

• Concept 2: State Machines for AI.
  Introduce LangGraph as a framework for building robust agent architectures as state machines.

• Explain the core primitives of LangGraph:

  • State: How we track the conversation, previous results, and control flow.
  • Nodes: The actions our agent can perform (e.g., call_model, execute_code).
  • Conditional Edges: The logic that governs the agent's decisions. "Should I execute more code or have I found the final answer?"

---

4. Observability: Debugging the Agent's Mind

• Concept: The challenge of the "black box." Why is it critical to understand an agent's decision-making process?
• Solution: Introduce LLM Observability using Langtrace. Explain how it allows us to visualize the agent's internal "thoughts" and actions.
• Cover key metrics to track: execution traces, latency, and custom attributes for better debugging and analysis.

---

5. Live Demo: The Mini Data Analyst in Action

• Tie all the concepts together with a live demonstration using the project codebase.
• Ask the agent questions about the Titanic dataset.
• Narrate the agent's actions in real-time as it cycles through the LangGraph state machine (calling the LLM, executing code).
• Show the resulting trace in the Langtrace UI to visually reinforce the concepts of the agent's workflow and observability.

---

6. Q&A

• Open the floor for audience questions.