%pip install boto3 %pip install botocore %pip install langchain %pip install langchain-aws %pip install python-dotenv %pip install dateparser %pip install langgraph %pip install langchain-community %pip install faiss-gpu %pip install grandalf

import warnings import boto3 from dotenv import load_dotenv import os from botocore.config import Config

warnings.filterwarnings(‘ignore’) load_dotenv()

my_config = Config( region_name = ‘us-west-2’, signature_version = ‘v4’, retries = { ‘max_attempts’: 10, ‘mode’: ‘standard’ } )

aws_access_key_id = os.getenv(“AWS_ACCESS_KEY”) aws_secret_access_key = os.getenv(“AWS_ACCESS_SECRET”)

boto3_bedrock = boto3.client(‘bedrock-runtime’, aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, config=my_config )

from langchain_aws import ChatBedrock from langchain_core.messages import HumanMessage, SystemMessage

llm = ChatBedrock( model_id="anthropic.claude-3-haiku-20240307-v1:0”, client=boto3_bedrock, model_kwargs=dict(temperature=0) )

messages = [ HumanMessage( content="what is the weather like in Seattle WA” ) ] ai_msg = llm.invoke(messages) ai_msg

Naive inferencing: The root challenge in creating Chatbots and Virtual Assistants and the Agentic solution:

As seen in previous tutorials, LLM conversational interfaces such as chatbots or virtual assistants can be used to enhance the user experience of customers. These can be improved even more by giving them context from related sources such as chat history, documents, websites, social media platforms, and / or messaging apps, this is called RAG (Retrieval Augmented Generation) and is a fundamental backbone of designing robust AI solutions.

One persistent bottleneck however is the inability of LLMs to assess whether data extracted and or its response, based on said data, is accurate and fully encapsulates a user requests (hallucinating). A way to mitigate this risk brought up by naive, inferencing with RAG is through the use of Agents. Agents are defined as a workflow that uses data, tools, and its own inferences to check that the response provided is accurate and meets users goals.

Amazon Bedrock - Agents Interface

Key Elements of Agents

  • Agents are designed for tasks that require multistep reasoning; Think questions that intuitively require multiple steps, for example how old was Henry Ford when he founded his company.
  • They are designed to plan ahead, remember past actions and check its own responses.
  • Agents can be made to deconstruct complex requests into manageable smaller sub-tasks such as data retrieval, comparison and tool usage.
  • Agents might be designed as standalone solutions or paired with other agents to enhance the agentic workflow.

Let’s build an agentic workflow from scratch to see how it works, for this use case we will use Calude 3 Sonnet to power our agentic workflow.

Architecture [Retriever with LangGraph]

The core benefit of agentic workflows lies in its flexibility to adjust to your needs. You have full control on the design the flow by properly defining what the agents do and what tools and information is available to them. One popular framework for the use of Agents is called Langgraph, a low-level framework that offers the ability of adding cycles (using previous inferences as context to either fix or build on it), controllability of the flow and state of your application, and persistence, giving the agents the ability to involve humans in the loop and the memory to recall past agentic flows.

For this scenario we’ll define 3 agents:

  1. We defined a supervisor agent responsible for deciding the steps needed to fulfill the users request, this can take the shape of using tools or data retrieval.
  2. Then a task-driven agent to retrieve documents which can be invoked only when the orchestrator agent deems it necessary to fulfill the users request.
  3. Finally, a data retriever agent will query an embedding database containing Medical history if its deemed necessary to use this information to answer the users question.

Dependencies and helper functions:

from langchain.chains import create_history_aware_retriever, create_retrieval_chain from langchain.chains.combine_documents import create_stuff_documents_chain from langchain_core.chat_history import InMemoryChatMessageHistory from langchain_core.runnables.history import RunnableWithMessageHistory from langchain_core.chat_history import BaseChatMessageHistory from langchain.document_loaders import CSVLoader from langchain.text_splitter import CharacterTextSplitter from langchain.vectorstores import FAISS from langchain.embeddings import BedrockEmbeddings import warnings from io import StringIO import sys import textwrap from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

warnings.filterwarnings(‘ignore’)

Define a print helper function

def print_ww(*args, width: int = 100, **kwargs): “““Like print(), but wraps output to width characters (default 100)””” _stdout = None buffer = StringIO() try: _stdout = sys.stdout sys.stdout = buffer print(*args, **kwargs) output = buffer.getvalue() finally: sys.stdout = _stdout for line in output.splitlines(): print("\n”.join(textwrap.wrap(line, width=width)))

Build the retriever chain to be used with LangGraph

  1. Create create_retriever_pain which is used when the solution requires data retrieval from our documents
  2. Define the system prompt to enforce the correct use of context retrieved, it also ensures that the agent does not hallucinate
  3. Define the vectorstore using FAISS, a light weight in-memory vector DB and our documents stored in ‘medi_history.csv’
  4. Define the sessions persistent memory store for the agents use

store = {} def get_session_history(session_id: str) -> BaseChatMessageHistory: if session_id not in store: store[session_id] = InMemoryChatMessageHistory() return store[session_id]

def create_retriever_pain():

br_embeddings = BedrockEmbeddings(model_id="amazon.titan-embed-text-v2:0", client=boto3_bedrock)

loader = CSVLoader("./medi_history.csv")
documents_aws = loader.load() 
print(f"Number of documents={len(documents_aws)}")

docs = CharacterTextSplitter(chunk_size=2000, chunk_overlap=400, separator=",").split_documents(documents_aws)

print(f"Number of documents after split and chunking={len(docs)}")
    
vectorstore_faiss_aws = FAISS.from_documents(
    documents=docs,
    embedding = br_embeddings
)

print(f"vectorstore_faiss_aws: number of elements in the index={vectorstore_faiss_aws.index.ntotal}::")

model_parameter = {"temperature": 0.0, "top_p": .5, "max_tokens_to_sample": 2000}
modelId = "anthropic.claude-3-sonnet-20240229-v1:0" #"meta.llama3-8b-instruct-v1:0" 
chatbedrock_llm = ChatBedrock(
    model_id=modelId,
    client=boto3_bedrock,
    model_kwargs=model_parameter, 
    beta_use_converse_api=True
)

qa_system_prompt = """You are an assistant for question-answering tasks. \
Use the following pieces of retrieved context to answer the question. \
If the answer is not present in the context, just say you do not have enough context to answer. \
If the input is not present in the context, just say you do not have enough context to answer. \
If the question is not present in the context, just say you do not have enough context to answer. \
If you don't know the answer, just say that you don't know. \
Use three sentences maximum and keep the answer concise.\

{context}"""

qa_prompt = ChatPromptTemplate.from_messages([
    ("system", qa_system_prompt),
    MessagesPlaceholder("chat_history"),
    ("human", "{input}")
])
question_answer_chain = create_stuff_documents_chain(chatbedrock_llm, qa_prompt)

pain_rag_chain = create_retrieval_chain(vectorstore_faiss_aws.as_retriever(), 
                                        question_answer_chain)

pain_retriever_chain = RunnableWithMessageHistory(
    pain_rag_chain,
    get_session_history=get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history",
    output_messages_key="answer",
)
return pain_retriever_chain

Testing the rag chain:

pain_rag_chain = create_retriever_pain()
result = pain_rag_chain.invoke( {“input”: “What all pain medications can be used for headache?”, “chat_history”: []}, config={‘configurable’: {‘session_id’: ‘TEST-123’}}, ) result[‘answer’]

Book / Cancel Appointments: An agent with tools:

In this module we will create an agent responsible for booking and canceling doctor appointments. This agent will take a booking request to create or cancel an appointment and its action will be guided by the 4 tools available to it.

  1. book_appointment: Used by the agent to book an appointment give the users request as long as it meets the criteria, valid date and time within office hours.
  2. cancel_appointment: If an exiting appointment is found, it will remove its respective ‘booking id’ from the list of appointments.
  3. reject_appointment: If an appointment cannot be booked due to inability or invalid date or time the agent will use this tool to reject the users request.
  4. need_more_info: Returns the earliest date and time needed for the booking an appointment back to the agent as well as informing the agent that it should request further details from the user.

from langchain.tools import tool from langchain.agents import AgentExecutor, create_tool_calling_agent from datetime import datetime, timedelta import dateparser

appointments = [‘ID_100’] # Default appointment def create_book_cancel_agent(): today = datetime.today() tomorrow = today + timedelta(days=1) formatted_tomorrow = tomorrow.strftime("%B %d, %Y”) start_time = datetime.strptime(“9:00 am”, “%I:%M %p”).time() end_time = datetime.strptime(“5:00 pm”, “%I:%M %p”).time()

def check_date_time(date: str, time: str) -> str:
    """Helper function is used by book appointment tool to check that the date and time passed by the user are within the date time params"""
    _date = dateparser.parse(date)
    _time = dateparser.parse(time)
    if not _date or not _time:
        return 'ERROR: Date and time parameters are not valid'
    
    input_date = _date.date()
    input_time = _time.time()
    if input_date < tomorrow.date():
        return f'ERROR: Appointment date must be at least one day from today: {today.strftime("%B %d, %Y")}'
    elif input_date.weekday() > 4:
        return f'ERROR: Appointments are only available on weekdays, date {input_date.strftime("%B %d, %Y")} falls on a weekend.'
    elif start_time > input_time >= end_time:
        return f'ERROR: Appointments bust be between the hours of 9:00 am to 5:00 pm'
    return 'True'
    
    
@tool("book_appointment")
def book_appointment(date: str, time: str) -> dict:
    """Use this function to book an appointment. This function returns the booking ID"""

    print(date, time)
    is_valid = check_date_time(date, time)
    if 'ERROR' in is_valid :
        return {"status" : False, "date": date, "time": time, "booking_id": is_valid}

    if appointments:
        last_appointment = appointments[-1]
        new_appointment = f"ID_{int(last_appointment[3:]) + 1}"
        appointments.append(new_appointment)
    else:
        new_appointment = "ID_100"
        appointments.append(new_appointment)
        
    return {"status" : True, "date": date, "time": time, "booking_id": new_appointment}

@tool("reject_appointment")
def reject_appointment() -> dict:
    """Use this function to reject an appointment if the status of book_appointment is False"""
    return {"status" : False, "date": "", "time": "", "booking_id": ""}
    
@tool("cancel_appointment")
def cancel_appointment(booking_id: str) -> dict:
    """Use this function to cancel an existing appointment and remove it from the schedule. This function needs a booking id to cancel the appointment."""

    print(booking_id)
    status = any(app == booking_id for app in appointments)
    if not status:
        booking_id = "ERROR: No ID for given booking found. Please provide valid id"
    appointments.remove(booking_id)
    return {"status" : status, "booking_id": booking_id}

@tool("need_more_info")
def need_more_info() -> dict:
    """Use this function to get more information from the user. This function returns the earliest date and time needed for the booking an appointment """
    return {"date after": formatted_tomorrow, "time between": "09:00 AM to 05:00 PM", "week day within": "Monday through Friday"}


prompt_template_sys = """
You are a booking assistant.
Make sure you use one the the following tools ["book_appointment", "cancel_appointment", "need_more_info", "reject_appointment"]
"""

chat_prompt_template = ChatPromptTemplate.from_messages(
        messages = [
            ("system", prompt_template_sys),
            ("placeholder", "{chat_history}"),
            ("human", "{input}"),
            ("placeholder", "{agent_scratchpad}"),
        ]
)

model_id = "anthropic.claude-3-sonnet-20240229-v1:0" #"us.anthropic.claude-3-5-sonnet-20240620-v1:0" 
model_parameter = {"temperature": 0.0, "top_p": .1, "max_tokens_to_sample": 400}
chat_bedrock_appointment = ChatBedrock(
    model_id=model_id,
    client=boto3_bedrock,
    model_kwargs=model_parameter, 
    beta_use_converse_api=True
)

tools_list_book = [book_appointment, cancel_appointment, need_more_info, reject_appointment]

# Construct the Tools agent
book_cancel_agent_t = create_tool_calling_agent(chat_bedrock_appointment, 
                                                tools_list_book, 
                                                chat_prompt_template)

agent_executor_t = AgentExecutor(agent=book_cancel_agent_t, 
                                 tools=tools_list_book, 
                                 verbose=True, 
                                 max_iterations=5, 
                                 return_intermediate_steps=True)
return agent_executor_t

appointments

Test the Booking Agent with history:

Add context for the agent to use

book_cancel_history = InMemoryChatMessageHistory() book_cancel_history.add_user_message(“can you book an appointment?")