LazyLM Core

Core LazyLM classes, types, and functions

This is the state called LazyState that will be passed to the language model. This should essentially be able to do the following:

source

LazyState

 LazyState (problem:str, steps:List[str]=<factory>, current_step:int=0)
state = LazyState(problem="What is the result of f(x) = 3x + 2 when x = 5?")
state.add_step("First, we need substitute x in the function with 5")
state
LazyState(problem='What is the result of f(x) = 3x + 2 when x = 5?', steps=['What is the result of f(x) = 3x + 2 when x = 5?', 'First, we need substitute x in the function with 5'], current_step=1)
state.get_context()
"Problem: What is the result of f(x) = 3x + 2 when x = 5? \n Steps so far: ['What is the result of f(x) = 3x + 2 when x = 5?', 'First, we need substitute x in the function with 5']"

To keep things general, we’ll have a very simple data class called LLM that will give us an interface for the language model client and the model version.

source

LLM

 LLM (client:anthropic.lib.vertex._client.AnthropicVertex, model:str)

This package will also keep support for our internal system prompt, called lazy_system_p

With this simple state manager, we have a way to track each step of the problem-solving process and get the current context to be used for a call to a language model.

Now, let’s set up a class LazyEvaluationClient that will do the heavy lifting of managing the state and calling the language model.

source

LazyEvaluationClient

 LazyEvaluationClient (llm:__main__.LLM, max_tokens:int=100,
                       state:Optional[__main__.LazyState]=None,
                       lazy_system_p:str='\n        You are a helpful
                       assistant that can help with math problems.\n
                       You will be given a problem and a list of steps as
                       context, the format will be:\n                \n
                       PROBLEM: <problem>\n        STEPS: <steps>\n\n
                       Your job is to complete the next step and only the
                       next step in the problem-solving process. You
                       should never give more than one step.\n        If
                       you evaluate that the problem is done, respond with
                       "PROBLEM DONE"\n        ')

The Lazy Evaluation Client

Type Default Details
llm LLM the language model to use, see LLM class
max_tokens int 100 the maximum number of tokens to generate
state Optional None
lazy_system_p str
You are a helpful assistant that can help with math problems.
You will be given a problem and a list of steps as context, the format will be:

PROBLEM:
STEPS:

Your job is to complete the next step and only the next step in the problem-solving process. You should never give more than one step.
If you evaluate that the problem is done, respond with “PROBLEM DONE”

Here is an example of using LLM, LazyState, and LazyEvaluationClient together.

lazy_state = LazyState(problem="What is the derivative of `2x^3 + x^2 + 2x + 1`? Give me the solution step-by-step")
client = AnthropicVertex(project_id=project_id, region=location)
llm = LLM(client=client, model=model)
lazy_lm = LazyEvaluationClient(llm=llm, state=lazy_state)

Let’s grab the current step

lazy_lm.get_current_step()
'What is the derivative of `2x^3 + x^2 + 2x + 1`? Give me the solution step-by-step'

Let’s get the next step

lazy_lm.get_next_step()
"To find the derivative of the given function, we'll use the power rule and the constant rule of differentiation. Let's start with the first term:\n\nThe derivative of 2x^3 is:\n3 * 2x^(3-1) = 6x^2"

and the next…

lazy_lm.get_next_step()
"Let's proceed with the next step in finding the derivative:\n\nThe derivative of x^2 is:\n2 * x^(2-1) = 2x"

Let’s look at our current state

for step in lazy_lm.state.steps:
    print("-"*10)
    print(step)
----------
What is the derivative of `2x^3 + x^2 + 2x + 1`? Give me the solution step-by-step
----------
To find the derivative of the given function, we'll use the power rule and the constant rule of differentiation. Let's start with the first term:

The derivative of 2x^3 is:
3 * 2x^(3-1) = 6x^2
----------
Let's proceed with the next step in finding the derivative:

The derivative of x^2 is:
2 * x^(2-1) = 2x

We need a way to take the current reasoning step and query it without having the model advance to the next step in the problem-solving process.

source

LazyEvaluationClient.ask_question

 LazyEvaluationClient.ask_question (question:str)

*Allows the user to ask a question about the current step without affecting the model’s ability to generate the next step.

Args: question (str): The question the user wants to ask about the current step.

Returns: str: The model’s response to the question.*

lazy_lm.get_current_step()
"Let's proceed with the next step in finding the derivative:\n\nThe derivative of x^2 is:\n2 * x^(2-1) = 2x"
lazy_lm.ask_question("This does not make sense to me")
"I apologize for the confusion. Let me explain the current step without advancing further:\n\nThe step we're looking at is finding the derivative of the x^2 term in the original expression 2x^3 + x^2 + 2x + 1.\n\nTo find the derivative of x^2, we use the power rule of differentiation. The power rule states that for a term ax^n, the derivative is n * ax^(n-1"

The Lazy Evaluation Flow

This simple framework effectivly shows how we can wrape a language model capable of step-by-step reasoning to create a lazy evaluator.

This approach follows these system design steps:

  • Problem Initialization: A state manager is initalized with a problem

  • Prompting Strategy: Prompt the language model to generate the next step given the context in the state manager.

  • State Update: State Manager records the newly generated step and updates.

  • User Interaction: User interaction is held within a different state manager question_history which does not affect the overall state of the current problem.

  • Adaptive Response: Based on the current input, the Lazy Evaluator decides to either 1) Generate the next step or 2) Provide a response to the user’s question given the current state of the problem and the question history.

Finally, let’s patch different model provider’s APIs

source

AnthropicVertex.lazy

 AnthropicVertex.lazy (problem:str)

Entry point of the LazyLM Framework for the AnthropicVertex client API