3  Using AI Functions

In this chapter, we are going to learn about equipping our AI with deterministic tools. This is a fantastic tool to equip your AI with certain skills you may need and reduces the risk of hallucinations.

This works by

• registering a function with your LLM,
• letting the LLM fill out the arguments based on the prompt,
• executing the function with the specified arguments in the background, and
• then passing the response to the LLM so that it can generate its response.

The {ellmer} docs summarize this via this neat infographic:

Now, this sounds like a lot of steps. But thanks to {ellmer} they are all automated. We just have to set this up. Let’s check out how that works. And like in every chapter, there’s also a corresponding video on YouTube:

3.1 Start with a regular chat

Like any workflow with {ellmer}, we need to have a chat object first. So as discussed in the previous chapter, let’s create one.

library(ellmer)
chat <- chat_openai(
  system_prompt = '
  You are a helpful assistant that can tell 
  the weather at the Washington Monument. 
  But only there. No other place'
) 
## Using model = "gpt-4o".

Let’s try out our AI and ask it what the weather at the Washington Monument will be in 3 hours.

chat$chat('What will the weather be at the Washington Monument will at 5 am local time?')
## I'm sorry, but I can't provide real-time weather forecasts. To get the most 
## accurate and up-to-date weather information for the Washington Monument, I 
## recommend checking a reliable weather website or app closer to your time of 
## interest.

Well, that was a bust. At least ChatGPT was honest and said it doesn’t know how to do that.

3.2 Creating a tool

Thus, let us build a tool (a good-ol’ function), that Gippity can use later on. (Also, look at how cool I can write GPT.) Thankfully, the weather data for the Washington monument is only a few API calls away.

And even better for us, I’ve actually covered how to do that in full detail in an old edition of my 3MW newsletter. Since making API requests isn’t really what I want to cover in this chaper, I’ll just throw together the code that we used in the newsletter. You can read up on the details there.

So, first we make an API request to the National Weather Service using the {httr2} package:

library(tidyverse)
library(httr2)

# Get forecast URL from NWS
NWS_base_url <- 'https://api.weather.gov'
NWS_response <- request(NWS_base_url) |> 
  req_url_path_append(
    'points',
    '38.8894,-77.0352'
  ) |> 
  req_perform()

# Extract forecast URL
forecast_url <- NWS_response |> 
  resp_body_json() |> 
  pluck('properties', 'forecastHourly')

# Get actual forecast data
forecast_response <- request(forecast_url) |> 
  req_perform()

And then we extract the data from the response:

# Bring data into tibble format
extracted_data <- forecast_response |> 
  resp_body_json() |> 
  pluck('properties', 'periods') |> 
  map_dfr( # iterates over each list and binds rows to a tibble
    \(x) {
      tibble(
        time = x |> pluck('startTime'),
        temp_F = x |> pluck('temperature'),
        rain_prob = x |> pluck('probabilityOfPrecipitation', 'value'),
        forecast = x |> pluck('shortForecast')
      )
    }
  )
extracted_data
## # A tibble: 156 × 4
##    time                      temp_F rain_prob forecast                
##    <chr>                      <int>     <int> <chr>                   
##  1 2025-02-08T17:00:00-05:00     33        92 Freezing Rain           
##  2 2025-02-08T18:00:00-05:00     33        52 Chance Light Rain       
##  3 2025-02-08T19:00:00-05:00     33        38 Chance Light Rain       
##  4 2025-02-08T20:00:00-05:00     33        35 Chance Light Rain       
##  5 2025-02-08T21:00:00-05:00     33        37 Chance Light Rain       
##  6 2025-02-08T22:00:00-05:00     34        40 Chance Light Rain       
##  7 2025-02-08T23:00:00-05:00     35        36 Chance Light Rain       
##  8 2025-02-09T00:00:00-05:00     36        23 Slight Chance Light Rain
##  9 2025-02-09T01:00:00-05:00     37        30 Chance Light Rain       
## 10 2025-02-09T02:00:00-05:00     38         6 Cloudy                  
## # ℹ 146 more rows

Cool! This shows us all the data our GPT friend needs to generate a meaningful response. Usually, though, AIs prefer JSON formats. So, let’s bring our data to JSON format.

jsonlite::toJSON(extracted_data, pretty = TRUE)
## [
##   {
##     "time": "2025-02-02T05:00:00-05:00",
##     "temp_F": 25,
##     "rain_prob": 0,
##     "forecast": "Mostly Clear"
##   },
##   {
##     "time": "2025-02-02T06:00:00-05:00",
##     "temp_F": 25,
##     "rain_prob": 0,
##     "forecast": "Sunny"
##   },
##   {
##     "time": "2025-02-02T07:00:00-05:00",
##     "temp_F": 26,
##     "rain_prob": 0,
##     "forecast": "Mostly Sunny"
##   },
##   ... (remaining rows cut for brevity)
## ]

This doesn’t look as nice to us humans. But apparently GPTs like this more. So, let’s give our AI overlords what they want, shall we? Anyway, let’s wrap all of this into a function.

get_weather_data <- function() {
  # Get forecast URL from NWS
  NWS_base_url <- 'https://api.weather.gov'
  NWS_response <- request(NWS_base_url) |> 
    req_url_path_append(
      'points',
      '38.8894,-77.0352'
    ) |> 
    req_perform()
  
  # Extract forecast URL
  forecast_url <- NWS_response |> 
    resp_body_json() |> 
    pluck('properties', 'forecastHourly')
  
  # Get actual forecast data
  forecast_response <- request(forecast_url) |> 
    req_perform()
  
  # Bring data into tibble format
  extracted_data <- forecast_response |> 
    resp_body_json() |> 
    pluck('properties', 'periods') |> 
    map_dfr( # iterates over each list and binds rows to a tibble
      \(x) {
        tibble(
          time = x |> pluck('startTime'),
          temp_F = x |> pluck('temperature'),
          rain_prob = x |> pluck('probabilityOfPrecipitation', 'value'),
          forecast = x |> pluck('shortForecast')
        )
      }
    )
  jsonlite::toJSON(extracted_data, pretty = TRUE)
}

3.3 Register a tool

Now, just because we have created a function doesn’t mean that our AI chat knows about that. So let’s register our tool. We can do so with the register_tool() method.

chat$register_tool(
  tool(
    .fun = get_weather_data,
    .description = 'Gets weather forecasts for Washington monument'
  )
)

Notice that I’ve also used the tool() function to define a tool. At this point, this tool() call only adds a description to our function. But we’ll soon see that more stuff can be used for this.

3.4 Let’s try again

For now, let’s try to ask GPT again about the weather at the Washington monument.

chat$chat('What will the weather be at the Washington Monument at 4 am local time?')
## At 4 AM local time, the weather at the Washington Monument is expected to be 
## mostly cloudy with a temperature of 39°F and a 0% chance of rain.

Aha! Now, GPT can tell us about the weather. But is the reply actually accurate and not some hallucinated non-sense? Let’s check with the data we extracted earlier.

extracted_data |> 
  filter(str_detect(time, 'T04'))
## # A tibble: 7 × 4
##   time                      temp_F rain_prob forecast                   
##   <chr>                      <int>     <int> <chr>                      
## 1 2025-02-09T04:00:00-05:00     39         0 Mostly Cloudy              
## 2 2025-02-10T04:00:00-05:00     33         5 Mostly Cloudy              
## 3 2025-02-11T04:00:00-05:00     31        12 Cloudy                     
## 4 2025-02-12T04:00:00-05:00     33        63 Light Snow Likely          
## 5 2025-02-13T04:00:00-05:00     36        93 Rain And Snow              
## 6 2025-02-14T04:00:00-05:00     32        11 Mostly Cloudy              
## 7 2025-02-15T04:00:00-05:00     30        21 Slight Chance Freezing Rain

Nice! This looks correct.

3.5 Modify location

Now, let’s do something more interesting. Last time, GPT didn’t have to fill any function arguments. So, let’s allow for a couple more places we can check the weather for.

#' Gets the weather forcast at a specifc location
#' 
#' @param coords A string containing 2 comma-separated coordinated
#' @return Hourly weather forcast for location in JSON format
get_weather_data <- function(coords) {
  # Get forecast URL from NWS
  NWS_base_url <- 'https://api.weather.gov'
  NWS_response <- request(NWS_base_url) |> 
    req_url_path_append(
      'points',
      coords
    ) |> 
    req_perform()
  
  # Extract forecast URL
  forecast_url <- NWS_response |> 
    resp_body_json() |> 
    pluck('properties', 'forecastHourly')
  
  # Get actual forecast data
  forecast_response <- request(forecast_url) |> 
    req_perform()
  
  # Bring data into tibble format
  extracted_data <- forecast_response |> 
    resp_body_json() |> 
    pluck('properties', 'periods') |> 
    map_dfr( # iterates over each list and binds rows to a tibble
      \(x) {
        tibble(
          time = x |> pluck('startTime'),
          temp_F = x |> pluck('temperature'),
          rain_prob = x |> pluck('probabilityOfPrecipitation', 'value'),
          forecast = x |> pluck('shortForecast')
        )
      }
    )
  jsonlite::toJSON(extracted_data, pretty = TRUE)
}

Notice that I have

• created a new argument called coords and
• added Roxygen2 comments to document the function.

The latter point will be useful when we register this function. But first, we have to create a new chat with a new system prompt to tell GPT that it knows specific locations.

chat <- chat_openai(
  system_prompt = '
  You can generate hourly weather forecasts for 
  the following locations:
  - Washington Monument (38.8894,-77.0352)
  - New York City (40.7306,-73.9352)
  - Chicago (41.8818,-87.6231)
  - Los Angeles (34.0522,-118.2436)'
)
## Using model = "gpt-4o".

3.6 Tool registration (again)

Now that we have a new chat, we can register our new function. Remember the Roxygen comments? This lets us pass our function to create_tool_def().

In theory, this will send our function (including Roxygen comments) to ChatGPT to generate the tool() call for us.

create_tool_def(get_weather_data)
## ```r
## tool(
##   get_weather_data,
##   "A function to retrieve weather data for specific coordinates.",
##   coords = type_unknown(
##     "The coordinates for which the weather data will be retrieved."  
##      # TODO: Replace with the appropriate type information.
##   )
## )
## ```

Unfortunately, this doesn’t always work perfectly as it is LLM-generated. So, we just take the code and adjust it to what we need. Here, what we have to change is type_unknown() to type_string(). Also, the description of the argument should also be more clear.

chat$register_tool(
  tool(
    get_weather_data,
    "Fetches weather data for the given coordinates.",
    coords = type_string(
      "A string containing 2 comma-separated coordinated"
    )
  )
)

Also notice that there are multiple type_*() functions that allows us to define input styles. For example, if we want the coords argument to be filled with

• texts/characters, then we have to use type_string().
• numbers, then we have to use type_number().

That way we can enforce how the LLM should fill in the function argument. Also, these type_*() functions are also pretty useful for structured outputs. But that’s a story for the next chapter. For now, let’s take our LLM for a test drive.

chat$chat('What is the weather in LA at 6 pm local time?')
## At 6 PM local time in Los Angeles, the weather is expected to be partly cloudy 
## with a temperature of 58°F. The chance of rain is 1%.

And let’s cross-check that with our function.

get_weather_data('34.0522,-118.2436') |> 
  jsonlite::fromJSON() |> 
  as_tibble() |> 
  filter(str_detect(time, 'T18:00'))
## # A tibble: 7 × 4
##   time                      temp_F rain_prob forecast         
##   <chr>                      <int>     <int> <chr>            
## 1 2025-02-08T18:00:00-08:00     58         1 Partly Cloudy    
## 2 2025-02-09T18:00:00-08:00     58         0 Mostly Cloudy    
## 3 2025-02-10T18:00:00-08:00     54         1 Partly Cloudy    
## 4 2025-02-11T18:00:00-08:00     54         5 Mostly Clear     
## 5 2025-02-12T18:00:00-08:00     52        50 Chance Light Rain
## 6 2025-02-13T18:00:00-08:00     56        92 Rain             
## 7 2025-02-14T18:00:00-08:00     54        25 Chance Light Rain

Hooray! This looks correct.