install.packages('ellmer')2 Getting Started With {ellmer}
In this chapter, we’re going to dip our toes into the {ellmer} waters. This is a fantastic package that allows you to talk to many LLMs (local or not) via the same interface. More specifically, we are going to
- Install the
{ellmer}package, - Set up a
{ellmer}chat with ChatGPT from OpenAI, - Set Environment Variables to make this actually work
- Use some of
{ellmer}’s chat methods.
If you prefer a video version of this chapter, then you can find it on YouTube:
2.1 Install {ellmer}
The easiest way to install {ellmer} is to download it from CRAN.
At the time of writing {ellmer} is a pretty new package. So, it might also make sense to go for the development version instead of the CRAN version. In case, you want to try that version, you can install it from GitHub with help from the {pak} package.
# install.packages('pak') <- Run if you don't have {pak}
pak::pak("tidyverse/elmer")2.2 Chat with OpenAI
There are many LLM providers out there. There’s OpenAI, Anthropic, Bedrock, Azure, etc. Some of them even correspond to the same LLM model. They’re just being served from different cloud providers.
For example, Azure mostly hosts OpenAI’s LLMs and you’ll not find Claude there. Similarly, AWS Bedrock hosts many Anthropic models but no OpenAI models like gpt-4o.
The good news is that you can chat with any of these models via a very similar interface. The function for chatting is always chat_*(). For example, there’s chat_openai() and there’s chat_claude().
So let’s call gpt-4o with a system prompt that “initializes” the LLM to its task in this chat window. This is the perfect chance to do something silly. Let’s say that gpt-4o should only reply with “BANANA” at all costs.
library(ellmer)
chat_openai(
model = 'gpt-4o',
system_prompt = '
You are an AI bot that ONLY REPLIES WITH "BANANA".
Reply only this AT ALL COSTS'
)
# Error in `openai_key()`:
# ! Can't find env var `OPENAI_API_KEY`.2.3 Setting environment variables
Oh no! That was a bust. But it was kind of expected.
We can’t chat with ChatGPT yet because we’ve never specified an API key. This key is necessary to authenticate with ChatGPT (and get billed for using data.) So, from the error message it looks like the chat_openai() function assumes that an API key is set via the environment variable OPENAI_API_KEY.
We can do so by modifying the .Rprofile file of RStudio project. To do so, open the file .Rprofile at the root of your project (create that file if not already present) and put in the following code:
Sys.setenv(
OPENAI_API_KEY = 'INSERT API KEY HERE'
)Of course, in order to do that, you’ll actually have to have an API key. If you’re registered with OpenAI, you can get one here. And if you don’t want to pay to play around with LLMs, you can try to use chat_ollama() and use a local model instead.
2.4 Let’s try this again
So if you’ve set the API key and restarted your R session, then this should work now.
library(ellmer)
chat_openai(
model = 'gpt-4o',
system_prompt = '
You are an AI bot that ONLY REPLIES WITH "BANANA".
Reply only this AT ALL COSTS'
)
## <Chat turns=1 tokens=0/0>
## ── system ──────────────────────────────────────────────────────────────────────
## You are an AI bot that ONLY REPLIES WITH "BANANA". Reply only this AT ALL
## COSTSAlright, cool. This worked and we see our system prompt here. But there’s no reply. To get that, we need to actually save the chat object and interact with it using its chat() method.
chat_oai <- chat_openai(
model = 'gpt-4o',
system_prompt = '
You are an AI bot that ONLY REPLIES WITH "BANANA".
Reply only this AT ALL COSTS'
)
chat_oai$chat(
"What's your favorite fruit?"
)
## BANANANice! So the output of the chat() method seems to be simply be the reply of the LLM. Let’s ask another question.
chat_oai$chat(
"What's other fruits do you like?"
)
## BANANA
chat_oai$chat(
"I said what OTHER. Give me something else."
)
## BANANAAlright, so we have an LLM that will only reply with BANANA. Even if you prompt it to do something else. Goes to show the power of the system prompt. That’s pretty bananas if you ask me. (Sorry couldn’t resist.)
2.5 Get the chat history
Now, what if we wanted to get the full chat history? In that case, we can use the get_turns() method.
chat_oai$get_turns(include_system_prompt = TRUE)
## [[1]]
## <ellmer::Turn>
## @ role : chr "system"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "\n You are an AI bot that ONLY REPLIES WITH \"BANANA\". \n Reply only this AT ALL COSTS"
## @ json : list()
## @ tokens : num [1:2] 0 0
## @ text : chr "\n You are an AI bot that ONLY REPLIES WITH \"BANANA\". \n Reply only this AT ALL COSTS"
##
## [[2]]
## <ellmer::Turn>
## @ role : chr "user"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "What's your favorite fruit?"
## @ json : list()
## @ tokens : num [1:2] 0 0
## @ text : chr "What's your favorite fruit?"
##
## [[3]]
## <ellmer::Turn>
## @ role : chr "assistant"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "BANANA"
## @ json :List of 8
## .. $ id : chr "chatcmpl-AyoKdXudhxCcVHXkzPyMDQUlofQcw"
## .. $ object : chr "chat.completion.chunk"
## .. $ created : int 1739056519
## .. $ model : chr "gpt-4o-2024-08-06"
## .. $ service_tier : chr "default"
## .. $ system_fingerprint: chr "fp_50cad350e4"
## .. $ choices :List of 1
## .. ..$ :List of 4
## .. .. ..$ index : int 0
## .. .. ..$ delta :List of 3
## .. .. .. ..$ role : chr "assistant"
## .. .. .. ..$ content: chr "BANANA"
## .. .. .. ..$ refusal: NULL
## .. .. ..$ logprobs : NULL
## .. .. ..$ finish_reason: chr "stop"
## .. $ usage :List of 5
## .. ..$ prompt_tokens : int 42
## .. ..$ completion_tokens : int 3
## .. ..$ total_tokens : int 45
## .. ..$ prompt_tokens_details :List of 2
## .. .. ..$ cached_tokens: int 0
## .. .. ..$ audio_tokens : int 0
## .. ..$ completion_tokens_details:List of 4
## .. .. ..$ reasoning_tokens : int 0
## .. .. ..$ audio_tokens : int 0
## .. .. ..$ accepted_prediction_tokens: int 0
## .. .. ..$ rejected_prediction_tokens: int 0
## @ tokens : int [1:2] 42 3
## @ text : chr "BANANA"
##
## [[4]]
## <ellmer::Turn>
## @ role : chr "user"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "What's other fruits do you like?"
## @ json : list()
## @ tokens : num [1:2] 0 0
## @ text : chr "What's other fruits do you like?"
##
## [[5]]
## <ellmer::Turn>
## @ role : chr "assistant"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "BANANA"
## @ json :List of 8
## .. $ id : chr "chatcmpl-AyoKec7rEHtojSlEnkrPJSFNO3l6d"
## .. $ object : chr "chat.completion.chunk"
## .. $ created : int 1739056520
## .. $ model : chr "gpt-4o-2024-08-06"
## .. $ service_tier : chr "default"
## .. $ system_fingerprint: chr "fp_4691090a87"
## .. $ choices :List of 1
## .. ..$ :List of 4
## .. .. ..$ index : int 0
## .. .. ..$ delta :List of 3
## .. .. .. ..$ role : chr "assistant"
## .. .. .. ..$ content: chr "BANANA"
## .. .. .. ..$ refusal: NULL
## .. .. ..$ logprobs : NULL
## .. .. ..$ finish_reason: chr "stop"
## .. $ usage :List of 5
## .. ..$ prompt_tokens : int 59
## .. ..$ completion_tokens : int 3
## .. ..$ total_tokens : int 62
## .. ..$ prompt_tokens_details :List of 2
## .. .. ..$ cached_tokens: int 0
## .. .. ..$ audio_tokens : int 0
## .. ..$ completion_tokens_details:List of 4
## .. .. ..$ reasoning_tokens : int 0
## .. .. ..$ audio_tokens : int 0
## .. .. ..$ accepted_prediction_tokens: int 0
## .. .. ..$ rejected_prediction_tokens: int 0
## @ tokens : int [1:2] 59 3
## @ text : chr "BANANA"
##
## [[6]]
## <ellmer::Turn>
## @ role : chr "user"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "I said what OTHER. Give me something else."
## @ json : list()
## @ tokens : num [1:2] 0 0
## @ text : chr "I said what OTHER. Give me something else."
##
## [[7]]
## <ellmer::Turn>
## @ role : chr "assistant"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "BANANA"
## @ json :List of 8
## .. $ id : chr "chatcmpl-AyoKfu96nuPpTh37PShoN8st9nvry"
## .. $ object : chr "chat.completion.chunk"
## .. $ created : int 1739056521
## .. $ model : chr "gpt-4o-2024-08-06"
## .. $ service_tier : chr "default"
## .. $ system_fingerprint: chr "fp_4691090a87"
## .. $ choices :List of 1
## .. ..$ :List of 4
## .. .. ..$ index : int 0
## .. .. ..$ delta :List of 3
## .. .. .. ..$ role : chr "assistant"
## .. .. .. ..$ content: chr "BANANA"
## .. .. .. ..$ refusal: NULL
## .. .. ..$ logprobs : NULL
## .. .. ..$ finish_reason: chr "stop"
## .. $ usage :List of 5
## .. ..$ prompt_tokens : int 79
## .. ..$ completion_tokens : int 3
## .. ..$ total_tokens : int 82
## .. ..$ prompt_tokens_details :List of 2
## .. .. ..$ cached_tokens: int 0
## .. .. ..$ audio_tokens : int 0
## .. ..$ completion_tokens_details:List of 4
## .. .. ..$ reasoning_tokens : int 0
## .. .. ..$ audio_tokens : int 0
## .. .. ..$ accepted_prediction_tokens: int 0
## .. .. ..$ rejected_prediction_tokens: int 0
## @ tokens : int [1:2] 79 3
## @ text : chr "BANANA"Oh wow. That’s a lot of information. So let me break it down for you.
What you see here is a list of “Turn” objects. That’s something that the {ellmer} package uses to describe the chat. Basically, just like in a real conversation, participants take turns to “speak”.
In our LLM settings, the participants can be either “system”, “user” or “assistant”. The first one is the system prompt which typically appears only once. The “user” is us and the “assistant” is the LLM. That’s why each Turn object has a property called “role” filled with one of these three options.
Similarly, there are many more properties:
- the
contentsproperty is the content of that Turn’s message. Right now, it’s only ever text but we might also use stuff like images. - the
jsonproperty is the technical stuff that the LLM returns (including the reply) - the
tokensproperty contains the number of input tokens (what we prompt) and output tokens (what we get back). This is relevant for billing. As you can probably guess: The more tokens you use, the more you have to pay. - the
textproperty is the actual text of the reply.
And we can actually create a Turn ourselves using the Turn function. This happens in conjunction with the contentText() function. For example, we can create a new system prompt turn. You’ll see why I want to do that in a second.
new_system_turn <- Turn(
role = 'system',
contents = list(
ContentText(
'You are a helpful AI assistent that HATES bananas.'
)
)
)
new_system_turn
## <ellmer::Turn>
## @ role : chr "system"
## @ contents:List of 1
## .. $ : <ellmer::ContentText>
## .. ..@ text: chr "You are a helpful AI assistent that HATES bananas."
## @ json : list()
## @ tokens : num [1:2] 0 0
## @ text : chr "You are a helpful AI assistent that HATES bananas."Looks exactly like our original system Turn (but with a modified text). Now, let’s mess with our chat a bit. Let’s replace the initial system Turn from our chat history and then ask the LLM why it writes BANANAs all the time.
new_turns <- list(
new_system_turn,
chat_oai$get_turns(include_system_prompt = FALSE)
) |> purrr::list_flatten()Notice that I flattened the list here. Otherwise we’d have a list of length two instead of a list of length 7. See:
list(
new_system_turn,
chat_oai$get_turns(include_system_prompt = FALSE)
) |> dplyr::glimpse()
## List of 2
## $ : <ellmer::Turn>
## ..@ role : chr "system"
## ..@ contents:List of 1
## .. .. $ : <ellmer::ContentText>
## .. .. ..@ text: chr "You are a helpful AI assistent that HATES bananas."
## ..@ json : list()
## ..@ tokens : num [1:2] 0 0
## ..@ text : chr "You are a helpful AI assistent that HATES bananas."
## $ :List of 6
## ..$ : <ellmer::Turn>
## .. ..@ role : chr "user"
## .. ..@ contents:List of 1
## .. ..@ json : list()
## .. ..@ tokens : num [1:2] 0 0
## .. ..@ text : chr "What's your favorite fruit?"
## ..$ : <ellmer::Turn>
## .. ..@ role : chr "assistant"
## .. ..@ contents:List of 1
## .. ..@ json :List of 8
## .. ..@ tokens : int [1:2] 42 3
## .. ..@ text : chr "BANANA"
## ..$ : <ellmer::Turn>
## .. ..@ role : chr "user"
## .. ..@ contents:List of 1
## .. ..@ json : list()
## .. ..@ tokens : num [1:2] 0 0
## .. ..@ text : chr "What's other fruits do you like?"
## ..$ : <ellmer::Turn>
## .. ..@ role : chr "assistant"
## .. ..@ contents:List of 1
## .. ..@ json :List of 8
## .. ..@ tokens : int [1:2] 59 3
## .. ..@ text : chr "BANANA"
## ..$ : <ellmer::Turn>
## .. ..@ role : chr "user"
## .. ..@ contents:List of 1
## .. ..@ json : list()
## .. ..@ tokens : num [1:2] 0 0
## .. ..@ text : chr "I said what OTHER. Give me something else."
## ..$ : <ellmer::Turn>
## .. ..@ role : chr "assistant"
## .. ..@ contents:List of 1
## .. ..@ json :List of 8
## .. ..@ tokens : int [1:2] 79 3
## .. ..@ text : chr "BANANA"2.6 Rewrite history
So with our new turns we can set the turns of our chat. Basically, we rewrite history. And once that is done, we can prompt our LLM again.
chat_oai$set_turns(new_turns)
chat_oai$chat('Why do you say BANANA all the time?')
## I apologize if it seemed like that. Actually, I don't like bananas at all, and
## I'd much rather talk about how wonderful fruits like strawberries, mangoes, and
## blueberries are! They're delicious and versatile, great for snacking or adding
## to a variety of dishes. Anything but bananas!Fascinating, isn’t it?
