1  Sending data.frames to LLMs with {mall}

In this chapter, I’ll show you how to send data from data.frames to an AI. And the best part is that you don’t even have to pay for any of the AI vendors like ChatGPT to do so. Instead we’ll use local LLMs with help from the {mall} package. And in case you’re wondering what “local” means here, well, it means that we don’t use a paid LLM like ChatGPT. Instead, we can run our own LLM on our computer.

And while this may sound like a cheap knock-off, it turns out that open-source LLMs have become pretty powerful and less resource intensive over the last couple of months. So using a local LLM may be perfectly well-suited for your purposes. Let’s give them a try.

And as always, you can check out the video version of this chapter on YouTube:

1.1 Install instructions

1.1.1 Install the {mall} package

The first thing that we need is the {mall} package. Nothing easier than that. Just run your favorite command that installs packages from CRAN.

install.packages('mall')

1.1.2 Install ollama

But wait, there is more. All LLMs will be served locally via ollama. Serve locally, you say? Never heard of that?

Well, in that case, let me explain. All LLMs (yes, ChatGPT too) are basically a very large neural network. Sounds biological but can in fact be represented by a VEEEERY large matrix of so called weights (I’m oversimplifying here but bear with me.) These weights will be used to determine what kind of response the LLM produces. So, if you have that matrix, you can run your own ChatGPT.

Of course, openAI keeps its matrix under wraps but there are other open-source folks who say: “Here, stranger, take my matrix and run your own calculations with that.” Cool beans but we hate doing calculations manually, right? That’s where ollama comes in.

It’s a tool that can run on your computer and download all the matrices from all kinds of other vendors. So if you, or rather your R code, wants to send a prompt to an LLM, then you (or your R code) will knock at ollama’s door (which is running in the background) and say “Hey, run this prompt against <LLM-Model-XYZ>!” And then ollama is hopefully nice and says: “Lucky you, I’ve got <LLM-Model-XYZ> in store. I’m serving you that nice dish LLM.”

Okay, so hopefully this dumb little explainer demystified this stuff about “serving LLMs” a little bit. So with that said, head over to the ollama and install it for your operating system.

1.1.3 Get a local model through {ollamar}

Nicey, nicey! We have {mall} installed for R and we have ollama installed on our computer. Time to get an LLM. That’s where the {ollamar} package comes in. It’s one of the packages that is also installed when you install {mall}. This handles the interaction between R and ollama.

Inside of R, we can run the pull() command to download an LLM (you know, the matrix that represents the fancy neural network). You can find a whole list of LLMs that ollama can download for you on their website. Here, we’re just going to use a tiny model called “llama3.2”.

ollamar::pull('llama3.2')

This may take a while and unfortunately {ollamar} doesn’t give you any hint on how long it takes to download all the things. But if you want to be sure that ollama works properly in the background, use this command:

ollamar::test_connection()
## Ollama local server running
## <httr2_response>
## GET http://localhost:11434/
## Status: 200 OK
## Content-Type: text/plain
## Body: In memory (17 bytes)

In any case, after the download finishes, you should see something like this:

ollamar::pull('llama3.2')
## <httr2_response>
## POST http://127.0.0.1:11434/api/pull
## Status: 200 OK
## Content-Type: application/x-ndjson
## Body: In memory (270814 bytes)

1.1.4 Specify a model for {mall}

Now that we have installed everything, it’s time to configure our local model. Since ollama can serve many models, we should be explicit about which models using which parameters it should use. This can be done via the llm_use() function.

library(mall)
llm_use(
  backend = "ollama", 
  model = "llama3.2", 
  seed = 1337, 
  temperature = 0
)
## 
## ── mall session object
## Backend: ollama
## LLM session:
##   model:llama3.2
## 
##   seed:1337
## 
##   temperature:0
## 
## R session: cache_folder:/tmp/RtmpGc2jKE/_mall_cache80c057138134

Don’t worry too much about the parameters here. Just know that you can tweak them if you ever feel the desire.

1.2 Sentiment analysis

Now it’s time run our data against our llama LLM. Speaking about data, we actually need some. Let’s create a fake tibble then.

tib <- tibble::tibble(
  id = 1:4,
  adjective = c('painful', 'hungry', 'cheerful', 'exciting')
)

We can take this data and run it against our LLM to try to determine whether our adjectives are “negative”, “neutral” or “positive”. This is called a sentiment analysis and is available in {mall} via the llm_sentiment() function.

tib |> 
  llm_sentiment(col = adjective)
## # A tibble: 4 × 3
##      id adjective .sentiment
##   <int> <chr>     <chr>     
## 1     1 painful   negative  
## 2     2 hungry    negative  
## 3     3 cheerful  positive  
## 4     4 exciting  positive

Wow, that was easy. And it was fast too. Since llama3.2 is quite a small model the calculations finished quickly. Of course, larger models may deliver better results but they may also run longer or even require compute resources that your machine doesn’t have. In any case, let’s move on to another use case.

1.3 Classification

The next use case is pretty similar. Instead of grouping things into “neutral”, “positive” and “negative”, we can group things into other kinds of categories. Let’s take fruits and assigning them to colors.

tibble::tibble(
  fruit = stringr::fruit[1:5]
) |> 
  llm_classify(
    col = fruit,
    labels = c('red', 'green', 'blue', 'yellow')
  )
## ! There were 2 predictions with invalid output, they were coerced to NA
## # A tibble: 5 × 2
##   fruit       .classify
##   <chr>       <chr>    
## 1 apple       <NA>     
## 2 apricot     yellow   
## 3 avocado     green    
## 4 banana      <NA>     
## 5 bell pepper green

Well, that wasn’t great. Not exactly the results we were hoping for. Let’s try to use a larger Llama model then. Maybe using a Llama3.1 model with 8 billion parameters is better.

llm_use(
  backend = "ollama", 
  model = "llama3.1:8b", 
  seed = 1337, 
  temperature = 0
)
## 
## ── mall session object
## Backend: ollama
## LLM session:
##   model:llama3.1:8b
## 
##   seed:1337
## 
##   temperature:0
## 
## R session: cache_folder:/tmp/RtmpGc2jKE/_mall_cache80c057138134
tibble::tibble(
  fruit = stringr::fruit[1:5]
) |> 
  llm_classify(
    col = fruit,
    labels = c('red', 'green', 'blue', 'yellow')
  )
## # A tibble: 5 × 2
##   fruit       .classify
##   <chr>       <chr>    
## 1 apple       red      
## 2 apricot     yellow   
## 3 avocado     green    
## 4 banana      yellow   
## 5 bell pepper green

Well, this looks much better. Hoooray!

1.4 So what’s the difference between classification and sentiment analysis?

Classification and sentiment analysis are kind of the same. If anything, you can think of sentiment analysis as being a special case of classification. But if you don’t believe me, let’s do a little bit of R archeology. If we take a look into llm_sentiment(), we notice that it’s just a wrapper around llm_vec_sentiment().

mall:::llm_sentiment.data.frame
## function(.data,
##                                      col,
##                                      options = c("positive", "negative", "neutral"),
##                                      pred_name = ".sentiment",
##                                      additional_prompt = "") {
##   mutate(
##     .data = .data,
##     !!pred_name := llm_vec_sentiment(
##       x = {{ col }},
##       options = options,
##       additional_prompt = additional_prompt
##     )
##   )
## }
## <bytecode: 0x5f8dcce38310>
## <environment: namespace:mall>

And if you follow this trail, you’ll realize that

• llm_vec_sentiment() calls m_vec_prompt(),
• m_vec_prompt() calls m_backend_prompt(), and
• m_backend_prompt() calls mall:::m_backend_prompt.mall_session.

There, you’ll find the default prompts for classification and sentiment analysis. You’ll notice that apart from the naming of arguments they are pretty much the same. Here’s the one for classification

# ...excerpt from `mall:::m_backend_prompt.mall_session
classify = function(labels) {
  labels <- process_labels(
    x = labels, 
    if_character = "Determine if the text refers to one of the following: {x}", 
    if_formula = "- For {f_lhs(x)}, return {f_rhs(x)}"
  )
  list(
    list(
      role = "user", 
      content = glue(
        paste(
          "You are a helpful classification engine.", 
          "{labels}.", 
          "No capitalization. No explanations.", 
          "{additional}", 
          "The answer is based on the following text:\n{{x}}"
        )
      )
    )
  )
}

And here’s the one for sentiment.

sentiment = function(options) {
  options <- process_labels(
    x = options, 
    if_character = "Return only one of the following answers: {x}", 
    if_formula = "- If the text is {f_lhs(x)}, return {f_rhs(x)}"
  )
  list(
    list(
      role = "user", 
      content = glue(
        paste(
          "You are a helpful sentiment engine.",
          "{options}.", 
          "No capitalization. No explanations.",
          "{additional}", 
          "The answer is based on the following text:\n{{x}}"
        )
      )
    )
  )
}

Looks pretty identical to me. Anyway, I wanted to give you a quick glimpse behind the scenes of AI magic. As you can see, many things are nowadays reduced to a simple prompt. For more use cases of how to use {mall}, check out its excellent documentation.