---
title: "Working with Embeddings"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Working with Embeddings}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
# This vignette runs against recorded, credential-free API fixtures. When the
# fixtures are present (they are recorded once with data-raw/record-doc-outputs.R
# and committed under vignettes/embeddings/), every foundry_*() call below is
# executed and its real output is shown. When they are absent, the API chunks are
# not evaluated so the vignette still builds anywhere without Azure credentials.
fixture_dir <- "embeddings"
recording <- nzchar(Sys.getenv("FOUNDRY_RECORD_DOCS"))
have_fixtures <- dir.exists(fixture_dir) && length(list.files(fixture_dir)) > 0
run_api <- requireNamespace("httptest2", quietly = TRUE) &&
  (recording || have_fixtures)

# Attach foundryR before start_vignette(): httptest2 only sources the package's
# inst/httptest2/start-vignette.R (which sets replay placeholders) from attached
# packages.
library(foundryR)

if (run_api) {
  httptest2::start_vignette(fixture_dir)
}

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = run_api
)
```

## What embeddings are for

Embeddings are numerical representations of text that capture semantic meaning.
When you convert text to an embedding, you get a vector of numbers (often 1,536
or 3,072 dimensions depending on the model). Texts with similar meanings tend to
have similar vectors.

Use embeddings when you need to:

- Find documents related to a query by meaning, not only keywords.
- Measure how similar two pieces of text are.
- Cluster open-ended responses into themes.
- Find near-duplicate responses or records.
- Feed text-derived numeric predictors into downstream models.

Unlike keyword matching, embeddings understand that "automobile" and "car" are
semantically similar, even though they share no letters.

## Generating embeddings with foundry_embed()

The examples below embed real sentences from Jane Austen's *Pride and Prejudice*,
available in the `janeaustenr` package. Using a well-known public-domain text
makes the output easy to reason about: the opening lines share vocabulary and
sentiment, so their embeddings should sit close together.

```{r austen-lines}
library(foundryR)

austen_lines <- c(
  "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.",
  "However little known the feelings or views of such a man may be on his first entering a neighbourhood.",
  "Mr. Bennet was so odd a mixture of quick parts, sarcastic humour, reserve, and caprice."
)

embedding <- foundry_embed(austen_lines[1], model = "text-embedding-3-small")
embedding
```

The result is a tibble with:

- `text`: the original input text.
- `embedding`: a list-column holding the numeric vector.
- `n_dims`: the dimensionality of the embedding.

### Embedding multiple texts

Pass a character vector to embed several texts in one call:

```{r multiple-embed}
doc_embeddings <- foundry_embed(austen_lines, model = "text-embedding-3-small")
doc_embeddings
```

### Controlling dimensions

The `text-embedding-3-small` and `text-embedding-3-large` models can return
shorter vectors. Smaller dimensions mean faster similarity computations and less
storage, with some trade-off in precision:

```{r reduced-dims}
compact <- foundry_embed(
  austen_lines[1],
  model = "text-embedding-3-small",
  dimensions = 256
)
compact$n_dims
```

## Computing similarity with foundry_similarity()

Cosine similarity measures how close two embeddings are, from -1 (opposite) to 1
(identical). `foundry_similarity()` computes every pairwise similarity in a tibble
of embeddings. Here we contrast the Austen lines with two sentences from a
different domain so the split is visible.

```{r similarity}
mixed <- c(
  "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.",
  "Mr. Bennet was so odd a mixture of quick parts, sarcastic humour, reserve, and caprice.",
  "The quarterly revenue report showed a sharp rise in cloud subscriptions.",
  "Analysts raised their earnings forecast after the strong cloud numbers."
)

similarities <- foundry_embed(mixed, model = "text-embedding-3-small") |>
  foundry_similarity()
similarities
```

Results are sorted by similarity. The two Austen lines pair together and the two
finance lines pair together, while cross-domain pairs score lower.
The plots in this vignette are shown when the suggested `ggplot2` package is
installed.

```{r similarity-heatmap, echo = FALSE, eval = run_api && requireNamespace("ggplot2", quietly = TRUE), fig.alt = "Heatmap of cosine similarity across four sentences from two domains."}
label_for <- function(x) {
  dplyr::case_when(
    startsWith(x, "It is a truth") ~ "Austen 1",
    startsWith(x, "Mr. Bennet") ~ "Austen 2",
    startsWith(x, "The quarterly") ~ "Finance 1",
    TRUE ~ "Finance 2"
  )
}

heatmap_data <- similarities |>
  dplyr::mutate(
    a = label_for(text_1),
    b = label_for(text_2)
  )

ggplot2::ggplot(heatmap_data, ggplot2::aes(x = b, y = a, fill = similarity)) +
  ggplot2::geom_tile(color = "white", linewidth = 0.6) +
  ggplot2::geom_text(ggplot2::aes(label = sprintf("%.2f", similarity)), size = 3.2) +
  ggplot2::scale_fill_gradient(low = "#E6F2FB", high = "#0078D4") +
  ggplot2::labs(
    title = "Cosine similarity separates the two domains",
    x = NULL, y = NULL, fill = "Similarity"
  ) +
  ggplot2::theme_minimal(base_size = 12) +
  ggplot2::theme(panel.grid = ggplot2::element_blank(), legend.position = "bottom")
```

## Use case: finding similar documents

A common application is ranking documents by relevance to a query. Embed the
documents and the query, then sort by cosine similarity:

```{r semantic-search}
library(dplyr)

documents <- c(
  "How to install R packages using install.packages()",
  "Data visualization with ggplot2 in R",
  "Introduction to machine learning with Python",
  "Statistical hypothesis testing explained",
  "Building web applications with Shiny",
  "Deep learning with TensorFlow and Keras"
)

doc_embeddings <- foundry_embed(documents, model = "text-embedding-3-small")
query_embedding <- foundry_embed(
  "How do I create charts and graphs in R?",
  model = "text-embedding-3-small"
)

cosine <- function(a, b) sum(a * b) / (sqrt(sum(a^2)) * sqrt(sum(b^2)))
query_vec <- query_embedding$embedding[[1]]

doc_embeddings |>
  mutate(similarity = vapply(embedding, cosine, numeric(1), b = query_vec)) |>
  arrange(desc(similarity)) |>
  select(text, similarity) |>
  head(3)
```

## Use case: clustering text

Embeddings work well as features for clustering. Here `stats::kmeans()` groups a
mix of programming, food, and sports sentences without any labels:

```{r clustering}
texts <- c(
  "Python is great for machine learning",
  "R excels at statistical analysis",
  "JavaScript powers modern web applications",
  "Italian pasta with tomato sauce",
  "Sushi is a popular Japanese dish",
  "French croissants are flaky and buttery",
  "Soccer is the world's most popular sport",
  "Basketball requires speed and agility",
  "Tennis matches can last for hours"
)

cluster_embeddings <- foundry_embed(texts, model = "text-embedding-3-small")
embedding_matrix <- do.call(rbind, cluster_embeddings$embedding)

set.seed(42)
clusters <- kmeans(embedding_matrix, centers = 3, nstart = 10)

cluster_embeddings |>
  mutate(cluster = clusters$cluster) |>
  arrange(cluster) |>
  select(text, cluster)
```

```{r projection, echo = FALSE, eval = run_api && requireNamespace("ggplot2", quietly = TRUE), fig.alt = "Two-dimensional PCA projection of sentence embeddings, colored by k-means cluster."}
pca <- prcomp(embedding_matrix, rank. = 2)
projection <- tibble::tibble(
  pc1 = pca$x[, 1],
  pc2 = pca$x[, 2],
  cluster = factor(clusters$cluster),
  label = substr(texts, 1, 18)
)

ggplot2::ggplot(projection, ggplot2::aes(pc1, pc2, color = cluster, label = label)) +
  ggplot2::geom_point(size = 3.2, alpha = 0.9) +
  ggplot2::geom_text(nudge_y = 0.15, size = 3, show.legend = FALSE) +
  ggplot2::scale_color_manual(values = c("#0078D4", "#107C10", "#5C2D91")) +
  ggplot2::labs(
    title = "A PCA projection makes the clusters visible",
    x = "PC 1", y = "PC 2", color = "Cluster"
  ) +
  ggplot2::theme_minimal(base_size = 12) +
  ggplot2::theme(legend.position = "bottom", panel.grid.minor = ggplot2::element_blank())
```

The clusters recover the three topics from the raw text alone.

## Tips for working with embeddings

### Choosing a model

| Model | Dimensions | Notes |
|-------|-----------|-------|
| text-embedding-ada-002 | 1,536 | Previous generation, widely used |
| text-embedding-3-small | 1,536 (configurable) | Newer, supports dimension reduction |
| text-embedding-3-large | 3,072 (configurable) | Highest quality, more expensive |

For most use cases, `text-embedding-3-small` balances quality and cost.

### Dimension trade-offs

Higher dimensions capture more nuance but need more storage, take longer to
compare, and may not improve simple tasks. Consider reduced dimensions (256-512)
for large-scale applications where speed matters more than precision.

### Handling large collections

1. **Batch processing**: embed documents in batches to respect rate limits.
2. **Caching**: store embeddings in a database rather than regenerating them.
3. **Approximate nearest neighbors**: use libraries like `RcppAnnoy` for fast
   similarity search on large datasets.

```{r batch-example, eval = TRUE}
# Defining this helper is local; calling it requires Azure credentials.
batch_embed <- function(texts, model, batch_size = 100) {
  n_batches <- ceiling(length(texts) / batch_size)
  results <- vector("list", n_batches)
  for (i in seq_len(n_batches)) {
    start_idx <- (i - 1) * batch_size + 1
    end_idx <- min(i * batch_size, length(texts))
    results[[i]] <- foundry_embed(texts[start_idx:end_idx], model = model)
    Sys.sleep(0.5)
  }
  dplyr::bind_rows(results)
}
```

```{r cleanup, include = FALSE}
if (run_api) {
  httptest2::end_vignette()
}
```