---
title: "Audio Workflows with Microsoft Foundry"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Audio Workflows with Microsoft Foundry}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
# This vignette runs against recorded, credential-free API fixtures. The
# fixtures are recorded once by a maintainer with real Azure OpenAI credentials
# (data-raw/record-doc-outputs.R) and committed under vignettes/audio/. When the
# fixtures are present every foundry_*() call below executes and shows its real
# output; when they are absent the API chunks are not evaluated so the vignette
# still builds anywhere without credentials. No output on this page is fabricated.
fixture_dir <- "audio"
recording <- nzchar(Sys.getenv("FOUNDRY_RECORD_DOCS"))
have_fixtures <- dir.exists(fixture_dir) && length(list.files(fixture_dir)) > 0
run_api <- requireNamespace("httptest2", quietly = TRUE) &&
  (recording || have_fixtures)

# Attach foundryR before start_vignette(): httptest2 only sources the package's
# inst/httptest2/start-vignette.R (which sets replay placeholders) from attached
# packages.
library(foundryR)

if (run_api) {
  httptest2::start_vignette(fixture_dir)
}

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = run_api
)

# Embed a self-contained <audio> player holding the real bytes foundry_speak()
# produced, so readers can play the output (not just read its size). On replay
# the file holds the recorded audio, so this works offline with no credentials.
embed_audio <- function(path) {
  if (!requireNamespace("base64enc", quietly = TRUE)) {
    return(invisible(NULL))
  }
  uri <- paste0("data:audio/mpeg;base64,", base64enc::base64encode(path))
  knitr::asis_output(
    sprintf(
      '<audio controls src="%s">Your browser does not support audio playback.</audio>',
      uri
    )
  )
}
```

```{r library, eval = TRUE}
library(foundryR)
```

Audio is one of the most useful Foundry additions for researchers. You can
transcribe interviews, lectures, field recordings, meetings, and focus groups,
then keep the result in a tibble with segment-level timing for downstream coding
or analysis.

## Two ways to reach speech models

foundryR can use audio models in two places:

- **Azure OpenAI deployments on your main resource** -- `whisper` for
  transcription and translation, and a text-to-speech model such as
  `gpt-4o-mini-tts` for synthesis. These reuse your main endpoint and key and
  are what the examples below use. Classic `whisper` is exposed only on the
  deployment path, so pass `api = "deployment"`.
- **A dedicated Speech (LLM Speech) resource** for MAI-Transcribe models. This
  chunk is illustrative and is not run:

```{r configure-speech, eval = FALSE}
# Only needed for MAI-Transcribe on a dedicated Speech resource:
foundry_set_speech_endpoint(Sys.getenv("AZURE_FOUNDRY_SPEECH_ENDPOINT"))
foundry_set_speech_key("your-speech-key")
```

## A real, public-domain sample

The examples below use a short excerpt from John F. Kennedy's 1961 inaugural
address ("And so, my fellow Americans..."). This clip ships with the package and
is the de facto "hello, world" of open-source speech recognition, so the
transcript is easy to check against a recording everyone knows.

```{r sample, eval = TRUE}
sample_audio <- system.file("extdata/samples/jfk.wav", package = "foundryR")
basename(sample_audio)
```

## Transcribe an audio file

`foundry_transcribe()` returns one row per file. The `text` column holds the
transcript and the `phrases` list-column holds segment-level timing. We use the
`whisper` deployment on the main resource; because classic whisper lives on the
deployment path we pass `api = "deployment"`, and `response_format =
"verbose_json"` asks the service for per-segment timing.

```{r transcribe}
transcript <- foundry_transcribe(
  sample_audio,
  service = "openai",
  model = "whisper",
  api = "deployment",
  response_format = "verbose_json"
)

transcript$text
```

The segment timing lives in the `phrases` list-column, one row per recognized
segment. A short clip like this one is a single segment; longer recordings
return many:

```{r transcribe-phrases}
head(transcript$phrases[[1]])
```

## Synthesize speech

`foundry_speak()` writes binary audio to disk and returns the file path and byte
count -- handy for experiment stimuli, accessibility assets, and demos. Use your
text-to-speech deployment name for `model`. The examples use temporary files and
remove them after use; choose an explicit path in your own workflow for audio
you want to keep.

```{r speak}
speech_path <- tempfile(fileext = ".mp3")
speech <- foundry_speak(
  "Hello, world.",
  model = "gpt-4o-mini-tts",
  voice = "alloy",
  path = speech_path
)

speech[, c("bytes", "model", "voice", "format")]
```

Those bytes are the real audio the model returned. Play them here when the
suggested `base64enc` package is installed:

```{r speak-play, echo = FALSE}
embed_audio(speech$path)
```

## Translate multilingual recordings

Use `foundry_translate_audio()` when you want an analysis corpus in a common
language. To keep the example fully reproducible we first synthesize a short
Spanish clip, then translate it to English with whisper -- both are real API
calls.

```{r translate-synth}
spanish_path <- tempfile(fileext = ".mp3")
spanish_clip <- foundry_speak(
  "La reunion fue muy util.",
  model = "gpt-4o-mini-tts",
  voice = "alloy",
  path = spanish_path
)
```

Listen to the synthesized Spanish input:

```{r translate-play, echo = FALSE}
embed_audio(spanish_clip$path)
```

Now translate it to English with the whisper deployment:

```{r translate}
translation <- foundry_translate_audio(
  spanish_clip$path,
  service = "openai",
  model = "whisper",
  api = "deployment"
)

translation$text
```

## Notes for researchers

- Inspect `head(transcript$phrases[[1]])` before processing long recordings so
  you know the segment structure your coding scheme has to handle.
- Keep raw audio out of your project repository; store transcripts and IDs.
- Request `response_format = "verbose_json"` to get segment-level timing from
  whisper; the default format returns the transcript text only.

```{r cleanup, include = FALSE, eval = TRUE}
if (run_api) {
  unlink(c(speech_path, spanish_path))
  httptest2::end_vignette()
}
```