The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
A Tool for Rating Text/Image/Audio Stimuli via Large Language Models
CRAN maintenance release — no new user-facing
features. 1.3.1 fixes two issues surfaced by
R CMD check on CRAN incoming:
stats::aggregate() import added to
NAMESPACE (was flagged as undefined global function in
rate_openai_weighted()).rank argument of alignment() is now
documented in man/alignment.Rd (was triggering a
Codoc mismatches WARNING).inst/CITATION is now a single APA 7 entry pointing at
the PsyArXiv preprint (doi:10.31234/osf.io/mje6w_v1).R CMD check is clean on CRAN incoming (Windows Server
2022, R-devel r90190, ucrt): Status: OK.
The probability-weighted scoring mode
(method = "weighted", top_logprobs,
include_probs) introduced in 1.3.0 remains the headline
feature of this release line — see the What’s New in 1.3.0
section below for details.
| Provider | Description | API Key Required? |
|---|---|---|
| openai | OpenAI GPT models | Yes |
| anthropic | Anthropic Claude models | Yes |
| ollama | Local models via Ollama | No |
| lmstudio | Local models via LM Studio | No |
| deepseek | DeepSeek models | Yes |
| groq | Groq inference | Yes |
| mistral | Mistral models | Yes |
| openrouter | Unified access to many models | Yes |
| openai_compatible | Custom endpoints (vLLM, etc.) | Depends |
# Install from CRAN (production version)
install.packages("chatRater")
pak::pkg_install("chatRater")
# Install from GitHub (development version)
remotes::install_github("ShiyangZheng/chatRater")library(chatRater)
# Basic usage with OpenAI
stim <- 'The early bird catches the worm'
res <- generate_ratings(
model = 'gpt-4o',
stim = stim,
provider = 'openai',
api_key = Sys.getenv("OPENAI_API_KEY"),
prompt = 'You are an expert in figurative language.',
question = 'Rate the creativity of this phrase on a scale of 1-10:',
scale = '1-10'
)
# Using Anthropic Claude
res <- generate_ratings(
model = 'claude-sonnet-4-20250514',
stim = stim,
provider = 'anthropic',
api_key = Sys.getenv("ANTHROPIC_API_KEY"),
scale = '1-5'
)# Make sure Ollama is running first
# Download from: https://ollama.com
res <- generate_ratings(
stim = 'Bite the bullet',
provider = 'ollama',
model = 'llama3.2',
scale = '1-7',
n_iterations = 3
)
# Or with LM Studio (run on port 1234 by default)
res <- generate_ratings(
stim = 'Hit the nail on the head',
provider = 'lmstudio',
model = 'your-model-name',
scale = '1-5'
)stim_list <- c('Kick the bucket', 'Beat around the bush', 'Cut to the chase')
results <- generate_ratings_for_all(
stim_list = stim_list,
provider = 'ollama',
model = 'llama3.2',
scale = '1-7',
n_iterations = 5
)# Rate an image from URL
res <- generate_ratings(
stim = 'https://example.com/image.jpg',
provider = 'openai',
model = 'gpt-4o',
api_key = Sys.getenv("OPENAI_API_KEY"),
question = 'Rate the visual quality:',
scale = '1-10'
)
# Rate a local image file
res <- generate_ratings(
stim = '/path/to/image.png',
provider = 'anthropic',
api_key = Sys.getenv("ANTHROPIC_API_KEY"),
scale = '1-5'
)# Audio is transcribed via OpenAI Whisper, then rated
res <- generate_ratings(
stim = '/path/to/audio.mp3',
provider = 'openai',
model = 'gpt-4o',
api_key = Sys.getenv("OPENAI_API_KEY"),
prompt = 'You are rating audio transcripts.',
question = 'Rate the formality of this speech:',
scale = '1-10'
)# Numeric (default): extracts numbers from LLM response
res <- generate_ratings(
stim = 'test',
provider = 'ollama',
model = 'llama3.2',
return_type = 'numeric',
scale = '1-10'
)
# Text: returns full LLM response text
res <- generate_ratings(
stim = 'test',
provider = 'ollama',
model = 'llama3.2',
return_type = 'text'
)
# Raw: returns raw API response
res <- generate_ratings(
stim = 'test',
provider = 'ollama',
model = 'llama3.2',
return_type = 'raw'
)# Option 1: Set environment variables in .Renviron
# OPENAI_API_KEY=sk-...
# ANTHROPIC_API_KEY=sk-ant-...
# Option 2: Pass directly
generate_ratings(
api_key = 'sk-...',
...
)
# Option 3: Use Sys.getenv()
generate_ratings(
api_key = Sys.getenv("OPENAI_API_KEY"),
...
)# For vLLM or other OpenAI-compatible servers
res <- generate_ratings(
stim = 'test',
provider = 'openai_compatible',
base_url = 'http://localhost:8080/v1',
api_key = NULL, # or your API key if required
model = 'your-model'
)If you use chatRater in your research, please cite the associated preprint (APA 7):
Zheng, S. (2026, May 16). chatRater: Validating LLM-Generated Psycholinguistic Norms with Probability-Weighted Scoring [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/mje6w_v1
You can also get these citations from within R:
citation("chatRater")chatRater 1.3.1 depends on: - llmcoder (>= 1.2.0): LLM integration backend - base64enc: For encoding local files - tools: For file extension detection - httr2: For HTTP requests - curl: For file uploads - jsonlite: For JSON parsing
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.