What makes mLLMCelltype different from other cell type annotation tools?

mLLMCelltype differs from traditional cell type annotation tools in several key ways:

1. No reference dataset required: Unlike reference-based methods, mLLMCelltype doesn’t require a pre-existing reference dataset.

2. Multi-model consensus: mLLMCelltype leverages multiple large language models to achieve more reliable annotations than any single model could provide.

3. Transparent reasoning: The package provides complete reasoning chains for annotations, making the process interpretable and transparent.

4. Uncertainty quantification: mLLMCelltype provides explicit uncertainty metrics (consensus proportion and Shannon entropy) to identify ambiguous cell populations.

5. Structured deliberation: For controversial clusters, mLLMCelltype initiates a structured discussion process among models to reach a more reliable consensus.

Which tissues and species does mLLMCelltype support?

mLLMCelltype can annotate cell types from virtually any tissue and species, as it relies on the biological knowledge embedded in large language models rather than pre-defined reference datasets. However, performance may vary depending on how well-characterized the tissue is in the scientific literature.

The package has been extensively tested on: - Human tissues (PBMC, bone marrow, brain, lung, liver, kidney, etc.) - Mouse tissues (brain, lung, kidney, etc.) - Other model organisms (zebrafish, fruit fly, etc.)

For very specialized or poorly characterized tissues, the uncertainty metrics will help identify clusters that may require expert review.

How accurate is mLLMCelltype compared to other methods?

In our benchmarks (Yang et al., 2025; see our paper), the consensus approach showed improvements over both traditional annotation methods and single-LLM approaches:

• Compared to reference-based methods (e.g., SingleR, Seurat label transfer), mLLMCelltype showed comparable or better performance without requiring a reference dataset.
• Compared to marker-based methods (e.g., SCINA, CellAssign), mLLMCelltype demonstrated higher accuracy and flexibility.
• Compared to single-LLM approaches, the consensus mechanism showed accuracy improvements on tested datasets.

The accuracy advantage is particularly pronounced for rare cell types and tissues with limited reference data.

How does mLLMCelltype handle cluster indices?

mLLMCelltype preserves your original cluster IDs as-is. Whether your clusters are numbered 0, 1, 2 (Seurat default) or 1, 2, 3 (R convention), or use custom names like “t_cells”, the package will use them directly without modification. The returned annotations use the same cluster IDs as your input.

What is the recommended number of marker genes per cluster?

The default setting uses the top 10 marker genes per cluster, which works well for most scenarios. However, you can adjust this using the top_gene_count parameter:

• For well-characterized cell types: 5-10 marker genes is usually sufficient
• For rare or poorly characterized cell types: 10-20 marker genes may be beneficial
• For noisy data: Fewer genes (5-7) might give better results by focusing on the strongest signals

The optimal number depends on the quality of your marker genes and the complexity of the tissue. We recommend starting with the default of 10 and adjusting based on the results.

How does caching work in mLLMCelltype?

mLLMCelltype implements a caching system to avoid redundant API calls, which saves time and reduces costs:

• By default, caching is enabled (cache = TRUE)
• The cache is based on a hash of the input data, model, and other parameters
• Results are stored in a local directory (default: a temporary directory)
• You can specify a custom cache directory using the cache_dir parameter

To clear the cache:

cache_manager <- CacheManager$new()
cache_manager$clear_cache()

Note: The annotate_cell_types function does not have built-in caching. If you need caching, you’ll need to implement it separately.

How does mLLMCelltype handle rate limits and API errors?

The package includes error handling for API calls:

• Detailed error messages: When an API call fails, the error message includes details to help diagnose the issue

If you’re processing many clusters, you might encounter rate limits. In this case:

1. Reduce the number of models used for initial annotation
2. Process batches of clusters separately with pauses between batches
3. Consider implementing your own retry mechanism if needed

How long does it take to run mLLMCelltype?

The runtime depends on several factors:

• Number of clusters: Each cluster requires separate API calls
• Number of models: More models means more API calls
• Discussion process: Controversial clusters require additional API calls for discussion
• API response times: Different providers have different response times
• Network conditions: Internet speed and reliability affect performance

Typical runtimes: - Single model, 10 clusters: 1-2 minutes - Multi-model consensus (3 models), 10 clusters: 3-5 minutes - Multi-model consensus with discussion, 10 clusters: 5-10 minutes

To optimize runtime: - Implement your own caching mechanism if needed - Start with fewer models for initial exploration - Use a higher controversy_threshold to reduce the number of controversial clusters - Process large datasets in batches

What are the API costs associated with using mLLMCelltype?

The API costs depend on the models you use and the number of clusters:

• OpenAI models (GPT-4o, etc.): $0.01-0.05 per cluster for annotation
• Anthropic models (Claude 3.7, etc.): $0.01-0.03 per cluster for annotation
• Google models (Gemini 1.5, etc.): $0.001-0.01 per cluster for annotation
• Other models: Generally lower cost
• OpenRouter free models: $0.00 (free with :free suffix)

For a typical dataset with 10-20 clusters: - Single model annotation: $0.10-1.00 total - Multi-model consensus (3 models): $0.30-3.00 total - With discussion process: Additional $0.10-1.00 - Using OpenRouter free models: $0.00 total

To reduce costs: - Implement your own caching mechanism to avoid redundant API calls - Start with more economical models - Use fewer models for initial exploration - Reserve multi-model consensus for final analysis - Consider using OpenRouter free models (see below)

How can I use OpenRouter free models?

OpenRouter provides access to several high-quality models for free:

1. Sign up for an OpenRouter account at openrouter.ai

2. Get your API key from the OpenRouter dashboard

3. Use models with the :free suffix:

# Set your OpenRouter API key
Sys.setenv(OPENROUTER_API_KEY = "your-openrouter-api-key")

# Use a free model
results <- annotate_cell_types(
  input = marker_data,
  tissue_name = "human PBMC",
  model = "meta-llama/llama-4-maverick:free",  # Note the :free suffix
  api_key = Sys.getenv("OPENROUTER_API_KEY")
  # No need to specify provider - it's automatically detected from the model name format
)

4. Recommended free models (Updated Oct 2025):
   • meta-llama/llama-4-maverick:free - Meta Llama 4 Maverick (256K context, best performance)
   • deepseek/deepseek-r1:free - DeepSeek R1 (advanced reasoning)
   • meta-llama/llama-3.3-70b-instruct:free - Meta Llama 3.3 70B (reliable)
   • venice/uncensored:free - Venice Uncensored (new model)
   • minimax/minimax-m2:free - MiniMax M2 (optimized for coding)
   • z-ai/glm-4.5-air:free - GLM 4.5 Air (lightweight)

Important Notes about Free Models (Updated Oct 2025): - Daily limits reduced: 50 requests/day for free accounts (down from 200) - Accounts with $10+ credits: 1000 requests/day - Rate limit: 20 requests/minute for all accounts - Some models removed: NVIDIA Nemotron and others have exited the free tier - Availability may change: Verify current free models at https://openrouter.ai/models?q=free - For production use: Consider using paid models for better reliability

How can I improve the accuracy of annotations?

To get the most accurate annotations:

1. Use multiple high-quality models: Include diverse, high-performing models like Claude 3.7, GPT-4o, and Gemini 1.5

2. Provide good marker genes: Use robust differential expression analysis to identify strong marker genes

3. Specify the correct tissue: Always provide the correct tissue name to give models the proper context

4. Review uncertainty metrics: Pay attention to consensus proportion and Shannon entropy to identify clusters that may need manual review

5. Examine discussion logs: For controversial clusters, review the discussion logs to understand the reasoning

6. Iterate if needed: If results are unsatisfactory, try adjusting parameters or providing additional context

Why am I getting different results with the same input?

There are several possible reasons for getting different results with the same input:

1. Model updates: LLMs are regularly updated, which can change their outputs

2. Temperature/sampling: Some randomness is inherent in LLM outputs

3. Context window limitations: Different runs might include slightly different context

4. API changes: Providers may change how their APIs work

To ensure reproducibility: - Implement your own caching mechanism to reuse results - Specify model versions explicitly when available - Save and document your results - Consider saving the raw API responses for future reference

I’m getting an error about invalid cluster indices. What should I do?

If you see an error about invalid cluster indices, check that your cluster column contains valid values. mLLMCelltype accepts any cluster IDs (numeric or character) and preserves them as-is. Common issues:

1. Ensure the cluster column exists in your data frame
2. Check for NA values in the cluster column
3. If using non-standard column names, rename to cluster before calling the function

How do I handle “API key not found” errors?

If you get an error about missing API keys:

1. Check environment variables: Ensure your API keys are set correctly in your .env file or environment

2. Provide keys directly: Pass the API key directly to the function:

results <- annotate_cell_types(..., api_key = "your-api-key")

3. Check provider name: Make sure you’re using the correct provider name for your API key:

# Set API key for a specific provider
Sys.setenv(ANTHROPIC_API_KEY = "your-anthropic-key")

4. Verify key validity: Check if your API key is still valid by testing it directly with the provider’s API

Why are some cell types not being correctly identified?

If specific cell types are not being correctly identified:

1. Check marker genes: Ensure the marker genes for these cell types are strong and specific

2. Provide more context: Specify the tissue type accurately to give models the right context

3. Use more models: Different models have different strengths; using multiple models improves coverage

4. Increase marker count: Try increasing top_gene_count to include more marker genes

5. Review discussion logs: For controversial clusters, examine the discussion to understand the reasoning

6. Consider rare cell types: Some cell types may be poorly represented in the training data of LLMs

How does mLLMCelltype integrate with Seurat?

mLLMCelltype integrates with Seurat:

1. Input: You can directly use Seurat’s FindAllMarkers() output as input

2. Output: Annotation results can be easily added to your Seurat object:

seurat_obj$cell_type_consensus <- plyr::mapvalues(
  x = as.character(Idents(seurat_obj)),
  from = names(consensus_results$final_annotations),
  to = consensus_results$final_annotations
)

3. Visualization: Use Seurat’s visualization functions with the added annotations:

DimPlot(seurat_obj, group.by = "cell_type_consensus", label = TRUE)

Can I use mLLMCelltype with Scanpy/AnnData in R?

Yes, you can use mLLMCelltype with Scanpy/AnnData objects in R:

1. Extract marker genes: Export marker genes from your Scanpy analysis to a CSV file

2. Run mLLMCelltype: Use the CSV file as input to mLLMCelltype

3. Import results: Add the annotation results back to your AnnData object

Alternatively, you can use the Python version of mLLMCelltype for direct integration with Scanpy.

How can I combine mLLMCelltype with traditional annotation methods?

mLLMCelltype can be used alongside traditional annotation methods:

1. Complementary approach: Use both methods and compare results

2. Validation: Use mLLMCelltype to validate annotations from reference-based methods

3. Hybrid approach: Use reference-based methods for well-characterized cell types and mLLMCelltype for novel or rare cell types

4. Ensemble method: Create a consensus between mLLMCelltype and traditional methods

How can I customize the prompts used by mLLMCelltype?

While mLLMCelltype uses carefully designed prompts, advanced users can customize them:

# Create a custom annotation prompt
custom_prompt <- create_annotation_prompt(
  marker_data = your_markers,
  tissue_name = "your_tissue",
  top_gene_count = 10,
  custom_instructions = "Also consider developmental stage and activation state."
)

# Use the custom prompt directly
response <- get_model_response(
  prompt = custom_prompt,
  model = "claude-sonnet-4-5-20250929",
  api_key = your_api_key
)

Can I add my own custom LLM models?

Yes, you can register custom models and providers:

# Register a custom provider
register_custom_provider(
  provider_name = "my_provider",
  api_url = "https://api.my-provider.com/v1/chat/completions",
  api_key_env_var = "MY_PROVIDER_API_KEY",
  process_function = function(prompt, api_key) {
    # Custom implementation
  }
)

# Register a custom model
register_custom_model(
  model_name = "my-custom-model",
  provider = "my_provider"
)

# Use the custom model
results <- annotate_cell_types(
  input = your_markers,
  tissue_name = "your_tissue",
  model = "my-custom-model",
  api_key = your_api_key
)

How can I contribute to mLLMCelltype?

We welcome contributions! Here are some ways to contribute:

1. Report issues: Report bugs or suggest features on our GitHub repository

2. Improve documentation: Help us improve documentation and examples

3. Add new models: Implement support for new LLM models

4. Share benchmarks: Share your benchmarking results with different tissues and species

5. Develop new features: Contribute code for new features or improvements

See our Contributing Guide for more details.