The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

using_time_models

Victor Navarro

Time models in calmr

Version 0.5 of calmr introduced its first time-based model, ANCCR (Jeong et al., 2022), and with it, I wrote several additional tools for future time-based models.

Changes to trial-based models

The biggest change in calmr version 0.5 is the use of the “>” character and its effect on trial-based models. With the advent of time-based models, some generalizations had to be made to enable those models to update across adjacent trial periods. You can learn more about this in the directional_models vignette.

Specifying a design for time-based models

The designs for time-based models are nearly identical to those for trial-based models. However, clever use of the “>” character will enrich them. Let’s specify a serial feature discrimination experiment:

library(calmr)
fpfn <- data.frame(
  group = c("FP", "FN"),
  phase1 = c("!100F>T>(US)/100T", "!100F>T/100T>(US)")
)
parse_design(fpfn)
#> CalmrDesign built from data.frame:
#>   group            phase1
#> 1    FP !100F>T>(US)/100T
#> 2    FN !100F>T/100T>(US)
#> ----------------
#> Trials detected:
#>   group  phase trial_names trial_repeats is_test stimuli
#> 1    FP phase1    F>T>(US)           100   FALSE  F;T;US
#> 2    FP phase1           T           100   FALSE       T
#> 3    FN phase1         F>T           100   FALSE     F;T
#> 4    FN phase1      T>(US)           100   FALSE    T;US

We can manually specify the timing for the above experiment by calling the get_timings() function. Manipulating the list returned by that function will result in a manipulation of the timing between the experimental events.

ts <- get_timings(fpfn, model = "ANCCR")
ts
#> $use_exponential
#> [1] TRUE
#> 
#> $sample_timings
#> [1] TRUE
#> 
#> $trial_ts
#>      trial post_trial_delay mean_ITI max_ITI
#> 1 F>T>(US)                1       30      90
#> 2        T                1       30      90
#> 3      F>T                1       30      90
#> 4   T>(US)                1       30      90
#> 
#> $transition_ts
#>      trial transition transition_delay
#> 1 F>T>(US)        F>T                1
#> 2 F>T>(US)     T>(US)                1
#> 3      F>T        F>T                1
#> 4   T>(US)     T>(US)                1

And now let’s get the parameters for the ANCCR model.

pars <- get_parameters(fpfn, model = "ANCCR")
# increase learning rates
pars$alpha_reward <- 0.8
pars$alpha <- 0.08
# increase sampling interval to speed up the model
pars$sampling_interval <- 5
pars
#> $reward_magnitude
#>  F  T US 
#>  1  1  1 
#> 
#> $betas
#>  F  T US 
#>  1  1  1 
#> 
#> $cost
#> [1] 0
#> 
#> $temperature
#> [1] 1
#> 
#> $threshold
#> [1] 0.6
#> 
#> $k
#> [1] 1
#> 
#> $w
#> [1] 0.5
#> 
#> $minimum_rate
#> [1] 0.001
#> 
#> $sampling_interval
#> [1] 5
#> 
#> $use_exact_mean
#> [1] 0
#> 
#> $t_ratio
#> [1] 1.2
#> 
#> $t_constant
#> [1] NA
#> 
#> $alpha
#> [1] 0.08
#> 
#> $alpha_reward
#> [1] 0.8
#> 
#> $use_timed_alpha
#> [1] 0
#> 
#> $alpha_exponent
#> [1] 1
#> 
#> $alpha_init
#> [1] 1
#> 
#> $alpha_min
#> [1] 0
#> 
#> $add_beta
#> [1] 0
#> 
#> $jitter
#> [1] 1

Let’s make the model’s experience and look at the first 20 entries.

experiment <- make_experiment(fpfn,
  parameters = pars,
  timings = ts,
  model = "ANCCR"
)
head(experiences(experiment)[[1]], 20)
#>    model group  phase tp       tn is_test block_size trial stimulus      time reward_mag
#> 1  ANCCR    FP phase1  2        T   FALSE          2     1        T  28.49805          1
#> 2  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     2        F  55.50233          1
#> 3  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     2        T  56.50233          1
#> 4  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     2       US  57.50233          1
#> 5  ANCCR    FP phase1  2        T   FALSE          2     3        T  70.39851          1
#> 6  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     4        F  77.58430          1
#> 7  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     4        T  78.58430          1
#> 8  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     4       US  79.58430          1
#> 9  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     5        F  85.42575          1
#> 10 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     5        T  86.42575          1
#> 11 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     5       US  87.42575          1
#> 12 ANCCR    FP phase1  2        T   FALSE          2     6        T 104.34924          1
#> 13 ANCCR    FP phase1  2        T   FALSE          2     7        T 161.84664          1
#> 14 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     8        F 164.21165          1
#> 15 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     8        T 165.21165          1
#> 16 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     8       US 166.21165          1
#> 17 ANCCR    FP phase1  2        T   FALSE          2     9        T 186.19612          1
#> 18 ANCCR    FP phase1  1 F>T>(US)   FALSE          2    10        F 249.42371          1
#> 19 ANCCR    FP phase1  1 F>T>(US)   FALSE          2    10        T 250.42371          1
#> 20 ANCCR    FP phase1  1 F>T>(US)   FALSE          2    10       US 251.42371          1

As you can see above, there are several rows per trial, each specifying a different stimulus. Time-based models like ANCCR run over a time log because they make ample use of the temporal difference between events.

Let’s run the model and see some plots.

experiment <- run_experiment(experiment)
# Action values
patch_plots(plot(experiment, type = "action_values"))

# ANCCR
patch_plots(plot(experiment, type = "anccrs"))

# Dopamine transients
patch_plots(plot(experiment, type = "dopamines"))

And that’s it! Easy, right?

References

Jeong, H., Taylor, A., Floeder, J. R., Lohmann, M., Mihalas, S., Wu, B., Zhou, M., Burke, D. A., & Namboodiri, V. M. K. (2022). Mesolimbic dopamine release conveys causal associations. Science (New York, N.Y.), 378, eabq6740. https://doi.org/10.1126/science.abq6740

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.