Usage

• Expected A Posterior estimates and 95% Credible Intervals for AUC and any other parameters.
• Comparison of AUC.
• Draw Curves, i.e., FROC and AFROC curves.

Basic words

We only focus TP and FP, thus we call it hit and false alarms, respectively. That is:

• Number of hits are denoted by h and number of false alarms are denoted by f in the R console, respectively.

• Number of hits are denoted by \(H\) in TeX and number of false alarms are denoted by \(F\) in TeX.

Notation and Symbols for FROC task.

Suppose that there are \(\color{red}{N_I}\) images (e.g., radiograph) in which there is \(\color{red}{N_L}\) lesions that should be detected by radiologists. Each image may contain no lesions. Radiologist identify suspicious locations of lesions for each image if he suspects that there are lesions with his confidence level that is number \(1,2, ..., c, ..., C\). So, radiologist can answer multiple locations for a single image, this multiples differ from ordinal ROC analysis which allows each reader only single dichotomous answer for each image. Summarizing his true positive \(H_c\) and false positive (false alarm) \(F_c\) for each confidence level generate a FROC dataset \((F_c,H_c)\). Now, we introduced the notations, \(N_L\), \(N_I\), \(H_c\), \(F_c\), \(C\). In the R console, these notations are represented by NL, NI, h, f, C.

If \(C=5\), then the dataset for FROC analysis is the follows;

Note that \(H_{c},F_c \in \mathbb{N} \cup\{0\}\) for \(c=1,2,...,5\).

The dictionary for data of single reader and single modality

Statistical model for FROC

\[\begin{eqnarray*} H_{c } & \sim &\text{Binomial} ( p_{c}, N_{L} ), \text{ for $c=1,2,...,C$.}\\ F_{c } & \sim &\text{Poisson}( (\lambda _{c} -\lambda _{c+1} )\times N_{I} ), \text{ for $c=1,2,...,C-1$.}\\ \lambda _{c}& =& - \log \Phi ( z_{c } ),\text{ for $c=1,2,...,C$.}\\ p_{c} &=&\Phi (\frac{z_{c +1}-\mu}{\sigma})-\Phi (\frac{z_{c}-\mu}{\sigma}), \text{ for $c=1,2,...,C-1$.}\\ p_C & =& 1-\Phi (\frac{z_{C}-\mu}{\sigma}),\\ F_{C} & \sim & \text{Poisson}( (\lambda _{C} - 0)N_I),\\ dz_c=z_{c+1}-z_{c} &\sim& \text{Uniform}(0,\infty), \text{ for $c=1,2,...,C-1$.}\\ \mu &\sim& \text{Uniform}(-\infty,\infty),\\ \sigma &\sim& \text{Uniform}(0,\infty),\\ \end{eqnarray*}\] Our model has parameters \(z_{1}, dz_1,dz_2,\cdots, dz_{C-1}\), \(\mu\), and \(\sigma\). Notation \(\text{Uniform}( -\infty,100000)\) means the improper uniform distribution of its support is the unbounded interval \(( -\infty,100000)\).

For the details, please see the authors paper. Note that this model is used if default value ModifiedPoisson = FALSE is retained .

The code for fitting

Example

This is a basic example which shows how to fit a model to data dataList of single reader and single modality.

#0) To avoid the following error in Readme file, I have to attach the Rcpp. I do not know why such error occur withou Rcpp. This error occurs only when I run the following R scripts from readme.

#Error
#in do.call(rbind,sampler_params) :second argument must be a list Calles:<Anonymous>...get_divergent_iterations ->sampler_param_vector =. do.call Execution halted




 library(Rcpp)  # This code can remove the above unknown error, if someone know why the error occur, please tell me.




#1) Build  data for singler reader and single modality  case.




  dataList <- list(c=c(3,2,1),     # c is ignored, can omit.
              h=c(97,32,31),
              f=c(1,14,74),
              NL=259,
              NI=57,
              C=3)





#  where,
#        c denotes confidence level, each components indicates that 
#                3 = Definitely lesion,
#                2 = subtle,  
#                1 = very subtle
#        h denotes number of hits 
#          (True Positives: TP) for each confidence level,
#        f denotes number of false alarms
#          (False Positives: FP) for each confidence level,
#        NL denotes number of lesions (signal),
#        NI denotes number of images,


#2) Fit the FROC model.


 
                  fit <- BayesianFROC::fit_Bayesian_FROC(
                                
                               
                    
                    dataList,
                                        
            #The number of MCMC chains                         
                    cha = 4
                                                         )




#  validation of fit via alculation of p -value of the chi square goodness of fit, which is 
#  calculated by integrating with  predictive posterior measure.
                  
                  
                    BayesianFROC::p_value_of_the_Bayesian_sense_for_chi_square_goodness_of_fit(fit, head.only = TRUE)
                   
                   

                                     
                  

Note that the above list object dataList representing the following FROC data;

#Interpretation of Outputs The results of BayesianFROC::fit_Bayesian_FROC(dat) are as follows:

The correspondence of notations between the R console and the author’s paper:

Work Flow 1

• Prepare data
  • Create by your hands: dataset_creator_new_version() or create_dataset()
  • Convert from Excel data of Jafroc or Rjafroc format: convertFromJafroc()
• Fitting: fit_Bayesian_FROC()
  • Estimates of your FROC model
  • Comparison of Modalities
• Draw Curves: DrawCurves()
  • FROC curve
  • AFROC curve
  • Cumulative False positive and cumulative true positives

Work Flow 2

Prepare data

#An example dataset for the case of Multiple readers and Multiple Modalities.
dat <- BayesianFROC::dataList.Chakra.Web

Fitting

# Fitting for your data with respect to the hierarchical Bayesian model.
fit<-BayesianFROC::fit_Bayesian_FROC(dat)

Draw Curves

 #Draw curves for the 1st modality and 2nd reader
 DrawCurves(

   #This is estimates 
        fit,
   
   # Modatity ID whose curves are drawn.
      modalityID =1,
   
   # Reader ID whose curves are drawn.
      readerID   =2)

Data for MRMC

BayesianFROC::dataList.Chakra.Web

BayesianFROC:::viewdata(BayesianFROC::dataList.Chakra.Web)

Fit

To fit the model, it is same as the case of single reader and single modality as follows:

# I do not why, but Machine cannot find some function in Rcpp. So I have to load the package Rcpp.
library(Rcpp)

# Prepare dataset
dat <- BayesianFROC::dataList.Chakra.Web

#Fitting
fit <-BayesianFROC::fit_Bayesian_FROC(dat)

It is different to the single reader and single modality case user should run the another function DrawCurves_MRMC_pairwise() to draw the curves such as FROC, AFROC, or cumulative hits and cumulative false alarms. Since drawing the curve is very heavy procedures, so the author separate this drawing process from fitting procedures.

Draw curves

library(Rcpp)



# Prepare a dataset
dat <- BayesianFROC::dataList.Chakra.Web


# Fitting
fit <- BayesianFROC::fit_Bayesian_FROC(dat)




#    Draw curves for the 1st modality and 2nd reader
 DrawCurves(
   
#    This is the above fitted model object
        fit,
   
#    Specify Modatity ID by vector whose curve are drawn.
      modalityID =1,
   
#    Specify readerID ID by vector whose curve are drawn.
      readerID   =2)

Note that you need to input your dataset dat and estimates in fit to draw the curves. Of course you should specify reader and modality whose curves you want to write.

To identify reader and modality ID you can use vector, for example, modalityID = c(1,3) for the first modality and the second modalities. readerID = c(1,2) for the first and the second readers.

Questions and Supports

If user has any questions, send me a mail.

tsunoda.issei1111 at gmail.com

Dictionary 1