r - How to get test data ROC plot from MLeval

Question

I am trying to return the ROC curves for a test dataset using the MLevals package.

  # Load data 
  train <- readRDS(paste0("Data/train.rds"))
  test <- readRDS(paste0("Data/test.rds"))
  
  # Create factor class
  train$class <- ifelse(train$class == 1, 'yes', 'no')
 

  # Set up control function for training
  ctrl <- trainControl(method = "cv",
                       number = 5, 
                       returnResamp = 'none',
                       summaryFunction = twoClassSummary(),
                       classProbs = T,
                       savePredictions = T,
                       verboseIter = F)
  
  gbmGrid <-  expand.grid(interaction.depth = 10,
                          n.trees = 18000,                                          
                          shrinkage = 0.01,                                         
                          n.minobsinnode = 4) 
  
  # Build using a gradient boosted machine
  set.seed(5627)
  gbm <- train(class ~ .,
               data = train,
               method = "gbm",
               metric = "ROC",
               tuneGrid = gbmGrid,
               verbose = FALSE,
               trControl = ctrl) 

# Predict results - 
  pred <- predict(gbm, newdata = test, type = "prob")[,"yes"]
  roc <- evalm(data.frame(pred, test$class))

I have used the following post, ROC curve for the testing set using Caret package, to try and plot the ROC from test data using MLeval and yet I get the following error message:

MLeval: Machine Learning Model Evaluation Input: data frame of probabilities of observed labels Error in names(x) <- value : 'names' attribute [3] must be the same length as the vector [2]

Can anyone please help? Thanks.

question from:https://stackoverflow.com/questions/65829802/how-to-get-test-data-roc-plot-from-mleval

Answer 1

Please provide a reproducible example with sample data so we can replicate the error and test for solutions (i.e., we cannot access train.rds or test.rds).

Nevertheless, the below may fix your issue.

pred <- predict(gbm, newdata = test, type = "prob")
roc <- evalm(data.frame(pred, test$class))