r - Shiny app does not reflect changes in update RData file

Question

I update my RData file on daily basis through a cron job for my shiny apps. However, shiny apps does not pick updates most of the time and keep showing the old data from old RData file.

Here is the minimum reproducible example. It works fine when data_processing.R is executed from my desktop. However, when it is done on a Rshiny server, shiny app does not read updated date and time stamp.

data_processing.R

rm(list=ls())
df <- iris
data_update_date_time <- Sys.time()
save.image("working_dataset.RData", compress = TRUE)

server.R

load("working_dataset.RData")

function(input, output, session) {

  # Combine the selected variables into a new data frame
  selectedData <- reactive({
    df[, c(input$xcol, input$ycol)]
  })

  clusters <- reactive({
    kmeans(selectedData(), input$clusters)
  })

  output$plot1 <- renderPlot({
    palette(c("#E41A1C", "#377EB8", "#4DAF4A", "#984EA3",
              "#FF7F00", "#FFFF33", "#A65628", "#F781BF", "#999999"))

    par(mar = c(5.1, 4.1, 0, 1))
    plot(selectedData(),
         col = clusters()$cluster,
         pch = 20, cex = 3)
    points(clusters()$centers, pch = 4, cex = 4, lwd = 4)
  })

  ## Data update date and time stamp
  output$update_date_time <- renderPrint(data_update_date_time)

}

ui.R

pageWithSidebar(
  headerPanel('Iris k-means clustering'),
  sidebarPanel(
    selectInput('xcol', 'X Variable', names(iris)),
    selectInput('ycol', 'Y Variable', names(iris),
                selected=names(iris)[[2]]),
    numericInput('clusters', 'Cluster count', 3,
                 min = 1, max = 9),
    br(),
    h4("Date update date time"),
    textOutput("update_date_time")
  ),
  mainPanel(
    plotOutput('plot1')
  )
)

Thanks for taking your time.

Answer 1

Edit

There is actually a function called reactiveFileReader in the shiny package that does exactly what you are looking for: Periodically checking if the files "last modified" time or size changed and rereading accordingly. However, this function can only be used in the server context, so the file will be read at least once for each user that connects to your app. Options 3 and 4 in my Answer do not have these inefficiencies.

Original Answer from here on

First and foremost, shiny does not have a way to keep track of filechanges AFAIK. Your implementation reloads the .RData file whenever

1. shiny-server gets restarted via bash or
2. the global variables get reloaded because the app became idle at some point.

There is no way of telling, when the second condition is met. Therefore, I would advocate using one of the following four options. Sorted from easy to you better know your shiny!.

Option 1: Put the load statement in server

Here, the image is reloaded whenever a new user connects with the app. However, this might slow down your app if your .RData file is huge. If speed is not an issue, I would pick this solution since it is easy and clean.

# server.R
function(input, output, session) {
  load("working_dataset.RData")
  ...
}

The data will also be reread whenever a user refreshes the page (F5)

Option 2: Restart shiny-server whenever you want to re-import your data

(Also see @shosacos answer). This forces the .Rdata file to be reloaded.

$ sudo systemctl restart shiny-server

Again, this might slow-down your production process depending on the complecity of your app. One advantage of this approach is that you can also use the imported data to build the ui if you load the data in global.R. (I assume you don't given the code you gave).

Option 3: Import according to "last modified"

The idea here is to check whether the .RData has changed whenever a user connects to the app. To do this, you will have to use a "global" variable that contains a timestamp of the last imported version. The following code is untested, but should give you an idea on how to implement this feature.

# server.R
last_importet_timestamp <- reactiveVal("")

function(input,output,session){
  current_timestamp <- file.info(rdata_path)$mtime 

  if(last_importet_timestamp() != current_timestamp){
    # use parent.frame(2) to make data available in other sessions
    load(rdata_path, envir = parent.fame(2))
    # update last_importet_timestamp
    last_importet_timestamp(current_timestamp) 
  }

  ...
}

Speed-wise, this should be more efficient than the first two versions. The data is never imported more than once per timestamp (unless shiny server gets restarted or becomes idle).

Option 4: Import "reactvely"

Basically, the same as option 3 but the file will be checked for changes every 50ms. Here is a full working example of this approach. Note that the data is not loaded unless a change in "last modified" is detected, so the resulting overhead is not too bad.

library(shiny)

globalVars <- reactiveValues()

rdata_path = "working_dataset.RData"

server <- function(input, output, session){
  observe({
    text = input$text_in
    save(text = text, file = rdata_path, compress = TRUE)
  })
  observe({
    invalidateLater(50, session)
    req(file.exists(rdata_path))
    modified <- file.info(rdata_path)$mtime
    imported <- isolate(globalVars$last_imported)
    if(!identical(imported, modified)){
      tmpenv <- new.env()
      load(rdata_path, envir = tmpenv)
      globalVars$workspace <- tmpenv
      globalVars$last_imported <- modified
    }
  })
  output$text_out <- renderText({
    globalVars$workspace$text
  })
}

ui <- fluidPage(
  textInput("text_in", "enter some text to save in Rdata", "default text"),
  textOutput("text_out")
)

shinyApp(ui, server)

If you find it inconvenient to use globalVars$workspace$text, you can use with to access the contents of globalVars$workspace directly.

  output$text_out <- renderText({
    with(globalVars$workspace, {
      paste(text, "suffix")
    })
  })