logstash - PowerShell Script As Service

Question

I've created a script that I want to run as a service. It uses Invoke-RestMethod to monitor an application on the server and, when it detects an unwanted condition, it restarts the application. When I run it with my user account, it works great, but when I run it as a service (As SYSTEM) nothing happens. I'm outputting most console output to a variable and then at the end of the loop, writing the contents of the variable to a file as a makeshift log. This file never gets written.

I used this guide to create the service and it seems to startup fine and run...it's just nothing happens.

Below is the script I am using that the service should execute.

#Configuration Options
$ESHost = ""
$ESPort = 9200
$ESPro = ""
$ESAPI = ""
$ESAPI.Cred = ""
$LogRoot = ""
$LSHost = ""
$LSPort = 9600
$LSPro = ""
$SMTP = @{
  server = ""
  to = ""
  from = ""
  port = 25
}

#Look for Logstash service
While (Get-Service Logstash) {
  $Date = Get-Date -Format "MMddyyyy_HHmmss"
  #Verify service is running
  if ((Get-Service Logstash).Status -eq "Running") {
    $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  *****Start Health Check*****`n"
    #Collect list of pipelines
    $Pipelines = ((Invoke-RestMethod -Uri "$($LSPro)://$($LSHost):$($LSPort)/_node/stats/pipelines").pipelines.PSObject.Properties | Where-Object {$_.MemberType -eq "NoteProperty"}).Name
    $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  Running pipelines: $($pipelines.count)`n"
    foreach ($pipeline in $pipelines) {
      $log += "  $($pipeline)`n"
    }
    $down = @()
    #Check health of each pipeline
    foreach ($pipeline in $pipelines) {
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  Starting health check on $($pipeline) pipeline`n"
      $pipelineinfo = Invoke-RestMethod -Uri "$($LSPro)://$($LSHost):$($LSPort)/_node/stats/pipelines/$($pipeline)"
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($pipeline) ephemeral id $($pipelineinfo.ephemeral_id)`n"
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($pipeline) queue size (bytes) $([string]::Format('{0:N0}',($pipelineinfo.pipelines.$pipeline.queue.queue_size_in_bytes)))`n"
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($pipeline) queue capacity (bytes) $([string]::Format('{0:N0}',($pipelineinfo.pipelines.$pipeline.queue.max_queue_size_in_bytes)))`n"
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($pipeline) queued events $($pipelineinfo.pipelines.$pipeline.queue.events)`n"
      if ($pipelineinfo.pipelines.$pipeline.hash -eq $null) {
        $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($pipeline) down`n"
        $down += $pipeline
      }
      if ($pipelineinfo.pipelines.$pipeline.queue.queue_size_in_bytes -eq $pipelineinfo.pipelines.$pipeline.queue.max_queue_size_in_bytes) {
        $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($pipeline) queue at max capacity"
        Try {
          Send-MailMessage -SmtpServer $smtp.server -To $smtp.to -From $smtp.from -Subject "Pipeline Queue Full" -Port $smtp.port -Body "Pipeline $($pipeline)'s queue appears to be at full capacity"
        }
        Catch {
           log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($error[0].CategoryInfo.Activity)"+": "+"$($error[0].Exception.Message)`n"
        }
      }
      if (($pipelineinfo.pipelines.$pipeline.queue.queue_size_in_bytes -ne $pipelineinfo.pipelines.$pipeline.queue.max_queue_size_in_bytes) -and ($pipelineinfo.pipelines.$pipeline.hash -ne $null)) {
        $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($pipeline) up`n"
      }
    }
    #Actions on unhealthy pipelines
    if ($down.count -gt 0) {
      foreach ($pipe in $down) {
        Write-Host "$(Get-Date)  Pipeline $($pipe) appears to be down. Collecting logs..."
        #Compress logs for each failed pipeline
        $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  Collecting $($pipe) logs...`n"
        Try {
          Compress-Archive -LiteralPath "$($LogRoot)pipeline_$($pipe).log" -CompressionLevel Fastest -DestinationPath "$($LogRoot)pipeline_$($pipe)_$($Date).zip"
        }
        Catch {
           log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($error[0].CategoryInfo.Activity)"+": "+"$($error[0].Exception.Message)`n"
        }
      }
      #Compress logstash log
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  Collecting logstash-plain.log...`n"
      Try {
        Compress-Archive -LiteralPath "$($LogRoot)logstash-plain.log" -CompressionLevel Fastest -DestinationPath "$($LogRoot)logstash-plain_$($Date).zip"
      }
      Catch {
        log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($error[0].CategoryInfo.Activity)"+": "+"$($error[0].Exception.Message)`n"
      }
      #Send compressed logs in email notification
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  Sending mail notification`n"
      Try {
        Send-MailMessage -SmtpServer $smtp.server -To $smtp.to -From $smtp.from -Subject "Pipeline Down Detected" -Port $smtp.port -Attachments (Get-ChildItem "$LogRoot*.zip")
      }
      Catch {
        log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  $($error[0].CategoryInfo.Activity)"+": "+"$($error[0].Exception.Message)`n"
      }
      #Remove compressed logs
      Remove-Item (Get-ChildItem "$LogRoot*.zip") -Force -Confirm:$false
      Restart-Service Logstash
      Remove-Variable down
      Do {
        (Get-Service Logstash).Status
        Start-Sleep -Seconds 5
      }
      Until (((Get-Service Logstash).Status) -eq "Running")
      Send-MailMessage -SmtpServer $smtp.server -To $smtp.to -From $smtp.from -Subject "Logstash Successfully Restarted" -Port $smtp.port
    #Actions on no unhealthy pipelines
    } else {
      $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  All pipelines running`n"
      Write-Host "$(Get-Date)  All pipelines up"
    }
  }
  $log += "$(Get-Date -Format 'MM-dd-yyyy HH:mm:ss')  *****End Health Check*****`n"
  $log | Out-File "$($LogRoot)MonitorService.log" -Append
  Remove-Variable log
  Start-Sleep -Seconds 300
}

Answer 1

Identified some issues that I overlooked.

1. Running the script with an interactive PowerShell session under user SYSTEM, I was able to identify a couple problems with file access rights. These were resolved by copying the file to a temp dir and then performing actions on them.

2. Most importantly, the guide I used did not work right. After running 'NSSM edit ', it showed that the service was only executing PowerShell with no arguments defining the script. So the service was just executing an empty powershell session *facepalm.