powershell - How to Use Multithreading script

Question

I found this Multithreading script at http://www.get-blog.com/?p=189

Param($Command = $(Read-Host "Enter the script file"), 
    [Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]$ObjectList,
    $InputParam = $Null,
    $MaxThreads = 20,
    $SleepTimer = 200,
    $MaxResultTime = 120,
    [HashTable]$AddParam = @{},
    [Array]$AddSwitch = @()
)

Begin{
    $ISS = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
    $RunspacePool = [runspacefactory]::CreateRunspacePool(1, $MaxThreads, $ISS, $Host)
    $RunspacePool.Open()

    If ($(Get-Command | Select-Object Name) -match $Command){
        $Code = $Null
    }Else{
        $OFS = "`r`n"
        $Code = [ScriptBlock]::Create($(Get-Content $Command))
        Remove-Variable OFS
    }
    $Jobs = @()
}

Process{
    Write-Progress -Activity "Preloading threads" -Status "Starting Job $($jobs.count)"
    ForEach ($Object in $ObjectList){
        If ($Code -eq $Null){
            $PowershellThread = [powershell]::Create().AddCommand($Command)
        }Else{
            $PowershellThread = [powershell]::Create().AddScript($Code)
        }
        If ($InputParam -ne $Null){
            $PowershellThread.AddParameter($InputParam, $Object.ToString()) | out-null
        }Else{
            $PowershellThread.AddArgument($Object.ToString()) | out-null
        }
        ForEach($Key in $AddParam.Keys){
            $PowershellThread.AddParameter($Key, $AddParam.$key) | out-null
        }
        ForEach($Switch in $AddSwitch){
            $Switch
            $PowershellThread.AddParameter($Switch) | out-null
        }
        $PowershellThread.RunspacePool = $RunspacePool
        $Handle = $PowershellThread.BeginInvoke()
        $Job = "" | Select-Object Handle, Thread, object
        $Job.Handle = $Handle
        $Job.Thread = $PowershellThread
        $Job.Object = $Object.ToString()
        $Jobs += $Job
    }

}

End{
    $ResultTimer = Get-Date
    While (@($Jobs | Where-Object {$_.Handle -ne $Null}).count -gt 0)  {

        $Remaining = "$($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).object)"
        If ($Remaining.Length -gt 60){
            $Remaining = $Remaining.Substring(0,60) + "..."
        }
        Write-Progress `
            -Activity "Waiting for Jobs - $($MaxThreads - $($RunspacePool.GetAvailableRunspaces())) of $MaxThreads threads running" `
            -PercentComplete (($Jobs.count - $($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).count)) / $Jobs.Count * 100) `
            -Status "$(@($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False})).count) remaining - $remaining" 

        ForEach ($Job in $($Jobs | Where-Object {$_.Handle.IsCompleted -eq $True})){
            $Job.Thread.EndInvoke($Job.Handle)
            $Job.Thread.Dispose()
            $Job.Thread = $Null
            $Job.Handle = $Null
            $ResultTimer = Get-Date
        }
        If (($(Get-Date) - $ResultTimer).totalseconds -gt $MaxResultTime){
            Write-Error "Child script appears to be frozen, try increasing MaxResultTime"
            Exit
        }
        Start-Sleep -Milliseconds $SleepTimer

    } 
    $RunspacePool.Close() | Out-Null
    $RunspacePool.Dispose() | Out-Null
}

I'm confused as how to use it though. In particular how to use the $ObjectList variable. I would like to use this ping script that I wrote to ping a list of 100,000 machines... But in it's current form it pings about 100 machines a minute. So it would take over 16 hours to complete.

$Computers = Get-Content -Path C:TempComps.txt

foreach ($Computer in $Computers) {

    if (test-Connection -ComputerName $Computer -Count 1) {   

        "$Computer is Pinging"

        }


    Else {

        "$Computer is not Pinging"

        }

    }

Any pointers as to how to integrate the ping script with the multithreading script to yield faster results?

I'm assuming that I would remove the $Computers variable from the ping script, and instead use the $ObjectList variable in the multithreading script. But all of my attempts to do so fail.

Answer 1

You can ping hosts asynchronously with [System.Net.NetworkInformation.Ping], no need to go crazy with runspaces. Moreover it will ping the host once, so it will be way much faster than test-connection. Below is a sample for a batch of hosts (about 90). I guess it's not a good idea to load 100K at once, probably split it in smaller batches and do one by one.

$hosts = "www.facebook.com,www.twitter.com,www.youtu.be,www.google.com,www.youtube.com,www.instagram.com,www.linkedin.com,www.pinterest.com,www.wordpress.com,www.blogspot.com,www.apple.com,www.adobe.com,www.tumblr.com,www.amazon.com,www.vimeo.com,www.flickr.com,www.microsoft.com,www.yahoo.com,www.godaddy.com,www.qq.com,www.vk.com,www.reddit.com,www.baidu.com,www.nytimes.com,www.buydomains.com,www.wp.com,www.statcounter.com,www.jimdo.com,www.blogger.com,www.github.com,www.weebly.com,www.soundcloud.com,www.myspace.com,www.addthis.com,www.theguardian.com,www.cnn.com,www.stumbleupon.com,www.gravatar.com,www.digg.com,www.addtoany.com,www.creativecommons.org,www.paypal.com,www.yelp.com,www.imdb.com,www.huffingtonpost.com,www.feedburner.com,www.issuu.com,www.wixsite.com,www.wix.com,www.dropbox.com,www.forbes.com,www.amazonaws.com,www.washingtonpost.com,www.bluehost.com,www.etsy.com,www.go.com,www.msn.com,www.wsj.com,www.weibo.com,www.fc2.com,www.eventbrite.com,www.parallels.com,www.ebay.com,www.livejournal.com,www.reuters.com,www.taobao.com,www.typepad.com,www.bloomberg.com,www.elegantthemes.com,www.eepurl.com,www.usatoday.com,www.about.com,www.medium.com,www.macromedia.com,www.xing.com,www.bing.com,www.time.com,www.tripadvisor.com,www.aol.com,www.constantcontact.com,www.latimes.com,www.list-manage.com,www.webs.com,www.opera.com,www.live.com,www.bandcamp.com,www.bbc.com,www.businessinsider.com,www.dailymotion.com,www.cpanel.com,www.disqus.com,www.sina.com.cn,www.spotify.com,www.wired.com,www.googleusercontent.com"
$hosts = $hosts -split ","

$tasks = @{}
foreach ($h in $hosts) { $tasks[$h] = [System.Net.NetworkInformation.Ping]::new().SendPingAsync($h)}

Write-Host "Waiting for batch is completed" -NoNewline
while($false -in $tasks.Values.IsCompleted) {sleep -Milliseconds 300; Write-Host "." -NoNewline}

$result = foreach($h in $hosts) {
$r = $tasks[$h].Result
[PSCustomObject]@{
 host = $h
 address = $r.Address.IPAddressToString
 status = if($r.Address.IPAddressToString){$r.Status}else{"Failed"}
 time = $r.RoundtripTime
 bytes = $r.Buffer.Count
 ttl = $r.Options.Ttl
  }
}


$result | Format-Table -AutoSize