powershell - Alternative to Get-Content

Question

I currently have the following line of code.

(Get-Content 'file.txt') |
  ForEach-Object {$_ -replace '"', ''} |
  Set-Content 'file.txt'

This worked when testing, but now I am trying to use it on real data files (13 GB) and this process of using Get-Content is causing Powershell to consume a large amount of RAM and ultimately all of the available RAM on the machine.

Is there a better way that I can achieve the same result without the same amount of overhead?

Seems I am doing the opposite of best practice but not sure what else would be cleaner/ less RAM intensive than the above.

Answer 1

Use a stream to read the file, then it won't put it all into memory, you can also use a stream to write the output. This should perform pretty well, and keep memory usage down:

$file = New-Object System.IO.StreamReader -Arg "c:estfile.txt"
$outstream = [System.IO.StreamWriter] "c:estout.txt"

while ($line = $file.ReadLine()) {
  $s = $line -replace '"', ''
  $outstream.WriteLine($s)
}
$file.close()
$outstream.close()