Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
703 views
in Technique[技术] by (71.8m points)

c# - Writing to file using StreamWriter much slower than file copy over slow network

I have a program that attempts to write quite a large amount of text to a file on a remote server overseas, which has a slow network connection.

Using the following code, where outputFileContent is a StringBuilder:

using (var outfile = new StreamWriter(myRemoteFilePath))
{
    outfile.Write(outputFileContent.ToString());
}

it is taking a seriously long time to run (several minutes), whereas if I first write to a local file and then copy it across to the remote location, it is much quicker (20-30 secs):

string tempFilePath = Path.GetTempFileName();
using (var outfile = new StreamWriter(tempFilePath))
{
    outfile.Write(outputFileContent.ToString());
}

System.IO.File.Copy(tempFilePath, myRemoteFilePath, true)

Any idea why this is happening? My only guess is that it is something to do with buffering across the network, or perhaps because the stream writer doesn't know how big it needs to be ahead of time.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

If you create your StreamWriter using default buffer sizes then the underlying SMB protocol will be issuing write requests in chunks no larger than 4096 bytes, which means a large number of round-trips over the network. You could increase your StreamWriter's buffer size up to a maximum of 64k in order to reduce the number of round trips:

using (var outfile = new StreamWriter(myRemoteFilePath, false, Encoding.ASCII, 0x10000))

Increasing the buffer size beyond 64k will not help in any circumstance, as the underlying SMB protocol does not support buffer lengths beyond 64k. Note that a direct file copy still uses the SMB protocol, so from a network traffic perspective there's little difference between the operations except for the buffer sizes.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...