You dont need maxBufferSize or maxBufferPoolSize set so high these are possibly causing your out of memory exception. The defaults should be fine.
Check out Large Data and Streaming on MSDN, specifically the section Special Security Considerations for Large Data this piece of the text is important
The MaxBufferSize property is required to constrain the memory that
WCF buffers. It is important to set this to a safe value (or keep it
at the default value) when streaming. For example, suppose your
service must receive files up to 4 GB in size and store them on the
local disk. Suppose also that your memory is constrained in such a way
that you can only buffer 64 KB of data at a time. Then you would set
the MaxReceivedMessageSize to 4 GB and MaxBufferSize to 64 KB. Also,
in your service implementation, you must ensure that you read only
from the incoming stream in 64-KB chunks and do not read the next
chunk before the previous one has been written to disk and discarded
from memory.
I put together a very simple example of streaming data from a self hosted service to a console client. To keep the post short I only added the server code and part of the client.
The service contract
using System.IO;
using System.ServiceModel;
namespace Service
{
[ServiceContract]
public interface IStream
{
[OperationContract]
Stream GetLargeObject();
}
}
The service implementation
using System;
using System.IO;
using System.ServiceModel;
namespace Service
{
[ServiceBehavior]
public class StreamService : IStream
{
public Stream GetLargeObject()
{
// Add path to a big file, this one is 2.5 gb
string filePath = Path.Combine(Environment.CurrentDirectory, "C:\Temp\BFBC2_PC_Client_R11_795745_Patch.exe");
try
{
FileStream imageFile = File.OpenRead(filePath);
return imageFile;
}
catch (IOException ex)
{
Console.WriteLine(String.Format("An exception was thrown while trying to open file {0}", filePath));
Console.WriteLine("Exception is: ");
Console.WriteLine(ex.ToString());
throw;
}
}
}
}
The service main
using System;
using System.ServiceModel;
namespace Service
{
class Program
{
static void Main(string[] args)
{
try
{
using (var serviceHost = new ServiceHost(typeof(StreamService)))
{
serviceHost.Open();
Console.WriteLine("Press Any Key to end");
Console.ReadKey();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
}
The service app.config
<?xml version="1.0"?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="StreamServiceBehavior">
<serviceMetadata httpGetEnabled="True" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netTcpBinding>
<binding name="NewBinding0" transferMode="Streamed"/>
</netTcpBinding>
</bindings>
<services>
<service behaviorConfiguration="StreamServiceBehavior" name="Service.StreamService">
<endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
bindingConfiguration="NewBinding0" bindingName="" contract="Service.IStream" />
<endpoint address="mex" binding="mexHttpBinding"
contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="http://localhost:8080/StreamService" />
</baseAddresses>
</host>
</service>
</services>
</system.serviceModel>
</configuration>
Launch the service, may need to run under admin account to open the socket. Create a client console application and add a service reference using the url http:// localhost:8080 / StreamService, using Service as the namespace for the generated client.
The client main
using System;
using System.IO;
using Client.Service;
namespace Client
{
class Program
{
static void Main(string[] args)
{
try
{
using (StreamClient streamClient = new StreamClient())
{
streamClient.Open();
using (FileStream fileStream = new FileStream("c:\temp\bigfile.exe",FileMode.Create))
{
streamClient.GetLargeObject().CopyTo(fileStream);
}
}
Console.WriteLine("Press any key to end");
Console.ReadKey();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
}
The generated client config file will need to be modified slightly, increase receiveTimeout and set maxReceivedMessageSize="4294967295"
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="NetTcpBinding_IStream" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:30:00" sendTimeout="00:01:00"
transactionFlow="false" transferMode="Streamed" transactionProtocol="OleTransactions"
hostNameComparisonMode="StrongWildcard" listenBacklog="10"
maxBufferPoolSize="524288" maxBufferSize="65536" maxConnections="10"
maxReceivedMessageSize="4294967295">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="Transport">
<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
<message clientCredentialType="Windows" />
</security>
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
bindingConfiguration="NetTcpBinding_IStream" contract="Service.IStream"
name="NetTcpBinding_IStream">
</endpoint>
</client>
</system.serviceModel>
Launch the service then the client. It will stream a large file without issue.
