This is trivial to do with Canvas. It is also possible to do with SVG, but it's convoluted. The following method works with conventional fully opaque images. First, you convert every non-matching color value in each channel to zero and every matching color value to 1 using a long ComponentTransfer (the index of the sole "1" in your 256 member tableValues arrays should match your r, g, and b replace value). Then you zero out the alpha of everything except the resulting white pixels using a color matrix. You use the result as a mask with an feFlood of your target color and composite the result on top of your original graphic. For example, the following code replaces a specific color - rgb(87, 78, 29) - with blue.
<svg width="600px" height="600px" viewBox="0 0 600 600">
<defs>
<filter id="color-replace" color-interpolation-filters="sRGB">
<!-- Replace rgb(87,78,29) with blue. -->
<feComponentTransfer >
<feFuncR type="discrete" tableValues="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"/>
<feFuncG type="discrete" tableValues="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"/>
<feFuncB type="discrete" tableValues="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"/>
</feComponentTransfer>
<feColorMatrix type="matrix" values="1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
1 1 1 1 -3" result="selectedColor"/>
<feFlood flood-color="blue"/>
<feComposite operator="in" in2="selectedColor"/>
<feComposite operator="over" in2="SourceGraphic"/>
</filter>
</defs>
<g filter="url(#color-replace)">
<rect x="50" y="50" height="100" width="100" fill="rgb(86,77,28)"/>
<rect x="250" y="50" height="100" width="100" fill="rgb(86,77,29)"/>
<rect x="450" y="50" height="100" width="100" fill="rgb(86,78,29)"/>
<rect x="50" y="250" height="100" width="100" fill="rgb(87,77,29)"/>
<rect x="250" y="250" height="100" width="100" fill="rgb(87,78,29)"/>
<rect x="450" y="250" height="100" width="100" fill="rgb(87,78,30)"/>
<rect x="50" y="450" height="100" width="100" fill="rgb(88,78,30)"/>
<rect x="250" y="450" height="100" width="100" fill="rgb(88,79,29)"/>
<rect x="450" y="450" height="100" width="100" fill="rgb(88,79,30)"/>
</g>
</svg>
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…