json - Swift JSONEncoder number rounding

Question

As with all IEEE 7540 systems, a number in Swift like 4.7 is treated as a value like 4.7000000000000002. So it isn't surprising that:

% swift
Welcome to Apple Swift version 5.2.4 (swiftlang-1103.0.32.9 clang-1103.0.32.53).
Type :help for assistance.
  1> 4.7
$R0: Double = 4.7000000000000002
  2> 4.7 == 4.7000000000000002
$R1: Bool = true

This is a well-understood reality of the world, and so does not need to be addressed with comments containing links to background articles on floating-point precision loss.

When encoding this number using the built-in JSONEncoder, we see:

  4> String(data: JSONEncoder().encode([4.7]), encoding: .utf8) 
$R2: String? = "[4.7000000000000002]"

This is not incorrect, as Wikipedia says this about JSON & floating point numbers:

The JSON standard makes no requirements regarding implementation details such as overflow, underflow, loss of precision, rounding, or signed zeros, but it does recommend to expect no more than IEEE 754 binary64 precision for "good interoperability". There is no inherent precision loss in serializing a machine-level binary representation of a floating-point number (like binary64) into a human-readable decimal representation (like numbers in JSON), and back, since there exist published algorithms to do this exactly and optimally.

However, other JavaScript environments tend to round these numbers. E.g. with JavaScriptCore:

% /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Helpers/jsc

>>> 4.7 == 4.7000000000000002
true
>>> JSON.stringify([4.7000000000000002])
[4.7]

And with node:

% node
Welcome to Node.js v13.13.0.
Type ".help" for more information.
> 4.7 == 4.7000000000000002
true
> JSON.stringify([4.7000000000000002])
'[4.7]'

The problem for me is that I have large collections of Swift doubles that, when serialized to JSON for storage and/or transmission, contain a lot of unnecessary chaff ("4.7000000000000002" has 6x more characters than "4.7"), thereby inflating the size of the serialized data considerably.

Can anyone think of a nice way to override Swift's numeric encoding to serialize doubles as their rounded equivalent, short of giving up on auto-synthesis of encodability and re-implementing the encoding of the entire type graph manually?

Answer 1

You can extend KeyedEncodingContainer and KeyedDecodingContainer and implement a custom encoding and decoding methods to send Decimal as plain data. You would just need to set the encoder/decoder dataEncodingStrategy to deferredToData. Another possibility is to encode and decode its base64Data or encode/decode it as plain string.

extension Numeric {
    var data: Data {
        var bytes = self
        return .init(bytes: &bytes, count: MemoryLayout<Self>.size)
    }
}

---

extension DataProtocol {
    func decode<T: Numeric>(_ codingPath: [CodingKey], key: CodingKey) throws -> T {
        var value: T = .zero
        guard withUnsafeMutableBytes(of: &value, copyBytes) == MemoryLayout.size(ofValue: value) else {
            throw DecodingError.dataCorrupted(.init(codingPath: codingPath, debugDescription: "The key (key) could not be converted to a numeric value: (Array(self))"))
        }
        return value
    }
}

---

extension KeyedEncodingContainer {
    mutating func encode(_ value: Decimal, forKey key: K) throws {
        try encode(value.data, forKey: key)
    }
    mutating func encodeIfPresent(_ value: Decimal?, forKey key: K) throws {
        guard let value = value else { return }
        try encode(value, forKey: key)
    }
}

---

extension KeyedDecodingContainer {
    func decode(_ type: Decimal.Type, forKey key: K) throws -> Decimal {
        try decode(Data.self, forKey: key).decode(codingPath, key: key)
    }
    func decodeIfPresent(_ type: Decimal.Type, forKey key: K) throws -> Decimal? {
        try decodeIfPresent(Data.self, forKey: key)?.decode(codingPath, key: key)
    }
}

---

Playground testing:

struct Root: Codable {
    let decimal: Decimal
}

// using the string initializer for decimal is required to maintain precision
let root = Root(decimal: Decimal(string: "0.007")!)

do {
    let encoder = JSONEncoder()
    encoder.dataEncodingStrategy = .deferredToData
    let rootData = try encoder.encode(root)
    let decoder = JSONDecoder()
    decoder.dataDecodingStrategy = .deferredToData
    let root = try decoder.decode(Root.self, from: rootData)
    print(root.decimal) // prints "0.007
" instead of "0.007000000000000001024
" without the custom encoding and decoding methods
} catch {
    print(error)
}

---

To keep the data size as low as possible You can encode and decode Decimal as string:

extension String {
    func decimal(_ codingPath: [CodingKey], key: CodingKey) throws -> Decimal {
        guard let decimal = Decimal(string: self) else {
            throw DecodingError.dataCorrupted(.init(codingPath: codingPath, debugDescription: "The key (key) could not be converted to decimal: (self)"))
        }
        return decimal
    }

}

---

extension KeyedEncodingContainer {
    mutating func encode(_ value: Decimal, forKey key: K) throws {
        try encode(String(describing: value), forKey: key)
    }
    mutating func encodeIfPresent(_ value: Decimal?, forKey key: K) throws {
        guard let value = value else { return }
        try encode(value, forKey: key)
    }
}

---

extension KeyedDecodingContainer {
    func decode(_ type: Decimal.Type, forKey key: K) throws -> Decimal {
        try decode(String.self, forKey: key).decimal(codingPath, key: key)
    }
    func decodeIfPresent(_ type: Decimal.Type, forKey key: K) throws -> Decimal? {
        try decodeIfPresent(String.self, forKey: key)?.decimal(codingPath, key: key)
    }
}

---

Playground testing:

struct StringDecimal: Codable {
    let decimal: Decimal
}

let root = StringDecimal(decimal: Decimal(string: "0.007")!)
do {
    let stringDecimalData = try JSONEncoder().encode(root)
    print(String(data: stringDecimalData, encoding: .utf8)!)
    let stringDecimal = try JSONDecoder().decode(StringDecimal.self, from: stringDecimalData)
    print(stringDecimal.decimal) // "0.007
"
} catch {
    print(error)
}

---

This will print

{"decimal":"0.007"}
0.007