The purpose of the pseudo header fields was to unify the way the request/response information was carried in SPDY and later in HTTP/2 (which is based on SPDY).
When SPDY was designed (but also HTTP/2) there was a need to transport request or response information that is formatted in different ways.
The HTTP headers are (key, value) pairs, that's easy.
However, there is the concept of the HTTP method. That happens to be the first token of a request line, so it's not a tuple; its key is defined by its position (the first token) and its value is the actual characters present on the request line that form the first token.
Same goes with the request target, and HTTP version: they are the second and the third token of the request line.
So conceptually, a HTTP request can be represented with pairs in this way, for example:
(method, GET)
(target, /)
(version, HTTP/1.1)
(Connection, close)
(Accept, *)
However, "method", "target" and "version" could not be used as plain HTTP headers, because they were never reserved as standard HTTP header names by the HTTP specification, and people could have used them as custom HTTP header names (imagine a REST API using the "version" header).
HTTP/2 needed a way to carry those pairs in a homogeneous way, as pairs, because that would have simplified (a lot) the protocol.
Hence, the introduction of special names for the extra information carried by the request and response lines. That extra information is positional in HTTP/1.1, but a normal pair in HTTP/2, making HTTP/2 more homogeneous in this respect: it just carries pairs.
So much so, that the HTTP/2 frame that carries request and response information is the same and it's just called HEADERS.
The pseudo header names where chosen to begin with a colon because that would be an illegal character for a header name in HTTP/1.1.
HTTP/1.1 does not use pseudo header names.
