Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
312 views
in Technique[技术] by (71.8m points)

python 3.x - How to read and insert bytea columns using psycopg2?

I am working on a Python script to replicate some Postgresql tables from one environment to another (which does a little more than pg_dump). It works except when I am copying a table that has bytea data type.

I read the source table data in memory, then I dump the memory in the target database with concatenated inserts.

Here is my method that produces an insert statement:

def generateInsert(self, argCachedRow):

    colOrd = 0;

    valClauseList = []

    hasBinary = False

    for colData in argCachedRow:
        colOrd += 1
        colName = self.colOrdLookup.get(colOrd)
        col = self.colLookup.get(colName)
        dataType = col.dataType

        insVal = None

        if colData is not None:

            strVal = str(colData)
            if dataType.useQuote:

                if "'" in strVal:
                    strVal = strVal.replace("'", "''")
                insVal = "'%s'" % strVal
            else:
                if dataType.binary:
                    hasBinary = True
                    #insVal = psycopg2.Binary(colData)
                #else:

                insVal = strVal
        else:
            insVal = "NULL"

        valClauseList.append(insVal)

    valClause = ", ".join(valClauseList)

    if hasBinary:
        valClause = psycopg2.Binary(valClause)

    result = "INSERT INTO %s VALUES (%s)" % (self.name, valClause)

    return result

which works with every table that doesn't have binary data.

I also tried (intuitively) to wrap just the binary column data in psycopg2.Binary, which is the commented out line and then not do it to the whole row value list but that didn't work either.

Here is my simple DataType wrapper, which is loaded by reading Postgres' information_schema tables:

class DataType(object):
    def __init__(self, argDispName, argSqlName, argUseQuote, argBin):
        self.dispName = argDispName
        self.sqlName = argSqlName
        self.useQuote = argUseQuote
        self.binary = argBin

How do I read and insert bytea columns using psycopg2?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

If you have this database structure:

CREATE TABLE test (a bytea,
                   b int,
                   c text)

then inserting binary data into the request can be done like so, without any wrappers:

bin_data = b'bytes object'
db = psycopg2.connect(*args)  # DB-API 2.0
c = db.cursor()
c.execute('''INSERT INTO test VALUES (%s, %s, %s)''', (bin_data, 1337, 'foo'))
c.execute('''UPDATE test SET a = %s''', (bin_data + b'1',))

Then, when you query it:

c.execute('''SELECT a FROM test''')

You'll receive a memoryview, which is easily converted back to bytes:

mview = c.fetchone()
new_bin_data = bytes(mview)
print(new_bin_data)

Output: b'bytes object1'

Also, I'd suggest you not to assemble queries by string formatting. psycopg2's built-in parameter substitution is much more convenient and you don't have to worry about validating data to protect from SQL injections.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

1.4m articles

1.4m replys

5 comments

57.0k users

...