Python/pyspark data frame rearrange columns

Question

I have a data frame in python/pyspark with columns id time city zip and so on......

Now I added a new column name to this data frame.

Now I have to arrange the columns in such a way that the name column comes after id

I have done like below

change_cols = ['id', 'name']

cols = ([col for col in change_cols if col in df] 
        + [col for col in df if col not in change_cols])

df = df[cols]

I am getting this error

pyspark.sql.utils.AnalysisException: u"Reference 'id' is ambiguous, could be: id#609, id#1224.;"

Why is this error occuring. How can I rectify this.

question from:https://stackoverflow.com/questions/42912156/python-pyspark-data-frame-rearrange-columns

Answer 1

You can use select to change the order of the columns:

df.select("id","name","time","city")