I have a dataframe with a few columns. Now I want to derive a new column from 2 other columns:
from pyspark.sql import functions as F
new_df = df.withColumn("new_col", F.when(df["col-1"] > 0.0 & df["col-2"] > 0.0, 1).otherwise(0))
With this I only get an exception:
py4j.Py4JException: Method and([class java.lang.Double]) does not exist
It works with just one condition like this:
new_df = df.withColumn("new_col", F.when(df["col-1"] > 0.0, 1).otherwise(0))
Does anyone know to use multiple conditions?
I'm using Spark 1.4.
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…