You can use pyspark.sql.functions.array_contains
method:
df.filter(array_contains(df['authors'], 'Some Author'))
from pyspark.sql.types import *
from pyspark.sql.functions import array_contains
lst = [(["author 1", "author 2"],), (["author 2"],) , (["author 1"],)]
schema = StructType([StructField("authors", ArrayType(StringType()), True)])
df = spark.createDataFrame(lst, schema)
df.show()
+--------------------+
| authors|
+--------------------+
|[author 1, author 2]|
| [author 2]|
| [author 1]|
+--------------------+
df.printSchema()
root
|-- authors: array (nullable = true)
| |-- element: string (containsNull = true)
df.filter(array_contains(df.authors, "author 1")).show()
+--------------------+
| authors|
+--------------------+
|[author 1, author 2]|
| [author 1]|
+--------------------+
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…