How to get the result based on a condition in spark scala

Question

I have the below dataframe with me

+---+-----+---+
| ID|Month|val|
+---+-----+---+
| M1|  Jan| 10|
| M1|  Feb| 20|
| M1|  Mar| 30|
| M2|  Jan| 22|
| M2|  Feb| 33|
| M3|  Jan| 30|
| M3|  Feb| 30|
| M4|  Jan| 40|
| M4|  Feb| 39|
+---+-----+---+

created by -

val df1=Seq(
("M1","Jan","10"),
("M1","Feb","20"),
("M1","Mar","30"),
("M2","Jan","22"),
("M2","Feb","33"),
("M3","Jan","30"),
("M3","Feb","30"),
("M4","Jan","40"),
("M4","Feb","39")
).toDF("ID","Month","val")

I have to find out those row whose val are lesser than or equal to the previous months.

Expected Result -

+---+-----+---+-------+
| ID|Month|val|    Res|
+---+-----+---+-------+
| M1|  Jan| 10| true  |
| M1|  Feb| 20| true  |
| M1|  Mar| 30| true  |
| M2|  Jan| 22| true  |
| M2|  Feb| 33| true  |
| M3|  Jan| 30| true  |
| M3|  Feb| 30| false |
| M4|  Jan| 40| true  |
| M4|  Feb| 39| false |
+---+-----+---+-------+

Answer 1

You can use lag over a window ordered by the month converted to a date:

import org.apache.spark.sql.expressions.Window

val df2 = df1.withColumn(
    "Res",
    coalesce(
        $"val" > lag($"val", 1).over(Window.partitionBy("ID").orderBy(to_date($"Month", "MMM"))),
        lit(true)
    )
).orderBy($"ID", to_date($"Month", "MMM"))

df2.show
+---+-----+---+-----+
| ID|Month|val|  Res|
+---+-----+---+-----+
| M1|  Jan| 10| true|
| M1|  Feb| 20| true|
| M1|  Mar| 30| true|
| M2|  Jan| 22| true|
| M2|  Feb| 33| true|
| M3|  Jan| 30| true|
| M3|  Feb| 30|false|
| M4|  Jan| 40| true|
| M4|  Feb| 39|false|
+---+-----+---+-----+