How to retrieve a value from struct in Spark Java?

Question

My Dataset ds has the following schema:

root
 |-- id: string (nullable = true)
 |-- type: string (nullable = true)
 |-- item: struct (nullable = true)
 |    |-- item: string (nullable = true)

Sample:

{"id":"1","type": "aaa", "item": {"item":"11"}}
{"id":"2","type": "bbb", "item" : {"item":"12"}}

How can I retrieve item from struct to get this result?

id   type    item
1    aaa     11
2    bbb     12

This is what I tried without success:

ds.select("id", "type", "item.0");

Please notice that I use Java. Do not post answers in Scala or Python unless they are identical for Java.

Answer 1

Suppose you have your sample file:

{"id":"1","type": "aaa", "item": {"item":"11"}}
{"id":"2","type": "bbb", "item" : {"item":"12"}}

You can test the following Java code:

public class SparkJavaTest {
    public static SparkSession spark = SparkSession
            .builder()
            .appName("JavaSparkTest")
            .master("local")
            .getOrCreate();


    public static void main(String[] args) {

    Dataset<Row> ds1 = spark.read().json("sample.json");

ds1.printSchema();

ds1.select("id", "type", "item.item").show(false);

The result would be:

root
 |-- id: string (nullable = true)
 |-- item: struct (nullable = true)
 |    |-- item: string (nullable = true)
 |-- type: string (nullable = true)


+---+----+----+
|id |type|item|
+---+----+----+
|1  |aaa |11  |
|2  |bbb |12  |
+---+----+----+