第44课：Spark 2.0编程实战之DataSet案例开发实战

 第44课：Spark 2.0编程实战之DataSet案例开发实战

people.json

{"name":"Michael"}
{"name":"Andy", "age":30}
{"name":"Justin", "age":19}

peopleScores.json

{"n":"Michael", "score":88}

{"n":"Andy", "score":100}

{"n":"Justin", "score":89}

 

运行结果

 

16/09/16 17:48:51 INFO CodeGenerator: Code generated in 13.484901 ms
+---+----+----+-----+
|age|name|   n|score|
+---+----+----+-----+
| 30|Andy|Andy|  100|
+---+----+----+-----

 

 

== Physical Plan ==
*HashAggregate(keys=[name#1], functions=[avg(score#25L), avg(age#0L)])
+- Exchange hashpartitioning(name#1, 200)
   +- *HashAggregate(keys=[name#1], functi