Hadoop 二次排序 八股文的样本例子
1、数据文本
[root@master IMFdatatest]#cat SecondarySort.txt 12 8 32 21 54 32 65 21 501 12 81 2 81 6 81 9 81 7 81 1 100 100
2、上传hdfs
[root@master IMFdatatest]#hadoop dfs -put SecondarySort.txt /libaray
3、编码
自定义IntPair ，放入两列值 重写比较
八股文定义好 SecondarySortGroupComparator MyPartitioner
4、hadoop 框架 自动排序key value
5、输出值，按字符串格式
[root@master IMFdatatest]#hadoop dfs -cat /library/outputSecondarySort/part-r-00000 DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it.
16/02/19 18:16:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 100 100 12 8 32 21 501 12
