Hadoop 二次排序 八股文的样本例子
1、数据文本
[root@master IMFdatatest]#cat SecondarySort.txt
12 8
32 21
54 32
65 21
501 12
81 2
81 6
81 9
81 7
81 1
100 100
2、上传hdfs
[root@master IMFdatatest]#hadoop dfs -put SecondarySort.txt /libaray
3、编码
自定义IntPair ，放入两列值 重写比较
八股文定义好 SecondarySortGroupComparator MyPartitioner
4、hadoop 框架 自动排序key value
5、输出值，按字符串格式
[root@master IMFdatatest]#hadoop dfs -cat /library/outputSecondarySort/part-r-00000
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
16/02/19 18:16:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
100 100
12 8
32 21
501 12
