hadoop源码编译 - 高董

gaodong 2021-08-07 原文


hadoop源码编译

hadoop 源码编译

以前写过一篇《yarn hadoop mapreduce 2.0 编译》,环境准备和那一篇一样。
本篇直接从svn下载最新源码,进行编译。

下载源码:
从subversion库check out:
[zhouhh@Hadoop48 hsrc]$ svn co http://svn.apache.org/repos/asf/hadoop/common/trunk

[zhouhh@Hadoop48 hsrc]$ cd trunk/
[zhouhh@Hadoop48 trunk]$ ls
BUILDING.txt hadoop-assemblies hadoop-common-project hadoop-hdfs-project hadoop-minicluster hadoop-project-dist hadoop-yarn-project
dev-support hadoop-client hadoop-dist hadoop-mapreduce-project hadoop-project hadoop-tools pom.xml

Maven 主要模块:

hadoop (Main Hadoop project)
– hadoop-project (Parent POM for all Hadoop Maven modules. )
(All plugins & dependencies versions are defined here.)
– hadoop-project-dist (Parent POM for modules that generate distributions.)
– hadoop-annotations (Generates the Hadoop doclet used to generated the Javadocs)
– hadoop-assemblies (Maven assemblies used by the different modules)
– hadoop-common-project (Hadoop Common)
– hadoop-hdfs-project (Hadoop HDFS)
– hadoop-mapreduce-project (Hadoop MapReduce)
– hadoop-tools (Hadoop tools like Streaming, Distcp, etc.)
– hadoop-dist (Hadoop distribution assembler)

编译:

[zhouhh@Hadoop48 trunk]$ mvn install -DskipTests -Pdist

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ………………………….. SUCCESS [0.605s]
[INFO] Apache Hadoop Project POM ……………………. SUCCESS [0.558s]
[INFO] Apache Hadoop Annotations ……………………. SUCCESS [0.288s]
[INFO] Apache Hadoop Project Dist POM ……………….. SUCCESS [0.094s]
[INFO] Apache Hadoop Assemblies …………………….. SUCCESS [0.088s]
[INFO] Apache Hadoop Auth ………………………….. SUCCESS [0.152s]
[INFO] Apache Hadoop Auth Examples ………………….. SUCCESS [0.093s]
[INFO] Apache Hadoop Common ………………………… SUCCESS [5.188s]
[INFO] Apache Hadoop Common Project …………………. SUCCESS [0.049s]
[INFO] Apache Hadoop HDFS ………………………….. SUCCESS [12.065s]
[INFO] Apache Hadoop HttpFS ………………………… SUCCESS [0.194s]
[INFO] Apache Hadoop HDFS BookKeeper Journal …………. SUCCESS [0.616s]
[INFO] Apache Hadoop HDFS Project …………………… SUCCESS [0.029s]
[INFO] hadoop-yarn ………………………………… SUCCESS [0.157s]
[INFO] hadoop-yarn-api …………………………….. SUCCESS [2.951s]
[INFO] hadoop-yarn-common ………………………….. SUCCESS [0.752s]
[INFO] hadoop-yarn-server ………………………….. SUCCESS [0.124s]
[INFO] hadoop-yarn-server-common ……………………. SUCCESS [0.736s]
[INFO] hadoop-yarn-server-nodemanager ……………….. SUCCESS [0.592s]
[INFO] hadoop-yarn-server-web-proxy …………………. SUCCESS [0.123s]
[INFO] hadoop-yarn-server-resourcemanager ……………. SUCCESS [0.200s]
[INFO] hadoop-yarn-server-tests …………………….. SUCCESS [0.149s]
[INFO] hadoop-yarn-client ………………………….. SUCCESS [0.119s]
[INFO] hadoop-yarn-applications …………………….. SUCCESS [0.090s]
[INFO] hadoop-yarn-applications-distributedshell ……… SUCCESS [0.167s]
[INFO] hadoop-mapreduce-client ……………………… SUCCESS [0.049s]
[INFO] hadoop-mapreduce-client-core …………………. SUCCESS [1.103s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher …. SUCCESS [0.142s]
[INFO] hadoop-yarn-site ……………………………. SUCCESS [0.082s]
[INFO] hadoop-yarn-project …………………………. SUCCESS [0.075s]
[INFO] hadoop-mapreduce-client-common ……………….. SUCCESS [1.202s]
[INFO] hadoop-mapreduce-client-shuffle ………………. SUCCESS [0.066s]
[INFO] hadoop-mapreduce-client-app ………………….. SUCCESS [0.109s]
[INFO] hadoop-mapreduce-client-hs …………………… SUCCESS [0.123s]
[INFO] hadoop-mapreduce-client-jobclient …………….. SUCCESS [0.114s]
[INFO] hadoop-mapreduce-client-hs-plugins ……………. SUCCESS [0.084s]
[INFO] Apache Hadoop MapReduce Examples ……………… SUCCESS [0.130s]
[INFO] hadoop-mapreduce ……………………………. SUCCESS [0.060s]
[INFO] Apache Hadoop MapReduce Streaming …………….. SUCCESS [0.071s]
[INFO] Apache Hadoop Distributed Copy ……………….. SUCCESS [0.069s]
[INFO] Apache Hadoop Archives ………………………. SUCCESS [0.061s]
[INFO] Apache Hadoop Rumen …………………………. SUCCESS [0.135s]
[INFO] Apache Hadoop Gridmix ……………………….. SUCCESS [0.082s]
[INFO] Apache Hadoop Data Join ……………………… SUCCESS [0.070s]
[INFO] Apache Hadoop Extras ………………………… SUCCESS [0.192s]
[INFO] Apache Hadoop Pipes …………………………. SUCCESS [0.019s]
[INFO] Apache Hadoop Tools Dist …………………….. SUCCESS [0.057s]
[INFO] Apache Hadoop Tools …………………………. SUCCESS [0.018s]
[INFO] Apache Hadoop Distribution …………………… SUCCESS [0.047s]
[INFO] Apache Hadoop Client ………………………… SUCCESS [0.047s]
[INFO] Apache Hadoop Mini-Cluster …………………… SUCCESS [0.053s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 32.093s
[INFO] Finished at: Wed Dec 26 11:00:10 CST 2012
[INFO] Final Memory: 60M/769M

打包命令:
创建二进制分发版,不带native code和文档:
[zhouhh@Hadoop48 trunk]$ mvn package -Pdist -DskipTests -Dtar
创建二进制分发版,带native code和文档:
[zhouhh@Hadoop48 trunk]$ mvn package -Pdist,native,docs -DskipTests -Dtar
创建源码分发版
[zhouhh@Hadoop48 trunk]$ mvn package -Psrc -DskipTests
创建二进制带源码分发版,带native code和文档:
[zhouhh@Hadoop48 trunk]$ mvn package -Pdist,native,docs,src -DskipTests -Dtar
创建本地版web页面,放在/tmp/hadoop-site
[zhouhh@Hadoop48 trunk]$ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site

参考

http://svn.apache.org/repos/asf/hadoop/common/trunk/BUILDING.txt

版权声明:本文为gaodong原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://www.cnblogs.com/gaodong/p/3333153.html

hadoop源码编译 - 高董的更多相关文章

  1. Mac系统常用快捷键及技巧 – 三戒1993

    Mac系统常用快捷键及技巧 2013-04-18 14:52  三戒1993  阅读(174)  评论(0)  […]...

  2. Execl如何在一个格子里打多行字,如何换行 – 自由一份子

    答:“ALT+回车”!...

  3. Android中AlarmManager使用示例(持续更新,已经更改) – 潘侯爷

    Android中AlarmManager使用示例(持续更新,已经更改) 现在普遍的手机都会有一个闹钟的功能,如 […]...

  4. CWMP开源代码研究1——开篇之作 – 京月飞鸿

    原创作品,转载请注明出处,严禁非法转载。如有错误,请留言! email:40879506@qq.com 淘宝店 […]...

  5. 剖析Unreal Engine超真实人类的渲染技术Part 3 – 毛发渲染及其它

    目录 四、毛发渲染 4.1 毛发的构造及渲染技术 4.1.1 毛发的构造 4.1.2 Marschner毛发渲 […]...

  6. web前端简单试题 – F-lower

    web前端简单试题 2013-10-02 09:58  F-lower  阅读(310)  评论(0)  编辑 […]...

  7. 什么是SDK什么是API? – Delphi学习之路

    什么是SDK什么是API?   SDK 就是 Software Development Kit 的缩写,就是“ […]...

  8. 实战记录轻松10分钟将Deepin深度系统安装到笔记本电脑中 – itbulu

    实战记录轻松10分钟将Deepin深度系统安装到笔记本电脑中 麦子在之前的文章中有记录到”5步制作deepin […]...

随机推荐

  1. 常用SAR软件资源清单

    InSAR处理软件 GMT5SAR   -基于GMT的 InSAR 处理软件. ISCE   – […]...

  2. HTML 编辑器大全

    HTMl编辑器大全:WebHtmlEditor 、SPAW Editor、YnetEditor、RTE、HHF […]...

  3. Git很麻烦?不存在的!掌握这几招就够了

    废话不多说,下面直接开始了!查看原文 确保代码库是最新的,先用这条命令把你的代码拉取到本地 git clone […]...

  4. 使用企业微信的API给指定用户发送消息

    上个月比较忙,等不忙了继续写点基础教程(五一还在高铁上写项目在)。因为公司的原因,其中有这样的一个要求,就是在 […]...

  5. 产品经理小技术:图片素材随手找,原型设计快又好

     画图——这项古老而精细的做法,是一代代产品狗们得以传承的立足之本。草图、线框图、思维导图、PPT插图、数据汇 […]...

  6. Django框架之session

    Session介绍: 在计算机中,尤其是在网络应用中,称为“会话控制”。Session 对象存储特定用户会话所 […]...

  7. RabbitMQ和Kafka到底怎么选?

    队列 前言 开源社区有好多优秀的队列中间件,比如RabbitMQ和Kafka,每个队列都貌似有其特性,在进行工 […]...

  8. 一个开源的仓储系统

    http://www.gitwms.com/Home/Welcome...

展开目录

目录导航