实用 _ 如何安装Cloudera Enterprise 5.pdf该脑图是介绍如何安装Cloude

文件名称: 实用 _ 如何安装Cloudera Enterprise 5.pdf

所属分类: Hadoop

开发工具:

文件大小: 2mb

下载次数: 0

上传时间: 2019-09-01

提供者: qq_38******

下载 (2mb)

不能下载？报告错误

详细说明：该脑图是介绍如何安装Cloudera Enterprise，请贡献给大家下载！RHEL/CentOS #在 etc/rc loca中增加一行: S>echo never >/sys/kemel/mm/redhat transparent hugepage/defrag SUSE 11 SP2. #在/ etc/init. d/after loca中增加一行: if test-f /sys/kemel/mm/transparent hugepageenabled: then echo never >/sys/kernel/mm/transparent hugepage/enabled 10.禁止交换(可选),内存页面交换在某些情况下会导致CDH性能下降,建议在 kerne在2.6.32-303之前的版本中将其关闭 S>vim/etc/sysctl. conf/ 增加一行: vm swampiness=.0 s>sudosysctlvmswampiness=0. 11.(可选)修改/etc/ security/limits.conf者在/etc/ security/limits.d下增加相应的配置文件,可以设置一些硬限制和软限制; Cloudera Manager节点会为所有节点自动做这些修改。通过 > cat /proc /limits#确认一些参数限制 12.在需要作为Repo库的节点上安装必要的软件,包含HTP服务和Rep创建工具 Centos, sp>yum install createrepo :S>yuminstallhttpd S>servicehttpdstarte :s>chkconfighttpdon SUSE 11 SP2. t please find the yum-metadata-parser and createrepo package in os repo directory, usually you can find the them in x86 64 or noarch folder. s>rpm-ivh yum-metadata-parser-xxx. rpm. s>rpmivhcreaterepo-xxx rpm s>pm-ivhyast2-http-server-2.17.15-0.5.2.noarch.rpm. 磁盘划分会影响磁盘存储的主要包含以下一些服务角色节点 RAD推荐。磁盘要求所有节点 RAIDI war/og空间不小于506B 所有节点。 JBD/RAD1。/ opt/cloudera空间不小于20GB mysql RAID1. 数据目录不小于50GB Host Monitor (A standalone RAIDI /var/ib/ feloudera-host- monitor不小于15GB service for Cloudera Manager )a (目录可调整,大小不可调整);可以单独部署 Service Monitor (A standalone RAIDIT /ar/ib/ cloudera- service-monitor不小于15GB service for Cloudera Manager 目录可调整,大小不可调整):可以单独部署。 Event server (A standalone RAIDI /ar/ib/ cloudera-scm-eventserver,可以通过调 service for Cloudera manager 整存储的最大事件数来控制存储空间占用 Reports Manager (A standalone RAIDlJ /var/lib/ cloudera-scm- headlamp,目录空间不确 service for Cloudera Manager )3 定,取决于 fsimage的大小和HDFS中文件或者目录的路径深度。 Navigator Audit Server (A RAID1, /var/b不小于10GB,可以通过调整保存的审 standalone service for Cloudera 计日志天数来调整存储 Manager )p Navigator Metadata Server (A RAIDI /ar/b不小于10GB standalone service for Cloudera Manager)a Name Node(Active/Standby )- J80D例如:/ /hadoop/dfs/nn不小于128GB JournalNode 」BoD 例如:/ hadoop/fs/jn不小于64GB Zookeeper. 8D/RAD1例如:/ab/e小于10B Datanodee jboD 增加 noatime的挂载选项角色划分(样例) 节点接入层角色。 Node 1e 接入层1 CM (and related), QIM Node 2e DN, NM, RS, HS2, HMS, MySQL Node 33 DN. NM RS HM. ZK. Node 4-10 DN, NM, RS Node 11. NN, RM, QJM Node 12 DN NM RS, HS2 HMS Node 13. DN,NM,RS,HM,zR。 Node 14-40 DN, NM, RS, Node41.接入层2.NN,RMQM Node 42. DN, NM, RS, HS2, HMS, MysQ Node 43. DN NM RS, HM. ZR Node 44-60. DN NM RS Node6le DN RS, ZR HMe Node 622 DN, RS, HM Node 63-70 DN RS Node 71e DN. RS ZK HMe Node 72-80. DN, RS 核心层核心层 1Gb 接入层接入层接入层接入层外部软件安装 1.使用已有的NTP服务或者配置新的NTP服务 2.外部数据库安装, Cloudera5推荐安装 MySQL5.5的版本,安裝前必须清除旧版本。 a)清除旧版本 rpm--e--nodepsmysql-libs. b)选择一个节点安装 MySQL数据库 MySQL Server Bundle; http://cdn.mysqlcom/archives/mysql-5.5/mysql-5.5.41-1.el6.x86_64.rpm-bundle.tar MysQL Connector Java: http://cdn.mysqlcom/downloads/connector-j/mysql-connector-jaVa-3.1.14.tar.gz S>rpm -ivh mysql-server-5541-1.sles11x86 64. rpm mysql-client-5541-1.sles11 xB6- 64. rpm. S>chkconfigmysqlon, S>servicemysg starta c运行 mysql_secure_installaticηn,配置root用户访问数据库的密码,禁止远程机器以root用户登录,禁止匿名登录创建root访问数据库的密码 #禁止远程机器以root用户登录 #禁止以匿名方式登录数据库 /usr/bin/mysql secure installation S sudo /usr/bin/mysql secure installation [ Enter current password for root (enter for none):* OK, successfully used password, moving Set root password? TY/] yu New password: 123456. Re-enter new password Remove anonymous users? YinYa Disallow root login remotely? Y/n]N, Remove test database and access to it Y/]Y, Reload privilege tables now? YinIY, Al‖ donel d)创建 metastore数据库 mysql-u root -password= - e create database metastore default character set utf8; e)创建 Cloudera Enterprise Hive访问 MySQLmetastore库的用户名和密码 mysql -u root-password= -e" CREATE USER hive %' IDENTIFIED BY 123456 GRANT ALL PRIVILEGES ON metastore, TO hive %, FLUSH PRIVILEGES,? f)类似地,创建其它组件用的数据库,包括: Cloudera Manager server, Activity Monitor, Reports Manager, Sentry Server, Cloudera Navigator Audit Server, Cloudera Navigator Metadata Server #若是安装了yam就不需要安全 activity monitor组件了 mysql-u root -password=123456'-e"create user 'aman'% identified by 123456 mysql-u root -password=123456'-e'create database amon default character set utf8 mysql-u root -password=123456'-e"grant all privileges on amon. to amon %" mysql-u root-password='123456'-e"create user'rman' %'iden tified by 123456 mysql-u root -password=123456'-e'create database rman default character set utf8 mysql-u root -password=123456'-e"grant all privileges on rman, to rman'%, mysql-u root -password=123456'-e"create user'sentry' %' identified by 123456", mysql-u root password=123456'-e"grant all privileges on sentry. to sentry'%r mysql -u root -password=123456'-e'create database sentry default character set utf8 mysql-u root -password=123456'-e"create user nav%identified by 123456w mysql-u root -password=123456'-e create database nav default character set utf8M mysql-u root -password=123456'-e"grant all privileges on nav. 'to 'nav%", lysql-u root -password=123456'-e"create user'navms'"%' identified by 123456 mysql-u root -password=123456'-e'create database navms default character set utf8, mysql-u root -password=123456'-e"grant all privileges on navms. to% mysql-u root -password=123456'-e"create user 'cm% identified by 123456, mysql-u root -password=123456'-e'create database cm default character set utf8. mysql-u root -password=123456-e grant all privileges on cm. to 'cm%"+ CDH软件下载与配置( Cloudera管理器节点) 1.下载 Cloudera管理器需要的rpm包 RHEL/CentoS wget-c -r-nd -np水 k-L-Arpn http://archive-primary.cloudera.com/cm5/redhat/6/x8664/cm/5/rpms/x8664/e SUsE 115P2. += wget -nd -np rpmhttp://archive- 4 primary cloudera. com/cm5/sles/11/x86 64/cm/5/RPMS/x86 64/2-.----- 2.下载 Parcel包(包含了CDH中的 Hadoop组件从以下地址选择合适版本的 parcel包: http:/archive-primary-cloudera.com/cdh5/parcels/latest+ 下载 manifest json文件: httplarchive-primary-cloudera.com/cdh5/parcels/latest/manifestison- 3.下载后将下载的软件放置为如下结构(该步骤不是必须的,只是为了后续说明的方便) rootdip-172-31-46-113cDH]#1s CDH-5.2.0-1 cdh5.2.0.p0. 36-e16 parcel cm manifest json rootip-172-31-46-113cDH]#1scm cloudera-manager-agent-5.2.0-1. cm520 p0 60 e16x86_64. rpm cloudera-manager-daemons-5.2.0-1. cm520 p060e16x86 64. rpm cloude ra-manager-server-5,2.0-1, cm520p0, 60, e16, x86_64, rpm cloudera-manager-server-db-2-5.2.0-1. cm520 p0 60e16 x86-64 rpm enterprise-debuginfo-5.2.0-1. cm520, p0.60.e16 x86_64.rpm jdk-6u31-Linux-amd64 rpm oracle-j2sdk1. 7-1. 7. 0+update67-1 x86-64,rpm 4.创建repo文件以支持本地yum的操作 S> cd cm. s>createrepo, w Spawning worker O with 7 pkgs Workers Finished. Gathering worker results. Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Saite DBs complete. S>Is, cloudera-manager-agent-520-1. cm520. p0, 60 el6x86 64. rpme cloudera-manager-daemons-520-1. cm520 p0 60 el6x86 64.rpm. cloudera-manager-server-520-1. cm520-p0 60 el6x86 64 rpma cloudera-manager-server-db-2-5,2.0-1. cm520 p0 60,el6 x86 64 rpme enterprise-debuginfo-5.2.0-1.cm520 p0 60el6 x86_64. pmw idk-6u31-linux-amd64pme oracle-j2sdk1,7-1.7.0+update 67-1x86_64.rpm- rodata. 执行完后,在cm目录下生成目录 repodata 5.将文件移动到特定的目录,确保可以通过HTTP协议进行访问 5>|s CDH-5.2.0-1cdh5.2.0.p0. 36-el6parcel cmmanifest sone s>mkdir -p /ar/www/html/cdh5/parcels/5.2.0/. S>mv CDH-520-1cdh5.2.0. P0. 36-el6. parcel /var/www/html/cdh5/ parcels/5.2.0/ S>mvmanifest json /var/www/html/cdh5/parcels/5, 2.0/, S>my cm /var/www/html/ $>chmod -R ugo+rX /ar/www/html PPaP pwaPweHm" Pp 现在,你应该可以使用浏览器对相关目录进行访问 4> C A D172.31.46.113/cdh5/parcels/5.2.0/ Index of /cdh5/parcels/5.2.0 Same Last mo dified size Description 4 Parent Director 2SDH-520-Lcdh5,2.0.20.316-e16parcel 12-0ct-2014 18:40 1.4G 的 manifestison 120t201418:4242K Apeche/2.2-15(CentoS) Server ar $4 69-224-63 Pord 80 中cn1723146113cm/ Index of /cm a Lau madified ze e 的bmnm52n2的题使mm1302041573sM 的mem2Lm题130x20141s374M 可dkmn症M16如6的m1302014153776 m cloadnumamager? 5 2 1cm 320 0 c xa6_65 rpm 130cM 2014 15:17.6K cntcrpcisc-asbw winfo-520-1-ae酸威 13201415:37672 的如3面 13201415376sM odkuyik1-TaLYHundatc7-1x5A-mm1 1c如15135M F:podaa 15D2140952 ipsha22.3(Cemrd,) servert 346922465 on sth 6.新建文件/ etc/yum. repos. d/myrepo repo [myrepoJ i hame -++ baseurl=htp:/172.31.46.113/cm enabledetrue. gpgcheck=false. 安装 Clouderaf管理器 1.安装JDK RHEL/Centos yum install oracle-j2sdk1.7+ SUSE 11 SP2, zypper install oracle-j2sdk17e 2.安装 Cloudera管理器服务器 RHEL/CentoS yum install cloudera-manager-daemons cloudera-manager-servera SUSE 11SP2 zypper install cloudera-manager-daemons cloudera-manager-servera 3在需要访问 MySQL的节点上安装mysq- connector-java cp mysql-connector-java-5.1.34. jar/usr/share/java/w In-s mysql-connector-java-5.1.34. jar mysql-connector-java jare 4.为 Cloudera管理器配置外部数据库 fusr/share/cmfschema/scm prepare database. sh SDB YPE>SPASSWORD>4 h< MYSOL HOST>可以不指定,默认是 localhost DB TYPE可以是mysq, oracle DATABASE即为之前为 Cloudera Manager配置的数据库 USERNAMEPASSWORD即为可以访问这个数据库的用户 5.启动 Cloudera管理器服务器 servicecloudera-scm-serverstarto 启动后就可以访问 Cloudera管理器页面了 Cloudera管理器的监听端口为7180 (17314Pm )合自吾李-如·日·≡ 回·回·回独回回贴回中 cloudera manager 杜受即最安装cDH集群 1.输入账户密码 admin/ admin,点击“登录” 此计积上偏片温题 2.选择要安装的集群版本(在这里我们选择试用版),点击“继续”

(系统自动生成,下载前可以参看下载内容)