hadoop-1.1.2 在Windows环境下的部署

1:先安装Cygwin

  参考http://blog.csdn.net/wind520/article/details/9223003

2:下载

3:解压在C:cygwinhadoop1

4:配置

hadoop1/conf下需要配置的文件:

1).hadoop-env.sh

2).core-site.xml

3).hdfs-site.xml

4).mapred-site.xml

如何修改请参考 http://blog.csdn.net/wind520/article/details/9225429

5:启动Hadoop

第一步,在hadoop1目录下创建目录logs,用于保存日志
第二步,Format a new distributed-filesystem: 格式化namenode,创建HDFS
执行命令: bin/hadoop namenode -format

  开始没有格式化,导致打不开

  • NameNode - http://localhost:50070/
    lenovo@lenovo-PC ~
    $ cd ..
    
    lenovo@lenovo-PC /home
    $ cd hadoop1
    -bash: cd: hadoop1: No such file or directory
    
    lenovo@lenovo-PC /home
    $ cd ../hadoop1
    
    lenovo@lenovo-PC /hadoop1
    $ bin/hadoop namenode -format
    [INFO ][mgmnt  ] Local JMX connector started
    13/07/03 15:44:30 INFO namenode.NameNode: STARTUP_MSG:
    /************************************************************
    STARTUP_MSG: Starting NameNode
    STARTUP_MSG:   host = lenovo-PC/10.6.3.180
    STARTUP_MSG:   args = [-format]
    STARTUP_MSG:   version = 1.1.2
    STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/b ranch-1.1 -r 1440782; compiled by 'hortonfo' on Thu Jan 31 02:03:24 UTC 2013
    ************************************************************/
    Re-format filesystem in 	mphadoop-lenovodfs
    ame ? (Y or N) Y
    13/07/03 15:44:35 INFO util.GSet: VM type       = 32-bit
    13/07/03 15:44:35 INFO util.GSet: 2% max memory = 20.0 MB
    13/07/03 15:44:35 INFO util.GSet: capacity      = 2^22 = 4194304 entries
    13/07/03 15:44:35 INFO util.GSet: recommended=4194304, actual=4194304
    13/07/03 15:44:35 INFO namenode.FSNamesystem: fsOwner=lenovo
    13/07/03 15:44:35 INFO namenode.FSNamesystem: supergroup=supergroup
    13/07/03 15:44:35 INFO namenode.FSNamesystem: isPermissionEnabled=true
    13/07/03 15:44:35 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
    13/07/03 15:44:35 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessK eyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
    13/07/03 15:44:35 INFO namenode.NameNode: Caching file names occuring more than  10 times
    13/07/03 15:44:36 INFO common.Storage: Image file of size 112 saved in 0 seconds .
    13/07/03 15:44:36 INFO namenode.FSEditLog: closing edit log: position=4, editlog =	mphadoop-lenovodfs
    amecurrentedits
    13/07/03 15:44:36 INFO namenode.FSEditLog: close success: truncate to 4, editlog =	mphadoop-lenovodfs
    amecurrentedits
    13/07/03 15:44:36 INFO common.Storage: Storage directory 	mphadoop-lenovodfs name has been successfully formatted.
    13/07/03 15:44:36 INFO namenode.NameNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down NameNode at lenovo-PC/10.6.3.180
    ************************************************************/

    第三步,启动Hadoop,执行命令: bin/start-all.sh

    需要输入3次密码

    lenovo@lenovo-PC /hadoop1
    $ bin/start-all.sh
    starting namenode, logging to /hadoop1/libexec/../logs/hadoop-lenovo-namenode-lenovo-PC.out
    lenovo@localhost's password:
    localhost: starting datanode, logging to /hadoop1/libexec/../logs/hadoop-lenovo-datanode-lenovo-PC.out
    lenovo@localhost's password:
    localhost: starting secondarynamenode, logging to /hadoop1/libexec/../logs/hadoop-lenovo-secondarynamenode-lenovo-PC.out
    starting jobtracker, logging to /hadoop1/libexec/../logs/hadoop-lenovo-jobtracker-lenovo-PC.out
    lenovo@localhost's password:
    localhost: starting tasktracker, logging to /hadoop1/libexec/../logs/hadoop-lenovo-tasktracker-lenovo-PC.out
    
    lenovo@lenovo-PC /hadoop1
    $ ps
          PID    PPID    PGID     WINPID   TTY     UID    STIME COMMAND
        59592       1   42744      60224  pty0    1000 15:45:31 /cygdrive/d/java/Tomcat6/jdk/bin/java
        58584   58340   58584      58612  pty0    1000 15:43:22 /usr/bin/bash
        62296       1   42744      63540  pty0    1000 15:45:48 /cygdrive/d/java/Tomcat6/jdk/bin/java
        58844   58584   58844      58936  pty0    1000 15:54:51 /usr/bin/ps
        62812       1   62208      63448  ?       1000 15:45:46 /cygdrive/d/java/Tomcat6/jdk/bin/java
        58340       1   58340      58340  ?       1000 15:43:22 /usr/bin/mintty
    
    lenovo@lenovo-PC /hadoop1
    $


    hadoop只运行3个JVM进程。


     hadoop默认的NameNode 和 JobTracker

原文地址:https://www.cnblogs.com/dyllove98/p/3170221.html