nginx+tomcat负载均衡+动静分离+redis集中管理session

1、服务器A安装ng，服务器B、C安装tomcat；

2、服务器A建立/data/www目录，用于发布静态文件；

3、ng无动静分离配置；

user root root;
worker_processes 8;
pid /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 102400;
events
{
use epoll;
worker_connections 102400;
}
http
{
  include       mime.types;
  default_type  application/octet-stream;
  fastcgi_intercept_errors on;
  charset  utf-8;
  server_names_hash_bucket_size 128;
  client_header_buffer_size 4k;
  large_client_header_buffers 4 32k;
  client_max_body_size 300m;
  sendfile on;
  tcp_nopush     on;
   
  keepalive_timeout 60;
   
  tcp_nodelay on;
  client_body_buffer_size  512k;
 
  proxy_connect_timeout    5;
  proxy_read_timeout       60;
  proxy_send_timeout       5;
  proxy_buffer_size        16k;
  proxy_buffers            4 64k;
  proxy_busy_buffers_size 128k;
  proxy_temp_file_write_size 128k;
   
  gzip on;
  gzip_min_length  1k;
  gzip_buffers     4 16k;
  gzip_http_version 1.1;
  gzip_comp_level 2;
  gzip_types       text/plain application/x-javascript text/css application/xml;
  gzip_vary on;
   
###2012-12-19 change nginx logs
log_format  main  '$http_x_forwarded_for - $remote_user [$time_local] "$request" '
              '$status $body_bytes_sent "$http_referer" '
              '"$http_user_agent"  $request_time $remote_addr';
               
upstream web_app {
 server 192.168.203.130:10086 weight=1 max_fails=2 fail_timeout=30s;
 server 192.168.203.131:10087 weight=1 max_fails=2 fail_timeout=30s;
}
 
####chinaapp.sinaapp.com
server {
    listen 80;
    server_name  chinaapp.sinaapp.com;
    index index.jsp index.html index.htm;
    # 发布目录:/data/www
    root  /data/www;
     
    location /
    {
        proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_set_header Host  $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://web_app;
        expires      3d;
    }
  
  }
 
}

View Code

4、ng动静分离配置；

user root root;
worker_processes 8;
pid /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 102400;
events
{
use epoll;
worker_connections 102400;
}
http
{
  include       mime.types;
  default_type  application/octet-stream;
  fastcgi_intercept_errors on;
  charset  utf-8;
  server_names_hash_bucket_size 128;
  client_header_buffer_size 4k;
  large_client_header_buffers 4 32k;
  client_max_body_size 300m;
  sendfile on;
  tcp_nopush     on;
   
  keepalive_timeout 60;
   
  tcp_nodelay on;
  client_body_buffer_size  512k;
 
  proxy_connect_timeout    5;
  proxy_read_timeout       60;
  proxy_send_timeout       5;
  proxy_buffer_size        16k;
  proxy_buffers            4 64k;
  proxy_busy_buffers_size 128k;
  proxy_temp_file_write_size 128k;
   
  gzip on;
  gzip_min_length  1k;
  gzip_buffers     4 16k;
  gzip_http_version 1.1;
  gzip_comp_level 2;
  gzip_types       text/plain application/x-javascript text/css application/xml;
  gzip_vary on;
   
###2012-12-19 change nginx logs
log_format  main  '$http_x_forwarded_for - $remote_user [$time_local] "$request" '
              '$status $body_bytes_sent "$http_referer" '
              '"$http_user_agent"  $request_time $remote_addr';
               
upstream web_app {
 server 192.168.203.130:10086 weight=1 max_fails=2 fail_timeout=30s;
 server 192.168.203.131:10087 weight=1 max_fails=2 fail_timeout=30s;
}
 
####chinaapp.sinaapp.com
server {
    listen 80;
    server_name  chinaapp.sinaapp.com;
    index index.jsp index.html index.htm;
    # 发布目录:/data/www
    root  /data/www;
     
    location /
    {
        proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_set_header Host  $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://web_app;
        expires      3d;
    }
    location ~ .*.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$
    {
        root /data/www; 
        #expires定义用户浏览器缓存的时间为3天，如果静态页面不常更新，可以设置更长，这样可以节省带宽和缓解服务器的压力
        expires      3d;
     
    }
  
  }
 
}

View Code

5、安装ng的时候不用指定编译参数，直接./configure --prefix=/usr/local/nginx & make & make install就行；

6、这里直接访问域名依然会请求tomcat，没搞清楚为什么，只有域名后面带有具体静态文件才会从ng请求；

7、下载https://github.com/jcoleman/tomcat-redis-session-manager，修改相关参数，编译，将下面的9个文件放到tomcat的lib目录下；

gradle.build修改成为：

apply plugin: 'java'
apply plugin: 'maven'
apply plugin: 'signing'

group = 'com.orangefunction'
version = '2.0.0'

repositories {
  mavenCentral()
}

compileJava {
  sourceCompatibility = 1.7
  targetCompatibility = 1.7
}

dependencies {
  compile group: 'org.apache.tomcat', name: 'tomcat-catalina', version: '7.0.67'
  compile group: 'redis.clients', name: 'jedis', version: '2.5.2'
  compile group: 'org.apache.commons', name: 'commons-pool2', version: '2.2'
  //compile group: 'commons-codec', name: 'commons-codec', version: '1.9'

  testCompile group: 'junit', name: 'junit', version: '4.+'
  testCompile 'org.hamcrest:hamcrest-core:1.3'
  testCompile 'org.hamcrest:hamcrest-library:1.3'
  testCompile 'org.mockito:mockito-all:1.9.5'
  testCompile group: 'org.apache.tomcat', name: 'tomcat-coyote', version: '7.0.67'
}

task javadocJar(type: Jar, dependsOn: javadoc) {
  classifier = 'javadoc'
  from 'build/docs/javadoc'
}

task sourcesJar(type: Jar) {
  from sourceSets.main.allSource
  classifier = 'sources'
}

artifacts {
  archives jar

  archives javadocJar
  archives sourcesJar
}

//signing {
//  sign configurations.archives
//}

task copyJars(type: Copy) {
  from configurations.runtime
  into 'dist'  
}

uploadArchives {
  repositories {
    mavenDeployer {
      beforeDeployment { MavenDeployment deployment -> signing.signPom(deployment) }

      //repository(url: "https://oss.sonatype.org/service/local/staging/deploy/maven2/") {
      //  authentication(userName: sonatypeUsername, password: sonatypePassword)
      //}
      //repository(url: "https://oss.sonatype.org/content/repositories/snapshots") {
      //  authentication(userName: sonatypeUsername, password: sonatypePassword)
      //}

      pom.project {
        name 'tomcat-redis-session-manager'
        packaging 'jar'
        description 'Tomcat Redis Session Manager is a Tomcat extension to store sessions in Redis'
        url 'https://github.com/jcoleman/tomcat-redis-session-manager'

        issueManagement {
          url 'https://github.com:jcoleman/tomcat-redis-session-manager/issues'
          system 'GitHub Issues'
        }

        scm {
          url 'https://github.com:jcoleman/tomcat-redis-session-manager'
          connection 'scm:git:git://github.com/jcoleman/tomcat-redis-session-manager.git'
          developerConnection 'scm:git:git@github.com:jcoleman/tomcat-redis-session-manager.git'
        }

        licenses {
          license {
            name 'MIT'
            url 'http://opensource.org/licenses/MIT'
            distribution 'repo'
          }
        }

        developers {
          developer {
            id 'jcoleman'
            name 'James Coleman'
            email 'jtc331@gmail.com'
            url 'https://github.com/jcoleman'
          }
        }
      }
    }
  }
}

View Code

8、编译完之后，上面的倒数第三那个是在build目录下面；

9、修改tomcat配置文件：

<Context>
      <Valve className="com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve" />
      <Manager className="com.orangefunction.tomcat.redissessions.RedisSessionManager"
           host="redis主机ip地址"
           port="6379"
           database="0"
           maxInactiveInterval="60" />
</Context>

View Code

10、上面的host是redis的地址，如果tomcat和redis没有在同一台机器上，那么tomcat的启动是比较慢的，需要耐心等候，不是报错；

11、一般来说，session中我们记录用户的信息，如果记录单个属性，那么没什么说得，如果session的属性是用户对象，那么该对象需要实现Serializable接口；

12、异常记录：由于我的数据库装载笔记本实体机上面，而ng，tomcat1，tomcat2分别装在vm的3个不同的机器里面的，所以导致vm里面的机器连接实体机的数据库时报错，无法连接，我们需要为数据库增加访问权限：

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION; 
FLUSH   PRIVILEGES;

13、另外：tomcat启动真的很慢，需要1分钟，搞不懂为什么，我这里把文件做了动静分离的，把静态文件放在ng的/data/www目录下面，而ajax请求会被做负载均衡。