memcache 集群配置(通过magent)

介绍:

  memcached本身没有内置分布式功能,无法实现使用多台Memcache服务器来存储不同的数据,最大程度的使用相同的资源;无法同步数据,容易造成单点故障。

  在 Memcached中可以保存的item数据量是没有限制的,只要内存足够 。
  Memcached单进程最大使用内存为2G,要使用更多内存,可以分多个端口开启多个Memcached进程 
  最大30天的数据过期时间,设置为永久的也会在这个时间过期,常量REALTIME_MAXDELTA 
  60*60*24*30控制 
  最大键长为250字节,大于该长度无法存储,常量KEY_MAX_LENGTH 250控制 
  单个item最大数据是1MB,超过1MB数据不予存储,常量POWER_BLOCK 1048576进行控制, 
  它是默认的slab大小 
  最大同时连接数是200,通过 conn_init()中的freetotal进行控制,最大软连接数是1024,通过 
  settings.maxconns=1024 进行控制 
  跟空间占用相关的参数:settings.factor=1.25, settings.chunk_size=48, 影响slab的数据占用和步进方式

  memcached是一种无阻塞的socket通信方式服务,基于libevent库,由于无阻塞通信,对内存读写速度非常之快。
  memcached分服务器端和客户端,可以配置多个服务器端和客户端,应用于分布式的服务非常广泛。
  memcached作为小规模的数据分布式平台是十分有效果的。

  memcached是键值一一对应,key默认最大不能超过128个字 节,value默认大小是1M,也就是一个slabs,如果要存2M的值(连续的),不能用两个slabs,因为两个slabs不是连  续的,无法在内存中 存储,故需要修改slabs的大小,多个key和value进行存储时,即使这个slabs没有利用完,那么也不会存放别的数据。

magent能够消除单点故障问题,但是不能实现负载均衡。(mcrouter流量分发)

1:cache安装脚本:

install_cache(){

cd $INSTALL_HOME

mkdir -p /opt

if [ -f libevent-1.4.14b-stable.tar.gz ] ; then
tar zxvf libevent-1.4.14b-stable.tar.gz >> $INSTALL_HOME/install.log
else
lftp -c "pget -n 10 $FTP_ADDR/softwares/memcache/libevent-1.4.14b-stable.tar.gz"
tar zxvf libevent-1.4.14b-stable.tar.gz >> $INSTALL_HOME/install.log
fi

cd libevent-1.4.14b-stable
./configure --prefix=/usr >> $INSTALL_HOME/install.log
make >> $INSTALL_HOME/install.log
make install >> $INSTALL_HOME/install.log
cd $INSTALL_HOME
rm -rf libevent-1.4.14b-stable*

echo 'install libevent succesed!'

cd $INSTALL_HOME

echo 'install the memcached server...'
cd $INSTALL_HOME

if [ -f memcached-1.4.5.tar.gz ] ; then
tar zxvf memcached-1.4.5.tar.gz >> $INSTALL_HOME/install.log
else
lftp -c "pget -n 10 $FTP_ADDR/softwares/memcache/memcached-1.4.5.tar.gz"
tar zxvf memcached-1.4.5.tar.gz >> $INSTALL_HOME/install.log
fi
cd memcached-1.4.5
./configure --prefix=/opt/memcached --with-libevent=/usr >> $INSTALL_HOME/install.log
make >> $INSTALL_HOME/install.log
make install >> $INSTALL_HOME/install.log
cd ..
rm -rf memcached-1.4.5*

ln -s /usr/lib/libevent-1.4.so.2 /usr/lib64/libevent-1.4.so.2

echo '========********install memcached successed!******* ======'

echo "/opt/memcached/bin/memcached -d -m 1024 -u root -p 11211" >>/etc/rc.local

}

2:安装magent代理

下载      magent-0.5.tar.gz

mkdir magent

tar -zxvf magent-0.5.tar.gz -C kmagent

[root@shell magent]# /sbin/ldconfig
[root@shell magent]# sed -i "s#LIBS = -levent#LIBS = -levent -lm#g" Makefile
[root@shell magent]# make
gcc -Wall -O2 -g -c -o magent.o magent.c
magent.c: 在函数‘writev_list’中:
magent.c:623: 错误:‘SSIZE_MAX’未声明(在此函数内第一次使用)
magent.c:623: 错误:(即使在一个函数内多次出现,每个未声明的标识符在其
magent.c:623: 错误:所在的函数内也只报告一次。)
make: *** [magent.o] 错误 1

参考:http://www.php-note.com/article/detail/820 

在开头加入:vim  magent.c

#ifndef SSIZE_MAX
#define SSIZE_MAX 32767
#endif

重新执行:

[root@shell magent]# make
gcc -Wall -O2 -g -c -o magent.o magent.c
gcc -Wall -O2 -g -c -o ketama.o ketama.c
gcc -Wall -O2 -g -o magent magent.o ketama.o -levent -lm

cp magent /usr/bin/magent

执行下magent试试

[root@shell ~]# magent
please provide -s "ip:port" argument

memcached agent v0.4 Build-Date: Nov 26 2015 09:58:35
Usage:
-h this message
-u uid
-g gid
-p port, default is 11211. (0 to disable tcp support)
-s ip:port, set memcached server ip and port
-b ip:port, set backup memcached server ip and port
-l ip, local bind ip address, default is 0.0.0.0
-n number, set max connections, default is 4096
-D don't go to background
-k use ketama key allocation algorithm
-f file, unix socket path to listen on. default is off
-i number, set max keep alive connections for one memcached server, default is 20
-v verbose

3:查看memcached缓存(服务器为:192.168.16.239,192.168.16.238,192.168.16.241)

[root@shell ~]# ps aux|grep cache
root 18751 0.0 0.0 335052 896 ? Ssl 10:05 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11211
root 18758 0.0 0.0 335052 892 ? Ssl 10:05 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11212
root 18765 0.0 0.0 335052 892 ? Ssl 10:05 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11213

[root@oracle10g src]# ps aux|grep cache
root 5411 0.0 0.0 335052 896 ? Ssl 10:15 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.238 -p 11211
root 5419 0.0 0.0 335052 896 ? Ssl 10:15 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.238 -p 11212
root 5426 0.0 0.0 335052 896 ? Ssl 10:15 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.238 -p 11213

1):先才用单台服务器进行测试

magent -u root -n 51200 -l 192.168.16.239 -p 12000 -s 192.168.16.239:11211 -s 192.168.16.239:11212 -b 192.168.16.239:11213

[root@shell ~]# telnet 192.168.16.239 12000
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
set key1 16 0 4
zhao
STORED
set key2 16 0 5
liang
STORED

[root@shell opt]# telnet 192.168.16.239 11211

Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
get key1
END
get key2
VALUE key2 16 5
liang
END

[root@shell opt]# telnet 192.168.16.239 11212
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
get key1
VALUE key1 16 4
zhao
END
get key2
END

[root@shell opt]# telnet 192.168.16.239 11213
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
get key1
VALUE key1 16 4
zhao
END
get key2
VALUE key2 16 5
liang
END

2)不用备份服务端,停掉一个server

[root@shell opt]# magent -u root -n 51200 -l 192.168.16.239 -p 12000 -s 192.168.16.239:11211 -s 192.168.16.239:11212
[root@shell opt]# /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11211
[root@shell opt]# /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11212

[root@shell ~]# telnet 192.168.16.239 12000
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
set key1 16 0 4
zhao
STORED
set key2 16 0 5
liang
STORED
set key3 16 0 5
liang
STORED

[root@shell opt]# telnet 192.168.16.239 11211
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
get key1
END
get key2
VALUE key2 16 5
liang
END
quit
Connection closed by foreign host.
[root@shell opt]# telnet 192.168.16.239 11212
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
get key1
VALUE key1 16 4
zhao
END
get key2
END

[root@shell ~]# ps aux|grep cache
root 30357 0.0 0.1 336076 1968 ? Ssl 14:54 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11211
root 30364 0.0 0.1 336076 1972 ? Ssl 14:54 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11212

[root@shell ~]# kill -9 30357

[root@shell ~]# ps aux|grep cache
root 30364 0.0 0.1 336076 1972 ? Ssl 14:54 0:00 /opt/memcached/bin/memcached -d -m 64 -u root -l 192.168.16.239 -p 11212

[root@shell opt]# telnet 192.168.16.239 11212
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
get key1
VALUE key1 16 4
zhao
END
get key2
END
get key1
VALUE key1 16 4
zhao
END
get key3
VALUE key3 16 5
liang
END
quit
Connection closed by foreign host.
[root@shell opt]# telnet 192.168.16.239 12000
Trying 192.168.16.239...
Connected to 192.168.16.239.
Escape character is '^]'.
get key3
VALUE key3 16 5
liang
END

脚本:

#!/bin/bash

for ((i=1; i<=100; i++))
do

printf "set key$i 0 0 5 12345 " | nc 192.168.16.239 11211

done

printf "get key$i " | nc 192.168.16.239 11211

原文地址:https://www.cnblogs.com/liangsky/p/4997819.html