架构图:
后面有脚本一键安装。。。。
第一步:安装6个迷你版的Linux
第二步:由于迷你版Linux缺少一些常用的工具库,操作起来不方便,推荐安装如下的工具库:
1、安装lrzsz, yum install lrzsz -y
2、安装wget, yum install wget -y
4、安装vim, yum install vim -y
5、安装unzip,yum install unzip -y
6、安装ifconfig,yum install net-tools -y
yum install lrzsz wget vim unzip net-tools -y
7、安装nginx及fastdfs需要的库依赖:
yum install gcc perl openssl openssl-devel pcre pcre-devel zlib zlib-devel libevent libevent-devel -y
第三步:在6个服务器节点上分别安装FastDFS,参考:FastDFS分布式文件系统介绍及单机版安装
第四步:在第1、2(133,135)这两台服务器节点配置tracker,并且启动
配置tracker请修改tracker.conf文件,修改内容如下:
mkdir -p /opt/fastdfs/tracker
cd /etc/fdfs
cp tracker.conf.sample tracker.conf
vim /etc/fdfs/tracker.conf
base_path=/opt/fastdfs/tracker
#测试一下tracker的负载均衡上传文件:修改tracker.conf文件如下的内容即可,该配置默认就为2,不用修改
# the method of selecting group to upload files
# 0: round robin
# 1: specify group
# 2: load balance, select the max free space group to upload file
# 指定负载均衡类型
store_lookup=2
注意要保证配置的文件路径要存在
第五步:在第3、4、5、6(136,137,138,139)这四台服务器节点上配置storage,并且启动
配置storage请修改storage.conf文件,修改内容如下:
mkdir -p /opt/fastdfs/storage
mkdir -p /opt/fastdfs/storage/files
cd /etc/fdfs
cp storage.conf.sample storage.conf
vim storage.conf
group_name=group1 #组名,根据实际情况修改(实际情况为:136,137的group_name为group1,138,139的group_name为group2),值为 group1、group2、group3......
base_path=/opt/fastdfs/storage
store_path0=/opt/fastdfs/storage/files
#两个tracke_server的地址端口
tracker_server=192.168.184.133:22122
tracker_server=192.168.117.135:22122
将指定的目录要创建好,然后就可以启动storage服务了。
至此一个FastDFS本身的集群就搭建好了,可以正常使用了。
注意:为了让storage服务能正常连接tracker,请关闭所有机器的防火墙:
systemctl status firewalld
systemctl disable firewalld
systemctl stop firewalld
启动133,135服务器上的tracker
fdfs_trackerd /etc/fdfs/tracker.conf
启动1136,137,138,139服务器上的storage
fdfs_storaged /etc/fdfs/storage.conf
在storage服务器上执行:
fdfs_monitor /etc/fdfs/storage.conf
状态都为ACTIV表示集群搭建成功,此处日志为只粘贴了136服务器的日志
[root@localhost opt]# fdfs_monitor /etc/fdfs/storage.conf
[2019-05-13 15:26:00] DEBUG - base_path=/opt/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0server_count=2, server_index=1tracker server is 192.168.184.135:22122group count: 2Group 1:
group name = group1
disk total space = 17394 MB
disk free space = 13133 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0Storage 1:id = 192.168.184.136ip_addr = 192.168.184.136 (localhost.localdomain) ACTIVEhttp domain = version = 5.11join time = 2019-05-13 15:18:48up time = 2019-05-13 15:18:48total storage = 17394 MBfree storage = 13133 MBupload priority = 10store_path_count = 1subdir_count_per_path = 256storage_port = 23000storage_http_port = 8888current_write_path = 0source storage id = 192.168.184.137if_trunk_server = 0connection.alloc_count = 256connection.current_count = 1connection.max_count = 1total_upload_count = 0success_upload_count = 0total_append_count = 0success_append_count = 0total_modify_count = 0success_modify_count = 0total_truncate_count = 0success_truncate_count = 0total_set_meta_count = 0success_set_meta_count = 0total_delete_count = 0success_delete_count = 0total_download_count = 0success_download_count = 0total_get_meta_count = 0success_get_meta_count = 0total_create_link_count = 0success_create_link_count = 0total_delete_link_count = 0success_delete_link_count = 0total_upload_bytes = 0success_upload_bytes = 0total_append_bytes = 0success_append_bytes = 0total_modify_bytes = 0success_modify_bytes = 0stotal_download_bytes = 0success_download_bytes = 0total_sync_in_bytes = 0success_sync_in_bytes = 0total_sync_out_bytes = 0success_sync_out_bytes = 0total_file_open_count = 0success_file_open_count = 0total_file_read_count = 0success_file_read_count = 0total_file_write_count = 0success_file_write_count = 0last_heart_beat_time = 2019-05-13 15:26:30last_source_update = 1970-01-01 08:00:00last_sync_update = 1970-01-01 08:00:00last_synced_timestamp = 1970-01-01 08:00:00 Storage 2:id = 192.168.184.137ip_addr = 192.168.184.137 ACTIVEhttp domain = version = 5.11join time = 2019-05-13 15:19:51up time = 2019-05-13 15:25:00total storage = 17394 MBfree storage = 16039 MBupload priority = 10store_path_count = 1subdir_count_per_path = 256storage_port = 23000storage_http_port = 8888current_write_path = 0source storage id = if_trunk_server = 0connection.alloc_count = 256connection.current_count = 1connection.max_count = 1total_upload_count = 0success_upload_count = 0total_append_count = 0success_append_count = 0total_modify_count = 0success_modify_count = 0total_truncate_count = 0success_truncate_count = 0total_set_meta_count = 0success_set_meta_count = 0total_delete_count = 0success_delete_count = 0total_download_count = 0success_download_count = 0total_get_meta_count = 0success_get_meta_count = 0total_create_link_count = 0success_create_link_count = 0total_delete_link_count = 0success_delete_link_count = 0total_upload_bytes = 0success_upload_bytes = 0total_append_bytes = 0success_append_bytes = 0total_modify_bytes = 0success_modify_bytes = 0stotal_download_bytes = 0success_download_bytes = 0total_sync_in_bytes = 0success_sync_in_bytes = 0total_sync_out_bytes = 0success_sync_out_bytes = 0total_file_open_count = 0success_file_open_count = 0total_file_read_count = 0success_file_read_count = 0total_file_write_count = 0success_file_write_count = 0last_heart_beat_time = 2019-05-13 15:26:29last_source_update = 1970-01-01 08:00:00last_sync_update = 1970-01-01 08:00:00last_synced_timestamp = 1970-01-01 08:00:00 Group 2:
group name = group2
disk total space = 17394 MB
disk free space = 16039 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0Storage 1:id = 192.168.184.138ip_addr = 192.168.184.138 ACTIVEhttp domain = version = 5.11join time = 2019-05-13 15:19:52up time = 2019-05-13 15:19:52total storage = 17394 MBfree storage = 16039 MBupload priority = 10store_path_count = 1subdir_count_per_path = 256storage_port = 23000storage_http_port = 8888current_write_path = 0source storage id = 192.168.184.139if_trunk_server = 0connection.alloc_count = 256connection.current_count = 1connection.max_count = 1total_upload_count = 0success_upload_count = 0total_append_count = 0success_append_count = 0total_modify_count = 0success_modify_count = 0total_truncate_count = 0success_truncate_count = 0total_set_meta_count = 0success_set_meta_count = 0total_delete_count = 0success_delete_count = 0total_download_count = 0success_download_count = 0total_get_meta_count = 0success_get_meta_count = 0total_create_link_count = 0success_create_link_count = 0total_delete_link_count = 0success_delete_link_count = 0total_upload_bytes = 0success_upload_bytes = 0total_append_bytes = 0success_append_bytes = 0total_modify_bytes = 0success_modify_bytes = 0stotal_download_bytes = 0success_download_bytes = 0total_sync_in_bytes = 0success_sync_in_bytes = 0total_sync_out_bytes = 0success_sync_out_bytes = 0total_file_open_count = 0success_file_open_count = 0total_file_read_count = 0success_file_read_count = 0total_file_write_count = 0success_file_write_count = 0last_heart_beat_time = 2019-05-13 15:26:30last_source_update = 1970-01-01 08:00:00last_sync_update = 1970-01-01 08:00:00last_synced_timestamp = 1970-01-01 08:00:00 Storage 2:id = 192.168.184.139ip_addr = 192.168.184.139 ACTIVEhttp domain = version = 5.11join time = 2019-05-13 15:19:54up time = 2019-05-13 15:25:43total storage = 17394 MBfree storage = 16039 MBupload priority = 10store_path_count = 1subdir_count_per_path = 256storage_port = 23000storage_http_port = 8888current_write_path = 0source storage id = if_trunk_server = 0connection.alloc_count = 256connection.current_count = 1connection.max_count = 1total_upload_count = 0success_upload_count = 0total_append_count = 0success_append_count = 0total_modify_count = 0success_modify_count = 0total_truncate_count = 0success_truncate_count = 0total_set_meta_count = 0success_set_meta_count = 0total_delete_count = 0success_delete_count = 0total_download_count = 0success_download_count = 0total_get_meta_count = 0success_get_meta_count = 0total_create_link_count = 0success_create_link_count = 0total_delete_link_count = 0success_delete_link_count = 0total_upload_bytes = 0success_upload_bytes = 0total_append_bytes = 0success_append_bytes = 0total_modify_bytes = 0success_modify_bytes = 0stotal_download_bytes = 0success_download_bytes = 0total_sync_in_bytes = 0success_sync_in_bytes = 0total_sync_out_bytes = 0success_sync_out_bytes = 0total_file_open_count = 0success_file_open_count = 0total_file_read_count = 0success_file_read_count = 0total_file_write_count = 0success_file_write_count = 0last_heart_beat_time = 2019-05-13 15:26:41last_source_update = 1970-01-01 08:00:00last_sync_update = 1970-01-01 08:00:00last_synced_timestamp = 1970-01-01 08:00:00
[root@localhost opt]#
测试之前,需要修改client.conf配置文件,修改两个配置:
base_path=/opt/fastdfs/client
tracker_server=192.168.184.133:22122
tracker_server=192.168.184.135:22122
新建文件夹:
mkdir -p /opt/fastdfs/client
测试:
fdfs_test /etc/fdfs/client.conf upload /root/testfastfds.txt
显示:
[root@localhost fdfs]# fdfs_test /etc/fdfs/client.conf upload /root/testfastfds.txt
This is FastDFS client test program v5.11Copyright (C) 2008, Happy Fish / YuQingFastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.[2019-05-13 15:34:10] DEBUG - base_path=/opt/fastdfs/client, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0tracker_query_storage_store_list_without_group: server 1. group_name=, ip_addr=192.168.184.138, port=23000server 2. group_name=, ip_addr=192.168.184.139, port=23000group_name=group2, ip_addr=192.168.184.138, port=23000
storage_upload_by_filename
group_name=group2, remote_filename=M00/00/00/wKi4ilzZHfKAQbXhAAAAIIe1k0c677.txt
source ip address: 192.168.184.138
file timestamp=2019-05-13 15:34:10
file size=32
file crc32=2276823879
example file url: http://192.168.184.138/group2/M00/00/00/wKi4ilzZHfKAQbXhAAAAIIe1k0c677.txt
storage_upload_slave_by_filename
group_name=group2, remote_filename=M00/00/00/wKi4ilzZHfKAQbXhAAAAIIe1k0c677_big.txt
source ip address: 192.168.184.138
file timestamp=2019-05-13 15:34:10
file size=32
file crc32=2276823879
example file url: http://192.168.184.138/group2/M00/00/00/wKi4ilzZHfKAQbXhAAAAIIe1k0c677_big.txt
去group2(138,139)里面看到都存储了相同的内容,上传成功
测试结果:把group2的storage的所有节点都停了,再次上传文件则会负载到group1中额storage节点中,如果重启group2中的138节点,则会再次向group2节点中的138节点上传文件,这时候启动group2中的139节点,则会同步group2中的138节点中的文件。
到此FastDFS集群搭建完毕
脚本:只是安装,配置文件自行修改,在任意目录下执行该文件,sh installFastdfs.sh所需要的安装包在/usr/home/soft/fastdfs,安装路径在/usr/local/fastdfs
vim installFastdfs.sh
#安装一些工具库,方便操作
yum install lrzsz wget vim unzip net-tools -y
#安装nginx及fastdfs需要的库依赖
yum install gcc perl openssl openssl-devel pcre pcre-devel zlib zlib-devel libevent libevent-devel -y
mkdir -p /home/soft/fastdfs
cd /home/soft/fastdfs
wget https://github.com/happyfish100/libfastcommon/archive/V1.0.36.tar.gz -O libfastcommon-1.0.36.tar.gz
wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz -O fastdfs-5.11.tar.gz
mkdir -p /usr/local/fastdfs
tar -zxvf libfastcommon-1.0.36.tar.gz -C /usr/local/fastdfs/
cd /usr/local/fastdfs/libfastcommon-1.0.36/
./make.sh
./make.sh install
#安装FastDFS
cd /home/soft/fastdfs
tar -zxvf fastdfs-5.11.tar.gz -C /usr/local/fastdfs/
cd /usr/local/fastdfs/fastdfs-5.11/
./make.sh
./make.sh install
#新建fastdfs配置文件中所需的路径
mkdir -p /opt/fastdfs/tracker
mkdir -p /opt/fastdfs/storage
mkdir -p /opt/fastdfs/storage/files
mkdir -p /opt/fastdfs/client
//一道亮丽的风景线//
接下搭建第二个维度的集群(通过Http访问FastDFS上的文件)
第一步:在第3、4、5、6(136,137,138,139)这四台服务器节点上安装带有FastDFS扩展模块的Nginx
参考:FastDFS分布式文件系统介绍及单机版安装:十二、安装Nginx的FastDFS扩展模块
第二步:配置第3、4、5、6(136,137,138,139)这四台服务器节点上的Nginx
vim /usr/local/fastdfs/nginx_fdfs/conf/nginx.conf
#nginx拦截请求路径:
location ~ /group[1-9]/M0[0-9] { ngx_fastdfs_module;
}
ngx_fastdfs_module;指定依赖一个配置文件,所以需要配置mod_fastdfs.conf,需要配置和修改的内容如下:
cp /home/soft/fastdfs/fastdfs-nginx-module-73869650b6171fd6e5bf81b5b8caa89f021d6c7b/src/mod_fastdfs.conf /etc/fdfs/
vim /etc/fdfs/mod_fastdfs.conf base_path=/opt/fastdfs/mod_fastdfs #保存日志目录
tracker_server=192.168.184.133:22122 #tracker服务器的IP地址以及端口号
tracker_server=192.168.184.135:22122
group_name=group1 #当前服务器的group名
url_have_group_name=true #文件url中是否有group名
store_path0=/opt/fastdfs/storage/files #存储路径
group_count = 2 #设置组的个数在末尾增加2个组的具体信息:
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/opt/fastdfs/storage/files[group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/opt/fastdfs/storage/files
以上配置注意:文件路径要存在,所以请先创建好路径,否则会报错;测试验证第3、4、5、6这四台服务器节点上的Nginx,通过Nginx访问已经上传到FastDFS上的文件;
启动4台服务器上的Nginx:
通过浏览器访问:
http://192.168.184.136/group1/M00/00/00/wKi4iVzZIXmALSF_AAAAJrfiD9E295.txt
http://192.168.184.137/group1/M00/00/00/wKi4iVzZIXmALSF_AAAAJrfiD9E295.txt
http://192.168.184.138/group2/M00/00/00/wKi4ilzZIZ6AZ90CAAAAJrfiD9E276.txt
http://192.168.184.139/group2/M00/00/00/wKi4ilzZIZ6AZ90CAAAAJrfiD9E276.txt
这里只列出一个storage服务的访问。
第三步:部署两个tracker server服务器上的Nginx,需要做的工作,这个Nginx作用不是集成fastDFS模块,只是负载转发
部署配置nginx负载均衡:
upstream fastdfs_group_server {server 192.168.184.136:80;server 192.168.184.137:80;server 192.168.184.138:80;server 192.168.184.139:80;}
#nginx拦截请求路径:
location ~ /group[1-9]/M0[0-9] { proxy_pass http://fastdfs_group_server;}
启动这两个Nginx,然后测试访问:
http://192.168.184.133/group1/M00/00/00/wKi4iVzZIXmALSF_AAAAJrfiD9E295.txt
http://192.168.184.135/group2/M00/00/00/wKi4ilzZIZ6AZ90CAAAAJrfiD9E276.txt
第四步:部署前端用户访问入口Nginx服务器,即访问本地windows上的Nginx,该Nginx负载均衡到后端2个tracker server上的Nginx;
部署配置访问入口nginx负载均衡:
upstream fastdfs_access_server { server 192.168.184.133:80; server 192.168.117.135:80;
}#nginx拦截请求路径:
location ~ /group[1-9]/M0[0-9] { proxy_pass http://fastdfs_group_server;
}
启动window上的nginx:
测试访问:
http://localhost/group1/M00/00/00/wKi4iVzZIXmALSF_AAAAJrfiD9E295.txt
http://localhost/group2/M00/00/00/wKi4ilzZIZ6AZ90CAAAAJrfiD9E276.txt
到此FastDFS集群搭建完毕
脚本:vim installFastdfsNginxMod.sh
cd /home/soft/fastdfs/
wget https://github.com/happyfish100/fastdfs-nginx-module/archive/73869650b6171fd6e5bf81b5b8caa89f021d6c7b.zip -O fastdfs-nginx-module.zip
wget http://nginx.org/download/nginx-1.12.2.tar.gz
unzip fastdfs-nginx-module.zip
tar -zxvf nginx-1.12.2.tar.gz
cd nginx-1.12.2
./configure --prefix=/usr/local/fastdfs/nginx_fdfs --add-module=/home/soft/fastdfs/fastdfs-nginx-module-73869650b6171fd6e5bf81b5b8caa89f021d6c7b/src
make && make install
cp /home/soft/fastdfs/fastdfs-nginx-module-73869650b6171fd6e5bf81b5b8caa89f021d6c7b/src/mod_fastdfs.conf /etc/fdfs/
mkdir -p /opt/fastdfs/nginx_mod
cd /usr/local/fastdfs/fastdfs-5.11/conf
cp http.conf /etc/fdfs/
cp mime.types /etc/fdfs/
有了这两个脚本安装FastDFS就如此简单了。。。