Run Cluster by Yum Tools
Yum tools to run a CubeFS cluster for CentOS 7+ is provided. The list of RPM packages dependencies can be installed with:
$ yum install https://ocs-cn-north1.heytapcs.com/cubefs/rpm/3.0.0/cfs-install-3.0.0-el7.x86_64.rpm
$ cd /cfs/install
$ tree -L 2
.
├── install_cfs.yml
├── install.sh
├── iplist
├── src
└── template
├── client.json.j2
├── console.json.j2
├── create_vol.sh.j2
├── datanode.json.j2
├── grafana
│ ├── grafana.ini
│ ├── init.sh
│ └── provisioning
├── master.json.j2
├── metanode.json.j2
└── objectnode.json.j2
Set parameters of the CubeFS cluster in iplist.
[master], [datanode] , [metanode] , [console], [monitor] , [client] modules includes member IP addresses of each role.
[cfs:vars] module includes parameters for SSH connection. So make sure the port, username and password for SSH connection is unified before start.
#master config module includes parameters of Master.
Key |
Type |
Description |
Mandatory |
|---|---|---|---|
master_clusterName |
string |
The cluster identifier |
Yes |
master_listen |
string |
Http port which api service listen on |
Yes |
master_prof |
string |
golang pprof port |
Yes |
master_logDir |
string |
Path for log file storage |
Yes |
master_logLevel |
string |
Level operation for logging. Default is info. |
No |
master_retainLogs |
string |
the number of raft logs will be retain. |
Yes |
master_walDir |
string |
Path for raft log file storage. |
Yes |
master_storeDir |
string |
Path for RocksDB file storage,path must be exist |
Yes |
master_exporterPort |
int |
The prometheus exporter port |
No |
master_metaNodeReservedMem |
string |
If the metanode memory is below this value, it will be marked as read-only. Unit: byte. 1073741824 by default. |
No |
#datanode config module includes parameters of DataNodes.
Key |
Type |
Description |
Mandatory |
|---|---|---|---|
datanode_listen |
string |
Port of TCP network to be listen |
Yes |
datanode_prof |
string |
Port of HTTP based prof and api service |
Yes |
datanode_logDir |
string |
Path for log file storage |
Yes |
datanode_logLevel |
string |
Level operation for logging. Default is info |
No |
datanode_raftHeartbeat |
string |
Port of raft heartbeat TCP network to be listen |
Yes |
datanode_raftReplica |
string |
Port of raft replicate TCP network to be listen |
Yes |
datanode_raftDir |
string |
Path for raft log file storage |
No |
datanode_exporterPort |
string |
Port for monitor system |
No |
datanode_disks |
string slice |
Format: PATH:RETAIN.
PATH: Disk mount point, must exists. RETAIN: Retain space. (Ranges: 20G-50G.)
RETAIN: The minimum free space(Bytes) reserved for the path.
|
Yes |
#metanode config module includes parameters of MetaNodes.
Key |
Type |
Description |
Mandatory |
|
|---|---|---|---|---|
metanode_listen |
string |
Listen and accept port of the server |
Yes |
|
metanode_prof |
string |
Pprof port |
Yes |
|
metanode_logLevel |
string |
Level operation for logging. Default is info |
No |
|
metanode_metadataDir |
string |
MetaNode store snapshot directory |
Yes |
|
metanode_logDir |
string |
Log directory |
Yes |
|
metanode_raftDir |
string |
Raft wal directory |
Yes |
|
metanode_raftHeartbeatPort |
string |
Raft heartbeat port |
Yes |
|
metanode_raftReplicaPort |
string |
Raft replicate port |
Yes |
|
metanode_exporterPort |
string |
Port for monitor system |
No |
|
metanode_totalMem |
string |
Max memory metadata used. The value needs to be higher than the value of metaNodeReservedMem in the master configuration. Unit: byte |
Yes |
#objectnode config module includes parameters of ObjectNodes.
Key |
Type |
Description |
Mandatory |
|---|---|---|---|
objectnode_listen |
string |
Listen and accept port of the server |
Yes |
objectnode_domains |
string slice |
Domain of S3-like interface which makes wildcard domain support
Format:
DOMAIN |
No |
objectnode_logDir |
string |
Log directory |
Yes |
objectnode_logLevel |
string |
Level operation for logging.
Default:
error |
No |
objectnode_exporterPort |
string |
Port for monitor system |
No |
objectnode_enableHTTPS |
string |
Enable HTTPS |
Yes |
#console config module includes parameters of Console.
Key |
Type |
Description |
Mandatory |
|---|---|---|---|
console_logDir |
string |
Path for log file storage |
Yes |
console_logLevel |
string |
Level operation for logging. Default is info |
No |
console_listen |
string |
Port of TCP network to be listen, default is 80 |
Yes |
#client config module includes parameters of Client.
Key |
Type |
Description |
Mandatory |
|---|---|---|---|
client_mountPoint |
string |
Mount point of the client |
Yes |
client_volName |
string |
Volume name |
No |
client_owner |
string |
Owner id |
Yes |
client_SizeGB |
string |
Initial size to create the volume if it not exists |
No |
client_logDir |
string |
Path for log file storage |
Yes |
client_logLevel |
string |
Level operation for logging. Default is info |
No |
client_exporterPort |
string |
Port exposed to monitor system |
Yes |
client_profPort |
string |
Pprof port |
No |
[master]
10.196.59.198
10.196.59.199
10.196.59.200
[datanode]
...
[cfs:vars]
ansible_ssh_port=22
ansible_ssh_user=root
ansible_ssh_pass="password"
...
#master config
...
#datanode config
...
datanode_disks = '"/data0:10737418240","/data1:10737418240"'
...
#metanode config
...
metanode_totalMem = "28589934592"
...
#objectnode config
...
#console config
...
For more configurations, please refer to Resource Manager (Master); Meta Subsystem; Data Subsystem; Client; Monitor; Console.
Start the resources of CubeFS cluster with script install.sh . (make sure the Master is started first)
$ bash install.sh -h
Usage: install.sh -r | --role [datanode | metanode | master | objectnode | console | monitor | client | all | createvol ]
$ bash install.sh -r master
$ bash install.sh -r metanode
$ bash install.sh -r datanode
$ bash install.sh -r objectnode
$ bash install.sh -r console
$ bash install.sh -r monitor
$ bash install.sh -r client
Check mount point at /cfs/mountpoint on client node defined in iplist .
Open http://consul.prometheus-cfs.local in browser for monitoring system(the IP of monitoring system is defined in iplist ).