tisample

command module

v0.0.0-...-a027a2a Latest Latest Go to latest Published: Jan 26, 2022 License: Apache-2.0 Imports: 1 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/luyomo/tisample

README ¶

Oh My TiUP

项目介绍

Oh My TiUP 的命名灵感来自于 Oh My Zsh 项目，希望能像 Oh My Zsh 一样成为一个趁手的生产力工具。Oh My TiUP 将基于 TiUP，围绕着易用性打造一系列甜品级的特性。

背景&动机

TiUP 作为 TiDB 系列所有产品线的入口，是非常重要的生态工具，可谓是 TiDB 的门神。目前 TiUP 作为部署工具已经收到了广泛地认可，但仍然在一些场景存在力不从心的情况。我们将以提高易用性，降低用户门槛为宗旨，为 TiUP 支持一系列甜品级的特性。

项目设计

TiUP Cluster on Cloud

背景

TiUP Cluster 用于在生产上部署 TiDB 集群，提供了非常丰富的功能以适用于生产上复杂的部署环境。但 TiUP Cluster 主要是为 on-premise 场景部署而设计的，并未考虑云上场景。虽然在云上场景中，tidb operator 已经能提供丰富的集群部署和管理能力，但 tidb operator 依赖 K8s，在追求轻量和易运维的场景中并不适用。目前在云上但不使用 K8s 的场景中，TiDB 集群的部署和管理仍然是一块空白。

产品设计

我们设计了一个新的名为 cloud-cluster 的 TiUP 组件，专门用于 TiDB 集群的云上部署和管理。

设计架构

alt text

部署集群

在部署集群时，cloud-cluster 不需要用户指定要部署的目标主机 IP，而是用户指定要部署的目标主机机型，Region、AZ 等信息，cloud-cluster 可以自动向 AWS 申请虚机并部署 TiDB 集群。

能力

deploy 集群时可以自动根据用户的配置向 AWS 申请 EC2（包含 VPC，EBS 配置）
EC2 创建并启动完毕后，tiup 可以自动将 TiDB 产品线族部署到目标 EC2
根据 ec2 的 region 和 az，自动设置实例的物理位置 label
tiup 可以自动创建 ELB，作为 TiDB 上层的负载均衡器

User Interface

官方 deploy 组件部署集群的命令如下： tiup cluster deploy tidb-test v5.3.0 ./topology.yaml --user root [-p] [-i /home/root/.ssh/gcp_rsa]

cloud-deploy 组件部署集群命令如下： tiup cloud-cluster deploy tidb-test v5.3.0 ./topology.yaml #TODO：加一些用于 AWS 授权的参数

相比与官方 deploy 指令，cloud-cluster 的 deploy 指令去除了 -p -i 这种用于 ssh 授权的指令，而添加一些用于 AWS 授权的指令

配置文件：

aws_topo_configs:
  general:
    imageid: ami-0ac97798ccf296e02            # Image ID for TiDB cluster's EC2 node
    keyname: jay.pingcap                      # key name to login from workstation to EC2 nodes
    cidr: 172.83.0.0/16                       # VPC cidr
    instance_type: m5.2xlarge                 # default instance type for EC2 nodes
    tidb_version: v5.2.0                      # TiDB version to deploy
  pd:
    instance_type: m5.2xlarge                 # PD instance type
    count: 3                                  # Number of PD nodes to generate
  tidb:
    instance_type: m5.2xlarge                 # TiDB instance type
    count: 2                                  # Number of TiDB nodes to generate
  tikv:
    instance_type: m5.2xlarge                 # TiKV instance type
    count: 3                                  # Number of TiKV nodes to generate
    volumeSize: 80                            # Volume Size of the TiKV nodes
  dm:
    instance_type: t2.micro                   # DM instance type
    count: 1                                  # Number of DM node to generate
  ticdc:
    instance_type: m5.2xlarge                 # TiCDC instance type
    count: 1                                  # Number of TiCDC nodes to generate

Generate aws resources and deploy tidb cluster

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms deploy hackathon ~/workspace/hackathon/aws-tidb-simple.yaml
Please confirm your topology:
AWS Region:      Tokyo
Cluster type:    tidb
Cluster name:    hackathon
Cluster version: v5.1.0
User Name:       admin
Key Name:        jay

Component    # of nodes  Instance Type  Image Name             CIDR           User
---------    ----------  -------------  ----------             ----           ----
Workstation  1           m5.2xlarge     ami-0ac97798ccf296e02  172.82.0.0/16  admin
TiDB         2           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
PD           3           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
TiKV         3           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
TiCDC        1           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
DM           1           t2.micro       ami-0ac97798ccf296e02  172.83.0.0/16  master
Attention:
    1. If the topology is not what you expected, check your yaml file.
        2. Please confirm there is no port/directory conflicts in same host.
        Do you want to continue? [y/N]: (default=N) y
+ Deploying all the sub components for tidb2ms solution service
  - Preparing workstation ... ⠙ Echo: Creating VPC
+ Deploying all the sub components for tidb2ms solution service
+ Deploying all the sub components for tidb2ms solution service
  - Preparing workstation ... ⠇ Echo: Creating VPC
tidb : Creating Security Group ... ...  ... ⠼ Echo: Creating security group
+ Deploying all the sub components for tidb2ms solution service
+ Deploying all the sub components for tidb2ms solution service
  - Preparing workstation ... Done
  - Preparing tidb servers ... Done
+ Deploying tidb2ms solution service ... ...
  - Prepare DMS servicer and additional network resources :22 ... ⠏ Echo: Deploying TiCDC
The error here is <&errors.errorString{s:"No RDB Instance found(No matched name)"}>
+ Deploying tidb2ms solution service ... ...
  - Prepare DMS servicer and additional network resources :22 ... Done
Verbose debug logs has been written to /home/pi/.tiup/logs/aws-nodes-debug-2022-01-08-09-05-27.log.

AWS resources

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms list hackathon
+ Listing aws resources
  - Listing VPC ... Done
  - Listing Subnets ... Done
  - Listing Route Tables ... Done
  - Listing Security Groups ... Done
  - Listing Transit gateway  ... Done
  - Listing Transit gateway vpc attachment ... Done
  - Listing EC2 ... Done
  - Listing Load Balancer  ... Done
Cluster  Type:      tisample-tidb2ms
Cluster Name :      hackathon

Resource Type:      VPC
Component Name  VPC ID                 CIDR           Status
--------------  ------                 ----           ------
tidb            vpc-094a70a2eda96a0d9  172.83.0.0/16  available
workstation     vpc-0ebe1f7a15535f32e  172.82.0.0/16  available

Resource Type:      Subnet
Component Name  Zone             Subnet ID                 CIDR            State      VPC ID
--------------  ----             ---------                 ----            -----      ------
tidb            ap-northeast-1a  subnet-05303c850d3e395bb  172.83.1.0/24   available  vpc-094a70a2eda96a0d9
tidb            ap-northeast-1c  subnet-0b187ad5016ce5c2c  172.83.2.0/24   available  vpc-094a70a2eda96a0d9
tidb            ap-northeast-1d  subnet-0e2116872a10b9188  172.83.3.0/24   available  vpc-094a70a2eda96a0d9
workstation     ap-northeast-1a  subnet-0dc7f2b4054aba960  172.82.11.0/24  available  vpc-0ebe1f7a15535f32e

Resource Type:      Route Table
Component Name  Route Table ID         DestinationCidrBlock  TransitGatewayId       GatewayId              State   Origin
--------------  --------------         --------------------  ----------------       ---------              -----   ------
tidb            rtb-0df8cd282cfe3a6b5  172.82.0.0/16         tgw-0a93c20132e7c22c9                         active  CreateRoute
tidb            rtb-0df8cd282cfe3a6b5  172.83.0.0/16                                local                  active  CreateRouteTable
workstation     rtb-09277e69270c614e7  172.82.0.0/16                                local                  active  CreateRouteTable
workstation     rtb-09277e69270c614e7  172.83.0.0/16         tgw-0a93c20132e7c22c9                         active  CreateRoute
workstation     rtb-09277e69270c614e7  0.0.0.0/0                                    igw-0329ce2e0baad348e  active  CreateRoute

Resource Type:      Security Group
Component Name  Ip Protocol  Source Ip Range  From Port  To Port
--------------  -----------  ---------------  ---------  -------
workstation     tcp          172.82.0.0/16    0          65535
workstation     tcp          0.0.0.0/0        22         22
workstation     icmp         172.82.0.0/16    -1         -1
tidb            tcp          0.0.0.0/0        8300       8300
tidb            tcp          0.0.0.0/0        1433       1433
tidb            tcp          0.0.0.0/0        20180      20180
tidb            tcp          172.83.0.0/16    0          65535
tidb            tcp          0.0.0.0/0        22         22
tidb            tcp          0.0.0.0/0        4000       4000
tidb            tcp          0.0.0.0/0        2379       2379
tidb            tcp          0.0.0.0/0        20160      20160
tidb            tcp          0.0.0.0/0        3306       3306
tidb            tcp          0.0.0.0/0        2380       2380
tidb            icmp         172.83.0.0/16    -1         -1
tidb            tcp          0.0.0.0/0        10080      10080

Resource Type:      Transit Gateway
Resource ID  :      tgw-0a93c20132e7c22c9    State: available
Component Name  VPC ID                 State
--------------  ------                 -----
tidb            vpc-094a70a2eda96a0d9  available
workstation     vpc-0ebe1f7a15535f32e  available

Load Balancer:      hackathon-cf7959a3b0c7415c.elb.ap-northeast-1.amazonaws.com
Resource Type:      EC2
Component Name  Component Cluster  State    Instance ID          Instance Type  Preivate IP   Public IP      Image ID
--------------  -----------------  -----    -----------          -------------  -----------   ---------      --------
dm              tidb               running  i-07d62c419ca2b3fe3  t2.micro       172.83.1.66                  ami-0ac97798ccf296e02
pd              tidb               running  i-0c2d80f6333f6a77b  m5.2xlarge     172.83.2.21                  ami-0ac97798ccf296e02
pd              tidb               running  i-05a19af4d8bfdfb7a  m5.2xlarge     172.83.1.117                 ami-0ac97798ccf296e02
pd              tidb               running  i-040ce4539e62fa6a5  m5.2xlarge     172.83.3.9                   ami-0ac97798ccf296e02
ticdc           tidb               running  i-0bb01a195b1f3cb32  m5.2xlarge     172.83.1.149                 ami-0ac97798ccf296e02
tidb            tidb               running  i-06fa2ff5edf5cee70  m5.2xlarge     172.83.1.97                  ami-0ac97798ccf296e02
tidb            tidb               running  i-02e57d8abdcc8c66f  m5.2xlarge     172.83.2.197                 ami-0ac97798ccf296e02
tikv            tidb               running  i-0e24e56798270fda6  m5.2xlarge     172.83.2.59                  ami-0ac97798ccf296e02
tikv            tidb               running  i-0f501a53bd0a8451c  m5.2xlarge     172.83.3.247                 ami-0ac97798ccf296e02
tikv            tidb               running  i-0ebb8edf4cfae6c08  m5.2xlarge     172.83.1.109                 ami-0ac97798ccf296e02
workstation     workstation        running  i-0a0e4c9b7167c6fa8  m5.2xlarge     172.82.11.69  54.65.110.179  ami-0ac97798ccf296e02

TiDB cluster status

pi@c1s11:~/workspace/tisample $ ssh -i '~/.ssh/jaypingcap.pem' admin@54.65.110.179
admin@ip-172-82-11-69:~$ tiup cluster display hackathon
Starting component `cluster`: /home/admin/.tiup/components/cluster/v1.8.1/tiup-cluster display hackathon
Cluster type:       tidb
Cluster name:       hackathon
Cluster version:    v5.2.0
Deploy user:        admin
SSH type:           builtin
Dashboard URL:      http://172.83.1.117:2379/dashboard
ID                  Role  Host          Ports        OS/Arch       Status  Data Dir                               Deploy Dir
--                  ----  ----          -----        -------       ------  --------                               ----------
172.83.1.149:8300   cdc   172.83.1.149  8300         linux/x86_64  Up      /home/admin/tidb/tidb-data/cdc-8300    /home/admin/tidb/tidb-deploy/cdc-8300
172.83.1.117:2379   pd    172.83.1.117  2379/2380    linux/x86_64  Up|UI   /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.2.21:2379    pd    172.83.2.21   2379/2380    linux/x86_64  Up      /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.3.9:2379     pd    172.83.3.9    2379/2380    linux/x86_64  Up|L    /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.1.97:4000    tidb  172.83.1.97   4000/10080   linux/x86_64  Up      -                                      /home/admin/tidb/tidb-deploy/tidb-4000
172.83.2.197:4000   tidb  172.83.2.197  4000/10080   linux/x86_64  Up      -                                      /home/admin/tidb/tidb-deploy/tidb-4000
172.83.1.109:20160  tikv  172.83.1.109  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.2.59:20160   tikv  172.83.2.59   20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.3.247:20160  tikv  172.83.3.247  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
Total nodes: 9

扩容集群

能力

扩容集群时可以自动向 AWS 申请 EC2，并自动完成部署
缩容时可以自动停止或销毁 EC2，由用户选择是否直接销毁 EC2

User Interface

官方扩缩容设计见：使用 TiUP 扩容缩容 TiDB 集群扩容配置文件（scale-out.yaml）：

aws_topo_configs:
  general:
    imageid: ami-0ac97798ccf296e02            # Image ID for TiDB cluster's EC2 node
    keyname: jay.pingcap                      # key name to login from workstation to EC2 nodes
    cidr: 172.83.0.0/16                       # VPC cidr
    instance_type: m5.2xlarge                 # default instance type for EC2 nodes
    tidb_version: v5.2.0                      # TiDB version to deploy
  pd:
    instance_type: m5.2xlarge                 # PD instance type
    count: 3                                  # Number of PD nodes to generate
  tidb:
    instance_type: m5.2xlarge                 # TiDB instance type
    #Scale one TiDB node from 2 to 3
    count: 3                                  # Number of TiDB nodes to generate 
  tikv:
    instance_type: m5.2xlarge                 # TiKV instance type
    # Scale three TiKV nodes from 3 to 6
    count: 6                                  # Number of TiKV nodes to generate
    volumeSize: 80                            # Volume Size of the TiKV nodes
  dm:
    instance_type: t2.micro                   # DM instance type
    count: 1                                  # Number of DM node to generate
  ticdc:
    instance_type: m5.2xlarge                 # TiCDC instance type
    count: 1                                  # Number of TiCDC nodes to generate

命令参数直接 follow 官方设计： tiup cloud-cluster scale scale-out.yaml

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms scale hackathon ~/workspace/hackathon/scale-out.yaml
Please confirm your topology:
AWS Region:      Tokyo
Cluster type:    tidb
Cluster name:    hackathon
Cluster version: v5.1.0
User Name:       admin
Key Name:        jay

Component    # of nodes  Instance Type  Image Name             CIDR           User
---------    ----------  -------------  ----------             ----           ----
Workstation  1           m5.2xlarge     ami-0ac97798ccf296e02  172.82.0.0/16  admin
TiDB         3           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
PD           3           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
TiKV         6           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
TiCDC        1           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
DM           1           t2.micro       ami-0ac97798ccf296e02  172.83.0.0/16  master
Attention:
    1. If the topology is not what you expected, check your yaml file.
    2. Please confirm there is no port/directory conflicts in same host.
Do you want to continue? [y/N]: (default=N) y
+ Initialize target host environments
  - Preparing workstation ... ⠦ Echo: Creating VPC
+ Initialize target host environments
  - Preparing workstation ... ⠋ Echo: Creating VPC
+ Initialize target host environments
  - Preparing workstation ... Done
  - Preparing tidb servers ... Done
+ Initialize target host environments
  - Prepare Ec2  resources :22 ... Done
Cluster `hackathon` scaled successfully

AWS resources

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms list hackathon
+ Listing aws resources
  - Listing VPC ... Done
  - Listing Subnets ... Done
  - Listing Route Tables ... Done
  - Listing Security Groups ... Done
  - Listing Transit gateway  ... Done
  - Listing Transit gateway vpc attachment ... Done
  - Listing EC2 ... Done
  - Listing Load Balancer  ... Done
Cluster  Type:      tisample-tidb2ms
Cluster Name :      hackathon

Resource Type:      VPC
Component Name  VPC ID                 CIDR           Status
--------------  ------                 ----           ------
tidb            vpc-094a70a2eda96a0d9  172.83.0.0/16  available
workstation     vpc-0ebe1f7a15535f32e  172.82.0.0/16  available

... ... ...
Load Balancer:      hackathon-cf7959a3b0c7415c.elb.ap-northeast-1.amazonaws.com
Resource Type:      EC2
Component Name  Component Cluster  State    Instance ID          Instance Type  Preivate IP   Public IP      Image ID
--------------  -----------------  -----    -----------          -------------  -----------   ---------      --------
dm              tidb               running  i-07d62c419ca2b3fe3  t2.micro       172.83.1.66                  ami-0ac97798ccf296e02
pd              tidb               running  i-0c2d80f6333f6a77b  m5.2xlarge     172.83.2.21                  ami-0ac97798ccf296e02
pd              tidb               running  i-040ce4539e62fa6a5  m5.2xlarge     172.83.3.9                   ami-0ac97798ccf296e02
pd              tidb               running  i-05a19af4d8bfdfb7a  m5.2xlarge     172.83.1.117                 ami-0ac97798ccf296e02
ticdc           tidb               running  i-0bb01a195b1f3cb32  m5.2xlarge     172.83.1.149                 ami-0ac97798ccf296e02
tidb            tidb               running  i-02e57d8abdcc8c66f  m5.2xlarge     172.83.2.197                 ami-0ac97798ccf296e02
tidb            tidb               running  i-06fa2ff5edf5cee70  m5.2xlarge     172.83.1.97                  ami-0ac97798ccf296e02
tidb            tidb               running  i-06d7e53de2c2adba8  m5.2xlarge     172.83.3.8                   ami-0ac97798ccf296e02
tikv            tidb               running  i-0ebb8edf4cfae6c08  m5.2xlarge     172.83.1.109                 ami-0ac97798ccf296e02
tikv            tidb               running  i-0533d0ad572d4cedb  m5.2xlarge     172.83.2.102                 ami-0ac97798ccf296e02
tikv            tidb               running  i-04b91ddcc78d1fa23  m5.2xlarge     172.83.1.124                 ami-0ac97798ccf296e02
tikv            tidb               running  i-0f501a53bd0a8451c  m5.2xlarge     172.83.3.247                 ami-0ac97798ccf296e02
tikv            tidb               running  i-0e24e56798270fda6  m5.2xlarge     172.83.2.59                  ami-0ac97798ccf296e02
tikv            tidb               running  i-045aa3b3445a4aa20  m5.2xlarge     172.83.3.244                 ami-0ac97798ccf296e02
workstation     workstation        running  i-0a0e4c9b7167c6fa8  m5.2xlarge     172.82.11.69  54.65.110.179  ami-0ac97798ccf296e02

TiDB cluster status

admin@ip-172-82-11-69:~$ tiup cluster display hackathon
Starting component `cluster`: /home/admin/.tiup/components/cluster/v1.8.1/tiup-cluster display hackathon
Cluster type:       tidb
Cluster name:       hackathon
Cluster version:    v5.2.0
Deploy user:        admin
SSH type:           builtin
Dashboard URL:      http://172.83.1.117:2379/dashboard
ID                  Role  Host          Ports        OS/Arch       Status  Data Dir                               Deploy Dir
--                  ----  ----          -----        -------       ------  --------                               ----------
172.83.1.149:8300   cdc   172.83.1.149  8300         linux/x86_64  Up      /home/admin/tidb/tidb-data/cdc-8300    /home/admin/tidb/tidb-deploy/cdc-8300
172.83.1.117:2379   pd    172.83.1.117  2379/2380    linux/x86_64  Up|UI   /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.2.21:2379    pd    172.83.2.21   2379/2380    linux/x86_64  Up      /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.3.9:2379     pd    172.83.3.9    2379/2380    linux/x86_64  Up|L    /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.1.97:4000    tidb  172.83.1.97   4000/10080   linux/x86_64  Up      -                                      /home/admin/tidb/tidb-deploy/tidb-4000
172.83.2.197:4000   tidb  172.83.2.197  4000/10080   linux/x86_64  Up      -                                      /home/admin/tidb/tidb-deploy/tidb-4000
172.83.3.8:4000     tidb  172.83.3.8    4000/10080   linux/x86_64  Up      -                                      /home/admin/tidb/tidb-deploy/tidb-4000
172.83.1.109:20160  tikv  172.83.1.109  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.1.124:20160  tikv  172.83.1.124  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.2.102:20160  tikv  172.83.2.102  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.2.59:20160   tikv  172.83.2.59   20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.3.244:20160  tikv  172.83.3.244  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.3.247:20160  tikv  172.83.3.247  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
Total nodes: 13

缩容集群

User Interface(Same to 扩容集群命令)

官方扩缩容设计见：使用 TiUP 扩容缩容 TiDB 集群扩容配置文件（scale-in.yaml）：

aws_topo_configs:
  general:
    imageid: ami-0ac97798ccf296e02            # Image ID for TiDB cluster's EC2 node
    keyname: jay.pingcap                      # key name to login from workstation to EC2 nodes
    cidr: 172.83.0.0/16                       # VPC cidr
    instance_type: m5.2xlarge                 # default instance type for EC2 nodes
    tidb_version: v5.2.0                      # TiDB version to deploy
  pd:
    instance_type: m5.2xlarge                 # PD instance type
    count: 3                                  # Number of PD nodes to generate
  tidb:
    instance_type: m5.2xlarge                 # TiDB instance type
    #Scale one TiDB node from 3 to 2
    count: 2                                  # Number of TiDB nodes to generate 
  tikv:
    instance_type: m5.2xlarge                 # TiKV instance type
    # Scale three TiKV nodes from 6 to 3
    count: 3                                  # Number of TiKV nodes to generate
    volumeSize: 80                            # Volume Size of the TiKV nodes
  dm:
    instance_type: t2.micro                   # DM instance type
    count: 1                                  # Number of DM node to generate
  ticdc:
    instance_type: m5.2xlarge                 # TiCDC instance type
    count: 1                                  # Number of TiCDC nodes to generate

命令参数直接 follow 官方设计： tiup cloud-cluster scale scale-out.yaml

缩容(todo)： tiup cloud-cluster scale-in --node 10.0.1.5:20160 [--retain]

缩容指定节点 --retain 停止 EC2 而非销毁 EC2，以保留数据 Scale out cluster using below commands

Command Example

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms scale hackathon ~/workspace/hackathon/scale-in.yaml
Please confirm your topology:
AWS Region:      Tokyo
Cluster type:    tidb
Cluster name:    hackathon
Cluster version: v5.1.0
User Name:       admin
Key Name:        jay

Component    # of nodes  Instance Type  Image Name             CIDR           User
---------    ----------  -------------  ----------             ----           ----
Workstation  1           m5.2xlarge     ami-0ac97798ccf296e02  172.82.0.0/16  admin
TiDB         2           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
PD           3           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
TiKV         3           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
TiCDC        1           m5.2xlarge     ami-0ac97798ccf296e02  172.83.0.0/16  master
DM           1           t2.micro       ami-0ac97798ccf296e02  172.83.0.0/16  master
Attention:
    1. If the topology is not what you expected, check your yaml file.
    2. Please confirm there is no port/directory conflicts in same host.
Do you want to continue? [y/N]: (default=N) y
+ Initialize target host environments
  - Preparing workstation ... ⠦ Echo: Creating VPC 
+ Initialize target host environments
  - Preparing workstation ... ⠹ Echo: Creating VPC 
+ Initialize target host environments
  - Preparing workstation ... Done
  - Preparing tidb servers ... Done
+ Initialize target host environments
  - Prepare Ec2  resources :22 ... Done
Cluster `hackathon` scaled successfully

AWS resources

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms list hackathon
... ... ...
Load Balancer:      hackathon-cf7959a3b0c7415c.elb.ap-northeast-1.amazonaws.com
Resource Type:      EC2
Component Name  Component Cluster  State    Instance ID          Instance Type  Preivate IP   Public IP      Image ID
--------------  -----------------  -----    -----------          -------------  -----------   ---------      --------
dm              tidb               running  i-07d62c419ca2b3fe3  t2.micro       172.83.1.66                  ami-0ac97798ccf296e02
pd              tidb               running  i-05a19af4d8bfdfb7a  m5.2xlarge     172.83.1.117                 ami-0ac97798ccf296e02
pd              tidb               running  i-040ce4539e62fa6a5  m5.2xlarge     172.83.3.9                   ami-0ac97798ccf296e02
pd              tidb               running  i-0c2d80f6333f6a77b  m5.2xlarge     172.83.2.21                  ami-0ac97798ccf296e02
ticdc           tidb               running  i-0bb01a195b1f3cb32  m5.2xlarge     172.83.1.149                 ami-0ac97798ccf296e02
tidb            tidb               running  i-02e57d8abdcc8c66f  m5.2xlarge     172.83.2.197                 ami-0ac97798ccf296e02
tidb            tidb               running  i-06d7e53de2c2adba8  m5.2xlarge     172.83.3.8                   ami-0ac97798ccf296e02
tikv            tidb               running  i-0e24e56798270fda6  m5.2xlarge     172.83.2.59                  ami-0ac97798ccf296e02
tikv            tidb               running  i-0f501a53bd0a8451c  m5.2xlarge     172.83.3.247                 ami-0ac97798ccf296e02
tikv            tidb               running  i-0ebb8edf4cfae6c08  m5.2xlarge     172.83.1.109                 ami-0ac97798ccf296e02
workstation     workstation        running  i-0a0e4c9b7167c6fa8  m5.2xlarge     172.82.11.69  54.65.110.179  ami-0ac97798ccf296e02

TiDB Cluster status

admin@ip-172-82-11-69:~$ tiup cluster display hackathon 
Starting component `cluster`: /home/admin/.tiup/components/cluster/v1.8.1/tiup-cluster display hackathon
Cluster type:       tidb
Cluster name:       hackathon
Cluster version:    v5.2.0
Deploy user:        admin
SSH type:           builtin
Dashboard URL:      http://172.83.1.117:2379/dashboard
ID                  Role  Host          Ports        OS/Arch       Status  Data Dir                               Deploy Dir
--                  ----  ----          -----        -------       ------  --------                               ----------
172.83.1.149:8300   cdc   172.83.1.149  8300         linux/x86_64  Up      /home/admin/tidb/tidb-data/cdc-8300    /home/admin/tidb/tidb-deploy/cdc-8300
172.83.1.117:2379   pd    172.83.1.117  2379/2380    linux/x86_64  Up|UI   /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.2.21:2379    pd    172.83.2.21   2379/2380    linux/x86_64  Up      /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.3.9:2379     pd    172.83.3.9    2379/2380    linux/x86_64  Up|L    /home/admin/tidb/tidb-data/pd-2379     /home/admin/tidb/tidb-deploy/pd-2379
172.83.2.197:4000   tidb  172.83.2.197  4000/10080   linux/x86_64  Up      -                                      /home/admin/tidb/tidb-deploy/tidb-4000
172.83.3.8:4000     tidb  172.83.3.8    4000/10080   linux/x86_64  Up      -                                      /home/admin/tidb/tidb-deploy/tidb-4000
172.83.1.109:20160  tikv  172.83.1.109  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.2.59:20160   tikv  172.83.2.59   20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
172.83.3.247:20160  tikv  172.83.3.247  20160/20180  linux/x86_64  Up      /home/admin/tidb/tidb-data/tikv-20160  /home/admin/tidb/tidb-deploy/tikv-20160
Total nodes: 9

销毁集群

能力

自动停止或销毁集群所使用的所有 EC2，由用户选择是否直接销毁 EC2

User Interface

➜  ~ tiup cloud-cluster destroy --help
Usage:
  tiup-cluster destroy <cluster-name> [flags]

Flags:
      --force                          Force will ignore remote error while destroy the cluster
  -h, --help                           help for destroy
      --retain-node-data stringArray   Specify the nodes or hosts whose data will be retained，指定需要保留的节点（停止 EC2 而非销毁 EC2）
      --retain-role-data stringArray   Specify the roles whose data will be retained，指定需要保留的节点（停止 EC2 而非销毁 EC2）

Command Example

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms destroy hackathon                                    
                                                   
+ Destroying tidb2ms solution service ... ...                                                         
+ Destroying tidb2ms solution service ... ...  
... ...
+ Destroying all the componets
  - Destroying EC2 nodes cluster hackathon  ... Done
  - Destroying aurora db cluster hackathon  ... Done
  - Destroying sqlserver cluster hackathon  ... Done
  - Destroying workstation cluster hackathon  ... Done

AWS resources

pi@c1s11:~/workspace/tisample $ ./bin/aws tidb2ms list hackathon
+ Listing aws resources
  - Listing VPC ... Done
  - Listing Subnets ... Done
  - Listing Route Tables ... Done
  - Listing Security Groups ... Done
  - Listing Transit gateway  ... Done
  - Listing Transit gateway vpc attachment ... Done
  - Listing EC2 ... Done
  - Listing Load Balancer  ... Done
Cluster  Type:      tisample-tidb2ms
Cluster Name :      hackathon

Resource Type:      VPC
Component Name  VPC ID  CIDR  Status
--------------  ------  ----  ------

Resource Type:      Subnet
Component Name  Zone  Subnet ID  CIDR  State  VPC ID
--------------  ----  ---------  ----  -----  ------

Resource Type:      Route Table
Component Name  Route Table ID  DestinationCidrBlock  TransitGatewayId  GatewayId  State  Origin
--------------  --------------  --------------------  ----------------  ---------  -----  ------

Resource Type:      Security Group
Component Name  Ip Protocol  Source Ip Range  From Port  To Port
--------------  -----------  ---------------  ---------  -------

Resource Type:      Transit Gateway
Resource ID  :          State:  
Component Name  VPC ID  State
--------------  ------  -----

Load Balancer:      
Resource Type:      EC2
Component Name  Component Cluster  State  Instance ID  Instance Type  Preivate IP  Public IP  Image ID
--------------  -----------------  -----  -----------  -------------  -----------  ---------  --------

Reference

Documentation ¶

There is no documentation for this package.

Source Files ¶

View all Source files

main.go

Directories ¶

Path	Synopsis
cmd
components
aws
aws/command
candle
candle/command
cluster
cluster/command
ctl
dm
dm/ansible
dm/command
dm/spec
dm/task
doc
errdoc
errdoc/errdoc-gen
errdoc/spec
playground
playground/instance
workstation
workstation/command
embed
pkg
aws/ansible
aws/api
aws/api/dmpb Package pb is a reverse proxy.	Package pb is a reverse proxy.
aws/api/typeutil
aws/audit
aws/clusterutil
aws/manager
aws/module
aws/operation
aws/spec
aws/task
aws/template
aws/template/config
aws/template/install
aws/template/scripts
aws/template/systemd
base52
candle/ansible
candle/api
candle/api/dmpb Package pb is a reverse proxy.	Package pb is a reverse proxy.
candle/api/typeutil
candle/audit
candle/clusterutil
candle/ctxt
candle/executor
candle/manager
candle/module
candle/operation
candle/spec
candle/task
candle/template
candle/template/config
candle/template/install
candle/template/scripts
candle/template/systemd
checkpoint
cluster/ansible
cluster/api
cluster/api/dmpb Package pb is a reverse proxy.	Package pb is a reverse proxy.
cluster/api/typeutil
cluster/audit
cluster/clusterutil
cluster/ctxt
cluster/executor
cluster/manager
cluster/module
cluster/operation
cluster/spec
cluster/task
cluster/template
cluster/template/config
cluster/template/install
cluster/template/scripts
cluster/template/systemd
crypto
crypto/rand
ctxt
environment
exec
executor
localdata
logger
logger/log
meta
proxy
queue
repository
repository/model
repository/store
repository/utils
repository/v1manifest
set
telemetry
tui
tui/progress
utils
utils/mock
version
workstation/ansible
workstation/api
workstation/api/dmpb Package pb is a reverse proxy.	Package pb is a reverse proxy.
workstation/api/typeutil
workstation/audit
workstation/clusterutil
workstation/ctxt
workstation/executor
workstation/manager
workstation/module
workstation/operation
workstation/spec
workstation/task
workstation/template
workstation/template/config
workstation/template/install
workstation/template/scripts
workstation/template/systemd
server
handler
package
rotate
session

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL