I tried to reproduce the error and I modified your template for my tests. The template modified is below, so you have to check it if you want. Also you would have to adjust it to your setup if you want to use it. For simplicity I put everything in public subnets, but I don't think it is the key issue here.
I think the core issue is that you are not setting up aws-auth-cm.yaml
which prohibits the node instances to register with the cluster.
To setup aws-auth-cm.yaml
please have a look at To enable nodes to join your cluster section. Once I did it, I could see my node in kubectl get nodes
when using the template below.
I have kubectl
running on my local workstation, not on the bastion host that you create. I only tested nodes joining cluster, not the functionality of any pods. Also EKS AWS console does not show the nodes, but kubectl get nodes
shows them.
AWSTemplateFormatVersion: "2010-09-09"
Description: Creates API gateway and services for my projects
Parameters:
ClusterName:
Type: String
Description: Cluster name
Default: eks-min-cluster
NodeAutoScalingGroupDesiredCapacity:
Type: Number
Default: 1
Description: Desired capacity of Node Group ASG.
NodeAutoScalingGroupMinSize:
Type: Number
Default: 1
Description: Minimum size of Node Group ASG.
KeyPair:
Type: AWS::EC2::KeyPair::KeyName
Default: jd-system
NodeAutoScalingGroupMaxSize:
Type: Number
Default: 3
Description: Maximum size of Node Group ASG. Set to at least 1 greater than NodeAutoScalingGroupDesiredCapacity.
BootstrapArguments:
Type: String
Default: ""
Description: "Arguments to pass to the nodes' bootstrap script. See files/bootstrap.sh in https://github.com/awslabs/amazon-eks-ami"
VpcCidr:
Description: Please enter the IP range (CIDR notation) for this VPC
Type: String
Default: 10.192.0.0/16
PublicSubnet1Cidr:
Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone
Type: String
Default: 10.192.20.0/24
PublicSubnet2Cidr:
Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone
Type: String
Default: 10.192.21.0/24
PrivateSubnet1Cidr:
Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone
Type: String
Default: 10.192.22.0/24
PrivateSubnet2Cidr:
Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone
Type: String
Default: 10.192.23.0/24
NodeImageIdSSMParam:
Type: "AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>"
Default: /aws/service/eks/optimized-ami/1.18/amazon-linux-2/recommended/image_id
Description: AWS Systems Manager Parameter Store parameter of the AMI ID for the worker node instances.
LatestAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
InternetGateway:
Type: AWS::EC2::InternetGateway
Vpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VpcCidr
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: !Sub "kubernetes.io/cluster/${ClusterName}"
Value: shared
- Key: Name
Value: MyEksVpc
VpcGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref Vpc
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
AvailabilityZone: !Select [ 0, !GetAZs '' ]
CidrBlock: !Ref PublicSubnet1Cidr
MapPublicIpOnLaunch: true
Tags:
- Key: kubernetes.io/role/elb
Value: 1
Tags:
- Key: !Sub "kubernetes.io/cluster/${ClusterName}"
Value: shared
PublicSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
AvailabilityZone: !Select [ 1, !GetAZs '' ]
CidrBlock: !Ref PublicSubnet2Cidr
MapPublicIpOnLaunch: true
Tags:
- Key: kubernetes.io/role/elb
Value: 1
- Key: !Sub "kubernetes.io/cluster/${ClusterName}"
Value: shared
PrivateSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
AvailabilityZone: !Select [ 0, !GetAZs '' ]
CidrBlock: !Ref PrivateSubnet1Cidr
MapPublicIpOnLaunch: true
Tags:
- Key: kubernetes.io/role/internal-elb
Value: 1
PrivateSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
AvailabilityZone: !Select [ 1, !GetAZs '' ]
CidrBlock: !Ref PrivateSubnet2Cidr
MapPublicIpOnLaunch: true
Tags:
- Key: kubernetes.io/role/internal-elb
Value: 1
SshSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref Vpc
GroupDescription: Enable SSH access via port 22
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
FromPort: 22
IpProtocol: tcp
ToPort: 22
- CidrIp: 0.0.0.0/0
FromPort: 8
IpProtocol: icmp
ToPort: -1
GatewayHostSshPortAddress:
Type: AWS::EC2::EIP
DependsOn: VpcGatewayAttachment
Properties:
Domain: vpc
AssociateGatewayHostSshPort:
Type: AWS::EC2::EIPAssociation
DependsOn: GatewayHostSshPortAddress
Properties:
AllocationId: !GetAtt GatewayHostSshPortAddress.AllocationId
NetworkInterfaceId: !Ref GatewayHostSshNetworkInterface
GatewayHostSshNetworkInterface:
Type: AWS::EC2::NetworkInterface
Properties:
SubnetId: !Ref PublicSubnet1
Description: Interface for controlling traffic such as SSH
GroupSet:
- !Ref SshSecurityGroup
SourceDestCheck: true
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref Vpc
PrivateSubnet1RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref Vpc
PrivateSubnet2RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref Vpc
DefaultPublicRoute:
Type: AWS::EC2::Route
#DependsOn: VpcGatewayAttachment
Properties:
RouteTableId: !Ref PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
PrivateSubnet1Route:
Type: AWS::EC2::Route
#DependsOn:
# - VpcGatewayAttachment
# - PrivateSubnet1NatGateway
Properties:
RouteTableId: !Ref PrivateSubnet1RouteTable
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref PrivateSubnet1NatGateway
PrivateSubnet2Route:
Type: AWS::EC2::Route
#DependsOn:
# - VpcGatewayAttachment
# - PrivateSubnet2NatGateway
Properties:
RouteTableId: !Ref PrivateSubnet2RouteTable
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref PrivateSubnet2NatGateway
PrivateSubnet1NatGateway:
Type: AWS::EC2::NatGateway
#DependsOn:
# - PrivateSubnet1NatGatewayEIP
#- PublicSubnet1
#- VpcGatewayAttachment
Properties:
AllocationId: !GetAtt PrivateSubnet1NatGatewayEIP.AllocationId
SubnetId: !Ref PublicSubnet1
PrivateSubnet2NatGateway:
Type: AWS::EC2::NatGateway
# DependsOn:
# - PrivateSubnet2NatGatewayEIP
# - PublicSubnet2
# - VpcGatewayAttachment
Properties:
AllocationId: !GetAtt PrivateSubnet2NatGatewayEIP.AllocationId
SubnetId: !Ref PublicSubnet2
PrivateSubnet1NatGatewayEIP:
DependsOn:
- VpcGatewayAttachment
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
PrivateSubnet2NatGatewayEIP:
DependsOn:
- VpcGatewayAttachment
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
PublicRouteTableToPublicSubnet1Association:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnet1
PublicRouteTableToPublicSubnet2Association:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnet2
PrivateRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref Vpc
# GatewayHost:
# Type: AWS::EC2::Instance
# DependsOn: [AssociateGatewayHostSshPort]
# Properties:
# ImageId: !Ref LatestAmiId
# InstanceType: t2.nano
# AvailabilityZone: !Select [ 0, !GetAZs '' ]
# KeyName: !Ref KeyPair
# NetworkInterfaces:
# -
# NetworkInterfaceId: !Ref GatewayHostSshNetworkInterface
# DeviceIndex: 0
# # Metadata:
# # AWS::CloudFormation::Init:
# # config:
# # files:
# # /etc/kong/kong.yml:
# # content: test-jd
# # #source:
# # mode: "000644"
# # owner: "root"
# # group: "root"
EksIamRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- eks.amazonaws.com
Action:
- 'sts:AssumeRole'
#RoleName: EksIamRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
- arn:aws:iam::aws:policy/AmazonEKSServicePolicy
################### CONTROL PLANE ###################
ClusterControlPlaneSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Cluster communication with worker nodes
VpcId: !Ref Vpc
EksCluster:
Type: AWS::EKS::Cluster
Properties:
Name: !Ref ClusterName
RoleArn: !GetAtt EksIamRole.Arn
ResourcesVpcConfig:
SecurityGroupIds:
#- !Ref SshSecurityGroup
- !Ref ClusterControlPlaneSecurityGroup
SubnetIds:
- !Ref PublicSubnet1
- !Ref PublicSubnet2
#- !Ref PrivateSubnet1
#- !Ref PrivateSubnet2
#DependsOn: [EksIamRole, PublicSubnet1, PublicSubnet2, PrivateSubnet1, PrivateSubnet2, SshSecurityGroup]
################### WORKER NODES ###################
NodeSecurityGroup:
Type: "AWS::EC2::SecurityGroup"
Properties:
GroupDescription: Security group for all nodes in the cluster
Tags:
- Key: !Sub kubernetes.io/cluster/${ClusterName}
Value: owned
VpcId: !Ref Vpc
NodeSecurityGroupIngress:
Type: "AWS::EC2::SecurityGroupIngress"
DependsOn: NodeSecurityGroup
Properties:
Description: Allow node to communicate with each other
FromPort: 0
GroupId: !Ref NodeSecurityGroup
IpProtocol: "-1"
SourceSecurityGroupId: !Ref NodeSecurityGroup
ToPort: 65535
ClusterControlPlaneSecurityGroupIngress:
Type: "AWS::EC2::SecurityGroupIngress"
#DependsOn: NodeSecurityGroup
Properties:
Description: Allow pods to communicate with the cluster API Server
FromPort: 443
GroupId: !Ref ClusterControlPlaneSecurityGroup
IpProtocol: tcp
Sourc
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…