amazon web services - Is there a way to specify a task placement strategy such that a new task is placed on the instance having most available memory?

Question

I have multiple EC2 instances reserved in the ECS cluster. Currently, the tasks are placed by spread. This sometimes causes an issue when certain instances have very little memory remaining, while the other instances have a lot of it.

Is there a way to create a placement strategy to place new tasks on the instance with the most available memory?

question from:https://stackoverflow.com/questions/65680345/is-there-a-way-to-specify-a-task-placement-strategy-such-that-a-new-task-is-plac

Answer 1

According to the documentation you can give the scheduler three kinds of strategies:

Amazon ECS supports the following task placement strategies:

binpack Tasks are placed on container instances so as to leave the least amount of unused CPU or memory. This strategy minimizes the number of container instances in use.

When this strategy is used and a scale-in action is taken, Amazon ECS will terminate tasks based on the amount of resources that will be left on the container instance after the task is terminated. The container instance that will have the most available resources left after task termination will have that task terminated.

random Tasks are placed randomly.

spread Tasks are placed evenly based on the specified value. Accepted values are instanceId (or host, which has the same effect), or any platform or custom attribute that is applied to a container instance, such as attribute:ecs.availability-zone. Service tasks are spread based on the tasks from that service. Standalone tasks are spread based on the tasks from the same task group.

When this strategy is used and a scale-in action is taken, Amazon ECS will select tasks to terminate that maintains a balance across Availability Zones. Within an Availability Zone, tasks will be selected at random.

What you seem to need seems like the opposite of binpack, so you might need to create a custom scheduler.

Alternatively you could increase the required memory in your task definitions with the memoryReservation parameter. That way your containers should get as much memory as they need for their operations.