A task is simply a collection of all the instantiated activities of an application.
For example:
If we have Application A with activities A1, A2 and A3 and A2 is opened after A1 and A3 is opened after A2, the task for Application A would look like this:
|A3|
|A2|
|A1|
Now if we press the back button, A3 will get popped off and A2 will be revealed to the user. The task for Application A would now look like this:
|A2|
|A1|
If we press back till all the activities are gone, then the task will be discarded and the next time we start Application A, Android will create a new task with Application A's main activity as the first activity.
Now, Let's open A1, A2 and A3 again in the same order. Application A's task is now back to:
|A3|
|A2|
|A1|
Now, suppose we press the home button and launch another app, Application B. This will cause Application A's entire task to be retained in the background and a new task is created for Application B with it's main activity instantiated. So now we have the situation that looks like this:
Application A Application B
|A3| |B1|
|A2|
|A1|
If we open more activities in Application B, they will be added to it's task just like Application A:
Application A Application B
|A3| |B3|
|A2| |B2|
|A1| |B1|
Now, if we switch back to Application A, we will bring A's task to the foreground and B's task will be retained in the background.
Multiple instances of the same activity can also exist on the same task. This behavior can be controlled.
If the system runs out of memory, it will start killing activities in the background. If all the activities of a task are gone, the task will also be destroyed. (UPDATE: According to this answer by Dianne Hackborn, it is not the individual activities but the entire process that hosts them that gets discarded. The docs may be a bit misleading in this regard and the confusion is yet to be resolved. I'll update it when I get more solid info.)
So, to sum up, a task is simply a collection of activities of an application. It is used to maintain a "stack" or "backstack" of all the instantiated activities of an app. It gets retained in the background when the all the activities of an application are in the background. When one of these activities is brought back to the foreground, the task is brought back as well and the task of the current activity is pushed to the background. If the system needs memory, background activities and tasks can be destroyed.
The official docs give a lot more info and I recommend reading them:
http://developer.android.com/guide/components/tasks-and-back-stack.html