Systems Manager Corner

Queueing - The Checkout Line

Articles 11/96

If a module is nearing 100% busy, response time will certainly slow down. However, some processes will slow more than others. This slowdown is a result of the processes waiting for a CPU to be freed up before the process is assigned to a CPU. This is a process's CPU wait time, and it is incurred each time a process is rescheduled. If a process is rescheduled 20 times to do a particular increment of work, it could take substantially longer to do that work than another process which is only rescheduled twice.

An example: Two people walk into a grocery store, each intending to purchase two items. One of the people selects the two items, goes through the checkout line (including the wait to get to the cashier), and pays, and is done. The second person selects the first item, goes through the checkout line and pays, and then goes back into the store to select the second item, and again goes to the checkout line and pays. They both did the same amount of "work," paying for two items. But one person did it with only one trip through the CPU (checkout line), while the other person took two trips, incurring the extra wait time.

As the module gets busier, it takes longer to perform a chunk of work. This chart shows the wait plus service time for a uniprocessor.

The Moral: Design your application to do as much work in as few requests as possible. For example, make one large request to a server from a client, instead of several small requests.

The penalty for queueing is paid at each step of the transaction: Waiting for a shared communications line, waiting for CPU, waiting for disk, waiting for a server.