[Operating System] {ud923} P2L5: Thread Performance Considerations

"Flash: An Efficient and Portable Web Server" by Pai

Which Threading Model is Better?

ERRATA

avg. time to complete order for boss worker should be: ((5*120) + (5*240) + 360) / 11

the quizzes refer to case with 6 threads total, in the boss worker model that means 1 boss and 5 workers.

Are Threads Useful?

Visual Metaphor

Performance Metrics Intro

Performance Metrics

Performance Metrics Summary

Really… Are Threads Useful?

ERRATA

Computer Header => Compute Header

Multi Threaded Web Server

Event-Driven Model

Paper

"Flash: An Efficient and Portable Web Server" by Pai

Figures

Event-Driven Model: Why

Event-Driven Model: How

sockets and files share the same data structure, called "file descriptior"

※ Select():

to determine which file descriptor has input, so to determine there's an event arriving in the system, the flash paper talks about using "Select()" call.

return the very first one that has some kind of input on it.

※ Poll():

The common drawback of the two APIs is that if there's a really large list of file descriptors, OS has to scan through them.

Likely there only a few to have inputs.

Lots of search time will be wasted.

※ epoll():

alternative API, eliminating some of the problems that select and poll have.

so it's used today.

Helper Threads and Processes

One thing that makes asynchronous calls possible is that the OS kernel is multithreaded.

So while the caller thread continues execution, another kernel thread does all the necessary work and all the waiting that's needed to perform the I/O operation, and then to also make sure the results become available to the appropriate user level context.

Also, asynchronous operations can benefit by the actual I/O devices.

We'll come back later. Now all you need to know about asynchronous operations is that:

our process will not be blocked in the kernel when performing I/O.

※ Asynchronos I/O calls weren't ubiquitously available in the past, and not available for all devices

※ In a general case, maybe the processing needs to be performed by our server, isn't reading a file where there's asynchronous call, instead maybe to call processing some accelerator or other devices only server have access to.

In AMPED, to support devices not supporting multi thread, the multi-process is used. and helpers only deal with block I/O operation, and the main one performs everything else.fl

In the other models, we had a separate thread for each of the requests or for each of pipeline stages.

In event driven model , we have handlers which are just procedures in that address space, and the helper threads only occur for blocking I/O operations.

Flash Web Server

an extra check before we read any file, it actually results in big savings because it prevents the full process from being blocked if it turns out that a blocking I/O operation is necessary.

cache:e eg. some pathname for the file (no need to recompute)

three caches here: 3 brown frames

Also, Flash does some optimizations that take advantage of the networking hardware and the network interface card.

It's easy to perform DMA operations without copying data.

DMA => Direct memory access??

Similarly, they use DMA operations that have scatter-gather support, and that really means that the header and the acutal data dont have to be alighed one next to another in memory.

So there's a copy that's avoided.

Apache Web Server