4.3 Distributed computingWhen one computer is not enough

Distributed computing is a computational model in which multiple devices are used to run a program. The devices are separate computers connected by a network, and each one does part of the work. Distributed computing allows problems to be solved that could not be solved on a single computer, because of either the processing time or the storage needed. It also lets much larger problems be solved quicker than they could be on one computer.
?Wait, how is that different from parallel computing?
| Problem | Why one computer is not enough | How it is split |
|---|---|---|
| Weather and climate forecasts | Too many calculations to finish before the forecast is needed | Each machine simulates one region of the map |
| A search engine's index of the Web | Far too much data to store on one machine | Each machine stores and searches part of the index |
| Volunteer science projects | Years of computing time on one machine | Home computers each analyze one small slice of data |
| Rendering an animated movie | Hours per frame, over 100,000 frames | Each machine renders different frames |
The same timing ideas apply, with one more cost: sending the data out and collecting the results takes time, and that part doesn't shrink when you add machines. Distributed systems are also a fault tolerance story: if one machine in a thousand fails, its slice can be handed to another machine, a kind of redundancy.
I thought about it. Show me
- (a) 1,000,000 × 0.5 = 500,000 seconds. That is about 5.8 days.
Why?
500,000 ÷ 86,400 seconds in a day ≈ 5.79. - (b) Each computer gets 1,000,000 ÷ 200 = 5,000 images, which take 5,000 × 0.5 = 2,500 seconds.
- All 200 run at the same time, so the parallel part is 2,500 seconds. Add the 300 seconds of sending and collecting: 2,800 seconds, under 47 minutes.
- (c) Speedup = 500,000 ÷ 2,800 ≈ 178.6.
Why?
Not 200, because of the 300 seconds that every run must pay. The more computers you add, the more that fixed cost matters.
Answer: (a) 500,000 seconds (about 5.8 days). (b) 2,800 seconds. (c) A speedup of about 179.
Stuck? Get a hint
- Start with the one-computer time: images × seconds per image.
- Each of the 200 computers does 5,000 images, all at the same time.
- Parallel time = 300 + the time for one computer's share.
- AA calculator app adding two numbers
- BA student sorting a list of 30 names on a laptop
- CThousands of servers each storing and searching part of an index too large for any one computer
- DA program that runs one instruction at a time on one processor
Show the answer
C. Distributed computing is for problems too big for one machine, in processing time or storage. The fourth choice describes sequential computing, and the first two are tiny jobs one device handles easily.
- Distributed computing: several separate devices on a network each run part of a program.
- It solves problems too large for one computer in processing time or storage, and solves big problems faster.
- Sending and combining data adds sequential time that limits the speedup.