All posts

How Download Managers Speed Up Downloads with Parallel HTTP Requests.

A download manager does not download the file four times. It asks for four different slices, fetches them together, then stacks the slices back in order.

HTTPNode.jsTypeScript

A normal download is one request. The server starts at the beginning of the file and keeps sending until the end.

A download manager opens several requests for the same URL. That only works because each request asks for a different stretch of bytes. Think of a book split across four people: each person copies different pages, then you stack the pages from first to last.

One file, several slices

A byte is one unit of the file. In a 1,000-byte file, the first byte sits at position 0 and the last sits at position 999. Four requests can share that file like this:

  1. 1MeasureLearn how big the file is
  2. 2SplitGive each request its own range
  3. 3FetchDownload the ranges together
  4. 4JoinWrite the slices in byte order

largefile.zip · one URL · 1,000 bytes

Part 00–249
Part 1250–499
Part 2500–749
Part 3750–999
byte 0byte 999
Four requests cover the file once. Each slice is 250 bytes, and the next slice starts where the previous one stopped.

You do not need to understand a ZIP file or a video to do this. You are cutting the raw bytes, not the format. One slice may not open on its own. The joined file is the thing that should open.

The samples below follow my TypeScript downloader gist. Node.js starts the requests together on its event loop. It does not need a worker thread per slice. Node.js describes that I/O model.

Ask the server for specific bytes

A normal GET says “send the file.” Add one header and it says “send only this part.”

You send

GET /largefile.zip

Range: bytes=250-499

One slice from the middle of the file.

Server replies

206 Partial Content

bytes 250-499/1000

250 bytes, taken from a 1,000-byte file.

The server sees an ordinary GET with a Range header. There is no special “parallel download” command.

Both ends of the range are included. bytes=0-2 is three bytes, not two. You get the length by counting every position from the start through the end.

Range: bytes=0-2

The highlighted cells are the response. 2 − 0 + 1 = 3 bytes.

206 Partial Content means you received a slice. Content-Range: bytes 250-499/1000 tells you where it belongs, and that the whole file is 1,000 bytes. Content-Length: 250 is the size of this response, not the size of the file. MDN documents the Range header.

Read the status before you save anything. 200 OK means the server ignored the range and sent the whole file. 416 Range Not Satisfiable means that slice does not exist. Neither one is a successful part.

You need the size before you can plan the slices. A HEAD request returns headers, including Content-Length, and skips the body. You can also ask for the first byte with Range: bytes=0-0. A useful answer looks like Content-Range: bytes 0-0/1000: ranges work, and the file is 1,000 bytes. MDN shows both checks.

Leave no gaps and no overlaps

Divide the size by the number of pieces, then walk from the start. Each new slice begins on the byte after the previous slice ends.

JavaScript · Plan the ranges
function planSegments(fileSize, pieces) {
  const segmentSize = Math.ceil(fileSize / pieces);
  const segments = [];

  for (let start = 0; start < fileSize; start += segmentSize) {
    segments.push({
      start,
      end: Math.min(start + segmentSize - 1, fileSize - 1),
    });
  }

  return segments;
}

planSegments(1000, 4) returns the four equal slices in the diagram above. A 5-byte file does not divide into four useful pieces. The same function stops at the end of the file:

5 bytes, asked for 4 pieces

0–12 bytes
2–32 bytes
41 byte
Three requests cover bytes 0 through 4 once. A fourth request would start past the end of the file.

The rule is small: start at byte 0, never skip a byte, never repeat a byte, and stop at the last byte.

Start every request, then wait

Waiting for one slice before starting the next is a normal download with extra steps. Start them all, then wait for the group:

JavaScript · Keep the planned order
const segments = planSegments(fileSize, 4);
const parts = await Promise.all(
  segments.map((segment) => downloadPart(url, segment))
);

The requests begin when downloadPart is called. Promise.all only waits. Its array stays in plan order even when part 3 finishes first. MDN documents that order.

One after another

Part 0
Part 1
Part 2
Part 3

Done when the last wait ends

At the same time

Part 0
Part 1
Part 2
Part 3

Done when the slowest part ends

The colored bars are download time. Sequential parts wait their turn. Concurrent parts overlap, so the clock stops with the slowest one.

Each request should accept only 206, and the saved slice should contain every byte in that range and nothing outside it. If one request fails, cancel the others. Promise.all will reject, but it will not stop the requests that are still running.

Join the slices in file order

Parts can finish in any order. Joining them in that order scrambles the file, even when every byte arrived.

Order they finished

  1. 1stPart 2bytes 500–749
  2. 2ndPart 0bytes 0–249
  3. 3rdPart 3bytes 750–999
  4. 4thPart 1bytes 250–499

Saving this order breaks the file

Order to save them

  1. 0Part 0bytes 0–249
  2. 1Part 1bytes 250–499
  3. 2Part 2bytes 500–749
  4. 3Part 3bytes 750–999

Saving this order rebuilds the file

Finish order can change from download to download. Save order is always part 0, part 1, part 2, part 3.

Promise.all already returns the slices in plan order, so you can write them from the start of that array to the end. Check that the finished file is 1,000 bytes. If the publisher gives you a checksum, compare that too. Write to a temporary file first, and rename it only after those checks pass.

One more mismatch is easy to miss. The file on the server can change while your requests are in flight. A slice from the old file and a slice from the new file can still add up to the right length. If the first response includes a strong ETag, send it back with If-Range so every slice is tied to that same version. MDN explains If-Range.

More requests are not always faster

Parallel downloads help when one connection is not allowed to use your full internet speed. Picture a server that limits each connection to 2 MB/s, while your connection can receive 10 MB/s. A 100 MB file takes about 50 seconds on one connection. Four connections can use about 8 MB/s and finish in about 12.5 seconds.

1 connection 50s
4 connections 12.5s
An example, not a measurement. Each connection is capped at 2 MB/s, and the link can carry 10 MB/s. Four connections use 8 MB/s.

If one connection already fills the link, four requests just share that same speed. They cannot turn 10 MB/s into 40 MB/s. Try one request, then two, then four, on the server you actually use.

Look at one slice with curl

Use a file URL you are allowed to download. This asks for the first byte and prints the response headers:

Shell · Ask for byte 0
curl --location --max-time 15 \
  --header 'Range: bytes=0-0' \
  --header 'Accept-Encoding: identity' \
  --dump-header - --output probe.bin \
  'https://downloads.your-domain.test/largefile.zip'

A working range server answers 206, returns a matching Content-Range, and leaves probe.bin as a one-byte file. A 200 means this server did not honor the range. Accept-Encoding: identity asks for the raw bytes, so compression does not shift the positions you planned.

After that, three checks decide whether the download is real:

  • Coverage. Every byte appears in exactly one slice.
  • The same file. Every slice belongs to the same version.
  • Order. You join the slices by byte position, not by which request finished first.

Concurrency is what creates the chance of a faster download. Those three checks are what make the file correct.