# HTTP and Networking
Course: [Node.js](https://www.devtutorials.ai/learn/nodejs) · Section: Networking and Parallelism
Source: https://www.devtutorials.ai/learn/nodejs/network-parallel/http-and-networking
> Sockets and streams, how net, http and tls relate, building an HTTP server with http.createServer, IncomingMessage and ServerResponse, reading the method, URL, headers and body, sending JSON, manual routing, serving files with streams, listen options and server events, HTTP clients with http.request and fetch, keep-alive and agents, server timeouts, why frameworks like Express exist, TCP with net, HTTPS and HTTP/2, UDP, DNS and TLS, and local sockets.
## HTTP and Networking
**Network** = computers connected so they can send messages to each other.
**HTTP** = the language web browsers and servers use: a **request** goes out, a **response** comes back.
- Node.js can be the **server** that answers requests.
- Node.js can also be the **client** that sends requests to other servers.
- The `node:http` module does both, with no framework.
**Simple lesson:** a server waits for requests and sends back responses.
### A Real-World Example
Think of the school office window.
- A parent comes to the window with a slip: "Fee receipt for Asha, Class 8". This is the **request**.
- The clerk reads the slip, finds the receipt and hands it over. This is the **response**.
- If the slip asks for something that does not exist, the clerk says "Not found". That is a **404**.
- The window stays open all day, waiting for the next parent. That is the server **listening**.
## What You Will Learn
In this lesson, you will learn:
1. Sockets, and how `net`, `http` and `tls` fit together
2. Making a server with `http.createServer()`
3. Reading the method, URL, headers and body
4. Sending text, JSON and files
5. Simple routing by hand
6. Calling other servers with `fetch`
7. Keep-alive, agents and server timeouts
8. Why frameworks like Express exist
9. TCP, HTTPS, HTTP/2, UDP, DNS and local sockets (advanced)
## Networking in Node.js
**Socket** = one end of a two-way connection between two programs.
- **`net`** = plain TCP sockets. Bytes in, bytes out.
- **`tls`** = TCP sockets with encryption.
- **`http`** = speaks HTTP on top of `net`. **`https`** = HTTP on top of `tls`.
- A socket is a **duplex stream**: you read from it and write to it (see **Buffers, Streams and Compression**).
HTTP is a set of rules for messages sent over TCP sockets.
**Simple lesson:** `http` is built on `net`, and a socket is a two-way stream.
## HTTP Server
**`http.createServer(handler)`** = make a server. The handler runs once for every request.
- **`req`** = an `IncomingMessage`: the request, with `method`, `url`, `headers`, and the body as a stream.
- **`res`** = a `ServerResponse`: you set the status and headers, and write the body.
- **`server.listen(3000)`** = start waiting for requests on port 3000.
Click **▶ Run**. The lab sends `GET /` to your server and shows the answer.
- **The server keeps running.** It waits for the next request until you press **■ Stop**.
- **Every path** gets the same answer, because the handler does not look at the URL yet.
Every request runs the same handler function on the same thread. Keep it quick, and never block it.
### Reading the Method, URL and Headers
- **`req.method`** = `GET`, `POST`, `PUT`, `DELETE` and so on.
- **`req.url`** = the path and query, such as `/students?class=8`.
- **`new URL(req.url, "http://localhost")`** = split the URL into `pathname` and `searchParams`.
- **`req.headers`** = all request headers, with lowercase names.
### Sending Responses
- **`res.writeHead(status, headers)`** = set the status and headers in one call.
- **`res.setHeader(name, value)`** = set one header.
- **`res.write(text)`** = send part of the body. **`res.end(text)`** = send the last part and finish.
- **JSON** = set `Content-Type: application/json` and send `JSON.stringify(data)`.
### Reading the Request Body as a Stream
- The body does not arrive with the handler. It arrives as a stream of chunks.
- Read it with `for await (const chunk of req)`, then join the chunks.
- Always limit the size. Otherwise one huge upload can fill your server's memory.
Never trust the body. Check its size, its JSON and every field before you use it.
**Simple lesson:** read `req`, then set the status, set the headers and end the response.
Every request gets exactly one response.
## Basic Manual Routing
**Routing** = choosing what to do from the method and the path.
Try each quick request on the right.
- **`/students/2`** found Ravi with a regular expression.
- **`/students/9`** gave **404**, because there is no such student.
- **`DELETE`** gave **405**, because that method is not allowed.
Send JSON errors like `{ "error": "No such student" }` with the right status code. Clients read both.
**Simple lesson:** routing is "if method and path match, do this", with 404 and 405 as fallbacks.
## Serving Files with Streams
- Use `fs.createReadStream(file).pipe(res)`. The file is sent chunk by chunk.
- Set the right `Content-Type`, such as `text/html` or `text/css`.
- Handle a missing file with **404**.
- Never build the file path straight from the URL. `/../../secret` could escape your folder. This is called **path traversal** (see **Cryptography and Security**).
**Simple lesson:** stream files to the response, set the type, and keep requests inside your folder.
## listen() Options and Server Events
- **`listen(port)`** = the port to use. **`listen(0)`** = any free port, handy in tests.
- **`listen({ port, host })`** = choose which address to listen on, such as `"127.0.0.1"` for this computer only.
- **`listening` event** = the server is ready.
- **`error` event** = it could not start, for example **`EADDRINUSE`**: the port is taken.
- **`connection` event** = a new socket connected.
- **`close` event** = the server has stopped. `server.close()` stops it.
**Simple lesson:** listen for `error` on a server, because a busy port is common.
## HTTP Client
- **`fetch(url, options)`** = the modern way to call other servers. Built into Node.js, based on a library called **undici**.
- Unlike in the browser, Node.js has no CORS rules and no cookies by default.
- **`http.get()` and `http.request()`** = the older, lower-level client. You write the body and read the response as streams.
- Always set a timeout with `signal: AbortSignal.timeout(ms)`. A server that never answers would otherwise hold your program.
```js
// main.mjs: the older http.request client
import http from "node:http";
const server = http.createServer((req, res) => res.end("Hello from the office"));
server.listen(0, () => {
const { port } = server.address();
const req = http.request({ port, path: "/", method: "GET" }, (res) => {
let body = "";
res.setEncoding("utf8");
res.on("data", (chunk) => (body += chunk));
res.on("end", () => {
console.log(res.statusCode, body);
server.close();
});
});
req.end(); // send the request
});
```
Output:
```text
200 Hello from the office
```
**Simple lesson:** use `fetch` with a timeout signal to call other servers.
## Connection Management and Server Timeouts
**Keep-alive** = reuse one TCP connection for many requests, instead of opening a new one each time.
- Opening a connection takes time. Reusing it is faster.
- **`http.Agent`** = the object that keeps and reuses connections for the client.
- **`maxSockets`** = the most connections an agent opens to one server at once.
- `fetch` reuses connections for you.
Server timeouts protect a server from slow or stuck clients:
- **`headersTimeout`** = how long to wait for the request headers.
- **`requestTimeout`** = how long to wait for the whole request.
- **`keepAliveTimeout`** = how long to keep an idle connection open after a response.
```js
// main.mjs
import http from "node:http";
const server = http.createServer();
console.log("headersTimeout:", server.headersTimeout, "ms");
console.log("requestTimeout:", server.requestTimeout, "ms");
console.log("keepAliveTimeout:", server.keepAliveTimeout, "ms");
const agent = new http.Agent({ keepAlive: true, maxSockets: 10 });
console.log("Agent keeps connections?", agent.keepAlive, "max sockets:", agent.maxSockets);
```
Output:
```text
headersTimeout: 60000 ms
requestTimeout: 300000 ms
keepAliveTimeout: 5000 ms
Agent keeps connections? true max sockets: 10
```
**Simple lesson:** reuse connections, and set timeouts so stuck clients cannot hold your server.
## Limits of the Raw HTTP Module
With only `node:http`, you write everything yourself:
- **Routing** = matching paths with `if` and regular expressions.
- **Parsing** = reading and checking JSON, forms and query strings.
- **Middleware** = shared steps, such as logging, auth and error pages, for every route.
- **Security headers, cookies, sessions and uploads.**
This is why frameworks like **Express** exist. Express is a thin layer on top of `node:http`. Everything in this lesson still happens underneath. The **Express.js** course starts from here.
**Simple lesson:** `node:http` gives you the engine, and Express gives you the steering wheel.
## TCP with net (Advanced)
- **`net.createServer(socket => ...)`** = a TCP server. Each client gets a socket.
- **`net.connect(port)`** = a TCP client.
- **Socket events:** `data`, `end`, `error`, `close`.
- **Message framing** = TCP is one long stream of bytes, not separate messages. You decide where a message ends, for example with a new line.
```js
// main.mjs: a TCP echo server and client, one message per line
import net from "node:net";
const server = net.createServer((socket) => {
socket.setEncoding("utf8");
let buffer = "";
socket.on("data", (chunk) => {
buffer += chunk;
let i;
while ((i = buffer.indexOf("\n")) !== -1) {
const message = buffer.slice(0, i);
buffer = buffer.slice(i + 1);
socket.write(`roll call: ${message.toUpperCase()}\n`);
}
});
});
server.listen(0, () => {
const client = net.connect(server.address().port);
client.setEncoding("utf8");
client.on("data", (d) => {
process.stdout.write(d);
if (d.includes("RAVI")) {
client.end();
server.close();
}
});
client.write("asha\nra");
setTimeout(() => client.write("vi\n"), 20); // one message split across two writes
});
```
Output:
```text
roll call: ASHA
roll call: RAVI
```
## HTTPS and HTTP/2 (Advanced)
- **`https.createServer({ key, cert }, handler)`** = an encrypted HTTP server.
- **Certificate** = a file that proves the server's name, signed by a trusted authority.
- **Local development certificates** = made with a tool such as `mkcert`, so `https://localhost` works on your own computer.
- In production, HTTPS is often handled by a proxy or the hosting platform in front of Node.js.
- **`node:http2`** = HTTP/2, which sends many requests over one connection at once. Browsers need it over TLS.
```js
import https from "node:https";
import { readFileSync } from "node:fs";
const server = https.createServer(
{ key: readFileSync("localhost-key.pem"), cert: readFileSync("localhost.pem") },
(req, res) => res.end("Hello over HTTPS"),
);
server.listen(8443);
```
## UDP, DNS and TLS (Advanced)
- **`dgram`** = UDP sockets. Fast, no connection, no delivery promise. Used for games, video and discovery.
- **`dns.lookup(name)`** = find an address the way the operating system does. Uses the thread pool and the `hosts` file.
- **`dns.resolve4(name)`** = ask DNS servers directly. Does not read the `hosts` file.
- **`tls.connect()`** = an encrypted TCP client, the base of `https`.
```js
// main.mjs
import dns from "node:dns/promises";
import dgram from "node:dgram";
console.log("localhost is", (await dns.lookup("localhost", { family: 4 })).address);
const server = dgram.createSocket("udp4");
server.on("message", (msg) => {
console.log("UDP message:", msg.toString());
server.close();
});
server.bind(0, () => {
const client = dgram.createSocket("udp4");
client.send("score 3-1", server.address().port, "127.0.0.1", () => client.close());
});
```
Output:
```text
localhost is 127.0.0.1
UDP message: score 3-1
```
## Local Sockets (Advanced)
- **Unix domain socket** = a socket that is a file on Mac and Linux, such as `/tmp/app.sock`. Fast, for programs on the same computer.
- **Windows named pipe** = the Windows version, such as `\\.\pipe\app`.
- Use them with `server.listen("/tmp/app.sock")` and `net.connect("/tmp/app.sock")`.
**Simple lesson:** under HTTP sit TCP, TLS and UDP, and Node.js gives you all of them.
## Common mistakes
| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Forgetting `res.end()` | The response sends itself | Always end every response | The browser waits forever |
| Reading `req.body` | The body is already parsed | Read the stream with `for await` | `undefined` |
| No size limit on the body | Clients send small bodies | Stop at a limit and reply 413 | Memory runs out |
| Building a file path from the URL | URLs are safe | Normalize and check the folder | Path traversal leaks files |
| No `error` listener on the server | Start-up never fails | Handle `EADDRINUSE` | The program crashes |
| `fetch` with no timeout | Servers always answer | Pass `AbortSignal.timeout(ms)` | Your program hangs |
| Sending JSON as `text/plain` | Clients guess the type | Set `Content-Type: application/json` | Clients misread the data |
## TL;DR
1. **Socket** = one end of a two-way connection. `http` sits on `net`.
2. **`http.createServer((req, res) => ...)`** = a server. **`listen(3000)`** starts it.
3. **`req`** = method, url, headers, and the body as a stream.
4. **`res`** = `writeHead`, `setHeader`, `write`, `end`.
5. **Routing by hand** = check method and path. Fall back to 404 and 405.
6. **Files** = `createReadStream(file).pipe(res)`, inside a safe folder.
7. **`fetch`** = the built-in client. Add `AbortSignal.timeout`.
8. **Keep-alive** = reuse connections. **Timeouts** = protect the server.
9. **Express** = routing, parsing and middleware on top of `node:http`.
## Check Yourself
## Exercises
## What to Remember
- A server answers requests. `http.createServer` makes one, `listen` starts it.
- The request body is a stream. Read it, limit it, then parse it.
- Set the status and `Content-Type`, and always `end()` the response.
- Route by method and path, with 404 and 405 as fallbacks.
- Call other servers with `fetch` and a timeout signal.
- `node:http` is the base that Express builds on.
Next: **Child Processes, Worker Threads and Clustering**. You will use more than one CPU core for heavy work.