HTTP from Scratch (over Raw TCP)

This repository is the source code companion to my articles on building an HTTP/1.1 server from raw TCP sockets:

• Dev.to: Building an HTTP Server from Scratch Using TCP
• Medium: Building an HTTP Server from Scratch Using TCP

I built this project to understand how HTTP actually works under the hood without relying on Go's net/http package to do any request parsing or response formatting. It uses only standard TCP sockets (net.Listen and net.Conn) to handle connections, parse raw text streams into structured requests, and serialize HTTP responses.

---

Codebase Navigation

Here is how the project is organized and where to find each part of the implementation:

.
├── main.go               # Server entry point, TCP listener & request routing
├── pkg/
│   ├── stream/           # Buffered socket reader for TCP streams
│   │   └── main.go
│   ├── request/          # Request parser (raw text -> Request struct)
│   │   └── request.go
│   └── response/         # Response builder and serializer (Response struct -> raw text)
│       └── response.go
├── requests/             # Raw text HTTP payloads for testing with netcat
│   ├── get.txt
│   └── post.txt
├── Makefile              # Shortcuts for running and testing
└── go.mod

1. main.go — The Entry Point & Connection Handler

• TCP Listener (main): Opens port 8080 using net.Listen("tcp", ":8080") and loops forever accepting client connections (listener.Accept()).
• Concurrency: Hands each incoming net.Conn to a separate goroutine (go handleConnection(conn)) so the server never blocks while handling a request.
• Routing (handleConnection): Takes the parsed request, matches req.Path, sets status codes / response bodies, and writes the response back to the client.

2. pkg/stream/main.go — Buffered Stream Reader

TCP is a continuous stream of bytes, not clean lines of text. Reading byte-by-byte from the network is slow, so I wrapped the socket in a buffered reader (bufio.Reader):

• ReadLine(): Reads bytes until it hits \n, trimming off trailing \r\n. Used for parsing the request line and individual headers.
• ReadExact(count): Reads an exact number of bytes into a fixed buffer using io.ReadFull. Used when reading the request body to avoid corrupting binary data or payload newlines.

3. pkg/request/request.go — Request Parser

Turns raw HTTP text into a structured Go Request struct:

• Request Line: Splits the first line by spaces to extract Method (e.g., GET, POST), Path (e.g., /users), and Version (e.g., HTTP/1.1).
• Headers: Reads line-by-line until it encounters an empty string (""), which marks the end of headers (\r\n\r\n). Header names are converted to lowercase so lookups are case-insensitive.
• Body: Checks if content-length exists in the headers. If present, it converts the value to an integer and uses reader.ReadExact(contentLength) to read the exact payload.

4. pkg/response/response.go — Response Serializer

Constructs and sends a valid HTTP/1.1 response over the socket:

• SetStatus(code): Sets the status code and maps it to a standard status text (e.g., 200 -> OK, 404 -> Not Found).
• SetBody(body, contentType): Sets the payload and automatically attaches the Content-Type header.
• Send(w): Formats the status line, automatically calculates and sets Content-Length, appends all headers separated by \r\n, adds the blank line delimiter (\r\n), and writes the headers followed by the raw body bytes to the connection.

5. requests/ — Raw HTTP Payloads

Contains sample HTTP request text files (get.txt and post.txt) with exact CRLF line endings. You can pipe these directly into the TCP port to test the server without needing curl or a browser.

---

The Request Lifecycle

When a client makes a request, data flows through the packages in this sequence:

1. Client connects (TCP handshake)
   │
2. main.go accepts conn -> spawns goroutine handleConnection(conn)
   │
3. pkg/stream wraps conn in bufio.Reader
   │
4. pkg/request parses:
   ├── ReadLine() -> Method, Path, Version
   ├── ReadLine() loop -> Headers (until empty line)
   └── ReadExact(Content-Length) -> Body
   │
5. main.go inspects req.Path and builds response
   │
6. pkg/response formats HTTP text and writes bytes to conn
   │
7. Connection closes (defer conn.Close())

---

Running and Testing

1. Start the Server

go run ./main.go
# or
make start

2. Test with curl

You can test the endpoints directly using the included Makefile targets:

# Test GET /
make test-get

# Test GET /users (JSON response)
make test-users

# Test POST /users (Sends a text payload with Content-Length)
make test-post

# Test 404 handler
make test-404

Or run curl manually with -v to inspect the raw HTTP headers:

curl -v http://localhost:8080/users

3. Test with Netcat (nc)

To verify that HTTP is just plain text over TCP, send raw text files directly into the socket:

# Send raw GET request
nc localhost 8080 < requests/get.txt

# Send raw POST request with headers and body
nc localhost 8080 < requests/post.txt

---

License

MIT