What are unix sockets?

While looking at cosmos-sdk, I noticed that they use unix sockets to communicate between the consensus and the application layers. I heard about unix sockets many times, but never really invested time to understand them properly, so this was a perfect opportunity.

Unix sockets are bidirectional communication channels enabled by the kernel for programs communicating on the same host. On a high level that's it. You need three components:

1. kernel
2. client
3. server

How do clients find sockets to connect to? The kernel creates a file, for example /tmp/yoursocket.sock. No data is stored in the file. The socket lives in the kernel and is associated with the file. The file exists so other processes can find it in a shared global namespace.

The kernel does all the data transport and maintains the channel between your programs.

1. client writes → kernel buffer → server reads
2. server writes → kernel buffer → client reads

You can control connectivity by setting permissions like you would for any file on your file system. Permissions are checked during connecting, so once you have a connection, you can communicate until it's closed.

You can also choose between:

1. stream based sockets -> think of tcp, message ordering and delivery guarantee
2. datagram based sockets -> think of udp, no ordering or delivery guarantee

Things to remember:

1. Can't connect remotely
2. More performant than tcp
3. Clean up your sock files once you're done using them, it's a global namespace
4. Harder to debug than http
5. Control who can connect using file system permissions (chmod, chown)

Usage

Any time you need to communicate between processes on the same host, unix sockets can work.

Docker uses it for communication between the docker runtime and the cli. If you've ever seen something like /var/run/docker.sock, you know why now.

Postgresql also allows connecting to your db via a unix socket: psql -h /var/run/postgresql -U user -d db

Many other dbs allow connecting via a socket as well as other services like:

1. container orchestration
2. observability tools
3. web servers
4. blockchain nodes
5. ...

Any time performance is important, and you are on the same host, unix sockets should be considered.

Here is a simple snippet in golang to send messages over a socket (error handling and synchronization ignored on purpose):

package main

import (
    "bufio"
    "fmt"
    "net"
    "os"
    "time"
)

const socketPath = "/tmp/example.sock"

func main() {
    // remove old sockets if they exist
    os.RemoveAll(socketPath)

    l, err := net.Listen("unix", socketPath)
    if err != nil {
        panic(err)
    }
    defer l.Close()

    fmt.Println("Listening on: ", socketPath)

    // connect a client to a socket and send some bytes over it
    go func() {
        time.Sleep(2 * time.Second)
        conn, _ := net.Dial("unix", socketPath)
        defer conn.Close()

        message := "hello unix socket\n"
        fmt.Println("client sending:", message)

        _, _ = conn.Write([]byte(message))

        reader := bufio.NewReader(conn)
        reply, _ := reader.ReadString('\n')

        fmt.Println("client received:", reply)
    }()

    // accept connections in an infinite loop and return echoed messages
    for {
        conn, err := l.Accept()
        if err != nil {
            fmt.Println("accept error:", err)
            continue
        }

        go func(conn net.Conn) {
            defer conn.Close()

            reader := bufio.NewReader(conn)
            msg, err := reader.ReadString('\n')
            if err != nil {
                fmt.Println("read error:", err)
                return
            }

            fmt.Println("server received:", msg)

            _, _ = conn.Write([]byte("echo: " + msg))
        }(conn)
    }

}

Also, if you want to see how to use rpcs via unix sockets, check out my previous blog that covers it.