Shell-Pool

A high-performance, persistent Rust application for managing a shell command execution pool. It enables efficient process lifecycle management, DAG-based task execution, and inter-process communication via JSON-RPC.

Architecture

graph TD
    Client[Client App / CLI] -->|JSON-RPC| Daemon[Daemon Process]
    Daemon --> Pool[Process Pool]
    Pool --> Exec[Command Executors]
    Exec --> Proc[OS Processes]
    
    subgraph Daemon_Responsibility
        Daemon
        Pool
        Exec
    end

Key Components

1. Daemon (daemon.rs)

The core manager. It runs as a persistent background service, maintaining a Unix domain socket for incoming JSON-RPC requests. It orchestrates the lifecycle of the pool and dispatches tasks.

2. Process Pool (pool.rs)

Manages a fixed or dynamic set of worker processes. It ensures resources are reused, preventing the overhead of frequent process spawning.

3. Execution Strategies (chain.rs, parallel.rs, dag.rs)

• Chain: Executes commands sequentially, where one's success might be a prerequisite for the next.
• Parallel: Executes commands concurrently, managing workload distribution across the pool.
• DAG: Manages complex dependency graphs, ensuring commands run only when their dependencies are satisfied.

4. Executors (executor.rs)

Wraps the actual system call execution, providing unified error handling, timeout management, and stdout/stderr capture.

Communication Protocol

The daemon communicates via a JSON-RPC 2.0 protocol over Unix sockets.

JSON-RPC API

execute — Batch Command Execution

Sends a complete command and waits for the full result.

{
    "jsonrpc": "2.0",
    "method": "execute",
    "params": {
        "command": "whoami",
        "args": [],
        "timeout": 1,
        "working_dir": "/tmp",
        "session_id": "session-abc123"
    },
    "id": 2
}

• working_dir — Optional. Sets the command's current directory via Command::current_dir().
• session_id — Optional. Runs the command with the session's environment variables.

Expected Response:

{
    "jsonrpc": "2.0",
    "result": {
        "duration_ms": 1,
        "exit_code": 0,
        "stderr": "",
        "stdout": "kade\n",
        "timed_out": false
    },
    "id": 2
}

execute_stream — Real-Time Streaming Execution

Keeps the connection open and streams stdout/stderr chunks as JSON-lines as the command runs.

{
    "jsonrpc": "2.0",
    "method": "execute_stream",
    "params": {
        "command": "cargo build",
        "args": [],
        "timeout": 300,
        "working_dir": "/home/kade/src/myproject"
    },
    "id": 3
}

Connection receives newline-delimited JSON chunks:

{"jsonrpc":"2.0","result":{"type":"stdout","data":"Compiling myproject v0.1.0\r\n"},"id":3}
{"jsonrpc":"2.0","result":{"type":"stdout","data":"    Finished dev [unoptimized + debuginfo] target(s)\r\n"},"id":3}
{"jsonrpc":"2.0","result":{"type":"exit","exit_code":0,"duration_ms":1234},"id":3}

session_create — Create a Named Session

Create a session with persistent environment variables.

{
    "jsonrpc": "2.0",
    "method": "session_create",
    "params": {
        "name": "my-session",
        "env_vars": {
            "RUST_BACKTRACE": "1",
            "MY_CUSTOM_VAR": "value"
        }
    },
    "id": 4
}

Expected Response:

{
    "jsonrpc": "2.0",
    "result": {
        "session_id": "session-abc123"
    },
    "id": 4
}

Pass session_id to execute or execute_stream to run commands with that session's environment.

session_list / session_destroy

List active sessions or destroy a specific session.

sequenceDiagram
    participant Client
    participant Daemon
    participant Pool
    participant Session
    
    Client->>Daemon: execute / execute_stream (JSON-RPC)
    Daemon->>Session: Get env_vars (if session_id)
    Daemon->>Pool: Dispatch Task (with working_dir + env)
    Pool->>Pool: Execute Command
    Pool-->>Daemon: Command Result (batch or streaming)
    Daemon-->>Client: Result (JSON-RPC or JSON-lines)

Setup & Configuration

• Socket: Defaults to /run/user/$UID/shell-pool/shell-pool.sock.
• Config: Managed via environment variables, loaded by config.rs.
• Deployment: Managed via Ansible, installed as a user-level systemd service.

Security

• No Root Required: Designed to run as a user-level daemon, minimizing potential exploit surface.
• Resource Constraints: Pool size and timeouts are configurable per request and globally to prevent resource exhaustion.

SSH Command Handling

Shell-pool constructs SSH commands programmatically via SshExecutor. All SSH commands include:

• -F /dev/null — Bypasses system-wide SSH configuration files. Required in container/sandbox environments where /etc/ssh may be owned by nobody:nobody instead of root:root, which causes ssh to reject configs with "Bad owner or permissions".
• -o StrictHostKeyChecking=no — Automates host key acceptance for programmatic connections.
• -o UserKnownHostsFile=/dev/null — Prevents polluting the user's known_hosts file with programmatic connection keys.

These options are safe because shell-pool explicitly controls all SSH parameters programmatically rather than relying on system-wide configuration.