curl & libcurl Wiki

Overview

curl is a command-line tool and library for transferring data using URLs. It supports 20+ protocols including HTTP, HTTPS, FTP, SFTP, SMTP, and many others. The project consists of two main components: libcurl (the reusable C library) and curl (the command-line tool that uses libcurl).

Architecture Overview

Loading diagram...

Core Components

libcurl is the heart of the project. It provides a C API for performing URL transfers with features like:

• Easy Interface (curl_easy_*): Simple synchronous API for single transfers
• Multi Interface (curl_multi_*): Asynchronous API for handling multiple transfers
• Share Interface (curl_share_*): Shared resources (DNS cache, cookies, SSL sessions) across handles
• Connection Pooling: Reuses connections for efficiency
• Protocol Support: Pluggable protocol handlers for different schemes

curl is a command-line wrapper around libcurl. It parses command-line arguments, configures a libcurl handle, and executes transfers. The main entry point is in src/tool_main.c, which initializes the library and delegates to the operate() function.

Key Data Structures

The Curl_easy structure (defined in lib/urldata.h) represents a single transfer handle. It contains:

• Connection state and metadata
• Protocol-specific options
• Request/response buffers
• Callback functions for progress, headers, and data

The Curl_share structure manages shared resources across multiple handles, including DNS caches, cookie jars, and SSL session caches.

Initialization Flow

1. Global Init: curl_global_init() initializes SSL, async DNS, and memory tracking
2. Easy Handle: curl_easy_init() creates a new transfer handle
3. Configuration: curl_easy_setopt() sets options (URL, headers, auth, etc.)
4. Transfer: curl_easy_perform() executes the transfer
5. Cleanup: curl_easy_cleanup() and curl_global_cleanup() free resources

Protocol Handling

curl uses a modular architecture where each protocol (HTTP, FTP, SFTP, etc.) has its own handler. Connection filters (cfilters) manage the protocol stack, allowing composition of features like proxies, TLS, and HTTP/2 upgrades.

Platform Support

The codebase is designed for C89 compatibility and runs on 32-bit and 64-bit systems. It supports Windows, Linux, macOS, and many other platforms. Build systems include autoconf/automake and CMake.

Architecture & Data Flow

Overview

libcurl's architecture is built on two complementary APIs: the Easy API for simple sequential transfers and the Multi API for concurrent operations. Both APIs share a common foundation of connection management, protocol handling, and data transfer mechanisms. The system uses a connection filter chain architecture to modularize protocol layers and enable flexible composition of features like TLS, proxies, and multiplexing.

Core Components

Easy vs. Multi API

The Easy API (curl_easy_*) provides a synchronous, blocking interface where each transfer completes before the next begins. Internally, it wraps the Multi API, creating a single-transfer multi handle for simplicity.

The Multi API (curl_multi_*) enables concurrent transfers by managing multiple easy handles within a single multi handle. It uses non-blocking I/O and event-driven processing, allowing applications to multiplex hundreds of transfers efficiently.

Connection Filter Chain

The connection filter chain is the core architectural innovation (introduced in curl 7.87.0). Each connection maintains a stack of filters that handle different protocol layers:

┌─────────────────────────────────────┐
│  Application (Easy/Multi API)       │
├─────────────────────────────────────┤
│  HTTP/1.1 or HTTP/2 Filter          │
├─────────────────────────────────────┤
│  TLS/SSL Filter (optional)          │
├─────────────────────────────────────┤
│  Proxy Filter (optional)            │
├─────────────────────────────────────┤
│  Socket Filter (TCP/UDP/UNIX)       │
└─────────────────────────────────────┘

Each filter implements a standard interface with callbacks for:

• Connect: Establish the connection layer
• Send/Recv: Read and write data
• Close/Shutdown: Graceful connection termination
• Query: Inspect filter properties (socket, TLS info, ALPN, etc.)
• Control: Handle events and state changes

Filters chain together, with each filter optionally transforming data before passing it to the next. Once connected, pass-through filters (like proxies) incur near-zero overhead.

Transfer State Machine

Each transfer progresses through a well-defined state machine managed by the multi handle:

Loading diagram...

States include: INIT, PENDING, SETUP, CONNECT, RESOLVING, CONNECTING, PROTOCONNECT, DO, DOING, PERFORMING, RATELIMITING, DONE, COMPLETED, and MSGSENT.

Connection Pooling

The multi handle maintains a connection pool (conncache) that reuses connections across transfers. Connections are cached by host and protocol, allowing subsequent requests to the same server to reuse existing TCP connections and TLS sessions. This dramatically improves performance for sequential or concurrent requests to the same host.

Data Flow

1. Request Setup: Application calls curl_easy_setopt() to configure the transfer, then curl_easy_perform() (Easy API) or curl_multi_add_handle() (Multi API).

2. URL Parsing: url.c parses the URL and extracts host, port, path, and protocol information into the Curl_easy handle.

3. Connection Establishment: The multi handle checks the connection pool. If no suitable connection exists, it creates a new one and builds the filter chain based on the protocol and options.

4. Filter Chain Execution: Each filter connects in sequence. For HTTPS, this means: socket connects → TLS handshake → HTTP protocol setup.

5. Data Transfer: transfer.c orchestrates sending the request and receiving the response through the filter chain. Filters handle protocol-specific details (chunked encoding, multiplexing, etc.).

6. Completion: The transfer moves to DONE state, and a completion message is queued for the application to retrieve.

Key Design Patterns

• Non-blocking I/O: The multi handle uses select() or platform-specific event mechanisms to wait on multiple sockets without blocking.
• Lazy Initialization: Buffers and resources are allocated only when needed.
• Filter Composition: Complex protocols (HTTP/2 over TLS over SOCKS proxy) are built by stacking filters, each handling one concern.
• Connection Reuse: Persistent connections and session caching minimize overhead for repeated requests.

Protocol Handlers

Protocol handlers are the core abstraction that enables curl to support multiple network protocols. Each protocol (HTTP, FTP, SMTP, IMAP, POP3, etc.) implements a standardized interface defined by the Curl_handler structure, allowing the transfer engine to work uniformly across all protocols.

Handler Architecture

The Curl_handler structure in urldata.h defines the protocol interface with function pointers for key operations:

• Core operations: do_it() (main transfer logic), done() (cleanup), setup_connection() (initialization)
• Multi-phase support: do_more() (continuation), connect_it(), connecting(), doing() (state progression)
• Event polling: proto_pollset(), doing_pollset(), domore_pollset(), perform_pollset() (for async operations)
• Lifecycle: disconnect() (teardown), attach() (connection binding)
• Metadata: scheme (URL scheme), defport (default port), protocol (protocol bit), flags (capabilities)

Handler Registration

Handlers are registered in a static hash table in url.c via Curl_getn_scheme_handler(). The table uses a simple hash function on the scheme name to enable O(1) lookup. Each protocol defines its handler as a global constant (e.g., Curl_handler_http, Curl_handler_ftp) and is conditionally included based on compile-time flags like CURL_DISABLE_HTTP.

Protocol Categories

Stateless protocols like HTTP use minimal state management. The handler directly executes the transfer in do_it().

Stateful protocols (FTP, SMTP, IMAP, POP3) use the pingpong abstraction in pingpong.h. This provides a generic request-response framework with buffering, timeouts, and state machine support. Protocols implement two callbacks: statemachine() (drives protocol logic) and endofresp() (detects response completion).

Protocol Flags

Handlers declare capabilities via flags:

• PROTOPT_SSL — supports TLS encryption
• PROTOPT_DUAL — uses two connections (e.g., FTP control + data)
• PROTOPT_NEEDSPWD — requires authentication
• PROTOPT_CREDSPERREQUEST — credentials needed per request, not per connection

Dispatch Flow

When a URL is processed, findprotocol() looks up the handler by scheme. The handler is attached to the connection, and the transfer engine calls handler methods in sequence: setup_connection() → do_it() → do_more() (if needed) → done(). For async operations, the multi interface polls handlers via their *_pollset() functions to determine when to resume execution.

Connection Management & Filters

Connection management in curl is built on a filter chain architecture that separates concerns and enables flexible protocol composition. Each connection maintains a chain of filters responsible for reading, writing, connecting, and closing operations.

Filter Chain Architecture

A connection filter is a modular component handling specific protocol or transport concerns. Filters are chained together, with each filter delegating to the next filter in the chain. This design allows curl to compose complex connection scenarios (e.g., HTTP/2 proxy with TLS) without tight coupling.

struct Curl_cfilter {
  const struct Curl_cftype *cft;  /* filter type with implementation */
  struct Curl_cfilter *next;      /* next filter in chain */
  void *ctx;                      /* filter-specific context */
  struct connectdata *conn;       /* parent connection */
  int sockindex;                  /* socket index in connection */
  BIT(connected);                 /* connection state */
  BIT(shutdown);                  /* shutdown state */
};

Filters are added at the top of the chain and process data top-down. Lower filters can override upper filter decisions (e.g., a TLS filter may remove POLL_OUT if it needs to read first).

Filter Types

Curl defines several filter types, each with specific capabilities:

• Socket Filters (TCP, UDP, UNIX): Raw I/O operations on sockets
• SSL Filter: TLS encryption, handshake, and backend management
• Proxy Filters (HTTP-PROXY, H1-PROXY, H2-PROXY, SOCKS-PROXY): Protocol tunneling
• HTTP Filters (HTTP/2, HTTP/3): Multiplexing and protocol handling
• Meta Filters (HAPPY-EYEBALLS, HTTPS-CONNECT): Manage sub-filter racing and selection

Each filter type is defined by a Curl_cftype structure containing function pointers for connect, send, recv, close, shutdown, and query operations.

Connection Pool Management

The connection pool (cpool) stores reusable connections organized by destination. Key operations include:

• Finding connections: Curl_cpool_find() searches for matching connections using callback filters
• Adding connections: Curl_cpool_add() adds new connections with limit checking
• Pruning: Curl_cpool_prune_dead() removes half-open/dead connections
• Idle management: Curl_cpool_conn_now_idle() handles idle connection cleanup

Connections are bundled by destination string, enabling efficient reuse for the same host.

Filter Operations

Common filter operations include:

• Connect: Establish connection through the filter chain
• Send/Recv: Pass data through filters (near-zero cost if no transformation)
• Query: Retrieve properties (socket, SSL info, ALPN, IP info, etc.)
• Control: Handle events like data setup, pause, flush
• Adjust Pollset: Modify poll events based on filter state

/* Example: Send data through filter chain */
CURLcode Curl_cf_send(struct Curl_easy *data, int sockindex,
                      const uint8_t *buf, size_t len, bool eos,
                      size_t *pnwritten);

Connection Lifecycle

1. Creation: Filter chain built based on protocol requirements
2. Connection: Filters connect top-down, each waiting for lower filters
3. Data Transfer: Send/recv operations flow through chain
4. Idle: Connection returned to pool if reusable
5. Cleanup: Filters shutdown gracefully or close immediately

The filter architecture enables curl to handle complex scenarios like proxied HTTPS connections, HTTP/2 multiplexing, and happy eyeballing with clean separation of concerns.

TLS & Security

Multi-Backend TLS Architecture

Curl implements a pluggable TLS backend system through the Curl_ssl abstraction layer. This allows compile-time selection of different SSL/TLS libraries: OpenSSL, GnuTLS, WolfSSL, mbedTLS, Rustls, Schannel (Windows), and Apple SecTrust (macOS/iOS). The vtls.c module acts as a dispatcher, routing all TLS operations to the selected backend via function pointers in the Curl_ssl struct.

Each backend implements the same interface for context initialization, session management, certificate verification, and connection handshakes. This design enables users to choose their preferred cryptographic library without recompiling the entire codebase.

Certificate Verification Pipeline

Certificate verification follows a multi-stage process controlled by verifypeer and verifyhost flags:

1. Peer Certificate Validation (verifypeer=TRUE): The backend verifies the certificate chain against trusted CA certificates using the system's certificate store or user-provided CA bundles.

2. Hostname Verification (verifyhost=TRUE): The certificate's Common Name (CN) or Subject Alternative Names (SANs) are matched against the requested hostname using RFC6125 rules. Wildcard matching is supported only for the leftmost label (e.g., *.example.com matches www.example.com but not sub.www.example.com).

3. OCSP Status Checking (verifystatus=TRUE): Online Certificate Status Protocol queries verify the certificate hasn't been revoked.

4. Public Key Pinning: Optional SHA256 hashes of the certificate's public key can be pinned to prevent MITM attacks via compromised CAs.

Authentication Mechanisms

Curl supports multiple authentication schemes through the SASL framework and HTTP-specific methods:

SASL Mechanisms (for IMAP, SMTP, POP3, LDAP):

• LOGIN, PLAIN, CRAM-MD5, DIGEST-MD5
• GSSAPI, EXTERNAL, NTLM
• SCRAM-SHA-1, SCRAM-SHA-256
• XOAUTH2, OAUTHBEARER

HTTP Authentication:

• Basic: Base64-encoded credentials (requires HTTPS for security)
• Digest: Challenge-response with MD5/SHA-256 hashing
• Bearer: OAuth 2.0 token-based authentication

The curl_sasl.c module decodes mechanism names and routes to appropriate handlers. Digest authentication (http_digest.c) implements RFC2617, computing response hashes based on server-provided nonces and realm values.

Session Caching & Resumption

TLS sessions are cached in vtls_scache.c to enable session resumption, reducing handshake overhead. Session IDs and tickets are stored per-host and reused when verifypeer is enabled. This is critical for performance in scenarios with repeated connections to the same server.

Key Security Features

• SNI (Server Name Indication): Hostname sent during handshake for virtual hosting support
• ALPN (Application-Layer Protocol Negotiation): Protocol selection (HTTP/1.1, HTTP/2, HTTP/3)
• ECH (Encrypted Client Hello): Hides SNI from network observers (OpenSSL 3.0+)
• Keylogging: Optional SSLKEYLOGFILE support for debugging with Wireshark
• Native CA Stores: Integration with system certificate stores (Windows, macOS)

curl Command-Line Tool

The curl command-line tool is the primary user-facing interface to libcurl. It transforms command-line arguments into libcurl API calls, enabling users to perform HTTP requests and other network operations from the shell.

Execution Flow

The tool follows a well-defined initialization and execution pipeline:

1. Startup (tool_main.c): The main() function initializes platform-specific features (Windows, VMS, Amiga), checks file descriptors, and sets up memory tracking if debugging is enabled.

2. Global Configuration (tool_cfgable.c): globalconf_init() creates the global configuration state, which manages all operation configurations and shared resources.

3. Operation Dispatch (tool_operate.c): The operate() function is the core orchestrator. It:

   • Loads .curlrc configuration files
   • Parses command-line arguments via parse_args()
   • Handles special requests (help, version, manual)
   • Executes transfers (serial or parallel)

Argument Parsing

The argument parsing system is centralized in tool_getparam.c and uses a lookup table approach:

• Aliases Table: A static alphasorted array of LongShort structures maps option names to command codes (e.g., --data <-> C_DATA).
• getparameter() Function: Processes individual flags and their arguments. It distinguishes between long options (--option), short options (-o), and combined forms (-ofoo).
• Option Types: Each option has metadata (ARG_BOOL, ARG_STRG, ARG_FILE) that determines how arguments are consumed and validated.

Configuration Data Structures

Two main structures hold configuration state:

OperationConfig (per-request): Contains URL lists, headers, authentication, proxy settings, output options, and protocol-specific parameters. Multiple configs can be chained for batch operations.

GlobalConfig (process-wide): Manages the linked list of OperationConfig nodes, global state (trace settings, SSL sessions, variables), and execution mode (serial vs. parallel).

Transfer Execution

Once arguments are parsed, the tool creates transfer objects and executes them:

• Serial Mode: Transfers execute sequentially via serial_transfers().
• Parallel Mode: Multiple transfers run concurrently via parallel_transfers() using libcurl's multi interface.
• Pre/Post Transfer Hooks: pre_transfer() and post_transfer() handle setup, cleanup, and retry logic.

Key Design Patterns

• Configuration Chaining: Multiple OperationConfig nodes allow batch operations with different settings per URL.
• Lazy Initialization: libcurl handles are created only when needed, reducing overhead for help/version requests.
• Error Propagation: ParameterError codes distinguish parsing errors from runtime errors, enabling precise error reporting.
• Platform Abstraction: Conditional compilation handles Windows, VMS, and POSIX differences transparently.

Testing Infrastructure

Curl's testing infrastructure consists of three complementary test suites that validate functionality at different levels: unit tests for individual functions, libcurl tests for library behavior, and curl tool tests for end-to-end scenarios.

Test Types

Unit Tests focus on individual libcurl functions in isolation. Located in tests/unit/, they require curl built with --enable-debug and use a custom test harness with macros like fail_unless() and fail_if(). Each test is a C source file named unitNNNN.c paired with a test data file tests/data/testNNNN.

Libcurl Tests verify library functionality through custom C programs in tests/libtest/. These tests link against libcurl and exercise the public API, allowing validation of library behavior in realistic scenarios. They use the same test data format as curl tool tests.

Curl Tool Tests validate the curl command-line tool itself. These tests run curl with various arguments and verify output, protocol exchanges, and error handling. Test data files in tests/data/testNNNN define expected behavior using XML-like sections for client commands, server responses, and verification criteria.

HTTP Test Suite provides advanced integration testing using pytest with real servers (Apache httpd, nghttpx). Located in tests/http/, it tests complex scenarios like HTTP/2, HTTP/3, proxies, and authentication that are difficult to simulate.

Test Execution

The main test runner is runtests.pl, a Perl script that orchestrates test execution. It supports parallel execution with the -j flag, allowing multiple tests to run simultaneously. The runner manages test servers, handles timeouts, collects results, and generates reports.

Key execution modes include:

• Standard: make test runs all enabled tests sequentially
• Parallel: make test TFLAGS="-j10" runs up to 10 tests concurrently
• Torture: make test TFLAGS="-t" runs tests repeatedly to detect memory leaks and race conditions
• Event-based: make test TFLAGS="-e" tests the curl_multi_socket_action() API
• CI Mode: make test TFLAGS="-a -p ~flaky ~timing-dependent -r --retry=5 -j20" optimized for continuous integration

Test Infrastructure

The test framework manages multiple protocol servers (HTTP, HTTPS, FTP, FTPS, SMTP, etc.) started on-demand. Server management code in servers.pm handles startup, shutdown, and port allocation. Tests can skip if required servers or features are unavailable.

Test data files use a structured format with sections like <client> (curl command), <server> (expected protocol), <reply> (server response), and <verify> (expected output). This declarative approach makes tests easy to read and maintain without writing C code.

The framework supports feature detection, allowing tests to skip gracefully when curl lacks required capabilities (e.g., IPv6, specific protocols, TLS backends). Tests can also be disabled via the DISABLED file or marked as flaky to ignore failures in CI.

Writing Tests

To add a new test, create tests/data/testNNNN with test definition and tests/libtest/libNNNN.c for libcurl tests. Unit tests go in tests/unit/unitNNNN.c. Register tests in Makefile.inc files. The test framework automatically discovers and runs new tests during make test.

Build System

curl supports two primary build systems: CMake (modern, cross-platform) and Autotools (traditional Unix). Both can generate static and shared libraries with extensive configuration options.

CMake Build System

CMake is the recommended approach for modern builds. The root CMakeLists.txt handles version detection, platform detection, and dependency discovery. Key features include:

• Platform Support: Detects Windows, macOS, Linux, BSD, Android, DOS, Amiga, and more
• Compiler Detection: Identifies GCC, Clang, MSVC, and applies appropriate flags
• Dependency Management: Automatically finds OpenSSL, GnuTLS, mbedTLS, Rustls, zlib, Brotli, and 20+ other libraries
• Feature Flags: Over 50 CURL_DISABLE_* options to customize protocol and feature support

cmake -B build -DCURL_USE_OPENSSL=ON -DBUILD_SHARED_LIBS=ON
cmake --build build

The lib/CMakeLists.txt defines library targets. It supports object library sharing (reusing compiled objects for both static and shared builds on compatible platforms) and versioned symbols on Unix-like systems.

Autotools Build System

The traditional Autotools approach uses configure.ac and Makefile.am files. It provides similar functionality but with a different workflow:

./buildconf
./configure --with-openssl --enable-ipv6
make

Key configuration options include TLS backend selection (--with-openssl, --with-gnutls, etc.), protocol support toggles, and compression library integration.

Build Targets

Both systems support:

• Static Libraries: libcurl.a or libcurl_static.lib
• Shared Libraries: libcurl.so or libcurl.dll
• Curl Executable: Command-line tool linking against libcurl
• Unit Tests: Special curlu library for testing (CMake only)
• Documentation: Man pages and HTML docs (requires Perl)

Dependency Configuration

The build system auto-detects dependencies via pkg-config or manual paths. Critical dependencies include:

• TLS Backends: OpenSSL, GnuTLS, mbedTLS, wolfSSL, Rustls, Schannel (Windows), SecTrust (macOS)
• Compression: zlib, Brotli, Zstandard
• HTTP/2 & HTTP/3: nghttp2, ngtcp2, nghttp3, quiche
• Other: c-ares (async DNS), libssh2, libpsl, libidn2

Testing

Run tests via:

make test              # Autotools
ctest                  # CMake

Tests require Perl and optional servers (Apache, nghttpx, vsftpd, sshd) for protocol testing.

Utilities & Helpers

Curl provides a collection of low-level utility libraries that form the foundation for data management and string processing throughout the codebase. These helpers handle dynamic memory allocation, data structure management, and URL encoding/decoding.

Dynamic Buffers (dynbuf)

The dynamic buffer utility provides a growable string buffer with automatic memory management. It's designed for safe, bounded allocation with configurable size limits.

Key Features:

• Automatic exponential growth strategy (starts at 32 bytes minimum)
• Configurable maximum size to prevent runaway allocations
• Printf-style formatting support
• Safe memory operations with debug assertions

Core Operations:

• curlx_dyn_init() - Initialize with size limit
• curlx_dyn_add() / curlx_dyn_addn() - Append strings or binary data
• curlx_dyn_addf() - Printf-style formatting
• curlx_dyn_ptr() - Get buffer pointer
• curlx_dyn_take() - Extract buffer and reset

Example Usage:

struct dynbuf buf;
curlx_dyn_init(&buf, 1024);
curlx_dyn_add(&buf, "Hello ");
curlx_dyn_addf(&buf, "%s", "World");
char *result = curlx_dyn_ptr(&buf);
curlx_dyn_free(&buf);

Hash Tables

The hash table implementation provides O(1) average-case lookup with chaining for collision resolution. It's used for caching connections, cookies, and other key-value data.

Architecture:

• Slot-based table with linked-list chains
• Customizable hash and comparison functions
• Per-element destructor support
• Iterator interface for traversal

Core Operations:

• Curl_hash_init() - Create hash with custom functions
• Curl_hash_add() / Curl_hash_add2() - Insert or update entries
• Curl_hash_pick() - Retrieve by key
• Curl_hash_delete() - Remove entry
• Curl_hash_start_iterate() - Begin iteration

Built-in Functions:

• Curl_hash_str() - DJB2-style string hashing
• curlx_str_key_compare() - Binary-safe string comparison

Linked Lists

A doubly-linked list with intrusive node design, where nodes are embedded in user structures rather than allocated separately. This reduces memory fragmentation and improves cache locality.

Design Pattern: Nodes are embedded in user structures, providing O(1) removal without searching:

struct my_data {
  int value;
  struct Curl_llist_node node;  /* embedded node */
};

Core Operations:

• Curl_llist_init() - Initialize with optional destructor
• Curl_llist_append() - Add to end
• Curl_llist_insert_next() - Insert after position
• Curl_node_remove() - Remove node
• Curl_llist_destroy() - Clean up entire list

URL Encoding/Decoding

Utilities for percent-encoding and hex conversion, essential for URL manipulation and data encoding.

Functions:

• curl_easy_escape() - Percent-encode strings (public API)
• Curl_urldecode() - Percent-decode with validation options
• Curl_hexencode() - Binary to hex ASCII conversion
• Curl_hexbyte() - Single byte to two-digit hex

Validation Options:

• REJECT_NADA - Accept all bytes
• REJECT_CTRL - Reject control characters (<0x20)
• REJECT_ZERO - Reject null bytes

Example:

char *encoded = curl_easy_escape(NULL, "hello world", 11);
/* Result: "hello%20world" */

Memory Safety

All utilities include debug-mode assertions to catch API misuse:

• Initialization markers detect use-after-free
• Size bounds prevent buffer overflows
• Null-pointer checks on critical operations

These helpers are designed for reliability in production environments while providing strict validation during development.

DNS Resolution & Networking

Overview

DNS resolution in curl is a multi-layered system supporting both synchronous and asynchronous name resolution. The architecture provides flexibility through compile-time configuration, allowing different resolver backends (system getaddrinfo, c-ares, or threaded) and modern protocols like DNS-over-HTTPS (DoH).

Resolution Methods

Curl supports three primary resolution strategies:

1. Synchronous Resolution (CURLRES_SYNCH) - Blocking calls to system getaddrinfo() or platform-specific functions
2. Asynchronous with c-ares (CURLRES_ARES) - Non-blocking resolution using the c-ares library
3. Threaded Resolution (CURLRES_THREADED) - Background thread-based resolution on Windows

The resolver backend is selected at compile time via configuration flags. Each backend implements the common async interface defined in asyn.h, ensuring consistent behavior across different builds.

DNS Caching

The DNS cache (Curl_dnscache) uses a hash table to store resolved entries, keyed by hostname and port. Key features include:

• Reference counting - Entries track usage to prevent premature cleanup
• TTL-based expiration - Entries age out based on configurable timeout (default 300 seconds)
• Negative caching - Failed lookups are cached and age twice as fast as successful ones
• Shared caching - Cache can be shared across multiple handles via CURLOPT_SHARE
• Pruning - Automatic cleanup of stale entries when cache reaches size limits

Cache entries are created with Curl_dnscache_mk_entry() and retrieved with Curl_dnscache_get(). The Curl_resolv() function is the main entry point for name resolution.

DNS-over-HTTPS (DoH)

DoH provides privacy-preserving DNS resolution by encapsulating DNS queries in HTTPS requests. The implementation:

• Encodes DNS queries into binary format using doh_req_encode()
• Sends queries to a DoH server via HTTPS
• Decodes responses with doh_resp_decode()
• Supports multiple simultaneous probes (IPv4, IPv6, HTTPS RR)
• Falls back to traditional DNS if DoH fails

DoH is controlled via CURLOPT_DOH_URL and can be disabled at compile time with CURL_DISABLE_DOH.

Async Resolution Flow

Loading diagram...

Happy Eyeballs Integration

For IPv6-capable systems, curl implements Happy Eyeballs (RFC 8305) to handle dual-stack scenarios. When both IPv4 and IPv6 addresses are available, the resolver attempts connections in parallel with staggered timing, improving user experience on IPv6-broken networks.

Socket Polling

The select.c module provides cross-platform socket polling via curl_multi_poll(). It abstracts platform differences:

• Uses select() on Unix-like systems
• Uses poll() where available
• Handles Windows socket semantics
• Integrates with async resolver file descriptors

Configuration Options

Key curl options affecting DNS resolution:

• CURLOPT_RESOLVE - Pre-populate cache with custom hostname-to-IP mappings
• CURLOPT_DOH_URL - Enable DNS-over-HTTPS with specified server
• CURLOPT_DNS_SERVERS - Set custom DNS servers (c-ares only)
• CURLOPT_DNS_INTERFACE - Bind DNS queries to specific interface
• CURLOPT_DNS_LOCAL_IP4/IP6 - Set local IP for DNS queries
• CURLOPT_TIMEOUT_MS - Overall timeout for resolution