Configuration

The configuration for each CCF node must be contained in a single JSON configuration file specified to the executable via the --config /path/to/config/file argument.

Tip

JSON configuration samples:

A single configuration file can be verified using the executable, but without launching a node, using the --check option:

$ /opt/ccf/bin/js_generic --config /path/to/config/file --check

Configuration Options

network

This section includes configuration for the interfaces a node listens on (for both client and node-to-node communications). Required Type: object

network.node_to_node_interface

Addresses (host:port) to listen on for incoming node-to-node connections (e.g. internal consensus messages). IPv6 literals must be bracketed, e.g. [::1]:8081. Required Type: object

bind_address
Local address the node binds to and listens on.
Required
Type: string
published_address
The published node address advertised to other nodes. This must be different on each node. If omitted, defaults to bind_address.
Type: string

network.rpc_interfaces

Interfaces to listen on for incoming client TLS connections, as a dictionary from unique interface name to RPC interface information. Required Type: object

network.rpc_interfaces.[name]

Type: object

bind_address
Local address the node binds to and listens on.
Required
Type: string
published_address
The published RPC address advertised to clients. If omitted, defaults to bind_address.
Type: string
protocol
The RPC protocol used (udp | tcp).
Type: string
Default: "tcp"
app_protocol
The application protocol used by all sessions on this interface.
Type: string
Default: "HTTP1"
max_open_sessions_soft
The maximum number of active client sessions on that interface after which clients will receive an HTTP 503 error.
Type: integer
Default: 1000
max_open_sessions_hard
The maximum number of active client sessions on that interface after which clients sessions will be terminated, before the TLS handshake is complete. Note that its value must be greater than the value of max_open_sessions_soft.
Type: integer
Default: 1010
network.rpc_interfaces.[name].http_configuration

Type: object

max_body_size
Maximum size (size string) of a single HTTP request body. Submitting a request with a payload larger than this value will result in the client session being automatically closed.
Type: string
Default: "1MB"
max_header_size
Maximum size (size string) of a single HTTP request header (key or value). Submitting a request with a header larger than this value will result in the client session being automatically closed.
Type: string
Default: "16KB"
max_headers_count
Maximum number of headers in a single HTTP request. Submitting a request with more headers than this value will result in the session being automatically closed.
Type: integer
Default: 256
max_concurrent_streams_count
HTTP/2 only. Maximum number of concurrent streams allowed per session.
Type: integer
Default: 100
Minimum: 1
initial_window_size
HTTP/2 only. Initial size (size string) of flow-control window per stream.
Type: string
Default: "64KB"
max_frame_size
HTTP/2 only. Maximum allowed size (size string) of HTTP/2 frames (min: 16KB, max: 16MB).
Type: string
Default: "16KB"
network.rpc_interfaces.[name].endorsement

Type: object

authority
The type of endorsement for the TLS certificate used in client sessions. If the endorsement is not available, client sessions will be terminated, before the TLS handshake is complete. ‘Node’ means self-signed, ‘Service’ means service-endorsed, ‘ACME’ is deprecated, ‘Unsecured’ means unencrypted traffic and no endorsement authority.
Required
Type: string
Values: "Node", "Service", "ACME", "Unsecured"
Default: "Service"
network.rpc_interfaces.[name].accepted_endpoints

An array of regular expressions that specify which URL paths are served on the interface. Optional; if not present, all paths are served. Type: array

[item]
Type: string
forwarding_timeout_ms
Timeout for forwarded RPC calls (in milliseconds).
Type: integer
Default: 3000
network.rpc_interfaces.[name].redirections

Configure how redirect responses should be produced on this interface. If this is omitted, then forwarding will be used instead. Type: object

network.rpc_interfaces.[name].redirections.to_primary

Configures how the Location header should be populated, when requests arrive on this interface that must be served by a primary while the receiving node is not a primary. Type: object

kind
Required
Values: "NodeByRole", "StaticAddress"
network.rpc_interfaces.[name].redirections.to_primary.target (network.rpc_interfaces.[name].redirections.to_primary.kind is "NodeByRole")

(Only applies if network.rpc_interfaces.[name].redirections.to_primary.kind is "NodeByRole") Type: object

role
Values: "primary", "backup"
Default: "primary"
network.rpc_interfaces.[name].redirections.to_primary.target (network.rpc_interfaces.[name].redirections.to_primary.kind is "StaticAddress")

(Only applies if network.rpc_interfaces.[name].redirections.to_primary.kind is "StaticAddress") Required Type: object

address
Required
Type: string
network.rpc_interfaces.[name].redirections.to_backup

Configures how the Location header should be populated, when requests arrive on this interface that must be served by a backup while the receiving node is currently a primary. Type: object

kind
Required
Values: "NodeByRole", "StaticAddress"
network.rpc_interfaces.[name].redirections.to_backup.target (network.rpc_interfaces.[name].redirections.to_backup.kind is "NodeByRole")

(Only applies if network.rpc_interfaces.[name].redirections.to_backup.kind is "NodeByRole") Type: object

role
Values: "primary", "backup"
Default: "primary"
network.rpc_interfaces.[name].redirections.to_backup.target (network.rpc_interfaces.[name].redirections.to_backup.kind is "StaticAddress")

(Only applies if network.rpc_interfaces.[name].redirections.to_backup.kind is "StaticAddress") Required Type: object

address
Required
Type: string
network.rpc_interfaces.[name].enabled_operator_features

An array of features which should be enabled on this interface, providing access to endpoints with specific security or performance constraints. Type: array

[item]
Type: string
Values: "SnapshotRead", "LedgerChunkRead", "SnapshotCreate"

command

This section includes configuration of how the node should start (either start, join or recover) and associated information. Required Type: object

type
Type of CCF node.
Required
Type: string
Values: "Start", "Join", "Recover"
service_certificate_file
For Start and Recover nodes, path to which service certificate will be written to on startup. For Join nodes, path to the certificate of the existing service to join.
Type: string
Default: "service_cert.pem"

command.start

(Only applies if command.type is "Start") Required Type: object

command.start.constitution_files

List of constitution files. These typically include actions.js, validate.js, resolve.js and apply.js. Required Type: array

[item]
Type: string
initial_service_certificate_validity_days
Initial validity period (days) for service certificate.
Type: integer
Default: 1
Minimum: 1
service_subject_name
Subject name to include in service certificate. Can only be set once on service start.
Type: string
Default: "CN=CCF Service"
command.start.cose_signatures

If specified, both issuer and subject are required. Type: object

issuer
Issuer, set in CWT_Claims of COSE ledger signatures. Can only be set once on service start.
Type: string
subject
Subject, set in CWT_Claims of COSE ledger signatures. Can only be set once on service start.
Type: string
command.start.members

List of initial consortium members files, including identity certificates, public encryption keys and member data files. Required Type: array

command.start.members.[item]

Type: object

certificate_file
Path to member x509 identity certificate (PEM).
Required
Type: string
encryption_public_key_file
Path to member encryption public key (PEM).
Type: string | null
data_json_file
Path to member data file (JSON).
Type: string | null
recovery_role
Whether the member acts as a recovery participant and gets assigned a share that can contribute towards a recovery threshold or as an owner and gets assigned a full recovery key.
Type: string
Values: "NonParticipant", "Participant", "Owner"
command.start.service_configuration

Type: object

recovery_threshold
Number of recovery members required to recover the service. Note that if the recovery threshold is set to 0, it is automatically set to the number of recovery members specified in ‘members’.
Required
Type: integer
Minimum: 0
maximum_node_certificate_validity_days
The maximum number of days allowed for node certificate validity period.
Type: integer
Default: 365
Minimum: 1
maximum_service_certificate_validity_days
The maximum number of days allowed for service certificate validity period.
Type: integer
Default: 365
Minimum: 1
recent_cose_proposals_window_size
Size of the window of COSE proposals recently received by the service, kept for the purpose of replay protection.
Type: integer
Default: 100
Minimum: 1

command.join

(Only applies if command.type is "Join") Required Type: object

target_rpc_address
Address (host:port) of a node of the existing service to join. IPv6 literals must be bracketed, e.g. [2001:db8::1]:8080.
Required
Type: string
retry_timeout
Interval (time string) at which the node sends join requests to the existing service. This should be less than the value of ‘consensus.election_timeout’ set on the primary node of the existing service to join.
Type: string
Default: "1000ms"
follow_redirect
Whether to follow redirects to the primary node of the existing service to join.
Type: boolean
Default: true
fetch_recent_snapshot
Whether to ask the target for a snapshot before joining. The node will ask the target what their latest snapshot is, and will fetch it via RPC before launching. Should generally only be turned off for specific test cases.
Type: boolean
Default: true
fetch_snapshot_max_attempts
Maximum number of attempts to fetch a recent snapshot from the target node.
Type: integer
Default: 3
Minimum: 1
fetch_snapshot_retry_interval
Interval (time string) between retries to fetch a recent snapshot from the target node.
Type: string
Default: "1000ms"
fetch_snapshot_max_size
Maximum size of snapshot this node is willing to fetch.
Type: string
Default: "10GB"
host_data_transparent_statement_path
Path to a SCITT Transparent Statement over the attested host_data of the node.
Type: string | null
Default: null

command.recover

(Only applies if command.type is "Recover") Required when command.type is Recover. Type: object

initial_service_certificate_validity_days
Initial validity period (days) for service certificate.
Type: integer
Default: 1
Minimum: 1
previous_service_identity_file
Path to the previous service certificate (PEM) file.
Required
Type: string

node_certificate

This section includes configuration for the node x509 identity certificate. Type: object

subject_name
Subject name to include in node certificate.
Type: string
Default: "CN=CCF Node"

node_certificate.subject_alt_names

List of iPAddress: or dNSName: strings to include as Subject Alternative Names (SAN) in node certificates. If none are set, the node certificate will automatically include the value of the main RPC interface ‘published_address’. Type: array

[item]
Type: string
curve_id
Elliptic curve to use for node identity key.
Type: string
Values: "Secp384R1", "Secp256R1"
Default: "Secp384R1"
initial_validity_days
Initial validity period (days) for node certificate.
Type: integer
Default: 1
Minimum: 1

node_data_json_file

Path to file (JSON) containing initial node data. It is intended to store correlation IDs describing the node’s deployment, such as a VM name or Pod identifier. Type: string | null

attestation

This section includes configuration for the attestation for AMD SEV-SNP platform. Type: object

snp_security_policy_file
Path to file containing the security policy (SEV-SNP only), can contain environment variables, such as $UVM_SECURITY_CONTEXT_DIR.
Type: string | null
snp_uvm_endorsements_file
Path to file containing UVM endorsements as a base64-encoded COSE Sign1 (SEV-SNP only). Can contain environment variables, such as $UVM_SECURITY_CONTEXT_DIR.
Type: string | null
snp_endorsements_file
Path to file containing AMD VCEK hardware endorsements (a PEM certificate chain), base-64 encoded. Can contain environment variables, such as $UVM_SECURITY_CONTEXT_DIR. Will be used in preference to snp_endorsements_servers if the tcbm in this file matches that of the attestation.
Type: string | null

attestation.snp_endorsements_servers

List of servers used to retrieve attestation report endorsement certificates (SEV-SNP only). The first server in the list is always used and other servers are only specified as fallback. Type: array

attestation.snp_endorsements_servers.[item]

Type: object

type
Type of server used to retrieve attestation report endorsement certificates (SEV-SNP only).
Type: string
Values: "Azure", "AMD", "THIM"
Default: "Azure"
url
Server URL used to retrieve attestation report endorsement certificates.
Type: string
max_retries_count
Maximum number of retries to fetch endorsements from the server.
Type: integer
Default: 10

service_data_json_file

Path to file (JSON) containing initial service data. It is used when the node starts in ‘Start’ or ‘Recover’ mode and is intended to store arbitrary information about the service. Type: string | null

ledger

This section includes configuration for the ledger directories and files. Type: object

directory
Path to main ledger directory.
Type: string
Default: "ledger"

ledger.read_only_directories

Paths to read-only ledger directories. Note that only ‘.committed’ files will be read from these directories. Type: array

[item]
Type: string
chunk_size
Minimum size (size string) of the current ledger file after which a new ledger file (chunk) is created.
Type: string
Default: "5MB"

snapshots

This section includes configuration for the snapshot directories and files. Type: object

directory
Path to snapshots directory.
Type: string
Default: "snapshots"
tx_count
Number of transactions after which a snapshot is automatically generated.
Type: integer
Default: 10000
Minimum: 1
min_tx_count
Minimum number of transactions that must have elapsed since the last snapshot before a time-based snapshot can be triggered. Has no effect if time_interval is not set. If lower than 2 while time_interval is enabled, snapshots will be triggered indefinitely.
Type: integer
Default: 2
time_interval
Time interval after which a snapshot should be triggered, provided more than min_tx_count transactions have elapsed since the last snapshot. Set this to 0s to disable time-based snapshotting.
Type: string
Default: "0s"
read_only_directory
Path to read-only snapshots directory. Deprecated: this option is deprecated and will be removed in a future release. Use join.fetch_recent_snapshot and snapshots.backup_fetch to have joining and/or backup nodes automatically fetch snapshots from the primary node instead.
Type: string | null

snapshots.backup_fetch

Configuration for automatic snapshot fetching by backup nodes. Type: object

enabled
If true, backup nodes will automatically fetch snapshots from the primary when snapshot evidence is detected.
Type: boolean
Default: false
max_attempts
Maximum number of fetch attempts before giving up.
Type: integer
Default: 3
Minimum: 1
retry_interval
Delay between retry attempts.
Type: string
Default: "1000ms"
target_rpc_interface
Name of the RPC interface on the primary node to use for downloading snapshots. Must have the SnapshotRead feature enabled.
Type: string
Default: "primary_rpc_interface"
max_size
Maximum size of snapshot this node is willing to fetch.
Type: string
Default: "200MB"

files_cleanup

This section includes configuration for periodic cleanup of old files (snapshots, ledger chunks). Type: object

max_snapshots
Maximum number of committed snapshot files to retain. When the number of committed snapshots exceeds this value, the oldest snapshots are deleted. Must be at least 1 if set. If null or unset, no automated snapshot garbage collection is performed.
Type: integer | null
Default: null
Minimum: 1
max_committed_ledger_chunks
Maximum number of committed ledger chunk files to retain in the main ledger directory. When the number of committed chunks exceeds this value, the oldest chunks are deleted, but only after verifying that an identical copy (by SHA-256 digest) exists in at least one read-only ledger directory. Chunks whose entries extend to or beyond the sequence number of the newest committed snapshot are never deleted, ensuring a complete ledger history from that snapshot for disaster recovery. Requires at least one ledger.read_only_directories entry; the node will refuse to start otherwise. If null or unset, no automated ledger chunk garbage collection is performed.
Type: integer | null
Default: null
interval
Time interval at which to scan and delete old committed files (snapshots and ledger chunks) that exceed the configured retention limits. This periodic cleanup executes regardless of the node’s status (primary or backup).
Type: string
Default: "30s"

identity_history_fetch

Configuration for the network identity endorsement chain fetch. Type: object

max_attempts
Maximum number of fetch attempts before giving up.
Type: integer
Default: 100
Minimum: 1
retry_interval
Delay between retry attempts.
Type: string
Default: "100ms"

logging

This section includes configuration for the logging of the node process. Type: object

host_level
Deprecated. Accepted for compatibility but ignored. Use the --log-level command-line option instead.
Type: string
Values: "Trace", "Debug", "Info", "Fail", "Fatal"
format
If Json, node logs will be formatted as JSON.
Type: string
Values: "Text", "Json"
Default: "Text"

consensus

This section includes configuration for the consensus protocol (note: should be the same for all other nodes in the service). Type: object

message_timeout
Maximum interval (time string) at which the primary node sends messages to backup nodes to maintain its primary-ship. This should be set to a significantly lower value than ‘election_timeout’.
Type: string
Default: "100ms"
election_timeout
Maximum timeout (time string) after which backup nodes that have not received any message from the primary node (or voted for a candidate) will trigger a new election. This timeout is also used by candidates to restart unsuccessful elections. This should be set to a significantly greater value than ‘message_timeout’ plus the expected network delay.
Type: string
Default: "5000ms"
max_uncommitted_tx_count
Maximum number of uncommitted transactions allowed before the primary refuses new transactions. Unlimited if set to 0.
Type: integer
Default: 10000

ledger_signatures

This section includes configuration for the ledger signatures emitted by this node (note: should be the same for all other nodes in the service). Transaction commit latency in a CCF network is primarily a function of signature frequency. A network emitting signatures more frequently will be able to commit transactions faster, but will spend a larger proportion of its execution resources creating and verifying signatures. Setting signature frequency is a trade-off between transaction latency and throughput. Type: object

tx_count
Number of transactions after which a signature transaction is automatically generated.
Type: integer
Default: 5000
Minimum: 1
delay
Maximum duration after which a signature transaction is automatically generated.
Type: string
Default: "1000ms"

jwt

This section includes configuration for JWT issuers automatic refresh. Type: object

key_refresh_interval
Interval at which JWT keys for issuers registered with auto-refresh are automatically refreshed.
Type: string
Default: "30min"
key_refresh_max_response_size
Maximum response body size accepted when fetching OpenID metadata and JWKS for JWT issuer auto-refresh.
Type: string
Default: "1MB"

output_files

This section includes configuration for additional files output by the node. Type: object

node_certificate_file
Path to self-signed node certificate output by node on startup.
Type: string
Default: "nodecert.pem"
pid_file
Path to file in which process identifier (PID) will be written to on startup.
Type: string
Default: "my_node.pid"
node_to_node_address_file
Path to file in which node address (hostname and port) will be written to on startup. This option is particularly useful when binding to port 0 and getting auto-assigned a port by the OS. No file is created if this entry is not specified.
Type: string
rpc_addresses_file
Path to file in which all RPC addresses (hostnames and ports) will be written to on startup. This option is particularly useful when binding to port 0 and getting auto-assigned a port by the OS. No file is created if this entry is not specified.
Type: string

tick_interval

Interval at which the enclave time will be updated by the host (modify with care!). Type: string Default: "10ms"

slow_io_logging_threshold

Maximum duration of I/O operations (ledger and snapshots) after which slow operations will be logged to node log. Type: string Default: "10ms"

node_client_interface

Address to bind to for node-to-node client connections. If unspecified, this is automatically assigned by the OS. This option is particularly useful for testing purposes (e.g. establishing network partitions between nodes). Type: string | null

client_connection_timeout

Maximum duration after which unestablished client connections will be marked as timed out and either re-established or discarded. Type: string Default: "2000ms"

idle_connection_timeout

Timeout for idle connections. Null is a valid option, and means idle connections are retained indefinitely. Type: string | null Default: "60s"

worker_threads

Number of additional threads processing incoming client requests. Type: integer Default: 0 Minimum: 0

memory

This section includes configuration for the host-enclave ring-buffer memory (modify with care!). Type: object

circuit_size
Size (size string) of the internal host-enclave ringbuffers (must be a power of 2).
Type: string
Default: "16MB"
max_msg_size
Maximum size (size string) for a message sent over the ringbuffer. Messages may be split into multiple fragments, but this limits the total size of the sum of those fragments.
Type: string
Default: "64MB"
max_fragment_size
Maximum size (size string) of individual ringbuffer message fragments. Messages larger than this will be split into multiple fragments.
Type: string
Default: "256KB"

ignore_first_sigterm

Ignore the first SIGTERM received by the process, but set “stop_notice” in /node/state. Type: boolean Default: false

node_to_node_message_limit

Cap at which node-to-node message channels will be closed, and a new channel will be created. Can be used to limit use of single cryptographic key. Type: integer Default: 23726566 Minimum: 0

historical_cache_soft_limit

Historical queries cache soft limit (as size string). Type: string Default: "512MB"

sealing_recovery

Optional. Controls the behaviour of sealing-based recovery. If set, enables sealing of ledger secrets using platform derived key capabilities (e.g. AMD SEV-SNP derived keys). This allows a future recovering node to unilaterally recover its ledger secrets on restart without needing to reconstruct them from recovery shares. Type: object

sealing_recovery.location

Required Type: object

name
Required
Type: string
address
Required
Type: string

sealing_recovery.recovery_decision_protocol

Type: object

sealing_recovery.recovery_decision_protocol.expected_locations

List of locations that the recovery_decision_protocol expects to be part of the previous network. Required Type: array

sealing_recovery.recovery_decision_protocol.expected_locations.[item]

Type: object

name
Required
Type: string
address
Required
Type: string
message_retry_timeout
Type: string
Default: "100ms"
failover_timeout
Timeout duration before failover forcibly advances the recovery_decision_protocol, allowing recovery to proceed even in the presence of unresponsive nodes. Set to 0 to disable failover.
Type: string
Default: "2000ms"

IPv6 Addresses

Note

IPv6 support is currently experimental.

Every address field accepts an IPv4 address, an IPv6 address, or a fully qualified domain name. IPv6 literals must be written in bracketed form, [host]:port, so that the colons in the address are not mistaken for the host/port separator. This applies to all address fields, including:

  • network.node_to_node_interface.bind_address and published_address

  • network.rpc_interfaces.[name].bind_address and published_address

  • command.join.target_rpc_address

For example, to bind a node’s interfaces to the IPv6 loopback address ::1 and publish an address on a different IPv6 host:

{
  "network": {
    "node_to_node_interface": { "bind_address": "[::1]:8081" },
    "rpc_interfaces": {
      "primary_rpc_interface": {
        "bind_address": "[::1]:8080",
        "published_address": "[2001:db8::1]:12345"
      }
    }
  }
}

A node joining over IPv6 sets command.join.target_rpc_address to the bracketed address of an existing node:

{
  "command": {
    "join": { "target_rpc_address": "[2001:db8::1]:12345" }
  }
}

When a node is identified by an IPv6 literal, the matching node_certificate.subject_alt_names entry uses the iPAddress: prefix with the unbracketed address, for example "iPAddress:2001:db8::1".

Note

published_address defaults to bind_address when omitted, and is the address embedded in redirect URLs returned to clients. Brackets are added automatically where required, so a published IPv6 address appears in a redirect as https://[2001:db8::1]:12345/....

Operator Features

The enabled_operator_features configuration field allows enabling or disabling specific operator features on a per-interface basis.

Currently supported features are:

  1. ‘SnapshotRead’: gates access to endpoints used to fetch snapshots directly from nodes (GET /node/snapshot, HEAD /node/snapshot, GET /node/snapshot/{snapshot_name} and HEAD /node/snapshot/{snapshot_name}).

  2. ‘LedgerChunkRead’: gates access to endpoints used to retrieve ledger chunks (GET /node/ledger_chunk, HEAD /node/ledger_chunk, GET /node/ledger_chunk/{chunk_name} and HEAD /node/ledger_chunk/{chunk_name}).

  3. ‘SnapshotCreate’: gates access to the operator endpoint used to create a snapshot on the next signature transaction (POST /node/snapshot:create).

Since these operations may require disk IO and produce large responses, these features should not be enabled on interfaces with public access, and instead restricted to interfaces with local connectivity for node-to-node and operator access.

Note

  • Size strings are expressed as the value suffixed with the size in bytes (B, KB, MB, GB, TB, as factors of 1024), e.g. "20MB", "100KB" or "2048" (bytes).

  • Time strings are expressed as the value suffixed with the duration (us, ms, s, min, h), e.g. "1000ms", "10s" or "30min".

Upgrading to COSE-Only Ledger Signatures

By default, CCF nodes emit dual ledger signatures: a traditional node signature and a COSE Sign1 signature. Applications control this via the ccf::get_ledger_sign_mode() weak-symbol callback declared in ccf/node/ledger_sign_mode.h, which returns one of three modes:

  • Dual (default) — emit both signature types, accept all joiners.

  • CoseAllowDualJoin — emit only COSE signatures, but still accept join requests from Dual-mode nodes. Use during rolling upgrades.

  • CoseOnly — emit only COSE signatures, reject Dual-mode joiners. Final state after upgrade.

The mode is set at link time by providing a strong definition in the application binary. Joining nodes advertise their signing mode in the join request.

A rolling upgrade from Dual to CoseOnly is a two-step process:

  1. CoseAllowDualJoin. Deploy a binary that returns CoseAllowDualJoin. Replace nodes one at a time. During this phase, new nodes running the old Dual binary can still join as replacements.

  2. CoseOnly. Once all nodes are upgraded, deploy a binary that returns CoseOnly. Replace nodes again. After this, Dual joiners are rejected.