Realm.Security Documentation

Appearance

Custom JSON Logs ​

Realm supports ingesting custom logs in JSON format from any source. Choose a transport method based on where your log source runs and how it delivers data.

TransportBest for
AWS S3Logs already delivered to an S3 bucket
Azure Blob StorageLogs already delivered to Azure Blob Storage
Splunk HECSources that support Splunk HTTP Event Collector output
Cloud HTTP (Webhook)Sources that can POST JSON over HTTPS
Cloud OTLPSources that can export JSON logs over OpenTelemetry (OTLP/HTTP)
Cloud SyslogSources that can send JSON-wrapped syslog to Realm Cloud
CollectorOn-premises sources that cannot reach Realm Cloud directly

Create a Source in Realm ​

Before configuring any transport, create a source in Realm:

  1. In Realm, go to Sources > Add Source.
  2. Enter a name for your source.
  3. Set Product Format to Custom JSON.
  4. Click Save.

Then follow the section below for your chosen transport method.

AWS S3 ​

Use this transport when your log source delivers JSON files to an S3 bucket.

Add an AWS S3 Input Feed ​

  1. On the Select Transport Method step in the Add Source flow, or from the source page, click Add Input Feed.
  2. Select AWS S3 and give it a name.
  3. Enter the AWS credentials and S3/SQS configuration:

Access Key ID: <iam_access_key>
Secret Access Key: <iam_secret_key>
SQS Queue URL: <sqs_queue_url>
S3 Bucket Name: <bucket_name>
AWS Region: <region>

  1. Click Save.

Configure AWS Resources ​

Set up the SQS queue, IAM policy, IAM user, and S3 event notifications to deliver object-create events to Realm. Follow the AWS S3 source setup guide for complete instructions — the setup is identical regardless of log format.

Azure Blob Storage ​

Use this transport when your log source delivers JSON files to an Azure Storage Container.

Add an Azure Blob Storage Input Feed ​

  1. On the Select Transport Method step in the Add Source flow, or from the source page, click Add Input Feed.
  2. Select Azure Blob Storage and give it a name.
  3. Enter the Azure credentials and storage configuration:

Tenant ID: <tenant_id>
Client ID: <client_id>
Client Secret: <client_secret>
Event Hub Namespace: <namespace>.servicebus.windows.net
Event Hub Name: <event_hub_name>
Storage Account Name: <storage_account_name>
Container Name: <container_name>

  1. Click Save.

Configure Azure Resources ​

Set up the App Registration, Event Hub, Storage Account, and Event Grid notification subscription so Azure notifies Realm when new blobs are created. Follow the Azure Blob Storage source setup guide for complete instructions — the setup is identical regardless of log format.

Splunk HEC ​

Use this transport when your log source can forward events using the Splunk HTTP Event Collector (HEC) protocol.

Add a Splunk HEC Input Feed ​

  1. On the Select Transport Method step in the Add Source flow, or from the source page, click Add Input Feed.
  2. Select Splunk HEC and give it a name.
  3. Enter a token if you have one or leave it empty to have the system generate a random token for you.
  4. Click Save.
  5. Click ... > Edit on the newly created input feed.
  6. Click on eye icon in the token field to view token value.
  7. Copy the HEC Endpoint URL and Token — you will need these when configuring your source vendor product.

Configure Your Log Source ​

Configure your log source to forward JSON events to the Realm HEC endpoint.

For sources with native HEC support or another pipeline product (e.g., Cribl, Fluentd, Logstash), set the HEC endpoint URL and token in the output connector configuration of your pipeline tool.

Note: Realm expects JSON-formatted events. Ensure your HEC output is configured to send raw JSON rather than wrapped Splunk event envelopes where possible.

For full Splunk HEC configuration details and troubleshooting steps, see the Splunk HEC Source guide.

Cloud HTTP (Webhook) ​

Use this transport when your log source can POST JSON payloads over HTTPS to a webhook URL.

Add a Cloud HTTP Input Feed ​

  1. On the Select Transport Method step in the Add Source flow, or from the source page, click Add Input Feed.
  2. Select Cloud HTTP and give it a name.
  3. Select an Authentication Method (Auth Token, HMAC, or Shared Secret) and fill in its fields. See Authentication Methods to choose one.
  4. Click Add.
  5. Click ... > View on the input feed and copy the Hostname.
  6. If you selected Auth Token, click ... > Edit on the input feed, then click the eye icon in the Token field to view the token value.
  7. By default, the webhook uses a public CA. You can switch to using a Realm private CA from the new or edit input feed form. After saving the feed, the Realm CA pem file will be shown on the View page. You can copy and paste it in to the source product.

Configure Your Log Source ​

Configure your application or log pipeline to POST JSON events to the Realm Hostname, authenticating with the method configured on the input feed. For example, with Auth Token:

shell
curl -X POST "<hostname>" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{"timestamp":"2024-01-01T00:00:00Z","event":"login","user":"alice"}'

Payload requirements:

  • Content-Type must be application/json
  • The body must be a valid JSON object or a newline-delimited sequence of JSON objects (one per line)
  • With Shared Secret, the body must be a single JSON object that contains the shared secret field
  • Each event must fit within the maximum payload size shown on the input feed details page

Note: Realm returns HTTP 200 on successful ingestion. Retry on 429 (rate limited) or 5xx responses using exponential backoff.

For HMAC and Shared Secret request examples and troubleshooting steps, see the Cloud HTTP guide.

Cloud OTLP ​

Use this transport when your log source or pipeline can export JSON logs using the OpenTelemetry Protocol (OTLP) over HTTP, such as an OpenTelemetry Collector.

Add a Cloud OTLP Input Feed ​

  1. On the Select Transport Method step in the Add Source flow, or from the source page, click Add Input Feed.
  2. Select Cloud OTLP and give it a name.
  3. Enter a token if you have one or leave it empty to have the system generate a random token for you.
  4. Click Add.
  5. Click ... > View on the input feed and copy the Hostname.
  6. Click ... > Edit on the input feed, then click the eye icon in the Token field to view the token value.
  7. Copy the Token — you will need it and the Hostname when configuring your OTLP exporter.
  8. By default, the endpoint uses a public CA. You can switch to using a Realm private CA from the new or edit input feed form. After saving the feed, the Realm CA pem file will be shown on the View page. You can copy and paste it in to your OTLP exporter configuration.

Configure Your Log Source ​

Configure your OTLP exporter to send JSON log records to the Realm Hostname with the token in an Authorization: Bearer <token> header.

Note: Cloud OTLP accepts OTLP over HTTP only. gRPC is not supported.

For a sample OpenTelemetry Collector configuration and troubleshooting steps, see the Cloud OTLP guide.

Cloud Syslog ​

Use this transport when your log source can send syslog messages with JSON payloads directly to Realm Cloud.

Add a Cloud Syslog Input Feed ​

  1. On the Select Transport Method step in the Add Source flow, or from the source page, click Add Input Feed.
  2. Select Cloud Syslog and give it a name.
  3. Click Save.
  4. Click ... > Edit on the newly created input feed.
  5. Copy the Syslog Hostname, Port, Protocol — you will need these to configure your log source.
  6. By default, the Cloud syslog uses a public CA. You can switch to using a Realm private CA from the new or edit input feed form. After saving the feed, Private CA value will be shown. You can copy and paste it in to the source product.

Configure Your Log Source ​

Update your log source to send syslog messages to the Realm Cloud syslog endpoint. The syslog message body must contain a valid JSON object.

Example syslog message format (RFC 5424):

<134>1 2024-01-01T00:00:00Z myhost myapp 1234 - - {"event":"login","user":"alice","result":"success"}
  • The syslog header fields (facility, severity, hostname, app name) are ignored during ingestion — only the message body (the JSON object) is parsed.
  • Configure TLS using the Realm CA Certificate saved above.

For Cloud Syslog connection details, see the Cloud Syslog guide. For rsyslog, see the Rsyslog guide for TLS configuration examples that apply here as well.

Collector ​

Use this transport when your JSON log source is on-premises and cannot send data directly to Realm Cloud. The Realm Collector runs in your environment and forwards logs on behalf of the source.

Collector support ingesting

  • JSON logs that are new line delimited or
  • JSON log wrapped in Syslog RFC-5424 over TCP or UDP.

Add a Collector Input Feed ​

  1. On the Select Transport Method step in the Add Source flow, select Collector > create a new collector or select an existing one > click Continue to add a stream.
  2. Configure the collector stream:
    • Name: Give the stream a name.
    • Protocol: Select TCP or UDP.
    • Port: Use the preconfigured port or change it to suit your environment.
    • Framing Trailer: Select the framing trailer that defines the delimiter for log messages.
  3. Click Create Stream.

Note: Alternatively, create a stream from the Collectors page by selecting a collector and clicking Add Stream.

  1. Configure the collector stream:
    • Name: Give the stream a name.
    • Protocol: Select TCP or UDP.
    • Port: Use the preconfigured port or change it to suit your environment.
    • Framing Trailer: Select the framing trailer that defines the delimiter for log messages.
    • Source: Select the source node in the Realm Fabric to which the logs should be forwarded to.
  2. Click Create Stream.

Configure the Realm Collector ​

If you have not yet deployed the Realm Collector in your environment, follow the Collector setup guide first.

Once the Collector is running, follow the steps above to add a Stream to the collector.

Note: The Collector is the recommended option when your log source is behind a firewall or on a private network that cannot reach Realm Cloud endpoints.