AWS ALB Access Logs

Realm can ingest AWS Application Load Balancer (ALB) access logs delivered to an S3 bucket via SQS event notifications. ALB access logs record details about every request processed by the load balancer — ingesting these logs into Realm enables visibility into web traffic patterns, client behavior, and potential threats at the edge.

Realm supports two approaches for receiving notifications when new ALB log files land in S3:

• Option 1 — Via SQS (S3 → SQS): Configure S3 to send event notifications directly to a new SQS queue.
• Option 2 — Via SNS & SQS (S3 → SNS → SQS): Subscribe a new SQS queue to an existing SNS topic that already receives S3 events from the ALB log bucket.

Both approaches share the same IAM policy and IAM user setup. Follow the relevant section below based on your setup.

Prerequisites

ALB access logging must be enabled and configured to deliver logs to an S3 bucket. If you haven't done this yet, follow the steps below. If your ALB is already writing access logs to an S3 bucket, skip ahead to the relevant option.

Enable ALB Access Logging

Skip this step if ALB access logging is already delivering logs to an S3 bucket. Copy the bucket name and proceed to your chosen option.

Create the S3 bucket:

1. Go to S3 > Buckets
2. Click Create bucket
3. Enter the bucket name:

Bucket name: rlm-alb-access-logs-<account_id>

4. Select the same AWS Region as your load balancer
5. Click Create bucket

Attach the required bucket policy:

ALB requires a bucket policy that grants the ELB service account permission to write logs.

1. Open the bucket and go to Permissions > Bucket policy
2. Click Edit and replace the contents with the following, substituting <bucket_name> with your bucket name and <account_id> with your AWS account ID:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::elb-account-id:root"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<bucket_name>/AWSLogs/<account_id>/*"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "logdelivery.elasticloadbalancing.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<bucket_name>/AWSLogs/<account_id>/*"
    }
  ]
}

Note: The elb-account-id is region-specific. Refer to the AWS ELB access logs documentation for the correct account ID for your region.

3. Click Save changes

Enable access logging on the load balancer:

1. Go to EC2 > Load Balancers and select your ALB
2. Go to the Attributes tab and click Edit
3. Under Monitoring, enable Access logs
4. Enter the S3 bucket name and optional prefix:

S3 bucket: rlm-alb-access-logs-<account_id>

5. Click Save changes

Copy the S3 bucket name — you will need it in the steps below.

---

Option 1: Via SQS (S3 → SQS)

Use this approach if your ALB log S3 bucket does not already have an SNS topic configured for event notifications.

Step 1: Create SQS Queue

1. Go to SQS > Queues
2. Click Create Queue
3. Enter the queue name:

Name: rlm-alb-notifications

4. Click Create Queue
5. Copy the ARN of the queue — you will need it in the steps below

Step 2: Create IAM Policy

1. Go to IAM > Policies
2. Click Create Policy
3. Click JSON and replace the contents with the following, substituting <s3_bucket_arn> with the ARN of your ALB log S3 bucket and <sqs_queue_arn> with the SQS queue ARN from Step 1

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "s3ReadAlbObjects",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "<s3_bucket_arn>",
                "<s3_bucket_arn>/*"
            ]
        },
        {
            "Sid": "sqsAlbNotifications",
            "Effect": "Allow",
            "Action": [
                "sqs:ReceiveMessage",
                "sqs:DeleteMessage"
            ],
            "Resource": [
                "<sqs_queue_arn>"
            ]
        }
    ]
}

4. Click Next
5. Enter the policy details:

Name: rlm-alb-read
Description: Grant read access to ALB access log S3 bucket and permission to receive notifications from the SQS queue.

6. Click Create Policy

Step 3: Create IAM User

1. Go to IAM > Users and click Create User
2. Enter the username:

Name: rlm-alb-user

3. Click Next
4. Select Attach policies directly
5. Search for and select the rlm-alb-read policy created in Step 2
6. Click Next, then Create user
7. Open the user that was just created and copy its ARN — you will need it in Step 4
8. Go to Security credentials and click Create access key
9. Select Third-party service, check the confirmation checkbox, then click Next
10. Enter a description:

Credentials for Realm.Security to read ALB access logs from S3

11. Click Create access key
12. Copy and save the Access Key and Secret access key — you will need these when configuring the input feed in Realm

Step 4: Update SQS Access Policy

The SQS queue policy must allow S3 to publish notifications to the queue and allow the IAM user to read from it.

1. Go to SQS > Queues
2. Select the rlm-alb-notifications queue
3. Go to Queue policies and click Edit
4. Replace the policy JSON with the following, substituting <sqs_queue_arn> with the SQS queue ARN and <iam_user_arn> with the IAM user ARN from Step 3

{
  "Version": "2012-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "AllowS3Publish",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "sqs:SendMessage",
      "Resource": "<sqs_queue_arn>"
    },
    {
      "Sid": "AllowRealmUser",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<iam_user_arn>"
      },
      "Action": [
        "sqs:ChangeMessageVisibility",
        "sqs:DeleteMessage",
        "sqs:ReceiveMessage"
      ],
      "Resource": "<sqs_queue_arn>"
    }
  ]
}

5. Click Save

Step 5: Configure S3 Event Notifications

1. Go to S3 > Buckets and select your ALB log bucket
2. Go to Properties > Event notifications
3. Click Create event notification
4. Fill out the details:

Event name: RlmAlbCreateEvents
Check All object create events
Destination: SQS Queue
Select rlm-alb-notifications from the dropdown

5. Click Save changes

Step 6: Configure the Input Feed in Realm

1. In Realm, go to Sources
2. Click Add Source and select AWS ALB
3. Fill in the connection details:

Name: AWS ALB
SQS Queue URL: The URL of the rlm-alb-notifications queue
Access Key: The access key from Step 3
Secret Access Key: The secret access key from Step 3

4. Click Save

Realm will begin polling the SQS queue and fetching ALB access log files from S3 as they arrive.

---

Option 2: Via SNS → SQS

Use this approach to subscribe a new SQS queue to an SNS topic that already receives S3 ObjectCreate notifications from the ALB log bucket.

Step 1: Create and Configure SNS Topic

Skip this step if you already have an SNS topic configured to receive S3 ObjectCreate notifications from your ALB log bucket. Copy its ARN and proceed to Step 2.

Create the SNS topic:

1. Go to SNS > Topics
2. Click Create topic
3. Select Standard as the type
4. Enter the topic name:

Name: rlm-alb-notifications-topic

5. Click Create topic
6. Copy the ARN of the topic — you will need it in the steps below

Update the SNS topic access policy:

The SNS topic policy must allow S3 to publish to it.

1. On the topic details page, go to Access policy and click Edit
2. Replace the policy JSON with the following, substituting <sns_topic_arn> with the SNS topic ARN and <s3_bucket_arn> with the ARN of your ALB log S3 bucket

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowS3Publish",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "sns:Publish",
      "Resource": "<sns_topic_arn>",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "<s3_bucket_arn>"
        }
      }
    }
  ]
}

3. Click Save changes

Configure S3 to send ObjectCreate notifications to the SNS topic:

1. Go to S3 > Buckets and select your ALB log bucket
2. Go to Properties > Event notifications
3. Click Create event notification
4. Fill out the details:

Event name: RlmAlbToSnsEvents
Check All object create events
Destination: SNS Topic
Select rlm-alb-notifications-topic from the dropdown

5. Click Save changes

Step 2: Create SQS Queue

Follow the same steps as Step 1 in Option 1 above to create a queue named rlm-alb-notifications and copy its ARN.

Step 3: Create IAM Policy

Follow the same steps as Step 2 in Option 1 above to create the rlm-alb-read policy.

Step 4: Create IAM User

Follow the same steps as Step 3 in Option 1 above. Save the IAM user ARN, Access Key, and Secret access key.

Step 5: Update SQS Access Policy

The SQS queue policy must allow the SNS topic to publish messages to the queue and allow the IAM user to read from it.

1. Go to SQS > Queues
2. Select the rlm-alb-notifications queue
3. Go to Queue policies and click Edit
4. Replace the policy JSON with the following, substituting <sqs_queue_arn> with the SQS queue ARN, <sns_topic_arn> with the SNS topic ARN from Step 1, and <iam_user_arn> with the IAM user ARN from Step 4

{
  "Version": "2012-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "AllowSNSPublish",
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": "sqs:SendMessage",
      "Resource": "<sqs_queue_arn>",
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "<sns_topic_arn>"
        }
      }
    },
    {
      "Sid": "AllowRealmUser",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<iam_user_arn>"
      },
      "Action": [
        "sqs:ChangeMessageVisibility",
        "sqs:DeleteMessage",
        "sqs:ReceiveMessage"
      ],
      "Resource": "<sqs_queue_arn>"
    }
  ]
}

5. Click Save

Step 6: Subscribe SQS Queue to SNS Topic

1. Go to SNS > Topics
2. Select the SNS topic from Step 1
3. Click Create subscription
4. Fill out the details:

Protocol: Amazon SQS
Endpoint: <sqs_queue_arn>

5. Click Create subscription

The subscription will be automatically confirmed since both resources are in the same AWS account.

Step 7: Configure the Input Feed in Realm

Follow the same steps as Step 6 in Option 1 above, using the Access Key and Secret access key from Step 4.