Overview

Export your notification logs to Amazon S3 for analytics, debugging, and compliance.

On the old S3 connector (v1.0)? v1.0 will be deprecated over time. v2.0 exports all log types and notification analytics and closes gaps in error logging that exist in v1.0—see the v1.0 doc for the migration path.

Export your SuprSend notification data directly to your S3 bucket. Build custom analytics dashboards, debug delivery issues, surface errors to your customers, or maintain compliance audit trails—all with data you fully own and control.

---

How it works

SuprSend syncs your notification data to S3 every 5 minutes as Parquet files, partitioned by hour. By default, each data point goes into its own top-level folder (the per_type path layout)—see Compression and path layout for the exact folder structure and how to change it.

A few details worth knowing:

• Updates rewrite files in place. Each sync re-writes the hourly Parquet files whose data has changed. Query engines like Athena, Spark, and Presto see the new state automatically. For warehouses that don’t overwrite rows (e.g., BigQuery), use the updated_at column to pick the latest version.
• Parquet is read natively by every common query engine and warehouse. For Athena specifically, see Query with Athena.
• Data is encrypted in transit (TLS 1.2+) and at rest (SSE-S3 or SSE-KMS).
• If you pause the sync, the pause period backfills automatically when you resume.

---

What you can export

You can pick which data points to sync based on what you’re trying to do. Most teams start with Messages (analytics and delivery troubleshooting) and add Workflow Executions and Requests when they need to debug end-to-end. Each data point maps to its own table in your warehouse or query engine.

Data points

Data point	What’s in it	Use it for
Messages	Delivery status, engagement, vendor info, failures	Analytics, delivery troubleshooting
Workflow Executions	Step-by-step workflow logs	Debugging workflow level errors or computations like user preferences
Requests	API payloads and their responses	API debugging, Audit trails, Workflow Trigger level errors

You’ll choose which data points to export when you create the connector—open the SuprSend dashboard at app.suprsend.com and go to Connectors → Add connector → Amazon S3 v2.0 to see the selection panel:

Errors logged in each table:

Table	Errors
Requests	API level errors, workflow trigger level errors (condition mismatch, user not found, etc.)
Workflow Executions	Workflow level errors (dynamic variables in workflow could not be resolved, template rendering failed, webhook returned a 404 response, etc.)
Messages	Delivery failures

Table schema

Columns and types below match the Parquet files SuprSend writes to S3. Several semantically-typed fields (JSON, boolean, integer, timestamp) are stored as string in Parquet—they’re called out in the description so you can cast them in your queries.

Requests

Column name	Description	Datatype
workspace_key	SuprSend workspace identifier	string
created_at	Time when the request was received by SuprSend (UTC)	timestamp
updated_at	Time when this entry was last updated	timestamp
api_type	Entity type for the API call	string
api_name	Workflow, event, or broadcast name passed in the API call	string
wf_trigger_type	Workflow trigger type	string
distinct_id_list	List of user distinct_id values, or object_type/id for object recipients	array<string>
actor	Actor passed in the event or workflow API request	string
tenant_id	Tenant ID for which the API request was sent	string
payload	Input payload passed in the trigger, including API call details (JSON serialized as string)	string
response	HTTP API response (JSON serialized as string)	string
metadata	SDK, machine, and location information for the request (JSON serialized as string)	string
errors	Request-level errors. Each element has error_code, error_description, error_type, severity, workflow_slug	array<struct>
executions	Workflow or broadcast executions triggered by this API call. Each element has distinct_id, exec_id, workflow_slug. exec_id joins to workflow_executions.execution_id	array<struct>
idempotency_key	Idempotency key passed in the API request. A UUID is generated if not provided	string
status	Status of the API request	string

status can have these values:

• completed: request is successfully processed.
• failure: request failed to process due to some error.
• partial_failure: request has been partially processed with some failure or has an acceptable warning (like workflow conditions evaluated to false).

Workflow Executions

Column name	Description	Datatype
workspace_key	SuprSend workspace identifier	string
created_at	Time when the workflow execution started	timestamp
updated_at	Time when the workflow step log was last updated	timestamp
execution_id	Unique identifier for a workflow execution	string
recipient_distinct_id	User distinct_id; for objects, object_type/id	string
tenant_id	Unique identifier of the tenant	string
idempotency_key	Idempotency key passed in the API request. A UUID is generated if not provided	string
parent_object_execution_id	Execution ID of the parent object when triggered on an object	string
parent_object	Parent object_type/id when the workflow runs for subscribers	string
workflow_slug	Unique slug of the workflow	string
workflow_version	Version of the workflow	string
node_id	Unique identifier of the node	string
node_name	Name of the node	string
node_type	Type of the node	string
execution_stage	Current execution stage of the node	string
status	Log status of the step (error, warning, info)	string
message	Short description of the event or error at this stage	string
properties	Additional input or output data for the node execution (JSON serialized as string)	string

Messages

Column name	Description	Datatype
workspace_key	SuprSend workspace identifier	string
created_at	Time when this entry was created	timestamp
updated_at	Time when the message status was last updated	timestamp
wf_execution_id	Workflow execution this message belongs to. Joins to workflow_executions.execution_id	string
broadcast_execution_id	Broadcast execution this message belongs to	string
message_id	Message identifier; present only when there is no execution error	string
recipient_distinct_id	User distinct_id; for objects, object_type/id	string
tenant_id	Unique identifier of the tenant	string
idempotency_key	Idempotency key passed in the API request. A UUID is generated if not provided	string
parent_object	Parent object_type/id when the workflow runs for subscribers	string
parent_object_execution_id	Execution ID of the parent object when triggered on an object	string
workflow_slug	Unique slug of the workflow	string
template_name	Name of the template group	string
template_slug	Unique slug of the template	string
message_status	Delivery status of the message	string
message_triggered_at	Timestamp when SuprSend sent the message request to the vendor	timestamp
message_delivered_at	Timestamp when the vendor reported delivery	timestamp
message_seen_at	Timestamp when the message was seen by the user	timestamp
message_clicked_at	Timestamp when the message was clicked by the user	timestamp
node_id	Unique identifier of the node	string
node_name	Name of the node	string
node_type	Type of the node	string
execution_failure_reason	Workflow-execution-level failure details with severity (JSON serialized as string)	string
delivery_failure_reason	Failure reason returned by the vendor	string
note	Additional note attached to the message	string
message_id_by_vendor	Vendor-generated identifier for this message	string
vendor_fallback_applicable	Whether vendor fallback was enabled (true / false as string)	string
vendor_fallback_level	Order in which this vendor was used during fallback, starting at 0 (integer as string)	string
vendor_nickname	Vendor nickname configured in SuprSend	string
vendor_slug	Vendor identifier combining vendor type and channel	string
is_smart	Whether the node used smart channel routing (true / false as string)	string
success_metric	Success metric defined for smart channel routing	string
success_achieved_at	Timestamp when the success metric was achieved (ISO timestamp as string)	string
wait_time_in_seconds	Wait time between channels for smart routing, in seconds (integer as string)	string
channel_slug	Communication channel	string
channel_value	Channel-specific value (for example, email address)	string
webhook_data	Request and response payload for webhook nodes (JSON serialized as string)	string

Linking different data points

These data points are linked, so you can trace a notification from the initial API request all the way to final delivery. The idempotency_key is shared across all three tables and is the easiest way to follow a single request end to end—it’s also the value you can pass in your API call and store on your side to correlate SuprSend logs with your internal ones.

From → To	Join on
Requests → Workflow Executions	UNNEST(requests.executions).exec_id = workflow_executions.execution_id
Workflow Executions → Messages	workflow_executions.execution_id = messages.wf_execution_id
Requests → Messages (shortcut)	requests.idempotency_key = messages.idempotency_key

Compression and path layout

You can configure how Parquet files are written when you add or edit the connector. The defaults work for almost every use case—change them only if you have a specific need.

Connectors created before 21 May 2026 still use the previous defaults—lz4 compression and the shared path layout. New connectors default to snappy + per_type. Changing either setting on an existing connector only affects files written after the change; everything already in your bucket keeps its original codec and folder.

If you’re on the old defaults and want to switch, write to [e-mail]—we’ll help you migrate and rewrite the historical files if you need them in the new layout.

Compression

Parquet files are written with snappy by default, which works well for almost every query engine and warehouse. You can switch the codec to lz4, gzip, zstd, brotli, or none if you have a specific need.

Don’t use lz4 if you plan to query the data with Amazon Athena—Athena can’t read lz4-compressed Parquet reliably (a known Athena limitation, not specific to SuprSend).

Path layout

Two layouts are available for how files are organized inside your bucket. Both use Hive-style date partitions (year=YYYY/month=MM/day=DD/hour=HH).

per_type (default)

Each data point gets its own top-level folder, then date-partitioned subfolders:

your-bucket/
├── messages/year=2025/month=01/day=15/hour=14/messages.parquet
├── workflow_executions/year=2025/month=01/day=15/hour=14/workflow_executions.parquet
└── requests/year=2025/month=01/day=15/hour=14/requests.parquet

Recommended for every query engine and warehouse. Each data point lives in its own prefix, so tables, schemas, and IAM scopes stay cleanly separated—a one-to-one mapping between table and folder. Required if you’re using Amazon Athena—see Query with Athena.

shared

All data points share the same date-partitioned path, distinguished only by file name:

your-bucket/
├── year=2025/month=01/day=15/hour=14/messages.parquet
├── year=2025/month=01/day=15/hour=14/workflow_executions.parquet
└── year=2025/month=01/day=15/hour=14/requests.parquet

Works with BigQuery and DuckDB if you already consume data from a single date-partitioned prefix. Not compatible with Amazon Athena.

Setup

Step 1: Create your S3 bucket

Open AWS S3 Console and create a bucket with these settings:

• Bucket name: Something like suprsend-logs-production (save this—you’ll need it)
• Region: Pick one close to you
• Block all public access: Yes
• Encryption: SSE-S3 (or SSE-KMS for compliance)

Step 2: Create an IAM policy

This gives SuprSend permission to write to your bucket. In IAM Console, create a policy with this JSON:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "SuprSendS3ExportAccess",
    "Effect": "Allow",
    "Action": ["s3:PutObject", "s3:ListBucket", "s3:GetObject"],
    "Resource": [
      "arn:aws:s3:::YOUR_BUCKET_NAME/*",
      "arn:aws:s3:::YOUR_BUCKET_NAME"
    ]
  }]
}

Replace YOUR_BUCKET_NAME with your actual bucket name. Save it as something like suprsend_s3_policy.

Step 3: Set up authentication

Two authentication methods are available:

Method	When to use	We recommend
IAM Role	Production, enterprise, multi-account setups	✅ Yes—credentials rotate automatically, no secrets to manage
IAM User	Development, testing, quick POCs	Only if IAM Role isn’t feasible. Requires manual key rotation every 90 days.

IAM Role (Recommended)

Use IAM Role when:

• Running in production environments
• Security compliance requires no long-lived credentials
• You have multi-account AWS setups
• You want zero credential management overhead

Steps to create IAM Role:

1. In IAM Console → Roles → Create Role
2. Select Another AWS Account and enter SuprSend’s account ID: 924219879248
3. Attach the policy you just created
4. Name it (for example, suprsend_s3_role)

Now configure the trust relationship. Generate an External ID at uuidgenerator.net, then update the trust policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "AWS": "arn:aws:iam::924219879248:root" },
    "Action": "sts:AssumeRole",
    "Condition": { "StringEquals": { "sts:ExternalId": "YOUR_EXTERNAL_ID" }}
  }]
}

Save these for the next step: Role ARN + External ID (case-sensitive, no extra spaces)

IAM User

Use IAM User when:

• Setting up for development or testing
• Quick proof-of-concept needed
• IAM Role setup isn’t feasible in your environment

IAM User credentials require manual rotation every 90 days for security compliance.

Steps to create IAM User:

1. In IAM Console → Users → Add users
2. Name it (for example, suprsend-s3-connector)
3. Attach your policy
4. Go to Security credentials → Create access key → Third party service

Save immediately: Access Key ID + Secret Access Key to add in the next step (AWS won’t show the secret again)

Step 4: Connect in SuprSend

In the SuprSend dashboard (app.suprsend.com), go to Connectors → Add connector → Amazon S3 v2.0.

IAM Role

IAM User

Enter your AWS credentials, select which data points to export, then Save and toggle Enable sync.

Step 5: Verify it’s working

Give it about 10 minutes, then check your S3 bucket. With the default per_type layout, you should see one folder per data point you enabled (messages/, workflow_executions/, requests/), each containing hourly partitions like year=2025/month=01/day=15/hour=14/....

Nothing showing up? Jump to FAQs for troubleshooting steps.

Step 6: Query the data with Athena (optional)

If you want to run SQL on the exported data, Amazon Athena reads the Parquet files in place—no warehouse to provision, no ingestion pipeline. The flow is:

1. Set the Athena query result location in the same region as your S3 bucket.
2. Create a database, e.g., suprsend_db.
3. Register one external table per data point (ss_requests, ss_workflow_executions, ss_messages) using partition projection on year/month/day/hour, so new hourly partitions show up automatically.
4. Query, always filtering on the partition columns to keep cost down.

For the full DDL, partition-projection settings, and example queries (including a join that traces a single request across all three tables), follow the Query with Athena guide.

---

Best practices

Security & access

• Use an IAM Role in production—credentials rotate automatically, with no long-lived secrets to manage.
• Use an External ID on the role’s trust policy to prevent “confused deputy” attacks.
• If you must use an IAM User, rotate the access keys every 90 days and never commit them to git.
• Assign policies to IAM groups rather than individual users to keep permissions easy to manage.
• Keep the bucket private: block all public access and enable encryption (SSE-S3 or SSE-KMS).

Querying efficiently

• Only sync the data points you need. Messages alone covers analytics and delivery troubleshooting; add Workflow Executions and Requests when you need to debug end-to-end.
• Filter on the partition columns first. Engines like Athena scan only the matching year/month/day/hour folders, which keeps queries fast and cheap.
• Use updated_at for incremental jobs. Pulling only rows where updated_at is newer than your last checkpoint avoids re-scanning history.

Storage hygiene

• Files accumulate over time. Set up an S3 Lifecycle rule to transition or expire old partitions if you don’t need them indefinitely.

---

FAQs

Files not appearing?

Work through this checklist:

1. Wait 10 minutes. The first sync takes time.
2. Bucket name matches exactly in AWS and SuprSend (case-sensitive).
3. Region matches in both AWS and SuprSend.
4. Credentials are right—Role ARN + External ID, or Access Key + Secret.
5. Policy grants s3:PutObject, s3:ListBucket, and s3:GetObject on the correct bucket ARNs.

How do I know it's working?

In the SuprSend dashboard, open Connectors → Amazon S3 v2.0. You should see:

• Sync toggle ON
• Status: Active
• At least one data point selected

Missing data?

• Data point might not be selected—check your export settings
• When setting up the connector first time, data points only export going forward (no historical backfill)
• Were notifications actually sent during that time?
• If sync was paused, data backfills when you resume

If you still find some gap in data, please contact support.

Can I change data points later, and how does backfilling work?

Yes—you can add or remove data points at any time. Here’s how each action behaves:

What you did	What happens
Paused then resumed the connector	Backfills everything from the pause window
Added a brand-new data point	Starts exporting going forward; no historical backfill
Removed a data point	Sync stops for it; data already in S3 stays there
Re-enabled a previously enabled data point	Backfills from when it was disabled