Email Migration Audit Log: What to Capture and Keep

An email migration audit log is the difference between "we moved 12,000 mailboxes successfully" and "we can prove, six months later, that mailbox UID 8472 in finance@example.com moved between 02:14 and 02:17 on 9 March, with all 8,341 messages and 4.2 GB of attachments intact". Auditors care about the second statement. Help desks care about the second statement. Lawyers responding to a discovery request care about the second statement. This post walks you through what to capture, how to structure it, and how long to keep it.

Why migration logs are different from normal application logs#

Application logs are usually high-volume, low-stakes records: who logged in, which endpoint they hit, whether it succeeded. They are kept for a window measured in days or weeks and rotated aggressively.

Migration logs are the opposite. The volume is moderate, the stakes are high, and the retention is long. Each entry represents a specific decision the migration made about a specific message in a specific mailbox. Lose the log and you lose the ability to reconstruct what happened.

Two scenarios put this in sharp focus:

• A user calls six weeks after the migration: "I'm missing an email from a vendor about contract terms. Did we move it?" Without per-message logs, you cannot answer. With per-message logs, you find the entry showing the source UID, the destination UID, the size, and the timestamp; you reconcile against what the user is seeing.
• An auditor pulls a sample three months after the migration: "Show me the controls operating for these three mailboxes." Without logs, you have screenshots and memory. With logs, you have evidence.

For a wider view of the compliance scaffolding that the audit log sits inside, the email migration compliance guide is the parent post. The SOC 2 email migration and GDPR email migration posts cover the specific control frameworks the logs feed.

The three log levels: message, mailbox, batch#

A useful migration log has three nested levels. Each answers a different question.

Per-message: did this specific email move#

The lowest level captures one row per message. Useful fields:

• Timestamp of the move attempt (ISO 8601, UTC).
• Mailbox identifier (email address on source, email address on destination if it changed).
• Folder path (source) and mapped folder path (destination).
• Source UID (IMAP UID on source server).
• Destination UID (IMAP UID on destination server after append).
• Message size in bytes.
• Message-ID header (the RFC 5322 one).
• Subject hash (SHA-256 of subject; do not log the subject in plaintext).
• Status: success, retry, skipped, failed.
• Failure reason if applicable (e.g. Message too large for destination).

The Message-ID and subject hash give you a way to reconcile messages even when IMAP UIDs change (which they do for many provider-side operations). The size in bytes gives you a way to spot truncated transfers.

This level of detail produces large logs. A 10,000-message mailbox produces 10,000 rows. A 200-mailbox migration with an average of 5,000 messages per mailbox produces a million rows. That is fine. Compressed CSV at this scale is tens of megabytes.

Per-mailbox: did this mailbox finish#

The middle level captures one row per mailbox. Useful fields:

• Mailbox identifier.
• Start timestamp (first message attempted).
• End timestamp (last message attempted or job marked complete).
• Total messages on source.
• Total messages on destination.
• Delta (positive or negative).
• Total bytes transferred.
• Folders processed with per-folder counts.
• Errors encountered (count and types).
• Final status: complete, complete-with-warnings, failed.

This is the level your dashboards summarise. It is also the level auditors sample at, because a row corresponds to one user's experience.

Per-batch: who ran what, when#

The top level captures one row per migration batch, where a batch is a set of mailboxes you grouped together to run on a schedule.

• Batch identifier (links back to change ticket).
• Operator identity (who ran it).
• Start and end timestamps.
• Configuration snapshot: source endpoint, destination endpoint, filters, concurrency settings.
• Mailbox count in the batch.
• Aggregate results: how many succeeded, warned, failed.

This level is what your auditor wants to see first. Everything else is sampled from here.

What not to log#

Logging is a discipline of subtraction as much as addition. Three things should never go into a migration audit log:

• Message bodies or attachments. The moment you log content, your log file becomes a copy of regulated data. It inherits all the data classification rules of the source mailbox. Skip this entirely.
• Subjects in plaintext. Subjects often contain confidential information (case numbers, customer names, salary references). Hash them if you need a stable identifier. If you do not need an identifier, do not log them at all.
• Credentials or tokens. OAuth refresh tokens, app passwords, and any kind of session bearer must not appear in the log. If your tool prints these to stdout, redact in the wrapper.

Log formats that work#

Auditors are pragmatic. They accept any format they can parse and sample from. In practice, three formats dominate:

• CSV with a documented schema. The lowest common denominator. Easy to open in a spreadsheet, easy to import into a query tool, accepted by every audit firm. Document the columns in a README that lives next to the file.
• JSON Lines (newline-delimited JSON). Better for structured fields, mixed types, and nested data. Auditors who run their own scripts prefer it. Pair with a JSON Schema describing the structure.
• Parquet for large logs. Useful for million-row datasets. Most audit firms can read it, but check before committing.

PDF summaries are useful but not sufficient. They are great for the executive summary at project closure. Auditors will ask to see the underlying data behind any PDF summary.

A practical CSV schema#

For per-message logs, the following CSV header works well:

timestamp_utc,batch_id,mailbox,folder_src,folder_dst,uid_src,uid_dst,message_id,subject_hash,size_bytes,status,error_code,error_message

For per-mailbox logs:

mailbox,start_utc,end_utc,batch_id,messages_src,messages_dst,delta,bytes_total,folders_count,errors_count,final_status

For per-batch logs:

batch_id,change_ticket,operator,start_utc,end_utc,source_endpoint,dest_endpoint,mailbox_count,succeeded,warned,failed

These are reasonable defaults. Adjust to match what your tool actually produces, and document any deviations.

Retention: how long to keep migration logs#

There is no universal answer, but there is a useful default: three years.

Three years covers:

• The typical SOC 2 audit lookback (12 months) plus the next audit cycle.
• Most regulator response windows.
• Most user "I'm missing an email from last year" scenarios.

For some industries you will need longer. Financial services often requires seven years for audit-relevant records. Healthcare retention is policy-driven and can be very long. Legal hold can extend any retention indefinitely.

What you should not do is keep migration logs forever by default. Logs that are not on a defined retention schedule eventually become a liability rather than an asset. Apply your retention policy, document it, and delete on schedule.

Where to store the logs#

Long-term migration log storage should be:

• Write-once or write-protected. The classic pattern is S3 with object lock, Azure Blob with immutable storage, or a WORM-protected file share. Auditors love seeing that logs cannot be silently modified after the fact.
• Encrypted at rest with a key you control.
• Access-restricted to a named group, not the whole IT team.
• Backed up to a separate location.

Do not leave logs on the migration workstation after the project finishes. The workstation is a transient piece of infrastructure; long-term logs belong in your long-term log system.

Reporting: turning logs into evidence#

Logs are raw data. Auditors and stakeholders want reports. A few reports that are worth producing at project closure:

The executive summary#

One page. Total mailboxes, total messages, total bytes, total elapsed time, success rate, any incidents. This is for the CIO and the steering committee. Keep it brief and accurate.

The per-mailbox reconciliation report#

One row per mailbox, with source count, destination count, delta, status, and any notes about anomalies. This is the report your auditor will sample. The companion post on post-migration validation covers what to put in the reconciliation column and how to handle non-zero deltas.

The error breakdown report#

Grouped by error type, then by frequency. This tells you what went wrong systematically versus what was a one-off. Patterns matter: 500 "Message too large for destination" errors point at one configuration issue; 500 different errors point at something else.

The access and credentials report#

Who held what privileges during the migration window, when they were granted, when they were revoked. This is for the access review part of your audit. Source it from your identity system, not from the migration tool's logs.

How to handle gaps in tool logging#

Not every migration tool produces enough detail for a clean audit log. Two patterns work when the tool itself is sparse.

Wrap the tool#

Run the migration tool from a shell script or scheduling system that captures:

• Start and end timestamps.
• The exact command line and configuration.
• Stdout and stderr captured to file.
• A SHA-256 of the configuration file.
• The operator's identity (whoever invoked the wrapper).

This produces a deterministic batch-level log even when the tool itself does not.

Reconcile against the source and destination#

You can always count what is on the source before the run and what is on the destination after. The IMAP SELECT command returns total message counts per folder. A simple before-and-after comparison gives you per-mailbox totals even if the tool's logs are unhelpful.

For Mailbox Taxi specifically, the desktop client produces per-message and per-mailbox logs by default, with optional verbose flags that dial up the detail. If you are using a different tool, check what it produces before the project starts; do not discover the gap halfway through.

Cross-checking the log against the destination#

A log can be wrong in two directions: it can claim a message moved when it did not, or fail to record a message that did. Spot-checking against the destination catches both.

A reasonable protocol:

1. Pick a sample of mailboxes (3–5% is plenty for an audit).
2. For each sampled mailbox, pick 10 messages distributed across folders.
3. For each sampled message, confirm it exists at the destination with matching Message-ID, subject, size, and date.
4. Record the check in a spreadsheet attached to the change ticket.

This is enough for SOC 2 sampling. It is also enough to catch a tool that silently fails on certain folder types (a common issue with IMAP folders containing non-ASCII characters in their names).

What auditors actually ask for#

A condensed list of artifacts that come up in real audits:

• The change ticket for the migration, with approvals.
• Per-batch logs showing operator identity and configuration.
• Per-mailbox reconciliation for a sample of mailboxes.
• Per-message logs for spot-check sampling within those mailboxes.
• Access grant and revocation records for the admin privileges used.
• Communication evidence that users were notified.
• Post-implementation review notes.

The migration audit log feeds the second, third, and fourth of these directly. Get the log right and most of the audit responses are exports rather than research.

When logs become incident evidence#

Occasionally a migration causes an incident: a mailbox is reported missing, a delivery fails, a user finds messages they did not expect. The logs become the primary evidence in the post-incident review.

For incident response:

• Freeze the relevant logs in a separate location to protect them from rotation or deletion.
• Pull the per-message rows for the affected mailbox and time window.
• Compare against the user's complaint systematically.
• Reconstruct the timeline from per-batch timestamps.

This is also where the value of comprehensive logging becomes obvious. You do not know in advance which mailbox will become the subject of an incident response. Logging everything means you are ready when it happens.

The complete email migration guide and the post-migration validation post cover the wider operational practice that surrounds logging. Read them together to see how validation, logging, and audit evidence weave into a single project workflow.

A short checklist for log-readiness#

Before the first production batch:

• Confirm your tool produces per-message, per-mailbox, and per-batch detail (or wrap it to add what is missing).
• Decide on log format (CSV is fine) and document the schema.
• Set up encrypted long-term storage with appropriate retention.
• Define who can access the logs and how.
• Test exporting and reopening a sample log file end-to-end.

After project closure:

• Move logs from the migration workstation to long-term storage.
• Generate the executive summary, per-mailbox reconciliation, and error breakdown reports.
• Attach all artifacts to the change ticket.
• Apply the retention schedule.
• Revoke any temporary access that was granted for the project.

Most of these steps take minutes. Done in advance and as routine practice, they turn audit season from an archaeology project into a search query.