AWS Snowball - Data Validation; Checksum Validation of Transferred Data; Common Validation Errors

AWS Snowball User Guide
Checksum Validation of Transferred Data
Data Validation in AWS Snowball
Following, you'll find information on how Snowball validates data transfers, and the manual steps you
can take to ensure data integrity during and after a job.
Topics
• Checksum Validation of Transferred Data (p. 87)
• Common Validation Errors (p. 87)
• Manual Data Validation for Snowball During Transfer (p. 88)
• Manual Data Validation for Snowball After Import into Amazon S3 (p. 89)
Checksum Validation of Transferred Data
When you copy a file from a local data source using the Snowball client or the Amazon S3 Adapter
for Snowball, to the Snowball, a number of checksums are created. These checksums are used to
automatically validate data as it's transferred.
At a high level, these checksums are created for each file (or for parts of large files). These checksums
are never visible to you, nor are they available for download. The checksums are used to validate the
integrity of your data throughout the transfer, and will ensure that your data is copied correctly.
When these checksums don't match, we won't import the associated data into Amazon S3.
Common Validation Errors
Validations errors can occur. Whenever there's a validation error, the corresponding data (a file or a part
of a large file) is not written to the destination. The common causes for validation errors are as follows:
• Attempting to copy symbolic links.
• Attempting to copy files that are actively being modified. This will not result in a validation error, but it
will cause the checksums to not match at the end of the transfer.
• Attempting to copy whole files larger than 5 TB in size.
• Attempting to copy part sizes larger than 5 GB in size.
• Attempting to copy files to a Snowball that is already at full data storage capacity.
• Attempting to copy files to a Snowball that doesn't follow the Object Key Naming Guidelines for
Amazon S3.
Whenever any one of these validation errors occurs, it is logged. You can take steps to manually identify
what files failed validation and why as described in the following sections:
• Manual Data Validation for Snowball During Transfer (p. 88) – Outlines how to check for failed files
while you still have the Snowball on-premises.
87