AWS Snowball User Guide
Using the Snowball Client
In addition to the previously defined Snowball client copy command options, there are some options
specific to transferring data from an HDFS cluster. The following table describes those options. For more
information on transferring from an HDFS cluster, see Importing Data from HDFS (p. 54).
Important
The --batch option for the Snowball client's copy command is not supported for HDFS
data transfers. If you must transfer a large number of small files from an HDFS cluster, we
recommend that you find a method of collecting them into larger archive files, and then
transferring those. However, these archives are what is imported into Amazon S3. If you want
the files in their original state, take them out of the archives after importing the archives.
HDFS-Specific
Command Option
Description
--hdfsconfig Used with the hdfs:// import schema, this option sets the path to a
custom XML configuration file on the server running your HDFS cluster.
This option must be repeated if you have multiple configuration files. For
example, the following specifies two configuration files.
--hdfsconfig src/core/Namenode-site.xml --hdfsconfig /
hdfs/corp/conf/hdfs-site.xml
-k On and set to false by default.
Used with the hdfs:// import schema and the -p option, this option
sets the path to the keytab file used to authenticate the Snowball client's
connection to the HDFS cluster before copying data to a Snowball.
Note
You must have both the principal and the keytab registered with
the Kerberos authentication server used to authenticate the HDFS
cluster. If you recently ran the kinit command on your terminal,
then you don't need to specify this option.
-n On and set to false by default.
Used with the hdfs:// import schema, this option copies data from a
nonsecure HDFS cluster.
-p On and set to false by default.
Used with the hdfs:// import schema and the -k option, this option sets
the principal used to authenticate the Snowball client's connection to the
HDFS cluster before then copying data to a Snowball.
Note
You must have both the principal and the keytab registered with
the Kerberos authentication server used to authenticate the HDFS
cluster. If you recently ran the kinit command on your terminal,
then you don't need to specify this option.
Syntax for the snowball cp Command
Copying data with the Snowball client's snowball cp command uses a syntax that is similar to Linux
cp command syntax. However, there are some notable differences. In the following topics, you can find
a reference for the syntax used by the snowball cp command. Failure to follow this syntax can lead to
unexpected results when copying data to or from a Snowball.
When copying data, define a source path and a destination path, as in the following example.
62
To Next Page
Loading...