iCommands is a collection of commands for Linux and Mac OS operating systems that are used in the iRODS system to interact with the CyVerse Data Store. Many commands are very similar to Unix utilities. For example, to list files and directories, in Linux you use
ls, but in iCommands you use
While iCommands are great for all transfers and for automating tasks via scripts, they are the best choice for large files (2-100 GB each) and for bulk file transfers (many small files). For a comparison of the different methods of uploading and downloading data items, see Downloading and Uploading Data.
iCommands can be used by CyVerse account users to download files that have been shared by other users and to upload files to the Data Store, as well as add metadata, change permissions, and more. Commonly used iCommands are listed below. Follow the instructions on Setting Up iCommands for how to download and configure iCommands for your operating system.
A CyVerse account is not required to download a public data file via iCommands. To see instructions just for public data download with iCommands, see the on Downloading Data Files Without a User Account.
Before you begin, you may want to watch a CyVerse video about iCommands.
About setup and configuration
Regardless of whether you want to download and upload data as a CyVerse user, or download public data without an account as an anonymous user, iCommands must be set up and configured on each user's local machine or server. For information on downloading and setting up iCommands for your operating system, see Setting Up iCommands.
For information on downloading public data files anonymously, see Downloading Data Files Without a User Account. To share data with a user, the user at minimum must have Read permission to the parent directory that contains the file and any other parent folders within the folder hierarchy.
In addition to using iCommands to share data from the command line, CyVerse users can upload and download data within the Discovery Environment. For more information, see Sharing Data Files and Folders Via the Discovery Environment.
Commonly used iCommands
These sample commands are entered from the command line once you have installed and configured iCommands on your computer.
For more information on using these and other iRODS commands, see the iRODS Documentation on the iRODS website, or enter the iCommand and -h to view the help file in a terminal window.
Enter any of the following iCommands in a command line.
|Command||Description (For help, enter the command followed by |
Changes the working directory.
Gives access to another user.
|Grants read-only permission level for specified user to selected file or folder.|
|Grants read and write permission level for specified user to selected file or folder.|
|Grants full ownership permission level for specified user to selected file or folder.|
|Removes the permission level for the user to the file or folder.|
Logs off/disconnects from the Data Store.
After you are successfully connected, you do not need to reenter your password to use the CyVerse Data Store from that computer unless you log out.
Downloads the file/directory from iRODS to local device.
In the above example, the
Initializes and starts the connection to iRODS.
Reports the client software version.
Reports the server version.
Lists the contents of the current working directory.
Lists the directory permissions.
Example: Directory permissions
Creates a new directory.
Uploads the file/directory from the local device to iRODS.
In the above example, the
See more iput examples below.
Shows the name and path of the current remote folder.
Moves a file to the trash.
Deletes a file.
Example: irm -f
Moves a folder to the trash.
Example: irm -r
Deletes a folder.
Example: irm -fr
Synchronize a folder from a local directory (e.g., your computer) to a directory in iRODs (or the CyVerse Data Store). The
Reliability and troubleshooting tips and tricks
If you are having problems with reliability, try the following command options:
Allows bulk uploads for transferring many small files at once. Cannot be used with
|Includes a folder or directory.|
Renews the socket connection every 10 minutes.
Lists the number of retries attempted in case of failure/error.
Saves the progress to a file so that only the files that were not sent successfully are sent when you restart a failed upload.
Saves the progress to an individual file so only the unsent portion of the file is sent when you restart the upload. Cannot be used when using
|Provides progress feedback.|
|Provides more information about a command.|
The versions must be the same on your client and on the server.
Troubleshooting a failed DE analysis that uses a file uploaded via iCommands
If the file used in the failed analysis was uploaded to the Data Store via iCommands, verify that the filename does not contain any special characters or spaces. This has been known to cause analyses to fail. For more information, see Using Special Characters in the DE and Renaming a Data File or Folder in the DE.
Bulk files transfer example
Single large file transfer example
In the above examples:
-K: To ensure data isn't corrupted during data transfer via iCommands, which can happen anytime data are transferred across the internet, CyVerse highly recommends that you include the
iput. This causes iCommands to verify that transferred files weren't corrupted during the transfer by comparing the Checksums computed before and after the transfer.
my-files-to-transfer/is the example name of the directory or folder for bulk transfers.
my files-to-transfer.txtis the example name for single file transfers.
- Any filename may be used for the checkpoint-file.