This box searches only documentation in the Data Store space. To search the entire CyVerse wiki, use the box at the upper right.


For overview information, see Data Store or Manage Data on the CyVerse website.

DATA STORE USER MANUAL
Skip to end of metadata
Go to start of metadata

IMPORTANT NOTE ABOUT UPGRADING TO 4.1.9

iCommands 4.1.9, the latest supported version for CyVerse services, has been released by iRODS. For a list of testing done on affected services, see Upgrading to iRODS 4.1.9.

iCommands is a collection of commands for Linux and Mac OS operating systems that are used in the iRODS system to interact with the CyVerse Data Store. Many commands are very similar to Unix utilities. For example, to list files and directories, in Linux you use ls, but in iCommands you use ils.

While iCommands are great for all transfers and for automating tasks via scripts, they are the best choice for large files (2-100 GB each) and for bulk file transfers (many small files). For a comparison of the different methods of uploading and downloading data items, see Downloading and Uploading Data.

iCommands can be used by CyVerse account users to download files that have been shared by other users and to upload files to the Data Store, as well as add metadata, change permissions, and more. Commonly used iCommands are listed below. Follow the instructions on Setting Up iCommands for how to download and configure iCommands for your operating system.

A CyVerse account is not required to download a public data file via iCommands. To see instructions just for public data download with iCommands, see the iCommands section on Downloading Data Files Without a User Account.

Before you begin, you may want to watch a CyVerse video about iCommands.

About setup and configuration

Regardless of whether you want to download and upload data as a CyVerse user, or download public data without an account as an anonymous user, iCommands must be set up and configured on each user's local machine or server. For information on downloading and setting up iCommands for your operating system, see Setting Up iCommands.

For information on downloading public data files anonymously, see Downloading Data Files Without a User Account. To share data with a user, the user at minimum must have Read permission to the parent directory that contains the file and any other parent folders within the folder hierarchy.

In addition to using iCommands to share data from the command line, CyVerse users can upload and download data within the Discovery Environment. For more information, see Sharing Data Files and Folders Via the Discovery Environment.

Commonly used iCommands

These sample commands are entered from the command line once you have installed and configured iCommands on your computer.

This is not a comprehensive guide for how to use iCommands

Icon

This section offers the most commonly used commands used with CyVerse data. If you are not comfortable using the command-line interface, ask your IT professional to assist you.

For more information and a complete list of all iCommands, enter the iCommand and -h in the terminal window to view the related Help information. More information is available by searching on "icommands" at http://irods.org/documentation/.

For more information on using these and other iRODS commands, see the iRODS Documentation on the iRODS website, or enter the iCommand and -h to view the help file in a terminal window.

Enter any of the following iCommands in a command line.

CommandDescription (For help, enter the command followed by -h)
icd

 

 

Changes the working directory.

 Example: icd
ichmod

Gives access to another user.

ichmod readGrants read-only permission level for specified user to selected file or folder.
ichmod write
Grants read and write permission level for specified user to selected file or folder.
ichmod ownGrants full ownership permission level for specified user to selected file or folder.
ichmod nullRemoves the permission level for the user to the file or folder.
iexit full

Logs off/disconnects from the Data Store.

  • To log out of the Data Store (especially important if on a shared or public computer), enter iexit full
  • To log in again on the same device, enter iinit 

After you are successfully connected, you do not need to reenter your password to use the CyVerse Data Store from that computer unless you log out.

iget

 

Downloads the file/directory from iRODS to local device.

Example:

In the above example, the -K flag is included to ensure data isn't corrupted during data transfer via iCommands, which can happen anytime data are transferred across the internet. CyVerse highly recommends that you include the -K option with both iget and iput. This causes iCommands to verify that transferred files weren't corrupted during the transfer by comparing the Checksums computed before and after the transfer.

iinit

Initializes and starts the connection to iRODS.

 Example: iinit

iinit -h

imiscsvrinfo

Reports the client software version.

Reports the server version.

ils

 

Lists the contents of the current working directory.

 Example: ils
ils -A

 

Lists the directory permissions.

 Example: Directory permissions
imkdir

 

Creates a new directory.

 Example: imkdir
iput

 

Uploads the file/directory from the local device to iRODS.

Example:

In the above example, the -K flag is included to ensure data isn't corrupted during data transfer via iCommands, which can happen any time data are transferred across the internet. CyVerse highly recommends that you include the -K flag with both iget and iput. This causes iCommands to verify that transferred files weren't corrupted during the transfer by comparing the Checksums computed before and after the transfer.

See more iput examples below.

ipwd

 

Shows the name and path of the current remote folder.

 Example: ipwd
irm

 

Moves a file to the trash.

 Example: irm
irm -f

Deletes a file.

 Example: irm -f
irm -r

Moves a folder to the trash.

 Example: irm -r
irm -fr

 

Deletes a folder.

 Example: irm -fr
irsync

Synchronize a folder from a local directory (e.g., your computer) to a directory in iRODs (or the CyVerse Data Store). The -l flag can be used to investigate if the directory needs to be synchronized. The -s flag can be used to use file size rather than checksums to determine if the directories are synchronized. 

Reliability and troubleshooting tips and tricks

If you are having problems with reliability, try the following command options:

PurposeCommand

Allows bulk uploads for transferring many small files at once. Cannot be used with --lfrestart.

-b

Includes a folder or directory.

-r

Renews the socket connection every 10 minutes.

-T

Lists the number of retries attempted in case of failure/error.

--retries

Saves the progress to a file so that only the files that were not sent successfully are sent when you restart a failed  upload.
Any filename may be used for the checkpoint-file.

-X <checkpoint-file>

Saves the progress to an individual file so only the unsent portion of the file is sent when you restart the upload. Cannot be used when using -b for a bulk upload. Any filename may be used for the checkpoint-lf-file.

--lfrestart <checkpoint-lf-file>
Provides progress feedback.
-P
Provides more information about a command.-V

The versions must be the same on your client and on the server.

  • Finds the version of iCommands on your system.
  • Finds the version of iCommands on your server.

 

  • iinit -h
  • imiscsvrinfo

Troubleshooting a failed DE analysis that uses a file uploaded via iCommands

If the file used in the failed analysis was uploaded to the Data Store via iCommands, verify that the filename does not contain any special characters or spaces. This has been known to cause analyses to fail. For more information, see Using Special Characters in the DE and Renaming a Data File or Folder in the DE.

Examples

Bulk files transfer example

Single large file transfer example

In the above examples:

  • -K: To ensure data isn't corrupted during data transfer via iCommands, which can happen anytime data are transferred across the internet, CyVerse highly recommends that you include the -K option with iget and iput. This causes iCommands to verify that transferred files weren't corrupted during the transfer by comparing the Checksums computed before and after the transfer.
  • my-files-to-transfer/ is the example name of the directory or folder for bulk transfers.
  • my files-to-transfer.txt is the example name for single file transfers.
  • Any filename may be used for the checkpoint-file.

1 Comment

  1. On a Mac, to log in with a different user on the same machine, edit ~/.irods/.irodsEnv. Change the CyVerse username behind irodsUserName.   Save the file and reinitialize with iinit.