This box searches only documentation in the Data Store space. To search the entire CyVerse wiki, use the box at the upper right.


For overview information, see Data Store or Manage Data on the CyVerse website.

DATA STORE USER MANUAL
 

 

 

 

Skip to end of metadata
Go to start of metadata

About the iDrop Desktop application

Icon

We recommend you use Cyberduck instead of iDrop.

iDrop Desktop is a cross-platform, high-throughput data transfer application that allows CyVerse account users to upload and download multiple large files (50-100 GB or larger) and directories from your local machine to the CyVerse Data Store, and move files and folders from your desktop. To effectively transfer large files such as NGS fastq or images from super resolution microscopes, iDrop establishes multiple connections between the source and destination, allowing parallel data transfer that is significantly faster than with traditional transfer methods such as ftp, sftp, or http.

About iDrop Desktop

iDrop is a Beta release that is in active development. Mac OS and Windows users may have more success using Cyberduck, a popular desktop application. Instructions for installation and configuration are here.

If you experience issues with iDrop or have questions about using iDrop, see the Troubleshooting section below.

To use iDrop on your desktop, you must first download the iPlantiDrop desktop application for your operating system (instructions below), which contains configurations already set for using iDrop with the CyVerse Data Store.

For users without a CyVerse user account, you can log in as an anonymous user (steps below), after downloading and configuring the application for your system. For more information about downloading CyVerse data files, see Downloading Data Files Without a User Account.

iDrop LITE has been removed from the DE

The iDrop Lite applet for use in the Discovery Environment has been removed.

Downloading the iPlantiDrop Beta release for your OS

These steps are for downloading the iPlantiDrop Beta release for your operating system from the CyVerse Wiki (links provided below).

The iPlantiDrop Beta release contains preset configurations that allow you to use it with the iPlant Data Store. If you have installed the public iDrop release, please go to the section below to manually set the configurations for use with the Data Store.

The release provided below is for the iPlantiDrop 2.0.1-RC1 release.

  1. Select the correct installer for your operating system (updated July 17, 2015):

  2. Follow the prompts to install iDrop.
  3. If required to create a pass phrase, enter a pass phrase. Make a note of it for future use.

Checking that your installed version is current

If you already have iDrop Desktop installed, verify your release is version 2.0.1.-RC1 or later.

  1. In iPlantiDrop, click iDrop Transfer Manager at the top left:
  2. Click About iDrop Transfer Manager and then view your release in the popup window.
  3. If you do not have version 2.0.1.-RC1 or later, download it.

Using iDrop

Logging in to iDrop

  1. Open the iDrop Transfer Manager application, enter your CyVerse username and password, and press Return.

    Don't have a CyVerse account?

    Icon

    If you don't have a CyVerse account, you can log in as an anonymous user by entering anonymous for the User Name and Password, and then pressing Return. You will have access to all data that has been made public and anonymous. For more information about downloading CyVerse data files, see Downloading Data Files Without a User Account.

  2. If required, enter the pass phrase you created in the previous section and click OK.

    Forget your pass phrase?

    Icon

    If you don't remember your pass phrase, click I Forgot, click Yes to the warning prompt that all your configuration for grid accounts will be lost/reset, and enter your new pass phrase. You will then need to reconfigure the settings for connecting to the CyVerse Data Store, as described in the section immediately below.

    Note: This step does not delete your account or any data you have in the Data Store; it is only for resetting the configuration information.

  3. In the iDrop: Grid Accounts window, either double-click the grid account to use, or click the account and then click .
  4. Under the Users folder in the left panel, click your username. Your local files and folders are displayed in the Local Directory panel on the left, and your Data Store files and folders are displayed in the File System panel on the right.

Creating an iDrop grid account

  1. If necessary, log in to iDrop (see steps above) and, if required, enter your pass phrase.
  2. Click at the top.
  3. In the iDrop: Grid Accounts window, click to open the Create Grid Account Information window.
  4. Enter the following:

    For this line:

    Enter this:Comments
    Host:
    • irodsHost (for iRODS)
    • Host (for iDrop)
    data.cyverse.org 
    Port1247If required

    User:

    • irodsUserName (for iRODS)
    • User (for iDrop)

    (your CyVerse username) 
    If no CyVerse account or only accessing public data sets, use anonymous.

     

    Zone:

    • irodsZone (for iRODS)
    • Zone (for iDrop)
    iplant 

    Password:

    • Current iRODS password (for iRODS)
    • Password (for iDrop)

    (your CyVerse password) 
    If using the anonymous username, no password is required.

     
    Default Resource(leave blank)Defaults to whatever is available
    About Firewalls

    Transfers to the Data Store require firewalls to allow TCP port 1247 and TCP and UDP ports 20000-20399 to be open. For more information on firewalls and connecting to the Data Store, see Configuring Your Firewall to Connect to the Data Store.

    If you notice that long running transfers are terminated abruptly: Some institutional firewalls have policies governing the duration allowed for idle connections that may interfere with data transfers. 

    CyVerse staff can provide additional networking and subnet information for firewall policies upon request. For more information, contact CyVerse Support at support@cyverse.org.

  5. Click Save.
  6. Back in the iDrop: Grid Accounts window, either double-click the grid account to use, or click the account and then click .
    Please be patient: The iDrop Desktop application may take a few seconds to start. Attempting to start iDrop more than once in several minutes may cause severe slowdown or freezing of your system or your Atmosphere instance.

    Is iDrop running?

    Once iDrop is running, is displayed in your system tray (Mac) or your menu bar (Windows) at the top. Simply click it to switch to the iDrop window.

    If prompted to configure the iDrop Synchronization service, click Do this later. Configuring the iDrop Synchronization service is not covered in this tutorial.

Downloading a file or folder from the iRODS file system to your local directory

  1. Find the file or folder to download in your File System folder on the right.

    Want to select a different folder?

    Icon

    To select from a different folder in the file system, see Maneuvering within the file system.

  2. Either drag the item to download from the File System folder on the right to your local folder on the left, and then click Yes to the confirmation prompt; or click the item to download from the File System folder on the right, click the destination local folder on the left, click at the top, and then click again.
  3. Click Refresh at the top to verify the item was successfully uploaded.

Uploading a file or folder to your iRODS file system

To upload a file or folder to the Data Store, you must have write or own permission to the destination directory.

  1. Find the file or folder to upload in your Local Directory Users folder on the left.
  2. If necessary, verify the iRODS permission level for the file or folder.
  3. Either drag the item to upload from your local folder on the left to the destination File System folder on the right, and then click Yes to the confirmation prompt; or click the item to upload from your local folder on the left, click the destination File System folder on the right, click at the top, and then click again.

    Icon

    To select a different folder in the file system, see Maneuvering within the file system.

  4. Click Refresh at the top to verify the item was successfully uploaded.

Verifying iRODS file or folder permissions

You must have write or own permissions to the iRODS folder to which you want to upload a file or folder. Follow the steps below to check it within iDrop, or you can also view the permission level within the DE Data window.

  1. To verify permission levels, click the file or folder in the File System panel on the right.

    Icon

    To select from a different folder in the file system, see Maneuvering within the file system.

  2. Click at the top and then click Permissions:
  3. Find the permission level for your user name.
  4. Click to close the window.

Maneuvering within the iRODS file system

There is an art to moving up and down the iRODS file system.

  1. Click :
  2. In the field, enter the path to the higher-level folder and then navigate to the directory to use, or copy and paste the path from the Data window in the Discovery Environment.

    Careful!

    Icon

    Clicking home will cause your system to hang.

    For example, to go to the Data Store Shared folder, enter the following:

  3. Press Enter.
  4. To return to your home folder, click your username.

Configuring iDrop for the CyVerse Data Store (not required for iPlantiDrop release)

Icon

This section is required only if you installed the public iDrop release, not the iPlantiDrop release. Be aware that any configurations you customize for this Beta release will be overwritten when you install a new version or binary.

  1. If necessary, log in to iDrop or open iDrop by clicking on the status bar to reopen the window.
    The first time you open iDrop, you are prompted to enter a "Pass Phrase". Your pass phrase will be used to retrieve grid information.
  2. Create your pass phrase and then click .
  3. Settings to use:

    Host

    data.cyverse.org

    Port

    1247 (if required)

    Zone

    iplant

    User

    <your_irods_username>

    Password

    <your_irods_password>

    Default Resource

    Leave blank (defaults to whatever is available)

    Starting Collection

    /iplant/home/your_irods_username

Troubleshooting tips (Hint: Use Cyberduck instead)

What to try first

If you experience issues with iDrop, try the following:

  • We recommend you use Cyberduck instead of iDrop.

  • Go to Ask CyVerse and read through the list of questions and answers about iDrop.
  • Update your Java version when notified by your operating system, including adjusting security permissions (to allow iDrop to run) when prompted by the Java updater. iDrop is developed using the Java programming language and requires updates to ensure its continued operation.
  • If you encounter an issue in analyses submitted through the DE that use a data file that was uploaded to the Data Store either via iDrop or iCommands, make sure that the filename does not contain any special characters or spaces. For more information, see Using Special Characters in the DE.

If you have problems or need assistance, see Get Help on the CyVerse website or email Support (support@cyverse.org).

Help for failing analyses or slow downloads

  • My downloads are failing and/or my analysis is taking a LONG time to complete. What can I do?
    There have been reports of failed analyses related to downloads, and iDrop that may take longer than two hours. Please contact CyVerse Support if you experience difficulties with failed analyses related to iDrop. Very often the issues are related to blocked firewall ports at your institution.
  • My DE analysis keeps failing with recently uploaded data. Is there a workaround?
    Check that the data file(s) you uploaded through the Data Store and used in the analysis don't have a special character or space in the file name. If so, rename the file and relaunch the analysis. For more information, see Using Special Characters in the DE. If this isn't your problem and you still have the problem, please contact CyVerse Support.

Help for syncing issues when copying public files

If you are experiencing sync issues when trying to copy files from the public data to your personal Data directory, be aware that a bug in iDrop Desktop makes it look as if the files weren't copied, when in fact they were. If you encounter this issue, you may want to use the irscync command (may run into issues if your folders have more than 1000 files), or use Cyberduck instead.

Questions about using iDrop Desktop with Mac

You can open iDrop Desktop from Mac, Windows, Linux, or from an Atmosphere image. iDrop Desktop is a Beta release that has been tested successfully on Mac OS X by iPlant staff and community members. Please note that although the application is being continually updated by iRODS, some of the features may not function as listed. For current iDrop development details, see the iDrop Desktop and Web GUI page on GitHub.