This box searches only documentation in the Data Store space. To search the entire CyVerse wiki, use the box at the upper right.


For overview information, see Data Store or Manage Data on the CyVerse website.

DATA STORE USER MANUAL
Maintenance: Tues, Jan 28, 2020

ACCESS TO OR USAGE OF THE FOLLOWING SERVICES WILL BE UNAVAILABLE OR DISRUPTED:

Discovery Environment         8:00am to 5:00pm MST
The Discovery Environment will be unavailable while patches and updates are applied.
        ** Currently running analyses will be terminated. Please plan accordingly.

Data Store                    8:00am to 5:00pm MST
The Data Store will be unavailable during the maintenance period.
 
Data Commons                  8:00am to 5:00pm MST
The Data Commons will be unavailable during the maintenance period.
 
Atmosphere and Cloud Services 8:00am to 5:00pm MST
Marana Cloud: Atmosphere instances in the Marana Cloud will be operational; however, you will not be able to use the Data Store within your instance, and you may not be able to access the Atmosphere web interface.
 
User Portal                   8:00am to 5:00pm MST
The User Portal, http://user.cyverse.org, will be unavailable while we perform maintenance and updates.
 
Agave/Science API             8:00am to 5:00pm MST
The Agave/Science API will be unavailable during this maintenance period.
 
DNA Subway                    8:00am to 5:00pm MST
DNA Subway will be unavailable during this maintenance period.
 
The following services will NOT be affected by the maintenance: CyVerse Wiki and JIRA

Keep up to date with our maintenance schedules on the CyVerse public calendar
http://www.cyverse.org/maintenance-calendar
Check your local timezone here https://bit.ly/36iVOkX 
 
Please contact support@cyverse.org for any questions, or concerns.

 

 

 

 

 

 

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Section
Column
width65%

What is metadata?

Metadata is data about a data file or folder that describes its contents and the context of its contents for purposes of locating and sharing that item. It is a useful way to recall specific information about a file or folder — such as a tool name, version, or settings you used when submitting an analysis — when you want to retrieve the item, help others retrieve it, or help collaborators with whom you share the item to understand the context of its contents (e.g, when and how it was collected, who collected it, or what were the input parameters or specimens used).

Metadata consists of attributes (a changeable characteristic of the item, i.e., filetype), values (the type of characteristic, i.e., text), and units — commonly referred to as AVUs or Attribute-Value-Unit triples.

About the metadata databases

Prior to the 2.7 release, all metadata both from within the Discovery Environment (DE) and via iCommands was written to and accessed from the iRODS metadata database. Because the iRODS metadata database allows only three fields (A, V, and U), it is not ideally suited for the dense data needs of semantic data (e.g., metadata linked to ontology terms).

As a result, CyVerse launched the CyVerse metadata database with the 2.7 release of the DE. This new database can store an unlimited number of metadata fields, providing a much more scalable framework for the needs of semantic data used in the CyVerse Data Store. Metadata created in the DE after the 2.7 release is stored in the new CyVerse metadata database.

Metadata created before the 2.7 release as part of a metadata template (e.g., SRA submission or DOI request) was already stored in and remains in the DE metadata database.

Metadata that was created before the 2.7 release without a template remains safely stored in the iRODS metadata database. That metadata can still be read within the DE, and both written read using iCommands imeta. Users can transfer their data from the iRODS metadata database to the CyVerse metadata database, but because it is no longer possible to create iRODS metadata from within the DE, the operation cannot be undone. User who require command-line access to metadata should continue to use iCommands until command-line access to the DE metadata database is available.

Both databases are indexed, searchable, and accessible through the DE.

Using metadata in the Discovery Environment

The CyVerse metadata database is the primary repository for DE metadata storage. Based on the familiar AVU-triples (Attributes-Values-Units) foundation of its predecessor iRODS metadata database, the new CyVerse metadata database allows an unlimited number of AVU combinations. For example, users now can use the same attribute with more than one value or more than one type of unit, significantly expanding the degree of metadata that can be stored for data analysis and retrieval.

In addition to the new metadata database, we have expanded the capabilities of metadata templates. Where previously, users could attach only one template to a data item, now users can attach an unlimited number of templates to the same data item. Once attached, any values or units that match an attribute in the AVU populate the fields in the template.

Access to DE metadata via command-line using the DE API will be available in a future release.

For information on using metadata and bulk metadata templates in the DE, see Using Metadata in the DE.

About metadata templates in the DE

You also can use the bulk metadata feature to add metadata to multiple files in the same or different folders.  

Using metadata via command-line with iRODS imeta commands

All data that was created prior to the 2.7 release without a metadata template can still be read and written after the 2.7 release with imeta iCommands (where it remains in the iRODS metadata database), and read through the DE. You can import that metadata to the CyVerse metadata database in the DE; however, be aware that this operation cannot be undone. For instructions on importing iRODS metadata to the CyVerse metadata database, see Using Metadata in the DE.

For information on adding metadata to a file or folder using imeta, see Adding Metadata to a File Using iRODS imeta (Metadata) Commands.

Column
width35%
Panel
bgColorWhiteSmoke
titleBGColorSteelBlue
titleHelpful Links
borderWidth1px
titleColorWhite

On This Page:

Table of Contents

Related Pages: