Data set

A data set is a collection of data (often from one event) in a scientific context organized in one matrix. Data in Pangaea are organized in predefined data sets which are quite similar to the original files uploaded - an exported from the archive.

The granularity of a data set depends on the type of data, the number of data points and is primarily in the decision of the data provider. In principle a Pangaea data set can have an unlimited number of columns and lines (excel 2003: 65,536 x 256; excel 2008: >1 Mio x 16,384) - Examples: A data set may contain one to many data series. One to many data sets may be grouped to one parent set. Access rights can be defined for a complete set only. Each data set consists of the data accompanied by metadata according to ISO standard fields (ISO 19115). A data set appears on the Internet with a metaheader which contains the information as described below.
 * 507 columns
 * 2,000,000+ lines
 * 22,600,000+ lines (in ascii: 551 MB; in ASE +index: 2.2 GB; export from IQ +DOI: 1.44 GB)

The frame title of the data set window shows its ID and the responsible curator (s. Fig.). Below are buttons to  and  the data set or to open it via its URL in a browser window . Checking the with cache new box will force a complete update in the data file cache when pressing Save.

Opening a data set in 4D will show four tabs named Config, Basics, Details and Web with metadata fields as described below:

Basics tab

 * Author(s) of the data set; one to many authors may be added by a multiple choice list related to the table [Staff]
 * Year, automaticaly set but can be changed
 * Title of the data set as free text; equivalent to the title of a publication; in case the data set is a table from a publication, the title should be the table caption, headed by the table numer, e.g. doi:0.1594/PANGAEA.693592
 * Source may contain the institution of the data origin and is relational to the table Institution; use only if data are not related to a reference.
 * Status of the data set with
 * pop-up menue with choices: questionable, not validated, validated, published, supplementary data, published & citable
 * Access rights button to set individual access to data sets not having the status published
 * citable to make data set a parent set with the citation added to a library catalog - to be used by the data librarian or editors only !
 * Registry: gives information about the registration process:
 * not to be registered if status is not published
 * registration is in the lead time for four weeks after setting the data set to status published and final editing
 * registered as the final status
 * login required may be checked for sets with status published but should still be protected (on request of the PI)
 * Reference(s) opens a multiple choice list related to the Reference table to select one to many papers relevant to the data set
 * Project(s) allows via a mutiple choice list to add one to many projects as provided by the Project table
 * Event(s) as used in the data set (can not be changed)
 * PI(s) lists all investigators related to the data series
 * Keywords is relational to the Thesaurus and can be set individualy

Config tab

 * Parameter window shows parameters with unit and short name, used in the data set. The buttons Add, Clear and PreSelect are used to compose new data sets.
 * Geocodes window lists all geocodes available; those used in the data set are highlighted and can be added to the Configuration.
 * Related metainformation window contains fields from the event table which can be added to the Configuration.
 * Configuration window lists geocodes, related metainformation and all parameters used in the order of the available data set. The Load and Save button can be used to save a configuration and load the same configuration to other similar data sets.
 * Format will show the number of digits before and after the decimal point of a numeric parameter if selected in the configuration window by a mouse click. Different formats can be selected from the pop-up menue or changed by hand. If the geocode Date/Time is selected, different types of ISO formats can be selected, depending on the required precision. Also the exponential-format is supported, e.g.
 * Split by event should not be used anymore.
 * Split by versions must be checked if a parameter occurs more than once.
 * Aggregate function may be used to internaly calculate statistic values - do not use.

Details tab

 * Citation as assembled from the fields on the Basics tab.
 * Comment to add individual comments as plain text; field size up to 32 kbyte. URIs might be included and will be resolvable in the metaheader (example ).
 * Keywords related to the Thesaurus; may be used to group sets by a keyword
 * Spatial coverage: fields showing min/max of the three spatial dimensions of the data set
 * Temporal coverage: min/max of Date/Time or Age
 * Topologic type is used to define the extension of a data set
 * Created: date/time or import; Updated: date/time of last change
 * Size of the data set

Web tab

 * Citation as assembled from the fields on the Basics tab.
 * URL as defined by the system for event-related data sets or as defined by the user for static links to files.
 * URL Data details to link files containing an extended description of the data set; the linked files should be *.txt for simple text or *.pdf if a text layout is needed. Field must contain a valid URI only and appears in the metaheader of a data set only if filled. (see URL comment on discussion page)
 * Export filename contains the data sets name if downloaded as text file to the users PC. The extension *.tab is added automaticaly.
 * Filenames usualy start with the event, followed by a specification of the content (e.g. M24_3-5_sedimentology).
 * File names of supplements start with the first authors name followed by the year, equivalent to the citation of references in a publication text; e.g. Smith_1998, Smith-Sandwell_1987 or Smith-etal_2007.
 * URL other version is an URI field to link to data sets of newer/older versions or any original source or other formats on the Internet. Field must contain a valid URI only and appears in the metaheader of a data set only if filled.
 * If a data set is deleted and substituted by a new version, this field in the deleted set should contain the DOI of the new version. In deleted sets only title and DOI will remain and the user if informed that the data set was substituted by an other version given by its new DOI: (this is only a test)
 * If a data set is requested which was deleted prior to registration or which has never been existed see e.g.
 * If a data set is requested which was deleted and missing a new version see e.g.