Intern:WRMC Curator

The following information are specially made for data curators who import new measurements and perform the WRMC.

Contact: Dr. Amelie Driemel, Alfred Wegener Institute, Bremerhaven

Data conversion and import to PANGAEA

 * (1) Move new incoming files from ftp.bsrn.awi.de/incoming to pub
 * Windows: putty account@ftp.bsrn.awi.de, macOS: ssh account@ftp.bsrn.awi.de (ask Amelie Driemel or Rainer Sieger for account)
 * cd ../incoming
 * find moveable files with lsbsrn
 * cd ../xxx (xxx = station ID)
 * smv * ../../pub/xxx
 * cd ..
 * ... (next)
 * exit


 * (2) Download, unpack, and check new station-to-archive files with the BSRN Toolbox
 * Station-to-archive -> download Station-to-archive files


 * (3) Metadata -> create all metadata files
 * if a PANGAEA metadata ID is missing (station, staff or method) ->
 * create missing entries in accordant 4D tables and
 * edit the BSRN-ID file
 * start the BSRN Toolbox and again create all metadata files


 * (4) Data -> create all data files
 * (5) Check data files: for LR0100 use the quality check of the BSRN Toolbox, see https://wiki.pangaea.de/wiki/BSRN_Toolbox#Quality_Check; the resulting file and the datafiles of the other logical records can be plotted with PanPlot (LR0100, LR0300, LR0500 etc.) or ODV (LR1100, radiosonde)
 * (6) Import -> create all import files
 * (7) Import files with 4D
 * (8) Check appearance and validity on the Internet and inform PI
 * (8) Edit files like BSRN_LR0100.txt and BSRN_Status.txt

Setting up a new BSRN-station

 * (1) To participate within the BSRN a station has to fulfill certain conditions.
 * (2) To become a BSRN-candidate the station must formally be accepted by the [mailto:Chuck.Long@noaa.gov BSRN project manager]. A brief overview document of the basic requirements of a BSRN station can be found here.
 * (3) The station manager from the BSRN-candidate station should provide [mailto:mailto:Amelie.Driemel@awi.de Amelie Driemel] with basic station/staff information.
 * (4) In return [mailto:Amelie.Driemel@awi.de Amelie Driemel] sends ...
 * the station manager the individual ftp-account for data submission from the new station,
 * the station identification number and further information needed to produce the station-to-archive file.
 * (5) Within the WRMC the data curator ...
 * imports the new station into PANGAEA as "Event" which automatically updates the BSRN stations table,
 * imports the new station/staff/etc. information into PANGAEA,
 * updates the BSRN-ID-file
 * updates Stations.kmz,
 * asks [mailto:Wolfgang.Cohrs@awi.de Wolfgang Cohrs] to update the maps,
 * asks [mailto:Rainer.Sieger@awi.de Rainer Sieger] to include the new station into the: BSRN_Toolbox
 * (6) The station manager creates, checks, compresses, and submits the first station-to-archive-file to ftp.bsrn.awi.de.
 * (7) After importing the first station-to-archive file into PANGAEA a new line for the new station must be included at the right alphabetic order in all files like BSRN_LR0100.txt and BSRN_status.txt.

Update Stations.kmz
Minor changes can be done manually according to the following instructions: Alternatively, Pan2Applic may be used. As inputfile the output of http://www.pangaea.de/ddi?request=bsrn/BSRNEvent&format=textfile&title=BSRN+Stations is a good starting point.
 * (1) download Stations.kmz on your local computer,
 * decompress the local version of the file (*.kmz-files are in fact *.zip-files). The result is a file named BSRN_Stations.kml,
 * edit BSRN_Stations.kml,
 * compress BSRN_Stations.kml into Stations.zip (e.g. using 7-Zip),
 * rename Stations.zip into Stations.kmz,
 * replace Stations.kmz in the web.

If the upper mentioned workflows do not succeed try opening the Stations.kmz in Google Earth, apply the edits and then go to "File - Save place as " (be sure to have marked Stations.kmz on the left side).

Editing PANGAEA data
Normally, archived data in PANGAEA will not be edited or deleted. Only in very specific cases (e.g. a single wrong default value) editing existing data makes more sense than importing a new version of a huge dataset. For editing single values 4D should be used:
 * searching for the dataset to be edited,
 * dragging the dataset on the "Data Series" button will open all data series from the dataset,
 * selecting the desired dataseries with a double click opens the dataseries,
 * by clicking on "List" the dataseries opens in a spread sheet which can be sorted (click on a parameter),
 * a selected datapoint can be edited or the whole (not the single value!!!) line can be deleted using "Strg D".
 * after editing/deleting please click first anywere to deselect the selection,
 * click the "Save" button,
 * close the "Data series" window,
 * open the corresponding dataset by doubleclicking,
 * tag "with cache new" and press the "Save" button to renew the edited dataset.

Updating PANGAEA data/New Versions of data
30 days after the import a permanent doi-number is attached to a dataset. Before, a dataset can be deleted without problems. Later, a dataset cannot be deleted anymore. If a new version of the dataset - carrying a new doi-number - exists, the old version has to be "hidden" from users by doing the following operations in 4D: Datasets, which have a replaced by reference link to another dataset are removed from the search results and can also not be downloaded via the data warehouse. If someone already cited the old version, however, the citation link still leads to the old version. Therefore, the title of old version has to be changed to "..., Version 1" and in the Commment field of the new version the information on what was changed and when has to be inserted (e.g. "Version 2, 2017-01-10, faulty LWD values corrected"). See also here for an example.
 * searching for the old dataset to be hidden (e.g. search for the *ID in the "Data Set" window) and open it
 * in the 'References' tab click on Add/Remove in the lower part (References for PANGAEA data sets)
 * search for the new version and insert it here by double-clicking on it and then pressing OK
 * choose "Replaced by" in the Relation column and press on save