Intern:WRMC Curator

The following information are specially made for data curators who import new measurements and perform the WRMC.

Contact: Dr. [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo], Alfred Wegener Institute, Bremerhaven

Data conversion and import to PANGAEA

 * (1) Move new incoming files from ftp.bsrn.awi.de/incoming to pub
 * Windows: putty account@ftp.bsrn.awi.de, OS X: ssh account@ftp.bsrn.awi.de (ask [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] or Rainer Sieger for account)
 * cd ../incoming
 * find moveable files with lsbsrn
 * cd ../xxx (xxx = station ID)
 * smv * ../../pub/xxx
 * cd ..
 * ... (next)
 * exit


 * (2) Download, unpack, and check new station-to-archive files with the BSRN Toolbox
 * Station-to-archive -> download Station-to-archive files


 * (3) Metadata -> create all metadata files
 * if a PANGAEA metadata ID is missing (station, staff or method) ->
 * create missing entries in accordant 4D tables and
 * edit the BSRN-ID file
 * start the BSRN Toolbox and again create all metadata files


 * (4) Data -> create all data files
 * (5) Check data files with PanPlot or ODV (radiosonde)
 * (6) Import -> create all import files
 * (7) Import files with 4D
 * (8) Check appearance and validity on the Internet and inform PI
 * (8) Edit files like BSRN_LR0100.txt and BSRN_Status.txt

Setting up a new BSRN-station

 * (1) To participate within the BSRN a station has to fulfill certain conditions.
 * (2) To become a BSRN-candidate the station must formally be accepted by the [mailto:Chuck.Long@noaa.gov BSRN project manager].
 * (3) The station manager from the BSRN-candidate station should provide [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] with basic station/staff information.
 * (4) In return [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] sends ...
 * the station manager the individual ftp-account for data submission from the new station,
 * the station identification number and further information needed to produce the station-to-archive file.
 * (5) Within the WRMC the data curator ...
 * imports the new station into PANGAEA as "Event" which automatically updates the BSRN stations table,
 * imports the new station/staff/etc. information into PANGAEA,
 * updates the BSRN-ID-file
 * updates Stations.kmz,
 * asks [mailto:Wolfgang.Cohrs@awi.de Wolfgang Cohrs] to update the maps,
 * asks [mailto:Rainer.Sieger@awi.de Rainer Sieger] to include the new station into the: BSRN_Toolbox
 * (6) The station manager creates, checks, compresses, and submits the first station-to-archive-file to ftp.bsrn.awi.de.
 * (7) After importing the first station-to-archive file into PANGAEA a new line for the new station must be included at the right alphabetic order in all files like http://www.pangaea.de/PHP/bsrn/BSRN_LR0100.txt and BSRN_Status.txt.

Update Stations.kmz
Minor changes can be done manually according to the following instructions: Alternatively, Pan2Applic may be used. As inputfile the output of http://www.pangaea.de/ddi?request=bsrn/BSRNEvent&format=textfile&title=BSRN+Stations is a good starting point.
 * (1) download Stations.kmz on your local computer,
 * decompress the local version of the file (*.kmz-files are in fact *.zip-files). The result is a file named BSRN_Stations.kml,
 * edit BSRN_Stations.kml,
 * compress BSRN_Stations.kml into Stations.zip (e.g. using 7-Zip),
 * rename Stations.zip into Stations.kmz,
 * replace Stations.kmz in the web.

PANGAEA import of new versions
If a dataset must be updated the new data should be imported as usual. Afterwards, the old version is flagged as such (by inserting a comment AND by flagging it in 4D) and contains a link to the new version. The earlier practice of deleting the old version is no longer supported by PANGAEA (as of 2016-01-01). The old version of the dataset will be under password protection and will not appear in the data warehouse nor in any retrievals, therefore guaranteeing that data are not redundant.

=> Flagging the old version in 4D
 * Open the old dataset (old version) in 4D
 * Under "References" insert the new version as a Reference (PANGAEA data sets, lower part of window)
 * Set the relation "replaced by" for the new version
 * Save

Editing PANGAEA data
Normally, archived data in PANGAEA will not be edited or deleted. Only in very specific cases (e.g. a single wrong default value) editing existing data makes more sense than importing a new version of a huge dataset. For editing single values 4D should be used:
 * searching for the dataset to be edited,
 * dragging the dataset on the "Data Series" button will open all data series from the dataset,
 * selecting the desired dataseries with a double click opens the dataseries,
 * by clicking on "List" the dataseries opens in a spread sheet which can be sorted (click on a parameter),
 * a selected datapoint can be edited or the whole (not the single value!!!) line can be deleted using "Strg D".
 * after editing/deleting please click first anywere to deselect the selection,
 * click the "Save" button,
 * close the "Data series" window,
 * open the corresponding dataset by doubleclicking,
 * tag "with cache new" and press the "Save" button to renew the edited dataset.

Updating PANGAEA data/New Versions of data
30 days after the import a permanent doi-number is attached to a dataset. Before, a dataset can be deleted without problems. Later, a dataset cannot be deleted anymore. If a new version of the dataset - carrying a new doi-number - exists, the old version has to be "hidden" from users by doing the following operations in 4D: Datasets, which have a replaced by reference link to another dataset are removed from the search results and can also not be downloaded via the data warehouse. If someone already cited the old version, however, the citation link still leads to the old version. Therefore, the title of old version has to be changed to xxx, Version 1 and in the Commment field of the new version the information on what was changed and when has to be inserted (e.g. "Version 2, 2017-01-10, faulty LWD values corrected"). See also here for an example.
 * searching for the old dataset to be hidden (e.g. search for the *ID in the "Data Set" window) and open it
 * in the 'References' tab click on Add/Remove in the lower part (References for PANGAEA data sets)
 * search for the new version and insert it here by double-clicking on it and then pressing OK
 * choose "Replaced by" in the Relation column
 * then go to the Basics tab, remove BSRN as a project (insert not_given as project) and press on save