Intern:WRMC Curator

The following information are specially made for data curators who import new measurements and perform the WRMC.

Contact: Dr. [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo], Alfred Wegener Institute, Bremerhaven

Data conversion and import to PANGAEA

 * (1) Move new incoming files from ftp.bsrn.awi.de/incoming to pub
 * Windows: putty account@ftp.bsrn.awi.de, OS X: ssh account@ftp.bsrn.awi.de (ask [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] or Rainer Sieger for account)
 * cd ../incoming
 * find moveable files with lsbsrn
 * cd ../xxx (xxx = station ID)
 * smv * ../../pub/xxx
 * cd ..
 * ... (next)
 * exit


 * (2) Download, unpack, and check new station-to-archive files with the BSRN Toolbox
 * Station-to-archive -> download Station-to-archive files


 * (3) Metadata -> create all metadata files
 * if a PANGAEA metadata ID is missing (station, staff or method) ->
 * create missing entries in accordant 4D tables and
 * edit the BSRN-ID file
 * start the BSRN Toolbox and again create all metadata files


 * (4) Data -> create all data files
 * (5) Check data files with PanPlot or ODV (radiosonde)
 * (6) Import -> create all import files
 * (7) Import files with 4D
 * (8) Check appearance and validity on the Internet and inform PI
 * (8) Edit files like BSRN_LR0100.txt and BSRN_Status.txt

Setting up a new BSRN-station

 * (1) To participate within the BSRN a station has to fulfill certain conditions.
 * (2) To become a BSRN-candidate the station must formally be accepted by the [mailto:Chuck.Long@noaa.gov BSRN project manager].
 * (3) The station manager from the BSRN-candidate station should provide [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] with basic station/staff information.
 * (4) In return [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] sends ...
 * the station manager the individual ftp-account for data submission from the new station,
 * the station identification number and further information needed to produce the station-to-archive file.
 * (5) Within the WRMC the data curator ...
 * imports the new station into PANGAEA as "Event" which automatically updates the BSRN stations table,
 * imports the new station/staff/etc. information into PANGAEA,
 * updates the BSRN-ID-file
 * updates Stations.kmz,
 * asks [mailto:Wolfgang.Cohrs@awi.de Wolfgang Cohrs] to update the maps,
 * asks [mailto:Rainer.Sieger@awi.de Rainer Sieger] to include the new station into the: BSRN_Toolbox
 * (6) The station manager creates, checks, compresses, and submits the first station-to-archive-file to ftp.bsrn.awi.de.
 * (7) After importing the first station-to-archive file into PANGAEA a new line for the new station must be included at the right alphabetic order in all files like http://www.pangaea.de/PHP/bsrn/BSRN_LR0100.txt and BSRN_Status.txt.

Update Stations.kmz
Minor changes can be done manually according to the following instructions: Alternatively, Pan2Applic may be used. As inputfile the output of http://www.pangaea.de/ddi?request=bsrn/BSRNEvent&format=textfile&title=BSRN+Stations is a good starting point.
 * (1) download Stations.kmz on your local computer,
 * decompress the local version of the file (*.kmz-files are in fact *.zip-files). The result is a file named BSRN_Stations.kml,
 * edit BSRN_Stations.kml,
 * compress BSRN_Stations.kml into Stations.zip (e.g. using 7-Zip),
 * rename Stations.zip into Stations.kmz,
 * replace Stations.kmz in the web.

PANGAEA import of new versions
If a dataset must be updated the new data should be imported as usual. Afterwards, the old version is flagged as such (by inserting a comment AND by flagging it in 4D) and contains a link to the new version. The earlier practice of deleting the old version is no longer supported by PANGAEA (as of 2016-01-01). The old version of the dataset will be under password protection and will not appear in the data warehouse nor in any retrievals, therefore guaranteeing that data are not redundant.

=> Flagging the old version in 4D
 * Open the old dataset (old version) in 4D
 * Under "References" insert the new version as a Reference (PANGAEA data sets, lower part of window)
 * Set the relation "replaced by" for the new version
 * Save

Editing PANGAEA data
Normally, archived data in PANGAEA will not be edited or deleted. Only in very specific cases (e.g. a single wrong default value) editing existing data makes more sense than importing a new version of a huge dataset. For editing single values 4D should be used:
 * searching for the dataset to be edited,
 * dragging the dataset on the "Data Series" button will open all data series from the dataset,
 * selecting the desired dataseries with a double click opens the dataseries,
 * by clicking on "List" the dataseries opens in a spread sheet which can be sorted (click on a parameter),
 * a selected datapoint can be edited or the whole (not the single value!!!) line can be deleted using "Strg D".
 * after editing/deleting please click first anywere to deselect the selection,
 * click the "Save" button,
 * close the "Data series" window,
 * open the corresponding dataset by doubleclicking,
 * tag "with cache new" and press the "Save" button to renew the edited dataset.

Deleting/Updating PANGAEA data
30 days after the import a permanent doi-number is attached to a dataset. Before, a dataset can be deleted without problems. Later, a dataset can be deleted only, if a new version of the dataset - carrying a new doi-number - exists. To delete a dataset 4D should be used:
 * searching for the dataset(s) to be deleted (e.g. search for the *ID after pushing "Data Set"). Multiple datasets can be selected,
 * push on "Delete Records" in the "List Special" menue,
 * after several warnings you will get asked for a doi if there is a new version available,
 * typing in a non existing doi (e.g. 4711) will simple delete the dataset,
 * typing in an existing doi of a new version from this dataset will delete the older dataset and link the old doi to the new dataset,
 * deleting takes as long as importing. Thus be patient! At the end you will get a notification that the deletion took place.