Intern:WRMC Curator

The following information are specially made for data curators who import new measurements and perform the WRMC.

Contact: Dr. [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo], Alfred Wegener Institute, Bremerhaven

Data conversion and import to PANGAEA

 * (1) Move new incoming files from ftp.bsrn.awi.de/incoming to pub
 * Windows: putty account@ftp.bsrn.awi.de, OS X: ssh account@ftp.bsrn.awi.de (ask [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] or Rainer Sieger for account)
 * cd ../incoming
 * find moveable files with lsbsrn
 * cd ../xxx (xxx = station ID)
 * smv * ../../pub/xxx
 * cd ..
 * ... (next)
 * exit


 * (2) Download, unpack, and check new station-to-archive files with the BSRN Toolbox
 * Station-to-archive -> download Station-to-archive files


 * (3) Metadata -> create all metadata files
 * if a PANGAEA metadata ID is missing (station, staff or method) ->
 * create missing entries in accordant 4D tables and
 * edit the BSRN-ID file at http://www.pangaea.de/PHP/bsrn/BSRN_IDs.txt
 * start the BSRN Toolbox and again create all metadata files


 * (4) Data -> create all data files
 * (5) Check data files with PanPlot or ODV (radiosonde)
 * (6) Import -> create all import files
 * (7) Import files with 4D
 * (8) Check appearance and validity on the Internet and inform PI
 * (8) Edit files like http://www.pangaea.de/PHP/bsrn/BSRN_LR0100.txt and BSRN_Status.txt

Setting up a new BSRN-station

 * (1) To participate within the BSRN a station has to fulfill certain conditions.
 * (2) To become a BSRN-candidate the station must formally be accepted by the [mailto:Joseph.Michalsky@noaa.gov BSRN project manager].
 * (3) The station manager from the BSRN-candidate station should provide [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] with basic station/staff information.
 * (4) In return [mailto:Gert.Koenig-Langlo@awi.de Gert König-Langlo] sends ...
 * the station manager the individual ftp-account for data submission from the new station,
 * the station identification number and further information needed to produce the station-to-archive file.
 * (5) Within the WRMC the data curator ...
 * imports the new station into PANGAEA as "Event" which automatically updates http://www.bsrn.awi.de/en/stations/listings/,
 * imports the new station/staff/etc. information into PANGAEA,
 * updates the BSRN-ID-file,
 * updates Stations.kmz,
 * asks [mailto:Wolfgang.Cohrs@awi.de Wolfgang Cohrs] to update the maps,
 * asks [mailto:Rainer.Sieger@awi.de Rainer Sieger] to include the new station into the: BSRN_Toolbox
 * (6) The station manager creates, checks, compresses, and submits the first station-to-archive-file to ftp.bsrn.awi.de.
 * (7) After importing the first station-to-archive file into PANGAEA a new line for the new station must be included at the right alphabetic order in all files like http://www.pangaea.de/PHP/bsrn/BSRN_LR0100.txt and BSRN_Status.txt.

Update Stations.kmz
Minor changes can be done manually according to the following instructions: Alternatively, Pan2Applic may be used. As inputfile the output of http://www.pangaea.de/ddi?request=bsrn/BSRNEvent&format=textfile&title=BSRN+Stations is a good starting point.
 * (1) download Stations.kmz on your local computer,
 * decompress the local version of the file (*.kmz-files are in fact *.zip-files). The result is a file named BSRN_Stations.kml,
 * edit BSRN_Stations.kml,
 * compress BSRN_Stations.kml into Stations.zip (e.g. using 7-Zip),
 * rename Stations.zip into Stations.kmz,
 * replace Stations.kmz in the web.

PANGAEA import of new versions
If a dataset must be updated the new data should be imported as usual. Afterwards, the old version should be deleted using 4D to avoid multible datasets. While deleting a file a 4D dialog ask to enter the doi of the new dataset to insure that the old doi is still valid and lead to the new dataset. Overwriting a dataset is possible, but not recommended. The "Import/overwrite dataset" option in the BSRN-toolbox may help by small datasets if the BSRN_Dataset_IDs.txt is up-to-date. In any case datasets with more than 10.000 items should not be overwritten!

Editing PANGAEA data
Normally, archived data in PANGAEA will not be edited or deleted. Only in very specific cases (e.g. a single wrong default value) editing existing data makes more sense than importing a new version of a huge dataset. For editing single values 4D should be used:
 * searching for the dataset to be edited,
 * dragging the dataset on the "Data Series" button will open all data series from the dataset,
 * selecting the desired dataseries with a double click opens the dataseries,
 * by clicking on "List" the dataseries opens in a spread sheet which can be sorted (click on a parameter),
 * a selected datapoint can be edited or the whole (not the single value!!!) line can be deleted using "Strg D".
 * after editing/deleting please click first anywere to deselect the selection,
 * click the "Save" button,
 * close the "Data series" window,
 * open the corresponding dataset by doubleclicking,
 * tag "with cache new" and press the "Save" button to renew the edited dataset.

Deleting/Updating PANGAEA data
30 days after the import a permanent doi-number is attached to a dataset. Before, a dataset can be deleted without problems. Later, a dataset can be deleted only, if a new version of the dataset - carrying a new doi-number - exists. To delete a dataset 4D should be used:
 * searching for the dataset(s) to be deleted (e.g. search for the *ID after pushing "Data Set"). Multiple datasets can be selected,
 * push on "Delete Records" in the "List Special" menue,
 * after several warnings you will get asked for a doi if there is a new version available,
 * typing in a non existing doi (e.g. 4711) will simple delete the dataset,
 * typing in an existing doi of a new version from this dataset will delete the older dataset and link the old doi to the new dataset,
 * deleting takes as long as importing. Thus be patient! At the end you will get a notification that the deletion took place.