Intern:File Upload

PANGAEA File Upload Workflow

This a short overview about the current state of file uploads and file archiving.

Create data submission ticket
An author creates a submission request: https://www.pangaea.de/submit/

As curator, check submissions: https://issues.pangaea.de/projects/PDI

Request “upload link” for large file uploads
As curator click on “Request file upload”.

https://issues.pangaea.de/browse/PDI-24552

Upload files
Author follows generated link and uploads files to, e.g.

https://issues.pangaea.de/upload/?code=5ef052c2338474.86421682

Web page must be open while uploading. A page reload cancels current uploads. So wait!

Author clicks “Confirm file upload” to comment on submission ticket.

Check uploaded files
As curator, check files and file names e.g. for invalid characters.

Upload files are located on Isilon and can be edited there. This is the source.

After editing, e.g. file rename, you can check the upload overview:

https://issues.pangaea.de/upload/?code=5ef052c2338474.86421682

Copy files
Until yet, files are located only on the Isilon and not archived. So files must be copied from upload (or staging) area to the archive area (HSM).

Open a command line, terminal or Putty

Have a look to your home directory.

Check and decide where files should be archived on HSM. This is the target.

If the directory does not exists, create it and set permissions.

Use the pcp.sh script located in you home directory to copy files from source to target.

Create file list
A file list is created automatically during the copy process.

Optional: have a look to your home directory.

7. Get file list for editorial
Download and optionally edit the generated file list. There are two ways:

a) Got to the source directory on the Isilon. There is a dataset-PDI-*.txt file which represent the data matrix.

b) Use Filezilla or similar to download the file from pangaea-im2.awi.de

Upload the file list to Editorial
Open 4D and goto Import > Data > Open.