VOCE Utilization
Here you can find a detailed recipe how to get access to VOCE computational resources and how to work in the VOCE environment.
The instructions for utilization of VOCE resources are also available for download in PDF format. If you have any comments or suggestions how this inscructions should be improved or modified, please contact us using the address voce@cesnet.cz.
Recommended Itinerary for Proper Utilization of VOCE Computing Resources
The EGEE Computing Grid Project
2005-04-08 (version 0.2)
Introduction
This tutorial contains step-by-step instructions concerning proper usage of computational resources belonging to VOCE infrastructure.
Recommended software
Software packages mentioned below are either part of the system installation or can be installed from the VOCE software repository (link will be added later).
SSH client
MS Windows: PuTTy (possibly with WinSCP )
Linux: usually part of the system installation
Internet browser (needed for user registration to VOCE)
MS Windows: Internet Explorer 5.0 or higher, Mozilla 1.6 or higher, FireFox 1.0 or higher
Linux: Mozilla 1.6 or higher, FireFox 1.0 or higher
Step 1: Certificate Management
Every VOCE user has to prove his/her identity before he/she is allowed to use VOCE resources. This requirement is fulfilled by usage of the X.509 Public Key Infrastructure (PKI) for user and service authentication. All communicating entities have to prove their identity by a valid certificate. Before beginning to use the grid environment it is necessary to obtain and properly install a user personal certificate. This has to be done by following subsequent steps
0) Prerequisities: Successful registration to VOCE is expected prior to login to VOCE facilities and utilization of VOCE resources.
1) Obtain your personal certificate from an approved certification authority (CA)
Currently each country participating in VOCE has a local CA, which has been accredited to issue certificates for Grid users. Follow links from http://www.eugridpma.org/members/worldmap/ to find a CA that serves users in your country (for Europe users). This link is for the others.
2) Login to VOCE UI (User Infertace)
VOCE UI (ui1.egee.cesnet.cz) is running under Unix operating system, therefore all commands written below have a syntax of an unix shell commands and have to be entered into a unix shell command line on UI. Moreover the commands for submitting a job, checking its status and so on are part of the grid middleware installed at UI therefore you have to connect to UI using tools listed above to be able to proceed with following steps. Please, note that Unix and grid middleware commands listed below utilize VOCE resources by the commad line interface (CLI) approach.
To login to VOCE UI (ui1.egee.cesnet.cz) use SSH client (listed above) using a login name and a password set during the registration process. Any time later you can change your password using thekpasswd command from the UI. You can also use authentication using your GSI proxy certificate. In order to use this method you will need a GSI-enabled version of the SSH client, source code for such a client can be downloaded. Binary versions for Linux and MS Windows will be provided later.
There is no need to install any UI on your computer if you use the P-GRADE portal for accessing VOCE. By P-GRADE portal you can graphically construct workflow (and single job) applications consisting of sequential and MPI jobs. P-GRADE portal takes care of your whole workflow's execution by managing the necessary job submissions and file transfers. It also provides on-line workflow execution visualization and animation as well as visual presentation of the VOCE information system. Supports certificate proxy management and grid resource customization. More information can be found at http://www.lpds.sztaki.hu/pgportal/.
Please, refer also how to import your certificate into a browser.
3) Copy your certificate and corresponding private key on your account on your preferred UI and store them in .globus subdirectory of your home directory
On the UI:
mkdir ~/.globus
chmod 700 ~/.globus
On your local machine:
scp <your_certificate> ui1.egee.cesnet.cz:.globus/usercert.pem
scp <your_private_key> ui1.egee.cesnet.cz:.globus/userkey.pem
Do not forget to set permission properly so you are the only person who can read the private key:
chmod 600 ~/.globus/userkey.pem
Step 2: Usage of Proxy Certificate
Your long-term private key is encrypted with a password, which must be entered before each use of the key. As the private key is needed whenever you perform authentication (e.g. during job submission) it would be uncomfortable for users to enter their password every time. In order to allow users easier work the Grid environment uses proxy certificates. Proxy certificate is a short-time certificate (usually valid for 12 hours) that is generated using the standard user's long-term credential. Private keys of proxy certificates are not encrypted and are stored on a local disk in files that are only readable for the proxy certificate owner. This proxy certificate is used instead of the long-term credential and applications use it automatically without requiring users to enter any additional passwords.
1) Create your grid proxy.
voms-proxy-init
Creates a proxy and stores it on a disk. You will be asked for your GRID pass phrase, which is the password to your private key set during the process of getting your certificate from your CA. The proxy certificate will be valid for 12 hours, so it is sufficient to issue this command once a day.
2) Check the existence of the grid proxy.
voms-proxy-info
3) Destroying the grid proxy
voms-proxy-destroy
It is more secure to explicitely destroy the proxy certificate when you finish your work with the Grid.
Step 3 MyProxy Management
!!! Before you start to use any commands related to MyProxy Management it is necessary to create your proxy certificate according to instructions described above !!!
The MyProxy server is a repository that allows user to store their certificates. A special password is used to secure access to certificates used in the MyProxy repository. The MyProxy server is used to issue a short-lifetime proxy on your behalf and is used for grid portals and support of long-time jobs. If you want to either run jobs which will last for couple of hours or you use graphical user interface (GUI) e.g. GENIUS, you will have to store your certificate to the VOCE MyProxy server (myproxy1.egee.cesnet.cz)
1) Create a delegator for proxies issuing.
myproxy-init -s myproxy1.egee.cesnet.cz
This creates the delegator that will issue proxies on your behalf. There is a hidden voms-proxy-*init in this command: it first creates a proxy to allow the MyProxy server to create the delegator. To create the delegator your certificate is used. The delegator has a lifetime of 1 week by default. After that, you need to re-run this step. You will be asked for the password of your certificate. You will be asked to enter a new password to be associated with your MyProxy delegation. It is recommended that this should be different to that in your certificate.
2) Print out the information concerning the delegator.
myproxy-info -s myproxy1.egee.cesnet.cz
Display information - e.g. remaining lifetime - about the delegator.
3) Erase available grid proxy (if any)
voms-proxy-destroy
4) Obtain a new proxy from MyProxy server.
myproxy-get-delegation -s myproxy1.egee.cesnet.cz
Get a new proxy from the MyProxy server. It has a default lifetime of 12 hours. This requires the password set in step 2. It is this step that GENIUS will perform when a user requests a Grid service.
5) Check the proxy info.
voms-proxy-info
Shows the proxy retrieved from the MyProxy server.
Step 4: Job Management
All operations concerning job submission, checking job status, retrieving job output and many more can be either performed using command line interface (CLI) or graphical user interface (GUI). The basic way how to continue using CLI approach is described below:
1) Check the available Computing Elements (CE) for your job.
glite-wms-job-list-match hostname.jdl
This shows the Computing Elements that are available for your job specified by the *.jdl file. The check of all requirements inside the *.jdl file will be performed and only CEs fully compatible with the requirements will be listed out.
2) Submit the job.
glite-wms-job-submit hostname.jdl
This command submits the job. The job will be assigned by a URI that is the job-identifier - a unique string.
3) Check the status of submitted job.
glite-wms-job-status https://grid....<rest of URI>
Gives the status of selected job; can be repeated until the command returns "Done" with "error code=0".
4) Prepare the output directory for finished job.
mkdir output
Make a directory to keep output away from other files.
5) Retrieve complete job output.
glite-wms-job-output --dir output https://grid....<rest of URI>
Gets your output from the Resource Broker (RB), which holds the output sandbox (so there is no direct contact with the machine on which the job was executed).
Comments
The detailed instructions are based on NA4 Open Meeting Quick Help tutorials.
Additional information can be found at following sources:
[1] https://gilda.ct.infn.it/video.html
[3] http://proj-lcg-security.web.cern.ch/proj-lcg-security/security_policy.html
[4] http://lcg.web.cern.ch/LCG/peb/grid_deployment/user_intro.htm