LB2.0 pre-certification testing

From EgeeWiki

This Document deals with Pre-certification testing of the L&B Service, primarily the 2.0 release.

Contents

[edit]

Info

[edit]

Testbed

[edit]

Certificates

Required steps: 

#(changed to EGI trustanchors)yum -c http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/lcg-CA.repo install -y lcg-CA
wget -O /etc/yum.repos.d/egi-trustanchors.repo http://repository.egi.eu/sw/production/cas/1/current/repo-files/egi-trustanchors.repo
yum install lcg-CA
[edit]

Installation

TODO: update this (clean instalation vs upgrade, using generated yum repositories...)

[edit]

Repositories

[edit]

Remote 

wget http://glite.web.cern.ch/glite/glite_key_gd.asc
rpm --import glite_key_gd.asc

cat << EOF > /etc/yum.repos.d/glite-UI.repo
# This is the official YUM repository string for the glite 3.2 User Interface
# Fetched from: http:///grid-deployment.web.cern.ch/grid-deployment/yaim/repos/glite-UI.repo
# Place it to /etc/yum.repos.d/ and run 'yum update' 

[glite-UI]
name=gLite 3.2 User Interface
baseurl=http://glitesoft.cern.ch/EGEE/gLite/R3.2/glite-UI/sl5/x86_64/RPMS.release/, http://glitesoft.cern.ch/EGEE/gLite/R3.2/glite-GENERIC/sl5/x86_64/RPMS.release/
gpgkey=http://glite.web.cern.ch/glite/glite_key_gd.asc
gpgcheck=1
enabled=1
exclude=glite-lb* glite-jp* glite-jobid* lcg-vomscerts* glite-security-gss* glite-security-gsoap-plugin*

[glite-UI_updates]
name=gLite 3.2 User Interface
baseurl=http://glitesoft.cern.ch/EGEE/gLite/R3.2/glite-UI/sl5/x86_64/RPMS.updates/, http://glitesoft.cern.ch/EGEE/gLite/R3.2/glite-GENERIC/sl5/x86_64/RPMS.updates/
gpgkey=http://glite.web.cern.ch/glite/glite_key_gd.asc
gpgcheck=1
enabled=1
exclude=glite-lb* glite-jp* glite-jobid* lcg-vomscerts* glite-security-gss* glite-security-gsoap-plugin*

[glite-UI_ext]
name=gLite 3.2 User Interface
baseurl=http://glitesoft.cern.ch/EGEE/gLite/R3.2/glite-UI/sl5/x86_64/RPMS.externals/, http://glitesoft.cern.ch/EGEE/gLite/R3.2/glite-GENERIC/sl5/x86_64/RPMS.externals/
gpgcheck=0
enabled=1
exclude=glite-lb* glite-jp* glite-jobid* lcg-vomscerts* glite-security-gss* glite-security-gsoap-plugin*
EOF
[edit]

Local

createrepo utility required: 

yum -y install createrepo

prepared a package URLs list required (to insert to savannach patch or got from the patch): 

cat > ~/packagelist.txt
...

steps: 

cat << EOF > /etc/yum.repos.d/glite-local.repo
[main]
[glite-local]
name=Local glite RPMS repository
baseurl=file:///var/cache/glite-local
enabled=1
EOF

mkdir /var/cache/glite-local
cd /var/cache/glite-local
wget -i ~/packagelist.txt
createrepo .
yum clean all

[edit]

Installation & check packages

  • from glite-LB metapackage - made within LB subsystem since LB 2.0/1.9 
yum install glite-LB
  • then check the rpm list by: 
cd /var/cache/glite-local
rpm -ivh *.rpm

There should be no new packages installed.

[edit]

Configuration

[edit]

YAIM 

cd ~/
mkdir -m 700 yaim
cd yaim

dobre nastavit:

  • SITE_EMAIL
  • MYSQL_PASSWORD 
cat << EOF > site-info.def
MYSQL_PASSWORD=set_this_to_a_good_password
SITE_NAME=krakonosovo
SITE_EMAIL="valtri@civ.zcu.cz"
GLITE_LB_TYPE="both"
#GLITE_RTM_ENABLED=true
#SITE_LAT=49.7481
#SITE_LONG=13.3742
# lze od L&B > 3.0.10:
#GLITE_LB_WMS_DN='/DC=cz/DC=cesnet-ca/O=CESNET/CN=wms1.egee.cesnet.cz,/DC=cz/DC=cesnet-ca/O=CESNET/CN=wms2.egee.cesnet.cz'
EOF

/opt/glite/yaim/bin/yaim -c -s ./site-info.def -n glite-LB
[edit]

Startup

  • all services should be already running from the previous (yaim) step
  • to run manually: 
/etc/init.d/gLite start
[edit]

Testplan

[edit]

Tests 

yum install cvs make gcc globus-proxy-utils voms-clients

# needed only for harvester test
yum install postgresql-server
/etc/init.d/postgresql initdb || :
#nebo: /usr/bin/postgresql-setup
sed -i 's/^\(local.*\)\(peer\|ident\)/\1trust/' /var/lib/pgsql/data/pg_hba.conf
/etc/init.d/postgresql restart
createuser -U postgres -d -R -S rtm

grid-proxy-init -valid 100:0
#export GLITE_LOCATION=/opt/glite
export GLITE_LOCATION=/usr
export LB_HOST=`hostname -f`
export PATH=$GLITE_LOCATION/bin:$GLITE_LOCATION/sbin:$GLITE_LOCATION/lib/glite-lb/examples:$GLITE_LOCATION/lib64/glite-lb/examples:$PATH
export GLITE_LB_SERVER_WPORT=9003
export GLITE_LB_SERVER_PORT=9000
export GLITE_LB_LOGGER_PORT=9002
export GLITE_WMS_QUERY_SERVER=$LB_HOST:$GLITE_LB_SERVER_PORT
export GLITE_WMS_NOTIF_SERVER=$LB_HOST:$GLITE_LB_SERVER_PORT
export GLITE_WMS_LOG_DESTINATION=$LB_HOST:$GLITE_LB_LOGGER_PORT
#export GLITE_WMS_LBPROXY_STORE_SOCK=/tmp/lb_proxy_ # maybe should be default?
ulimit -c unlimited

cvs -d :pserver:anonymous@glite.cvs.cern.ch:/cvs/glite co org.glite.testsuites.ctb/LB
cd org.glite.testsuites.ctb/LB/tests
make
./lb-test-binaries.sh
sudo ./lb-test-logger-local.sh # or glite account
sudo ./lb-test-server-local.sh # or glite account
./lb-test-job-registration.sh
./lb-test-event-delivery.sh
./lb-test-job-states.sh
./lb-test-notif.sh
./lb-test-notif-recovery.sh
./lb-test-notif-switch.sh
./lb-test-notif-stream.sh
./lb-test-ws.sh
./lb-test-bdii.sh
./lb-test-changeacl.sh 
./lb-test-https.sh
./lb-test-sandbox-transfer.sh
./lb-test-threaded.sh
./lb-test-acl-authz.sh
X509_USER_PROXY_BOB=... ./lb-test-switch-owner.sh

# super user required
X509_USER_CERT=$HOME/.certs/hostcert.pem \
X509_USER_KEY=$HOME/.certs/hostkey.pem \
  ./lb-test-statistics.sh 
X509_USER_CERT=$HOME/.certs/hostcert.pem \
X509_USER_KEY=$HOME/.certs/hostkey.pem \
  ./lb-test-purge.pl --i-want-to-purge $LB_HOST:9000
X509_USER_CERT=$HOME/.certs/hostcert.pem \
X509_USER_KEY=$HOME/.certs/hostkey.pem \
  ./lb-test-job-registration.sh

#
# local tests
#
./lb-test-logevent.sh /var/glite/log/dglogd.log
# requires glite user
# requires GLITE_LB_TYPE="both" or GLITE_LB_TYPE="proxy"
./lb-test-proxy-delivery.sh
# glite user account required
./lb-test-il-recovery.sh
GLITE_MYSQL_ROOT_PASSWORD='[Edited]' ./lb-test-harvester.sh
./lb-test-permissions.sh

# specialities:
# - may require increase timeouts in tests
# - >10 requires more slaves
# - >50 requires increasing conpool
for i in `seq 1 10`; do ./lb-test-notif.sh & done; wait
for i in `seq 1 10`; do ./lb-test-notif-switch.sh & done; wait
[edit]

Real jobs (through VOCE or the world)

Make sure we have CA certificates on LB server: 

yum install ca_policy_eugridpma

Prepare environment: 

grid-proxy-init -old -valid 100:0
gsissh ui1.egee.cesnet.cz
 voms-proxy-init -voms voce -noregen -valid 100:0
 cvs -d :pserver:anonymous@glite.cvs.cern.ch:/cvs/glite co org.glite.testsuites.ctb
 cd org.glite.testsuites.ctb/LB/tests/

Normal launch would be: 

 ./lb-test-wild.sh LBSERVER:PORT

Less intrusive launch: 

 ./lb-test-wild.sh -n 1 LBSERVER:PORT

Even less intrusive, with each test separated: 

./lb-test-wild.sh -n 1 -f html --test done delwin.fi.muni.cz:9000
./lb-test-wild.sh -n 1 -f html --test cancel delwin.fi.muni.cz:9000
./lb-test-wild.sh -n 1 -f html --test fail delwin.fi.muni.cz:9000

./lb-test-wild.sh -n 1 -f html --test done_coll delwin.fi.muni.cz:9000
./lb-test-wild.sh -n 1 -f html --test fail_coll delwin.fi.muni.cz:9000
./lb-test-wild.sh -n 1 -f html --test cancel_coll delwin.fi.muni.cz:9000
[edit]

glite-PX

site-info: 

cd ~/
mkdir -m 700 yaim
cd yaim
cat << EOF > site-info.def
SITE_NAME=krakonosovo
PX_HOST=`hostname -f`
GRID_AUTHORIZED_RETRIEVERS="\*"
EOF

launch yaim: 

/opt/glite/yaim/bin/yaim -c -s ./site-info.def -n glite-PX

quick test: 

export PATH=$PATH:/opt/globus/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/globus/lib:/opt/glite/lib64

PX_HOST=`hostname -f`
myproxy-init -s $PX_HOST
myproxy-get-delegation -s $PX_HOST
myproxy-destroy -s $PX_HOST
Retrieved from "http://egee.cesnet.cz/mediawiki/index.php/LB2.0_pre-certification_testing"