User:Valtri/Jenkins

Frantisek Dvorak — Fri, 17 May 2013 08:33:38 GMT

/* Info */ - more gpg keys

New page

= Info =

* '''URL''': https://emian.zcu.cz:8443/jenkins/
* '''credentials''': root@emain.zcu.cz:/etc/tomcat6/tomcat-users.xml
* '''platforms''': SL5/32, SL5/64, SL6/32, SL6/64, Fedora 17/64, Fedora 18/64, Debian 6/64
* '''gpg keys''':
** http://scientific.zcu.cz/repos/jenkins-builder.asc (main) - 7BBFDF7C
** http://scientific.zcu.cz/repos/jenkins-builder-sl5.asc (for SL5) - 64FA8786
** http://scientific.zcu.cz/repos/valtri.asc (externals) - 7157882F

= Builds =

There are two parts of each build:
# '''source''' build (canl-c-sources-*, gridsite-sources-*, glite-lb-soures-*, ...)
# '''binary''' build (canl-c-build-*, gridsite-build-*, glite-lb-build-*, ...)

The reason is to have the same source tarballs for all binary builds.

There are different builds for each branch:
# '''devel''' - current development branches (master, g_with_canl_1_0, ...)
# '''release''' - latest tagged versions

== Cloning configurations ==

The origin for all configurations should be the development ones.

Create or update release configurations:

# create a new job COMPONENT-build-release
#* clone of COMPONENT-build-devel (or clone of other active branch)
#* (keep "Discard Old Builds")
#* replace download in Build/Commands: from '*-sources-devel" to "*-sources-release"
# create a new job COMPONENT-sources-release
#* clone of COMPONENT-sources-devel (or clone of other active branch)
#* (keep "Discard Old Builds")
#* edit git tag (or update build commands if checkout is there)
#* remove "Build/Triggers/Poll SCM"
#* comment out sed on project/version.properties (and update Makefile.inc if it's gridsite)
#* change "Post-build Actions/Build other project": from COMPONENT-build-devel to COMPONENT-build-release

= Tricks =

'''Trigger Build''':
wget -q http://localhost:8080/jenkins/job/log4c/build?token=BUILD

= Installation =

* '''Key preparation''':
# X509 -> pkcs12
# (enter some passphrase)
openssl pkcs12 -export -in server.crt -inkey server.key \
-out server.p12 -name some-alias \
-certfile concatenedCAFiles.pem
# pkcs12 -> java
# (use the same password for store and for key)
keytool -importkeystore \
-deststorepass changeit -destkeypass changeit -destkeystore server.keystore \
-srckeystore server.p12 -srcstoretype PKCS12 -srcstorepass some-password \
-alias some-alias

* '''User''':
adduser --disabled-password jenkins
usermod -a -G jenkins tomcat6
#? usermod -a -G tomcat6 jenkins
chmod -R g+w /home/jenkins

* '''Tomcat''':
<tt>/etc/tomcat6/server.xml</tt> - <tt>Connector</tt>:

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="want" sslProtocol="TLS"
keystorePass="*CENSOORED*" URIEncoding="UTF-8" />

<tt>/etc/default/tomcat6</tt> - java opts:
# increase maximal memory heap
# set jenkins home
# for example:
JAVA_OPTS="-Djava.awt.headless=true -Xmx512m -XX:+UseConcMarkSweepGC -DJENKINS_HOME=/home/jenkins"

* '''Tomcat6 & Debian''':
sed -i 's/.*\(TOMCAT6_SECURITY\)=.*/\1=yes/' /etc/default/tomcat6

# TODO: specific properties
# TODO2: properties for who?
cat >/etc/tomcat6/policy.d/04webapps.policy <<EOF
grant {
permission java.util.PropertyPermission "*", "read";
};
grant codeBase "file:/var/lib/tomcat6/webapps/jenkins/-" {
permission java.security.AllPermission;
};
grant codeBase "file:/home/jenkins/-" {
permission java.security.AllPermission;
};
EOF

* '''GUI''':
** Configure Global Security
*** Matrix-based security ...
*** Security Realm: Delegate to servlet container (then edit <tt>/etc/tomcat6/tomcat-users.xml</tt>)
** Manage Plugins
*** update
*** install "Git Plugin", "Jenkins Mailer", "GitHub Plugin"
** copy https://software.sandia.gov/trac/fast/browser/hudson/master/launch_krb5_slave.sh to <tt>~jenkins/scripts</tt> and adjust

= Node preparation =

* we need gpg key for Jenkins
** beware of SL 5: there is limited support of keys, use SL 5 gnupg defaults to generate

Now, let's compare SL/Fedora build system setup (one useradd) with the Debian machinery. :-)

== Scientific Linux ==
~/guest/epel.sh

# SELinux
...
# for SL5:
semanage fcontext -a -s system_u -t user_home_t '/var/lib/jenkins/.+'
semanage fcontext -a -s system_u -t krb5_home_t '/var/lib/jenkins/\.k5login'
restorecon -Rv /var/lib/jenkins
# for SL6:
semanage fcontext -a -e /home/SOME_USER /var/lib/jenkins
restorecon -R -v /var/lib/jenkins

yum install createrepo rpm-sign
# install old version of mock and add exception to repository
#rpm -i mock-1.0.28-1.el5.noarch.rpm
#rpm -i mock-1.1.21-1.el6.noarch.rpm
...

yum install policycoreutils-python
yum install java-1.7.0-openjdk

useradd -m -d /var/lib/jenkins -u 1000 jenkins
echo "jenkins/emian.zcu.cz@ZCU.CZ" > ~jenkins/.k5login
chown jenkins:jenkins ~jenkins/.k5login
usermod -a -G mock jenkins
echo "%_gpg_name Jenkins Builder <jenkins@emian.zcu.cz>" >> ~jenkins/.rpmmacros
chown jenkins:jenkins ~jenkins/.rpmmacros

# copy .gnupg
# beware for SL5: RSA >= 2048 bit doesn't work, use gnupg default
...
chown -R jenkins:jenkins ~jenkins/.gnupg/
#rpm --import ...
...

== Fedora ==
yum install createrepo rpm-sign
# install old version of mock and add exception to repository
#rpm -i mock-1.1.26-2.fc17.noarch.rpm
#rpm -i mock-1.1.26-2.fc18.noarch.rpm
...

useradd -m -d /var/lib/jenkins -u 1000 jenkins
echo "jenkins/emian.zcu.cz@ZCU.CZ" > ~jenkins/.k5login
chown jenkins:jenkins ~jenkins/.k5login
usermod -a -G mock jenkins
echo "%_gpg_name Jenkins Builder <jenkins@emian.zcu.cz>" >> ~jenkins/.rpmmacros
chown jenkins:jenkins ~jenkins/.rpmmacros

# copy .gnupg
...
chown -R jenkins:jenkins ~jenkins/.gnupg/
#rpm --import ...
...

== Debian ==

'''pbuilder machinery''':
cat > /root/.pbuilderrc <<EOF
OTHERMIRROR='deb http://scientific.zcu.cz/repos/EMI3-external/debian/ stable/|deb file:///home/jenkins/tomcat-debian-externals/ stable/|deb file:///home/jenkins/tomcat-debian/ stable/'
BINDMOUNTS="/home/jenkins/tomcat-debian-externals /home/jenkins/tomcat-debian"
EOF

cat > ~jenkins/.pbuilderrc <<EOF
OTHERMIRROR='deb http://scientific.zcu.cz/repos/EMI3-external/debian/ stable/|deb file:///home/jenkins/debian-externals/ stable/|deb file:///home/jenkins/debian/ stable/'
BINDMOUNTS="/home/jenkins/debian-externals /home/jenkins/debian"
EOF

cat > ~jenkins/.mini-dinstall.conf <<EOF
[DEFAULT]
archive_style=flat
archivedir=~/debian
architectures=all, i386, powerpc, sparc, amd64
mail_on_success=false
generate_release=true

[stable]
alias=squeezy

[unstable]
alias=wheezy
EOF

cat > ~jenkins/.mini-dinstall-externals.conf <<EOF
[DEFAULT]
archive_style=flat
archivedir=~/debian-externals
architectures=all, i386, powerpc, sparc, amd64
mail_on_success=false
generate_release=true

[stable]
alias=squeezy

[unstable]
alias=wheezy
EOF

cat > ~jenkins/.dput.cf <<EOF
[local]
fqdn = localhost
method = local
incoming = ~/debian/mini-dinstall/incoming
run_dinstall = 0
post_upload_command = /usr/bin/mini-dinstall -v --debug --batch ~/debian

[local_externals]
fqdn = localhost
method = local
incoming = ~/debian-externals/mini-dinstall/incoming
run_dinstall = 0
post_upload_command = /usr/bin/mini-dinstall -v --debug --batch -c ~/.mini-dinstall-externals.conf ~/debian-externals
EOF

chown jenkins:jenkins ~jenkins/.pbuilderrc ~jenkins/.mini-dinstall.conf ~jenkins/.mini-dinstall-externals.conf ~jenkins/.dput.cf
mkdir -p /home/jenkins/debian/mini-dinstall/incoming /home/jenkins/debian-externals/mini-dinstall/incoming
chown jenkins:jenkins /home/jenkins/debian/mini-dinstall/incoming /home/jenkins/debian-externals/mini-dinstall/incoming

# to build by Jenkins under tomcat too
cd ~jenkins
cp .dput.cf .mini-dinstall*.conf .pbuilderrc ~tomcat6/
cd ~tomcat6
sed -i 's,~,/home/jenkins,' .dput.cf .mini-dinstall*.conf
chown tomcat6:tomcat6 .dput.cf .mini-dinstall*.conf .pbuilderrc

cp -rp ~jenkins/.gnupg ~tomcat6/
chown -R ~tomcat6/.gnupg

'''/etc/sudoers''':
jenkins,tomcat6 ALL=(root) NOPASSWD: SETENV: /usr/sbin/pbuilder --build --buildresult /home/jenkins/[a-zA-Z0-9_/-]* --debbuildopts [ ]--debbuildopts [ ]--hookdir /home/jenkins/[a-zA-Z0-9_/-]* *
jenkins,tomcat6 ALL=(root) NOPASSWD: /bin/rm -rfv results, /bin/rm -rfv results/pbuilder

'''initial image''':
pbuilder --create --distribution squeeze

User:Valtri

Frantisek Dvorak — Thu, 16 May 2013 20:05:59 GMT

 * VOMS setup-script: [[Image:Voms-install.sh]]
+'''Jenkins'''':
+* [[User:Valtri/Jenkins]]

EgeeWiki - Recent changes [en]

User:Valtri/Jenkins

User:Valtri