As I mentioned in my previous post, I’m not really a fan of giant shell scripts which ask for unrestricted sudo access without telling you what they’re going todo. Unfortunately DevStack is one such script :-( So I decided to investigate just what it does to a Fedora 17 host when it is run. The general idea I had was
- Install a generic Fedora 17 guest
- Create a QCow2 image using the installed image as its backing file
- Reconfigure the guest to use the QCow2 image as its disk
- Run DevStack in the guest
- Compare the contents of the original installed image and the DevStack processed image
It sounded like libguestfs ought to be able to help out with the last step, and after a few words with Rich, I learnt about use of virt-ls for exactly this purpose. After trying this once, it quickly became apparent that just comparing the lists of files is quite difficult because DevStack installs a load of extra RPMs with many 1000’s of files. So to take this out of the equation, I grabbed the /var/log/yum.log file to get a list of all RPMs that DevStack had installed, and manually added them into the generic Fedora 17 guest base image. Now I could re-run DevStack again and do a file comparison which excluded all the stuff installed by RPM.
RPM packages installed (with YUM)
- apr-1.4.6-1.fc17.x86_64
- apr-util-1.4.1-2.fc17.x86_64
- apr-util-ldap-1.4.1-2.fc17.x86_64
- augeas-libs-0.10.0-3.fc17.x86_64
- binutils-2.22.52.0.1-10.fc17.x86_64
- boost-1.48.0-13.fc17.x86_64
- boost-chrono-1.48.0-13.fc17.x86_64
- boost-date-time-1.48.0-13.fc17.x86_64
- boost-filesystem-1.48.0-13.fc17.x86_64
- boost-graph-1.48.0-13.fc17.x86_64
- boost-iostreams-1.48.0-13.fc17.x86_64
- boost-locale-1.48.0-13.fc17.x86_64
- boost-program-options-1.48.0-13.fc17.x86_64
- boost-python-1.48.0-13.fc17.x86_64
- boost-random-1.48.0-13.fc17.x86_64
- boost-regex-1.48.0-13.fc17.x86_64
- boost-serialization-1.48.0-13.fc17.x86_64
- boost-signals-1.48.0-13.fc17.x86_64
- boost-system-1.48.0-13.fc17.x86_64
- boost-test-1.48.0-13.fc17.x86_64
- boost-thread-1.48.0-13.fc17.x86_64
- boost-timer-1.48.0-13.fc17.x86_64
- boost-wave-1.48.0-13.fc17.x86_64
- ceph-0.44-5.fc17.x86_64
- check-0.9.8-5.fc17.x86_64
- cloog-ppl-0.15.11-3.fc17.1.x86_64
- cpp-4.7.2-2.fc17.x86_64
- curl-7.24.0-5.fc17.x86_64
- Django-1.4.2-1.fc17.noarch
- django-registration-0.7-3.fc17.noarch
- dmidecode-2.11-8.fc17.x86_64
- dnsmasq-utils-2.63-1.fc17.x86_64
- ebtables-2.0.10-5.fc17.x86_64
- euca2ools-2.1.1-2.fc17.noarch
- gawk-4.0.1-1.fc17.x86_64
- gcc-4.7.2-2.fc17.x86_64
- genisoimage-1.1.11-14.fc17.x86_64
- git-1.7.11.7-1.fc17.x86_64
- glusterfs-3.2.7-2.fc17.x86_64
- glusterfs-fuse-3.2.7-2.fc17.x86_64
- gnutls-utils-2.12.17-1.fc17.x86_64
- gperftools-libs-2.0-5.fc17.x86_64
- httpd-2.2.22-4.fc17.x86_64
- httpd-tools-2.2.22-4.fc17.x86_64
- iptables-1.4.14-2.fc17.x86_64
- ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch
- iscsi-initiator-utils-6.2.0.872-18.fc17.x86_64
- kernel-headers-3.6.6-1.fc17.x86_64
- kpartx-0.4.9-26.fc17.x86_64
- libaio-0.3.109-5.fc17.x86_64
- libcurl-7.24.0-5.fc17.x86_64
- libmpc-0.9-2.fc17.2.x86_64
- libunwind-1.0.1-3.fc17.x86_64
- libusal-1.1.11-14.fc17.x86_64
- libvirt-0.9.11.7-1.fc17.x86_64
- libvirt-client-0.9.11.7-1.fc17.x86_64
- libvirt-daemon-0.9.11.7-1.fc17.x86_64
- libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64
- libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64
- libvirt-python-0.9.11.7-1.fc17.x86_64
- libwsman1-2.2.7-5.fc17.x86_64
- lzop-1.03-4.fc17.x86_64
- m2crypto-0.21.1-8.fc17.x86_64
- mod_wsgi-3.3-2.fc17.x86_64
- mx-3.2.3-1.fc17.x86_64
- mysql-5.5.28-1.fc17.x86_64
- mysql-libs-5.5.28-1.fc17.x86_64
- MySQL-python-1.2.3-5.fc17.x86_64
- mysql-server-5.5.28-1.fc17.x86_64
- netcf-libs-0.2.2-1.fc17.x86_64
- numad-0.5-4.20120522git.fc17.x86_64
- numpy-1.6.2-1.fc17.x86_64
- parted-3.0-10.fc17.x86_64
- perl-AnyEvent-5.27-7.fc17.noarch
- perl-AnyEvent-AIO-1.1-8.fc17.noarch
- perl-AnyEvent-BDB-1.1-7.fc17.noarch
- perl-Async-MergePoint-0.03-7.fc17.noarch
- perl-BDB-1.88-5.fc17.x86_64
- perl-common-sense-3.5-1.fc17.noarch
- perl-Compress-Raw-Bzip2-2.052-1.fc17.x86_64
- perl-Compress-Raw-Zlib-2.052-1.fc17.x86_64
- perl-Config-General-2.50-6.fc17.noarch
- perl-Coro-6.07-3.fc17.x86_64
- perl-Curses-1.28-5.fc17.x86_64
- perl-DBD-MySQL-4.020-2.fc17.x86_64
- perl-DBI-1.617-1.fc17.x86_64
- perl-Encode-Locale-1.02-5.fc17.noarch
- perl-Error-0.17016-7.fc17.noarch
- perl-EV-4.03-8.fc17.x86_64
- perl-Event-1.20-1.fc17.x86_64
- perl-Event-Lib-1.03-16.fc17.x86_64
- perl-Git-1.7.11.7-1.fc17.noarch
- perl-Glib-1.241-2.fc17.x86_64
- perl-Guard-1.022-1.fc17.x86_64
- perl-Heap-0.80-10.fc17.noarch
- perl-HTML-Parser-3.69-3.fc17.x86_64
- perl-HTML-Tagset-3.20-10.fc17.noarch
- perl-HTTP-Date-6.00-3.fc17.noarch
- perl-HTTP-Message-6.03-1.fc17.noarch
- perl-IO-AIO-4.15-1.fc17.x86_64
- perl-IO-Async-0.29-7.fc17.noarch
- perl-IO-Compress-2.052-1.fc17.noarch
- perl-IO-Socket-SSL-1.66-1.fc17.noarch
- perl-IO-Tty-1.10-5.fc17.x86_64
- perl-LWP-MediaTypes-6.01-4.fc17.noarch
- perl-Net-HTTP-6.02-2.fc17.noarch
- perl-Net-LibIDN-0.12-8.fc17.x86_64
- perl-Net-SSLeay-1.48-1.fc17.x86_64
- perl-POE-1.350-2.fc17.noarch
- perl-Socket6-0.23-8.fc17.x86_64
- perl-Socket-GetAddrInfo-0.19-1.fc17.x86_64
- perl-TermReadKey-2.30-14.fc17.x86_64
- perl-TimeDate-1.20-6.fc17.noarch
- perl-URI-1.60-1.fc17.noarch
- ppl-0.11.2-8.fc17.x86_64
- ppl-pwl-0.11.2-8.fc17.x86_64
- pylint-0.25.1-1.fc17.noarch
- python-amqplib-1.0.2-3.fc17.noarch
- python-anyjson-0.3.1-3.fc17.noarch
- python-babel-0.9.6-3.fc17.noarch
- python-BeautifulSoup-3.2.1-3.fc17.noarch
- python-boto-2.5.2-1.fc17.noarch
- python-carrot-0.10.7-4.fc17.noarch
- python-cheetah-2.4.4-2.fc17.x86_64
- python-cherrypy-3.2.2-1.fc17.noarch
- python-coverage-3.5.1-0.3.b1.fc17.x86_64
- python-crypto-2.6-1.fc17.x86_64
- python-dateutil-1.5-3.fc17.noarch
- python-devel-2.7.3-7.2.fc17.x86_64
- python-docutils-0.8.1-3.fc17.noarch
- python-eventlet-0.9.17-1.fc17.noarch
- python-feedparser-5.1.2-2.fc17.noarch
- python-gflags-1.5.1-2.fc17.noarch
- python-greenlet-0.3.1-11.fc17.x86_64
- python-httplib2-0.7.4-6.fc17.noarch
- python-iso8601-0.1.4-4.fc17.noarch
- python-jinja2-2.6-2.fc17.noarch
- python-kombu-1.1.3-2.fc17.noarch
- python-lockfile-0.9.1-2.fc17.noarch
- python-logilab-astng-0.23.1-1.fc17.noarch
- python-logilab-common-0.57.1-2.fc17.noarch
- python-lxml-2.3.5-1.fc17.x86_64
- python-markdown-2.1.1-1.fc17.noarch
- python-migrate-0.7.2-2.fc17.noarch
- python-mox-0.5.3-4.fc17.noarch
- python-netaddr-0.7.5-4.fc17.noarch
- python-nose-1.1.2-2.fc17.noarch
- python-paramiko-1.7.7.1-2.fc17.noarch
- python-paste-deploy-1.5.0-4.fc17.noarch
- python-paste-script-1.7.5-4.fc17.noarch
- python-pep8-1.0.1-1.fc17.noarch
- python-pip-1.0.2-2.fc17.noarch
- python-pygments-1.4-4.fc17.noarch
- python-qpid-0.18-1.fc17.noarch
- python-routes-1.12.3-3.fc17.noarch
- python-setuptools-0.6.27-2.fc17.noarch
- python-sphinx-1.1.3-1.fc17.noarch
- python-sqlalchemy-0.7.9-1.fc17.x86_64
- python-suds-0.4.1-2.fc17.noarch
- python-tempita-0.5.1-1.fc17.noarch
- python-unittest2-0.5.1-3.fc17.noarch
- python-virtualenv-1.7.1.2-2.fc17.noarch
- python-webob-1.1.1-2.fc17.noarch
- python-wsgiref-0.1.2-8.fc17.noarch
- pyxattr-0.5.1-1.fc17.x86_64
- PyYAML-3.10-3.fc17.x86_64
- qemu-common-1.0.1-2.fc17.x86_64
- qemu-img-1.0.1-2.fc17.x86_64
- qemu-system-x86-1.0.1-2.fc17.x86_64
- qpid-cpp-client-0.18-5.fc17.x86_64
- qpid-cpp-server-0.18-5.fc17.x86_64
- radvd-1.8.5-3.fc17.x86_64
- screen-4.1.0-0.9.20120314git3c2946.fc17.x86_64
- scsi-target-utils-1.0.24-6.fc17.x86_64
- seabios-bin-1.7.1-1.fc17.noarch
- sg3_utils-1.31-2.fc17.x86_64
- sgabios-bin-0-0.20110622SVN.fc17.noarch
- spice-server-0.10.1-5.fc17.x86_64
- sqlite-3.7.11-3.fc17.x86_64
- tcpdump-4.2.1-3.fc17.x86_64
- vgabios-0.6c-4.fc17.noarch
- wget-1.13.4-7.fc17.x86_64
- xen-libs-4.1.3-5.fc17.x86_64
- xen-licenses-4.1.3-5.fc17.x86_64
Python packages installed (with PIP)
These all ended up in /usr/lib/python2.7/site-packages :-(
- WebOp
- amqplib
- boto (splattering over existing boto RPM package with older version)
- cinderclient
- cliff
- cmd2
- compressor
- django_appconf
- django_compressor
- django_openstack_auth
- glance
- horizon
- jsonschema
- keyring
- keystoneclient
- kombu
- lockfile
- nova
- openstack_auth
- pam
- passlib
- prettytable
- pyparsing
- python-cinderclient
- python-glanceclient
- python-novaclient
- python-openstackclient
- python-quantumclient
- python-swiftclient
- pytz
- quantumclient
- suds
- swiftclient
- warlock
- webob
Files changed
- /etc/group (added $USER to ‘libvirtd’ group)
- /etc/gshadow (as above)
- /etc/httpd/conf/httpd.conf (changes Listen 80 to Listen 0.0.0.0:80)
- /usr/lib/python2.7/site-packages/boto (due to overwriting RPM provided boto)
- /usr/bin/cq (as above)
- /usr/bin/elbadmin (as above)
- /usr/bin/list_instances (as above)
- /usr/bin/lss3 (as above)
- /usr/bin/route53 (as above)
- /usr/bin/s3multiput (as above)
- /usr/bin/s3put (as above)
- /usr/bin/sdbadmin (as above)
Files created
- /etc/cinder/*
- /etc/glance/*
- /etc/keystone/*
- /etc/nova/*
- /etc/httpd/conf.d/horizon.conf
- /etc/polkit-1/localauthority/50-local.d/50-libvirt-reomte-access.pkla
- /etc/sudoers.d/50_stack_sh
- /etc/sudoers.d/cinder-rootwrap
- /etc/sudoers.d/nova-rootwrap
- $DEST/cinder/*
- $DEST/data/*
- $DEST/glance/*
- $DEST/horizon/*
- $DEST/keystone/*
- $DEST/noVNC/*
- $DEST/nova/*
- $DEST/python-cinderclient/*
- $DEST/python-glanceclient/*
- $DEST/python-keystoneclient/*
- $DEST/python-novaclient/*
- $DEST/python-openstackclient/*
- /usr/bin/cinder*
- /usr/bin/glance*
- /usr/bin/keystone*
- /usr/bin/nova*
- /usr/bin/openstack
- /usr/bin/quantum
- /usr/bin/swift
- /var/cache/cinder/*
- /var/cache/glance/*
- /var/cache/keystone/*
- /var/cache/nova/*
- /var/lib/mysql/cinder/*
- /var/lib/mysql/glance/*
- /var/lib/mysql/keystone/*
- /var/lib/mysql/mysql/*
- /var/lib/mysql/nova/*
- /var/lib/mysql/performance_schema/*
Thoughts on installation
As we can see from the details above, DevStack does a very significant amount of work as root using sudo. I had fully expected that it was installing RPMs as root, but I had not counted on it adding extra python modules into /usr/lib/python-2.7, nor the addition of files in /etc/, /var or /usr/bin. I had set the $DEST environment variable for DevStack naively assuming that it would cause it to install everything possible under that location. In fact the $DEST variable was only used to control where the GIT checkouts of each openstack component went, along with a few misc files in $DEST/files/
IMHO a development environment setup tool should do as little as humanely possible as root. From the above list of changes, the only things that I believe justify use of sudo privileges are:
- Installation of RPMs from standard YUM repositories
- Installation of /etc/sudoers.d/ config files
- Installation of /etc/polkit file to grant access to libvirtd
Everything else is capable of being 100% isolated from the rest of the OS, under the $DEST directory location. Taking that into account my preferred development setup would be
$DEST
+- nova (GIT checkout)
+- ...etc... (GIT checkout)
+- vroot
+- bin
| +- nova
| +- ...etc...
+- etc
| +- nova
| +- ...etc...
+- lib
| +- python2.7
| +- site-packages
| +- boto
| +- ...etc...
+- var
+- cache
| +- nova
| +- ...etc...
+- lib
+- mysql
This would imply running a private copy of qpid, mysql and httpd, ideally all inside the same screen session as the rest of the OpenStack services, using unprivileged ports. Even if we relied on the system instances of qpid, mysql, httpd and did a little bit more privileged config, 95% of the rest of the stuff DevStack does as root, could still be kept unprivileged. I am also aware that current OpenStack code may not be amenable to installation in locations outside of / by default, but the code is all there to be modified to cope with arbitrary install locations if desired/required.
My other wishlist item for DevStack would be for it to print output that is meaningful to the end user running it. Simply printing a verbose list of every single shell command executed is one of the most unkind things you can do to a user. I’d like to see
# ./devstack.sh
* Cloning GIT repositories
- 1/10 nova
- 2/10 cinder
- 3/10 quantum
* Installing RPMs using YUM
- 1/30 python-pip
- 2/30 libvirt
- 3/30 libvirt-client
* Installing Python packages to $DEST/vroot/lib/python-2.7/site-packages using PIP
- 1/24 WebOb
- 2/24 ampqlib
- 3/24 boto
* Creating database schemas
- 1/10 nova
- 2/10 cinder
- 3/10 quantum
By all means still save the full list of every shell command and their output to a ‘devstack.log’ file for troubleshooting when things go wrong.
When first getting involved in the OpenStack project as a developer, most people will probably recommend use of DevStack. When I first started hacking, I skipped this because it wasn’t reliable on Fedora at that time, but these days it works just fine and there are even basic instructions for DevStack on Fedora. Last week I decided to finally give DevStack a go, since my hand-crafted dev environment was getting kind of nasty. The front page on the DevStack website says it is only supported on Fedora 16, but don’t let that put you off; aside from one bug which does not appear distro specific, it all seemed to work correctly. What follows is an overview of what I did / learnt
Setting up the virtual machine
I don’t like really like letting scripts like DevStack mess around with my primary development environment, particularly when there is little-to-no-documentation about what changes they will be making and they ask for unrestricted sudo (sigh) privileges ! Thus running DevStack inside a virtual machine was the obvious way to go. Yes, this means actual VMs run by Nova will be forced to use plain QEMU emulation (or nested KVM if you are brave), but for dev purposes this is fine, since the VMs don’t need todo anything except boot. My host is Fedora 17, and for simplicity I decided that my guest dev environment will also be Fedora 17. With that decided installing the guest was a simple matter of running virt-install on the host as root
# virt-install --name f17x86_64 --ram 2000 --file /var/lib/libvirt/images/f17x86_64.img --file-size 20 --accelerate --location http://mirror2.hs-esslingen.de/fedora/linux//releases/17/Fedora/x86_64/os/ --os-variant fedora17
I picked the defaults for all installer options, except for reducing the swap file size down to a more sensible 500 MB (rather than the 4 G it suggested). NB if copying this, you probably want to change the URL used to point to your own best mirror location.
Once installation completed, run through the firstboot wizard, creating yourself an unprivileged user account, then login as root. First add the user to the wheel group, to enable it to run sudo commands:
# gpasswd -a YOURUSERNAME wheel
The last step before getting onto DevStack is to install GIT
# yum -y install git
Setting up DevStack
The recommended way to use DevStack, is to simply check it out of GIT and run the latest code available. I like to keep all my source code checkouts in one place, so I’m using $HOME/src/openstack for this project
$ mkdir -p $HOME/src/openstack
$ cd $HOME/src/openstack
$ git clone git://github.com/openstack-dev/devstack.git
Arguably you can now just kick off the stack.sh script at this point, but there are some modifications that are a good idea to do. This involves creating a “localrc” file in the top level directory of the DevStack checkout
$ cd devstack
$ cat > localrc <<EOF
# Stop DevStack polluting /opt/stack
DESTDIR=$HOME/src/openstack
# Switch to use QPid instead of RabbitMQ
disable_service rabbit
enable_service qpid
# Replace with your primary interface name
HOST_IP_IFACE=eth0
PUBLIC_INTERFACE=eth0
VLAN_INTERFACE=eth0
FLAT_INTERFACE=eth0
# Replace with whatever password you wish to use
MYSQL_PASSWORD=badpassword
SERVICE_TOKEN=badpassword
SERVICE_PASSWORD=badpassword
ADMIN_PASSWORD=badpassword
# Pre-populate glance with a minimal image and a Fedora 17 image
IMAGE_URLS="http://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz,http://berrange.fedorapeople.org/images/2012-11-15/f17-x86_64-openstack-sda.qcow2"
EOF
With the localrc created, now just kick off the stack.sh script
$ ./stack.sh
At time of writing there is a bug in DevStack which will cause it to fail to complete correctly – it is checking for existence paths before it has created them. Fortunately, just running it for a second time is a simple workaround
$ ./unstack.sh
$ ./stack.sh
From a completely fresh Fedora 17 desktop install, stack.sh will take a while to complete, as it installs a large number of pre-requisite RPMs and downloads the appliance images. Once it has finished it should tell you what URL the Horizon web interface is running on. Point your browser to it and login as “admin” with the password provided in your localrc file earlier.
Because we told DevStack to use $HOME/src/openstack as the base directory, a small permissions tweak is needed to allow QEMU to access disk images that will be created during testing.
$ chmod o+rx $HOME
Note, that SELinux can be left ENFORCING, as it will just “do the right thing” with the VM disk image labelling.
UPDATE: if you want to use the Horizon web interface, then you do in fact need to set SELinux to permissive mode, since Apache won’t be allowed to access your GIT checkout where the Horizon files live.
$ sudo su -
# setenforce 0
# vi /etc/sysconfig/selinux
...change to permissive...
UPDATE:If you want to use Horizon, you must also manually install Node.js from a 3rd party repositoryh, because it is not yet included in Fedora package repositories:
# yum localinstall --nogpgcheck http://nodejs.tchol.org/repocfg/fedora/nodejs-stable-release.noarch.rpm
# yum -y install nodejs nodejs-compat-symlinks
# systemctl restart httpd.service
Testing DevStack
Before going any further, it is a good idea to make sure that things are operating somewhat normally. DevStack has created an file containing the environment variables required to communicate with OpenStack, so load that first
$ . openrc
Now check what images are available in glance. If you used the IMAGE_URLS example above, glance will have been pre-populated
$ glance image-list
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| 32b06aae-2dc7-40e9-b42b-551f08e0b3f9 | cirros-0.3.0-x86_64-uec-kernel | aki | aki | 4731440 | active |
| 61942b99-f31c-4155-bd6c-d51971d141d3 | f17-x86_64-openstack-sda | qcow2 | bare | 251985920 | active |
| 9fea8b4c-164b-4f54-8e74-b53966e858a6 | cirros-0.3.0-x86_64-uec-ramdisk | ari | ari | 2254249 | active |
| ec3e9b72-0970-44f2-b442-58d0042448f7 | cirros-0.3.0-x86_64-uec | ami | ami | 25165824 | active |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
Incidentally the glance sort ordering is less than helpful here – it appears to be sorting based on the UUID strings rather than the image names :-(
Before booting a instance, Nova likes to be given an SSH public key, which it will inject into the guest filesystem to allow admin login
$ nova keypair-add --pub-key $HOME/.ssh/id_rsa.pub mykey
Finally an image can be booted
$ nova boot --key-name mykey --image f17-x86_64-openstack-sda --flavor m1.tiny f17demo1
+------------------------+--------------------------------------+
| Property | Value |
+------------------------+--------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | NsddfbJtR6yy |
| config_drive | |
| created | 2012-11-19T15:00:51Z |
| flavor | m1.tiny |
| hostId | |
| id | 6ee509f9-b612-492b-b55b-a36146e6833e |
| image | f17-x86_64-openstack-sda |
| key_name | mykey |
| metadata | {} |
| name | f17demo1 |
| progress | 0 |
| security_groups | [{u'name': u'default'}] |
| status | BUILD |
| tenant_id | dd3d27564c6043ef87a31404aeb01ac5 |
| updated | 2012-11-19T15:00:55Z |
| user_id | 72ae640f50434d07abe7bb6a8e3aba4e |
+------------------------+--------------------------------------+
Since we’re running QEMU inside a KVM guest, booting the image will take a little while – several minutes or more. Just keep running the ‘nova list’ command to keep an eye on it, until it shows up as ACTIVE
$ nova list
+--------------------------------------+----------+--------+------------------+
| ID | Name | Status | Networks |
+--------------------------------------+----------+--------+------------------+
| 6ee509f9-b612-492b-b55b-a36146e6833e | f17demo1 | ACTIVE | private=10.0.0.2 |
+--------------------------------------+----------+--------+------------------+
Just to prove that it really is working, login to the instance with SSH
$ ssh ec2-user@10.0.0.2
The authenticity of host '10.0.0.2 (10.0.0.2)' can't be established.
RSA key fingerprint is 9a:73:e5:1a:39:e2:f7:a5:10:a7:dd:bc:db:6e:87:f5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.0.2' (RSA) to the list of known hosts.
[ec2-user@f17demo1 ~]$ sudo su -
[root@f17demo1 ~]#
Working with DevStack
The DevStack setup runs all the python services under a screen session. To stop/start individual services, attach to the screen session with the ‘rejoin-stack.sh’ script. Each service is running under a separate screen “window”. Switch to the window containing the service to be restarted, and just Ctrl-C it and then use bash history to run the same command again.
$ ./rejoin-stack.sh
Sometimes the entire process set will need to be restarted. In this case, just kill the screen session entirely, which causes all the OpenStack services to go away. Then the same ‘rejoin-stack.sh’ script can be used to start them all again.
One annoyance is that unless you have the screen session open, the debug messages from Nova don’t appear to end up anywhere useful. I’ve taken to editing the file “stack-screen” to make each service log to a local file in its checkout. eg I changed
stuff "cd /home/berrange/src/openstack/nova && sg libvirtd /home/berrange/src/openstack/nova/bin/nova-compute"
to
stuff "cd /home/berrange/src/openstack/nova && sg libvirtd /home/berrange/src/openstack/nova/bin/nova-compute 2>&1 | tee nova-compute.log"
I use the wonderful Digikam application for managing my photos on Fedora 16. I don’t mind that I’m running a KDE based application under GNOME shell, since it themes itself to match & its featureset easily beats any other open source desktop photo management application. Normally the UI theme when running Digikam under GNOME looks like this:
And then a strange thing happened last week. When I launched Digikam on my laptop the UI style suddenly changed to this:
Notice in particular the different tree view expander icons and the different scrollbars. More interestingly though, the overall UI felt more responsive when interacting with Digikam. Next time I launched Digikam, it was back to the “normal” GNOME compatible theme. Wierd. And now just last night the same behaviour occurred on my other laptop – Digikam launched in a different theme, but upon restart, went back to the original theme. WTF ?
I tried playing with the ‘Themes’ menu in Digikam itself and all I can change is the colour scheme, not the widget styling. Trying to change the KDE application theme in KDE Control Center had precisely zero effect on the Digikam theme.
Can anyone explain this behaviour ? Is there some trick to controlling KDE application themes when running under GNOME on Fedora 16 ? Most importantly how I can get back this alternative Digikam theme? I really liked how responsive it felt to interact with, compared to the standard GNOME-like theme.
With Fedora 16 out, I have to perform upgrades on my Fedora machines, two laptops, one mac mini and five servers. Officially the only supported way to upgrade Fedora is to use the regular Anaconda installer, or use pre-upgrade. In the years I’ve been using Fedora (since Fedora Core 5), I have never used Anaconda for upgrades because I rarely have direct access to the machine to boot CDs/DVDs, and whenever I have tried pre-upgrade it always fails. This time it failed on one server because /boot was on software-RAID, and failed on the other two servers because /boot was too small. So instead I’ve always ended up doing live upgrades using yum directly. The first few times I did this, there were some surprises, but in the end I’ve settled into a recipe that has a high success rate.
I’m NOT encouraging people follow my approach, but if you have ended up in a situation where a live yum upgrade is your only option, these notes might help you avoid some pitfalls.
- Tip 1: Only attempt to upgrade 1 release at a time, don’t try to skip over releases. This reduces the chances of problems with yum calculating a suitable upgrade path
- Tip 2: Avoid having any 3rd party repositories configured with yum, unless you know they already support both the current and target distro release. In practice this means you want to avoid anything except official Fedora, RPM Fusion and Livna repositories.
The actual upgrade process I follow is this:
- Step 1: Ensure the current install is fully updated.
# yum -y update
- Step 2: Remove any orphaned packages, ie locally installed packages which are not present in any of your active YUM repositories. Orphaned packages are the most common cause for unresolvable RPM dependencies during upgrade. If you choose to skip this step be aware that you will almost certainly need to revisit it, if yum fails to resolve an upgrade path.
# package-cleanup --orphans
# rpm -e ...for each orphan you want to remove...
- Step 3: Purge all YUM cached data about current repos. This just frees up space by removing cached data files that won’t be needed anymore
# yum clean all
- Step 4: Install the fedora-release, fedora-release-rawhide and fedora-release-notes RPMs for the new release.
# wget http://mirror.bytemark.co.uk/fedora/linux/releases/16/Everything/x86_64/os/Packages/fedora-release-16-1.noarch.rpm
# wget http://mirror.bytemark.co.uk/fedora/linux/releases/16/Everything/x86_64/os/Packages/fedora-release-notes-16.1.0-1.fc16.noarch.rpm
# wget http://mirror.bytemark.co.uk/fedora/linux/releases/16/Everything/x86_64/os/Packages/fedora-release-rawhide-16-1.noarch.rpm
# rpm -Uvh fedora-release*16*.rpm
- Step 5: Perform the actual upgrade.
# yum update
Sometimes even after removing all orphan packages and 3rd party packages, the last step will still fail with unresolvable dependencies. This might be the case if an RPM was purged from latest Fedora and thus not having an upgrade path. When this happens, it is usually sufficient to just remove the obsolete package and retry the upgrade.
One significant change between Fedora 15 and 16 that is important for the upgrade process, is the switch from Grub1 to Grub2. While the live YUM upgrade will result in grub2 getting installed, it does not update your bootsector. So there are two post-upgrade tasks that must be done before reboot.
- Generate the master grub2 config file
# grub2-mkconfig > /etc/grub2.cfg
- Install grub2 into the boot sector
# grub2-install /dev/sda
As mentioned earlier, on one of the machines I had software RAID configured, so I wanted grub2 installed on both disks
# grub2-install /dev/sda
# grub2-install /dev/sdb
After rebooting into the new kernel, I once again run ‘package-cleanup –orphans’ to ensure find any obsolete packages which can be killed off.
That’s all there was to it. Where I have followed these instructions I have a 100% success rate in live yum upgrades. The only upgrade which failed was the one where I forgot to generate the grub2 config file before rebooting :-)