Do you feel brave enough to try OpenStack? OSAD!


Updated: November 14, 2015

This is a very good question. For many reasons. One, OpenStack is a sort of a big, fancy buzzword nowadays, and anyone who thinks they are someone will tell you that you should be using this thing. Like the word cloud, the resistance is futile. Two, it's got something to do with virtualization after all, so maybe you should learn more about it.

Three and four and so on, the reasons shall become apparent later on this article. But the basic premise is, can you - and should you - dabble in OpenStack? Now, I am asking this question from the perspective of a person with natural technical intuition and significant experience dabbling in the likes of OpenStack. Today's article slash guide will show you how to get underway with this technology.

Teaser

Note: Image taken from Wikipedia, licensed under CC BY-SA 3.0.

Expectations versus reality

All right. So OpenStack it is. But then, do you really know anyone, apart from three or four big hosting providers, who might actually be using this? Everyone is talking about it, but you won't find too many real-life practical examples. This really got me curious and wondering. Why not?

I was naturally intrigued, so I decided to embark on my OpenStack journey, to see whether I could quickly and easily master the technology, and then offer you a tutorial or three that would help you along the way. Very soon, it became apparent that this software is not to be trifled with, and that it has immense power to humble even the most intuitive of geeks out there. To wit, I changed my tune, my strategy, and went from trying to showcase a step-by-step guide to an existential question around the relevance of OpenStack, as well as a genuine attempt to bridge the gap between your (and my) expectations versus the harshness of reality.

OpenStack is hard

Indeed, several long painful hours of command line wizardry into my adventure, I realized that OpenStack could not be just tamed the way I did with KVM or Docker. Nope. A few quick examples, some man page perusing and trial and error in a shell were far from being sufficient to even remotely begin using the software. OpenStack looked like a snow-capped mountain, and I only had flip flops for the climb.

The more I was playing with it, the more I realized it's a woefully complex and over-architected solution. The more i was tinkering and trying to get things done, the more it was eluding me, slipping away, becoming less and less interesting and appealing, and making the existential question all the more philosophical.

What not to do

So before I do show you how to get underway with OpenStack, there are several critical bits and pieces we need to clarify. First, architecture. Highly complex, highly modular, with each little component acting as a separate service, both in name and functionality, as having lots of components is considered a good thing today.

Architecture

Image taken from the official openstack.org documentation; click to enlarge.

The second piece is, business relevance. Why do you want to use OpenStack? It is a first and foremost an infrastructure service. If you need to run your business, then you do not need to know OpenStack, merely use it. Then, do you have sufficient time, money, skill, and manpower to invest in learning this technology? How quickly do you need it, and how deeply do you need to know it? What are you trying to achieve? Are you going to be a provider or a consumer of OpenStack? If you think of using this for your business, how large is it? Tens of virtual machines? Hundreds? Thousands? Tens of thousands?

This critical piece is often overlooked by clueless managers too eager to jump on the buzzword wagon, and they confuse themselves and their employees by blurring the difference between provisioning and utilization of services, by making infrastructure act as a business when it's not. And then OpenStack starts making even less sense, because people are busy trying to get it running, so they can run something else on top of it, end up wasting several months in futile effort and giving up. Which is why OpenStack only exists at the edges of the usage spectrum - either massive deployments or none at all.

What to do

Now, I did mention reasons, and I highlighted a couple. Here are some more. OpenStack makes sense if you want to be at the forefront of technology. It makes sense if you are trying to build private or public clouds, if you intend to be a provider of service, and also if you seek to establish mastery and dominance in the cloud field, which may be a legitimate if secondary objective of your business.

OpenStack can also be useful if just want to test software and run your whatnot on top of it. You can approach it as a blackbox solution, or you could develop a rudimentary understanding, so if something goes wrong, you can troubleshoot.

Now, the big question is, how do you go about it. As I've highlighted earlier, the manual process of getting things going is dreadful. You need a couple of books, several weeks of time, and lots and lots of command line typing, creating users, groups, roles, databases, network bridges, containers, and many other cruel little bits and pieces.

We want to skip all of that. Normally, I would suggest a linear learning process, but OpenStack is too complex for that. We will begin with an automated mechanism to get everything running, blindly believing the automated setup. Once that has been fully and successfully configured, we will slowly retrace our steps.

OpenStack Ansible Deployement (OSAD)

If you search around the Web, you will see there are dozens of solutions all claiming to be the holy grail of management and deployment for OpenStack. Yes, they exist because OpenStack is too difficult to use, and rather than fixing the core problem, the Internet is brimming with ad-hoc overlay solutions designed to make it easier to use, monitor and such. Much like containers really. OSAD is one such community project, although there are many, many others, like Fuel, Inception, TripleO, Compass, PackStack, and still more. In this tutorial, we will focus on OSAD and follow up with other similar tools. P.S. I am affiliated with this project, so if you smell bias, feel free to disregard this article.

Basic setup

Technically, you only need to run two commands. One, grab the master script from the GitHub repo, and two, run the Ansible setup. For those wondering what this strange little thing is, well, Ansible is a configuration management tool. Similar to Puppet, Chef, CFEngine, and friends, it is designed to automate deployment of software policies and configurations on target machines. The deployment logic is encoded inside files called Playbooks using the YAML language syntax. The software parses the Playbooks and then dispatches the necessary configurations to target hosts using SSH. That's the basic magic of Ansible.

Warning!

Now, I will not go into too many details how Ansible works. I will just briefly highlight what it does and how it behaves. The syntax is fairly intuitive, and once you understand how it works, you should be able to follow up the logic of OSAD yml files. Let's take a look at a sample Playbook from the online documentation:

---
- hosts: webservers
  vars:
    http_port: 80
    max_clients: 200
  remote_user: root
  tasks:
  - name: ensure apache is at the latest version
    yum: pkg=httpd state=latest
  - name: write the apache config file
    template: src=/srv/httpd.j2 dest=/etc/httpd.conf
    notify:
    - restart apache
  - name: ensure apache is running (and enable it at boot)
    service: name=httpd state=started enabled=yes
  handlers:
    - name: restart apache
      service: name=httpd state=restarted

What do we have here? Let's walk through the Playbook:

This briefly explains how Ansible playbooks work. Now, once we deploy OSAD, you can then go through the available yml files and explore the setup. You will see a lot of complicated stuff, including containers, bridges and whatnot, but that's the whole idea. You avoid the mess, and let someone else sort it out for you. At least, initially.

Getting underway

Anyhow, our two lovely commands:

bash <(curl http://git.openstack.org/cgit/stackforge/os-ansible-deployment/
plain/scripts/run-aio-build.sh)

/opt/stackforge/os-ansible-deployment/openstack-ansible setup-everything.yml

This will begin the setup, and depending on your hardware, it can take about 40-50 minutes. You will definitely need ample CPU, memory and disk space to get this running well, then again, less than you would expect. Four cores, 8 GB RAM and 40 GB disk should suffice, so even a decent laptop can do this fairly well. Don't expect any stellar performance.

Ansible deploying

Ansible deploying, more

...
+ unset -f deactivate
+ exit 0

/opt/stackforge/os-ansible-deployment
----------------------------------------------------------------

- [ Run Time = 2422 seconds || 40 minutes ] 
----------------------------------------------------------------

- [ Status: Success ] 
----------------------------------------------------------------
+ cat
++ ip -o -4 addr show dev eth0
++ awk -F '[ /]+' '/global/ {print $4}'
+ chmod +x /etc/update-motd.d/20-openstack
+ cat
++ pwd
+ chmod +x /etc/update-motd.d/21-galera

If you see the output as above, then your initial setup has succeeded well.

Playing with Nova

The next logical step is to make sure you can actually run one or two commands. Now, inspecting the whole stack of what you have on your system might not really be possible. But you should see a whole bunch of network interfaces, including bridges and virtual devices created when spawning containers. Somewhat like what we saw with Docker.

Nova commands are ugly and long, and you are better off setting up several environment variables than having to input all of them every single time you run a command:

nova --os-user-name admin --os-password secrete --os-tenant-name admin --os-auth-url http://172.29.236.100:5000/v2.0 image-list
+------------------------------------+--------+--------+--------+
| ID                                 | Name   | Status | Server |
+------------------------------------+--------+--------+--------+
| 896fa8ca-682b-4f4e-859e-9f08b...   | cirros | ACTIVE |        |
+------------------------------------+--------+--------+--------+

nova --os-user-name admin --os-password secrete --os-tenant-name admin --os-auth-url http://172.29.236.100:5000/v2.0 hypervisor-list
+----+---------------------+-------+---------+
| ID | Hypervisor hostname | State | Status  |
+----+---------------------+-------+---------+
| 1  | server-001          | up    | enabled |
+----+---------------------+-------+---------+

Indeed, setup the environment variables:

export OS_USERNAME=admin
export OS_PASSWORD=secrete
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://172.29.236.100:5000/v2.0

Then, you can use these shorter nova commands to boot your hosts. Basically, you need to specify the image, flavor and name. Plus of course all the identification details, but we sorted those only moments ago. Now, just as importantly, we will need SSH keys and some firewall rules so we can connect to our virtual machines. Otherwise, it won't work.

Security rules

nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0

nova keypair-add Dedoimedo > dedoimedo.pem
chmod 0600 dedoimedo.pem

nova boot First --image 896fa8ca-682b-4f4e-859e-9f08b82b47fb --flavor 3 -key-name Dedoimedo
+--------------------------------+------------------------------+
| Property                       | Value                        |
+--------------------------------+------------------------------+
| OS-DCF:diskConfig              | MANUAL                       |
| OS-EXT-AZ:availability_zo      | nova                         |
| OS-EXT-SRV-ATTR:host           | -                            |
| OS-EXT-SRV-ATTR:hyperviso      | -                            |
| OS-EXT-SRV-ATTR:instance_      | instance-00000007            |
| OS-EXT-STS:power_state         | 0                            |
...
| created                        | 2015-06-04T09:48:59Z         |
...
| flavor                         | m1.medium (3)                |
...
+--------------------------------+------------------------------+

First boot

The output information will include the default auto-generated password for the host, and you will be able to get its IP address from the list of running hosts. Then, it's the simple SSH thingie, and if you succeed, then you have just miraculously managed to get OpenStack running, and the first obstacle has been removed.

ping -c 1 172.29.248.13
PING 172.29.248.13 (172.29.248.13) 56(84) bytes of data.
64 bytes from 172.29.248.13: icmp_seq=1 ttl=64 time=0.634 ms

--- 172.29.248.13 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.634/0.634/0.634/0.000 ms

ssh -i dedoimedo.pem cirros@172.29.248.14
The authenticity of host '172.29.248.14 (172.29.248.14)'
can't be established. RSA key fingerprint is 5b:cf:ed:99:b8:a9:2a:2f:ce:58:f0:3c:eb:db:7c:c1.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.29.248.14' (RSA) to the list of known hosts.

SSH success

Problems you may encounter

Whether you go for the manual path or an automated setup, you may encounter some problems here and there. This section details some of the common issues you could face when playing with OpenStack, Ansible, or both.

openstack-ansible setup-everything.yml
Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e @/etc/openstack_deploy/user_variables.yml "
ERROR: Inventory script (inventory/dynamic_inventory.py) had an execution error: Traceback (most recent call last):
File "/openstack/os-ansible-deployment/playbooks/inventory/
dynamic_inventory.py", line 21, in <module>
import netaddr
ImportError: No module named netaddr

You are missing critical Python modules, which you need to install:

pip install netaddr
Collecting netaddr
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/
packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/
security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading netaddr-0.7.14-py2.py3-none-any.whl (1.5MB)
100% |                               | 1.5MB 312kB/s
Installing collected packages: netaddr
Successfully installed netaddr-0.7.14

While playing with OpenStack Ansible Deployment framework, you may see errors pertaining to All in One (AIO) build, which could indicate SSH problems, even though you may believe everything is peachy and should work:

GATHERING FACTS ***************************************************************
fatal: [aio1] => SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh

TASK: [apt_package_pinning | Add apt pin preferences] ***************************************************************
FATAL: no hosts matched or all hosts have already failed -- aborting

These errors could stem from the following two bugs, so please take a look yonder. Then, a third typical error that could come up is if you try to clone the repo into a non-empty directory:

+ git clone -b master https://github.com/stackforge/os-ansible-deployment.git /opt/stackforge/os-ansible-deployment
fatal: destination path '/opt/stackforge/os-ansible-deployment' already exists and is not an empty directory.

More reading

If you felt this tutorial was way too easy for your exquisite taste:

Some slides that could be interest.

Red Hat Distribution of OpenStack (RDO)

Mirantis OpenStack Deployment and Management - Fuel

Conclusion

As you can see, there are way too many incarnations of OpenStack. In a way, it replicates the complex, thriving and confusing Linux landscape into its own four dimensions, and we will probably have to endure a long decade of more buzzwords and name dropping until some kind of standard is finally reached, and equilibrium in the community achieved. OSAD is okay, however it is based on Ubuntu 14.04 + LXC, and you may actually want something like Red Hat or CentOS for your production. Makes sense, which means you should test other setups and tools, too. But that's something you can try after you've had your baptism of fun with OpenStack.

All that said, if you don't feel like waiting a few years before the techno-storm passes, you can start playing with OpenStack, regardless of your objective, be it business or backbone. Whatever you do, the first steps will be overwhelming, so you should definitely grab yourself a helper framework and then trace back to core technology.

OSAD is a decent example, if far from being easy and perfect. But it is light years ahead of what OpenStack offers on its own, and it's actually usable, as it does bring back some of the expected technical intuition to the table. Hopefully, this tutorial has helped you make some progress. It's not easy, it's messy, and I don't like it too much. Still, it's probably the best you can do with the tools at hand. Overall, it comes down to: try not to overthink it, avoid manual setups, go for Ansible, fix a few first-use errors, and then get the hang of Nova and friends. After a while, revisit OpenStack and slowly tweak until you get the monster tamed. That would be all for this guide.

P.S. I have officially presented this topic in a much more fun and entertaining way during this year's CloudOpen conference in Dublin, Ireland. More details at the behest and click of your mouse device. Go.

Cheers.

RSS Feed icon

del.icio.us del.icio.us stumbleupon stumble digg digg reddit reddit slashdot slashdot



Advertise!

Would you like to advertise your product/site on Dedoimedo?

Read more

Donate to Dedoimedo!

Do you want to
help me take early retirement? How about donating
some dinero to
Dedoimedo?

Read more

Donate