First of all, you need to know that by default only the « admin » tenant can manage flavors because of default policy in Nova :

"compute_extension:flavormanage": "rule:admin_api"

If you want to let the possibility to all tenants to create flavors, you can delete the rule to have :

"compute_extension:flavormanage": ""

Now we are going to create a flavor :

nova flavor-create flavor-name flavor-ID RAM-in-MB root-disk-in-GB VCPUs-number 

--ephemeral ephemeral-disk-in-GB --swap swap-in-MB --is-public False

Example :

nova flavor-create enocloud-xxl 50 32 200 8 --is-public False

The next step is to associate the flavor to the tenant :

nova flavor-access-add <flavor-id> <tenant-id>

Example :

nova flavor-access-add 50 4f1b0b9ce3354a439db8ef10cf456d6f

Hope that helps !

For that we have created a project called swiftsync hosted in github: https://github.com/enovance/swiftsync

The swiftsync project provides two binaries called :

• swsync (The synchronizer)
• swfiller (A swift filler)

swfiller is a tool designed to ease you the filling process of a swift cluster. This tool can be useful for testing the swsync capabilities on a test platform before using it on a production platform.

The second tool called swsync is the synchronization tool. It will migrate all swift content from an origin server to a destination server using only the API of the both proxy servers. Content that will be synchronized is the following :

• account and account metadata
• container and container metadata
• object and object metadata

swsync will take care of data (accounts metadata, containers/objects, containers/objects metadata) that are already stored on destination cluster in order to speed up and optimize data copying. Indeed only data that is more recent on origin will be synchronized. The first run of swsync will migrate all data from origin to destination but next runs will only migrate modified data. The process is as follow :

• will synchronize account metadata if they has changed on origin
• will delete container on destination if no longer exists on origin
• will create container on destination if not exists
• will synchronize destination container metadata if not same as origin container.
• will remove container object if no longer exists in origin container
• will synchronize object and metadata object if the last-modified header is the latest on the origin.

swsync has been designed to be run and run again and not ensuring that the first pass goes well, if for example there is a network failure swsync will just skip it and hope to do it on the next run. So the tool can for instance be launched by a cron job to perform a diff synchronization each night.

swsync will need to use a user that own the ResellerAdmin role. This role will let the user perform all kind of API operations on all swift account so swsync will be able to explore all origin and destination accounts to evaluate which data it need to synchronize.

Both tools come with unit and functional tests. Unit tests are managed by tox as all openstack projects.

Important thing to mention is that the synchronization tool is currently a work in progress and has not been really tested on a cluster that own a huge amount of data.

Have a look to the project README file for further informations about the project.

This is something I have been asked and I was at first under impression this was only available in v3, digging a bit more into the code there is actually a way to do that in v2 when you are using PKI tokens. Since I could not find much documentation is here is some steps describing how to do it.

First let’s generate a new PKI token, you can do it like this:

$ curl -s -d '{"auth": {"tenantName": "tenant", "passwordCredentials": 

{"username": "user", "password": "password"}}}'

 -H 'Content-type: application/json' http://localhost:5000/v2.0/tokens

or get my script here :

http://p.chmouel.com/ks

and use it like that :

eval $(bash ks -s localhost tenant:user password)

It will give you a variable $TOKEN and a variable $STORAGE_URL that you can use further down.

Now let’s try to use it with our swift :

$ curl -i -H "X-Auth-Token: $TOKEN" ${STORAGE_URL}
HTTP/1.1 204 No Content
Content-Length: 0
Accept-Ranges: bytes
X-Timestamp: 1366666887.01151
X-Account-Bytes-Used: 0
X-Account-Container-Count: 0
Content-Type: text/html; charset=UTF-8
X-Account-Object-Count: 0
X-Trans-Id: tx5b50dc6d01d04923a40a1486c13dd94d
Date: Mon, 22 Apr 2013 22:01:00 GMT

all good here,

So now go inside your keystone.conf and get your admin/service token or use that friendly copy and paste command line :

$ ADMIN_TOKEN=$(sed -n '/^admin_token/ { s/.*=[ ]*//;p }' /etc/keystone/keystone.conf)

and use it to DELETE the token we do that request directly to our keystone which is localhost here point it wherever you want:

$ curl -X DELETE -i -H "X-Auth-Token: $ADMIN_TOKEN" http://localhost:5000/v2.0/tokens/$TOKEN
HTTP/1.1 204 No Content
Vary: X-Auth-Token
Content-Length: 0
Date: Mon, 22 Apr 2013 22:01:08 GMT

We can still use it because the token is still in the cache. By default tokens are cached in memcache as good as 5 minutes but the
revocation list is fetched every seconds or so.

$ curl -i -H "X-Auth-Token: $TOKEN" ${STORAGE_URL}
204 No Content
Content-Length: 0
Accept-Ranges: bytes
X-Timestamp: 1366666887.01151
X-Account-Bytes-Used: 0
X-Account-Container-Count: 0
Content-Type: text/html; charset=UTF-8
X-Account-Object-Count: 0
X-Trans-Id: tx9018045ce1324203a91e882ec6d27ac3
Date: Mon, 22 Apr 2013 22:01:12 GMT

but after a bit (like over a minute or so) we are getting a proper denied:

$ curl -i -H "X-Auth-Token: $TOKEN" ${STORAGE_URL}
HTTP/1.1 401 Unauthorized
Content-Length: 131
Content-Type: text/html; charset=UTF-8
X-Trans-Id: tx9133daf949204f0facf45152a43836bb
Date: Mon, 22 Apr 2013 22:27:23 GMT

>h1<Unauthorized<

This server could not verify that you are authorized to access the document you requested.

and from log messages:

proxy-server Token 49d94a8ca068013b6efe79e3463627c8 is marked as having been revoked
proxy-server Token validation failure.#012Traceback (most recent call last):#012 

File "/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 689, in _validate_user_token#012   

verified = self.verify_signed_token(user_token)#012  File "/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 1045, 

in verify_signed_token#012    raise InvalidUserToken('Token has been revoked')#012InvalidUserToken: Token has been revoked

[..]
proxy-server Marking token MIIGogYJK...... as unauthorized in memcache

bingo the token has been now revoked properly.

What’s devops?

Funnily enough, there doesn’t seem to be a consensus around the definition of a devops. This seems to be intentional since the community carefully refuses to venture into giving a specific definition.

However here are some values that seem shared by all the participants today:

• A very strong emphasis on regular and thorough communication between different actors of the team.
• Sharing the responsibilities between operations and development. If an incident happens, we’re all concerned.
• Focus on multidisciplinary teams.
• Deploying code in production should be automated
• If you don’t have a CI, you’re doing it wrong.

The conference

The conference attracted people working mainly in the web field, but also banks, financial institutions, software publishers, webhosts, etc. The seven talks given during the 2 days consisted mainly of participants sharing their experiences in teams where flexibility, agility and the broader « devops » mentality was applied. We got speakers talking about applying devops to customer support, governmental work, web dev shops, etc.

Another interesting aspect of the conference were the open Spaces, a time where participants would decide what they wanted to talk about. Topics ranged from the highly technical (discussing specific tools like Vagrant or Puppet) to highly organizational (« How to apply devops when working remotely? », « How to apply devops and Kanban? »). Small groups formed very easily, discussion came naturally and it was a generally highly entertaining experience.

We were here

We were there to hold a stand to push the info that we’re hiring (did I mention we’re hiring?). We ended up meeting very interesting people dealing with problems similar to ours. A young and energetic community of IT workers is forming globally, based around the desire to provide excellence, flexibility, reactivity and communication to our customers. eNovance is proud to be part of it.

Also, did I mention we’re hiring?

Heat session : Updates in Havana

• parallel resource creation
• further improve OpenStack Networking support 
• Rolling updates 
• Support for new/extended template language 
• Add autoscaling API actions 
• Move to Ceilometer for metrics/monitoring/alarms 
• More UpdateStack improvements 
• Further improved security (trusts/in-in-instance credentials) using the new features in Keystone
• Native resource types. Goal: allow to use when no AWS interface is not present/activated
• Stack suspend resume 
• Configurable Load Balancer (use LBaaS) previously used Ha-proxy directly

OpenStack Networking session : Updates in Havana

Services :

• Firewall
• Load balancing: integrate more plugins 
• VPN: implementation API 
• Improved IPv6 support 
• Improved bare metal support: SRIOV, tripleo 
• Updated Client Library

Community initiatives :

• Database profiling
• Improving testing
• Exploring nova-net migration paths

By the way, nova-network will be supported for a couple of releases. The goal is to have Quantum as the default networking in Openstack in the future release while keeping nova-networking around.There is no plan for dynamic routing protocols.

Nova session : Updates in Havana

Big themes :

• Live upgrades
• Security 
• Scale, performance 
• Reliability

Internal object model :

• no leaks of db models
• objects wirh built-in serialization 
• decouple from db schema

And also : Graceful service shutdown, RPC version control, better state handling, Nova v3 API

Nova extension framework :

• code cleanup
• entrypoint 
• shared framework

Scheduling :

• allow scheduling nased on more data such as utilization
• whole host reservation 
• cross projet scheduling 
• group scheduling

Cells :

• address some feature gaps
• hopefully more people start using it

Major cleanup around migration/live-migrate/resize/evacuate.

Horizon session : Havana updates

API :

• multiple version support
• extension detection

Keystone :

• V3 API
• Domains 
• Groups 
• Roles 
• Policy management

Networking :

• Security group
• Admin IP management 
• Quotas 
• Even cooler topology visualization/interface

Heat :

• Stack management
• Visualizing stacks

Ceilometer :

• « Top consumers »
• Visualizations: will use d3.js for visualization. 
• Data everywhere: data accoring to context

Nova :

• Per-project flavors
• Instance action history 
• Zones 
• Better hypervisor information

Other items :

• Asynchronous communication… RPC bus integration; async communication backend; socket.io
• Remixing the Openstack dashboard 
• Mobile

Glance session : Updates in Havana

• Better direct end-user use
• Don’t hide behind Openstack compute 
• Improving image transfer performance 
• Supporting image conversion 
• Feature parity with EC2 CopyImage 
• Supporting rolling database migrations

Vision :

• Deprecation Path for Registry
• ACLs for Image Proterties 
• Image Quotas 
• Parity with EC2 CopyImage 
• Asynchronous image workers to support: 
• Conversion 
• Validation (?)
• Directly exposing image underlying locations 
• Better network performance

Keystone session : Updates in Havana

• External authentication
• OAuth 1.0a 
• x509 
• Client support 
• Middleware: auth_token 
• Command line: openstackclient 
• Web UI: Horizon 
• Event notifications 
• Availability zones and region management

V3: tenant renamed project :

• Key management (incubated)
• LDAP integration 
• Centralized quotas 
• Secure endpoint-endpoint communication 
• Fine-grained access control

Olso session (OpenStack common) : Updates in Havana

• oslo.messaging
• oslo.log 
• oslo.rootwrap 
• oslo.i18n 
• packaging and build tools
• DB 
• Service infrastructure 
• Scheduler 
• WSGI

Cinder session : updates in Havana

• expand fibre channel support
• ACL’s (shared volumes across tenants) 
• Volume migration 
• Shared storage libraries

OpenStack Infrastructure : Improve logging

The volume of testing that the OpenStack Infrastructure team does as part of the daily workflow of patch submission and merging, results in a large volume of logs. The size of logs will only increase with additional tests plus the volume of logs is increasing due to the growing number of patches submitted. This situation requires some rethinking of the status quo regarding log archiving and storage.

Working with the current goal of archiving a 6 month history of test logs, the Infrastructure team has had to deal with a rapid consumption of available storage space for logs in several occasions precipitating an the need to increase storage space. Storing this volume of logs in their current verbose state is proving fairly unusable due to the detailed nature of the logging output.

OpenStack Infra has recently made logstash available: http://logstash.openstack.org and searching logs will soon look prettier with a Kibana front end: http://kibana.org/ Here is the patch if you want to track it: https://review.openstack.org/#/c/27089/6

Having consistent logging formats among the projects will make parsing the files with logstash easier and more useful. So the use-case of the stored logs may now be prettier going forward.

That was most of our sessions for today.

By the way, did you know that the next OpenStack Summit will happen in Asia ? See you in Hong-Kong !

How much for an Openstack Cloud please?

That’s it! You’re ready! You know everything there is to know about OpenStack. It’s architecture, features and a bunch of other cool stuff. But finally: How much does an OpenStack Cloud cost?

During this talk, Raphael showed the work on two real production use cases to provide you with a detailed, yet simple, financial analysis that will help you budget your future cloud projects. He also presented the case of a basic simple infrastructure and a second example will treat of a much more complex, high end platform.

Raphaël’s slides are here

High Availability from DevOps side

I and Sebastien explained how to make some OpenStack services highly available in active / active mode, or active / passive for some of them. We share our experiences on production environments where High Availabity is one of our top priority. This session was not a lesson, but more a talk in which that we would share what we are doing at eNovance, and how we achive High Availability on our deployments. We also shown our contribution on Pacemaker Resources Agents, on the documentation and also on Puppet modules for making OpenStack easier and faster to install.

You can check the slides here

From zero to success : The French Touch

Very nice (and funny talk) from Raphaël and Chmouel who explained the « coq au vin » recipe or « How to be a key player in the OpenStack industry ». French people are good cooks aren’t they?

You can see their story here

Havana and Python 3

Python 3 Compatability Team meets every Thursday @ 1600 UTC in #openstack-meeting.

• single code base that is Python 2.x and 3.3 compatible.
• get library projects compatible first (oslo, clients)
• compatibility for current dependencies
• ensure future dependencies are compatible

More informations here.

RackSpace Party

To finish the day, we were at RackSpace party which was really awesome. They’ve done a great job last night and thank’s to them we were able to spend a great time together.

Here some picture of our developers (including me) :

Tomorrow is the last day, and we won’t miss the interesting sessions about OpenStack development. So I’ll see you tomorrow for the last daily report about OpenStack Summit !

Hope you’ll enjoy it!!!

Working @eNovance from eNovance on Vimeo.

My day began fine because I’ve had breakfast with people from the community. I’ve been able to understand what is Outreach Program for Women and by the way I attended the session with other interns on OpenStack. It was very interesting to see that interns can be involved into the development of OpenStack projects.

Did you know that eNovance is hiring some interns ? Yes, we actually looking for some people who want to be involved in one of the most important Open-Source project in Cloud-Computing.

Quantum & Horizon

A great job has been done in Grizzly since more features has been coming in OpenStack Networking : load-balancing, metadata, overlapping, etc. The main new features that have been improved in the last release are : Architecture Overview, Load balancing management, L3 support. Thanks to the developers, we have now a great support of Quantum in Horizon.

OpenStack Networking : SDN controller improvement

Edouard Thuleau from CloudWatt and Nick Barcet led a session about improving the features in the OVS plugin for OpenStack Networking. It was very interesting to see that there is two kind of plugins : third party controller & SDN controller. The community is waiting for that OVS plugin being able to scale and being Highly Available at least with the features that we had in Nova-Network (Multi-Host). The session was explaining that OVS plugin should use both data & controle planes for managing the virtual network.

You can check the slides at this URL.

Tomorrow, eNovance will attend several sessions, and of course we are going to share here our experience and let you know what has been said.

First of all, I was not surprised by the huge number of newcomers at the Summit since the community has grown so fast. For the first time, eNovance has a booth to present what we are doing. Thanks to Raphael (CEO & Co-founder), we have great pictures of today that you can see here.

Our engineers have followed most of the sessions today, and here are some reports that we want to share as we did for last summit in San Diego.

Upgrade from Grizzly to Havana

No magic here, either painful BIG upgrade or easy small hops, but needs strong (continuous) testing & automation.

It’s still currently hard to upgrade Openstack (even when following trunk) without some service interruptions.

Goals:
• allow to have components from version N coexist with components from version N+1 (APIs, message bus, etc. …)
• have minimal downtime during database migrations

Nova and Quantum : Gap status

I was happy to meet the new PTL of Quantum : Mark McClain working at Dreamhost.

A big part of the conversation was concerning Multi-host feature which is the most wanted in Quantum but not ready yet. Grizzly provides scheduling for getting multiple L3 & DHCP agents, but they are not working on the same way like the multi-host in nova-network. People is the room was wondering how things are going in Havana. No decision has been taken at this time and we still don’t know how to scale L3 in Quantum. The second part was focusing on the DNS part of Nova (which is actually not working quite good) : do we need it in Quantum ? Can we connect it to the existing project « Moniker » ? That’s those kind of questions that core developers were wondering today.

Quantum API Updates

The topics which have been mentioned are :

• Backward compatibility with OpenStack N-1 version
• Moving some extensions to core ? (i.e. L3) and also improving framework to optimize code
• New features : XML support, Pagination and Sorting
• New extensions : VPN, Edge firewalling
• API transaction support
• Proposal : for each service, have a core API for which we apply the same strict rules :

   Example: current core –> L2 core

                     router, floating ips –> L3 core

                     vips, pools, etc –> LB core <– (aaS dropped intentionally)
• Improve API documentation

What’s next for SDN ?

This session was a panel in which we had some participant from main SDN companies of today : BigSwitch, Midokura, NTT, NEC and Cisco.

About Overlay :

• Overlay and OpenFlow can coexist
• Customers need automation, OpenFlow stays an option

That’s it for today, I would like to thank Mirantis and Red Hat for the parties !

By the way, if you are at the summit, don’t forget to come at the booth to catch the official eNovance duck !