I find that I always re-iterate how I name things. This is especially hard I think when naming functions when programing or servers when infrastructuring (if that was a valid verb).

But, I think I have found a rule of thumb now when it comes to servers. Isn't this super basic stuff?, you ask. And of course it is, but in my experience the basic things are some times hard to get right. And cumbersome to fix in the future.

And maybe for you it does not matter that much, perhaps you just have like 10 servers. But I argue that to have a good naming scheme is part of how you are able to scale when needed.

When you name a server you want the name to include just enough information. Enough so that when it turns up in a log or an alert you have enough information to act.

I try to include the following in a server name: * location, like data center or cluster * environment, like production or staging * type, like web server or reverse proxy * identification or counter number, like random characters or an iterated number

The above might not work for you and I might overlook something. But, this is what I've come up with as a start.

I like to have each part of the name to be about three of four characters joined with dashes. So, location-environment-type-identification.

If, heavens forbid, you have to run Windows Server you also need to account for the fact that you can't have more than 15 characters in the server name due to limitations from the 1980s. So, I simply chant “serenity now” and accept this fact. Then always name servers with 15 characters. No more, no less.

Why bother with a name convention

Lets say you have a centralized logging solution and all logs are sent to that. And lets say you have 5 servers running the same web server. If all servers are named something “web” and perhaps even with an iteration like “web-01” you might be good for a while, but what if these servers resides in different parts of your infrastructure. Then, the logs are not telling the full picture.

If the name includes some tidbits about what it is (or serve), where it is and who it is you can build better automation and alerts with the server name as a key. Another example of why it is good to have a naming convention is that if perhaps one location has an issue, you will immediately see that pattern emerge in the logs and alerts.

Location

The location might be the actual host where an instance is located. But, in this context I'm thinking more of an infrastructure where you work with different availability zones and data centers. And if you also deploy your work loads at different providers the data centers and availability zones of course differs.

So, either you include the name of the provider and datacenter in the location. Or you create your own locations that are pointers to the so to speak real locations.

As an example, if I use Safespring as a provider and I create a server in one of their three datacenters, lets say Stockholm South. How would I abbreviate this into the location part of the name?

Well, “Safespring Stockholm South”, could be abbreviated. However, the amount of the letter “s” makes this hard. A usual way to abbreviate is to choose the first letter or letters of the words. In this case it would result in “SSSS”. And, how do I protect against a name change? As an example a couple of years ago City Network change their name to Cleura. What if Safespring re-brands in the future?

So, I argue that we need to create our own namespace for the location of our servers.

Let's assume we use two datacenters at Safespring and one from Cleura. We could simply create our own of location names like this: * DC01, Safespring Stockholm South * DC02, Safespring Stockholm North * DC03, Cluera Karlskrona

Or of course instead abbreviate location like with an L or Loc: * LOC01, Safespring Stockholm South * LOC02, Safespring Stockholm North * LOC03, Cluera Karlskrona

However, I really do like it when each part of a name has a length of three or four. So, I would choose to use the “DC01” or “L01” to save on length. Also, if you want to include locations that are not a physical location like a datacenter but instead maybe a cluster, I would use the “L01” abbreviation since “DC” implies datacenter. Perhaps even L001 to allow for 999 locations. But, I think that 99 locations are enough for most.

This makes it possible to include location in the name of a server or service in a scalable way.

Environment

It is necessary to include the environment that the server is part of. The obvious reason is of course that it would become quite easy to make terrible mistakes otherwise. Like shutting down the wrong server because you thought it was just a development environment server.

So, as with location, you should create a name convention. Two common environment types are staging and production. These could be abbreviated to s and p to keep it short.

Given the example for location names we could name a server dc01-p to mean “Safespring Stockholm South production environment”.

It might be hard to only allow one character for the environment, but I think it is a good practice. Then we can use more characters for the type part of our name when needed and still keep the 15 character limit for Windows Server.

I mean, p could be production, s staging, d could med dmz and so on. One just have to decide which letter will represent each type of environment one has.

Type

The type of server can also be tricky. One quick way is to set the type based on the service running on the server. If it is a web server running on it call the type “web”.

RabbitMQ could become “rmq” and Apache Airflow could become “airflow”. But that is too long in my opinion and does not have a given three or four character abbreviation.

So again, we need to come up with a nomenclature that fits our use case. Web and SQL database servers are quite easy to choose type name for.

RabbitMQ could also be shortened “msq” for message queue. And Apache Airflow could be shortened to flow or batch (since it is a batch workflow kind of service). As long as we keep the rule to three characters when dealing with Windows Server we can allow us to be more creative with the type nomenclature when needed.

What you really need to consider is to perhaps not mix names of a particular brand of a service with what the service is providing. Because, applications rebrand at times too. And RabbitMQ it is a message queue application, but there are others like it too. If we let the type be less about the brand name and more about what type of service we are using we improve the agility of the server name convention we build.

Identification or iteration

I tend to like the iteration number at the end because it is easy to verbalize which server you are talking about with team members. But, you run into the risk of duplicated names and using the same number on different types of servers. There might be a “sql-001” and a “web-001”. Or a “p-web-001” for a production server and “s-web-001” for a staging server.

When I create servers with OpenTofu or Terraform I use the https://search.opentofu.org/provider/hashicorp/random/latest to generate a UUID for the server I'm creating. I push this UUID as an environment variable to the server as well. It can be used to verify your records of what servers you have created. Not at all as an identity of the server but as a simple way of checking your records.

A UUID is made up of five parts separated by a dash, like so: 38f25e0a-23d4-4b02-8d8a-4d666ee05c08. I use the second part as the identifier. Since UUIDs are very, very unique a reckon it is very unlikely that these four characters will ever be the same for the servers I create.

The reason to use UUID parts is that it is quite natural to verbalize 23d4. You can say “twenty three dee four”, or “dee four” for short. If the full server name is dc01-p-web-23d4 you could say “we have an issue with web server twenty three dee four” or “we have an issue with our production web server dee four” and everyone would have little to no trouble to grasp which server you are talking about even in a crappy MS Teams meeting.

A unique identifier opposed to a iteration number makes it easy to search for the server in internal ledgers or when searching through metric or log management systems.

Conclusion

So, there you have it. A lot of words about naming servers. My server name would end up something like this for a web server: lo01-p-web-23d4

and for an configuration management tool like Ansible, I could name the controller server: lo02-m-cfg-f2j2 or perhaps even lo02-m-ansible-f2j2 since it is running on Linux and I could expense with the longer name.

This is simply my take on this matter. This is not the only way to name servers and I am probably missing some thing that I should think about when naming servers. However, that is normal because this is an iterative process. As is most processes.

If you have any questions or want to comment on this, you'll find me on https://mastodon.nu/@jdurehed with the handle https://mastodon.nu/@jdurehed.

Joakim Durehed

I don't use US providers if I don't have to. I will do a post on why some time in the future. Because AWS S3 has been such a success many tools have created support for the S3 object storage. So, both OpenStack Swift and Ceph have built compatability for the S3 API. And Ceph has the benefit of being compatible with both S3 API and OpenStack Swift API.

Ceph is the object storage solutuion at Cleura. They have quite good documentation on how to get get started. Like how to create ec2 credentials with OpenStack, so I suggest you follow their guide.

To use S3 backend with OpenTofu you first need to create a bucket. Then you can configure the backend and initialize with OpenTofu. And from there on you are rocking a remote backend set up with OpenTofu.

In OpenTofu the configuration needed is not not complicated. The snippet below shows the configuration needed for OpenTofu:

terraform {
  required_version = ">= 1.8.0"
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 2.1.0"
    }

  }

  backend "s3" {
    bucket = "a-cool-bucket-name"
    key    = "a_cool_state_name.tfstate"
    endpoints = {
      s3 = "https://s3-kna1.citycloud.com"
    }
    skip_s3_checksum            = "true"
    region                      = "us-east-1"
    use_path_style              = "true"
    skip_credentials_validation = "true"
    skip_requesting_account_id  = "true"
    skip_metadata_api_check     = "true"
    access_key                  = var.s3_access_key
    secret_key                  = var.s3_secret_key
  }
}

The acces_key and even more so the secret_key should be provided in a secure way. Either by your OpenTofu runner of choice or using Terragrunt and SOPS or something like that.

At Cleura the Karlskrona datacenter, Kna1, has an Object Storage with S3 compatability with features such as Object-Lock, Object-Versioning and Server Side Encryption (SSE).

Since I am using OpenTofu, I want to use state encryption later. That means I will encrypte the state file on the client side and thus protecting the state file before it reaches the object storage. The benefit of that is that I will not need to provide the encryption key to Cleura.

To create a bucket you might use the AWS S3 cli tool. The following snippet shows how to create a bucket with object lock enabled which also enables object versioning.

aws --profile PROFILENAME \
  s3api create-bucket \
  --bucket BUCKETNAME \
  --object-lock-enabled-for-bucket

When you have created the bucket and the backend configuration is complete you can perform tofu init.

If you already have a state file, make sure it is located inside your project root folder, called terraform.tfstate and perform tofu init -migrate-state

In a future post I will dig a bit deeper into state encryption and backups of the state file and how to use SOPS with OpenTofu.

#terraform #opentofu #openstack #s3 #objectstorage #remotebackend #cleura

Joakim Durehed

Regrettably, I have to work with Windows Server every day. I won't go into why I think this is regrettable in this post. But, one, of many, thing that sucks when running Windows Server is that so much information on the amazing world wide web is geared towards running applications on some sort of Linux. In the “put something on a server” world, some kind of Linux is the norm.

Anyway, I wanted to create a local Administrator account without having to push a password through CloudBase-Init in the user_data configuration. As an example one can do something like this:

users:
  - name: cool_boy_82
    groups: Administrators
    passwd: very-cool-and-strong-password
    ssh_authorized_keys:
      - ssh-rsa AAAB...byV

However, even if the above do work you get a little bit of a security issue with this approach. Say that you forget to reset the password you give user cool_boy_82 and you have some kind of breach. It can be possible to either read the logs from CloudBase-Init in C:\Program files\Cloudbase or to fetch the logs from the metadata service. Because if you put the password in clear text, it will be available in the logs in clear text. You might use this as a default account that you always add when you create new instances in OpenStack. Well, then you have a security issue.

Let CloudBase-Init generate user password

One solution for this is to let Cloudbase-Init to do the password generation for you. Then you retrieve it later through the magic of encryption. However, I ran into some issues. Whenever I tried to retrieve the password from the OpenStack CLI I got the following error:

b'unable to load Private Key [...] PEM routines [...]

So, something was wrong. And as so many times before, the error were somewhere behind the keyboard.

When you create an instance in OpenStack you tell it to use a ssh-key public key of your choice. The ssh-key public key can be used to encrypt the password that Cloud-Base Init generates so that ONLY the one who has access to the private key can decrypt it.

It is quite easy, but I ran in to a gotcha. Turns out that it is very important to create the ssh-key in a specific format and type of key. Also, if you password protect your private key, you can't retrieve the password from the OpenStack Horizon UI.

We need to use a ssh-key with type rsa and make sure that it is in the pem format. I use ed25519 as my type of choice. Especially after I started using a TKey from Tillitis wich only generates ed25519 keys.

So to generate a ssh-key of the correct type and format the following should suffice:

ssh-keygen -t rsa -m PEM -f "cool_boy_82"

So, then to retrieve the password generated for an instance use the OpenStack CLI with the following command:

nova get-password {instanceID} {PathTo: private key file}

So, all is well. If you simply do things the right way things work better. One thing that I do feel is not so “well” is the documentation from OpenStack and some of the OpenStack providers. I did search a hell of a lot to find information about this issue and I could not find conclusive, to me at least, information about what OpenStack required in terms of ssh-keys and formats. One good reasource is from the Swedish OpenStack provider Safespring. They have a very good blog post about ssh-keys and OpenStack.

Anyway, I put this out on the wide wide wide web to let any body else have the chance to find this quicker than I did.

#openstack #sshkey #cloudbaseinit #cloudinit #tkey #tillitis

Joakim Durehed

I've started to build on my project during my vacation. And even though I have not made a lot of progress in terms of getting something up and running for show I have made progress on a personal level.

It has been a long time, plus 8 years, since I did anything remotely web-based and development-like. I'm not a developer and so I've had to find a CSS framework to use, because I'm can't cope with doing anything in CSS from scratch.

I found Bulma which I chose for two reasons. The license (MIT) and the fact that I found that it was quite easy to get my head around. Also, a bonus is that it is not a project from Google or any such big player. Another benefit is that it is only CSS and quite a large community.

Also, somebody have made a Gem to make it very easy to install with a Ruby on Rails application. Which I managed.

And that is kind of the progress I have made. Getting myself accustom to the Rails ecosystem, to web development with CSS and such, and so on.

I have also made progress in finding resources where I can get data about prices for electricity in the EU and so on. I also managed to import a CSV that I had exported from my solar power system.

And finally I also found a charting Gem, ChartKick that uses Chart.js so that I can start building nice graphs rather effortless with Rails.

Just being able to create a layout, data imports and showing a table with the imported data is quite powerful. This I managed in about 4-8 hours over the course of a week. Generally I can tinker early in the morning or when my children are tired and just wanna watch TV.

But perhaps most important I have now some idea of how I can build my app. I have decided to create a micro service that is responsible for fetching electricity prices. I will probably not do that right away, but I think that my app will be much easier to maintain with a simple API request to my own creation managed in a separate repository. On reason is also that one does not have more than 400 requests per hour to the API at Entsoe. So I will store the results in my backend and let my app be able to fetch the data when needed. This way I can control limitations better I reckon.

In general I find that Ruby fits me. I think it really fits me, not much code has to be written to perform quite large tasks. And with the Rails framework you get so much done. If I had been working with web development for ages I might have had a different take on this. Perhaps I would have more objections to how things are done in Rails. But for me the opinionated approach of using a framework that pushes or nudges you in the right directions suits me.

I hope I will be able to get a working demo of what I want to achieve in a 1.0 version of the application in like two or three months time.

#ruby #rubyonrails #buildinganapp

Joakim Durehed

I really like that there is a free and Open Source fork of Terraform now. For me, the future is Open Source, and specifically Free and Open Source.

I don't believe this because I like to not have to pay for software. I believe this because it is no other way to move forward with digital transformation at a high pace effectively. Very much so in the public sector, but also for organizations in general.

OpenStack is my platform of choice. First off, it is Open Source, and second off, a lot of public cloud providers deploy OpenStack. So there are a lot of places to put your infrastructure.

And this is where a cool thing about infrastructure as code comes through. It is so easy to deploy resources to different providers and platforms that it was almost lost on me. I had to try it out and then I started to get it.

But, anyhow, lets do something with OpenTofu and OpenStack.

The basic stuff first

We are doing this without extra tools like Terragrunt and Gitlab. I usually create a variable to hold all parameters to connect to OpenStack. I keep the sensitive part, the application credential secret in a separate file called connect.auto.tfvars. This way, that file, containing the secret will be read by OpenTofu at plan, apply, destroy and so on.

But, when working in a team, this would preferably be read from a secret manager of some sort or in your build server so that you don't even need to store the secret locally.

Below is the file I call main.tf because it is the starting point in my mind with the provider connection parameters and the OpenTofu configuration.

terraform {
  required_version = ">= 0.14.0"
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 1.53.0"
    }
  }
}

provider "openstack" {
  application_credential_id     = var.connection.application_credential_id
  application_credential_secret = var.application_credential_secret
  auth_url                      = var.connection.auth_url
  region                        = var.connection.region
  endpoint_type                 = var.connection.endpoint_type
}

To use the main.tf file we also need a variables.tf file. You could call this whatever you want, but I will probably add more variables to this file, so instead of calling it connect.tf to categorize it with the connect.auto.tfvarsfile, I just call it varibles.tf.

I use Binero as the public cloud provider. Binero is based in Sweden, as am I.

I use an application credential instead of a user account because I like to differentiate using user accounts from API oriented accounts. Application credentials were designed for this purpose.

variable "connection" {
  type = object({
    application_credential_id = string
    auth_url                  = string
    endpoint_type             = string
    region                    = string
  })

  default = {
    application_credential_id = "application_id"
    auth_url                  = "https://auth.binero.cloud:5000/v3"
    region                    = "europe-se-1"
    endpoint_type             = "public"
  }
}

variable "application_credential_secret" {
  type      = string
  sensitive = true
}

Create a network

So with a main.tfvars and variables.tf accompanied with the connect.auto.tfvars file we have all we need to make some resources with OpenTofu.

In this case I will create a network and a subnet to that network.

I add a variable for the network cidr in the variables.tf. This could be more comprehensive, but works for our example.

variable "main_network_cidr" {
  type    = string
  default = "10.10.10.0/24"
}

And we use the above variable in the resource declaration in network.tf:

resource "openstack_networking_network_v2" "main" {
  name           = "main"
  description    = "Main network for general purpose."
  admin_state_up = "true"
}

resource "openstack_networking_subnet_v2" "main" {
  name        = "main"
  description = "Main network subnet for general purpose."
  network_id  = openstack_networking_network_v2.main.id
  cidr        = var.main_network_cidr
  ip_version  = 4
}

I would normally create input variables for all the above arguments in the resource blocks. But again, this is just an example. The idea of using a variable for each argument is that you then can re-use the resource block in other projects and create a module of it. And it makes it possible to add variables when performing tofu apply.

The above should get you running with creating a network and subnet in OpenStack. Remember that the connection arguments might differ between OpenStack providers.

Also, this is just an example, I will create more comprehensive projects in the future where I will gear towards something that is production ready. I will also share my code in a public repository (as soon as my public repository is ready for that)

I hope you found this useful. As stated, more is to come.

#terraform #opentofu #openstack #network

Joakim Durehed

I have solar on my barns roof. I am a micro energy producer. So naturally I'm interested in knowing about how much money and energy I save or how much I don't save any given day or month.

This perspective is not available to me without me taking the data I do have and perform calculations on it. My project is therefore very simple. I want a web app that performs the calculations for me and present it in a decent dashboard format for me.

And to make it a bit more challenging, since the above is quite easy to do if I settle for a cron-job doing the fetching and calculations for me and then just report it to me, I will do this web app as a software as a service.

So the gist of the app is to import data from energy producing systems, pare that with energy prices, other incentives and deductibles and as much data as possible really, and create a central birds eye view of everything about your energy consumption and production.

I'm not sure I will cope and actually do this, but I have this day dream of having a side hustle that is serious enough to make it possible to focus on. I want to trick my brain into focusing on this and some times I have to imagine some audience or receiver to achieve this.

I will not add more about this project before I had more time to outline it for my self, but I have decided on a few things.

It will be built in Ruby with Ruby on Rails and the app will target a European audience. Meaning that all the input (costs, incentives and so on) are from EU sources. So the app will focus on the EU-based micro energy producers.

I genuinely think this could be a viable product that I can charge for if I get it up and running. This is because most resellers of solar energy systems are creating their own ecosystems and they are generally bad at putting useful data in front of their users. And another important factor, if you change the energy reseller historic data will be lost in time. If there were a central solution with an open format data export feature, like the one I want to build, a user is able to opt out at any time and still be able to retain their data.

This post is a way for me to kick me in the butt to stop thinking about making this and start doing it.

Joakim Durehed

Today is the first time I use OpenTofu the Open Source Terraform contender.

HashiCorp made the terrible decision to go for a less open license called BSL. A license aimed at stifle any competition on creating services on top of the products under this license. In this case, it will make it impossible to create Terraform Cloud competitor for instance.

It is of course not illegal for Hashicorp to do this. They are completely free to change their license. They should have expected the backlash. And I think they did since they have not said a word about it really since the license change as far as I can tell.

I might do a write up on why a license change like this is a shitty move, but for now I'll point you to this good write up on that.

Back to OpenTofu. The first release, OpenTofu 1.6, is compatible with Terraform 1.6. They have made it very, very simple to install and to migrate from Terraform to OpenTofu. I'm not migrating anything today but simply creating some stuff with OpenTofu for the fun of it. And to just start using it.

You still use the .tf file ending and the provider configuration block is still named Terraform as you can see below.

terraform {
  required_providers {
    openstack = {
      version = ">= 1.54.1"
      source  = "terraform-provider-openstack/openstack"
    }
  }
}

What I lack today is a a way to run tofu apply with a remote state. There are some tools that do this, but most do it a way that really don't fit how I work right now. Of course this can be done with Terraform using Terraform Cloud. Which I do like and use to some extent. But, it of course comes with a price in different forms. For one, I can only us Terraform, not OpenTofu. It runs on AWS if I'm not mistaken and that is not really my taste (GDPR/FISA). And it also ties me to Hashicorp and their willingness to increase prices.

Some times it is a good fit to use a SaaS and just pay as you go. But some times, it is not. For me the problem is that I need to store critical credentials in Terraform Cloud. Credentials that allows for Terraform to create, modify and delete resources. I'm not saying that Terraform Cloud is not secure. It is probably very, very secure. But every something-as-a-service you use adds to the list of suppliers you need to trust. And even though I do trust Hashicorp to be very security first minded and extremely competent. I mean, really, competent. They are extremely talented. Period. No one is immune to attacks or mistakes.

So, as silly as it might sound to some, but when it comes to this level of trust, I rather be the one who do the mistakes than others. That is because when shit hits the fan, which it will at some point, I will have most if not all the pieces to the puzzle of what happened. And that to me is extremely important. I don't need anyone to blame.

Anyways, I will create OpenTofu related content here that is more practical and not like this post, a bunch of mumbling.

I also want to make it clear that even though I don't like the choice Hashicorp has made in regards to licensing. I still really like what they do in general and have so much to thank them for. They have made my job so much better by creating Packer, Vagrant, Terraform and so on.

#opentofu #terraform #openstack

Joakim Durehed