The Reluctant Tecchie
Automating the Mundane to Advance the Interesting.
Friday, February 20, 2015
Tuesday, January 21, 2014
Getting Instance Details at a Glance with awscli
The Nova command line interface has me completely spoiled. After becoming so used to 'nova list', the usual output of EC2's 'describe instances' makes me feel... disparaged.
Here's a one-liner I assembled to give a Nova-like experience with awscli (i.e. I just need a few high-level details of all my instances).
This yields output like the following:
Here's a one-liner I assembled to give a Nova-like experience with awscli (i.e. I just need a few high-level details of all my instances).
aws ec2 describe-instances --profile my-profile --output table --query \ 'Reservations[*].{\ id:Instances[0].InstanceId,\ key:Instances[0].KeyName,\ security_group:Instances[0].SecurityGroups[0].GroupName,\ public_ip:Instances[0].PublicIpAddress,\ status:Instances[0].State.Name,\ name:Instances[0].Tags[0].Value}'
This yields output like the following:
--------------------------------------------------------------------------------------------------- | DescribeInstances | +------------+--------------------+-----------------+--------------+------------------+-----------+ | id | key | name | public_ip | security_group | status | +------------+--------------------+-----------------+--------------+------------------+-----------+ | i-3dd9871f| user-01-dev | salt-master-01 | 192.168.1.1 | linux | running | | i-41173864| user-34-dev | None | None | mine | stopped | | i-3001421a| user-02-dev | toolbox | None | apache | stopped | +------------+--------------------+-----------------+--------------+------------------+-----------+That is all.
Monday, January 13, 2014
Transferring AWS S3 Buckets
tl;dr - Install awscli (pip install awscli), then:
Install awscli
A quick Google search sent me to this StackOverflow answer, which made awscli news to me. I went with installing via Pip so:
Configure awscli
I have a few profiles to manage, so, following the docs, I configured each of them with:
Transfer the Buckets
Once I tested out each profile with a describe-instances, copying an entire bucket to another region was simple:
And that was it! I love it when things are easy...
aws s3 sync s3://srv-salt . --profile myFirstProfile aws s3 mb srv-salt --profile mySecondProfile aws s3 sync . s3://srv-salt --profile mySecondProfile
Install awscli
A quick Google search sent me to this StackOverflow answer, which made awscli news to me. I went with installing via Pip so:
apt-get install python-pip pip install awscli
Configure awscli
I have a few profiles to manage, so, following the docs, I configured each of them with:
aws configure --profile myFirstProfile aws configure --profile mySecondProfile
Transfer the Buckets
Once I tested out each profile with a describe-instances, copying an entire bucket to another region was simple:
aws s3 sync s3://srv-salt . --profile myFirstProfile aws s3 mb srv-salt --profile mySecondProfile aws s3 sync . s3://srv-salt --profile mySecondProfile
And that was it! I love it when things are easy...
Sunday, January 5, 2014
Salt states for salt-api
If you read my previous post and would like to play the home version, I've written a few states to turn a Salt minion into a (insecure, testbed) install of salt-api.
You can find them over here on Github.
Share and Enjoy!
You can find them over here on Github.
Share and Enjoy!
Thursday, January 2, 2014
Salt-API, A Crash Course
The salt-api docs do a great job but it's a bit unclear on what it takes to get a barebones salt-api proof of concept up and running. No official deployment, no top cover from a web server. Just a Salt master with salt-api running on top of it. We'll cover that here.
I (heavily) referenced the rest_cherrypy section of the docs to put this together.
This post assumes that you already have a Salt master and at least one Minion connected to it.
Install salt-api
The docs seem to mention that Pip is the best method. Also, we're going the CherryPy route so:
Configure CherryPy
Very simple. In your master config file:
Notice the SSL lines. We can create a self-signed keypair very easily with Salt:
Configure External Auth
Salt-api leans on the eauth system for authentication. For our quick and dirty, we can just allow our user to do everything. Back in the master config file:
We're done configuring! With the Salt master & minion running, start salt-api:
You can optionally pass salt-api the -d option to put it into daemon mode.
With salt-api runnig. Test it out:
Success!
We'll need to log in. That looks like this:
This gives us a token. In this example the token is f72309d0ee425193bc8b763a0092470bbabab6bc. Let's use it to run a test.ping.
Not bad. Let's try making it actually do something.
Neat!
You can see where we can go from here. Once you've got a RESTful interface to your Salt master, your creativity is the only limit from what you can do from there.
I (heavily) referenced the rest_cherrypy section of the docs to put this together.
This post assumes that you already have a Salt master and at least one Minion connected to it.
Install salt-api
The docs seem to mention that Pip is the best method. Also, we're going the CherryPy route so:
pip install salt-api cherrypy
Configure CherryPy
Very simple. In your master config file:
rest_cherrypy: port: 8000 ssl_crt: /etc/pki/tls/certs/localhost.crt ssl_key: /etc/pki/tls/certs/localhost.key
Notice the SSL lines. We can create a self-signed keypair very easily with Salt:
salt-call tls.create_self_signed_cert
Configure External Auth
Salt-api leans on the eauth system for authentication. For our quick and dirty, we can just allow our user to do everything. Back in the master config file:
external_auth: pam: saltuser: - .*Turn On the Awesome
We're done configuring! With the Salt master & minion running, start salt-api:
salt-api
You can optionally pass salt-api the -d option to put it into daemon mode.
With salt-api runnig. Test it out:
curl -k https://localhost:8000 {"status": "401 Unauthorized", "return": "Please log in"}
Success!
We'll need to log in. That looks like this:
curl -ksi https://localhost:8000/login \ > -H "Accept: application/json" \ > -d username='saltuser' \ > -d password='password' \ > -d eauth='pam' {"return": [{"perms": [".*"], "start": 1388722947.535586, "token": "f72309d0ee425193bc8b763a0092470bbabab6bc", "expire": 1388766147.535588, "user": "saltuser", "eauth": "pam"}]}
This gives us a token. In this example the token is f72309d0ee425193bc8b763a0092470bbabab6bc. Let's use it to run a test.ping.
curl -ksi https://localhost:8000 \ -H "Accept: application/x-yaml" \ -H "X-Auth-Token: f72309d0ee425193bc8b763a0092470bbabab6bc" \ -d client='local' \ -d tgt='*' \ -d fun='test.ping' HTTP/1.1 200 OK Content-Length: 25 Vary: Accept-Encoding Server: CherryPy/3.2.4 Allow: GET, HEAD, POST Cache-Control: private Date: Fri, 03 Jan 2014 04:31:13 GMT Access-Control-Allow-Origin: * Content-Type: application/x-yaml Set-Cookie: session_id=f72309d0ee425193bc8b763a0092470bbabab6bc; expires=Fri, 03 Jan 2014 14:31:13 GMT; Path=/ return: - salt-api: true
Not bad. Let's try making it actually do something.
curl -ksi https://localhost:8000 \ -H "Accept: application/x-yaml" \ -H "X-Auth-Token: f72309d0ee425193bc8b763a0092470bbabab6bc" \ -d client='local' \ -d tgt='*' \ -d fun='cmd.run' \ -d arg='echo "hi salt-api!"' HTTP/1.1 200 OK Content-Length: 33 Vary: Accept-Encoding Server: CherryPy/3.2.4 Allow: GET, HEAD, POST Cache-Control: private Date: Fri, 03 Jan 2014 04:34:19 GMT Access-Control-Allow-Origin: * Content-Type: application/x-yaml Set-Cookie: session_id=f72309d0ee425193bc8b763a0092470bbabab6bc; expires=Fri, 03 Jan 2014 14:34:19 GMT; Path=/ return: - salt-api: hi salt-api!
Neat!
You can see where we can go from here. Once you've got a RESTful interface to your Salt master, your creativity is the only limit from what you can do from there.
Sunday, December 15, 2013
The Industrial Model Isn't Dead - It's Migrating
People love something 'made by hand'. Tell them that, to make your table, you collected driftwood from the beach. Or your vanilla extract is different because you've met the farmers and you get your alcohol for extraction from the (craft) distillery in your town. A beer made with intent will always taste better than a beer made for a bottom line. Tell someone a story, or how you painstakingly made each groove, and people listen.
Software is not like that. Systems, deployments, IT, and data centers are not like that.
No one (okay, very few people) cares about the love that was put into some software. They're not concerned with the initial itch that needed scratching. They don't look forward to hearing about how the author put countless hours of effort into automating some mundane routine so that the user wouldn't even notice that it had been done for them.
No one builds data centers by hand now and no one wants to hear about how someone built a data center by hand (rather - the people who do still build them by hand soon won't; either because they'll have upgraded their job or been left behind).
This leads to an interesting contrast between hand made goods and software. We like to hear that a coffee table took two weeks of carving and staining. But a data center? We want to know that a system can be deployed 'with the push of a button'. Never mind made with care, we want the actual creation of a system to be as automated and hands free as possible. Where goods are moving away from the bulk, industrial manufacturing model, IT systems and application deployment is running straight into it. More systems, faster, for less work (and, thus, cost).
The industrial model is still very much alive. We're just moving it from farms and goods over to data centers.
Tuesday, November 26, 2013
Getting Religion - Salt
Apparently I was on a quest and didn't even know it. Automation is great (essential, even), but at some point you need to wrap your arms around the whole thing. After spending some time with Puppet (and deciding it wasn't for me), I found Chef to be pretty approachable. I like the native versioning of recipes and the hands-free method of creating workstation separate from the master (the fewer hands that touch the master, the better). But there was a definite learning curve to be overcome.
Then I found Salt.
Pure simplicity.
How do I install Salt on CentOS?
yum install salt-master
How do I use Salt to install Apache?
salt myMinion pkg.install httpd
What about all my webservers?
salt -N myWebServers pkg.install httpd
Install Apache, pull down the latest version of my code, and restart Apache if the code changes!
my-awesome-app:
pkg.install:
- name: httpd
git.latest:
- name: http://github.com/awesome/sauce.git
- target: /var/www/html/awesomesauce
service:
- name: httpd
- enable: True
- running
- watch:
- git: my-awesome-app
And done! Every server. Latest code. Apache keeps up. All in easy to read yaml.
So simple.
Hell, it even works with Powershell. Fully automated MSSQL installation? Done. Local user management? Easy. I'm confident that, with the proper Salt setup, you can do away with System Center Config Manager. The implications are huge for people willing and able to do it themselves (instead of paying for Microsoft licenses and consultants).
I'm hoping to make some big changes in my environment with Salt. The more work we can get the machine to do, the more time we have to work on more interesting problems.
Then I found Salt.
Pure simplicity.
How do I install Salt on CentOS?
yum install salt-master
How do I use Salt to install Apache?
salt myMinion pkg.install httpd
What about all my webservers?
salt -N myWebServers pkg.install httpd
Install Apache, pull down the latest version of my code, and restart Apache if the code changes!
my-awesome-app:
pkg.install:
- name: httpd
git.latest:
- name: http://github.com/awesome/sauce.git
- target: /var/www/html/awesomesauce
service:
- name: httpd
- enable: True
- running
- watch:
- git: my-awesome-app
And done! Every server. Latest code. Apache keeps up. All in easy to read yaml.
So simple.
Hell, it even works with Powershell. Fully automated MSSQL installation? Done. Local user management? Easy. I'm confident that, with the proper Salt setup, you can do away with System Center Config Manager. The implications are huge for people willing and able to do it themselves (instead of paying for Microsoft licenses and consultants).
I'm hoping to make some big changes in my environment with Salt. The more work we can get the machine to do, the more time we have to work on more interesting problems.
Subscribe to:
Posts (Atom)