Tuesday, January 21, 2014

Getting Instance Details at a Glance with awscli

The Nova command line interface has me completely spoiled. After becoming so used to 'nova list', the usual output of EC2's 'describe instances' makes me feel... disparaged.

Here's a one-liner I assembled to give a Nova-like experience with awscli (i.e. I just need a few high-level details of all my instances).

aws ec2 describe-instances --profile my-profile --output table --query \
'Reservations[*].{\
id:Instances[0].InstanceId,\
key:Instances[0].KeyName,\
security_group:Instances[0].SecurityGroups[0].GroupName,\
public_ip:Instances[0].PublicIpAddress,\
status:Instances[0].State.Name,\
name:Instances[0].Tags[0].Value}'

This yields output like the following:

---------------------------------------------------------------------------------------------------
|                                        DescribeInstances                                        |
+------------+--------------------+-----------------+--------------+------------------+-----------+
|     id     |        key         |      name       |  public_ip   | security_group   |  status   |
+------------+--------------------+-----------------+--------------+------------------+-----------+
|  i-3dd9871f|  user-01-dev       |  salt-master-01 |  192.168.1.1 |  linux           |  running  |
|  i-41173864|  user-34-dev       |  None           |  None        |  mine            |  stopped  |
|  i-3001421a|  user-02-dev       |  toolbox        |  None        |  apache          |  stopped  |
+------------+--------------------+-----------------+--------------+------------------+-----------+

That is all.

Monday, January 13, 2014

Transferring AWS S3 Buckets

tl;dr - Install awscli (pip install awscli), then:


aws s3 sync s3://srv-salt .   --profile myFirstProfile
aws s3 mb srv-salt --profile mySecondProfile
aws s3 sync . s3://srv-salt --profile mySecondProfile


Install awscli
A quick Google search sent me to this StackOverflow answer, which made awscli news to me. I went with installing via Pip so:


apt-get install python-pip
pip install awscli

Configure awscli
I have a few profiles to manage, so, following the docs, I configured each of them with:

aws configure --profile myFirstProfile
aws configure --profile mySecondProfile

Transfer the Buckets
Once I tested out each profile with a describe-instances, copying an entire bucket to another region was simple:

aws s3 sync s3://srv-salt .   --profile myFirstProfile
aws s3 mb srv-salt --profile mySecondProfile
aws s3 sync . s3://srv-salt --profile mySecondProfile

And that was it! I love it when things are easy...

Sunday, January 5, 2014

Salt states for salt-api

If you read my previous post and would like to play the home version, I've written a few states to turn a Salt minion into a (insecure, testbed) install of salt-api.

You can find them over here on Github.

Share and Enjoy!

Thursday, January 2, 2014

Salt-API, A Crash Course

The salt-api docs do a great job but it's a bit unclear on what it takes to get a barebones salt-api proof of concept up and running. No official deployment, no top cover from a web server. Just a Salt master with salt-api running on top of it. We'll cover that here.

I (heavily) referenced the rest_cherrypy section of the docs to put this together.

This post assumes that you already have a Salt master and at least one Minion connected to it.

Install salt-api
The docs seem to mention that Pip is the best method. Also, we're going the CherryPy route so:

pip install salt-api cherrypy

Configure CherryPy
Very simple. In your master config file:
 
rest_cherrypy:
  port: 8000
  ssl_crt: /etc/pki/tls/certs/localhost.crt
  ssl_key: /etc/pki/tls/certs/localhost.key

Notice the SSL lines. We can create a self-signed keypair very easily with Salt:

salt-call tls.create_self_signed_cert


Configure External Auth
Salt-api leans on the eauth system for authentication. For our quick and dirty, we can just allow our user to do everything. Back in the master config file:
external_auth:
  pam:
    saltuser:
      - .*
Turn On the Awesome
We're done configuring! With the Salt master & minion running, start salt-api:
salt-api 

You can optionally pass salt-api the -d option to put it into daemon mode.

With salt-api runnig. Test it out:
curl -k https://localhost:8000

{"status": "401 Unauthorized", "return": "Please log in"}



Success!

We'll need to log in. That looks like this:
 curl -ksi https://localhost:8000/login \
> -H "Accept: application/json" \
> -d username='saltuser' \
> -d password='password' \
> -d eauth='pam'

{"return": [{"perms": [".*"], "start": 1388722947.535586,
"token": "f72309d0ee425193bc8b763a0092470bbabab6bc",
"expire": 1388766147.535588,
"user": "saltuser",
"eauth": "pam"}]}


This gives us a token. In this example the token is  f72309d0ee425193bc8b763a0092470bbabab6bc. Let's use it to run a test.ping.

 curl -ksi https://localhost:8000 \
-H "Accept: application/x-yaml" \
-H "X-Auth-Token: f72309d0ee425193bc8b763a0092470bbabab6bc" \
-d client='local' \
-d tgt='*' \
-d fun='test.ping'

HTTP/1.1 200 OK
Content-Length: 25
Vary: Accept-Encoding
Server: CherryPy/3.2.4
Allow: GET, HEAD, POST
Cache-Control: private
Date: Fri, 03 Jan 2014 04:31:13 GMT
Access-Control-Allow-Origin: *
Content-Type: application/x-yaml
Set-Cookie: session_id=f72309d0ee425193bc8b763a0092470bbabab6bc; expires=Fri, 03 Jan 2014 14:31:13 GMT; Path=/

return:
- salt-api: true




Not bad. Let's try making it actually do something.
curl -ksi https://localhost:8000 \
-H "Accept: application/x-yaml" \
-H "X-Auth-Token: f72309d0ee425193bc8b763a0092470bbabab6bc" \
-d client='local' \
-d tgt='*' \
-d fun='cmd.run' \
-d arg='echo "hi salt-api!"'

HTTP/1.1 200 OK
Content-Length: 33
Vary: Accept-Encoding
Server: CherryPy/3.2.4
Allow: GET, HEAD, POST
Cache-Control: private
Date: Fri, 03 Jan 2014 04:34:19 GMT
Access-Control-Allow-Origin: *
Content-Type: application/x-yaml
Set-Cookie: session_id=f72309d0ee425193bc8b763a0092470bbabab6bc; expires=Fri, 03 Jan 2014 14:34:19 GMT; Path=/

return:
- salt-api: hi salt-api!



Neat!

You can see where we can go from here. Once you've got a RESTful interface to your Salt master, your creativity is the only limit from what you can do from there.