Puppet Solo (AKA: supply_drop)

Similar to knife-solo for use with chef, supply_drop allows you to provision servers using puppet without the need for a puppet master server. It uses capistrano for executing commands on the remote server. I put together a working sample set of puppet and supply_drop deployment scripts for provisioning a postgres server.

See the github repo at  https://github.com/house9/puppet-solo-hello-world

Resources

• http://www.confreaks.com/videos/2479-railsconf2013-devops-for-the-rubyist-soul
• https://speakerdeck.com/jtdowney/devops-for-the-rubyist-soul-at-rubynation-2013

WAL-e chef cookbook

Postgres has various backup and restore options

• http://www.postgresql.org/docs/9.2/static/backup.html

Do you have a recovery plan in case your Postgres server crashes - your daily pg_dump is probably not going to cut it.

Postgres uses Write-Ahead Logging (WAL)

Write-Ahead Logging (WAL) is a standard method for ensuring data integrity. A detailed description can be found in most (if not all) books about transaction processing. Briefly, WAL's central concept is that changes to data files (where tables and indexes reside) must be written only after those changes have been logged, that is, after log records describing the changes have been flushed to permanent storage. If we follow this procedure, we do not need to flush data pages to disk on every transaction commit, because we know that in the event of a crash we will be able to recover the database using the log: any changes that have not been applied to the data pages can be redone from the log records. (This is roll-forward recovery, also known as REDO.)

• from http://www.postgresql.org/docs/9.2/static/wal-intro.html

Setting up Continuous Archiving and Point-in-Time Recovery (PITR) for your Postgres WAL files is very complex, lucky for us the WAL-e project has simplified this process greatly. WAL-e has utilities for sending all WAL files to an AWS S3 bucket as the log files are being generated. More Information, see:

• http://www.postgresql.org/docs/9.2/static/continuous-archiving.html

• https://github.com/wal-e/wal-e

Recently I put together a chef cookbook which installs WAL-e on a Postgres instance, see:

• https://github.com/house9/wal-e-cookbook

• https://github.com/house9/use_wal_e

Vagrant and puppet postgres 9.2 on ubuntu

Some vagrant configuration to install postgres 9.2 using puppet

	exec {
	"apt-get update":
	path => "/usr/bin"
	}
	# manage_package_repo => true will satisfy apt_postgresql_org
	# include postgresql::package_source::apt_postgresql_org
	class { 'postgresql':
	version => '9.2',
	manage_package_repo => true,
	charset => 'UTF8',
	locale => 'en_US.UTF-8'
	}
	class { 'postgresql::server':
	}
	postgresql::db { 'mydatabasename':
	user => 'mydatabaseuser',
	password => 'mypassword'
	}

view raw default.pp hosted with ❤ by GitHub

	# 1) install puppet locally
	# - install modules to local modules directory
	# puppet module install puppetlabs/postgresql --force -i modules
	# puppet module install puppetlabs/stdlib --force -i modules
	# puppet module install puppetlabs/firewall --force -i modules
	# puppet module install puppetlabs/apt --force -i modules
	# puppet module install ripienaar/concat --force -i modules
	# OR
	# download tar file from https://forge.puppetlabs.com/puppetlabs/postgresql
	# and place into modules directory
	# 2) create a manifests/default.pp file - see https://gist.github.com/house9/5762219
	Vagrant.configure("2") do |config|
	config.vm.box = "precise64"
	config.vm.provision :puppet do |puppet|
	puppet.module_path = "./modules"
	end
	end
	# 3) vagrant up

view raw Vagrantfile hosted with ❤ by GitHub

Resources

• https://forge.puppetlabs.com/puppetlabs/postgresql
• https://github.com/puppetlabs/puppetlabs-postgresql

Vagrantfile for setting up multiple machines with chef configuration

Example Vagrantfile below for setting up multiple virtual machines and using chef to provision them

	Vagrant.configure("2") do |config|
	# assumes your chef.json files are in cookbooks or site-cookbooks directory
	servers = [
	{
	id: :db1,
	ip: "172.16.2.111",
	node_json: "master-db.json"
	},
	{
	id: :db2,
	ip: "172.16.2.222",
	node_json: "standby-db.json"
	},
	{
	id: :web1,
	ip: "172.16.2.333",
	node_json: "web.json"
	}
	]
	# for testing remove some servers from provision
	db_servers.reject! { |item| item[:id] == :db2 }
	db_servers.each do |server_settings|
	config.vm.define server_settings[:id] do |db|
	db.vm.box = "precise32"
	db.vm.network :private_network, ip: server_settings[:ip]
	db.vm.provision :chef_solo do |chef|
	chef.cookbooks_path = ["./cookbooks", "./site-cookbooks"]
	# http://jbbarth.com/posts/2011-03-20-vagrant-provisioning-with-chefsolo.html
	["./nodes/#{server_settings[:node_json]}"].each do |node_file|
	json = JSON.parse(File.read(node_file))
	json["run_list"].each { |run| chef.add_recipe(run) }
	chef.json.merge!(json)
	end
	end
	end
	end
	end

view raw Vagrantfile.multibox.chef.rb hosted with ❤ by GitHub

Resources

• http://docs.vagrantup.com/v2/multi-machine/index.html 
• http://jbbarth.com/posts/2011-03-20-vagrant-provisioning-with-chefsolo.html 

HipChat is now free!

Just got an email from HipChat - now free for small groups - awesome!

> random tricks you can do with Rails

There are bunch of really good nuggets in this presentation:

• http://www.confreaks.com/videos/889-railsconf2012-ten-things-you-didn-t-know-rails-could-do 
• https://speakerdeck.com/jeg2/10-things-you-didnt-know-rails-could-do