server_remote - A Rails plugin for easily accessing servers

Note: this post is out of date. This tool is now a gem instead of a plugin; read all about it here.

I just released server_remote on github. It is a plugin for connecting to remote servers. Once installed, it provides commands via script/remote:

* remote shell – same as ssh’ing to the server (this is the default command, so it can be called with just remote)

* remote console – executes a script/console on the server

* remote logtail – executes tail -f log/<environment>.log on the server

Configuration is in config/server_remote.yml, and is grouped into profiles.

Here is the output of remote usage:

 Executes commands on a remote server over ssh. Configuration is in:
/Users/tobias/customers/DropZite/restaurantZite/config/server_remote.yml
 
You can override the profile used with -p profile. The default profile is: app
 
Learn more in the readme:
/Users/tobias/customers/DropZite/restaurantZite/vendor/plugins/server_remote/lib/README.textile
 
remote commands are:
 
DEFAULT COMMAND   shell
 
commands          List all 'remote' commands
console           executes remote console
help              Provide help documentation for a command
logtail           executes remote tail -f on the log
shell             executes remote shell
usage             prints usage message
 
For help on a particular command, use 'remote help COMMAND'.

It uses remi’s simplecli gem.





I plan to add other commands as needed, or you can open up Remote::Commands and add your own. Install with: script/plugin install git://github.com/tobias/server_remote.git

Generating a TAGS file from a git hook

Any decent programmer’s text editor provides some form of symbol lookup. With TextMate, the lookup table generation is built in. If you are using vim or emacs, you will need to use an external program to generate the lookup table. This post covers generating the lookup table for emacs (and with some slight modification, vi), and adding a hook to git to regenerate the table after a pull/merge. For more information on using symbol (tag) lookup with emacs or vim, see this wikia entry for vim, or this emacswiki entry for emacs.

ctags

Traditionally, the ctags/etags executable was used to generate lookup tables for vi/emacs respectively. ctag would create a tags file for vi, and etag would create a TAGS file for emacs. Unfortunately, the etags/ctags that ships with MacOS X does not parse ruby code. For that, you will need to use Exuberant Ctags. You can install Exuberant Ctags on MacOS via MacPorts with sudo port install ctags.

git hooks

There are some nice hooks that you can define for git to call after/before it performs certain actions. You can read the full list here. Hooks are defined by placing properly named executables into .git/hooks/ in your repo. The two hooks we care about here are post-merge and post-commit ( Update: the script has been modified to register for the post-checkout hook as well, based on a suggestion from Bryan Liles). The post-merge hook will get called after you do a git pull to merge from another repository, and post-commit is called after a local commit. An important thing to remember about these hooks (and anything else in .git) is that they will not be pushed to the remote repo. So if you develop on multiple machines, or want to share hooks with coworkers, you will need to install them in each local repo.

the kitchen sink

Here is the script I use to handle generating tags from ruby code explicitly, generating tags from a git hook, and installing the git hooks. It really should be two different scripts, but… well… isn’t.

You can read it and see what it does. To have it install the hooks in the current repo, call it with the -i@ option. If you use vim instead of emacs, you will need to remove the @-e option and change TAGS to tags on the ctags call.

Once it is installed, your tags will be regenerated after every commit or pull. You can manually regenerate the tags by calling the script from the repo base directory.

Update: to see a video of this script in action, check out Bryan Liles’ post on smartic.us.

Setting options for Webby

I’m using Webby to manage the static Hand Built Software site, and had a difficult time finding how to set defaults for the webby rake tasks (specifically the deploy:ssh task in this case). Here is what I figured out: you set the values at the top of your project Sitefile like so:

#-*-ruby-*-
SITE.user = 'tobias'
SITE.host = 'handbuiltsoftware.com'
SITE.remote_dir = '/path/to/site/'
 
task :default => :build
 
desc 'deploy the site to the webserver'
task :deploy => [:build, 'deploy:ssh']
 
# EOF

For reference, here are the webby defaults which can be overridden (from lib/webby.rb)

  # call-seq:
#    Webby.site    => struct
#
# Returns a struct containing the configuration parameters for the 
# Webby site. These defaults should be overridden as needed in the
# site specific Rakefile.
#
def self.site
return @site if defined? @site
@site = OpenStruct.new(
:output_dir    => 'output',
:content_dir   => 'content',
:layout_dir    => 'layouts',
:template_dir  => 'templates',
:exclude       => %w(tmp$ bak$ ~$ CVS \.svn),
:page_defaults => {
'layout'     => 'default'
},
:find_by       => 'title',
:base          => nil,
:create_mode   => 'page',
:blog_dir      => 'blog',
:tumblog_dir   => 'tumblog',
 
# Items for running the heel webserver
:use_web_server => true,
:heel_port      => 4331,
 
# Items used to deploy the website
:user       => ENV['USER'] || ENV['USERNAME'],
:host       => 'example.com',
:remote_dir => '/not/a/valid/dir',
:rsync_args => %w(-av),
 
# Global options for HAML and SASS
:haml_options => {},
:sass_options => {},
 
# Options passed to the 'tidy' program when the tidy filter is used
:tidy_options => '-indent -wrap 80',
 
# List of valid URIs (these automatically pass validation)
:valid_uris => [],
 
# Options for coderay processing
:coderay => {
:lang => :ruby,
:line_numbers => nil,
:line_number_start => 1,
:bold_every => 10,
:tab_width => 8
},
 
# Options for graphviz processing
:graphviz => {
:path => nil,
:cmd => 'dot',
:type => 'png'
},
 
# Options for tex2img processing
:tex2img => {
:path => nil,
:type => 'png',
:bg => 'white',
:fg => 'black',
:resolution => '150x150'
},
 
# Options for ultraviolet syntax highlighting
:uv => {
:lang => 'ruby',
:line_numbers => false,
:theme => 'mac_classic'
},
 
# XPath identifiers used by the basepath filter
:xpaths => %w(
/html/head//base[@href]
/html/head//link[@href]
//script[@src]
/html/body[@background]
/html/body//a[@href]
/html/body//object[@data]
/html/body//img[@src]
/html/body//area[@href]
/html/body//form[@action]
/html/body//input[@src]
)
# other possible XPaths to include for base path substitution
#   /html/body//object[@usemap]
#   /html/body//img[@usemap]
#   /html/body//input[@usemap]
)
end

Note: this applies to webby 0.9.3

Stepping off the Rails - adventures with Sinatra (Part 2)

In this episode, our trusty adventurer actually runs his Sinatra app, and deploys it to Passenger on shared hosting. See Part 1 for details of building the application.

Launching locally: two ways to skin a cat

I realized that in Part 1 I did not cover launching the app to hit it from the browser. It’s easy! To launch the app, use ruby app_name.rb. This will fire up a mongrel instance on port 4567 for your viewing pleasure.

Once you have defined your rack configuration (see below), you can also run the app inside rack locally with rackup config.ru to test your rack settings. This will launch rack inside mongrel on port 9292. Thanks to @jnunemaker for the tip. In fact, John’s article is what I used as a basis for my deployment, with a few changes to support file based view templates.

Rack Configuration

To configure the application to run on rack under Passenger, you need to define a rack configuration file. Passenger expects that file to be named config.ru (see Deploying a Rack-based Ruby application in the Passenger documentation). Here is a minimum config.ru to run the application:

require 'rubygems'
require 'sinatra'
 
disable :run
 
require 'urlunwind.rb'
run Sinatra.application

The only real configuration there is disable :run. This prevents Sinatra from starting a mongrel on port 4567 when the app file is required. This configuration would work for an app that uses no file based templates, and does not reference anything in public/. If you run this config with rackup on url_unwind (rackup config.ru), you’ll see errors like:

Errno::ENOENT: No such file or directory - /opt/local/bin/views/index.haml









This is because rack by default sets its base directory to be the path to the called executable (and on my machine, rackup lives in /opt/local/bin/). So we’ll need to explicitly set the paths:

require 'rubygems'
require 'sinatra'
 
disable :run
set :views, File.dirname(__FILE__) + '/views'
set :public, File.dirname(__FILE__) + '/public'
set :app_file, __FILE__
 
require 'urlunwind.rb'
run Sinatra.application

Now the app should run properly with rackup.

Logging

Sinatra uses rack’s built in logging to log requests, and these log messages get printed to standard out. It would be nice to log these to a file, and we can do that as well in config.ru (based on this tip from Chris Schneider). We’ll also need to turn error raising back on, since by default Sinatra swallows errors in production mode:

require 'rubygems'
require 'sinatra'
 
disable :run
set :env, :production
set :raise_errors, true
set :views, File.dirname(__FILE__) + '/views'
set :public, File.dirname(__FILE__) + '/public'
set :app_file, __FILE__
 
log = File.new("log/sinatra.log", "a")
STDOUT.reopen(log)
STDERR.reopen(log)
 
require 'urlunwind.rb'
run Sinatra.application

This allows you to add your own logging messages as well, just puts, print, or p@ whatever you want to log. *Note:* you will need to create the @log/ directory for this to work.

Deploy with Capistrano

I used Capistrano to deploy to Dreamhost, and based my Capfile of off John’s directions here. I modified it a bit to pull from github, and to create the tmp/ and log/ directories if they do not exist. Here is my Capfile:

#-*-ruby-*-
load 'deploy' if respond_to?(:namespace) # cap2 differentiator
 
default_run_options[:pty] = true
 
# be sure to change these
set :user, 'app_user'
set :domain, 'urlunwind.com'
set :application, 'url_unwind'
set :git_path_prefix, "git@github.com/tobias/"
 
# the rest should be good
set :repository, "#{git_path_prefix}#{application}.git"
set :deploy_to, "/home/#{user}/#{domain}"
set :deploy_via, :remote_cache
set :scm, 'git'
set :branch, 'master'
#set :git_shallow_clone, 1
set :scm_verbose, true
set :use_sudo, false
 
server domain, :app, :web
 
namespace :deploy do
task :restart do
run "test -d #{current_path}/tmp || mkdir #{current_path}/tmp"
run "test -d #{current_path}/log || mkdir #{current_path}/log"
run "touch #{current_path}/tmp/restart.txt"
end
end

Since the Sinatra gem is not installed on Dreamhost, I put it in vendor/ within the app. Since it is not a gem, it must be required with the full path to the base .rb file, both in config.ru and in the app file itself (urlunwind.rb in this case). Here is the final config.ru:

require 'rubygems'
require 'vendor/sinatra/lib/sinatra.rb'
 
disable :run
set :env, :production
set :raise_errors, true
set :views, File.dirname(__FILE__) + '/views'
set :public, File.dirname(__FILE__) + '/public'
set :app_file, __FILE__
 
log = File.new("log/sinatra.log", "a")
STDOUT.reopen(log)
STDERR.reopen(log)
 
require 'urlunwind.rb'
run Sinatra.application

Stepping off the Rails - adventures with Sinatra (Part 1)

In this episode, our trusty adventurer leaves the warm and fuzzy confines of Rails, and explores an ‘alternative’ Ruby web framework. Part 1 will cover the app itself, and Part 2 will cover deployment.

I must admit it: I’m one of those folks that came to Ruby because of Rails. And don’t get me wrong – I like Rails, but I frickin’ love Ruby. I’m not a clever person, but Ruby makes me feel clever every day. Ruby is a good friend, and I want to get to know it better outside of its relationship with Rails. I’m also interested in building simple web apps that don’t need the full weight of Rails. So I turned to Sinatra.

For those of you that don’t know all about it, Sinatra is a lightweight Ruby web application framework that takes a different approach than Rails. With Sinatra, you can write an entire web app in one file, with all of your templates inline (you can also have your templates in separate files instead, as we will see in a minute). Sinatra is rackable, and can be deployed in a Rack container, making it straightforward to deploy under Passenger (deployment is covered in Part 2). There are lots of other good intros out there on Sinatra, and I’ll list a few at the bottom of this post.

The Controller

Let’s get started! The sample application here is URLUnwind. You can see it in operation at urlunwind.com, and see the full source on github. It is a simple short URL unwinder, using Net::HTTP to make a HEAD request and then pulling out the ‘Location:’ header.

Sinatra apps are based around one controller file. You can break out your controller actions into separate files, but will need to manually require them into the primary controller. For this app, the controller is urlunwind.rb. Right now, we are just interested in the actions:

get '/' do
haml :index
end
 
post '/unwind' do
@url, @unwound_url, @unwind_error = unwind(params[:url])
haml :index
end
 
get '/unwind.json' do
content_type 'application/json'
 
@url, @unwound_url, @unwind_error = unwind(params[:url])
{ :from_url => @url, :to_url => @unwound_url, :error => @unwind_error }.to_json
end
 
get '/stylesheets/style.css' do
content_type 'text/css'
 
sass :style
end

To define an action, you use a simple DSL. It starts with the method (one of: ‘get’, ‘post’, ‘put’, ‘delete’, where ‘put’ and ‘delete’ are handled with a ‘_method’ parameter on a post), followed by the route that should trigger the action. The routes are searched in order they are defined until a match is found. You can include named parameters (with :symbol) or splat parameters (with *) in the routes as well (see the Sinatra page for examples of this). In a similar fashion to Rails, parameters matched in the route, provided in the post body, or included in the url all end up in a params hash available to your action.

If a request fails to match any route, Sinatra looks for a matching file path in a public/ directory in the same directory as the called executable ($0@) (this path can be overridden). I use that feature to provide a "SASS":http://haml.hamptoncatlin.com/docs/rdoc/classes/Sass.html stylesheet, but I have other stylesheets in @public/stylesheets/ that are served statically.

After the action does it’s business, whatever it returns is given to the client.

Rendering Views

View templates can be defined in four ways: inline on the render call, embedded in the controller, using the template method, or in individual files. I prefer the cleanliness of having a separate file for each view, so used that approach. Sinatra will look for file based views in a views/ directory in the same directory as the called executable ($0@) (this path can also be overridden). You render a template by calling a method for the type of template it is. This app has only two templates: @index.haml and style.sass (rendered with haml :index and sass :style, respectively).

As in Rails, ivars defined in the action are available within the template. You can also pass locals on the render call.

You can override the content-type for a particular action. I do that in the stylesheet and json actions with content_type.

Testing

Sinatra provides helpers for Test::Unit, Test::Spec, and RSpec. Here is a snippet from my Test::Unit tests:

 def test_index
get_it '/'
assert_match /Unwind It/, @response.body
end
 
def test_unwind_with_blank_url
post_it '/unwind', :url => ''
assert_match /Please enter/, @response.body
end

Sinatra provides helpers to call the actions: get_it, post_it, delete_it, put_it. After calling any of these, the response is available in the @respose ivar. The test helpers have a little growing to do – I was only able to assert_match against the body of the response. It would be nice to have access to the assigned ivars as well (this may be changed in edge, I built all of this against the 0.3.2 gem version of Sinatra).

Since Sinatra looks for views relative to the executable, calling the tests with ruby test/urlunwind_test.rb had it looking for test/views/, which did not exist. I linked test/views/ to ../views/, and all was good. You could also probably define a :test environment configuration that set the path to the views as well. I’ll cover overriding the default paths in Part 2, and you can read about configurations on the Sinatra site.

Project Structure

Here is a snap of the project structure:

layout I’ll discuss the purpose of the vendor/ and config.ru entries in Part 2 as part of my deployment discussion.

Resources

There are some good references out there for Sinatra (this list is by no means exhaustive):

* The Sinatra Homepage – In case you didn’t see the dozen links to it above

* The Sinatra Book – Which I discovered after writing this app. Wish I had seen it earlier.

* Chris Schneider’s blog – A good resource of Sinatra tips. Chris also maintains the Sinatra Book.

* #sinatra on irc.freenode.net – Folks there were quick to answer my deployment questions, and prodded me to provide a web service from the app.

* The Sinatra mailing list

* faithfulgeek’s ’10 Minute Web App’ screencast

* Update: Ruby Inside article – Sinatra: 29 Links and Resources For A Quicker, Easier Way to Build Webapps