[Home] [Blog] [Contact] - [Talks] [Bio] [Customers]
twitter linkedin youtube github rss

Patrick Debois

Rails Syndrome: It works on my PC but I don't know for how long

Rails projects tend to suffer from the 'it works on my PC syndrome'. People seem to struggle with keeping their application environment under control so it can be reproduced on another system. One of the reasons is that rails and ruby evolve fast to new concepts and ideas. While this is clearly a good thing, it does require additional steps to keep a rails project under control. There exist a lot of documentation in books and blogs, but that also suffers from the same problem: the recipe working for version X of rails, might not work anymore in the new version.
This document will try to document how to control it, additionally listing older approaches and indicating that they have become deprecated.
NOTE:
  • We assume that you have a working version of ruby and ruby-gems installed
  • This document uses unix commands useful for linux or macosx variants, Windows users should use their equivalent
  • This is work in progress. But I found it more important to get feedback before finishing the whole project
STEP 0: Create an initial directory structure for our projects let's make the project structure we would like
projects + project1 .... + projectX projectX.ENV +gem (where our project gems go) +rails (where our rails code goes) 
export PROJ_NAME=projectX
export PROJ_PATH=`pwd`/$PROJ_NAME
cd $PROJ_NAME
STEP 1: control your RUBY The first thing to decide is the ruby version you will use. This is usually a step to people tend to skip, they will use whatever ruby that is in the path Lets see where the ruby binary is installed 
RUBY_BINARY=`which ruby`
echo $RUBY_BINARY
RUBY_VERSION=`ruby --version`
echo $RUBY_VERSION
To fix the ruby version you want to use the environment variable RUBY_HOME If this variable is set, everything will be executed from this environment variable 
export RUBY_HOME='/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr'
echo $RUBY_HOME
export PATH=$RUBY_HOME/bin:$PATH
Let's write this variable to a file so that we can set it again everytime we need it 
echo "export RUBY_HOME=$RUBY_HOME" >> ENV.$PROJ_NAME
STEP 2: control your RAILS Just as we decided which version of ruby, we have to do the same with our rails version Again in most documents you will find a 'gem install rails' installing gems either in the system environment or the per user environment @REF: http://blog.divergentsoftware.com/2007/11/gemhome-is-where-partys-at.html @REF: https://we.riseup.net/rails/subversion-for-rails If you want to protect this project from other gem installs and gem updates we can dedicate a directory for this The path where gem install writes it's gems is in GEM_HOME 
export GEM_HOME=$PROJ_PATH/gem
mkdir $GEM_HOME
The path where the are read is in GEM_PATH. We set this to be excluvely to the gems in this directory. So no use of other f.i. system gems 
export GEM_PATH=$GEM_HOME
Let's write this again to our ENV file 
 echo "export GEM_HOME=$GEM_HOME" >> ENV.$PROJ_NAME
 echo "export GEM_PATH=$GEM_PATH" >> ENV.$PROJ_NAME
Let get the version we want 
RAILS_VERSION="2.1"
gem install rails -v $RAILS_VERSION
let's check if the files are indeed installed in the right place 
ls $GEMHOME/
To check the path of the installed versions 
gem list -d
STEP 3: let's setup a subversion to check our code in @REF: https://wincent.com/wiki/Checking_a_new_Rails_application_into_an_existing_Subversion_repository Chances are you have one allready but, this should show the basic setup of setting up yourself The place where you store your repository = REPO_PATH 
REPO_PATH=$HOME/subversion-repository
#REPO_URL="file://"$REPO_PATH/$PROJ_NAME
REPO_URL="svn://localhost/"$PROJ_NAME
Go within the repository path 
cd $REPO_PATH
and create the svn repository for our project = $PROJ_NAME 
svnadmin create ./$PROJ_NAME
Add some userT with password passwordT 
export SVN_USER="userT"
export SVN_PASSWORD="passwordT"
echo "$SVN_USER = $SVN_PASSWORD" >> $REPO_PATH/$PROJ_NAME/conf/passwd
echo "auth-access = write" >> $REPO_PATH/$PROJ_NAME/conf/svnserve.conf
echo "password-db = passwd" >> $REPO_PATH/$PROJ_NAME/conf/svnserve.conf
echo "realm = Project $PROJ_NAME" >> $REPO_PATH/$PROJ_NAME/conf/svnserve.conf
Start the listener to listen on the network 
svnserve -d -r $REPO_PATH
This command will create a trunk, tags, branches directory and commit the version of the project 
svn --username=$SVN_USER --password=$SVN_PASSWORD mkdir --message="standard project start" "$REPO_URL/trunk" "$REPO_URL/branches" "$REPO_URL/tags"
STEP 4: Create our first rails skeleton for the application Let's go to the our project directory we created in STEP 0 
cd $PROJ_PATH
Remember to set your RUBY_HOME, GEM_HOME, GEM_PATH We can use our project ENV to set things up again 
. ENV.$PROJ_NAME
Within our PROJ_PATH we will checkout the trunk to the rails directory Remember that REPO_URL includes PROJ_NAME (we assume that each project will have it's own trunk, tags, release) 
svn checkout "$REPO_URL/trunk" rails
cd rails
Set the database type (DB_TYPE) we would like to use 
DB_TYPE=mysql
rails . -d $DB_TYPE
Take the generated and add it to the files to check in 
svn add --force .
BUT: we don't want everything to be checked in so we need to set a set of ignores @REF = Addison Wesley - The Rails Way, 2008 : page 811
We don't want to have .DS_Store files to be checked in 
svn propset svn:ignore -R ".DS_Store" . --force
We don't want to have the log files to be checked in 
svn revert log/*
svn propset svn:ignore "*.log" log
We don't want to have generated documentation to be checked in 
svn propset svn:ignore "*" doc
@REF = http://www.sepcot.com/blog/2008/04/svn-rake-tasks-1 We don't want to have generated documentation to be checked in 
svn propset svn:ignore "*" doc/api
svn propset svn:ignore "*" doc/app
@REF = Addison Wesley - The Rails Way, 2008 : page 811 
svn propset svn:ignore "*" tmp
svn propset svn:ignore "*" tmp/sessions tmp/cache tmp/sockets
svn propset svn:ignore "*.q" db/ --force
svn propset svn:ignore "*.db" db/ --force
# We don't want to have developers overwrite eachother or worse production databases definition # Instead we create a template file that can be copied but it should never be checked in 
cp config/database.yml config/database.yml.template
svn add config/database.yml.template
svn revert config/database.yml
svn propset svn:ignore "database.yml" config
We can set some scripts as executable so that they can be started using GUI tools This goes for all dispatchers 
svn propset svn:executable "*" public/dispatch.*
and for all scripts inside the script directory 
svn propset svn:executable "*" `find script -type f | grep -v '.svn'`
Make sure we set the correct native eol style
@REF = Pragmatic Programmers - Deploying Rails Application : p 28 
svn propset svn:eol-style native public/dispatch.*
@REF = Pragmatic Programmers - Deploying Rails Application : p 28 
svn propset svn:ignore "engine_files" public
@REF = http://blog.teksol.info/2006/03/09/subversion-primer-for-rails-projects We want to make sure that the default page is not shown, so that it never looks like a default installation 
svn revert public/index.html
@REF = http://blog.teksol.info/2006/03/09/subversion-primer-for-rails-projects # This URL proposes to ignore the schema.rb because it is generated but the next one opposes this view # svn propset svn:ignore "schema.rb" db
@REF = Professional Ruby on Rails : p.45 @REF = http://books.google.be/books?id=aB4B13xGEv4C&pg=PA45&lpg=PA45&dq=subversion+schema.rb&source=bl&ots=fec_xmHyMa&sig=6PGXK4irrfzNYHxplVIJ2syoXgQ&hl=en&ei=ke64SfThCsm4-QaRm6HRBA&sa=X&oi=book_result&resnum=8&ct=result#PPA50,M1 They propose to insert the schema.rb within subversion, developers can start without applying all migrations and it might help
Let's see if we got all files marked and with correct status 
svn status
svn commit -m "initial rails checkin with ignores"
Step 5: Freezing rails To handle this, rails seem to have moved through different solutions Option 1 (DEPRECATED): define the vendor/rails directory in your project as an SVN External, but the rails repository moved to GIT and so the latest versions can't be used @REF = http://blog.teksol.info/2006/03/09/subversion-primer-for-rails-projects @REF = Wrox - Professional Ruby on Rails - page 55 
#svn propset svn:externals "rails http://dev.rubyonrails.org/svn/rails/branches/ 2-2-stable" vendor
#svn update vendor
#svn commit -m "vendor integration"
Option 2 (BACKWARDS COMPATIBLE): to solve this, there exist a gem called 'Piston' that allows you to handle git repositories and integrate them into a subversion repository @REF = http://blog.logeek.fr/2008/1/4/how-to-use-piston-to-ease-your-upgrades @REF = http://piston.rubyforge.org/
Option 3 (RECENT VERSIONS) : the rails seems to have a command option to tell it to freeze @REF = Wrox - Professonal Ruby on Rails - p 56 
#rails . -f
Option 4 (RECENT VERSIONS): rails now includes a rake task to freeze it into your project directory This will download the rails tar.gz from the edge site and we can tell to use a specific version using RELEASE= @REF = http://support.tigertech.net/freeze-rails @REF = http://codeclimber.blogspot.com/2008/09/freeze-those-rails.html 
rake rails:freeze:edge RELEASE=$RAILS_VERSION
svn add vendor/rails
svn commit -m "frozen rails to version $RAILS_VERSION"
Should we set RAILS_HOME after freezing? @REF = http://snippets.dzone.com/posts/show/4730 # PATH=`pwd`/vendor/rails/railties/bin:$PATH # export PATH
Step 6: Freezing rails gems Recent versions only: Together with the rake task for rails there comes a rake task to freeze gems.
If you have frozen rails in Step 5, there is no additional need to do this. Use this only if you want to freeze the rails gems and NOT rails itself. 
rake rails:freeze:gems
svn add vendor/gems
svn commit -m "frozen rails gems $RAILS_VERSION"
Step 7: Freezing non-rails gems Option 1 (DEPRECATED): the first solution that appeared was to unpack of a gem in vendor , copy the files to the correct place and remove the unpack
This is a tedious job because you have to go into the structure and move files around
@REF = http://www.vaporbase.com/postings/Using_Subversion_and_Rails 
# cd vendor
# gem unpack gemname
# cp -Rf gemname-n.n.n/lib/* .
# cp -Rf gemname-n.n.n/MIT-LICENSE LICENSE-gemname
# cp -Rf gemname-n.n.n/README README-gemname
# svn add aaaa bbbb cccc dddd.rb LICENSE-gemname README-gemname
# svn propset version "n.n.n (Gem)" gemname.rb
# rm -Rf gemname-n.n.n
# cd ..
Option 2 (CUSTOM): instead of copying the files, the gems are left unpacked in the vendor directory and the then the config.load_paths is adapted to include dynamically include the libs of all subdirs @REF = Oreilly - Advanced Rails - Large Projects p. 312 
# cd vendor
# mkdir gems
# cd gems
# gem unpack gemname
# cat ../../config/environment.rb |\
# sed -e 's@# config.load_paths += %W( #{RAILS_ROOT}/extras )@config.load_paths + = Dir["#{RAILS_ROOT}/vendor/gems/**"].map do |dir|\n\tFile.directory?(lib = "#{dir}/lib") ? lib : dir\nend@' > ../../config/environment.rb
# cd ..
# cd ..
Option 3 (CUSTOM) : do the same unpack of the gem in the vendors directory but use the GEM_PATH to include the paths where the gems are unpacked @REF = http://errtheblog.com/posts/50-vendor-everything 
# cd vendor
# mkdir gems
# cd gems
# gem unpack gemname
# cd ..
# cd ..
# echo "ENV['GEM_PATH'] = File.join(RAILS_ROOT, 'vendor', 'gems')" >> config/boot.rb
Option 4 (BACKWARDS COMPATIBLE) : gemsonrails @REF = http://gemsonrails.rubyforge.org/
Option 5 (RECENT) : similar to the rails freeze, there is now a rake task that allows you to unpack the gems your project needs The first step is to list the required gems and their version and optional their source to the config/environment.rb file @REF = http://www.pathf.com/blogs/2009/03/keeping-up-with-the-joneses-keeping-rails-and-its-extensions-up-to-date/ 
export GEM_NAME="hpricot"
export GEM_VERSION="0.6.0"
cat config/environment.rb | sed -e 's@# config.gem "bj"@config.gem "$GEM_NAME", :lib => "$GEM_NAME", :version => "$GEM_VERSION"@' > config/environment.rb.$$
mv config/environment.rb.$$ config/environment.rb
To use the rake task unpack, we first need to install the gem then we can execute the gem installation. Note: because we still have the GEMS_PATH and GEM_HOME set, the install will be specific to our project 
rake gems:install
Now that it is installed we can unpack the gems, this will go in the vendor/gems directory 
rake gems:unpack:dependencies
svn add vendor/gems
svn commit -m "frozen additional gem $GEM_NAME to version $GEM_VERSION"
To list the gems state there is a rake task (this only seems to work ok in Rails version 2.3.1 (otherwise you seem to get random nil.dependencies errors) 
rake gems
Option 6: warbler
@REF = http://caldersphere.rubyforge.org/warbler/
If you need to transfer your application to another system, another approach would be to create a war file of your application and run your rails application with jruby.
Option 7: gems2rpm
@REF =http://www.fooplanet.com/projects/gem2rpm/
Instead of putting your gems inside your project, it might be more useful to package your gems as an rpm. This is exactly what the gem2rpm script does. It only works with non-native gems
Step 8: Freezing native gems
the procedures from Step 7 will work but for gems requiring native when you unpack a gem, it will only unpack the ruby code that does not need to be compiled @REF = http://www.theagiledeveloper.com/localizing-gems-with-native-extensions.html @REF = http://agilewebdevelopment.com/plugins/gems_with_architecture @REF = http://www.linuxtopia.org/online_books/programming_books/ruby_tutorial/Extending_Ruby_Creating_a_Makefile_with_extconf.rb.html
Option 1: works but needs recompile on other architectures 
rake gems:build
Option 2: puts the binaries in a place where you want to
We prepare a directory called native-lib where we put all the compiled stuff from the native gems 
mkdir native-lib
ARCHDIR=`pwd`/native-lib/`ruby -e "puts RUBY_PLATFORM"`
export ARCHDIR
Next we search these directories for extconf.rb Then execute ruby extconf.rb , which will create a makefile, we do a make Then we install this to the native-lib directory we created, otherwise it would install itself in the system ruby directory 
find vendor/gems/ -name "extconf.rb" | xargs dirname | xargs -I {}  ksh 'cd {} ; ruby extconf.rb; make; make RUBYARCHDIR=$ARCHDIR install'
We add the native-lib directory to the load_paths in config.environment so that these can be found when executing 
cat config/environment.rb | sed -e 's@# config.load_paths += %W( #{RAILS_ROOT}/extras )@ config.load_paths += %W( #{RAILS_ROOT}/native-lib/#{RUBY_PLATFORM} )@' > config/environment.rb.$$
mv config/environment.rb.$$ config/environment.rb
Now you can argue that we file is generated , so we might not need to check in But on some systems there isn't a compiler available (on Windows, no standard C compiler is installed f.i. , the same goes for stripped production installations) 
svn add $ARCHDIR
svn commit -m "commited native libs"
Step 9: Checking it all out on another system
  • Create the project structure (see STEP 0)
  • Set the ENV's (RUBY_HOME, GEM_HOME, GEM_PATH) (see STEP1)
  • Check the project out in the rails directory (see STEP )
  • Copy the database template
  • We need to repeat the svn ignores stuff (see STEP )
  • Generate the cross compiled stuff (see STEP 8). Otherwise you get rails to ask you to install the gem.
@REF = http://tenderlovemaking.com/2008/11/21/cross-compiling-ruby-gems-for-win32/
You can create a custom rake task for doing all this stuff. If you create a file in PROJECTDIR/rails/lib/tasks 
namespace :svn do
        desc "Sets the correct ignores and other stuff for a correct checkin of rails project in an svn repository"
        task :propset do
                system 'svn propset svn:ignore -R ".DS_Store" . --force'
                system 'svn propset svn:ignore "*.log" log/'
                system 'svn propset svn:ignore "*.db" db/'
                system 'svn propset svn:ignore "*.sqlite3" db/'
                system 'svn propset svn:ignore "database.yml" config/'
                system 'svn propset svn:ignore "*" doc/'
                system 'svn propset svn:ignore "*" doc/api'
                system 'svn propset svn:ignore "*" doc/app'
                system 'svn propset svn:ignore "*" tmp/'
                system 'svn propset svn:ignore "*" tmp/sessions tmp/cache tmp/sockets'
                system 'svn propset svn:executable "*" public/dispatch.*'
                system 'svn propset svn:eol-style native public/dispatch.*'
                system 'svn propset svn:ignore "engine_files" public/ '
                system 'svn propset svn:ignore index.html public/'
        end
end
then you can repeat this tasks with: 
$ rake -T |grep svn
rake svn:propset                          # Sets the correct ignores and other stuff for a correct checkin of rails project in an svn repository
Step 10: Do you first scaffold. If you supply the -svn , the files generated will automatically be add for the next commit 
./script/generate scaffold -svn
svn commit -m "first scaffold"
# TODO : Freeze plugins http://www.vaporbase.com/postings/Using_Subversion_and_Rails # TODO : how to update to a new version of rails : rake rails:unfreeze, rake rails:update (seems to delete the who vendor dir?) #        rake rails:update -v 2.2.2 #        @REF = http://stackoverflow.com/questions/313176/migrate-from-rails-211-to-222 # TODO : how to update to a new version of a gem # TODO : show how you need to set this to work in a production environment (f.i. mod_rails)
# TODO : describe piston and gemsonrails more in detail # TODO : show how you need to set this to work in a production environment (f.i. mod_rails)