Just Enough Developed Infrastructure

Rails Syndrome: It works on my PC but I don't know for how long

Rails projects tend to suffer from the 'it works on my PC syndrome'. People seem to struggle with keeping their application environment under control so it can be reproduced on another system. One of the reasons is that rails and ruby evolve fast to new concepts and ideas. While this is clearly a good thing, it does require additional steps to keep a rails project under control. There exist a lot of documentation in books and blogs, but that also suffers from the same problem: the recipe working for version X of rails, might not work anymore in the new version.
This document will try to document how to control it, additionally listing older approaches and indicating that they have become deprecated.
NOTE:

• We assume that you have a working version of ruby and ruby-gems installed
• This document uses unix commands useful for linux or macosx variants, Windows users should use their equivalent
• This is work in progress. But I found it more important to get feedback before finishing the whole project

STEP 0: Create an initial directory structure for our projects let's make the project structure we would like
projects

• project1 ....
• projectX projectX.ENV +gem (where our project gems go) +rails (where our rails code goes)

  export PROJ_NAME=projectX
  export PROJ_PATH=pwd/$PROJ_NAME
  cd $PROJ_NAME

  STEP 1: control your RUBY The first thing to decide is the ruby version you will use. This is usually a step to people tend to skip, they will use whatever ruby that is in the path Lets see where the ruby binary is installed

  RUBY_BINARY=which ruby
  echo $RUBY_BINARY
  RUBY_VERSION=ruby --version
  echo $RUBY_VERSION

  To fix the ruby version you want to use the environment variable RUBY_HOME If this variable is set, everything will be executed from this environment variable

  export RUBY_HOME='/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr'
  echo $RUBY_HOME
  export PATH=$RUBY_HOME/bin:$PATH

  Let's write this variable to a file so that we can set it again everytime we need it

  echo "export RUBY_HOME=$RUBY_HOME" >> ENV.$PROJ_NAME

  STEP 2: control your RAILS Just as we decided which version of ruby, we have to do the same with our rails version Again in most documents you will find a 'gem install rails' installing gems either in the system environment or the per user environment @REF: http://blog.divergentsoftware.com/2007/11/gemhome-is-where-partys-at.html @REF: https://we.riseup.net/rails/subversion-for-rails If you want to protect this project from other gem installs and gem updates we can dedicate a directory for this The path where gem install writes it's gems is in GEM_HOME

  export GEM_HOME=$PROJ_PATH/gem
  mkdir $GEM_HOME

  The path where the are read is in GEM_PATH. We set this to be excluvely to the gems in this directory. So no use of other f.i. system gems

  export GEM_PATH=$GEM_HOME

  Let's write this again to our ENV file

   echo "export GEM_HOME=$GEM_HOME" >> ENV.$PROJ_NAME
  echo "export GEM_PATH=$GEM_PATH" >> ENV.$PROJ_NAME

  Let get the version we want

  RAILS_VERSION="2.1"
  gem install rails -v $RAILS_VERSION

  let's check if the files are indeed installed in the right place

  ls $GEMHOME/

  To check the path of the installed versions

  gem list -d

  STEP 3: let's setup a subversion to check our code in @REF: https://wincent.com/wiki/Checking_a_new_Rails_application_into_an_existing_Subversion_repository Chances are you have one allready but, this should show the basic setup of setting up yourself The place where you store your repository = REPO_PATH

  REPO_PATH=$HOME/subversion-repository
  #REPO_URL="file://"$REPO_PATH/$PROJ_NAME
  REPO_URL="svn://localhost/"$PROJ_NAME

  Go within the repository path

  cd $REPO_PATH

  and create the svn repository for our project = $PROJ_NAME

  svnadmin create ./$PROJ_NAME

  Add some userT with password passwordT

  export SVN_USER="userT"
  export SVN_PASSWORD="passwordT"
  echo "$SVN_USER = $SVN_PASSWORD" >> $REPO_PATH/$PROJ_NAME/conf/passwd
  echo "auth-access = write" >> $REPO_PATH/$PROJ_NAME/conf/svnserve.conf
  echo "password-db = passwd" >> $REPO_PATH/$PROJ_NAME/conf/svnserve.conf
  echo "realm = Project $PROJ_NAME" >> $REPO_PATH/$PROJ_NAME/conf/svnserve.conf

  Start the listener to listen on the network

  svnserve -d -r $REPO_PATH

  This command will create a trunk, tags, branches directory and commit the version of the project

  svn --username=$SVN_USER --password=$SVN_PASSWORD mkdir --message="standard project start" "$REPO_URL/trunk" "$REPO_URL/branches" "$REPO_URL/tags"

  STEP 4: Create our first rails skeleton for the application Let's go to the our project directory we created in STEP 0

  cd $PROJ_PATH

  Remember to set your RUBY_HOME, GEM_HOME, GEM_PATH We can use our project ENV to set things up again

  . ENV.$PROJ_NAME

  Within our PROJ_PATH we will checkout the trunk to the rails directory Remember that REPO_URL includes PROJ_NAME (we assume that each project will have it's own trunk, tags, release)

  svn checkout "$REPO_URL/trunk" rails
  cd rails

  Set the database type (DB_TYPE) we would like to use

  DB_TYPE=mysql
  rails . -d $DB_TYPE

  Take the generated and add it to the files to check in

  svn add --force .

  BUT: we don't want everything to be checked in so we need to set a set of ignores @REF = Addison Wesley - The Rails Way, 2008 : page 811
  We don't want to have .DS_Store files to be checked in

  svn propset svn:ignore -R ".DS_Store" . --force

  We don't want to have the log files to be checked in

  svn revert log/
  svn propset svn:ignore ".log" log

  We don't want to have generated documentation to be checked in

  svn propset svn:ignore "" doc

  @REF = http://www.sepcot.com/blog/2008/04/svn-rake-tasks-1 We don't want to have generated documentation to be checked in

  svn propset svn:ignore "" doc/api
  svn propset svn:ignore "" doc/app

  @REF = Addison Wesley - The Rails Way, 2008 : page 811

  svn propset svn:ignore "" tmp
  svn propset svn:ignore "" tmp/sessions tmp/cache tmp/sockets
  svn propset svn:ignore ".q" db/ --force
  svn propset svn:ignore "*.db" db/ --force

  We don't want to have developers overwrite eachother or worse production databases definition

  Instead we create a template file that can be copied but it should never be checked in

  cp config/database.yml config/database.yml.template
  svn add config/database.yml.template
  svn revert config/database.yml
  svn propset svn:ignore "database.yml" config

  We can set some scripts as executable so that they can be started using GUI tools This goes for all dispatchers

  svn propset svn:executable "" public/dispatch.

  and for all scripts inside the script directory

  svn propset svn:executable "" find script -type f | grep -v '.svn'

  Make sure we set the correct native eol style
  @REF = Pragmatic Programmers - Deploying Rails Application : p 28

  svn propset svn:eol-style native public/dispatch.

  @REF = Pragmatic Programmers - Deploying Rails Application : p 28

  svn propset svn:ignore "engine_files" public

  @REF = http://blog.teksol.info/2006/03/09/subversion-primer-for-rails-projects We want to make sure that the default page is not shown, so that it never looks like a default installation

  svn revert public/index.html

  @REF = http://blog.teksol.info/2006/03/09/subversion-primer-for-rails-projects

  This URL proposes to ignore the schema.rb because it is generated but the next one opposes this view

  svn propset svn:ignore "schema.rb" db

  
  @REF = Professional Ruby on Rails : p.45 @REF = http://books.google.be/books?id=aB4B13xGEv4C&pg=PA45&lpg=PA45&dq=subversion+schema.rb&source=bl&ots=fec_xmHyMa&sig=6PGXK4irrfzNYHxplVIJ2syoXgQ&hl=en&ei=ke64SfThCsm4-QaRm6HRBA&sa=X&oi=book_result&resnum=8&ct=result#PPA50,M1 They propose to insert the schema.rb within subversion, developers can start without applying all migrations and it might help
  Let's see if we got all files marked and with correct status

  svn status
  svn commit -m "initial rails checkin with ignores"

  Step 5: Freezing rails To handle this, rails seem to have moved through different solutions Option 1 (DEPRECATED): define the vendor/rails directory in your project as an SVN External, but the rails repository moved to GIT and so the latest versions can't be used @REF = http://blog.teksol.info/2006/03/09/subversion-primer-for-rails-projects @REF = Wrox - Professional Ruby on Rails - page 55

  #svn propset svn:externals "rails http://dev.rubyonrails.org/svn/rails/branches/ 2-2-stable" vendor
  #svn update vendor
  #svn commit -m "vendor integration"

  Option 2 (BACKWARDS COMPATIBLE): to solve this, there exist a gem called 'Piston' that allows you to handle git repositories and integrate them into a subversion repository @REF = http://blog.logeek.fr/2008/1/4/how-to-use-piston-to-ease-your-upgrades @REF = http://piston.rubyforge.org/
  Option 3 (RECENT VERSIONS) : the rails seems to have a command option to tell it to freeze @REF = Wrox - Professonal Ruby on Rails - p 56

  #rails . -f

  Option 4 (RECENT VERSIONS): rails now includes a rake task to freeze it into your project directory This will download the rails tar.gz from the edge site and we can tell to use a specific version using RELEASE= @REF = http://support.tigertech.net/freeze-rails @REF = http://codeclimber.blogspot.com/2008/09/freeze-those-rails.html

  rake rails:freeze:edge RELEASE=$RAILS_VERSION
  svn add vendor/rails
  svn commit -m "frozen rails to version $RAILS_VERSION"

  Should we set RAILS_HOME after freezing? @REF = http://snippets.dzone.com/posts/show/4730

  PATH=pwd/vendor/rails/railties/bin:$PATH

  export PATH

  
  Step 6: Freezing rails gems Recent versions only: Together with the rake task for rails there comes a rake task to freeze gems.
  If you have frozen rails in Step 5, there is no additional need to do this. Use this only if you want to freeze the rails gems and NOT rails itself.

  rake rails:freeze:gems
  svn add vendor/gems
  svn commit -m "frozen rails gems $RAILS_VERSION"

  Step 7: Freezing non-rails gems Option 1 (DEPRECATED): the first solution that appeared was to unpack of a gem in vendor , copy the files to the correct place and remove the unpack
  This is a tedious job because you have to go into the structure and move files around
  @REF = http://www.vaporbase.com/postings/Using_Subversion_and_Rails

  # cd vendor
  gem unpack gemname
  cp -Rf gemname-n.n.n/lib/* .
  cp -Rf gemname-n.n.n/MIT-LICENSE LICENSE-gemname
  cp -Rf gemname-n.n.n/README README-gemname
  svn add aaaa bbbb cccc dddd.rb LICENSE-gemname README-gemname
  svn propset version "n.n.n (Gem)" gemname.rb
  rm -Rf gemname-n.n.n
  cd ..

  Option 2 (CUSTOM): instead of copying the files, the gems are left unpacked in the vendor directory and the then the config.load_paths is adapted to include dynamically include the libs of all subdirs @REF = Oreilly - Advanced Rails - Large Projects p. 312

  # cd vendor
  mkdir gems
  cd gems
  gem unpack gemname
  cat ../../config/environment.rb |\
  sed -e 's@# config.load_paths += %W( #{RAILS_ROOT}/extras )@config.load_paths + = Dir["#{RAILS_ROOT}/vendor/gems/**"].map do |dir|\n\tFile.directory?(lib = "#{dir}/lib") ? lib : dir\nend@' > ../../config/environment.rb
  cd ..
  cd ..

  Option 3 (CUSTOM) : do the same unpack of the gem in the vendors directory but use the GEM_PATH to include the paths where the gems are unpacked @REF = http://errtheblog.com/posts/50-vendor-everything

  # cd vendor
  mkdir gems
  cd gems
  gem unpack gemname
  cd ..
  cd ..
  echo "ENV['GEM_PATH'] = File.join(RAILS_ROOT, 'vendor', 'gems')" >> config/boot.rb

  Option 4 (BACKWARDS COMPATIBLE) : gemsonrails @REF = http://gemsonrails.rubyforge.org/
  Option 5 (RECENT) : similar to the rails freeze, there is now a rake task that allows you to unpack the gems your project needs The first step is to list the required gems and their version and optional their source to the config/environment.rb file @REF = http://www.pathf.com/blogs/2009/03/keeping-up-with-the-joneses-keeping-rails-and-its-extensions-up-to-date/

  export GEM_NAME="hpricot"
  export GEM_VERSION="0.6.0"
  cat config/environment.rb | sed -e 's@# config.gem "bj"@config.gem "$GEM_NAME", :lib => "$GEM_NAME", :version => "$GEM_VERSION"@' > config/environment.rb.$$
  mv config/environment.rb.$$ config/environment.rb

  To use the rake task unpack, we first need to install the gem then we can execute the gem installation. Note: because we still have the GEMS_PATH and GEM_HOME set, the install will be specific to our project

  rake gems:install

  Now that it is installed we can unpack the gems, this will go in the vendor/gems directory

  rake gems:unpack:dependencies
  svn add vendor/gems
  svn commit -m "frozen additional gem $GEM_NAME to version $GEM_VERSION"

  To list the gems state there is a rake task (this only seems to work ok in Rails version 2.3.1 (otherwise you seem to get random nil.dependencies errors)

  rake gems

  Option 6: warbler
  @REF = http://caldersphere.rubyforge.org/warbler/
  If you need to transfer your application to another system, another approach would be to create a war file of your application and run your rails application with jruby.
  Option 7: gems2rpm
  @REF =http://www.fooplanet.com/projects/gem2rpm/
  Instead of putting your gems inside your project, it might be more useful to package your gems as an rpm. This is exactly what the gem2rpm script does. It only works with non-native gems
  Step 8: Freezing native gems
  the procedures from Step 7 will work but for gems requiring native when you unpack a gem, it will only unpack the ruby code that does not need to be compiled @REF = http://www.theagiledeveloper.com/localizing-gems-with-native-extensions.html @REF = http://agilewebdevelopment.com/plugins/gems_with_architecture @REF = http://www.linuxtopia.org/online_books/programming_books/ruby_tutorial/Extending_Ruby_Creating_a_Makefile_with_extconf.rb.html
  Option 1: works but needs recompile on other architectures

  rake gems:build

  Option 2: puts the binaries in a place where you want to
  We prepare a directory called native-lib where we put all the compiled stuff from the native gems

  mkdir native-lib
  ARCHDIR=pwd/native-lib/ruby -e "puts RUBY_PLATFORM"
  export ARCHDIR

  Next we search these directories for extconf.rb Then execute ruby extconf.rb , which will create a makefile, we do a make Then we install this to the native-lib directory we created, otherwise it would install itself in the system ruby directory

  find vendor/gems/ -name "extconf.rb" | xargs dirname | xargs -I {}  ksh 'cd {} ; ruby extconf.rb; make; make RUBYARCHDIR=$ARCHDIR install'

  We add the native-lib directory to the load_paths in config.environment so that these can be found when executing

  cat config/environment.rb | sed -e 's@# config.load_paths += %W( #{RAILS_ROOT}/extras )@ config.load_paths += %W( #{RAILS_ROOT}/native-lib/#{RUBY_PLATFORM} )@' > config/environment.rb.$$
  mv config/environment.rb.$$ config/environment.rb

  Now you can argue that we file is generated , so we might not need to check in But on some systems there isn't a compiler available (on Windows, no standard C compiler is installed f.i. , the same goes for stripped production installations)

  svn add $ARCHDIR
  svn commit -m "commited native libs"

  Step 9: Checking it all out on another system
  • Create the project structure (see STEP 0)
  • Set the ENV's (RUBY_HOME, GEM_HOME, GEM_PATH) (see STEP1)
  • Check the project out in the rails directory (see STEP )
  • Copy the database template
  • We need to repeat the svn ignores stuff (see STEP )
  • Generate the cross compiled stuff (see STEP 8). Otherwise you get rails to ask you to install the gem.
  @REF = http://tenderlovemaking.com/2008/11/21/cross-compiling-ruby-gems-for-win32/
  You can create a custom rake task for doing all this stuff. If you create a file in PROJECTDIR/rails/lib/tasks

  namespace :svn do
    desc "Sets the correct ignores and other stuff for a correct checkin of rails project in an svn repository"
    task :propset do
            system 'svn propset svn:ignore -R ".DS_Store" . --force'
            system 'svn propset svn:ignore "*.log" log/'
            system 'svn propset svn:ignore "*.db" db/'
            system 'svn propset svn:ignore "*.sqlite3" db/'
            system 'svn propset svn:ignore "database.yml" config/'
            system 'svn propset svn:ignore "*" doc/'
            system 'svn propset svn:ignore "*" doc/api'
            system 'svn propset svn:ignore "*" doc/app'
            system 'svn propset svn:ignore "*" tmp/'
            system 'svn propset svn:ignore "*" tmp/sessions tmp/cache tmp/sockets'
            system 'svn propset svn:executable "*" public/dispatch.*'
            system 'svn propset svn:eol-style native public/dispatch.*'
            system 'svn propset svn:ignore "engine_files" public/ '
            system 'svn propset svn:ignore index.html public/'
    end
  end

  then you can repeat this tasks with:

  $ rake -T |grep svn
  rake svn:propset                          # Sets the correct ignores and other stuff for a correct checkin of rails project in an svn repository

  Step 10: Do you first scaffold. If you supply the -svn , the files generated will automatically be add for the next commit

  ./script/generate scaffold -svn
  svn commit -m "first scaffold"

  TODO : Freeze plugins http://www.vaporbase.com/postings/Using_Subversion_and_Rails

  TODO : how to update to a new version of rails : rake rails:unfreeze, rake rails:update (seems to delete the who vendor dir?)

  rake rails:update -v 2.2.2

  @REF = http://stackoverflow.com/questions/313176/migrate-from-rails-211-to-222

  TODO : how to update to a new version of a gem

  TODO : show how you need to set this to work in a production environment (f.i. mod_rails)

  
  

  TODO : describe piston and gemsonrails more in detail

  TODO : show how you need to set this to work in a production environment (f.i. mod_rails)

blog comments powered by Disqus

---