This is the first part of the Rails boots up process, which is based on this book Dive Deep Rails.

Development environment: rbenv and rvm

Starting Rails

In a local Rails development environment and on a Rails application folder, Rails application is booted up with:

rails s

These are the output of the above command:

=> Booting WEBrick
=> Rails 4.2.0 application starting in development on http://localhost:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
[2015-06-21 15:36:47] INFO  WEBrick 1.3.1
[2015-06-21 15:36:47] INFO  ruby 2.2.1 (2015-02-26) [x86_64-linux]
[2015-06-21 15:36:47] INFO  WEBrick::HTTPServer#start: pid=6943 port=3000

So how does this command work? On any UNIX system, these executable files like rails lives in a specific folder. This folder is often included in the PATH environment variable. To find where a command (executable script) live, using which command:

which rails

This will find the place of the rails command in the user's PATH. With non Ruby version control system, the output will look something like this:

~/.gem/ruby/2.2.1/bin/rails

With rvm for Ruby version control, here's the output:

/home/hatt/.rvm/gems/ruby-2.1.5/bin/rails

With rbenv, which rails show the shim path:

/home/hatt/.rbenv/shims/rails

In Ruby world, the original executables or so-call "binaries" of a Ruby programs are usually wrapped within scripts and they are called binstubs. Binstub's purpose is to prepare the environment before dispatching the call to the original executable. To understand more about binstub, read understanding binstub.

So, the which rails command actually find the binstub of the real executable rails. With rvm or non Ruby version control systems, the rails binstub is the one that is generated by Rubygem. But with rbenv systems, it is generated by rbenv itself which is written in bash shell command.

To find the binstub that Rubygem generated for all system, use this command:

rbenv exec gem environment | grep bin

Result:

  - RUBY EXECUTABLE: ~/.rbenv/versions/2.2.1/bin/ruby
  - EXECUTABLE DIRECTORY: ~/.rbenv/versions/2.2.1/bin
     - ~/.rbenv/versions/2.2.1/bin
     - ~/.rbenv/plugins/ruby-build/bin
     - ~/.rbenv/plugins/ruby-build/bin
     - ~/.rbenv/bin
     - ~/.rbenv/plugins/ruby-build/bin
     - ~/.rbenv/bin
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /usr/bin
     - /sbin
     - /bin

The binstubs can be found from directories listed in EXECUTABLE DIRECTORY.

In my local development environment (using rbenv), rails binstub live at:

/home/hatt/.rbenv/versions/2.2.1/bin/rails

(binstub path)/bin/rails

Let's open and see the content of this file:

#!/home/hachan/.rbenv/versions/2.2.1/bin/ruby

#"/usr/bin/env ruby" for non Ruby version control system.

#/usr/bin/env ruby_executable_hooks for rvm system.
#
# This file was generated by RubyGems.
#
# The application 'railties' is installed as part of a gem, and
# this file is here to facilitate running it.
#

require 'rubygems'

version = ">= 0"

if ARGV.first
  str = ARGV.first
  str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
  if str =~ /\A_(.*)_\z/ and Gem::Version.correct?($1) then
    version = $1
    ARGV.shift
  end
end

gem 'railties', version
load Gem.bin_path('railties', 'rails', version)

This file is an executable and uses the shebang !# to parse its remain content through a Ruby interpreter. With rbenv system, ruby interpreter is located at /home/hachan/.rbenv/versions/{ruby-version}/bin/ruby and with non version control system, ruby interpreter is located at /usr/bin/env ruby. Without this line, it would be interpreted as whatever default shell scripting language, which probably going to be Bash.

The require rubygems provides RubyGems classes and methods, which are Gem::Version and Gem.bin_path

The if ARGV.first block enable this feature:

rails _4.1.5_ new app

It allows different version of Rails to be load and run that instead of the default picked by RubyGems. The code check the first argument of rails command (ARGV.first) matches the regular expression /\A_(.*)_\z/. If it matched, and it is a correct version (Gem::Version.correct?($1)), that version is used for:

load Gem.bin_path('railties', 'rails', version)

The version which is loaded through the previous step or not (default >=0) is used to find the bin path of the specific version of the railties gem and load the rails script.

To identify the rails script path loaded by the above command, add:

put load Gem.bin_path('railties', 'rails', version)

before it and then run rails s command and look at the result printed in the terminal. Or running this command:

ruby -rubygems -e "puts Gem.bin_path('railties', 'rails', '>=0')"

The >=0 indicate that it will find the latest version of the railties gem that is installed in the system. >=0 can be replaced by a particular version of Rails. This command prints something like this (rbenv):

/home/hatt/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/railties-4.2.0/bin/rails

for rvm:

/home/hatt/.rvm/gems/ruby-2.1.5/gems/railties-4.1.10/bin/rails

(railties gem path)/bin/rails

The content of the railties/bin/rails:

#!/usr/bin/env ruby

git_path = File.expand_path('../../../.git', __FILE__)

if File.exist?(git_path)
  railties_path = File.expand_path('../../lib', __FILE__)
  $:.unshift(railties_path)
end
require "rails/cli"

This file is used to check whether the rails command is ran from the Rubygem installation version (through gem install rails), which every dependencies are loaded to the $LOAD_PATH, or it's ran directly from the cloned version of rails (git clone https://github.com/rails/rails.git).

If it is the cloned version, then there is a .git folder in it. And with this case, some dependencies gem, like railties, might not be already in the $LOAD_PATH. Then it will add the railties directory path to the $LOAD_PATH. The $: is the shorthand of the $LOAD_PATH and since the $LOAD_PATH is an array of string of path, unshift will add the string path to the $LOAD_PATH array.

The final line in this file require the file rails/cli which is also in the railties gem's lib folder.

(railties gem path)/lib/rails/cli.rb

File's content:

require 'rails/app_rails_loader'

# If we are inside a Rails application this method performs an exec and thus
# the rest of this script is not run.
Rails::AppRailsLoader.exec_app_rails

require 'rails/ruby_version_check'
Signal.trap("INT") { puts; exit(1) }

if ARGV.first == 'plugin'
  ARGV.shift
  require 'rails/commands/plugin'
else
  require 'rails/commands/application'
end

First, it requires rails/app_rails_loader, and then runs the method Rails::AppRailsLoader.exec_app_rails which defined in the app_rails_loader file. According to the comment, if we're inside a Rails application folder and run the rails command (like rails server, rails console), the exec_app_rails will be executed and then leave the script without executing the rest of it, and if we're not in any rails app folder and run rails command (like rails new) then the rest of the script will be executed.

Let skip the "in rails app folder" path for this moment and find out what's the rest of this script do first.

Next is a line that requires rails/ruby_version_check, which is this file:

(railties gem path)/lib/rails/ruby_version_check.rb

if RUBY_VERSION < '1.9.3'
  desc = defined?(RUBY_DESCRIPTION) ? RUBY_DESCRIPTION : "ruby #{RUBY_VERSION} (#{RUBY_RELEASE_DATE})"
  abort <<-end_message

    Rails 4 prefers to run on Ruby 2.1 or newer.

    You're running
      #{desc}

    Please upgrade to Ruby 1.9.3 or newer to continue.

  end_message
end

This file checks the RUBY_VERSION defined by the Ruby interpreter that are being used. If the version is less than Ruby 1.9.3, this script will abort the process using the abort method and puts the message in the here document end_message.

Because there is some Rails's code isn't compatible with the earlier version of Ruby (Ruby 1.9.1 and 1.9.2), the Rails Core Team have decided to only support Ruby 1.9.3 and above.

Back with the (railties gem path)rails/cli file. The remaining code which is:

Signal.trap("INT") { puts; exit(1) }

if ARGV.first == 'plugin'
  ARGV.shift
  require 'rails/commands/plugin'
else
  require 'rails/commands/application'
end

The Signal.trap will trap an INT signal, which is typically issued by Ctrl + C or Command + C shortcuts. If this script encounters with these signal, it will exit with a code of 1 to indicate that the process didn't run correctly.

The if block checks if the rails command is ran with the first argument provided to it is plugin, like rails plugin new, then it will require rails/commands/plugin file. If the argument isn't plugin, like rails new, then it's will load the rails/command/application to do something with the application instead.

Let's get back the "in rails application folder" path. In order to know what's the Rails::AppRailsLoader.exec_app_rails method does, let's open the rails/app_rails_loader files.

(railties gem path)/lib/rails/apprailsloader.rb

require 'pathname'

module Rails
  module AppRailsLoader
    extend self

    RUBY = Gem.ruby
    EXECUTABLES = ['bin/rails', 'script/rails']
    BUNDLER_WARNING = <<EOS
Looks like your app's ./bin/rails is a stub that was generated by Bundler.

In Rails 4, your app's bin/ directory contains executables that are versioned
like any other source code, rather than stubs that are generated on demand.

Here's how to upgrade:

  bundle config --delete bin    # Turn off Bundler's stub generator
  rake rails:update:bin         # Use the new Rails 4 executables
  git add bin                   # Add bin/ to source control

You may need to remove bin/ from your .gitignore as well.

When you install a gem whose executable you want to use in your app,
generate it and add it to source control:

  bundle binstubs some-gem-name
  git add bin/new-executable

EOS

    def exec_app_rails
      original_cwd = Dir.pwd

      loop do
        if exe = find_executable
          contents = File.read(exe)

          if contents =~ /(APP|ENGINE)_PATH/
            exec RUBY, exe, *ARGV
            break # non reachable, hack to be able to stub exec in the test suite
          elsif exe.end_with?('bin/rails') && contents.include?('This file was generated by Bundler')
            $stderr.puts(BUNDLER_WARNING)
            Object.const_set(:APP_PATH, File.expand_path('config/application', Dir.pwd))
            require File.expand_path('../boot', APP_PATH)
            require 'rails/commands'
            break
          end
        end

        # If we exhaust the search there is no executable, this could be a
        # call to generate a new application, so restore the original cwd.
        Dir.chdir(original_cwd) and return if Pathname.new(Dir.pwd).root?

        # Otherwise keep moving upwards in search of an executable.
        Dir.chdir('..')
      end
    end

    def find_executable
      EXECUTABLES.find { |exe| File.file?(exe) }
    end
  end
end

The RUBY constant contains the executable ruby path which is the same result with which ruby command. The EXECUTABLE constant contains the paths that the rails executable may reside. The BUNDLER_WARNING contains a message which warning when you use bundler bin stub instead of the original rails.

The exec_app_rails method will attempt to find the rails executable (through find_executable method) in either bin or script folder at the current path. If it can't find it then it will go up a directory (Dir.chdir('..')) and look for it there. This process will continue when either it finds the rails executable or it reach the root directory.

When it success in finding the executable rails in either bin or script folder, it will check this rails file's content. If this rails file is generated by Rails, which it will contain a constant APP_PATH or ENGINE_PATH, then it will execute this code:

if contents =~ /(APP|ENGINE)_PATH/
  exec RUBY, exe, *ARGV

If the file is not generated by Rails, which mean it doesn't contain APP_PATH or ENGINE_PATH, then it will execute the rest of the elsif path. This is because when running bundle install --binstubs, the bundler binstubs will override Rails's default bin/rails. The code in this elsif block will do exactly what the proper bin/rails would've done to boot the application.

In the previous section, the comment above Rails::AppRailsLoader.exec_app_rails said that:

If we are inside a Rails application this method performs an exec and thus the rest of this script is not run.

This happen when the bin/rails is found and is generated by the Rails, then this line exec RUBY, exe,ARGVis executed. And because theexecute` method when successfully executed will exit the ruby script immediately, then the rest of the script will be skipped.

Typically, a Rails's application will always follow the exec path unless the bundle install --binstubs is used. Let's follow this path to see the bin/rails script generated by the default Rails.

(application path)/bin/rails

#!/usr/bin/env ruby
begin
  load File.expand_path("../spring", __FILE__)
rescue LoadError
end
APP_PATH = File.expand_path('../../config/application',  __FILE__)
require_relative '../config/boot'
require 'rails/commands'

Since this script would be run by either directly (i.e. bin/rails c) or through exec method, the first line need to include the shebang to indicate that this script will be interpreted by ruby interpreter.

The APP_PATH constant contains the path to config/application which indicates that it's booting an application and not an engine. When booting a rails app, it will use the setting written in this config/application file.

Next, this file require config/boot:

# Set up gems listed in the Gemfile.
ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)

require 'bundler/setup' if File.exist?(ENV['BUNDLE_GEMFILE'])

This file configures the BUNDLE_GEMFILE environment variable if it isn't set before. Then it requires the bundler/setup, which is responsible for adding all the gems in the Gemfile to the $LOAD_PATH. It makes all the gems available in the rails application when the application is booted, without having to require specific gems in each file.

The final line of the bin/rails requires the rails/command file, which is in the railties gem.

(railties gem path)/lib/rails/commands.rb

ARGV << '--help' if ARGV.empty?

aliases = {
  "g"  => "generate",
  "d"  => "destroy",
  "c"  => "console",
  "s"  => "server",
  "db" => "dbconsole",
  "r"  => "runner"
}

command = ARGV.shift
command = aliases[command] || command

require 'rails/commands/commands_tasks'

Rails::CommandsTasks.new(ARGV).run_command!(command)

This file defines all the shortcuts for the rails commands like rails s and rails c. First, if the ARGV is empty which mean the rails command is ran with no params, then it assume that the user want some help by adding the --help into the ARGV. If the ARGV is not empty, then the first argument is extracted (through ARGV.shift) and is checked with the aliases hash to see if it is a shortcut or not. If it is not a shortcut then it's assume to be the command itself and it's used immediately.

The rails/commands/commands_tasks is required to provide the method that run the command extracted from the previous process. First, a Rails::CommandsTasks instance is initialized and then calling run_command! with the command as an argument on that.

(railties gem path)/lib/rails/commands/commands_tasks.rb

Since this file provide method that execute all of the command for rails, it's pretty large. For booting process, only the server command is considered here. So let's look at the run_command! and then the server command on this file.

The run_command!:

COMMAND_WHITELIST = %w(plugin generate destroy console server dbconsole application runner new version help)

def run_command!(command)
  command = parse_command(command)
  if COMMAND_WHITELIST.include?(command)
    send(command)
  else
    write_error_message(command)
  end
end

This method relies on a few other methods defined in this file. First, it calls the parse_command to check if it's a rails operation commands or help command like -h or -v etc.

parse_command

def parse_command(command)
  case command
  when '--version', '-v'
    'version'
  when '--help', '-h'
    'help'
  else
    command
  end
end

Next, it checks if the command is a valid command in COMMAND_WHITELIST constant. If the command is not valid (not listed in the COMMAND_WHITELIST) then the write_error_message is called to handle the invalid message:

write_error_message

def write_error_message(command)
  puts "Error: Command '#{command}' not recognized"
  if %x{rake #{command} --dry-run 2>&1 } && $?.success?
    puts "Did you mean: `$ rake #{command}` ?\n\n"
  end
  write_help_message
  exit(1)
end

This method will check to see if there is a Rake task with the same name as the command. This is because people might be confusing between rake and rails command so they might run something like rails db:migrate. Finally, this method will put a help text message and then exit the script. The help message is the common message that displayed on the terminal when you mistype a rails command.

When running the rails server or rails s command, the run_command!, after checking the argument passed is a valid command, uses send method to call the method that match the command name which in this case, it is server:

server

def server
  set_application_directory!
  require_command!("server")

  Rails::Server.new.tap do |server|
    # We need to require application after the server sets environment,
    # otherwise the --environment option given to the server won't propagate.
    require APP_PATH
    Dir.chdir(Rails.application.root)
    server.start
  end
end

First, the server method call the set_application_directory method:

# Change to the application's path if there is no config.ru file in current directory.
# This allows us to run `rails server` from other directories, but still get
# the main config.ru and properly set the tmp directory.
def set_application_directory!
  Dir.chdir(File.expand_path('../../', APP_PATH)) unless File.exist?(File.expand_path("config.ru"))
end

Then, it calls the require_command method:

def require_command!(command)
  require "rails/commands/#{command}"
end

The command argument's value here is server, so this method will require rails/commands/server file to the server method, which defines the Rails:Server constant in this method.

The final lines of the server method initialize an instance of Rails::Server, setting things up and then start it. To understand what Rails::Server is, let's look at the rails/commands/server.rb file.

(railties gem path)/lib/rails/commands/server.rb

require 'fileutils'
require 'optparse'
require 'action_dispatch'
require 'rails'

module Rails
  class Server < ::Rack::Server
  #..

  def initialize(*)
    super
    set_environment
  end

  #..
end

The Rails::Server is inherited from the ::Rack::Server class, and its initialize method use super to delegate the work for its parent (::Rack::Server).

Since fileutils and optparse is the Ruby's built-in libraries, the complexity of this file mostly come from the action_dispatch and rails. They will be considered in the next blog post.