Saturday, April 30, 2011

Overwrite Devise Registrations Controller in Rails 3

I have a Rails application with
Rails version-3.0.5,
Ruby version- "ruby 1.8.7 (2010-08-16 patchlevel 302) [i386-mingw32]" and devise gem.

I wanted to overwrite some of the methods in Registrations Controller used by Devise. As it is done by Devise internally, I needed to create a controller that inherits from Devise::SessionsController. Here is how I did it:

Changed the routes as:
devise_for :users ,:controllers => { :registrations => "registrations" }

Added one controller named "registrations_controller" with following codes:
class RegistrationsController < Devise::RegistrationsController
  before_filter :authenticate_user!, :only => :token

  def new
      super
    end

 
  def create
    @user = User.new(params[:user])
    if @user.save
      flash[:notice] = "You have signed up successfully. If enabled, a confirmation was sent to your e-mail."
      redirect_to root_url
    else
      render :action => :new
    end
  end

  def update
    super
  end
end
'new' aciton with 'super' keyword is required. Super will executes the code from the inherited method from the Devise controller.
This is very simple 'create' action which can be customized.

Thursday, April 28, 2011

Getting "undefined method `name' for nil:NilClass" when running command "rake devise:setup"

I have a Rails application with
Rails version-3.0.5,
Ruby version- "ruby 1.8.7 (2010-08-16 patchlevel 302) [i386-mingw32]" and devise gem which is working fine. But when I am running command "rake devise:setup" it gives me following error:
rake aborted!
undefined method `name' for nil:NilClass

(See full trace by running task with --trace)

On running command with full trace following is the error:

** Invoke db:schema:dump (first_time)
** Invoke environment
** Execute db:schema:dump
** Invoke environment
** Execute devise:setup
rake aborted!
undefined method `name' for nil:NilClass
C:/Ruby187/lib/ruby/gems/1.8/gems/activesupport-3.0.5/lib/active_support/whiny_nil.rb:48:in `method_missing'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:57:in `visit_Arel_Nodes_InsertStatement'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:56:in `map'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:56:in `visit_Arel_Nodes_InsertStatement'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:15:in `send'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:15:in `visit'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:5:in `accept'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:19:in `accept'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:111:in `with_connection'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:17:in `accept'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/tree_manager.rb:20:in `to_sql'
C:/Ruby187/lib/ruby/gems/1.8/gems/arel-2.0.9/lib/arel/select_manager.rb:217:in `insert'
C:0:in `__send__'
C:0:in `insert'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/persistence.rb:270:in `create'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/timestamp.rb:47:in `create'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/callbacks.rb:281:in `create'
C:/Ruby187/lib/ruby/gems/1.8/gems/activesupport-3.0.5/lib/active_support/callbacks.rb:420:in `_run_create_callbacks'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/callbacks.rb:281:in `create'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/persistence.rb:246:in `create_or_update'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/callbacks.rb:277:in `create_or_update'
C:/Ruby187/lib/ruby/gems/1.8/gems/activesupport-3.0.5/lib/active_support/callbacks.rb:418:in `_run_save_callbacks'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/callbacks.rb:277:in `create_or_update'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/persistence.rb:39:in `save'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/validations.rb:43:in `save'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/attribute_methods/dirty.rb:21:in `save'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/transactions.rb:240:in `save'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/transactions.rb:207:in `transaction'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/transactions.rb:240:in `save'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/transactions.rb:251:in `rollback_active_record_state!'
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/transactions.rb:239:in `save'
C:/Ruby187/lib/ruby/gems/1.8/gems/devise-1.3.3/lib/devise/models/token_authenticatable.rb:41:in `reset_authentication_token!'
C:/Ruby187/lib/ruby/gems/1.8/gems/devise-1.3.3/lib/devise/models/token_authenticatable.rb:51:in `ensure_authentication_token!'
C:/projects/restaurant/lib/tasks/setup.rake:9
C:/Ruby187/lib/ruby/gems/1.8/gems/activerecord-3.0.5/lib/active_record/validations.rb:33:in `create!'
C:/projects/restaurant/lib/tasks/setup.rake:5
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'
C:/Ruby187/lib/ruby/1.8/monitor.rb:242:in `synchronize'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'
C:/Ruby187/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake:31
C:/Ruby187/bin/rake:19:in `load'
C:/Ruby187/bin/rake:19

Thanks in advance for any help/solution.

Thursday, April 14, 2011

Simple Ruby Screen Scraper using Mechanize, Hpricot and XPath

In my previous post, simple-ruby-screen-scraper-in-just-5.html, I explained how to do Screen Scraping in just 5 lines without using XPath. In this post, I will achieve the same task using XPath, before going through this, I would recommend you to go through previous post to get an idea about the complete scenario.

Here also I will be scraping a link "Quality Assurance" given on the left panel under "My Blogs" section on my website "http://www.kumarritesh.com/" with XPath. Here is the code:


require 'rubygems'
require 'mechanize'
require 'hpricot'

agent = Mechanize.new
page = agent.get('http://www.kumarritesh.com')
@response = page.content
doc = Hpricot(@response)
(doc/"/html/body/div/div/div[2]/div[2]/div[3]/div/ul/li/a")[0].innerHTML


All the code is same as mentioned in the previous post, only last line changes.
In my previous post, I parsed the page by searching the div class, li id and anchor link.
Here, I will make use of XPath. 

To get XPath, inspect the "Quality Assurance" link in Firebug. You will get the code:
<a href="/index.php/quality-assurance">Quality Assurance</a>

Right click and select "Copy XPath" option. You will get following code:
/html/body/div/div/div[2]/div[2]/div[3]/div/ul/li/a
 
Pass the code as displayed in the last line, it will return all the links, so to get the first link we have taken the zeroth element and then its HTML format. Similarly, we can have the XPath of any text, label, links and can scrap it very easily.

Simple Ruby Screen Scraper in just 5 lines without using XPath

Simple Ruby Screen Scraper in just 5 lines, true, its very -very basic scrapper that I will  explain in this article. You can call it Screen Scraping, Web Screen Scraping, Data Scraping, Website Scraping, Web Page Scraping, Web Crawler .... ....... my main aim is to explain how to extract a particular link or label from a web page.

For scraping, I am using Ruby version - 1.8.7 which should be installed on your system. Along with this, Mechanize (1.0.0) and Hpricot  (0.8.4) gem. I will be scraping a link "Quality Assurance" given on the left panel under "My Blogs" section on my website "http://www.kumarritesh.com/". So lets start.

1. Open command prompt (I am working on Windows 7). Then open Interactive Ruby Shell (irb) prompt.

C:\Users\ritesh>irb
irb(main):001:0>

2. Open site "http://www.kumarritesh.com/" and inspect the div in which link "Quality Assurance" is present. Web page Code of that block looks like the code given below:

<div id="module_77" class="tabcontent tabopen" tabindex="-1" role="tabpanel" aria-hidden="false" aria-expanded="true" aria-labelledby="link_77">
  <ul class="menu">
    <li id="item-466"><a href="/index.php/quality-assurance">Quality Assurance</a></li>
    <li id="item-467"><a href="/index.php/search-engine-optimization">Search Engine Optimization</a></li>
    <li id="item-468"><a href="/index.php/ruby-on-rails">Ruby on Rails</a></li>
    <li id="item-470"><a href="/index.php/blogs-links">Blog Links</a></li>
  </ul>
</div>

 3. Write the codes as follows:
require 'rubygems'
require 'mechanize'
require 'hpricot'

agent = Mechanize.new
page = agent.get('http://www.kumarritesh.com')
@response = page.content
doc = Hpricot(@response)
doc.search("//div[@class='tabcontent tabopen']").search("//li[@id ='item-466']").search("a").innerHTML

Last line will give you the output as "Quality Assurance". Very much confused with the code, let me take you through the code step by step.

First three lines load hpricot, rubygems and mechanize
require 'rubygems'
require 'mechanize'
require 'hpricot'
Instantiate a new mechanize object:
agent = Mechanize.new  
Now we'll use the agent we've created to fetch website page "http://www.kumarritesh.com/" and store the page content in an object, then passing that object to Hpricot. Hpricot loads the contents into a document object.
page = agent.get('http://www.kumarritesh.com')
@response = page.content
doc = Hpricot(@response)

 Now, last line will give you the expected output by parsing the doc
doc.search("//div[@class='tabcontent tabopen']").search("//li[@id ='item-466']").search("a").innerHTML

 What it does is it searches for the div class 'tabcontent tabopen' and then inside that div class, searches for li with id 'item-466' and then finally searches for anchor link.
'innerHTML' will remove all HTML elements and thus you get the result as "Quality Assurance"

In the coming articles, I will be taking you deep into the ocean of scraping where you will be diving like a big whale here and there finding some useful & interesting concepts, so stay tuned