curl - Ruby - net/http - following redirects

Question

I've got a URL and I'm using HTTP GET to pass a query along to a page. What happens with the most recent flavor (in net/http) is that the script doesn't go beyond the 302 response. I've tried several different solutions; HTTPClient, net/http, Rest-Client, Patron...

I need a way to continue to the final page in order to validate an attribute tag on that pages html. The redirection is due to a mobile user agent hitting a page that redirects to a mobile view, hence the mobile user agent in the header. Here is my code as it is today:

require 'uri'
require 'net/http'

class Check_Get_Page

    def more_http
        url = URI.parse('my_url')
        req, data = Net::HTTP::Get.new(url.path, {
        'User-Agent' => 'Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_2 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8H7 Safari/6533.18.5'
        })
        res = Net::HTTP.start(url.host, url.port) {|http|
        http.request(req)
            }
        cookie = res.response['set-cookie']
        puts 'Body = ' + res.body
        puts 'Message = ' + res.message
        puts 'Code = ' + res.code
        puts "Cookie 
" + cookie
    end

end

m = Check_Get_Page.new
m.more_http

Any suggestions would be greatly appreciated!

Answer 1

To follow redirects, you can do something like this (taken from ruby-doc)

Following Redirection

require 'net/http'
require 'uri'

def fetch(uri_str, limit = 10)
  # You should choose better exception.
  raise ArgumentError, 'HTTP redirect too deep' if limit == 0

  url = URI.parse(uri_str)
  req = Net::HTTP::Get.new(url.path, { 'User-Agent' => 'Mozilla/5.0 (etc...)' })
  response = Net::HTTP.start(url.host, url.port, use_ssl: true) { |http| http.request(req) }
  case response
  when Net::HTTPSuccess     then response
  when Net::HTTPRedirection then fetch(response['location'], limit - 1)
  else
    response.error!
  end
end

print fetch('http://www.ruby-lang.org/')