Forever Learning

Feed Rss

Amazon Product Advertising API w/ Ruby

07.01.2010, Business, Tech, by .

(This is a popular article from my old blog. Given that about 200 people a month come to this guide I 301’d it here when I stopped writing the old blog. It has been backdated so that the original publish date is what you see above.)

The Amazon Product Advertising API (formerly AWS or AWS-EC) is a way to programmatically perform many of the actions a user can perform on the Amazon.com website. This includes searching for an item, viewing an items details, or adding an item to your cart.  In order to use this API it is required that you register as a registered affiliate in the Amazon Affiliate Marketing program. Any use that is outside the scope of the AM program is expresely forbidden by the TOS.  The data obtained through this API is generally used to, for example, provide a set of dynamically generated custom links for advertising amazon products that are relevant to some user generated content in your application or website.

This API is not to be confused with the other major Amazon Web Services APIs (which are now also called “AWS”) such as AWS-S3 (Simple Storage Service) or AWS-EC2 (Elastic Computing Cloud).  Amazon had formerly named this particular API “AWS-EC” for Affiliate Web Services ECommerce API, but that name was deprecated when they decided to call the aforementioned pay-per-unit cloud based APIs “Amazon Web Services.”  And to top it off an Amazon Affiliate that wants to use the API formerly known as AWS must signup for an “AWS” account since the Product Advertising API is of course an Amazon Web Service.  (Did I mention the reference docs still say AWS?)

Prerequisites

  1. You have obtained an Amazon Affiliate Program account (here)
    • be sure to actually read the TOS for this service because Amazon will terminate if you’re caught in violation
  2. You have installed Ruby, Rails, & RubyGems (here)
  3. You have a basic understanding of programing and Ruby.

Technologies

The focus of this tutorial is both pragmatic and practical.  If you wish to understand SOAP here is the O’REILEY text on the subject.  The short answer is that SOAP is a, for better or for worse, widely adopted specification for data exchange.  Most large companies that provide any sort of API to the outside world provide a SOAP based XML API. For this tutorial we will be using:

  • Ruby (v 1.8.7)
  • Rails (v 2.3.8)
  • RubyGems (v 1.3.7)
  • Ruby/AWS (v 0.8.1)

Additional, less helpful, guides are available from amazon for Perl, Java, PHP, C# here.  Perhaps I will writeup some supplemental plug-in guides for the other versions if there is interest.

Obtaining an AWS Developer Key

Login with your Amazon.com credentialsIn addition to being an affiliate program member Amazon also requires users of their Affiliate Product Advertising API to sign up for an AWS account in order to obtain an access key with which to sign their requests.  The registration page is pretty straight forward.

  1. Verify your location
  2. As Amazon puts it “Success”

Once logged in at the Amazon Web Services home page go to Account >> Security Credentials.  Your newly generated API key will be waiting for you under the Access Credentials Section.

Be sure to keep your private key private!

https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key

Ruby/AWS

Ruby/AWS is the project name of the GPL RubyGem, written and maintained by Ian Macdonald, that handles the lower level communication with tha Amazon SOAP API.  You can think of Ruby/AWS as a tangible implementation of the interface defined by the WSDL.  This means the creator took the datatypes and operations specified in the WSDL and implemented them as classes and methods in ruby code.  As of the time of this writing the Ruby/AWS Gem on RubyForge is version 0.7.0.  Due to the significant re-factoring and the statement that v 0.8.1 “more or less fully  supports” the Amazon AWS v 4.0 API, I choose to install the most up-to date version available at the bottom of this rdoc page.

wget http://caliban.org/files/ruby/ruby-aaws-0.8.1.gem
sudo gem install ruby-aaws-0.8.1.gem

That’s it for the installation, pretty easy ehh. Well there is one more thing you need to do and that’s setup a file named .amazonrc in your home dir.  This file holds configuration information and should look something like this:

key_id = 'someshorterpublickey'
secret_key_id = 'somelongprivatekeythatseemsthislong'
associate = 'some-associate-code'
cache = false
locale = 'us'
encoding = 'iso-8859-15'

Your associate Tracking ID can be found in the upper left hand of your logged in associate main page:

https://affiliate-program.amazon.com/gp/associates/network/main.html

Now that .amazonrc is setup we can write a little script to test things out.

Hello World

(or the demonstrable AWS equivalent)

#!/usr/bin/ruby -w

require 'amazon/aws/search'

include Amazon::AWS
include Amazon::AWS::Search

is = ItemSearch.new( 'Baby',
 {
 'Keywords' => 'pants',
 'MinimumPrice' => '2500',
 'MaximumPrice' => '4999'
 } )
is.response_group = ResponseGroup.new( 'Small' )

req = Request.new
req.locale = 'us'

resp = req.search( is )
items = resp.item_search_response[0].items[0].item

items.each { |item| puts item, '' }

ItemSearch

Starting on line 8 we create a new ItemSearch object and give it our search parameters. ItemSearch’s constructor takes searchIndex which is roughly equivalent to the department dropdown on Amazon.com, with a few extras like “Music” and “Video” or a particular “Merchant Id.” The parameters are the conditions to search on which are once again roughly equivalent to the text input field and the additional filters of Amazon’s search results.  The complete list of search indices and parameters is available in the “AWS E-Commerce Service” API reference (yes, another name for it) entry for ItemSearch.  One thing to mention about ItemSearch is that you should be sure to check the search index compatibility matrix because not all indices are available on all locales.  The US locale currently supports all but 3 of them, for the other locales support is sparse.

ResponseGroup

A ResponseGroup is amazon’s way of you telling them how much and what data you want.  Valid ResponseGroups include:

  • Small, Medium, and Large
  • Reviews, EditorialReview
  • Offers
  • SalesRank
  • Accessories
  • etc…

You cannot combine ResponseGroups so requests for Mutually exclusive pieces of data will require multiple requests.  There is some limited batching functionality, but you may still come across cases where you need to make more than one request for a page.

Request

The request object is what is going to actually make the request to the web service like you would if you put a URL in your browsers address bar.  It then receives an XML response from Amazon and parses that into the appropriate set of objects for the given Operation.

Terse Version

This package has also made sure we have some common case optimizations built in. In this case the first script was setup with us providing the Amazon specified defaults as our parameters. Since we can leave the defaults out that script returns equivalent results to this much shorter version.

#!/usr/bin/ruby -w

require 'amazon/aws/search'

include Amazon::AWS
include Amazon::AWS::Search

resp = Amazon::AWS.item_search( 'Baby',
 {
 'Keywords' => 'pants',
 'MinimumPrice' => '2500',
 'MaximumPrice' => '4999'
 } )

items = resp.item_search_response.items.item

items.each { |item| puts item, '' }

Output

When you run the script you should see something along the lines of:

...
similar_products = similar_product = title = Pampers Easy Ups Value Pack for Girls, Size 5, 80-Count
 Box
asin = B0027VT9J8title = Pampers Easy Ups Value Pack for Boys, Size 6, 66-Count Box   asin = B0027VT9IEtitle = Pampers Cruisers Dry Max Diapers, Economy Plus, Size 6 (35+ Lbs), 100 Diapers
asin = B00347AFGU
title = Huggies Pull-Ups Learning Designs Training Pants, Girls, 4T-5T, 44-Count Biggie Pack
asin = B003D7LA8I
title = Pull-Ups Training Pants with Learning Designs, 3T-4T (32-40 lbs), Mega, 40 training pants
asin = B000096OKL

offer_summary = total_used = 0
lowest_new_price = currency_code = USD
formatted_price = $25.00amount = 2500
total_collectible = 0
total_new = 5total_refurbished = 0

image_sets = image_set = thumbnail_image = width = 75height = 75
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL75_.jpg
small_image = width = 75
height = 75
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL75_.jpg 

large_image = width = 500
height = 500
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL.jpg

tiny_image = width = 110
height = 110
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL110_.jpg

swatch_image = width = 30
height = 30
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL30_.jpg
...

That’s it.  This is the string representation of the objects that were generated from Ruby/AWS parsing the XML returned from Amazon.  The fruits of your labor have been realized and you can process amazon data (that is relevant for your Affiliate Marketing purposes) till your hearts content.  As long contentment comes before you average 1 request per second.  Did I mention to read the TOS?

XML –> Objects

Because XML makes me hate life you get a direct quote from a subsection of the README:

In Ruby/AWS, each unique XML element name forms a class of the same name. All
such classes are subclasses of AWSObject. For example, OperationRequest is a
class, as is ItemAttributes.

As the XML tree is traversed, each element is converted to an instance of the
class of the same name. Every such object has instance variables, one per
unique child element name. The name of the instance variable is translated to
comply with Ruby convention by adding an underscore ('_') character at word
boundaries and converting the name to lower case.

For example, given the following XML:

<ItemAttributes>
 <Author>Philip Pullman</Author>
 <Manufacturer>Scholastic</Manufacturer>
 <ProductGroup>Book</ProductGroup>
 <Title>The Ruby in the Smoke (Sally Lockhart Quartet)</Title>
</ItemAttributes>

the following statements would all be true:

- ItemAttributes, Author, Manufacturer, ProductGroup and Title would all be
dynamically defined subclasses of AWSObject.

- An instance of the ItemAttributes class would be created, with instance
variables @author, @manufacturer, @product_group and @title.

- To each of these instance variables would respectively be assigned an array
of Author objects, an array of Manufacturer objects, an array of
ProductGroup objects and an array of Title objects. In the above case, these
would all be single element arrays, because there's only one instance of
each kind of tag in the XML.

- The Author, Manufacturer, ProductGroup and Title objects would have no
instance variables of their own, because the corresponding XML elements
have no children, just a value. These objects are therefore directly
assigned the value in question.
Well that was easy, I should teach from the book more often… Thanks Ian, moving on.

Common Errors

“`require’: no such file to load — amazon/aws/search (LoadError)”
solution: don’t forget to set RUBYOPT in your .bash_profile via: export RUBYOPT=rubygems
“Your request should have atleast 1 of the following parameters: Signature, Timestamp.”
solution: This error is the general purpose error Amazon gives you when your request does not appear to have been signed correctly. Since the Ruby API we’re using does that for us the likely cause of this error is that you didn’t set your API secret key correctly in the .amazonrc file.

The README also points out that Amazon considers requests timestamped as being more than 15 minutes old to be invalid, so make sure your server’s closk is being sync’d appropriately.

Thanks

As always Comments are welcome and if you find any gaps in this tutorial, gaps in the Ruby/AWS API, or see that a new version is available, throw me a comment so I can update the tutorial and alert the maintainer.  For more information you can consult the 4000 word README doc in the rubygems directory for this gem.

API References

  • ruby-aaws: (rdoc) (or of course ‘gem server’)
  • Amazon documentation: