Forever Learning

Feed Rss

I was having lunch with some middle manager friends of mine at one of the companies I’ve worked for and two of them started to recount a conversation they had with their VP earlier that week.  He was asking them to set something up so that he had a mailbox where he could leave messages that would be transcribed and emailed to him.  Basically he wanted to followup when he was back at the office if he had an idea while he was out for a run or something.  Sounds reasonable, pretty straightforward, nothing to write home about here.  But then one of them mentions “He said he has a million dollar idea every day and he doesn’t want to lose them.” , to which the group bursts into laughter.  Wait, laughter?  I stop for a second and think, “Why are they laughing?”.  Every one of these people was a moderately successful, educated, product development manager with at least a half decade of experience successfully executing on complex projects to justify their 6-figure salaries.  Yet, none of them thinks that it’s reasonable to frequently have great ideas?

Do vs. Don’t

This is one of the experiences that helped me realize what separates those that do from those that don’t.  When you become a student of business (not to be confused with a business student) you build up knowledge through books, case studies, blogs, examples, observation, and often times though the work that you are doing starting up your own company.  After a while you start to see the world differently.  You start to understand why companies do the things they do.  You start to look at company X and realize their potential, even if they currently have $0 in revenue.  You appreciate the beauty of watching a new product fit a market.  You start to think in terms of providing value to a market.  Every day turns into an exercise in “wow, how has no one solved this problem yet”.  The gears turn in your head and eventually you get to a point where you have a million dollar idea at least once a week.  In the past I’ve called this actively thinking and actively thinking is what generates the ideas that feed the engine that drives innovation in our world.

Ideas and Startups

So, if ideas are what feed the engine, then they are valuable, right?  A founder with a great idea deserves to get a big chunk of a company and have other people just execute on all of his visionary ideas while he sites back and waits for his big fat check from Google, right?  You want my opinion?  Not a chance.

Any good startup is going to have a hundred great ideas in their first year alone.  The entire founding team is going to be contributing great ideas every day.  Chances are good that the first great idea, the idea that got the team together and got the ball rolling, isn’t even going to be what makes your company successful.  Heck, in the end it might not even make the top 10.  The problem isn’t a lack of great ideas, the problem is choosing which ideas to execute on, and then executing on those ideas.  Tenacity, resilience, perseverance… these are all the traits that VCs look for when they are investing in a person (notice I didn’t say company).  When it comes to ideas I care less about a person’s actual idea and more about whether they can duplicate that effort.  As we’ve discussed, a million dollar idea isn’t all that amazing.  But, ten million-dollar ideas… compounding on top of each other, backed by a solid group of people who can execute on those ideas–that’s starting to get a little special.  I care more about them understanding how to learn what a market’s needs are, and understanding how to meet those needs, than I do about any particular idea.  You need to continually refine your ideas, continually come up with new ideas, and continually choose to throw out the bad ones.  In fact, choosing to not work on a bad idea is much harder, and arguably more important, than coming up with a good one.  For every truly great idea there are 9 seemingly superb ones that are actually not going to get you where you want to be.  The resources wasted executing on a bad idea, while leaving a good one waiting in the wings, is time that you might as well have spent sitting on your hands.  This is the power of the lean methodologies.

Ideas + Lean = Success

The lean startup methodology advocates for wasting as little resources as possible validating the assumptions of an idea before you start to invest significant resources into fully implementing that idea.  Why take 6 months and spend $50,000 going down the wrong path if you can spend 6 days and $2000 learning not to go that way?  A lean engine (build, measure, learn, repeat) coupled with a steady stream of pretty good ideas will separate the wheat from the chaff and, given successful execution by the project team, will inevitably succeed.

The Takeaway

As an “idea guy”, AKA anyone who isn’t directly building or selling(+markting) the product, your job is not done when you come up with an idea.  You need to also determine how to test that idea in a non resource-intensive way in order to prove its value.  You need to keep coming up with great ideas, as do your fellow execution-oriented cofounders btw, and you need to keep from assigning inherent value to your ideas based on the fact that they were your ideas.  You need to partake in actively thinking so that your output of great ideas is high, and you need to make understand your market so that your output of bad ideas is low.  Finally, you need to let the process work so that the truly good ideas will prove themselves, and the bad ones will fall away.

Thanks for reading, as always feel free to contribute to the discussion.

This blog post is an MVP, please use the comments to provide positive and negative feedback from which I can learn and iterate on my idea.

I am thinking of writing a CTO guide for early stage startups[1] and entrepreneurs.  There are tons of people walking around with big ideas, but don’t know how to implement them.  Many of these people have a good mind for problem solving and understanding complex logic, but are missing some critical piece of the puzzle that would enable them to be a very effective technical co-founder or CTO.  To be a stud tech-cofounder you need to be someone who is comfortable with technology in general, knows how to code, knows how to develop a product, knows how to maintain systems, and understands how to best apply technology to serve the needs of the business. Throughout their time as a technical cofounder the CTO of a successful startup will have developed all of these skills, and this guide is meant to help enable or accelerate that process.

Target Market

As with any product, you need to define your market and understand who your audience is. I’ve identified 4 user-types who would benefit most from this knowledge.

The inexperienced generalist

This persona is a reader who understands the basics about most technical things, but has several gaps in several areas between where they are and where they would need to be in order to serve as a solid technical co-founder or CTO.  They’ve probably rented a shared server before and tweaked some wordpress sites, they understand the basics of SEO, have an interest in business, and taught their parents how to set the timer on the VCR in 1992… at the age 10 (apparently they’re turning 30 this year, what a coincidence).  They typically have a good understanding of windows or OS X, and they might have booted or installed ubuntu before to see what it was about.  A terminal window doesn’t scare them, but they’re not proficient by any means.  They probably know what a compiler is, but have no idea what “strongly typed” or “weakly typed” means.  They have probably built one or more computers for themselves or their friends/family in their lives, but most likely haven’t worked as a sysadmin.  These people have demonstrated a general technical proficiency which indicates the capacity to be the technical leader of an early stage venture.  My book should be able to bring the knowledge level of these people up significantly and serve as a general reference that they can turn to as the go meet the challenges of building a new company.

The awesome business cofounder

You know who I’m talking about.  To some it may seem like a myth, but there is such a thing as a business cofounder who is not only able to effortlessly squeeze funding out of a VC, close a deal like every prospect is their mom, and understand exactly what the market wants… but is able to get the analogy of a database being like a bunch of spreadsheets and whose head doesn’t blow up at the sight of a WHERE clause.  Basic programming control flow statements intuitively make sense to them (even if recursion doesn’t), and they were pretty good at chemistry, or physics, or math.  Most of all they have the perseverance and the desire to learn what they need to learn in order to fully understand their product.  This person cannot, and should not, be taking on the role of a technical cofounder or CTO, but they might need to bring one on board. They also might need to build the initial prototype of their idea in order to show people a working product (with some market uptake) and help get funding or make equity in their company more attractive to that software architect friend of theirs who make $180K + stock options + ESPP at their well-established company.  The best business cofounders are the ones who have a good grasp of technology and how their products are built.  The best technical cofounders are the ones who understand business and understand the needs of their market.  There are a multitude of ways that the business cofounder can benefit from my hypothetical book to make themselves more valuable to their venture.

The specialist

This is your prototypical enterprise developer or sysadmin.  They’ve done amazing things in code, but don’t know how to create a DNS server.  Or they’ve built systems that serve tens of millions of requests a day for *insert corporation X here*, but their coding is limited to 20 – 30 line bash or perl scripts that essentially just wrap a series of shell commands that they would have typed by hand to do the same job.  (I’m not knocking perl, I like writing perl… we’ll leave it at that.)  This person would typically be a very valuable cog in someone else’s wheel, but after getting sick of seeing the need to develop “other skills” in order to succeed in the corporate world (like rubbing the right people the right way at the right times) they’ve decided they want to stop trading their life for a paycheck and build their own wheel.   The journey from being a specialist to being the technical lead for a company can be difficult.  Specialists are used to knowing everything about their silo, and being a subject matter expert that everyone goes to for help on some complex issue.  Looking for help on some 17 year old web-designer’s blog when you’re trying to figure out why your button doesn’t work in IE is quite a change.  This person can benefit greatly from my hypothetical book by ramping up quickly in the areas where they are not particularly strong.  I’m not going to make them an expert over-night (they probably know more than me about their specialty [unless they are a developer ;-) ]), but I can give them a good start and help them move their momentum in the right direction.

The tech-only leader

This person, like the specialist, is strong in one technical area but is also decently well versed in others.  They have probably been developing professionally for years, but have also maintained production systems either on the side or in some past role.  They can code the hell our of an app in development, can deploy the app to a production environment, and with a little goggling can scale the app up to say a million users without breaking much of a sweat.  So what’s the problem?  Well this person has never been interested in the business aspect of things.  Intheir current form, they’re most useful as a “Lead Developer” or a “Lead Administrator”.  If you were to hand them a “completed” product they wouldn’t really know what to do with it, how to market it, how to sell it, etc…  They are a user of software and have product development experience so they can brainstorm use cases, but they don’t understand how to guide product development and how to best leverage the product to meet the needs of their market.  Sales = forget about it, funding = forget about it, marketing = they probably know what a “genetic algorithm” is, so they should understand the power of A/B testing fairly quickly, but they don’t know to even do it.  Networking, who needs that?  I put this persona last because the tech-only leader is the person who needs the least help from me of this group.  In fact a few years ago I definitely fit this persona.  I would guess (no data to back this up) that most successful technical co-founders come from this group, but require a very solid business lead to balance them out.  The tech-only leader will eventually, maybe begrudgingly, learn the lessons in my book but, unless they devote considerable time to developing their business skills (which they probably won’t), it will take them longer than it should.   These people probably won’t buy the book due to their lack of desire to learn business stuff, but maybe the “awesome business cofounder” will give them a copy.

Who I would NOT be targeting

I’ve also identified two user-types who do not fit the profile of someone who would benefit from reading my book.

The “Idea guy”

Sorry “idea guy” but you need more help than I can provide in one book.  Problem #1 is that you consider yourself “the idea guy”.  Feel free to buy my product, but you’re going to have to give it to that friend of yours who likes to watch those boring TED talks, or the one who showed you how to make those “balls of fire” (it’s called plasma) in your microwave, and convince them to do the work for you.  Those with no technical aptitude whatsoever, no desire to learn, and lacking the perseverance to execute would be better off spending their time refining that elevator pitch again or tweaking the copy on your marketing materials.  To be clear, I’m not saying you’re useless, you’re just not useful in this capacity, and by considering yourself the idea guy you have indicated to me that you just don’t have what it takes.  In fact, here is a blog post by someone complaining about business co-founders not being given enough credit (from which I stole the above graphic).  This guy eventually pushes through and builds a prototype, an act which moves him farther from the “idea guy” persona and closer to the “awesome business cofounder” persona.  But, what really made me link to that blog post was the top comment on it.  A guy going by Josh Strike (who I’m assuming is this guy: https://github.com/joshstrike), posts what I consider to be the single greatest random-internet-blog-comment about this topic ever written.  I will probably have a future post dedicated to the awesomeness that is this blog comment.  Go read it.

The person who has already figured it all out

Congratulations, you are one of the few people who already know everything in my book.  You have probably run a startup that has a large customer base (pay or not) and you were a strong business founder who learned the technical stuff or a strong technical founder who learned the business stuff.  If this persona is you, and somehow you are still here reading this, please give me some comments below to help improve this idea and help everyone else.  Also, if you have a large network to help distribute this product, and would like to help… then let me know.

Topics

Here is a proposed table of contents or outline of the book.  I lead with MVP because well, that is something that people should learn up front, before they start building out some massive web-scale (buzzword) system and call it an MVP, because they’ve seen people use that word on TechCrunch.  I follow that up with business because I want people to get that information early and have it in mind when they’re using the rest of the book to help develop their products.  I move to outsourcing next as a bit of a segway between business and technical, and ironically I was thinking of outsourcing this section to a (stateside) buddy of mine.  Once we get into the technical stuff I start with a high-level discussion of platforms, move to a lower but still fairly high level discussion of frameworks, and then get into product development.  Dev covers mostly process, but also UX (user experience) and bug tracking/maintenance.  Coding will be a significant portion of the book and I’m thinking I’ll probably focus on ruby+rack while touching on some other server side technologies.   We’ll be using bootstrap, jQuery, sass, haml, coffeescript, and mustache plus a few small libraries I’ll probably open source for this book in order to build our dynamic application.  Obviously I can’t write an opus on just coding in the middle of a book like this, but my goal is to give you a sufficient level of expertise to build & scale a prototype while using standard patterns.  Next we move on to systems where I focus on Linux (sorry windows developers) with ubuntu/centos, apache, nginx, ruby enterprise, MySQL, etc., general operational needs, and hosting.  What I dont’ talk about here is installing ram and patch cables, though I might tough on hardware in the colocation section.  We’ll see.  Finally we close with a bang by giving a full chapter, that you will hopefully have to use, on scaling your stack. This covers both code techniques and hardware configuration.  Ohh, and I will throw a little something about workstation administration or something.  I did spend a year with that technically being part of my job title in college, and it is sort of necessary.  I guess. I admit reluctantly. I’m thinking the appendix will contain some pointers out to other resources to further your knowledge so that this book both give you the foundation and points you in the right direction for more information.

  • MVP
    • Does P equal “Product” or “Prototype”?
    • build
    • measure
    • learn
  • business
    • vision
    • planning & finance
    • marketing (I might make this its own section)
    • selling
    • funding (I might make this its own section)
    • influencing people
    • handling business founders and the management team
    • hiring employees
    • talent acquisition (yes, this is different from the previous one)
    • operations
    • people management
  • outsourcing
  • platforms
    • web
    • mobile
    • desktop
    • enterprise
    • social (and the perils of depending on someone else’s platform)
    • persistent storage (mysql, mongo, cassandra, etc)
  • frameworks
    • MVC
    • Ruby + something (rails, merb, sinatra)
    • PHP + something (Yii, cake)
    • Python + something (django)
    • Javascript + something (so much can go here)
    • iOS SDK + xcode
    • Android SDK + Eclipse
    • PhoneGap (maybe?)
  • product development & maintenance
    • UX
    • waterfall
    • agile
    • lean
    • project management
    • bug tracking
  • coding
    • Git, HG, and SVN
    • hello world
    • dynamic web pages
    • asset handling
    • databases driven applications
    • webservices
    • mobile
    • jobs & queueing
    • desktop (afterthought?  I suppose.)
    • optimization
    • intro to scaling
  • systems
    • becoming a unix terminal pro
    • maintaining software (compiled & managed packages)
    • your stack
    • your first system… or two, or three
    • security
    • monitoring
    • VPS hosting (Linode, etc.)
    • Amazon Web Services (and hopefully some other competitors)
    • colocation
  • scaling
    • scalable code & caching
    • vertical/horizontal
    • DNS & Load Balancing
    • Type of servers: Web, DB, Cache
    • the different systems you’ll use)
  • Your corporate infrastructure (workstations & corporate systems = boring)
  • Appendix
    • Jeff’s ultimate (periodic) list of books for technical entrepreneurs
    • Jeff’s ultimate (periodic) list of websites & blogs for technical entrepreneurs
    • The Stack Overflow Podcasts
    • Tools for remote teams

So there it is, any takers?  Definitely, definitely, definitely feel free to pick this apart in the comments.  I read all comments and would love to crowd-source this into a great outline that I can then use to build a great product.

EndNotes:

[1] – by “startup” I really mean technology startup, and by “technology startup” I really mean young company that will build a product that is composed of machine instructions that will be executed on a CPU.  If your startup is a new steel company (good luck) then this product isn’t for you.  Similarly, if your startup is a dog walking service, you probably don’t quite need a CTO just yet.    

TL;DR? – Yes, yes it is.  But how about changing your business model?  Please read on.

I was reading this blog post over at the appcubby.com blog and thought it worth a little extra narration and argumentation.  The point of the article is to explain why the iOS email app Sparrow and their development team were recently “acquihired” by Google and essentially shut down.  The author uses his own experience with his iOS apps as well as some historical charting and statements made by the Sparrow team to perform a pretty good analysis determining why the team (and its investors) were willing to kill their baby in favor of punching a clock.

In a Nutshell

Sparrow was arguably a very successful app, both in the mac and iOS app stores.  But, due to the dynamics of these app stores, it is very difficult for a dev shop to sustain itself off of the very low one-time payments that are now the the expectation and the norm.  Gone are the days where a dev shop can build an app and sell it for $99, $199, or a $499 one time fee that the user will have to pay again in 2 – 3 years to get an update.  With the expectations set by the $0.99 prices in the original iOS app store, consumers have come to expect the software they purchase to be extremely cheap.  With the success of the first wave of indie developers in the original iOS store the expectations from developers has been, “if you build it, they will come” (and I’m gonna be rich!).  The dime-novel-esque stories of the iOS developer who makes $250K a month on his $1.99 utility have caused the supply of apps in the iOS store to exceed 650,000, while only the top few hundred are able to actually get any exposure to the public.  The convergence of this need to be in the top 100 and the rosy outlook by developers has kept significant downward pressure on prices.  Consumers are generally not going to pick the $49 app over the $4.99 app with a similar name but only one more star in the reviews.  This has also lead to the proliferation of 400,000 zombie iOS apps due to developers who have abandoned their now defunct applications that failed to adequately fill their bank accounts.

Why is this bad?

“So what, I like apps being $0.99.  I wish gas was that cheap.”  This might not seem all that tragic to you, but 400,000 zombie apps, and the fact that you need to remain in the top roughly 250 out of 650,000 in order to make a decent living (according to the aforementioned blog analysis), means that the market has not allocated resources efficiently and tens or even hundreds of millions of dev hours have been wasted and potential innovation has been lost.  Let me put the top 250 out of 650,000 in a little perspective.  A lot has been made in the last year about “the top 1%” and how rich you need to be to be in that top 1%.  It’s generally agreed upon that being in the top 1% of income earners is difficult to do.  Spot #250 on the app store means that you are in the 0.04th percentile.  As in 0.04% of apps have done better than you, and 99.96 have done worse.  This is a huge disincentive to developers.  Many developers, myself included, have abandoned the platform altogether as a viable means of making a living.  Why should I build my apps on this platform only to be put along side 650,000 others and need to remain in the top 0.04% in order to have a sustainable business?  The chances of success starting up a Subway are significantly higher than that.  Now that word has gotten out about just how difficult it is to succeed without already having a network of a few hundred thousand people eagerly awaiting news about your new app, the days of tens of thousands of new quality apps at your fingertips every month are over.

Developers are getting the hint that, it ain’t 1849 anymore. Not only that, but apps like sparrow are no longer being updated because the value prop is just not there.  Why stay splitting $30K a month among 5 team members when you can take a payout from Google AND have a $15K/month salary on to of it?  As time goes by, expect more of these acquihires and if you’re a consumer you can expect a less satisfying experience in the app store(s). At least until the pendulum swings back in the other direction and it once again makes sense to be an Indie app store developer again.

So what’s a dev to do?

The path toward success on the original app store was to just have an app.  Within a year that wasn’t good enough, now you needed to devote some resources to making sure that your app would pop within the first few days of its launch in order to make the charts, and then you would be successful.  Now its not about putting a little resources in, it’s about having massive resources at your disposal.  You need to drive thousands of user to the app store with the sole intention of buying your app, you need to price it low, and you need to have a good app.  But what about pricing it extremely low, as in free?  The new secret to app store success is to simply use the app store as a lead generation source.  Don’t depend on app store revenue, instead depend on revenue from some other thing that you’re selling, for example a subscription-based web app, and just use the app store to generate traffic and leads.  The recipe for software success that I see going forward is not revolving around getting 2,000,000 downloads for $0.99 a pop, but getting 20,000 subscribers who are willing to pay $9.99 a month for your service, or getting 2,000 who are willing to pay $99.  You need to take your prospects out of the “buying software” mindset and into the voluntarily choosing to pay for a service mindset.  Take spotify for example.  They have a free iOS app and a free android app.

But you can’t use them without having a subscription to their $9.99 a month service.  Spotify wouldn’t be able to exist as a $0.99 download when you consider their licensing fees, and no one would be willing to pay per-song for songs that they could only play in Spotify. Plus everyone who is willing to pay per song or album has already developed the habbit of just going to iTunes. But, turn their business into a reasonably prices service and you’ve struck gold.  Cable companies have known this for years, and they too would not be able to survive without subscription pricing (and a whole lot of ignorant learned behavior).  If I walked up to you and said “If you give me $1500, I’ll attach a cable to your house which will enable you to pay $500 in order to purchase a screen that will let me serve up advertisement in your home with a little programming mixed in around them” you’d look at me like I had 6 heads.  ”Ohh and after 2 years is up, you need to give me $1500 again to keep allowing me to show you ads”.  But you take that same proposition and turn it into “Give me $60 a month and you’ll have access to amazing HD content (television sold separately)” and you have a winner.

So, the gold rush is over when it comes to making millions in the app store by just building something useful and letting it fly on its own.  But, that doesn’t mean that us developers need to keep buying into the hype and giving away tremendous value for very little reward.  Re-frame your next business idea in terms of delivering a service that brings in recurring revenue and you can realize the success that you thought your little iOS taking app was going to bring you.

Update: I threw this over on hacker news if anyone wants to discuss this with some hackers.  Though (shameless plug) you’ll probably need to upvote if you actually want discussion  :-)

love the cover

I sat down to write a boring post this AM, but first I swung by TechCrunch for a daily dose of overly sensationalized headlines.  I generally don’t bother with videos but seeing “confessions of a media manipulator” was enough of a headline to at least read the abstract.  The abstract was then enough to convince me to watch the video, the video was enough to make me go take a look at a few blog posts, and the blog was enough to make me go pre-order the book.  In other words Ryan Holiday just made some portion of $12.99, from me, thanks to a 15 minute interview that he did on one of these manipulating blogs and the work he’s done building up his own blog over the last few years.

Wait, bloggers aren’t all pure and angelic?

To some extent we all already know that these bloggers and media outlets aren’t doing things out of the goodness of their own hearts.  We know that people operate popular blogs in order to make money, and very few popular bloggers (who are making money) are doing so without ulterior motives.  Personally, I’m very up front with the fact that I write this blog in order to indirectly and eventually benefit myself (and others).  My narcissism has convinced me that the stream of conscience going through my head occasionally spits out something that other people might find valuable.  I take these valuable bits, figure out how to write something that is targeted at a particular market, and then stick said valuable bit on twitter with an @mention directed at the thought leaders in that target market.  (Note, this is not what’s going on here as Mr. Holiday isn’t exactly a twit-a-holic so the chances of a retweet are slim, and the power of his twitter network is currently somewhat limited.)  My last post: What is a “Minimum Viable Product”? for example was retweeded by Eric Ries and that retweeting resulted in an extra 500 uniques that day.  Five hundred extra uniques in a day isn’t anything to write home about but, as Jim Collins will tell you, a successful business is like a flywheel in need of constant small pushes.  So far I’ve only been using twitter, but there are hundreds of other legitimate techniques out there for spreading your message.  Five hundred visitors here, a thousand there, and pretty soon we’re talking about some real traffic.  The expectation is that if I can articulate those valuable bits, promote my work, and can give enough value to the rest of the world then a few people might think I have a clue about that world and pay attention to what I have to say.  That built up reputation can then be utilized to make a buck if I find certain other opportunities to give value to people through avenues like consulting or creating products.  That’s basically been the game for the last dozen years when it comes to honest blogging.

But…

But, the suggestions from Ryan’s interview about his book suggests that the most popular bloggers and media outlets, and in some cases the supposed thought leaders in their industry, are generally the ones who have been doing the most to deceive their readers.  That is a little disturbing, but undoubtedly true.  In addition these self-promotion initiatives are spilling over into real media outlets via services like “HARO” (Help a Reporter Out) where people with a desire to get their message out and reporters who need something to write about go to meet up and create the news.  ”Wait, what do you mean create the news?”  I literally mean create the news.  HARO has built a user base of journalists who work for various traditional media outlets and a user base of people willing to pay HARO to get their message out.  The people wishing to be promoted (called sources) submit their story and journalists from news outlets such as Fox, ABC, and the Associated Press pick them up and run with them.  This essentially makes it difficult to discern advertisements from actual news because the news is now essentially an advertisement.

Why write and read this book?

The answer given in the interview for why he wrote this book was that he’s gotten bored with being one of the media manipulators and figured he’s pull off the vail in the hopes of making the world a better place, etc. etc.  I’m not going to try and infer what’s going through someone else’s head, but it just so happens that one of the books I’m reading right now is “Free Marketing: 101 Low and No-Cost Ways to Grow Your Business, Online and Off ” by Jim Cockrum.  The chapter I’m in the middle of is talking about the benefits of putting free content online, and how much value you can get out of exposing your own secrets.  The premise is that, whether it’s you or your competitors, sooner or later someone’s going to expose your secrets.  The ancillary benefits of being the person doing the exposing far out weight the cost of them being exposed, and who would you rather have reaping those benefits… you or your competitors?  Jim then goes on to give a few compelling case studies of this phenomenon.  I’d guess that’s part of what’s going on here.  My primary goal in reading this book is to survey the landscape and see what is being done by promoters like Ryan in order to fully understand when I am and am not being lied to.  I want to recognize the tactics that are sleezy and be able to separate the wheat from the chaff in terms of influential online personalities.   I want to survey these techniques and determine which ones I think are immoral, and which ones can be applied or tweaked to work in a way that is not subversive.  I also think that this book will be big this year, so I want to know what I’m talking about when I come across the various articles that will undoubtedly pop up as a result.  (check back in a few days/weeks because I’ll probably write a review)

I decided to write this because I see two basic camps forming when it comes to the idea of creating a Minimum Viable Product (AKA MVP) for your product or service. Bear with me as I proceed to make some broad generalizations that nearly everyone will take some sort of issue with.

The Static HTML Page Camp

The first camp is comprised primarily of people who read books, but does include plenty of people who execute as well.  (The sets of “people who execute” and “people who read books” are not mutually exclusive.)  This group uses the definition provided to us by Eric Reis in his book The Lean Startup to argue that a good MVP is usually comprised of one single, simple, static html page + some google ad words.  Now, let’s look at the definition that Eric proposed: “The minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.”  This is a pretty good definition of what a new startup should be doing at the beginning of its life, but is a static html ad-words landing page a minimum viable product?

The Minimum Viable Prototype Camp

The other camp is comprised of people who have usually spent a significant amount of time building or enhancing products and services.  This group understands that you need insight but thinks that, in most cases, a single static web page does not meet the “maximum amount of validated learning” portion of that definition.  They have typically picked up the terminology of Minimum Viable Product from someone else (potentially well before that book came out) and imposed their own definition based on the fairly intuitive nature of the wording.  They have looked at what successful companies are doing, looked at the time they have available, read numerous blog posts and comments extolling the virtues of a lean prototypes, and have determined that a 1 – 2 month prototype is the way to start a new venture.  They then call this prototype “their MVP”.

 Build, Measure, Learn... repeat.

So, who is correct?

Like everything else in life, both camps are partially correct, it’s just a matter of what you’re doing.

Yes, you want to do an extremely tiny amount of work to validate base assumptions at the outset of any venture.  Yes you want to be lean and agile while getting your prototype to market as quickly as possible.  But whether or not you need a functional prototype for people to use or something as simple as said static landing page is totally, completely, and utterly dependent on the core assumptions of your business.

There have been many books (one, two, three) written that explain how important it is to do your initial validation testing with what amounts to more than a single day’s worth of effort.  This helps prevent you from needlessly venturing down an inefficient path and incurring a huge opportunity cost during a time period where your assumptions aren’t very strong.  But, that acronym “M.V.P.” does include a V, as in viable.  We are validating assumptions, and what if your core assumption is that users will see enough value in a product, once they’ve used it, to share that product with their friends?  Good luck validating that assumption from a static landing page because it actually involves the behavior of a user once they’ve actually used the product.  Similarly, throughout Eric’s book he writes about gathering data from users and using that data to inform your future decisions.  How much data can you gather, that is related to your core assumptions, on a static landing page other than sales conversion statistics?

Many “get rich” books are written with the idea of listing and selling products primarily utilizing Google Ad Words for traffic.   The people who read these books (including the 3 I just linked to) start to think that an MVP is always about measuring the sales conversion rate for a hypothetical product through an Ad Words landing page.  These people also tend to frequent entrepreneurial blogs/message boards and run into people who are creating other types of companies, for example the ever-present, cliché San Francisco Tech startup.  These other companies often times have more involved products, services, and business plans.  They have businesses requiring more than just a high conversion rate on Ad Word traffic, and they need to validate much more complex assumptions.  (Remind me again, what’s your static HTML page for validating the viability of Instagram?).

In the end, when I see good, contributive, common sense blog posts like this one over at techcrunch, it frustrates me that the top few comments are all “well, that’s not really how to build an MVP”.  It’s certainly possible for a static HTML landing page to be an MVP, but it’s not a one-size-fits-all approach to building anyone’s Minimum Viable Product.

To the casual outside observer, AKA me, entering into small-scale retail sales is an intriguing opportunity.  We all live in a retail world where getting 20% off MSRP is a good deal, and 30% off is considered a steal.  I have one friend in particular who has purchased countless items that he didn’t need, or in some cases even want, just because the deal seemed so “unbelievable” at 40, 50, or 60% off.  So you can see why the prospect of getting “wholesale prices” can appeal to a retail business lay-person like myself.  This post will cover the fundamentals of how commerce works and include some ideas on finding products.  If you’re looking for instructions on how to setup an e-commerce website then I might throw together a more comprehensive post on that in the future (just go down to the comments and ask), but for now I would suggest that you just pick the most painless option and use Shopify.  I did: http://jeffjason.com/2012/07/why-im-doing-the-shopify-build-a-business-competition/ (and I’m a developer who already runs multiple websites).

How the System Works

Long ago when a person needed something they would either produce it themselves, or produce something that could be used to trade for that item.  They would head on down to a local market or trading post, bringing along excess goods that they didn’t really need, and exchange those goods for some other item or service that they did need.  Fast forward a few thousand years and this system has evolved into a much more complex and in some ways efficient version of the original, with the production and distribution goods and services being handled by highly specialized entities, and items “changing hands” multiple times on their way between the producers and the consumers.

While it is certainly still possible to produce an item and sell it directly to consumers, as is the case with things like custom built furniture and cabinetry that one might get from a local carpenter, these days that is more the exception than the rule.  Chances are that the desk at which you are sitting or the table at which you ate dinner last night was created by one entity, sold to another, sold to yet another, and then finally sold to you before making its way into your home.  How this works is that the initial producer, called a manufacturer, creates an item like a refrigerator or a television and then, rather than sell each individual item to a consumer, signs deals with wholesalers/distributors for the wholesalers to purchase large quantities of the the item for distribution.  The producer does not have the network needed to distribute the good so in order to get around the limitation of only being able to sell to his limited network he, for the most part, opts to pass on making high margin/low volume sales in exchange for getting the product in front of more people by using this distribution partner.  This of course means that in order for the producer to get the distributor to do distribution for them, the producer is only going to be able to charge the wholesaler a small fraction of what the item would fetch on the open consumer market.  This is of course done with the expectation of making more money overall by moving enough units to make up for the lower profit per item.

On top of that there is still another layer of indirection, however, as the wholesalers don’t typically own the shops where the actual consumer sales take place.  Their core competencies are more centered around signing deals (with producers and retailers), supply chain management, and these days importing/exporting of goods from asia than they are around attracting consumers and selling individual products.  That last part is taken care of by the retailers.  Now this 3 tiered system isn’t necessary in all cases, Home Depot for example combines the wholesale competencies with the retail competencies by essentially opening the doors of a warehouse to consumers and sticking dozens of brightly colored sales staff at strategic points throughout the “store”.  They deal direct with manufacturers because they can support the sort of volumes necessary to strike the same deals that the wholesalers would normally get from the manufacturer, and they deal directly with the consumers because they have built thousands of brick and mortar store fronts in the US and internationally. Fun fact from wikipedia: Home Depot’s largest store, located in Union, New Jersey is nearly the length of three football fields feet long, the length of one football field wide, and comes in at a massive 225,000 ft².

The Dynamics

As is the case with any set of business relationships, there are certain dynamics that spring up when you have businesses of different sizes interacting with each other.  Traditional producers/manufacturers generally don’t want to deal with the hassle of small fish like consumers or retailers and instead  want to focus on their core competency of producing a good quality product.  They don’t want to partake in chasing down 1500 retailers for payment of their invoice and they don’t want to have to develop the skills needed for running marketing campaigns, executing consumer sales, maintaining the operational staff necessary to own traditional brick-and-mortar retail stores, and they don’t own the regional distribution centers that make it easier for distributors to distribute their products.  Unless of course you’re one of the behemoths like Home Depot/Walmart/etc, where they own the entire supply chain.  The advent of the web has reduced some of the operational issues, and there are some manufacturers who have setup online store fronts, but for the most part they still only want to sell their goods in bulk to people who can move large volumes of products.  Similarly the wholesalers would rather not deal with the little guys who don’t account for much of their revenue because each additional retailer you have presumably adds additional support costs that is not proportional to their purchasing.  A wholesaler with 1000 customers who each average $100/month in revenue is much worse off than one with 100 customers who each average $1000/month in revenue.

To give an example from The 4-Hour Workweek, the author Tim Ferris ran a manufacturing operation (actually he outsourced it all, but in effect he was the manufacturer here) where he produced a particular athletic supplement.  He had about 100 – 120 customers (the wholesalers) and after spending a few years running around like a chicken with his head cut off trying to react to every support request he gave up on the idea of working 80 hour weeks and figured out how to automate the business.  Step one was determining what percentage of the business (revenue) could be attributed to what percentage of the customer base.  It turned out that more than 80% of his revenue came from less than 20% of the customers (the Pareto principle is a theme throughout the book), and that most of his revenue came from the same 5 – 10 wholesalers.  So why then was he running around responding to support requests from the other 110 customers?  This line of thinking is what keeps wholesalers and manufacturers from selling to Joe Six Pack who only wants 1 or 2 units of some item per month, and leads to minimum quantities and other qualification requirements to do business with them.

Drop-Shipping

Drop-Shipping

Finally, the retailers have the competing goal of all wanting to get as close to the manufacturer as possible because each additional person in the chain between them and the producer takes an additional cut.  This of course and adds overhead which makes the items they are looking to sell necessarily cost more.  To give a hypothetical example with completely fake numbers, a manufacturer might produce a product for $10 of cost to them and sell 1000 unitis of it to a distributor for an average price of $40 per item.  That distributor might turn around and sell that item to a retail store owner in quantities of 20 items for $80 per item, and the retailer will turn around and list that item for $124.99 for a single one.  The retailer would love to buy his 20 item lots from the manufacturer for $10 instead of $80, but the manufacturer has no interest in selling 20 item lots for the same price that they would sell 1,000 items.  Recently, with the advent of online e-commerce and in some cases “ebay retailers” there have been a ton of noobs throwing their hat into the mix (I would unfortunately have to count myself in that category right now).  This has caused the advent of a new layer of faux wholesalers who are providing no service other than marking up products and making sure that their “wholesale” membership websites show up high on google search results.  Ohh, and finally there is drop-shipping, which is usually done by the wholesalers.  This is basically outsourced inventory and shipping management for businesses who are looking to be totally virtual and strictly deal with advertising and making the end sale, but don’t want to handle anything between the sale and the producer.  This is obviously the lowest margin option, but it makes some things that might otherwise not be possible (like running an online e-commerce site when you have multiple kids and a fulltime job), possible.

So what’s a new retailer to do?

Something else maybe, lol.  In my case I’m not going to do something else, I’m just going to solve the problem.  As I mentioned in a previous post I just entered the  Shopify Build-A-Business competition, for a number of reasons, and I’m not about to back out of that just because of not yet having something to sell.  The rest of this post is going to be centered around ideas that I am brainstorming for how to get products to fill the virtual shelves and do so at the lowest prices with the least amount of inventory.  I will then write a subsequent post with the result of having attempted the ideas mentioned or the reason for why I decided not to attempt an idea.  (For example, see the mention of World Wide Brands below.)

So, finally, here is my not-so-comprehensive list of ideas for obtaining products.

  • Google – So the first step in just about any information gathering exercise these days is to throw a few words/phrases into google and see what she spits back out.  In this case I’m thinking of searching for things like “finding wholesalers”, “buy direct from manufacturer”, “wholesale pricing”,   “purchasing inventory”, “starting up an e-commerce site”, “starting up an ebay store”, “getting started on shopify”, “getting started with e-commerce”, and anything else that comes to mind.  One thing to note is that this exercise will inevitably result in me finding a bunch of non-wholesalers who buy from wholesalers and re-sell at a marked up price that they are call wholesale.  I’ll have to determine whether someone is a real distributer/wholesaler of one of these faux wholesalers.  This activity will probably also supplement my list of ideas, but you have to be weary of what you find on the first page of google results for things like “finding wholesalers”… because it’s information that is available to everyone.  Due to supply and demand, it’s generally not a good idea to do what everyone else is doing.
  • Contact the manufacturer.  Products that I want to sell are invariably for sale elsewhere on the internet.  In most cases I should be able to use Google Shopping and Amazon to find retail listings for these products and the listings should have the name of the manufacturer on them.  You can then google the manufacturer, find their website, and get their contact info.  In most cases they will shoot you down if you are looking for them to sell directly to you at a small e-commerce site’s volume, but they are generally more than happy to give you contact info for one or more of their distributors.
  • Find a distributor first.  Since it can be hard to get a distributor to give you a good price, or even work with you, if you don’t have high volumes (you know, like when when you’re first starting out), you might need to expend significant effort to find a distributor who is willing to work with you.  I’m thinking that I might be able to find a distributor first, and then shape my retail offering based on what they distribute.  The idea being that, you don’t have the volume to support only buying one product from them, but if you sell ten of the products they distribute you might be a big enough fish for them to invite you to dinner.
  • Go to ebay and see what products ebay sellers are selling. Since ebay is the wild-west of online retail it makes some sense to take a look at what ebay sellers are selling.  These people will be half pros and half amateurs, but if the amateurs can make a buck then you and I should be able to as well.
  • Find trade publications.  Like anything else I’m sure there are some sorts of publications aimed at retailers and wholesalers that can be leveraged to find the resources these people need.  Not sure what they are just yet, but this will likely be part of my search.
  • Find tradeshows.  Same reasoning as above, just with “expos” that might be swinging by my area that I can drive out to and peruse.
  • Find message boards and other online wholesale/retail communities.  Once again same as the last two, but this time in electronic form.
  • Wholesale pre-screeners (like worldwidebrands) – I’ve heard mixed things about this route.  Online bloggers love to suggest them (and of course provide you with affiliate links), but if you research what people’s opinion is of them it’s hit or miss, mostly miss.  They common complaint is that there isn’t enough supply of good wholesalers through worldwidebrands to meet the demand from retailers who use them, so you end up with essentially a watered down list of wholesalers who are going to give you inflated “wholesale” prices.  On the other side of the isle is the fact that it’s not exactly easy to put together a list of distributors who are all willing to do drop-shipping and will les you buy in reduced quantities.  Do I want to drop $300 on joining this club?  Probably not, but it’s at least worth researching.
  • Find bargin-bin items at places that you already know are cheap, or cost $0. Yard sales, things you own but don’t use, flea markets, consignment, things owned by friends of yours, etc. can all be stuck online and sold in a “bargin-bin” type area to make a quick buck.  Don’t sell junk though, you don’t want to get bad word of mouth going.

OK, that’s all I got for now, I’ll determine the success of these tactics and writeup a future post about it.

I signed up with Shopify today to enter their Build-A-Business competition.

Why Not?

The first reason is because… why not?  Looking at the Shopify Pricing Page the lowest level store is only $29/month + a 2% transaction fee.  This means that even if I sold $0 in products between now and the end of the competition, I will have lost at most $210.  Even a store that is for the most part a failure should net more than $200 in income over the next 7 months.  In addition the signup and administration process through Shopify has basically zero friction.  I’ve setup and tested Magento before for someone else and while it gets the job done, the experience was a little painful (even for a developer).  I also found Magento to be quite bloated and slow.

Experience

I’ve spent a decent amount of time learning and testing how to sell ideas using various marketing channels, but I’ve never actually testing selling a product.  I want to add value to the world, through my blog and other ventures, and what better way to do so than through things that I have not yet done.  Selling products is definitely something where I am not an experienced veteran so participating in this promotion should give me a better understanding of the business world, while helping me appeal to a wider audience.

Access to a Mentor

I’ve read books by two of the four mentors.  Tim Ferris, best known for his book “The 4-Hour Workweek” and his blog, is a self-promotion legend who was able to turn a little snake-oil into a multi-million dollar business.  He then followed that up with a NYTimes Best Seller and was voted Wired magazine’s Greatest Self-Promoter of All Time in 2008.  In other words, Tim knows how to do a lot with a little.  I considered entering this just based on the fact that I think Tim would be an interesting person to interact with though, if you have read one of this books then you know that, it’s likely you’ll just end up interacting with one of his outsourced virtual assistants.  The person I ended up picking is more of an up-and-coming star in the startup/entrepreneurial world.  Since releasing his book “The Lean Startup” last fall Eric Ries has become a bit of a thought leader in the realm of “how to run your early stage startup”.  The methodology he developed, as the CTO of a small virtual-world avatar-based startup, is simple: prototype ideas, test, gather data, make informed decisions, prototype ideas, test, gather data, make informed decisions, lather, rinse, repeat.  In addition you need to track certain derivative metrics that are a better at indicating the relative health of the company, and pivot when you are not making a positive enough affect on those metrics.  What I’m hoping to get from Eric is some specific suggestions on how best to set up my analytics.  FYI, I own and have read have both of these books cover to cover.  If you haven’t read them–you should.  Eric’s book gets a little repetitive toward the end, and Tim’s book is 10% crap.  But neither of those things detract enough to make them not worth owning.  Actually come to think of it, these are the only two books that I’ve bought in both audible and printed text format.

E-commerce Can (eventually) Be Put On Autopilot

Like everyone else, I’m busy.  E-commerce does take work to setup, but after the initial setup it is as easy of a business as there is to put on auto-pilot.  Yes, the margins might not be quite as high as some other ways of making money, but the hourly pay rate can be as good as it gets.  When you have many other projects going on, income streams that are more or less able to be put on auto-pilot are a nice thing to have.

Additional Exposure

An e-commerce store is yet another place to get myself some exposure and refer traffic to my blog.  I haven’t quite come up with the precise specs for how I will do this, but owning another online property certainly can’t hurt in that respect.

I Might Just Win

Given the number of entries in past competitions, it’s a long shot.  But I might just win.  Ohh, and signing up gets you $100 in free adwords, and $100 in free mail chimp.   Plus they throw in a free $12.99 domain which brings the total promotional value to just over that aforementioned $210.

Been meaning to spend a little more time researching kickstarter, so here goes. I went to kickstarter.com and I’m greeted with a “staff pick”. Ok, I’ve always thought staff picks were lame anywhere but a site like CNet where all they’re doing is reviewing the top X digital cameras and whatnot. But, I understand why a business, when it comes to their home page, wants to direct users toward revenue maximizing opportunities.

In any event I then scanned the categories and thought “OK, lets see what they have for technology projects”. I clicked on technology and saw a link that read: “See all 50 Technology projects”. Awesome, from there I should be able to browse through some list of projects with filtering and sorting or something. At the very least I’d expect to “See all 50 Technology projects” rather than more things like “staff picks”, lol. Instead, I’m brought to a page where there are 4 sections: Staff Picks (sigh), Popular This Week, Recently Successful, & Most Funded. Disappointed with their decision to make me click at least an extra time to get my list of 50 technology projects, I look around for the “browse” or “view all” link. Hmm, nothing. I can “see more staff picks”, “see more popular this week”, etc. Fine, I’ll let you hold my hand rather than giving me the ability to just browse the open projects. So I think, “what would I like to see?” Answer: Lets see what they have for projects that have not met their goal. Maybe I’m weird, but projects that have already reached their goal are not particularly interesting to me, nor are project that have already been completed. Looking at the aforementioned sections:

  • Staff Picks – OK, Not to beat the dead horse too much, but this is at the top of the page rather than somewhere less significant, where it belongs. This means that their staff essentially gets to pick the winners and losers? (i. e. who shows up at the top of each category page, and who ends up buried somewhere.)
  • Popular This week – OK, this is a start. There is one of the 3 that they gave me that is not already past 100% funding, but many are already funded, and many are likely just last week’s “Staff Picks”. (note: this is actually the way to get to all 51 open projects.. via clicking “See more popular projects”.  Was that intuitive to you?)
  • Recently Successful – These are all already funded by definition. Also, they likely spent significant time as a staff pick or “popular this week”. Either way, not what I’m looking for.
  • Most Funded – All already funded, all completed, and all useless to me.

I get that they are trying to maximize the chances of a person pledging money to a project so that they can take their cut. I get that people are most likely to pledge to projects that other people have already made the decision to pledge to (same idea as an e-commerce site putting the most popular products on front pages). I get that a pledge going to a project that will not be funded instead of one that will or already is funded is essentially lost revenue for Kickstarter. But, please… just give me a list of all of the projects in the category.

Where is the page where I can “See all 50 Technology projects” and do basic things like sorting/filtering of those 50 items? Does it exist? I don’t know. Because it’s certainly not the page I’m given when I click “See all 50 Technology projects”.

This is the story of my first encounter with giving people what they want, marketing, the power of exponential growth, and providing the best service.  It is also the story of how I learned to not depend on infrastructure that was controlled by people whose goals were not aligned with mine.

To Set the Scene

Let us take a trip back to my sophomore year in college.  Napster had just gotten shut down, Kazaa was the file sharing equivalent of a war zone (where an innocent looking DIVX download of a file named something like “Gone in 60 Seconds.avi” might end up being a snuff film), and BitTorrent was a mostly un-used infant technology.  What did exist, and worked pretty well, was a piece of file-sharing software named Direct Connect: http://en.wikipedia.org/wiki/Direct_Connect_(file_sharing).  Direct Connect consists of client software, server software (called hubs), and some number of central servers which contain lists of hubs.  The main list (at least the one I always connected to) had maybe 200 hubs and each hub contained anywhere from 0 – 500ish clients.  The idea of DC is that a user signs on with their client software and rather than connecting to a central server, à la Napster, they pick a hub from some list.  Once the user is connected, the hub handles search indexing, chat multiplexing, etc, etc.

In the Beginning

It was fall of 2001, I was 18, and after a summer of waiting tables for $7 – $10 hour (including tips) I decided that the $2000 I had left was burning a hole in my pocket.  Naturally that meant that I should build a new $2000 computer.  So, I started with a 1400 Mhz AMD K7 with 133 FSB, threw that in with an entire Gigabyte of “DDR” ram (WOAH a Gig!), added an ATI Radeon All-in-Wonder card w/TV Tuner & Video capture, bought a $400 20″ CRT monitor that weighed at least 50 lbs (while overheating my room), and then headed off to school.  The previous semester a friend of mine from WPI had introduced me to Direct Connect.  I thought it was cool, both the file sharing aspect and the community aspect, so I decided to spin up a hub on my new machine.  For the first day or two I had anywhere from 0 – 5 people on at any given time and I was a little disappointed that I wasn’t getting more traffic.  This wasn’t a business venture, I wasn’t trying to make money, I just wanted the community.    So I said to myself: “self, how can I get more users?”.  The answer I came up with was pretty simple: “Well, self, why don’t you make sure that your hub has the things that people want.”  This was a very simple idea, and one that would turn out to be pretty powerful (once I got the word out).

So what did people want?

I fired up DC++ and spent a few hours spelunking around the DC server list.  I took a look at the top hubs for trends that might indicate why they became popular.  I went onto the chat rooms and watched what people were asking for.  From this research I noticed that the hottest items were 3 pieces of yet-to-be released software.

  1. A release candidate developer preview of a cracked version of Windows XP
  2. The video game Aliens vs. Predator II
  3. and finally the game Civilization III.

So I took the time to find all 3 of the installers, download them, and verified that they all worked.  This took me a day or two because the most popular hubs were capped at a certain number of users so it was hard to connect to them and it was hard to find these pieces of software anywhere but those most popular hubs.  I then stuck the installers on my machine, connected to my hub, and waited for the flood.  It never came.

Ohh, you need to tell people what you have?

With my new software offerings I was doing a better job of retaining users, but not a very god job of growing.  Now the 2 – 3 new users who went to me hub each day would say, but after a week that rate of 2 – 3 new users had only climbed to 4 – 5 new users a day.  The problem was the obvious oversight that I forgot to update my hub description after upgrading my service offering.  So I put a “Guaranteed to have AVP2, CIV3, and Win-XP” in the description of my hub, connected to a few of the more popular hubs that didn’t seem to have AVP2/CIVIII/XP, threw a message out there that I had those things, and went to bed.  The next morning I woke up with 50 users (the 25 I had previously, and 25 more).  I was like “wow, cool… it worked.”  I went to class, came back and had 50 more users.  ”Awesome, more then twice as many as this morning” By that night I was up to 200 users and noticed that this growth was progressing exponentially.  After a day or two I had joined the other people are the top of the hub list with my user cap set at 500.  I also noticed that this wasn’t smoking my machine, and I still had adequate network available to do more.

Sitting at the Top

I was on a college network, meaning my up/down rates were actually set by the hardware (a guaranteed 5Mbit) rather than arbitrary cable company caps (whose upload caps were, at the time, typically in the 1Mbit range … and not guaranteed) I had the network capacity available to beat the competition.  I had also just build a new machine that summer (and no one on DC was sitting there with load balanced server farms) so I was able to handle more requests and index more clients than anyone else.  After running at 500 users for a few days, with no noticeable degradation in service, I decided to bump it up to 600.  Still no degradation, lets try 700.  Eventually around 800 users I noticed that my machine still wasn’t slowing down, but I had taken up available network capacity.  So I kept things at 800 and had what was, at the time, the largest hub in the world.  As an aside, I just took a look at some current hub lists and the “Maxusers” attribute in the xml list files are set as high as 15,000 for newer machines with good connections.

Providing Better Service

Now that I was on top, I wanted to enhance my service offering so that my users were getting the best without having to leave and go elsewhere.  We brainstormed a few ideas (I say “we” now because I had cultivated a set of about a dozen “ops” [think moderators] who would help me run the hub) and ended up doing things like recruiting other hubs to be part of the “UMT” network and handle overflow, coming up with lists of must-have software/divx movies, and making sure that discussion in chat was not going south or being overtaken by spammers.  I like to think that our hub developed a pretty good reputation in the community.  From there things got pretty ridiculous.  I was an athlete in college and my coach asked me to supervised the varsity weight room at night.  One time I was walking through there and I overheard a conversation where a couple guys were talking about how the umassthrower hub had all the best stuff on it.  I mentioned that it was my hub and they both looked at me start struck.  I guess that moment was my 15 minutes of fame.

And the Cracks Began to Show…

After my machine was accidentally shut down one afternoon while I was at class or practice, I came back to a ton of emails/IMs with my ops trying to get a hold of me.  I rebooted the machine, everything was fine, and by the next day I was back up to 800 users.  In the meantime, my ops were legitimately angry at me for the two hours of downtime.  In fact they even insisted that I needed to give them my cell number so that they could call me if the hub went down when I wasn’t around.  It’s somewhat astonishing how much people care about free services in which they have invested a significant amount of time.  Thanksgiving break was where I saw the writing on the wall.  I lived about 120 miles from school and went home for thanksgiving on Wednesday.  Over the weekend the hub went down and I didn’t have any means of getting it back up.  I went back to school on Sunday and my machine was off (must have been a powerfailure?) so I booted it up and watched as I went back up to about 200 users that night.  But over the next week I only went back up to 400 users and never again reached that 800 mark.  I knew that Christmas break was coming, it was December now so I was in season for track, and I knew that the hub wasn’t going to stay up through the 5 weeks I would be gone.  I also knew that my parents cable provider was not going to be able to handle the upload rates necessary to index that many users.  Finally, I had started to come to the realization that I could get royally fucked by providing this free service.  There was some news of the first few college students getting hit with huge lawsuits… and loosing.  My family was not well off and outside of a 1985 F250 that I had bought for $500, I had nothing of value to be sued for, but the idea of a $250,000 judgement against me for doing something as seemingly innocent as sharing a file was a little concerning.  So I prepared for things to go away… and they did.

The End

Half way through break my machine, which I had left in the dorm to give people the maximum amount of time they would have left with my hub, got disconnected.  When I started my machine up that spring semester, for some reason, I wasn’t able to get on the network.  I went down to OIT (Office of Information and Technology) and waited in line for help.  When I got to the front the person asked for my student ID (still SSN back then, lol) and after punching a few things in asked me to step over to the side and he called a supervisor over to talk to me.  They had shutoff my network access due to me using up too much off-campus network resources (surprise, surprise).   Turns out I was the biggest offender on campus for eating up off-campus bandwidth, and I was responsible for 6 terabytes of off-campus data transfers that previous semester.  At current rates that would have cost them about $750, but it was probably more expensive back then.  I didn’t have to pay because I didn’t break any official rules.  They hadn’t written the rules about off-campus networking until that semester, and the spring was the first time that they were going into affect.  But, the paperwork I had to sign said I promised to stay under 1GB/day of off campus transfers.  Faced with the prospect of having to pay real money, that I certainly didn’t have, to keep the free service alive and the possibility of being sued for providing that free service I closed up shop and resigned myself to the idea that “it was fun while it lasted”.

i2Hub

In case you were wondering I had nothing to do with i2Hub, the service launched by another UMass CS student.  Wayne Chang‘s i2Hub was a branch of direct connect that was closed to anyone who wasn’t on a whitelist of certain university IPs.  I think I used the service once or twice, but I pretty much stopped filesharing after my hub was gone and students started getting sued.  In the end it was probably best that my hub got taken down as a number of UMass students and Wayne himself ended up having to deal with a lawsuits related to file-sharing.

I have found myself somewhat overwhelmed lately.  My last couple years of college I worked 35 – 40 hours a week while taking 16 – 20 credits each semester + then I added 50 hours of work and 4 – 12 credits during the summer.  This meant that when I got out I felt like I had a ridiculous amount of free time.  As a result I found myself constantly looking for side projects.  I spent a year building out my own web application framework (a wonderfully fulfilling, educational, and completely useless/impractical experience).  I spent a year running a friends landscaping business (a wonderfully fulfilling, educational, and completely useless/impractical experience).  Then I hooked up with a bootstrapping startup named goingout.com (a wonderfully fulfilling, educational, and TBD experience).  Over this same period of time I bought a house, moved into it, married my college sweetheart, stood by as she did all of the work to bring our daughter into this world, bought another house, moved out of the first one into the second one, rented the first one, stood by as my wife once again did all of the work to bring my son into this world, and then found myself, once again, with no time.  As a result I have decided to prioritize my life and come up with a schedule/routine to get shit done.

This is part I, defining the list.

 

The List

In no particular order, here are the things that currently take up most of my time.

  • Mental well-being
    • Beer
    • Maintaining friendships – Weekly meetup with friends
    • Recreation
  • Work
    • Fulltime job – I’m sure I don’t have to tell you all what this entails.
    • Startup (business contributions) – as a CTO & cofounder I consider myself every bit as important as anyone else when it comes to building the team’s business knowledge and guiding us in the right direction.
    • Startup (non-development technical contributions) - scalability, security, hiring, managing the other developers, convincing people that we don’t want to use red for things that are not errors or alerts… you know, those sort of things.
    • Startup (development) – self explanatory
    • Income Property - I own a multi-family that occasionally needs to be rented and maintained
    • Consulting services – It would be a waste for me to NOT partake in this.
  • Rest - Sleep – 4 – 7 hours a night
  • Mentorship – I spend some time mentoring young developers and entrepreneurs.
  • Self Promotion (online presence)
    • Blogging
    • Facebooking
    • Twittering
    • etc.
    • Social Analytics – This is basically dedicating resources (time/money) to understanding the best ways to market using social
  • Learning
    • Tech – I try to spend some amount of time each day/week making sure the the world is not passing me by in the technical realm (usually consists of reading high scalability & hacker news)
    • Business – This is basically an exercise in trying to understand everything there is to know about business, that’s all.
  • Exercise/Diet
  • Husbandly Duties
    • Chores – exactly what you think
    • Working on the house/cars – I am a hands on kind of guy.  My portfolio includes a year doing electrical work, about 700 sq’ of framing, 1500sq’ of hardwood flooring, a couple bathroom renovations, brake jobs, replacing radiators, alternators, fuel pumps, electric window motors, etc.  I just don’t trust mechanics that I’m not related to, got screwed one too many times in college.
    • Yardword – about 2 hours a week during the summer, and this turns into snow removal in the winter
  • Family Time/Parenting – This is one I’d like to maximize, but often times falls by the wayside

So, now that I’ve got my list the next posts will be about prioritization and scheduling.

I went out to lunch with a few developers yesterday, including a fresh college grad names James.  Jim’s quite outgoing, as far as developers go, to the point where a previous recruiter asked him if he was interested in doing technology sales.  When we sat down at Chipotle  James threw a statement out there that was particularly well aimed considering my interests.  ”So if you guys want to volunteer any investing advice, feel free to just put it out there.”  This sentence fluttered through the air, into my ear, and flipped that little switch inside me labeled “desire to enthusiastically mentor young people.” My lips immediately jumped into action and I was soon filling their heads with the basic knowledge about what to do with your money that they don’t, but should, teach you in school.  This got me to thinking about how little I knew about money and finances when I was one of these wet behind the ears twenty somethings (as of this posting, I have 6 more months left of being a dry behind the ears twenty something).  There I was with a salary more than 3 times anything I had ever made before, and with no idea how to spend money.  I went out and bought a brand new car, surprise surprise. I was renting a fairly expensive apartment in Needham, MA (a very expensive community).  I was not contributing to my 401K, with the excuse that I had some left over credit card debt from college (which was true, thought I wasn’t exactly paying it down).  And, even though I was taking home more than double what my take home pay was for a 40 hour week 2 months earlier, my bank account curiously stayed the same each month.

This experience has inspired me to give some advice to recent college graduates in the finance arena. Expect a future post on the subject.

Setup my Blog

06.20.2012, Comments, Journal, by .

If you count ramblings about love from 2004, on some blogging platform of which I don’t even remember the name, this blog is my fifth.  The first one was started because I needed to get things out of my head.  The second one was a wordpress.com blog about politics, the economy, and other mostly useless wheel spinning, mental-masterbation-type posts from a 24 year old who had just joined the fulltime workforce and was trying to make his mind up about the world.  The third was like the second, but this time I moved to a shared hosting provider in order to be able to install my own plugins.  The fourth was intended to be a place for me to add notes about technical things I had learned, but with the goal that people would find them useful and I would end up with a bunch of traffic and make a bunch of money from relevant advertising.  (lol, yea, right)

What did I learn from blogs #1 – 4?

What I quickly realized with blog number four was that, for the most part, the “striking it rich because people like your blog” ship had already sailed.  That’s not to say there is no value in blogging.  Blog articles are great for driving traffic to other projects that make you money.  But, given the risk/reward proposition (opportunity cost considering how long it takes to put together quality blog posts vs. the benefit you receive from having built a great blog), using a blog for the sole purpose of revenue is highly likely to result in you making much less than the minimum wage.

So then, what’s the purpose of blog #5?

Well the details of that will become more clear as this web log matures.  In short, there is a correlation between successful blogging and successful people, especially in the areas of technology and entrepreneurship.  A blog unto itself is fairly unremarkable but… A blog, along with active social habits, along with the ability to produce great content, along with something to promote, and along with tools like buffer helps allow you to catch eyes.  Catch enough eyes and you increase the likelihood of success for just about any endeavor in which you choose to partake.  Btw, if you’re trying to synchronize your social media postings then go checkout the guys at buffer.   That link does not contain a referral, I am not affiliated with them in any way, I benefit in no way from you going through the link other than the possibility of you being appreciative for having pointed you in the direction of a startup where the founders are doing everything the right way.  I found them through their blogs c/o two posts they made that were trending on hacker news one weekend.  Good stuff.

What are my expectations?

Well, this section that I’m writing in right now (the “journal” category) is intended to hold the mostly worthless-to-you articles that people don’t really care about.  The hope is that I’ll take 10-20 minutes each day to brain dump journal entries that will serve as a pretty cool log for me to personally look back on.  This section will intentionally be rough and potentially somewhat incoherent at times.  Beyond the journal, there currently are spaces for “Tech”, “Business”, and “Reviews”.  The first is because, well, tech is what I do.  I’m a developer, I find technology interesting, and I hope some day to be able to build some awesome robots and home automation.  (Think the nest thermostat, but for everything you do in your home).  I expect that I can write a few tech-centric blog posts that are worth reading and would perform pretty well when put along side the other things that I see on techcrunch or hacker news.  The next category of Business is there because I’ve started to become pretty passionate about learning when it comes to business and entrepreneurship.  The goal of this section is to allow me to crystalize the ideas about business that are constantly swirling around in my head, and put them somewhere that they won’t vanish.  (that reminds me, need to setup backups)  I expect this section to eventually contain some gems that will hopefully be beneficial to many people in the future.  Finally that brings us to the Reviews category.  The intention of this page is to distil the pieces that I learn from the texts and other pieces of media that I am constantly consuming.  I have read seven books in the last six months, and it pains me to think about how much of those I did not absorb because they went in and out in one fleeting month.  My expectations with this section are that I will be able to internalize what I’m learning when reading a non-fiction book, have something to go back to after I’ve read it, have good content that people will want to see and therefore come to the site for, and allow other young entrepreneurs like myself to have something to go by when making purchase decisions for these books.  I will probably embed Amazon product referral links with these book titles, but I don’t expect any significant revenue from them.  To not provide a referral link would just be a waste, lol.

What should you expect?

A good looking, clean website with good content.  That’s simple enough, right?

Room for Improvement

I hope to add a beer section to this blog where I am able to do something like review a beer a week.  My initial thought was a beer a day.  While I almost drink enough quality microbrews to make that happen already, I’m not sure I can keep that up from a posting standpoint while still being able to provide a quality review.  I’m also going to add an “about” page with some relevant details to fill in the blanks without you having to head over to facebook.  I’ll probably add a section dedicated to my skillset and what you can do to hire me on a contract basis.  But, that is of secondary concern right now because I currently have a tremendous amount on my plate and simply do not have time for consulting.  Finally, I’d like to add a page that is comprised of all the awesome things I come across on the internet each day.  This can serve to act as a bookmark repository for me and other people who are interested in the same things that I am.  I think I can accomplish this with the press-this plugin, but I haven’t verified that yet.  Hopefully I’ll come up with a few more things, but that’s what I have for now.

Above all else…

Above all else, I need to make sure that this blog stays genuine.  The people I read who have the most impact on me are those who seem truly genuine in their writing.  I have facebook comments enabled so you should be able to post with ease.  Feel free to call me out if it looks like I’ve ventured away from this ideal.

So I couldn’t find anything good online for converting a google maps search radius to a zoom level.  Google’s docs said that their zoom levels followed a geometric series (not in so many words) so I jumped in the way-back machine, dusted off the old math degree, and performed the simple high-school level algebra/trig needed to calculate this.

Step 1: Figure out miles to pixels.

So no you can’t really calculate the scale of a google map on your monitor because that depends on your resolution, but you can calculate the distance between 2 pixels a certain number of pixels apart in the grid.  So I went on over to http://gmap-pedometer.com/ and did a rough calculation of pixels to miles at zoom level 14 because 14 just seems about right for figuring out a mile radius on my screen.  Conveniently enough for me, since the map I’m working on is 500px wide, 500px at zoom 14 is about 2.3 miles across.  This means that if I want to show a 1 mile search radius on a 500X500 map I simply need to set it to zoom level 14 and it will show just slightly more than the 1 mile.

Step 2: The series

So zoom level 14 is a 1 mile radius search, that’s nice, what about 15, 16, 13, 12, …  Well from the google documentation we know we have a geometric series:

Maps on Google Maps have an integer “zoom level” which defines the resolution of the current view. Zoom levels between 0 (the lowest zoom level, in which the entire world can be seen on one map) to 21+ (down to individual buildings) are possible within the default roadmap maps view.

Google Maps sets zoom level 0 to encompass the entire earth. Each succeeding zoom level doubles the precision in both horizontal and vertical dimensions. More information on how this is done is available in the Google Maps API documentation.

What does that mean?  Zoom 0 shows approximately 25000 miles across what looks like 1000 pixels on my laptop’s screen. If I go to zoom 1 then that same screen area shows 1/4 as much of the globe, 1/2 the width and 1/2 the height.  In terms of search radius, we’ve cut the search radius in half because we could search x miles in any direction and now 1 level closer we can only show x/2 miles (on a square screen map region obviously).  So what does this series look like in the more practical zoom levels?

16 = 1/4 mile, 15 = 1/2 mile, 14 = 1 mile, 13 = 2 miles, 12 = 4 miles, 11 = 8 miles, …

Not convinced?  Go on over to the aforementioned site and test it out.  500px on my screen at zoom 16 gave diameters of  .577 miles, zoom 15 gives 1.15 miles, and zoom 13 gives 4.4 miles.  Seems pretty consistent.

Step 3: The Function (show your work)

//15 = 1/2, 14 = 1, 13 = 2, 12 = 4, 11 = 8, 10 = 16
//-1 = 1/2, 0 = 1, 1 = 2, 2 = 4, 3 = 8, 4 = 16
//2^(14-zoom) = radius
//ln(x^y) = y*ln(x)
//ln(2^(14-zoom)) = ln(radius) = (14-zoom) * ln(2)
//ln(radius)/ln(2) -14 = -zoom
//zoom = 14 - ln(radius)/ln(2)
//throw  round on there to make sure we get an integer
//and don't make the g-maps API angry
radiusToZoom: function(radius){
    return Math.round(14-Math.log(radius)/Math.LN2);
}

Step 4: Testing

console.log(this.map.radiusToZoom(.5));
console.log(this.map.radiusToZoom( 1));
console.log(this.map.radiusToZoom( 2));
console.log(this.map.radiusToZoom( 4));
console.log(this.map.radiusToZoom( 8));
console.log(this.map.radiusToZoom(16));
console.log(this.map.radiusToZoom(32));

gives:
15
14
13
12
11
10
9

“Bingo, I got bingo, I win again.” -T. Brown

Step 5: Normalization

To use this with an arbitrary size map you would need to normalize for the number of pixels the radius is being projected on.  I am going to leave that as an exerrcise for the reader, since I don’t need it right now :-) .  Hint: a straightforward scaling of the radius should do fine.

Abstract

This quickie tells how to run multiple websites on one server with 1 ip address using Apache Http Server version 2.x.

Prerequisites

  1. Apache Http Server 2.0+  sudo apt-get install apache2 apache2-mpm-worker apache2-utils apache2.2-common (note: php.net recommends mpm-prefork for mod_php enabled servers)
  2. A domain (or more specifically 2, but we’ll quickly hit setting up subdomains)
  3. And two server roots with index files

DNS Setup

First you have to register your domain with your hosting provider’s name servers.  I use Linode and they have a pretty straightforward web utility for setting up your zone files.  For the purposes of this quickie you’re looking to add hostname entries to make your zone files look something like this:

; jeffreyjason.com [102453]
$TTL 86400
@	IN	SOA	ns1.linode.com. umassthrower.gmail.com. (
					2010071855
					14400
					14400
					1209600
					86400
				)
@		NS	ns1.linode.com.
@		NS	ns2.linode.com.
@		NS	ns3.linode.com.
@		NS	ns4.linode.com.
@		NS	ns5.linode.com.
@			MX	10	mail.jeffreyjason.com.
@			A	97.107.129.122
www			A	97.107.129.122
mail			A	97.107.129.122
redmine			A	97.107.129.122
xcache			A	97.107.129.122

httpd.conf setup

Inside your apache configuration file you need to add the following line to turn enable name based virtual hosting

NameVirtualHost *:80

Next you setup the virtual host configuration for your sites.  Here is a minimalist example of 2 of mine with the cruff stripped out:

www.jeffreyjason.com

<VirtualHost *:80>
        ServerName jeffreyjason.com
        DocumentRoot /var/www/jeffreyjason.com/
</VirtualHost>

redmine.jeffreyjason.com

<VirtualHost *:80>
        ServerName redmine.jeffreyjason.com
        DocumentRoot /var/www/redmine/
</VirtualHost>

Finally you restart apache and, assuming your subdomain entries have propagated, you should now be serving 2 sites from one server. The process is the same with 2 separate domains, you will just have 2 separate zone files pointing to the same IP. Before we go don’t forget to add ServerAlias directives to your virtual host configurations in order to have your visitors not end up with an error message when they inevitable try to access www.subdomain.domain.tld.

(This is a popular article from my old blog. Given that about 200 people a month come to this guide I 301′d it here when I stopped writing the old blog. It has been backdated so that the original publish date is what you see above.)

The Amazon Product Advertising API (formerly AWS or AWS-EC) is a way to programmatically perform many of the actions a user can perform on the Amazon.com website. This includes searching for an item, viewing an items details, or adding an item to your cart.  In order to use this API it is required that you register as a registered affiliate in the Amazon Affiliate Marketing program. Any use that is outside the scope of the AM program is expresely forbidden by the TOS.  The data obtained through this API is generally used to, for example, provide a set of dynamically generated custom links for advertising amazon products that are relevant to some user generated content in your application or website.

This API is not to be confused with the other major Amazon Web Services APIs (which are now also called “AWS”) such as AWS-S3 (Simple Storage Service) or AWS-EC2 (Elastic Computing Cloud).  Amazon had formerly named this particular API “AWS-EC” for Affiliate Web Services ECommerce API, but that name was deprecated when they decided to call the aforementioned pay-per-unit cloud based APIs “Amazon Web Services.”  And to top it off an Amazon Affiliate that wants to use the API formerly known as AWS must signup for an “AWS” account since the Product Advertising API is of course an Amazon Web Service.  (Did I mention the reference docs still say AWS?)

Prerequisites

  1. You have obtained an Amazon Affiliate Program account (here)
    • be sure to actually read the TOS for this service because Amazon will terminate if you’re caught in violation
  2. You have installed Ruby, Rails, & RubyGems (here)
  3. You have a basic understanding of programing and Ruby.

Technologies

The focus of this tutorial is both pragmatic and practical.  If you wish to understand SOAP here is the O’REILEY text on the subject.  The short answer is that SOAP is a, for better or for worse, widely adopted specification for data exchange.  Most large companies that provide any sort of API to the outside world provide a SOAP based XML API. For this tutorial we will be using:

  • Ruby (v 1.8.7)
  • Rails (v 2.3.8)
  • RubyGems (v 1.3.7)
  • Ruby/AWS (v 0.8.1)

Additional, less helpful, guides are available from amazon for Perl, Java, PHP, C# here.  Perhaps I will writeup some supplemental plug-in guides for the other versions if there is interest.

Obtaining an AWS Developer Key

Login with your Amazon.com credentialsIn addition to being an affiliate program member Amazon also requires users of their Affiliate Product Advertising API to sign up for an AWS account in order to obtain an access key with which to sign their requests.  The registration page is pretty straight forward.

  1. Verify your location
  2. As Amazon puts it “Success”

Once logged in at the Amazon Web Services home page go to Account >> Security Credentials.  Your newly generated API key will be waiting for you under the Access Credentials Section.

Be sure to keep your private key private!

https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key

Ruby/AWS

Ruby/AWS is the project name of the GPL RubyGem, written and maintained by Ian Macdonald, that handles the lower level communication with tha Amazon SOAP API.  You can think of Ruby/AWS as a tangible implementation of the interface defined by the WSDL.  This means the creator took the datatypes and operations specified in the WSDL and implemented them as classes and methods in ruby code.  As of the time of this writing the Ruby/AWS Gem on RubyForge is version 0.7.0.  Due to the significant re-factoring and the statement that v 0.8.1 “more or less fully  supports” the Amazon AWS v 4.0 API, I choose to install the most up-to date version available at the bottom of this rdoc page.

wget http://caliban.org/files/ruby/ruby-aaws-0.8.1.gem
sudo gem install ruby-aaws-0.8.1.gem

That’s it for the installation, pretty easy ehh. Well there is one more thing you need to do and that’s setup a file named .amazonrc in your home dir.  This file holds configuration information and should look something like this:

key_id = 'someshorterpublickey'
secret_key_id = 'somelongprivatekeythatseemsthislong'
associate = 'some-associate-code'
cache = false
locale = 'us'
encoding = 'iso-8859-15'

Your associate Tracking ID can be found in the upper left hand of your logged in associate main page:

https://affiliate-program.amazon.com/gp/associates/network/main.html

Now that .amazonrc is setup we can write a little script to test things out.

Hello World

(or the demonstrable AWS equivalent)

#!/usr/bin/ruby -w

require 'amazon/aws/search'

include Amazon::AWS
include Amazon::AWS::Search

is = ItemSearch.new( 'Baby',
 {
 'Keywords' => 'pants',
 'MinimumPrice' => '2500',
 'MaximumPrice' => '4999'
 } )
is.response_group = ResponseGroup.new( 'Small' )

req = Request.new
req.locale = 'us'

resp = req.search( is )
items = resp.item_search_response[0].items[0].item

items.each { |item| puts item, '' }

ItemSearch

Starting on line 8 we create a new ItemSearch object and give it our search parameters. ItemSearch’s constructor takes searchIndex which is roughly equivalent to the department dropdown on Amazon.com, with a few extras like “Music” and “Video” or a particular “Merchant Id.” The parameters are the conditions to search on which are once again roughly equivalent to the text input field and the additional filters of Amazon’s search results.  The complete list of search indices and parameters is available in the “AWS E-Commerce Service” API reference (yes, another name for it) entry for ItemSearch.  One thing to mention about ItemSearch is that you should be sure to check the search index compatibility matrix because not all indices are available on all locales.  The US locale currently supports all but 3 of them, for the other locales support is sparse.

ResponseGroup

A ResponseGroup is amazon’s way of you telling them how much and what data you want.  Valid ResponseGroups include:

  • Small, Medium, and Large
  • Reviews, EditorialReview
  • Offers
  • SalesRank
  • Accessories
  • etc…

You cannot combine ResponseGroups so requests for Mutually exclusive pieces of data will require multiple requests.  There is some limited batching functionality, but you may still come across cases where you need to make more than one request for a page.

Request

The request object is what is going to actually make the request to the web service like you would if you put a URL in your browsers address bar.  It then receives an XML response from Amazon and parses that into the appropriate set of objects for the given Operation.

Terse Version

This package has also made sure we have some common case optimizations built in. In this case the first script was setup with us providing the Amazon specified defaults as our parameters. Since we can leave the defaults out that script returns equivalent results to this much shorter version.

#!/usr/bin/ruby -w

require 'amazon/aws/search'

include Amazon::AWS
include Amazon::AWS::Search

resp = Amazon::AWS.item_search( 'Baby',
 {
 'Keywords' => 'pants',
 'MinimumPrice' => '2500',
 'MaximumPrice' => '4999'
 } )

items = resp.item_search_response.items.item

items.each { |item| puts item, '' }

Output

When you run the script you should see something along the lines of:

...
similar_products = similar_product = title = Pampers Easy Ups Value Pack for Girls, Size 5, 80-Count
 Box
asin = B0027VT9J8title = Pampers Easy Ups Value Pack for Boys, Size 6, 66-Count Box   asin = B0027VT9IEtitle = Pampers Cruisers Dry Max Diapers, Economy Plus, Size 6 (35+ Lbs), 100 Diapers
asin = B00347AFGU
title = Huggies Pull-Ups Learning Designs Training Pants, Girls, 4T-5T, 44-Count Biggie Pack
asin = B003D7LA8I
title = Pull-Ups Training Pants with Learning Designs, 3T-4T (32-40 lbs), Mega, 40 training pants
asin = B000096OKL

offer_summary = total_used = 0
lowest_new_price = currency_code = USD
formatted_price = $25.00amount = 2500
total_collectible = 0
total_new = 5total_refurbished = 0

image_sets = image_set = thumbnail_image = width = 75height = 75
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL75_.jpg
small_image = width = 75
height = 75
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL75_.jpg 

large_image = width = 500
height = 500
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL.jpg

tiny_image = width = 110
height = 110
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL110_.jpg

swatch_image = width = 30
height = 30
url = http://ecx.images-amazon.com/images/I/51%2BUWKsDyGL._SL30_.jpg
...

That’s it.  This is the string representation of the objects that were generated from Ruby/AWS parsing the XML returned from Amazon.  The fruits of your labor have been realized and you can process amazon data (that is relevant for your Affiliate Marketing purposes) till your hearts content.  As long contentment comes before you average 1 request per second.  Did I mention to read the TOS?

XML –> Objects

Because XML makes me hate life you get a direct quote from a subsection of the README:

In Ruby/AWS, each unique XML element name forms a class of the same name. All
such classes are subclasses of AWSObject. For example, OperationRequest is a
class, as is ItemAttributes.

As the XML tree is traversed, each element is converted to an instance of the
class of the same name. Every such object has instance variables, one per
unique child element name. The name of the instance variable is translated to
comply with Ruby convention by adding an underscore ('_') character at word
boundaries and converting the name to lower case.

For example, given the following XML:

<ItemAttributes>
 <Author>Philip Pullman</Author>
 <Manufacturer>Scholastic</Manufacturer>
 <ProductGroup>Book</ProductGroup>
 <Title>The Ruby in the Smoke (Sally Lockhart Quartet)</Title>
</ItemAttributes>

the following statements would all be true:

- ItemAttributes, Author, Manufacturer, ProductGroup and Title would all be
dynamically defined subclasses of AWSObject.

- An instance of the ItemAttributes class would be created, with instance
variables @author, @manufacturer, @product_group and @title.

- To each of these instance variables would respectively be assigned an array
of Author objects, an array of Manufacturer objects, an array of
ProductGroup objects and an array of Title objects. In the above case, these
would all be single element arrays, because there's only one instance of
each kind of tag in the XML.

- The Author, Manufacturer, ProductGroup and Title objects would have no
instance variables of their own, because the corresponding XML elements
have no children, just a value. These objects are therefore directly
assigned the value in question.
Well that was easy, I should teach from the book more often… Thanks Ian, moving on.

Common Errors

“`require’: no such file to load — amazon/aws/search (LoadError)”
solution: don’t forget to set RUBYOPT in your .bash_profile via: export RUBYOPT=rubygems
“Your request should have atleast 1 of the following parameters: Signature, Timestamp.”
solution: This error is the general purpose error Amazon gives you when your request does not appear to have been signed correctly. Since the Ruby API we’re using does that for us the likely cause of this error is that you didn’t set your API secret key correctly in the .amazonrc file.

The README also points out that Amazon considers requests timestamped as being more than 15 minutes old to be invalid, so make sure your server’s closk is being sync’d appropriately.

Thanks

As always Comments are welcome and if you find any gaps in this tutorial, gaps in the Ruby/AWS API, or see that a new version is available, throw me a comment so I can update the tutorial and alert the maintainer.  For more information you can consult the 4000 word README doc in the rubygems directory for this gem.

API References

  • ruby-aaws: (rdoc) (or of course ‘gem server’)
  • Amazon documentation: