Recently, I had a customer ask me to put together an estimate for some work that would "scrape" pages off the internet, resulting in a rapidly-built directory of businesses and customers. The option was to pay a lot of employees a lot of money to go "use the Interwebs" and collect that same information -- instead, a smaller set of people could sift through less information and add more value to these data in the process. I run a development firm, so I immediately jumped to "let's build a program that can do that." Boy, was I wrong!
First, I contacted my Scraper-Builder Ninja (doesn't everybody have one of those laying about?) and we put together a scope of work (in estimate form) to scrape and deliver a particular set of information from eight sites. Wouldn't you know it? They actually have EIGHTY sites to scrape. They were hoping to extrapolate by a factor of ten (80/8=10) based on the count records collected. Where I was thinking in terms of the actual effort to build the tool, they were focused on results. Namely, in filling up spreadsheets with information. Each source site requires its own ruleset for the scraper to do its business *on that site*. Our focus was on the effort required to build that tool.
Who's right? Well, the customer is right, obviously. They just probably had the wrong vendor. You see, I want to build software that does stuff. In the moment of creating this tool, we would make it as efficient as possible and capable of collecting dozens to tens of thousands of records -- scale doesn't matter. What the client wanted was a fixed price, some way to scale in a predictable fashion. My developer-brain thinks, "My GOD, but you could scale cheaply by changing a few characters and running the scraper for a different state!" But my customer is solely business-focused, thinking, "I just want a dollar amount I can budget for every new market."
As a businessman, it would be a slam-dunk to put a number around this effort and charge a fixed amount for, say, every 10,000 records retrieved. Every 10,000 more from the same or similar site would be pure profit for me. In the end, that may be our compromise. I have to say, though, that I'd be making out pretty well in that deal; in the end, I want my client to succeed and I'll help them do that however I can.
So I'm in the process of helping the customer do one of a few things:
- find another vendor who focuses on services like this
- start providing this product as a service
- convince the client that my "just buy the code" approach is cheaper, which will probably go hand-in-hand with the "find another vendor" option
At the end of the day, I'm glad to have these problems. Offering products as a service is definitely a great business model, just not one that I have the staff to manage. But if my clients are willing to learn along with me, I think a hybrid development and services offering can work.
Software as a service is
Software as a service is defintely the way you want to go with Drupal. All of the site duplication + bulk importing/ exporting features make it so that you can really capitalize on unknowing customers. Most people do not understand or believe that they can use automated/ bulk systems for their own information services, so they tend to default to the historical method of paying per task (automated/bulk action systems are relatively new historically since computers are needed to do such, and furthermore they are only now becoming avaialble to the masses through open source software (which for some reason most people do not nkow about....).) So since most customers only dream of doing such activities (since they dont think it is personally possible), you (as the designer/ implementor) have a lot of leeway in your pricing schemes/ methods...
Bulk scraping, bulk importing, and auto tagging truly make the service model more desireable than the development model (if you want to make more money in less time).