Major Goals Realized for Game Price Drop

Good news for the Game Price Drop project. All major features have officially been implemented. It has a lot to be smoothed out but, on the whole, that it's gotten this far is a big deal.

First off, the dataset has expanded considerably. The MySQL database now includes Images, link, and release date information. Also migrated in from the web service world is information on Xbox 360 and Wii titles. The Gamepro API served as a resource for review data that also sits in the database. Efforts expanded the Python code base considerably making it ripe for some refactoring.

The application has a working web front end. The server side of it is PHP based with some front end scripting done in JQuery. JQuery is an easy Javascript library to pick up. Going from knowing nothing to having an Ajax front end with image transition effects is a testament to the quality of work the developers put into this library.

The project has seen environment changes too. It was born on Vista with Xampp serving as the Apache server. Everything now sits in Linux and runs on that Apache now. Also added to the development process was version control.  Mercurial brought peace of mind into the project. Using a VCS based on a distributed repository model made sense.  There's nothing to the project that needs the central server that subversion or cvs would require.

Overall, this has evolved into an interesting end-to-end development project. There is plenty of work to do for refining this project. It's difficult to say when or if this will go live on the web. Making it fit for public usage, though, is a worthwhile goal.

Game Data, AWS, and MySQL Challenges

Okay, here is my first report on the results of my hundred minute hack experiment.  The good news is the database has video game data from Amazon and I was able to stuff those results into an Excel file.  Getting data into a web page is next.  The pressure is on to find work in case I can't turn this idea profitable somehow.  There's good news even in that regard since I now have good samples I can use to market with to prospective employers.  Now for the details...

A third of the time spent on this project was getting that first information request to Amazon.  That involved grappling with encryption, url parameter manipulation, unicode issues, and other stuff that Amazon Web Services requires in order for you to get an information request through to them.  Eventually, I shifted strategies and turned to APIs for Amazon and MySQL.  After some gotchas with versions, platforms, dependencies, and so forth, the first request from Amazon to Python to MySQL made it through.

As data was concerned, a "cheap game" was defined as a physical PS3 game costing anywhere between a penny to 49.99.   Getting that data presented difficulties.  Just asking for game data causes Amazon to spit back a whole lot of Playstation console bundles, controllers, and so on along with game titles.

Building a Playstation hardware dataset helped even though it meant two more rounds of web service requests.  One round was needed for the hardware and accessories while a separate one was needed for game controllers.  A MySQL trigger cleared unwanted hardware from the game dataset.  The trigger approach made sense because it removed bad data from the game list once it was discovered rather than burden the system needlessly with subqueries and table joins at every single request.

So there you have it.  Several hundred web service requests done in 3 parts leading up to a collection of cheap games.  All told, that amounted to somewhere between 500 and 750 web service requests which translates to 5000 to 7500 records processed.  All this done to amass a collection of 341 Playstation 3 titles.

I have quite a bit to do on this project and I look forward to seeing where all this leads me.  I'll share as things progress.

First Steps

When my contract ended recently, I decided to try my hand at self-employment and build something profitable. I don't know what I'm going to build let alone how I'm going to make money on it but I figured “Why not?”

So without a product or a clue how to run a business, I figured I'd start with an interesting idea and a productivity technique to drive me forward.

The interesting idea is a database of Playstation 3 video games that keeps track of prices. The whole idea is that it would a nice list of all titles, prices, and when those prices change. Beyond that, I don't know.

The productivity technique I came up with is called the “Hundred Minute Hack”. The basic idea is I throw together a todo list of four or five things I'd like to do in 100 minutes. Then I set a timer and I go at it and I don't let myself stop for anything short of coffee or a bathroom break.

The four or five related todo items is a pretty nice trick. If I get sick of doing one thing, I can just do something else on the list. It's a handy way to keep myself from being bored, frustrated, or intimidated by the project.

I'm committing to doing these 100 minute runs two to three times a day. Some flexibility is important and sometimes I have to settle for 50 minute runs. But that's what's pretty sweet about the whole system. On a bad day, I still find I've spent over two hours hacking away on my project. On a good day, I get up to five.

The entries that follow this will probably be a mix of progress reports and little tricks I'm picking up along the way. It's hard to say where this experiment will lead. In the best case scenario, I have a profitable business. In the worst case, it'll be a fun topic to bring up during job interviews.