Big Data is a term applied to data with a size such that commonly used tools are unable to capture, manage, and process it within a tolerable elapsed time. Big Data sizes are a constantly moving target, currently ranging from a few dozen terabytes to many petabytes of data in a single set.

In this talk we will discuss how web sites such as TripAdvisor use Hadoop and associated projects -- like Hive, Oozie, and Sqoop -- combined with SQL Server. These tools provide data warehouse teams with the ability to process vast amounts of log information, the prominent source of "Big Data." The talk will also cover how we integrate with SQL Server for end-user reporting, as well as some of the challenges with working with these new and bleeding-edge projects. And, believe it or not, TripAdvisor does this all with a team of just three engineers.


Don O'Neill is the Senior Director of Engineering, Operations and Infrastructure at TripAdvisor, overseeing the company's software, hardware and networking infrastructure and operations, and planning and executing its scalability strategy. Don has eighteen years’ experience in Software Development mostly on the Microsoft platform. Don has a Masters degree in Software Engineering from Brandeis University.