Seminar on Big Data Generation Techniques for Software Development

Developing an application often requires test data for debugging and system testing. When developing modern applications (e.g., Web 2.0 applications, real-time analytics), we may require generation of big test data (e.g., millions of web transactions per second, terabytes of transaction records, trillions of tweet messages).

This seminar, sponsored by Hong Kong Innovation and Technology Fund, aims to share with the attendance the state-of-art free-and-efficient tools for generating terabytes of test data.  The topics include:

  • Introduction to YCSB and TPC-C — a tool to generate and pose millions of transactions to a backend system (e.g., NoSQL key-value store, relational database).
  • Introduction to BigFrame — a tool to generate the two ‘V’s (volume and variety) of big data.   The variety of data can include a mixture of social network (graph), tweet (text), and e-commerce (relational) data.
  • Introduction to DBScale — a tool to upscale (e.g., x100) your existing enterprise data (for stress testing).

Date: 3 Jan 2014