The ins and outs of data mining analytics on digital data"Data Discovery: Tell me something that I don't know" is the definition of data mining - discovering unexpected patterns and relationships in data. In this session Neil explores the approach to insight generation through data mining and predictive analytical technologies. Using real world case studies he covers the ins and outs of data mining analytics on digital data, which types of techniques can be used to solve which kinds of problems and some of the challenges that you will inevitable face along the way. Discover what your data can tell you if you ask it the right questions.
Data Mining Process and Predictive AnalyticsIn Part I (out of three), Neil discusses the difficulty of extracting signals from all the noise present in the overwhelming quantity of data we deal with nowadays. While we now have tools for free and the cost of collecting data is diminishing, Neil questions: "what do we do with all this data?" He introduces other tools that can help us to address some of these challenges:
- Data Mining is about discovering things we don't already know. It is about uncovering patterns and relationships in data that we may not have already thought about.
- Predictive Analytics is using that understanding to think about what might happen in the future; it is about applying those historical patterns to predict those future outcomes.
- Business Understanding: what problem are we trying to solve? What is the business trying to achieve?
- Data Understanding: do we have the data to be able to answer this questions? If not, what is the cost of acquiring that additional information?
- Data Preparation: all data is dirty and needs to be cleaned and transformed. This is the heavy lifting stage.
- Analysis & Modeling: the tools must be chosen based on what the business is trying to understand and the data available.
- Evaluate Outcomes: how well does the model actually works from a statistical point of view (significance) and from a business point of view (actionability)?
- Deployment: driving the insight into the business.