Davies Meyer – home
    Data3 min read

    Data Mining

    Data mining systematically examines data for patterns, anomalies and relationships using methods including statistics and machine learning. Findings may support new questions or predictions, but do not automatically explain causes or produce a useful recommendation for action.

    Data Mining explained

    Start with the decision the investigation should support. Do you want to understand recurring order combinations or find unusual steps in a process? Check which data actually represents that question. A large collection offers little advantage if periods, definitions or recorded events do not align. Keep origin, missing values and relevant changes traceable.

    Let the task determine the method. Clustering forms groups using selected similarities; classification assigns predefined categories. Regression estimates numerical values, while association analysis searches for co-occurrence. These are different tools. A discovered customer group is not a natural, permanent property of everyone assigned to it.

    Check patterns against data that did not already determine their selection or development. For predictions, use only information that would have been available at the prediction point. Otherwise an apparently accurate forecast may simply exploit later knowledge. Correlation also differs from causation, and an unusual order is not automatically fraud.

    Creative Engineering connects verified analysis with an understandable application. We take responsibility for the concept and quality. An observed order combination might suggest a useful recommendation. Test that recommendation separately rather than booking the association as revenue impact. Include data maintenance, implementation and possible decision errors in the effort.

    Examples

    Hypothetical application

    A retailer studies craft-material orders and discovers frequent combinations. The team checks whether an existing promotion created them and whether they also appear in another period. A resulting complementary-product recommendation is tested separately. The joint purchase alone does not establish additional value from the recommendation.

    Key Points

    • Assess the question and data together.
    • Choose methods according to the claim needed.
    • Validate patterns separately from their later application.

    Practical application

    Define a concrete question, check the data and select a suitable method. Validate the finding and investigate the effect of a resulting intervention separately.

    Useful measures

    Testable patterns

    Assess stability and meaning using appropriate further data.

    Task-specific errors

    Record consequential misclassifications or prediction errors.

    Application value

    Investigate the implemented intervention’s outcome, including effort.

    Common mistakes

    • Presenting arbitrary patterns as business insight.
    • Accidentally including later information in a forecast.
    • Confusing model training with evidence of intervention impact.

    Sources and context

    • IBM: Data mining

      Foundations of pattern analysis, method selection and interpretation limits.

    Frequently Asked Questions about Data Mining

    No. Patterns can suggest hypotheses. A causal explanation needs an appropriate study design and suitable assumptions.

    Loading related terms…

    All Terms