Digital information · Lesson 38 · Approx. 7 minutes including practice
Data cleansing and validation
Validate duplicates, required values, reference integrity and business rules before import. Separate confirmed corrections from uncertain records. Keep an exception log and have the data owner approve changes to master data.
Step 1: Validation rules
A documentation workflow, not an equipment operating sequence. Animation starts only when you choose Play.
Separate confirmed corrections from uncertain data
Cleansing should improve usability without hiding uncertainty. Start with validation rules approved by the data owner: required fields, uniqueness, reference integrity, allowed values and business relationships. Some errors can be corrected deterministically, such as an agreed formatting rule. Others require a decision, such as whether two similar suppliers are actually the same legal entity. Keep uncertain records in an exception workflow rather than silently merging them.
Prepare and check the work
Retain the original data and log each applied correction or rule version. Compare before and after counts and review representative changed records. Treat missing values, duplicates and invalid references as different issue types with different owners. Avoid removing a record simply because it fails an import. Quarantine it, explain the failure and obtain an authorised resolution. Re-run validation after changes because a correction can introduce a new conflict.
Worked example · demonstration only
Two suppliers share a trading name but have different registered identifiers. Do not deduplicate them by name alone. Ask the business owner to verify the entities and decide how they should be represented in the target system.
Guided practice and review evidence
Use only a demonstration dataset for this exercise: Identify duplicates and missing parent references in a small sample dataset. Before starting, write down the question the finished record should answer and identify the source information you would need on a real job. Keep a separate list of missing inputs. Prepare a first draft, then compare it with the topic-specific checks above. Ask another person to follow your references without explaining them verbally. Record where they hesitate, what information they cannot locate and any different interpretation. Keep a brief review log with four columns: observation, proposed action, responsible reviewer and closure evidence. Revise the example and repeat the information-finding check. Your evidence of completion is the revised demonstration record plus that log, not simply time spent reading. Do not upload client information to the academy or practise on operating equipment.
What good completion looks like
A useful result can be understood by someone who did not prepare it. They can identify its purpose, applicable item, source and review status, and they can distinguish confirmed information from unresolved questions. Pay particular attention to this failure mode: Do not merge records just because their descriptions look alike. Check the output in the format the recipient will use. A native file, spreadsheet and exported PDF can expose different problems. Keep the source revision and the issued output linked, and record any limitations of the handover. On a real project, follow the customer's authorised checking and approval arrangements before the information is used. The knowledge check below tests understanding of this lesson; it does not assess practical competence or authorise regulated work.
Practical workflow
- Establish the source: identify the approved input, document revision and person responsible for its technical content. Record missing information as an open question.
- Prepare a small representative example before applying the approach to a whole project. Check identifiers, units, references and the intended reader’s needs.
- Ask the nominated reviewer to check technical meaning. Capture the response, resolve comments and issue only under the agreed document-control process.
Review checklist
- Can another reader identify the source and applicable revision?
- Are assumptions and unresolved decisions clearly recorded?
- Is there evidence of the required review before issue?
Educational material and demonstration examples. Not accredited training, practical competency assessment or authority to perform regulated work. Use current approved sources and your responsible technical authority on real projects.