Skip to main content

Guide · Data and knowledge

Data foundations for useful AI

Build a source register, resolve ownership and freshness, and test the information that an AI workflow depends on before connecting more tools.

The short answer

A useful AI workflow starts with information the business can identify, trust and keep current. Define the task, register its sources, name the owners, establish permissions and reconcile the meanings of key fields. Then test retrieval and outputs against known cases. The first deliverable is a source-readiness brief: what can enter the workflow, what needs repair and who owns the next step.

Start with a decision and a useful output

Begin with the question the workflow must answer. For a customer renewal review, the team needs the operative agreement, payment status, service issues and renewal deadline. It does not need every document the company has collected. Define the required output before choosing an index, database or model.

LEXKIT maps the information behind that decision and the systems responsible for it. We use the map to specify the source register, entity relationships, access boundaries and checks that the workflow needs. This gives the business and engineering teams a shared basis for implementation.

Build a register that names the authority

For each source, record its system, owner, access approver, record identifier, classification and intended use. Include the refresh method, last successful update, retention rule and the process for removing access. A path or connector name alone does not explain whether the source is suitable.

Separate the source of record from a convenient copy. An exported invoice sheet may be useful for analysis, but the billing system still owns payment status. Label the export’s coverage and date so that a later payment is not mistaken for an unpaid balance.

  • Source and business owner
  • Authoritative fields and stable identifiers
  • Read scope and access approver
  • Refresh, retention and removal rules
  • Known gaps and the person resolving them

Resolve names, entities and business meaning

A customer can appear under a legal name in billing, a trading name in the CRM and an abbreviation in meeting notes. Define how those records connect. Preserve source identifiers and approved aliases; send uncertain matches for review rather than merging records because their names look similar.

Agree what important fields mean. “Revenue” can mean a signed deal, issued invoice or received payment. “Owner” can mean the current service contact or the person credited with the original sale. Keep those concepts separate. A technically successful join can still produce a misleading business answer.

Make freshness and precedence explicit

Set the age limit for each input according to the decision. A policy changes differently from an account balance. Record both the source event time and the time your system collected it. A recent import can contain old information.

Define the rule when sources disagree. A signed amendment can supersede a clause; a later email may only ask for a change. Keep the original records and the reason for selecting an operative value. If the rule cannot resolve the conflict, the output should show the disagreement and name its owner.

Carry permissions through the whole workflow

Apply the source boundary to search results, extracted passages, generated answers and saved outputs. Check that a user cannot recover restricted information from a preview, a cached answer or an exported report. Test removal as well as access: a revoked source should no longer appear through a derivative path.

Keep private operational knowledge separate from public visibility research. A source approved for an internal decision is not automatically approved for a public prompt, a training dataset or a client-facing output. Record the allowed use instead of inferring it from availability.

Test quality with known cases

Choose questions with known answers and include difficult records: a duplicate customer, a missing attachment, an unreadable scan, a revised policy and a stale export. Verify that the output uses the correct source, preserves important qualifications and exposes missing information.

Keep extraction checks separate from reasoning checks. First confirm that a date, amount or clause was read correctly. Then check whether the workflow interpreted it correctly. A low-confidence scan can explain an extraction failure; it does not justify inventing the missing text.

  • Correct entity and source version
  • Required fields present or visibly missing
  • Units, currencies and dates interpreted consistently
  • Source passages support the answer
  • Restricted or revoked records stay outside the result

Turn the findings into an implementation brief

Group sources as ready, ready with conditions or held for repair. State why, assign a responsible owner and define what clears the condition. For example, a billing export can support historical analysis while remaining unsuitable for current-balance decisions until its refresh process is agreed.

The handoff includes the source register, entity map, precedence rules, access decisions, quality checks and unresolved issues. Use those artifacts to build the next workflow. Review them when the source schema, business process or owner changes; the data foundation is part of operating the system.

Worked example

Sample artifactSource-readiness brief: renewal review
Source
Billing export used alongside the customer agreement and service record
Authority
Billing system owns payment status; the export is a dated analytical copy
Entity rule
Join on the approved customer identifier; route unmatched names for review
Freshness
Show source event time and collection time; hold current-balance decisions when the refresh condition is unmet
Access
Read only the approved customer scope; keep the export outside public prompts
Readiness decision
Ready for historical analysis; current-balance use awaits the finance owner’s refresh rule
Next action
Finance owner confirms refresh requirements; implementation owner tests stale and duplicate records
Sample readiness record. The sources, conditions and decisions are illustrative implementation examples.

Checklist

  1. Name the decision and required output before collecting sources
  2. Register the owner, authority, identifier and allowed use for every source
  3. Distinguish original records from copies and exports
  4. Define entity matching and send uncertain joins for review
  5. Agree business meanings for amounts, dates, status and ownership
  6. Record source event time, collection time and refresh conditions
  7. Set precedence rules and keep unresolved conflicts visible
  8. Test permissions across search, answers, caches and exports
  9. Test duplicates, stale records, missing files and unreadable material
  10. Deliver a readiness decision with a named owner for each open condition

Review responsibilities

Business owner
Defines the decision, useful output and meaning of the information used.
Source owner
Confirms authority, quality, freshness and the permitted use of the source.
Data and integration owner
Implements entity mapping, updates, lineage and quality checks.
Access reviewer
Approves scope and tests restricted, revoked and derivative access paths.
Workflow owner
Accepts the readiness brief and resolves conditions before dependent actions proceed.

Limitations

  • An organized source register does not make incomplete information complete; unresolved gaps must remain visible.
  • A successful connection does not establish authority, correct business meaning or permission for every use.
  • Source-backed answers can still be wrong when extraction, interpretation or entity matching fails.
  • Quality and access change over time. Repeat the affected checks when sources, owners or workflows change.

FAQ

Do we need a data warehouse before using AI?

Choose storage for the task. A bounded workflow may use existing systems and a small approved index. Cross-system history and analysis can require a broader data platform. Decide after mapping the sources, update needs and operating responsibilities.

Should every document enter the knowledge base?

No. Include sources that serve the defined work and have an approved use. More records can add duplication, stale guidance and permission complexity without improving the output.

Is connecting data the same as training a model?

No. Retrieval supplies relevant information to a task. Model training changes model parameters. Specify which technique the workflow needs and how its data is handled rather than using the terms interchangeably.

What happens when two systems disagree?

Preserve both records, apply the agreed authority and freshness rules, and show any unresolved conflict. Assign an owner when the rule cannot determine the right value.

What do we receive from a data-foundation engagement?

A source and ownership map, entity definitions, access boundaries, freshness and precedence rules, quality checks and a prioritized implementation plan. The scope follows the business workflow and its systems.

Put the guidance to work

Bring the business goal, systems, source owners and operating constraints. We use the conversation to identify a useful scope and the next architecture or implementation step.

Discuss your architecture

Describe the system or process you want to connect. HAI reviews each request and agrees the scope with you.

Next guidePlatform operations: decide who owns the running system