Independent DPDP educationBrowser-only workspace · no accounts, analytics or submissions

AI and product decisions · DP087

Search indexes and embeddings belong in your deletion map

Map a source document’s searchable and derived copies so an erasure decision reaches the relevant index, metadata and cached outputs.

A precise platinum aperture guides a small violet beam through a clear prism, separating the chosen path from unused inputs.

Deleting a file from a document folder may leave its contents available through search. A system may have extracted text, divided it into chunks, created embeddings, stored metadata and saved an answer that quoted the original. Each artifact needs an identified place in the lifecycle.

Build the deletion map when the search feature is designed. It is much harder to reconstruct the relationship between source records and derived artifacts after a request arrives.

Use a source reference that survives processing

Give the source a stable reference and carry an appropriate link into its chunks, indexed records and related metadata. Record which process creates each artifact and which team owns its removal or restriction.

The reference should support the task without unnecessarily exposing an identity in every technical record. It can still be personal data where it is linkable. Under section 2(t), the relevant question is identifiability by or in relation to the data, not whether a field looks technical.

Do not assume an embedding is anonymous simply because it is a vector rather than readable text. A primary embedding-inversion study reconstructed text and personal information under its tested conditions; that demonstrates a risk, not that every vector can always be reversed. Assess the associated metadata, links, access and the actual information the system can retrieve or expose. The assessment belongs to the whole arrangement.

As of 22 October 2026, the relevant DPDP erasure duties are scheduled for 13 May 2027 under the commencement notification. The future section 8(7) and section 12 framework must be read with applicable retention requirements, including rule 8(3) and the Seventh Schedule. That future rule requires covered personal data, associated traffic data and processing logs to be retained for at least one year from processing for the specified Schedule purposes, with further retention where required by law or government notification.

First decide which processing should stop, which artifacts should be erased and which records require restricted retention for a justified reason. Then translate that decision into system actions. A technical delete button should not silently decide a legal exception, and a legal decision should not be marked complete merely because one database accepted a command.

Keep retained records out of active retrieval where the assessed purpose does not permit that use. Retention for one reason is not permission to keep serving the content in ordinary search results.

A hypothetical knowledge-assistant deletion

A service business adds customer correspondence to an internal search assistant. One source email produces several text chunks, vector records, metadata and a cached summary. The team uses a synthetic email to test the deletion workflow before relying on it for real records.

The job identifies all artifacts linked to the source, applies the approved action to each and checks the relevant search and direct-access paths. It also inspects the cached summary and any export created during the test. The result records both completed actions and unresolved artifacts.

The team discovers that deleting the source email does not automatically remove a separately stored summary. It adds that artifact to the lifecycle map and assigns a responsible process. The lesson is about the proposed system; it is not a claim about any named product’s automatic behaviour.

Understand the storage service’s boundaries

Read the documentation for the actual index and API in use. As one concrete illustration, Pinecone’s deletion documentation describes operations scoped to documents or records in an index namespace and distinguishes the relevant APIs. It also discusses when changes become visible. Those details belong to that service and configuration; do not generalise them into a universal deletion guarantee.

A successful response can establish that a request was accepted or processed according to an API’s semantics. Verification of the intended absence through the relevant access paths is a separate observation. Do not invent a propagation time or mark an uncertain state as complete.

Prevent deleted content from returning

Review scheduled ingestion, retry queues and restored backups. A source that remains in an import feed can be indexed again after an otherwise successful cleanup. Include a controlled way to preserve the applicable deletion or restriction decision across restoration and reprocessing.

Distinguish an index deletion from model training. Removing a retrieval record does not, by itself, establish that learned model parameters have changed. If training is part of the workflow, it needs a separate map and assessment rather than an unsupported promise of universal erasure.

Keep evidence without recopying the content

A useful completion record can identify the request, source reference, artifact classes, actions, verification results and exceptions. It should not reproduce the full personal content merely to prove that it was removed.

The recurring failure is defining deletion by one storage location. Follow the source into every relevant searchable or derived artifact, apply the assessed decision, and verify what the user can still reach. That is the basis for a credible completion statement.

Follow the source into its derived artifacts

A source produces chunks, index records and cached outputs, each connected to a deletion-verification step.
The deletion decision needs an artifact map; removing the original file alone does not establish the state of every derived copy. Open full-size diagram
Text version of the diagram
  1. Source document: Keep a stable source reference
  2. Chunks and metadata: Track extracted text and identifiers
  3. Search index: Link vectors and indexed records to the source
  4. Cached outputs: Identify saved answers and exports
  5. Deletion verification: Apply the decision and test each relevant path
  • Source document → Chunks and metadata
  • Chunks and metadata → Search index
  • Search index → Cached outputs
  • Source document → Deletion verification
  • Chunks and metadata → Deletion verification
  • Search index → Deletion verification
  • Cached outputs → Deletion verification

Sources and reading context

These references support the original 22 October 2026 research. Source links and the date above make the scope of this article explicit.

  1. Commencement notification for provisions of the DPDP Act · paragraph (c)

    Dates future erasure duties

  2. Digital Personal Data Protection Act, 2023 · sections 2(t), 8(7), 12

    Identifiability and erasure framework

  3. Digital Personal Data Protection Rules, 2025 · rule 8(3); Seventh Schedule

    Retention qualification

  4. Pinecone: Delete data · Scoped record and document deletion; data freshness

    Illustrates index-specific deletion boundaries

  5. Text Embeddings Reveal (Almost) As Much As Text, EMNLP 2023 · Embedding-inversion study and evaluated conditions

    Supports treating vectors as potentially revealing rather than inherently anonymous