Preparing your Data for GenAI

Process

The processing of data can be supported with GenAI in a way that it can assist you in the cleansing of data which will increase the data quality, and the processing time.

Note: A data quality of 100% will also not be reached with GenAI. For mission critical applications you would still need a human in the loop.

Risk: When processing data, particularly when employing systems or models that do not belong to you, it is crucial to anonymize the data. Consequently, while utilizing GenAI for data processing, there exist various situations that may result in the inadvertent disclosure of information such as excessively transparent models or inadequate and unsuccessful anonymization techniques. Such instances might enable sensitive information to fall into unauthorized possession.

Do:

• Data Validation: Verify the data processing steps to prevent data leakage.
• Data Quality Assessment: Regularly assess data quality after processing to ensure data integrity.

Don’t:

• Excessive Data Alteration: Avoid excessive data manipulation that may compromise the original data's accuracy and integrity and quality. This can be done through generating clean data or replacing missing or erroneous values.

Analyze

Utilizing data synthesis, GenAI has the capability to offer supplementary data points for analysis or facilitate the investigation of hypothetical situations, provided that the data follows recognized patterns.

Risk: It is recommended to exercise caution as this may result in overfitting, potentially producing data that does not accurately represent the genuine underlying patterns. Additionally, it is important to make sure that any generated data is adequately labeled or documented to avoid misinterpretation.

Do:

• Cross-Validation: Use cross-validation to assess the quality and generalizability of generated data.
• Validation Metrics: Establish appropriate validation metrics to evaluate the generated data's fidelity.

Don’t:

• Overreliance: Don't solely rely on generated data; combine it with real data for comprehensive analysis.

“Data hallucinations” are situations in which the AI model produces completely made-up or wrong outputs, often without any connection to the training data or input given. These results can be misleading, irrelevant, or even nonsensical. Hallucinations may arise due to limitations in the model's training data or structure and can pose problems in applications where accuracy and faithfulness to the input data matter a lot, like natural language processing or image generation. For instance, let's consider a text generation tool powered by generative AI. It might experience hallucinations by generating sentences or information that is factually incorrect or unrelated to what was provided as input. Tackling and minimizing these hallucinations remains an ongoing challenge during the development of generative AI models because it is crucial for ensuring trustworthy and reliable content generated by AI systems – especially when precision and truthfulness hold utmost importance within specific applications.

Store

Through using generative models it is possible to compress, thus reducing the storage space needed for certain datasets without losing essential features needed from the data.

Risk: The possibility of storing more data puts you at risk from a data retention point of view. 

Do:

• Data Retention Policy: Implement a clear and compliant data retention policy, especially for generated data.

Don’t:

• Insecure Storage: Avoid storing data in unsecured locations or formats.

Share

In the sharing process GenAI can be used to create synthetic or anonymized versions of data for sharing, preserving privacy while allowing third parties to use representative data.

Risk: The act of sharing sensitive information, even when adequately anonymized, may potentially result in violations of privacy or unauthorized exploitation for unintended objectives.

Do:

• Consent and Agreements: Ensure that data sharing adheres to consent agreements and legal requirements.
• Data Minimization: Share only the necessary data, keeping it as minimal as possible.

Don’t:

• Uncontrolled Sharing: Avoid sharing data without proper consent, agreements, and control mechanisms.

Archive

Concerning archiving data, GenAI is able to assist you in before mentioned ways, like summarizing, compressing or analyzing archived data. Moreover, it can aid you in categorizing and arranging stored information. Through training models to identify patterns and content within the data, GenAI is able to automatically assign relevant keywords or labels to facilitate easy searching and retrieval of archived information.

Risk: If not carefully managed, GenAI may inadvertently alter or distort archived data, potentially leading to data loss or degradation.

Do:

• Data Preservation: Either make use of GenAI to enhance data conservation by generating duplicate copies, recovering damaged data, and guaranteeing that the data will be accessible in upcoming formats or systems.
• Data Compression: Or Utilize GenAI to decrease storage requirements through compression techniques that maintain the accuracy of information.
• Data Transformation: Employ GenAI to convert data into standard, open, or long-lasting preservation formats for future compatibility.
• Regular Quality Assessment: Regularly evaluate the quality of stored data using Generative AI technology to identify deterioration or irregularities.
• Data Summarization:  Leverage GenAI to generate brief summaries or outlines of stored data, improving its usability.

Don’t:

• Data Leakage Oversight: Ensure that GenAI models used in archiving are properly configured to prevent data leakage or the exposure of sensitive information.
• Lack of Data Recovery Measures: It's important to not depend exclusively on GenAI for restoring data without incorporating extra backup and recovery tactics.

Delete

GenAI has the capability to determine which data should be removed, when it should be deleted, and for what reasons according to specific policies. Additionally, in cases where sensitive data needs to be erased, GenAI can produce substitute information in order to preserve the integrity of databases and applications that depend on the eliminated data.

Risk: When using GenAI in the data deletion process it can leave residues, so that the data can be retrieved by unwanted parties. Also if not properly configured it can delete data which shouldn’t be deleted and are irretrievably lost.

Do:

• Secure Deletion Techniques: Ensure the safety of sensitive information by employing secure techniques for data deletion, such as overwriting or cryptographic deletion, to prevent any potential retrieval.
• Dependency Analysis: Before carrying out deletion tasks, thoroughly examine data dependencies within systems and applications to identify possible consequences.
• Data Replacement: To maintain the structure and functionality of systems that rely on deleted data, utilize GenAI when necessary to generate replacement data.

Don’t:

• Incomplete Deletion: Avoid incomplete or insecure data deletion methods that leave traces of data behind.
• Rushed Deletion: Take your time during the process of deleting data; rushing can lead to mistakes. Assess dependencies carefully and guarantee a secure deletion procedure.
• Data Deletion Without Backup: Don't delete data without a proper backup or data recovery plan in place, especially if the data is critical or irreplaceable.
• Deletion Without Documentation: Always document the entire process of deleting data, including reasons for deletion and any actions taken.