Skip to main content

Personal Identifiable Information and sensitive information tagging through metadata scanning and profiling

Utilize existing open source libraries ( https://microsoft.github.io/presidio/structured/ ,

https://pypi.org/project/pii-Scanner/ , or similar) to identify columns likely to contain PII during the metadata scan or profiling, and then add tags to the data element to allow clients to filter and flag columns that may require additional handling or security. This could also allow for tagging for the type of data like phone numbers, emails, address data, or more.

This is an extremely common request we get from clients when demoing Catalog, metadata scanning, and profiling. This data could then be carried over into snapshot management, dataset design, mapping, and reporting in Migrate.

Log in to comment and vote

No comments yet

Be the first to share your thoughts.