Given a large set of CSV files with thousands of paragraphs each, how would you detect duplicates wi

Question

Given a large set of CSV files with thousands of paragraphs each, how would you detect duplicates within each file, and how would you scale this solution for many files?

Given a large set of CSV files with thousands of paragraphs each, how would you detect duplicates within each file, and how would you scale this solution for many files?

Top community answer

Have a better answer?