node-file-dedup
A node module that removes duplicate lines from one or more files.
The output file will be sorted.
Example
const nodeFileDedup = require('node-file-dedup'); nodeFileDedup.dedup([ 'input1.txt', 'input2.txt', 'input3.txt' ], 'output.txt', function() { console.log('Files merged and deduplicated lines');});
API
The API is pretty straight forward.
nodeFileDedup.dedup(files, output, callback)
- files: either one file or an array of files
- output: the output file
- callback: is called when deduplication is finished