csv-concurrent-reader

CSV Concurrent Reader

Introduction

This project provides a concurrent CSV reader capable of processing CSV files line by line, concurrently executing actions for each data row.

Installation

Using npm:

$ npm install csv-concurrent-reader

Using yarn:

$ yarn add csv-concurrent-reader

Usage

import csvConcurrentReader from 'csv-concurrent-reader'

const filePath = path.resolve(__dirname, "path", "to", "your", "file.csv")

async function main() {
    async function callback(data) {
      // for each data row  
    }

    await csvConcurrentReader(filePath, callback, 100)
}

main()

Overview

Main function: csvConcurrentReader(filePath, callback, concurrency, queueSize?, options?) accepts several parameters:

Parameter	Type	Default	Description
filePath	string or ReadStream	-	CSV file path or a Node.js ReadStream.
callback	function	-	A callback function executed for each data row.
concurrency	number	-	The number of functions to execute simultaneously.
queueSize	number	10000	The total size of the internal queue.
options	object	-	CSV parsing options. Refer to the documentation of csv-parser for more details

Dependencies

• fastq: A lightweight, high-performance task queue.
• csv-parser: A CSV parsing library.