@cumulus/ingest
TypeScript icon, indicating that this package has built-in type declarations

18.2.0 • Public • Published

@cumulus/ingest

@cumulus/ingest is a collection of modules for discovering and ingesting data.

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Installation

npm install @cumulus/ingest

Testing

Running tests locally requires localstack.

With localstack running, you can run tests using:

LOCALSTACK_HOST=localhost npm test

Modules

All modules are accessible using require: require('@cumulus/ingest/<MODULE_NAME>') or import: import <MODULE_NAME> from '@cumulus/ingest/<MODULE_NAME>'.

  • consumer - consumer for SQS messages
  • crypto - provides encryption and decryption methods with a consistent API but differing mechanisms for dealing with encryption keys
  • ftp - for accessing FTP servers
  • granule - discovers and ingests granules
  • http - for accessing data via HTTP and HTTPS
  • lock - creates locks for S3 data
  • log - stringifies JS object logs for ElasticSearch indexing
  • parse-pdr - tools for validating PDRs and generating PDRD and PAN messages
  • pdr - discovers and ingests pdrs
  • queue - creates queues for ingesting data
  • recursion - handles recursion of a FTP/SFTP list operation
  • sftp - for accessing SFTP servers

Contributing

To make a contribution, please see our contributing guidelines.

Readme

Keywords

Package Sidebar

Install

npm i @cumulus/ingest

Weekly Downloads

63

Version

18.2.0

License

Apache-2.0

Unpacked Size

632 kB

Total Files

82

Last publish

Collaborators

  • cumuluspublisher
  • nsidc_kovarik
  • jennyhliu
  • npauzenga