nxus-data-manager
Import file contents as arrays of objects. Includes parsers and exporters for:
- CSV / TSV
- JSON
- GeoJSON / ArcJSON
Provides convenience integration with nxus-storage to import and save to a storage model.
Installation
npm install nxus-data-manager --save
Options
All responses accept an options argument, in addition to any parser-specific options you can indicate:
mapping: object of {field: newField} name mappingsidentityFields: for importing to models, array of fields to be used for createOrUpdate querytruncate: for importing to models, true/false for deleting existing collection data before import. Ignored ifidentityFieldsis provided.strict: defaults to true. Only import columns/data that matches the attribute names for the model. Set to false to import everything.
Events
You can modify the records during import with the following specific events:
records.type: e.g.dataManager.after('records.csv', (results) => {})record.type: e.g.dataManager.after('record.csv', (one) => {})models.identity: e.g.dataMangaer.after('models.user', (results) => {})model.identity: e.g.dataManager.after('model.user', (user) => {})
record_ events occur after parsing and name mapping model_ events occur after record events and before models are created/updated.
API
parser
Provide a parser for a particular type (file extension)
Parameters
typestring The type (e.g. 'html') this renderer should handlehandlerfunction Function to receive (content, options) and return parsed array of result objects
exporter
Provide an exporter for a particular type (file extension)
Parameters
typestring The type (e.g. 'html') this exporter createshandlerfunction Function to receive (content, options) and return formatted output content
export
Request formattted output based on type
Parameters
typestring The type (e.g. 'html') of the output contentrecords[object] The records to exportoptsobject Options for the exporter context
Returns Promise String of formatted output
import
Request parsed results based on type
Parameters
typestring The type (e.g. 'html') of the contentcontentstring The contents to parseoptsobject Options for the parser context
Returns Promise Array of parsed result objects
importFile
Request parsed results from a file path
Parameters
Returns Promise Array of parsed result objects
importToModel
Import string contents to a model
Parameters
modelstring The identity of the model to populatetypestring The type (e.g. 'html') of the contentcontentstring The contents to parseoptsobject Options for the parser context
Returns Promise Array of instances
importFileToModel
Import file contents to a model
Parameters
modelstring The identity of the model to populatefilenamestring The filename to read and parseoptsobject Options for the parser context
Returns Promise Array of instances
fixture
Import a data file as fixture data. Can specify environment option to only load for e.g. test
Parameters
modelIdstring The identity of the model to importpathstring The path to a fileoptionsobject Options to pass to data-loader.importFile
API
Import file contents as arrays of objects
Installation
npm install nxus-data-loader --save
Options
All responses accept an options argument, in addition to any parser-specific options you can indicate:
mapping: object of {field: newField} name mappingsidentityFields: for importing to models, array of fields to be used for createOrUpdate querytruncate: for importing to models, true/false for deleting existing collection data before import. Ignored ifidentityFieldsis provided.strict: defaults to true. Only import columns/data that matches the attribute names for the model. Set to false to import everything.
Events
You can modify the records during import with the following specific events:
records.type: e.g.app.get('data-loader').after('records.csv', (results) => {})record.type: e.g.app.get('data-loader).after('record.csv', (one) => {})models.identity: e.g.app.get('data-loader').after('models.user', (results) => {})model.identity: e.g.app.get('data-loader).after('model.user', (user) => {})
record_ events occur after parsing and name mapping model_ events occur after record events and before models are created/updated.
API
parser
Provide a parser for a particular type (file extension)
Parameters
typestring The type (e.g. 'html') this renderer should handlehandlerfunction Function to receive (content, options) and return parsed array of result objects
exporter
Provide an exporter for a particular type (file extension)
Parameters
typestring The type (e.g. 'html') this exporter createshandlerfunction Function to receive (content, options) and return formatted output content
export
Request formattted output based on type
Parameters
typestring The type (e.g. 'html') of the output contentrecords[object] The records to exportoptsobject Options for the exporter context
Returns Promise String of formatted output
import
Request parsed results based on type
Parameters
typestring The type (e.g. 'html') of the contentcontentstring The contents to parseoptsobject Options for the parser context
Returns Promise Array of parsed result objects
importFile
Request parsed results from a file path
Parameters
Returns Promise Array of parsed result objects
importToModel
Import string contents to a model
Parameters
modelstring The identity of the model to populatetypestring The type (e.g. 'html') of the contentcontentstring The contents to parseoptsobject Options for the parser context
Returns Promise Array of instances
importFileToModel
Import file contents to a model
Parameters
modelstring The identity of the model to populatefilenamestring The filename to read and parseoptsobject Options for the parser context
Returns Promise Array of instances
_storeResultsWithModel
Simple do-storage function to help mock up storage for tests, other applications.
Parameters
model[type] [description]values[type] [description]uniqueCriteria[type] [description]
Returns [type] [description]
export
Stringify an array of results into JSON. Assumes top-level is array, unless opts.key is provided to wrap results in an object.
Parameters
parse
Parse JSON into an array of results. Assumes top-level is array, unless opts.key is provided to pick a top-level key from parsed object as results.
Parameters