cky-mongo-decay

1.1.1 • Public • Published

cky-mongo-decay

cky-mongo-decay is a tool that helps you to split your mongodb into smaller mongodb to serve the needs of fragmented storage. Use environment DEBUG=DecayMongo for show log debug

All bug or issue please send to email: Chickyky@gmail.com

Features!

  • split your mongodb into multi mongodb with your special condition
  • use mongoose package for upsert document into your pieces mongodb

Installation

$ npm install cky-mongo-decay --save

function

  • contructor (options)

    • fnDecay *** (require)***: The function used to filter the document into mongodb you want
      • example:
        fnDecay: (obj) => {
            return moment(obj.publishDate).utcOffset(420).format('YYYYMM');
        }
    • schema *** (require)***: Use for upsert your mongodb
    • modelName *** (require)***: Model name, use for upsert your mongodb
    • piecesOfDecay *** (require)***: Object setup your pieces mongodb with :
      • key: result from fnDecay, which you want your document was filtered
      • value: Connection string of your piece mongodb
      • example:
        piecesOfDecay: {
            '201907': 'mongodb://user:pwd@host:port/db1',
            '201908': 'mongodb://user:pwd@host:port/db2',
            '201909': 'mongodb://user:pwd@host:port/db3',
            '201910': 'mongodb://user:pwd@host:port/db1'
        }
        
    • limitDecay: ******: limit async run in parallel (lib use package async), default is 5
    • stopDecayWhenError ******: if you want skip error when upsert into your pieces mongodb then set it is false, default is true (Stop and return error callback when error happen)
  • init (callback)

    • Run first for init this lib, it will process connect all connection of your peices mongodb, callback when done with format (err, result)
    • callback: error when one of host connect fail
    • after init success, you can use this lib
  • decay (findObj, arrObjDecay, callback)

    • findObj : use for query upsert,
      • example:
        {
            slug: 1
        }
        
      • this example above mean: query key slug with value of document you want upsert into your peice mongodb, it will become:
        {
            slug: 'this-is-slug-for-find-query-of-mongodb'
        }
        
    • arrObjDecay : list multi or one document you want insert to your peice mongodb, if you pass Array then result in callback is Array[String] else is String
      • example:
        {
            "slug" : "this-is-slug-for-find-query-of-mongodb",
            "updatedAt" : new Date("2019-11-21T16:55:00.794+07:00"),
            "createdAt" : new Date("2019-11-21T12:19:02.454+07:00"),
            "title" : "Hello world!",
            "publishDate" : new Date("2019-11-27T14:00:00.000+07:00")
        }
        
    • callback : format (err, result)
      • err: error when process fail and flag stopDecayWhenError is true
      • result: is Array[Object] if arrObjDecay is Array else Object. Element in Array is object have field decayId is new ObjectId mongodb, in case you use stopDecayWhenError = false and upsert error then decayId is null.
        • example:
          [
              {
                  "find": {
                      "slug": "slug-1"
                  },
                  "decayId": "5dd83b2bd82b317c61758d61"
              },
              {
                  "find": {
                      "slug": "slug-2"
                  },
                  "decayId": null
              },
          ]
          // or
          {
              "find": {
                  "slug": "slug-2"
              },
              "decayId": "5dd83b2bd82b317c61758d61"
          }
          // or
          {
              "find": {
                  "slug": "slug-3"
              },
              "decayId": null
          }
  • close (callback)

    • Close all connection of your peices mongodb you was setup when init
    • callback is function (err), err != null when close error

Usage Example

const moment = require('moment');
const DecayMongo = require('cky-mongo-decay');
 
const decay = new DecayMongo({
    fnDecay: (obj) => {
        return moment(obj.publishDate).utcOffset(420).format('YYYYMM');
    },
    schema: require('./FeedSchema'),
    modelName: 'Feed',
    limitDecay: 1,
    stopDecayWhenError: false,
    piecesOfDecay: {
        '201907': 'mongodb://user:pwd@host:port/db1',
        '201908': 'mongodb://user:pwd@host:port/db2',
        '201909': 'mongodb://user:pwd@host:port/db3',
        '201910': 'mongodb://user:pwd@host:port/db1'
    }
});
 
let obj4Decay = [{
    "slug" : "slug-1",
    "title" : "Hello world!",
    "publishDate" : new Date("2019-07-27T14:00:00.000+07:00")
}, {
    "slug" : "slug-2",
    "title" : "Hello world!",
    "publishDate" : new Date("2019-08-27T14:00:00.000+07:00")
}, {
    "slug" : "slug-3",
    "title" : "Hello world!",
    "publishDate" : new Date("2019-09-27T14:00:00.000+07:00")
}, {
    "slug" : "slug-4",
    "title" : "Hello world!",
    "publishDate" : new Date("2019-10-27T14:00:00.000+07:00")
}, {
    "slug" : "slug-5",
    "title" : "Hello world!",
    "publishDate" : new Date("2019-11-27T14:00:00.000+07:00")
}]
 
decay.init((err, result) => {
    if (err) {
        return console.log('init fail, err=', err, 'result=', result);
    }
 
    decay.decay({
        slug: 1
    },
    obj4Decay,
    (err, result) => {
        console.log('done err=', err);
        console.log('done result=', result);
 
        decay.close(() => {
            process.exit(0);
        })
    })
})

result will like this:

[
  {
    "find": {
      "slug": "slug-1"
    },
    "decayId": "5dd83b2bd82b317c61758d61"
  },
  {
    "find": {
      "slug": "slug-2"
    },
    "decayId": "5dd83b2bd82b317c61758df8"
  },
  {
    "find": {
      "slug": "slug-3"
    },
    "decayId": "5dd83b2cd82b317c61758ea5"
  },
  {
    "find": {
      "slug": "slug-4"
    },
    "decayId": "5dd83b2cd82b317c61758ea5"
  },
  {
    "find": {
      "slug": "slug-5"
    },
    "decayId": null
  }
]

License

ISC

- Chickyky -

Readme

Keywords

Package Sidebar

Install

npm i cky-mongo-decay

Weekly Downloads

0

Version

1.1.1

License

ISC

Unpacked Size

11.6 kB

Total Files

3

Last publish

Collaborators

  • chickyky