Redis Data Lake
Use Redis as a document database with extensive search capabilities
Installation
USING NPM
$ npm install datalake --save
In node.js
const datalake = ;const dl = ;
Implementation
Code:
const dl = ;const datalake = ; datalake catch { console; };
Available Methods
S.No | API Name | Method | Required Fields | Optional Fields | Sample PostData | Notes |
---|---|---|---|---|---|---|
2 | /CloseConnection | GET | ------------ | ------------ | ------------ | ------------ |
2 | /ConfigureSearchIndex | POST | ShortCodes(sc, type, csv), Schema, Keyword | ------------ | /ConfigureSearchIndex | ------------ |
2 | /CreateBackupData | POST | Schema, Guid | ------------ | /CreateBackupData | ------------ |
2 | /CreateConnection | POST | host,dbname | port,password,maxConnection | /CreateConnection | ------------ |
2 | /Fetch | POST | ||||
2 | /GetAllSchemaData | POST | Schema | Keyword, Guid | /GetAllSchemaData | This API cannot used in big data (Server will crash) |
2 | /GetCacheData | POST | Schema,Key | ------------ | /GetCacheData | ------------ |
2 | /getDistinctRecords | POST | ||||
2 | /InsertData | POST | Schema, Keyword, MetaData | ------------ | /InsertData | ------------ |
2 | /ListSchemas | GET | ------------ | ------------ | ------------ | ------------ |
2 | /ListSearchIndex | POST | Schema, Keyword | ------------ | /ListSearchIndex | ------------ |
2 | /RefreshSchemaSearchIndex | POST | Schema | Guid | /RefreshSchemaSearchIndex | ------------ |
2 | /RemoveData | POST | ||||
2 | /RestoreData | POST | Schema, Guid, Version | ------------ | /RestoreData | ------------ |
2 | /Search | POST | Schema, Keyword | (SearchFields), recordFrom, recordUpto, Guid | /Search | Schema, (SearchFields) can be single value or can be comma (,) seperated or can be ranged (~). Either (SearchFields) or Guid should be given else This API cannot used in big data (Server will crash) |
2 | /SearchDataByProperty | POST | Schema, Keyword, (SearchFields) | ------------ | /SearchMultipleData | (SearchFields) can be single value or can be comma (,) seperated or can be ranged (~) |
2 | /SearchMultipleData | POST | Schema, Keyword | PropertyField, PropertyValue, Guid | /SearchDataByProperty | Schema, PropertyField, PropertyValue can be single value or can be comma (,) seperated or can be ranged (~). Either (SearchFields) or Guid should be given else This API cannot used in big data (Server will crash) |
2 | /SearchTopRecords | POST | Schema, Keyword, Nextbatch, BATCHCOUNT, field | ------- | /SearchTopRecords | Nextbatch will be first element of response data |
2 | /SetCacheData | POST | Schema, Key, Data | Timeout | /SetCacheData | If Timeout is not given, Data will be stored in redis permanently |
2 | /ShowConnectionStatus | GET | ------------ | ------------ | ------------ | ------------ |
Sample Input
createConnection
POST
: createConnection() → {undefined}
This will Create Redis Connection pool object.
Post Data:
closeConnection
GET
: closeConnection() → {undefined}
This will close already created Redis connection pool if any.
showStatus
GET
: showStatus() → {JSON}
Returns the Status of Redis connection
Response:
SetupSearchHash
POST
: SetupSearchHash() → {JSON}
Defining Redis Schema index keys
Post Data:
InsertData
POST
: InsertData() → {JSON}
This will insert Key value pair against the Schema given
Post Data:
GetAllSchemaData
POST
: GetAllSchemaData() → {JSON}
This will search records for given Keyword, Schema and returns all Key-value pair available in the schema. Result can be filtered with particular Guid
Post Data:
SearchTidalPoolHash
POST
: SearchTidalPoolHash() → {JSON}
This will search record for given PropertyField, PropertyValue, Keyword and SchemaName including comma(,) separated PropertyValues and range of PropertyValue
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
GetDatafromSchemas
POST
: GetDatafromSchemas() → {JSON}
This Method is the combination of GetSearchHashSchema, GetTidalPoolSchema, and SearchTidalPoolHash. This will search the records for given PropertyField, PropertyValue, Keyword and SchemaName. Multi-value search of records using comma(,) separated list of PropertyValue(s) or range between the PropertyValue(s) or it can be filtered with particular Guid. It can also performs multiple comma(,) separated Schema Name search.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
GetSchemaList
GET
: GetSchemaList() → {undefined}
GetSchemaList will return list of Schema Names available in the Selected Redis DB.
Implementation
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
GetSearchHashSchema
POST
: GetSearchHashSchema() → {JSON}
This will return the defined Redis Schema index keys for the given Schema Name.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
GetTpSearchHash
POST
: GetTpSearchHash() → {JSON}
This will return the defined Redis Schema index keys for the given Schema Name.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
SetKeyData
POST
: SetKeyData() → {JSON}
Set cache data into the Redis DB for the given key-value pair with TTL(Time to live) provided. If TTL(Time to live) not provided, it will set cache data as permanent data.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
GetKeyData
POST
: GetKeyData() → {JSON}
Get the value from cache memory for given search key before the TTL(Time to live) expire
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
RefreshTidalPoolData
POST
: RefreshTidalPoolData() → {JSON}
Refresh/Re-Build all the search index for the given Schema Name
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
UpdateTidalPoolHash
POST
: UpdateTidalPoolHash() → {JSON}
Refresh/Re-Build particular search index for the given Guid, Schema Name.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
SnapshotTidalPoolData
POST
: SnapshotTidalPoolData() → {JSON}
It Creates Backup copy of given Guid, Schema Name.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
RemoveTidalPoolData
POST
: RemoveTidalPoolData() → {JSON}
It Creates Backup copy of given Guid, Schema Name and Removes Key-value pair from the Schema.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response:
RollbackTidalPoolData
POST
: RollbackTidalPoolData() → {JSON}
It Creates Backup copy of current Key-Value pair for the Given Guid-Schema and Replaces old Key-Value pair from Archive.
Implementation
Post Data:
Code:
const datalake = ;const dl = ;dl catch { console; };
Response: