AnyWhichWay
A JavaScript schema optional extensible key-value backed datastore supporting element level security, triggers, query pipelines, graphs, objects, documents, joins and full-text indexing.
Introduction
AnyWhichWay can be backed by almost any key-value store that exposes get(key)
, set(key,data)
, del(key)
or similar methods synchronously or asynchronously.
It is in an early BETA and currently supports:
-
Graph navigation over fully indexed data.
-
Object retrieval based on patterns.
-
Optional schema to support indexing and object validation.
-
Query joins.
-
Streaming analytics triggers for put/insert, patch/update, and delete.
-
"Smart" serialization. The database "learns" about new classes as they are inserted and restores data into appropriate class instances.
-
Over 30 piped array like query commands such as
first(n)
,last(n)
,map(f)
,mapReduce(mapper,reducer)
,pop()
,reduce(f,init)
. -
Custom graph navigation and piped commands in as little as one line of code.
-
Inspection and automatic "polyfilling" of passed in storage, e.g. storage can provide
del
,delete
,remove
, orremoveItem
. -
Security using graph paths. This allows the application of security at any level desired, i.e. object, property, and even value.
-
Automatic data expiration using durations or specific dates.
-
Full text stemmed and tri-gram indexes and search.
This version has been tested with: localStorage and idbkvstore in the browser plus blockstore, kvvs, Redis, and scattered-store on the server. Addining testing with keyv in on the roadmap. AnyWhichWay has built in logic to find the appropriate get
, set
, delete
, and clear
methods.
The key value stores that will not work are those that generate file names for the keys without hashing, e.g. node-localstorage. This is because the keys generated internally by AnyWhichWay are often not valid file names.
Note: idbkvstore
is currently very slow for put
. In fact, things will run faster against a remote Redis store. This is a function of the underlying IndexedDB, which in our experience tends to be slow for simple key/value type tasks due but fast for more complex tasks.
Why AnyWhichWay
When processing data in JavaScript, one usually ends-up collecting things into arrays and then processing the arrays to sort, filter, reduce etc. This can consume large amounts of RAM and also means that result processing can't start until an array is fully populated, particularly with respect to simulating joins. AnyWhichWay provides a full set of array like functions that are actually backed by asynchronous generators. Even the join processor is an asynchronous generator that processes one tuple at a time. When processing large volumes of data, this ensures the non-blocking return of initial results as fast as possible while limiting RAM usage.
With AnyWhichWay, if you know the functions available on an array you know more than 80% of what is required to query and process data in a manner far more rich than many alternatives with a smaller footprint. It is currenlty about 50K uncompressed. Granted, AnyWhichWay is in BETA now, so there are also things missing, e.g. conflict resolution, full transaction management. We anticipate it will be about 25K compressed when PRODUCTION is released.
Contents
Example Capabilities
-
Graph navigation over fully indexed data supporting:
a) exact matching, e.g.
get("Person/email/'jsmith@somewhere.com'")
retrieves the Person with the provided e-mail.b) inline tests, e.g.
get("Person/age/gt(27)")
retrieves all Persons over 27.c) existing value retrieval, e.g.
get("Person/age")
will return all the possible ages for Persons.d) wild cards, e.g.
get("*/email/*")
, will retrieve all objects that have an email property, not just Persons.e) inline functions, e.g.
get("Person/age/(value) => value > 27 ? value)
also retrieves all Persons over 27, although more slowlyf) object methods, e.g. `get("Car/#/.makeAndModel()) will return all makeAndModel strings for all Cars.
-
Object retrieval based on patterns supporting:
a) exact matching, e.g.
get({email:"jsmith@somewhere.com"},Person)
retrieves the Person with the provided e-mail.b) tests, e.g.
get({age:value => value > 27 ? value,name:"Joe"},Person)
retrieves all Persons over 27 with name "Joe".c) coerced classes, e.g.
get({email:"jsmith@somewhere.com", instanceof:"Person"})
retrieves the Person with the provided e-mail. -
Object joins supporting:
a) patterns, e.g.
join({instanceof:Guest},{instanceof:Guest},([guest1,guest2]) => guest1["#"]!=guest2["#"] ? [guest1,guest2] : false)
retrieves all possible Guest pairsb) paths, e.g.
join("Object/age/*","Object/age/*",([a,b]) => a.age && b.age && a.age!==b.age ? [a,b] : false)
retrieves all pairs of differently aged Objects -
Triggers:
a) e.g.
on('Object/optin/true',"set",({type,target,key,value}) => console.log(type,target,key,value)
will log"put",<some object>,"optin",true
whenever an object with theoptin
property set totrue
is added to the database. -
Security using graph path strings or arrays:
a)
/@Object/<security rule>
- controls all Objectsb)
/@Object/SSN/<security rule>
- controls all SNN property data on all Objects -
Data expiration:
a)
putObject({user:"joe.jones@myco.com",passwordResetKey: 12345},30*60*1000})
- inserts a key for password reset that expires and is deleted in 30 minutes.
Assist Us
If you like the concept of AnyWhichWay give us a star here on GitHub or npmjs.
Help spread the word by a comment or up vote on echojs or a Tweet.
Installation
npm install anywhichway
AnyWhichWay will run in current versions of Chrome and Firefox.
Node v9.7.1 (the most recent at this writing) must be run with the --harmony
flag.
Babel transpiled code will not work. Babel does not seem to generate correct asynchronous generators.
Documentation Notes
When "Object" is capitalized it refers to a direct instance of the class Object. When "object" is lower case it refers to an instance of any type of class except Array, which uses the term "array".
The property "#" is the default used for unique uuidv4 ids on objects in AnyWhichWay. See the section Metadata for more info.
The property "^" is the default used for object metadata. See the section Metadata for more info.
Starting A Database
Databases are instantiated with 2 arguments, a storage instance and an options object.
const mydb = localStorageinline:true expirationInterval=10*1000
or
const redis = redis mydb = redisinline:truepromisify:true;
The storage argument can be anything that supports the localStorage
API or something similar with get
and set
. If the storage argument is null an in-memory store is created.
The options object supports the following:
context
- The object to use as the this
scope for triggers and security rules. Defaults to null
.
awaitIndexWrite:boolean
- A flag to indicate if object insertion should return prior to writing the index. Defaults to true
. Setting to false
will speed-up inserts. Note: Objects are still written before the index, this just affects return behavior. Recommend setting to false
when there is a slow network connection to the backing key-value store.
cacheStep:number
- Sets the number of records to free if the cache reaches maxCache
. If a float, then treated as a percentage. If an integer, treated as an absolute count. Defaults to 0.1 (10 percent).
expirationInterval:number
- The number of milliseconds between each scan and deletion of expiring data. The default is 30 minutes, i.e. 30*60*1000
. If <= 0, auto expiration is turned off. This can be changed at any time by setting <db>._options.expirationInterval
.
getCurrentUser:function
- The function to call to get the current userid. Although AnyWhichWay supports mappings between Users
and Roles
, it does not provide password management or an authentication mechanism. It is up to the developer to provide these capabilities, perhaps through OAuth
, and support the delivery of an authenticated user id to the security mechanisms of AnyWhichWay. The function can take one argument, which is the database instance. It should return a Promise for an object with the property id
, usually an e-mail address.
inline:boolean
- A flag indicating whether or not to support and compile inline functions in graph paths. Defaults to 'false'. Warning: Setting to true
opens up a code injection risk.
indexDates:boolean
- Index dates by their component UTC milliseconds, seconds, minutes, etc. Off by default. Otherwise, just the seconds since January 1st, 1970 are indexed.
maxString:number
- All strings are full text or trigram indexed, but those longer than maxString
can only be matched using search
. Defaults to 64.
maxCache:number
- Sets max size of LFRU Cache. The default is 10,000.
promisify:boolean
- If true
, the passed in storage uses callbacks that will need to be wrapped with Promises.
textIndex:"stem"|"trigram"|*
- Sets the full text index type. Use *
to index both ways'
Storing Data
To use AnyWhichWay as a regular key/value store, just use an asynchronous version of the localStorage API, e.g.
await mydb;const joe = mydb;await mydb;
To store an object and have it indexed, use the extended API putObject(object)
. Then use graph paths (see Queyring Data) or getObject(id)
, matchObject(pattern)
, and removeObject(patternOrId)
, e.g.
// uses plain old Objectslet joe = await mydb
For putObject
, you can also set a duration in milliseconds or expiration date/time with a second argument:
// expires in approximately one yearawait mydb; // expires Wed Aug 01 2040 00:00:00await mydb;
Note: Note the default expiration processing interval in AnyWhichWay is 30 minutes, so durations less than 30*60*1000
are not useful unless this is changed using the start-up option expirationInterval
.
The indexes can be accessed like file paths and take the form:
<@classname>/<property>/<value>/<property>/<value>...
value
- Value of the property. Strings are surrounded by "", e.g. /@Object/name/"Joe"
. This allows the indexed to be typed, i.e. @Object/age/27
vs /@Object/cutomerid/"123456"
.
[/<property>/<value>]
- Optional property/value pairs are the nested objects, if any, e.g `/@Object/address/city/"Seattle".
See Graph Navigation below for how to use these paths.
Querying Data
Graph Navigation
Data can be retrieved using a graph path, e.g.:
mydball;
Paths can also be handed in as arrays, e.g.:
mydball;
Unless you are using AnyWhichWay as a simple key value store, graph references generally start with a @ followed by a property and a value or another property if the value is itself an object, e.g.
address:city:"Bainbridge Island"state:"WA" zipcode:base:98110plus4:0000
is matched by:
@Object/address/city/"Bainbridge Island" @Object/address/zipcode/base/98110
Any location in the path can also be the * wildcard, a compiled inline test, a dynamic inline test, a ".." relative location, e.g.
/@Object/address/city/
- matches all Objects with an address and a city property and returns the Objects
/@Object/address/city
- returns all city names for Objects that have an address property
/@Object/address/state/"WA"/
- return all objects with addresses in WA. Note the quotes around "WA". Strings must be quoted so that the index can distinguish between numbers and strings, e.g. 27 vs "27".
/@Object/address/state/in(["WA","OR"])/
- return all objects with addresses in WA or OR
/@Object/address/zipcode/base/between(98100,98199,true)/
- return all Objects with an address in the zipcode base range of 98100 to 98199
Note the use of the between
predicate above, this is one of many predicates available:
Inline Predicates
gt(testValue)
- True for values that are > testValue
.
gte(testValue)
- True for values that are >= testValue
.
eq(testValue,depth)
- True for values that are == testValue
. If depth
> 0, tests objects to the depth. Object having equal ids satisfy eq
.
between(bound1,bound2,edges)
- True for values that are between bound1
and bound2
. Optionally allows the value to equal boundary.
outside(bound1,bound2)
- True for values that are outside bound1
and bound2
.
eeq(testValue) - True for values that are ===
testValue`.
echoes(testValue)
- True for values that sound like testValue
.
matches(testValue)
- True for values that match testValue
where testValue
is a RegExp. If testValue
is a string it is converted into a RegExp.
contains(testValue)
- True for values where value.indexOf(testValue)>=0
so it works on strings and arrays.
neq(testValue)
- True for values that are != testValue
.
lte(testValue)
- True for values that are <= testValue
.
lt(testValue)
- True for values that are < testValue
.
in(testValue)
- True for values where testValue.indexOf(value)>=0
so it works on strings and arrays.
nin(testValue)
- True for values where testValue.indexOf(value)===-1
so it works on strings and arrays.
not(f)
- True for values where !f(value)
is truthy.
type(testValue)
- True for values where typeof(value)===testValue
.
Tertiary nodes after the "#" selector can be property names or method calls, e.g.
@Car/#/model
- gets all model names for all Cars doing a table scan, Car/model
is faster.
@Car/#/.makeAndModel()
- However, method names can only be invoked as a result of a table scan.
It should not be overlooked that by design graph paths can be escaped and passed directly over a network as get
requests!
Arrays
Values stored in arrays can be accessed directly as children or via an array offset, e.g.
name: "Joe" favoriteColors: "red""green"
/@Object/favoriteColors/"red"/
- Returns Objects where "red" is anywhere in the array.
/@Object/favoriteColors/0/"red"/
- Returns Objects where "red" is the first element.
Dynamic Inline Tests
/@Object/address/zipcode/base/(value) => value>=98100 && value<=98199)
- return all Objects with an address in the zipcode base range of 98100 to 98199
/@Object/*/(value) => ... some code/
- a "table scan" across all Objects and all properties, returns all Objects with property values satisfying the inline
/@Object/#/(value) => ... some code/
-a "table scan" across all Objects, returns all Objects satisfying the inline
*/*/(value) => ... some code/
- a "table scan" across an entire database, returns all objects satisfying the inline
*/#/(value) => ... some code/
- a "table scan" across instances of all classes, returns all objects satisfying the inline
Notes: Dynamic in-line tests MUST use parentheses around arguments, even if there is just one.
Dynamic in-line tests expose your code to injection risk and must be enabled by setting inline
to true in the options object when a database connection is created. Any in-line test can be added as a compiled test to avoid this issue. See Extending AnyWhichWay.
For convenience, the following are already defined:
Query Patterns
Query patterns are objects. If the query pattern is an instance of a specific kind of object, then only those kinds of objects will be matched.
Property values in query patterns may be literals or functions that return 'falsy' if they should fail.
// yield all Person's over 27 in the state of Washington.mydball;
Query Commands
Queries are effectively a series of pipes that change the nature of the results or produce other side-effects. Internally, all query commands are wrapped in a generator functions that yield
each result to the next command.
Queries take the forms:
<db><edgeYield><edgeYield>...<valueYield><valueYield>|<pipelineStep>...<valueYield>|<pipelineStep><invoker>`; <db>.query().<valueYield>[.<valueYield>|<pipelineStep>...[.<valueYield>|<pipelineStep>].<invoker>]`; // only `provide` can be used as a `valueYield` when no edge is yielded first.
For example:
mydbvalueall;
.get("Object/age/*")
- yields all edges for Objects with ages.
.value()
- extracts the value from each edge.
.filter(object => object.age>=27)
- filters for all objects with age >= 27.
.all()
- executes the query and returns a promise for an array of objects.
Edge Yielding Commands
get(pathOrPattern)
- get
always yields an edge, except when the last portion is /#/.<methodCallOrPropertyName>
, in which case it yields a value. See Query Patterns below.
Value Yielding Commands
delete(pathOrPatternOrId)
- Deletes the specified item. Or, if no argument is provided, deletes the item yielded by the previous step in the query. Yields no result.
edge(raw)
- Yields the edge itself as a JSON value. If raw
is true, yields the generator that represents the edge.
edges(raw)
- Yields the child edges of the edge itself as JSON values. If raw
is true, yields the generators that represents the children.
fetch(url,options)
- If an argument is provided, fetches the url with the provided options using the value it is passed as the body
. If no arguments are provided, assumes the passed value will be a string representing a URL or an object of the form {url,options}
. Also available as a Pipeline Step.
fork()
- Creates a separate worker or Node process to run the command. NOT YET IMPLEMENTED.
join(...pathsOrPatterns,test)
- Yields arrays of values combinations that satisfy test(<array combination>)
. See Joins below.
keys()
- Yields the keys on all values that are objects. Any non-object values result in no yield. Also available as a Pipeline Step.
provide(...values)
- Yields no values except those in values
. Usually used directly after <db>.query()
, often for testing purposes.
random(float)
- Yields a random percentage of the values it is passed. The random
method may also be used as part of a process step sequence. Also available as a Pipeline Step.
render(template)
- Renders each value provided into the template
and yields {value,string}
. The template
should use JavaScript template literal type substitution. The scope is bound to the value passed in. The render
method may also be used as part of a process step sequence. Also available as a Pipeline Step.
search(keywordString,options)
- The keywordString
is tokenized and stems are used to search an index. After candidate documents are returned, exact matching of tokens is applied. The options
object can take the flags {percentage:float,exact:true|false,stem:true|false,path:pathString,index:"stem"|"trigram"}
If percentage
is set, the provided percentage of tokens must be matched; otherwise, just one token is sufficient. If stem
is true, then the candidate documents just have stems checked. If exact
is true, then percentage
and stem
are ignored and there must be an exact match for the keywordString
. By default all properties of an object are searched. If path
is set as a dot or slash delimited string, then just the property or sub-property matching the path is searched. index
specifies which index to use. It defaults to the database options property textIndex
. If textIndex
is *
(both), search defaults to "stem".
value
- Yields the value on the edge.
values(objectsOnly)
- Yields all the values of properties on any objects passed down or primitive values. If objectsOnly
is true, then any primitive values from edges are not yielded. Also available as a Pipeline Step.
Pipeline Step
Pipeline steps are almost identical to the calling interface of an array, with some enhancements.
aggregate([pattern,]aggregates)
- Behaves similar to MongoDB aggregate except uses AnyWhichWay functional pattern matching and aggregators. See matches
for functional pattern matching. The pattern {}
matches all objects. You can also exclude the pattern
argument and its is assumed to be {}
, which will match all objects passed down the pipeline. The aggregates
specification is an object the properties of which are the keys to group by. Their values are also objects naming the property to aggregate which are also objects naming the aggregated field and defining the aggregator functions. Aggregated filed names must be unique across the entire aggregate. The functional aggregators are simply reduce functions that take the accumulated value, the next value, and the aggregation as arguments. Aggregators are called in the order they are specified, so results can be based on aggregates computed earlier. The count
property is always available on the aggregation. The first time aggregators are called the accumulator will be undefined
. You can default it to 0 the first value with a single short statement, e.g. see the definition of sum and average below which will sum and average the balance across all objects.
;
The slightly modified example will group by customer id:
;
concat(...args)
- Adds the args
to the end of the yielded query results.
collect()
- Collects values into an array.
distinct(key,objectsOnly)
- Yields only unique values. The key
argument is optional and will yield only unique values for key
when objects are passed in. If objectsOnly
is falsy, primitive values will be also be yielded.
every(f)
- Yields all values so long as every value satisfies f(value)
.
filter(f)
- Only yields values for which f(value)
returns truthy.
flatten()
- Assumes an array is being passed from upstream and yields each value separately. If it is not an array, just yields the value.
forEach(f)
- Calls f(value,index)
for all values.
group(grouping,flatten)
- grouping
is an array of fields to group by. flatten
will force the return of an array; otherwise an object of the form ... is returned.
map(f)
- Calls f(value,index)
and yields the value returned for each value it receives. f
should return undefined
to prevent anything from being passed down the processing chain.
mapReduce(map,reduce[,{key:string,value:string}])
- Takes upstream value as query result and then behaves like MongoDB mapReduce to yields the results as {key:string,value:any}
pairs. The
optional third argument supports the replacement of the key
and value
property names with custom names.
matches(pattern,value)
- If pattern
a primitive yields value
if pattern===value
. If pattern
is a function value
is yielded if pattern(value)
is truthy. If pattern
is an object, recursively calls matches
on each property value of pattern
and value
.
pop(f)
- Pulls the last value and does not yield it. If f
is not null calls f(value)
with the popped value.
push(value)
- Yields value
into the end of the results.
reduce(f,initial)
- Yields the result of <all values>.reduce(f,initial)
.
reduceRight(f,initial)
- Yields the result of <all values>.reduceRight(f,initial)
.
reverse()
- Yields results in reverse order.
seen(f)
- Calls f(value)
once for each unique value received but yields all values.
slice(begin,end)
- Yields just the values between the index begin
and end
. Using negative values uses offsets from end of yield process.
shift(f)
- Does not yield the first value. If f
is not null, calls f(value)
with the shifted value.
some(f)
- Yields all values so long as some value satisfies f(value)
.
sort(f)
- Yields values in sorted order based on f
. If f
is undefined, yields in alpha sorted order.
split(...queryFragmentFunctions)
- creates multiple yield streams with copies of the value passed to split
. If an array is passed and it contains objects, each object is also copied. A queryFragmentFunction
takes the form:
// "this" takes the place of "<database>.query()"// there should be no "all()" or "exec()" at the end. { return this<command 1><command 2>...<command n>; }
test(test,value)
- If test
is a primitive or a RegExp it is treated as a RegExp; otherwise, it should be a function. Yields value
if test(value)
or test.test(value)
is truthy.
splice(start,deleteCount,...items)
- Yields items
values after values up to start
have been yielded and up to deleteCount
items have not been yielded.
unshift(value)
- Add value
to the end of the yield stream.
Invokers
all()
- Returns a promise for the values of a query as an array.
exec()
- Processes a query without collecting any results. Conserves memory as a result.
Full Text Stemmed and Trigram Indexing and Search
To be written. For now see search
in API documentation above.
Joins
join(...pathsOrPatterns,test)
- Yields arrays of value combinations that satisfy test(<array combination>)
.
By convention you should destructure the argument to test. The test below will only yield combinations where the names are identical:
object1object2 object1name && object2name && object1name===object2name;
Metadata
The signature of metadata is: {created:Date,updated:Date,expires:Date,v:number}
.
v
is the version number.
Unique object uuidv4 ids are stored on objects themselves under the key #
rather than in metadata. Their signature is: <classname>@<uuidv4>
.
Dates have the id signature Date@<milliseconds>
.
Schema
Schemas are optional with AnyWhichWay. They can be used to:
-
Validate data
-
Ensure referential integrity across objects/documents
-
Support records retention and expiration
-
Provide declarative security
By convention schemas are defined as static mmembers of classes/constructors. However, they can also be declared when registering classes using <database>.register(ctor,name,schema)
.
The surface of a schema may contain any of the following properties:
-
index
-
integrity
-
management
-
security
-
validation
Each should point to an object as described below:
Index
If no schema index definition is provided, all properties on an object are indexed. If a property name is not listed it is indexed by default.
Personschema = index: SSN: false
Integrity - NOT YET IMPLEMENTED
Supports cascading deletes and the creation of reflexive relationships. Validates the arity (n-ary nature) of relationships.
The integrity value should be an object with a property for every case where an instance is expects to have another object as a value.
Personschema = integrity: address: kind: "heirarchical" arity: 1 favoriteColors: kind: "heirarchical" arity: 3 siblings: kind: "relational" arity: Infinity reflect: "siblings" cascade: "update"
A person might then look like this:
address: ... favoriteColors: ... siblings: ...ids
Management
Sets default durations. Unspecified durations are effectively Infinty. Turns off time stamping and versioning. By default creation and update time stamps are created and versions are incremented.
Personschema = management: duration: milliseconds timestamp: true|false version: true|false
Security - NOT YET IMPLEMENTED
Personschema = security: SSN: group:"admin"read:truewrite:true group:"owner"read:truewrite:true // owner is a special group true:
Validation
The validation value should be a function without closure scope that validates an entire object and returns a possible empty array of errors, or an object that has the same structure as the expected class instance, including nested objects. However, each property value should either be:
-
a function without closure scope that when passed a value, key, object from an instance returns
true
for valid or an Error object if invalid. -
a validation specification taking the form:
type: "boolean"|"number"|"string"|"object"|"<classname>" default: <value>|
Nested objects can either be validated by a single function, or can have functions and validation specifications on nested object properties.
For example:
{ Object; }Personschema = validation: typeofvalue==="string" || "Person.name is required and must be a string" value===undefined || typeofvalue==="number" || "Person.age must be a number" optin: type:"boolean"default:false
Security
In addition to schema based security, security can expressed using graph paths and a special command secure(path,action,function)
. This allows the application of security at any level desired, e.g.
<db>.secure("Object","set",<security rule>)
- controls set for all Objects and their child paths.
<db>.secure("Object/SSN","get",<security rule>)
- controls get of all data stored in SNN property on all Objects
Security rules are just a special type of function receiving a security event object:
... your code ...
Note: Wild cards and inline functions are NOT supported in security paths; however, heirarchies are supported. In the example above, both of the rules will be applied to Objects that contain an SSN property since the first rule applies to ALL Objects.
The key
will be the full path to the data being accessed. path
will be the current level in the security heirarchy.
You are free to modify value
to obscure data or restore it from the original. As a result, security rules can be very powerful, you could examine a value to see if it looked like an SSN regardless of the key it is stored under and return a masked version.
The original
field will be present when using set
and contains the current values on disk. This allows unmasking of previously masked data prior to save. Changes to original
are ignored.
At the moment it is up to the implementor to look-up session ids, user ids and groups using the if they are needed. The rules this
will be bound to will be the value of the context
property of the options
used to start the database.
Returning undefined
, will prevent the action.
Transactions
Writes of objects that do not contain nested objects or that contain only nested objects of classes that are not registered with schema are atomic and done before indexing. This is somewhat transactional; however, if there is a failure, indexes may be incomplete.
Writes of objects that contain nested objects that are registered as schema are not currently atomic. Each object is a separate atomic write. If the system fails between these writes, then there could be an inconsistent state and definitely an incomplete index.
All objects are written prior to indexing. Currently schema must be registered every time a database is started using the register(ctor,name=ctor.name,schema=ctor.schema)
method on the database. Passing true
as the value for schema
will force indexing, but no validation will be done on the instances of ctor
. As can be seen from the signature for register
, by convention schema are defined as static properties on class definitions; however, this can be changed simply by calling register
with three arguments.
The above being said, a base design for full transactional capability has been envisioned and transactional integrity is possible.
Triggers
<db>.on(key,eventType,callback)
- Ads callback
as a trigger on key
for the event types set
, delete
.
The callback is invoked with an event object of the below form.
keyaction:"set"|"delete"value:<any value>oldvalue:<any value>objectkey`
The optional members object
and key
will only be present if the value at the path is an object.
Thecallback
will be serialized between database sessions. So, it should not contain any closure scoped variables. Its this
will be bound to the value of the context
property of the options
used to start the database.
Extending AnyWhichWay
Predicates
Predicates, e.g. gt
, gte
, can be added using the tests
property on the options
object when starting a database.
The below example woud just replace the outside
predicate with what happens to be the same code as its internal representation.
{ return value != b1 && value !=b2 && b2 > b1 ? value<b1 || value>b2 : value>b1 || value<b2;}const db = storetests:outside;
Note: The function outside
returns a function that take a single argument, value
. This function is invoked by AnyWhichWay with the current value of the data item being tested.
Benchmarks
Proper benchmarking is udner development; meanwhile, below are the typical times to run the unit tests:
Redis (remote high speed connection): 10s Redis (remote low speed connection): 12s localStorage: 5s idbKVStore: 16s scattered-store: 2s Blockstore: 450ms KVVS Speed Optimize: 1s KVVS Memory Optimized: 2s
Release History (reverse chronological order)
2018-04-23 - BETA v0.0.27b Eliminating test data pushed with earlier build.
2018-04-20 - BETA v0.0.26b Tested with KVVS (Key Value Versioned Store).
2018-04-16 - BETA v0.0.25b Added user and role creation. Fixed issues with instanceof
.
2018-04-13 - BETA v0.0.24b Enhanced documentation. Added to start-up option awaitIndexWrite
.
2018-04-13 - BETA v0.0.23b Implemented MongoDB like mapReduce
. Allow null storage for memory only database. Renamed fork
to split
. Created multi-process fork
. Enhanced schema support.
2018-04-08 - ALPHA v0.0.22a Major re-write of internals. 20% smaller. 2-3x faster. Index names prefixed with @
. Added a type of LFRU Cache. split
fixed. Next release will be a BETA.
2018-04-01 - ALPHA v0.0.21a Predicates in the query pipleine have been deprecated. Use map
instead. last(n)
has been deprecated, use slice(-n)
instead. first(n)
has been deprecated, use slice(0,n)
instead. patch(pattern,data,expires)
has been deprecated, use get(pattern).assign(object,true,expires)
instead. NOTE: relative paths broken.
2018-03-22 - ALPHA v0.0.20a full text search added, NOTE: relative paths and split broken.
2018-03-17 - ALPHA v0.0.19a object patching now implemented
2018-03-16 - ALPHA v0.0.18a enhanced documentation.
2018-03-16 - ALPHA v0.0.17a moved triggers to their own partition for performance.
2018-03-16 - ALPHA v0.0.16a enhanced documentation, added ondelete
handling.
2018-03-14 - ALPHA v0.0.15a enhanced documentation.
2018-03-14 - ALPHA v0.0.14a tested with blockstore
, improved split
.
2018-03-14 - ALPHA v0.0.13a fixed some string escape issues for idbkvstore
.
2018-03-13 - ALPHA v0.0.12a enhanced documentation.
2018-03-13 - ALPHA v0.0.11a enhanced documentation, added assign
, default
, fetch
, fork
.
2018-03-13 - ALPHA v0.0.10a enhanced documentation, added auto expiration, published to NPM.
2018-03-12 - ALPHA v0.0.9a enhanced documentation, published to NPM.
2018-03-11 - ALPHA v0.0.8a delete now working for objects, paths pending.
2018-03-11 - ALPHA improved metadata support, started work on delete.
2018-03-11 - ALPHA enhanced documentation, added events and instanceof support.
2018-03-09 - ALPHA enhanced documentation, added schema validation.
2018-03-08 - ALPHA enhanced documentation, improved relative paths and nested objects.
2018-03-07 - ALPHA Enhanced documentation, pattern matching, joins, and unit tests.
2018-03-04 - ALPHA Publication of major re-write and release track ALPHA.
License
MIT License
Copyright (c) 2018 Simon Y. Blackwell, AnyWhichWay, LLC
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.