TOC
About
fastcall is a foreign function interface library which aim is to provide an easy to use, 100% JavaScript based method for developers to use native shared libraries in Node.js, without needing to touch anything in C++ and without sacrificing too much performance. It's designed with performance and simplicity in mind, an it has comparable function call overhead with hand made C++ native modules. See the benchmarks.
Why?
There is a a popular dynamic binding library for Node.js: node-ffi. Then why we need another one could ask? For performance! There is a good 20x-40x function call performance overhead when using node-ffi compared to hand made C++ native module, which is unacceptable in most cases (see the benchmarks).
Features
- uses CMake.js as of its build system (has no Python 2 dependency)
- based on TooTallNate's excellent ref module and its counterparts (ref-struct, ref-array and ref-union), it doesn't try to reinvent the wheel
- has an almost 100% node-ffi compatible interface, could work as a drop-in replacement of node-ffi
- RAII: supports deterministic scopes and automatic, GC based cleanup
- supports thread synchronization of asynchronous functions
Requirements
- CMake
- A proper C/C++ compiler toolchain of the given platform
- Windows:
- Visual C++ Build Tools or a recent version of Visual C++ will do (the free Community version works well)
- Unix/Posix:
- Clang or GCC
- Ninja or Make (Ninja will be picked if both present)
- Windows:
Install
npm install --save fastcall
Clone and Build
git clone --recursive https://github.com/cmake-js/fastcall.gitcd fastcallnpm install
For subsequent builds, type:
cmake-js compile
But don't forget to install CMake.js as a global module first:
npm install -g cmake-js
Benchmarks
To run benchmarks, type:
node benchmarks
Results:
There are 3 tests with synchronous and asynchronous versions:
- addNumbers: simple addition, intended to test function call performance
- concat: string concatenation, intended to test function call performance when JavaScript string to C string marshaling required, which is costly.
- callback: simple addition with callbacks, intended to test how fast native code can call JavaScript side functions
All tests implemented in:
- native: C++ code. Assuming function call's cost is almost zero and there is no need for marshaling, its numbers shows the run time of test function bodies
- native module: Nan based native module in C++. This is the reference, there is no way to be anything to be faster than this. fastcall's aim is to get its performance as close as possible to this, so the overhead ratio got counted according to its numbers
- node-ffi: binding with node-ffi in pure JavaScript
- fastcall: binding with fastcall in pure JavaScript
Yeah, there is a room for improvement at fastcall side. Improving marshaling and callback performance is the main target of the future development.
Documentation and Tutorials
fastcall.Library
Central part of fastcall is the Library
class. With that you can load shared libraries to Node.js' address spaces, and could access its functions and declared types.
; isSymbolExists(name); release(); declare(); declareSync(); declareAsync(); function(); syncFunction(); asyncFunction(); struct(); union(); array(); callback(); get functions(); get structs(); get unions(); get arrays(); get callbacks(); get interface();}
Constructor:
libPath
: path of the shared library to load. There is no magical platform dependent extension guess system, you should provide the correct library paths on each supported platforms (os
module would help). For example, for OpenCL, you gotta passOpenCL.dll
on Windows, andlibOpenCL.so
on Linux, and so on.options
: optional object with optional properties of:defaultCallMode
: either the defaultLibrary.callMode.sync
, which means synchronous functions will get created, orLibrary.callMode.async
which means asynchronous functions will get created by defaultsyncMode
: either the defaultLibrary.syncMode.lock
, which means asynchronous function invocations will get synchronized with a library global mutex, orLibrary.syncMode.queue
which means asynchronous function invocations will get synchronized with a library global call queue (more on that later)
Methods:
isSymbolExists
: returns true if the specified symbol exists in the libraryrelease
: release loaded shared library's resources (please note thatLibrary
is not aDisposable
because you'll need to call this method in very rare situations)declare
: parses and process a declaration string. Its functions are declared with the default call modedeclareSync
: parses and process a declaration string. Its functions are declared as synchronousdeclareAsync
: parses and process a declaration string. Its functions are declared as asynchronousfunction
: declares a function with the default call modesyncFunction
: declares a synchronous functionasyncFunction
: declares an asynchronous functionstruct
: declares a structureunion
: declares an unionarray
: declares an arraycallback
: declares a callback
Properties:
functions
: declared functions (metadata)structs
: declared structures (metadata)unions
: declared unions (metadata)arrays
: declared arrays (metadata)callbacks
: declared callbacks (metadata)
ref
Let's take a look at ref before going into the details (credits for TooTallNate). ref is a native type system with pointers and other types those are required to address C based interfaces of native shared libraries. It also has a native interface compatible types for structs, unions and arrays.
In fastcall there are a bundled versions of ref, ref-array, ref-struct and ref-union. Those are 100% compatible with the originals, they are there because I didn't wanted to have a CMake.js based module to depend on anything node-gyp based stuff. Bundled versions are built with CMake.js. The only exception is ref-array, fastcall's version contains some interface breaking changes for the sake of a way much better performance.
const fastcall = ; // 100% ref@latest built with CMake.js:const ref = fastcallref; // 100% ref-struct@latestconst StructType = fastcallStructType; // 100% ref-union@latestconst UnionType = fastcallUnionType; // modified ref-struct@latestconst ArrayType = fastcallArrayType;
ref-array changes:
See the original FAQ there: https://github.com/TooTallNate/ref-array
There are two huge performance bottleneck exists in this module. The first is the price of array indexer syntax:
const IntArray = 'int';const arr = 5;arr1 = 1;arr0 = arr1 + 1;
Those [0] and [1] are implemented by defining Object properties named "0" and "1" respectively. For supporting a length of 100, there will be 100 properties created on the fly. On the other hand those indexing numbers gets converted to string on each access.
Because of that, fastcalls version uses get
and set
method for indexing:
const IntArray = 'int';const arr = 5;arr;arr;
Not that nice, but way much faster than the original.
The other is the continous reinterpretation of a dynamically sized array.
const IntArray = 'int';const outArr = ref;const outLen = ref; myLib;const arr = outArr;const len = outLen;// arr is an IntArray with length of 0// but we know that after the pointer// there are a len number of element elements,// so we can do: for let i = 0; i < len; i++ console;
So far so good, but in each iteration arr
will gets reinterpreted to a new Buffer with a size that would provide access to an item of index i
. That's slow.
In fastcall's version array's length is writable, so you can do:
arrlength = len;for let i = 0; i < len; i++ console;
Which means only one reinterpret, and this is a lot of faster than the original.
Unfortunately those are huge interface changes, that's why it might not make it in a PR.
declaring functions
The interface uses ref and co. for its ABI.
For declaring functions, you can go with a node-ffi like syntax, or with a C like syntax.
Examples:
const fastcall = ;const Library = fastcallLibrary;const ref = fastcallref; const lib = 'bimbo.dll' // arg. names could be omitted;
Declared functions are accessible on the lib's interface:
const result = libinterface;
Sync and async:
About sync and async modes please refer for fastcall.Library's documentation.
If a function is async, it runs in a separate thread, and the result is a Bluebird Promise.
const lib = ...; libinterface;
You can always switch between a function's sync and async modes:
const lib = ...; const mulAsync = libraryinterfacemulasync;const mulSync = mulAsyncsync;assert;assert;
You get the idea.
Concurrency and thread safety:
By default, a library's asynchronous functions are running in parallel distributed in libuv's thread pool. So they are not thread safe.
For thread safety there are two options could be passed to fastcall.Library's constructor: syncMode.lock
and syncMode.queue
.
With lock, a simple mutex will be used for synchronization. With queue, all of library's asynchronous calls are enqueued, and only one could execute at once. The former is a bit slower, but that allows synchronization between synchronous calls of the given library. However the latter will throw an exception if a synchronous function gets called while there is an asynchronous invocation is in progress.
Metadata:
In case of:
const lib = ...;
lib.functions.mul
gives the same metadata object's instance as lib.interface.mul.function
.
- properties:
name
: function's nameresultType
: ref type of the function's resultargs
: is an array of argument objects, with properties ofname
: name of the argument (arg[n] when the name was omitted)type
: ref type of the given argument
- methods
toString
: gives function's C like syntax
declaring structs and unions
Easy as goblin pie. (Note: struct interface is ref-struct, and union is ref-array based.)
You can go with the traditional (node-ffi) way:
const fastcall = ;const StructType = fastcallStructType; // Let's say you have a library with a function of:// float dist(Point* p1, Pint* p2); const Point = x: 'int' y: 'int'; const lib = ...; const point1 = x: 42 y: 42 ;const point2 = x: 43 y: 43 ;const dist = libinterface;
But there is a better option. You can declare struct on library interfaces, and this way, fastcall will understand JavaScript structures on function interfaces.
Declaring structures:
// object syntaxlib; // C like syntaxlib;
After that the structure is accessible on library's structs
property:
const pointMetadata = libstructsPoint;
metadata properties:
name
: struct's nametype
: struct's ref type (exactly likePoint
in the first example)
The real benefit is that, by this way - because the library knows about your structure - you can do the following:
lib; // --- OR --- lib; // --- THEN --- const result = libinterface;
Way much nicer and simpler syntax like the original, ain't it?
About unions:
Declaring unions is exactly the same, except 'union' used instead of 'struct' on appropriate places. (Note: struct interface is ref-union based.)
lib; const f = libinterface; // f is 42.2
declaring arrays
Declaring arrays is not much different than declaring structures and unions. There is one difference: arrays with fixed length are supported.
// C code: // a struct using fixed length array struct S5 ; // a astruct using an arbitrary length array struct SA ; // example functions: // prints five values void // print length number of values // length is a parameter, because SA knows nothing about // the length of SA.ints void
Access those in fastcall:
lib // arbitrary length // or: IntArray ints; libinterface;libinterface;
Of course object syntax is available too:
lib // arbitrary length;
After that the array is accessible on library's arrays
property:
const intArrayMetadata = libarraysIntArray;
metadata properties:
name
: array's namelength
: number if fixed length, null if arbitrarytype
: array's ref type
Of course you can declare fixed length arrays directly:
lib;
And any fixed or arbitrary length array type could get resized on usage:
lib
declaring everything :)
fastcall.Library's declare
, declareSync
and declareAsync
functions could declare every type of stuff at once.
So the array example could be written like:
libdeclare'int[] IntArray;' + 'struct S5 { IntArray[5] ints; };' + 'struct SA { IntArray[] ints; };' + 'void printS5(S5* s);' + 'void printSA(SA* s, int len);';
callbacks
You can create native pointers for arbitrary JavaScript functions, and by this way you can use JavaScript functions for native callbacks.
// C code typedef int (*TMakeIntFunc)(float, double); int bambino(float fv, double dv, TMakeIntFunc func){ return (int)func(fv, dv) * 2;}
To drive this, you need a callback and a function on your fastcall based library defined. It could get declared by object syntax:
lib; const result = libinterface;// result is 42
Or with a familiar, C like syntax:
libdeclare'int (*TMakeIntFunc)(float, double);' + 'int bambino(float fv, double dv, TMakeIntFunc func)'; const result = libinterface;// result is 42
Callback's metadata are accessible on library's callbacks
property:
const TMakeIntFuncMetadata = libcallbacksTMakeIntFunc;
- metadata properties:
name
: callbacks's nameresultType
: ref type of the callbacks's resultargs
: is an array of argument objects, with properties ofname
: name of the argument (arg[n] when the name was omitted)type
: ref type of the given argument
- methods
toString
: gives callbacks C like syntax
pointer factories
If you wanna use you callbacks, structs, unions or arrays more than once (in a loop, for example), without being changed, you can create a (ref) pointer from them, and with those, function call performance will be significantly faster. Callback, struct, union and array factories are just functions on library's property: interface
.
For example:
// ref-struct way const StructType = StructType; // or require('ref-struct')const MyStruct = a: 'int' b: 'double' ; const struct = a: 1 b: 11 ;const ptr = struct; // ptr instanceof Buffer libinterface; // fastcall way libdeclare'struct StructType { int a; double b; }'; const ptr = libinterface;// ptr instanceof Buffer // You can create the modifyable struct instance of course// by using the metadata property: type const structMetadata = libstructsStructType;const struct = structMetadatatype a: 1 b: 11 ;// ...structa = 2;structb = 01;// ...const ptr = struct;// orconst ptr = libinterface;
Or with callbacks:
libdeclare'void (*CB)(int)'; const callbackPtr = libinterface;// callbackPtr instanceof Buffer libinterface;// that's a slightly faster than:libinterface;
string pointers:
For converting JavaScript string to (ref) pointers back and forth there are ref.allocCString() and ref.readCString() methods available.
However for converting JavaScript strings to native-side read-only strings there is a much faster alternative available in fastcall:
fastcall.makeStringBuffer([string] str)
Example:
const fastcall = ;// ... libdeclare'void print(char* str)'; const ptr = fastcall; libinterface;// that's faster than writing:libinterface;
RAII
Native resources must get freed somehow. We can rely on Node.js' garbage collector for this task, but that would only work if our native code's held resources are memory blocks. For other resources it is more appropriate to free them manually, for example in try ... finally blocks. However, there are more complex cases.
Let's say we have a math library that works on the GPU with vectors and matrices. A complex formula will create a bunch of temporary vectors and matrices, that will hold a lot memory on the GPU. In this case, we cannot rely on garbage collector of course, because that knows nothing about VRAM. Decorating our formula with try ... finally blocks would be a horrible idea in this complex case, just think about it:
const vec1 = lib;const vec2 = lib;const result = lib; // would be something like: const vec1 = lib;const vec2 = lib;let tmpAdd = null tmpAbs = null tmpMul = null;let result;try tmpAdd = lib; tmpAbs = lib; tmpMul = lib; result = lib;finally tmpAdd && tmpAdd; tmpMul && tmpMul; tmpAbs && tmpAbs;
So we need an automatic, deterministic scoping mechanism for JavaScript for supporting this case. Like C++ RAII or Python's __del__
method.
The good news is all three mentioned techniques are supported in fastcall.
Disposable
For accessing fastcall's RAII features you need a class that inherits from Disposable
. It's a very simple class:
; dispose(); resetDisposable(disposeFunction, aproxAllocatedMemory);}
disposeFunction
: could be null or a function. If null, thenDisposable
doesn't dispose anything. If a function, then it should release your object's native resources. It could be asynchronous, and should return a Promise on that case (any thenable object will do). Please note that this function has no parameters, and only allowed to capture native handles from the source object, not a reference of the source itself because that would prevent garbage collection! More on this later, please keep reading!aproxAllocatedMemory
: in bytes. You could inform Node.js' GC about your object's native memory usage (calls Nan::AdjustExternalMemory()). Will get considered only if it's a positive number.dispose()
: will invokedisposeFunction
manually (for the mentioned try ... catch use cases). Subsequent calls does nothing. You can override this method for implementing custom disposing logic, just don't forget to call its prototype'sdispose()
if you passed adisposeFunction
tosuper
. IfdisposeFunction
is asynchronous thendipsose()
should be asynchronous too by returning a Promise (or any thenable object).resetDisposable(...)
: reinitializes the dispose function and the allocated memory of the givenDisposable
. You should call this, when the underlying handle changed. WARNING: the originaldisposeFunction
is not called implicitly by this method. You can rely on garbage collector to clean it up, or you can calldispose()
explicitly prior calling of this method.
Example:
const lib = ;const fastcall = ;const ref = fastcallref;const Disposable = fastcallDisposable; { const out = ref; lib; const handle = outdefer; super { // you should NEVER mention "this" there // that would prevent garbage collection lib; } 42 /*Bytes*/; thishandle = handle; } // later let stuff = null;try stuff = ; // do stuff :)finally stuff && stuff; // replace the underlying handle: const stuff = ; // get some native handle from anywhereconst out = ref;lib;const someOtherHandle = outdefer; // now, stuff should wrap that // you should implement this in a private method:stuff; // old handle gets releasedstuff;stuffhandle = someOtherHandle;
For prototype based inheritance please use Disposable.Legacy
as the base class:
{ DisposableLegacy;} util;
automatic cleanup (GC)
When there is no alive references exist for your objects, they gets disposed automatically once Node.js' GC cycle kicks in (lib.releaseStuff(handle)
would get called from the above example). There is nothing else to do there. :) (Reporting approximate memory usage would help in this case, though.)
scopes
For deterministic destruction without that try ... catch mess, fastcall offers scopes. Let's take a look at an example:
const scope = scope; // Let's say, we have Stuff class// from the previous example. ;
Kinda C++ braces.
Rules:
- scope only affects objects whose classes inherits from
Disposable
, we refer them as disposables from now on - scope will affect implicitly created disposables also (like that hidden temporary result of the inner
fuseStuff
call of the above example) - all disposables gets disposed on the end of the scope, except returned ones
- returned disposables are propagated to parent scope, if there is one - if there isn't a parent, they escape (see below)
Escaping and compound disposables:
A disposable could escape from a nest of scopes at anytime. This will be handy for creating classes those are a compound of other disposables. Escaped disposable would become a free object that won't get affected further by the above rules. (They would get disposed manually or by the garbage collector eventually.)
{ // MyClass is disposable but has no // direct native references, // so its disposeFunction is null supernull; thisstuff1 = null; thisstuff2 = null; ; // at this point this.stuff1 and // this.stuff2 will be alive // despite their creator scope got ended } { // we should override dispose() to // free our custom objects thisstuff1; thisstuff2; } // later ;// However poo got created in the above scope,// so it gets disposed at the end,// and MyClass.stuff1 and MyClass.stuff2 gets disposed at that point// because of the overridden dispose().
Result:
scope
's result is the result of its function.
const result = ;// result is the new Stuff
Rules of propagation is described above. Propagation also happens when the function returns with:
- an array of disposables
- a plain objects holding disposables in its values
- Map of disposables
- Set of disposables
This lookup is recursive, and apply for scope.escape()
's argument, so you can escape an array of Map of disposables for example with a single scope.escape()
call.
Async:
If a scope's function returns a promise (any then-able object will do) then it turns to an asynchronous scope, it also returns a promise. Same rules apply like with synchronous scopes.
const result = ; result;
Coroutines:
Coroutines supported by calling scope.async
. So the above example turns to:
const result = scope
Way much nicer, eh? It does exactly the same thing.
node-ffi compatible interface
If you happen to have a node-ffi based module, you can switch to fastcall with a minimal effort, because there is a node-ffi compatible interface available:
const fastcall = ; // inconst ffi = fastcallffi; // you'll haveffiLibraryffiFunctionffiCallbackffiStructType // == fastcall.StructType (ref-struct)ffiUnionType // == fastcall.UnionType (ref-union)ffiArrayType // == fastcall.ArrayType (ref-array)
Works exactly like node-ffi and ref modules. However there are some minor exceptions:
- only
async
option supported in library options asyncPromise
property is available in functions along withasync
, you don't have to relyPromise.promisify()
like features- fastcall's version of Library doesn't add default extensions and prefixes to shared library names, so
"OpenCL"
won't turn magically to"OpenCL.dll"
on windows or"libOpenCL.so"
on Linux - fastcall's version of ref-array is modified slightly to have better performance than the original
Showcase
- NOOOCL: I have recently ported NOOOCL from node-ffi to fastcall. It took only a hour or so thanks to fastcall's node-ffi compatible interface. Take a look at its source code to have a better idea how ref and fastcall works together in a legacy code.
- ArrayFire.js: as soon as I finish writing this documentation, I'm gonna start to work on a brand new, fastcall based version of ArrayFire.js. That will get implemented with fastcall from strach, so eventually you can take a look its source code for hints and ideas of using this library.
Credits
- Daniel Adler and Tassilo Philipp - for dyncall
- TooTallNate - ref, ref-array, ref-struct, ref-union and of course node-ffi for inspiration and ideas
License
Copyright 2016 Gábor Mező (gabor.mezo@outlook.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.