@factoshi/zeetl
TypeScript icon, indicating that this package has built-in type declarations

0.1.1 • Public • Published

Zeetl

An ETL for ZEE. Run in it Docker or import it as a Node libary.

Requirements

  • Postgres
  • Node 16.x or Docker

Run in Docker

The ETL can be started with Docker. Its config is set exclusively via environment variables. The following table dictates options and requirements.

Environment Variable Description Default Required
STAKING_PROGRAM_ID The address of the ZEE staking program - true
SOLANA_URL URL of the Solana RPC API - true
POSTGRES_PASSWORD Postgres password - true
POSTGRES_USER Postgres user admin false
POSTGRES_HOST Postgres host localhost false
POSTGRES_PORT Postgres port 5432 false
POSTGRES_DATABASE Postgres database zeetl false
SOLANA_RATE_LIMIT Min time between API requests in ms 250 false
LOG_LVL The log level info false
LOG_SILENT Set logs to silent false false

The easiest way to pass environment variables is to create a .env file containing your configuration. Here is an example:

POSTGRES_PASSWORD=dummy
SOLANA_URL=https://api.devnet.solana.com
SOLANA_RATE_LIMIT=250
STAKING_PROGRAM_ID=7vo1tfi7A7DfLi5viwb1eNwv9WUuphV2QS3TNv1nPUo5

Next, migrate the database schema.

// Insert the correct version.
docker run --rm --network host --env-file .env ormembaar/zeetl:VERSION bin/migrate

Finally, start the ETL.

// Again, insert the version
docker run -d --network host --env-file .env --name=zeetl ormembaar/zeetl:VERSION

Import as library

The ETL can also be used as a library.

const connection = new Connection('https://api.devnet.solana.com', 'confirmed');
const etl = new ETL({
    connection,
    postgresURL: 'postgres://...',
    stakeProgramId: '7vo1tfi7A7DfLi5viwb1eNwv9WUuphV2QS3TNv1nPUo5',
    rateLimit: 250
});
etl.on('error', (err) => log.error('ETL error', err));
etl.start();

Then you can query the database directly via the Prisma client embedded in the ETL.

const foo = await etl.client.stakeInitInstruction.findFirst();
console.log(foo);
/*
{
  id: 1,
  signatureId: 4,
  payer: 'FuGYQLy3jPhdUvRiAMtz71TdXwSdEVuV8xHq2XDPBAE6',
  staker: '5hWFiih4FNjGBJghozv1ZYBh4nEXyTNoCHSJgW1gg2Nj',
  stakerFund: 'BvQM3N9fw7bodh7eAjEvrmBeGkazi9wbQKHgMyEQpPGK',
  stakerBeneficiary: 'AKdjvmGqwbPQg3AtZeEzZ7fPHuDjRnXAWbznU2YzfHmt',
  communityAccount: 'ALismzdo2CZk96rAq9VMwCfRcHzhfCaA8JmjiJvc4UC',
  stakeAccount: 'BqRvpXgiT7svyn9Y83Ss6gqKkBcTECgLg7AtqnXFcFbX',
  zeeTokenMint: 'E3iGfQ6AdPLT2Q39JMNKnC78hEvoNTgqCmt7dydSAzpc'
}
*/

See the Prisma docs for help using the client. The client is generated from prisma/schema.prisma, which should be used for reference.

/@factoshi/zeetl/

    Package Sidebar

    Install

    npm i @factoshi/zeetl

    Weekly Downloads

    1

    Version

    0.1.1

    License

    MIT

    Unpacked Size

    99.5 kB

    Total Files

    38

    Last publish

    Collaborators

    • ormembaar