@antv/a8
TypeScript icon, indicating that this package has built-in type declarations

0.0.1 • Public • Published

@antv/a8

A library for audio visualization using the following rendering techniques:

We provide the following effects now:

gpu sinegpu stardust gpu blackhole

Getting Started

Install from NPM.

npm install @antv/a8

Create a audio, set effect and start playing.

import { Audio, Sine } from '@antv/a8';

const audio = new Audio({
  canvas: $canvas,
});
audio.data($audio).effect(new Sine()).play();

API

Constructor

new Audio({
  canvas: $canvas,
});
  • canvas HTMLCanvasElement

data()

Pass in an HTMLAudioElement, create a AudioContext and Analyser later.

audio.data($audio);

effect()

Mount an effect.

import { Sine } from '@antv/a8';

audio.effect(new Sine());
audio.effect(new Sine()); // switch to another

style()

Update style options of effect.

audio.style({ blur: 1 });

play()

Start visualizing the audio.

audio.play();

destroy()

Destroy rAF and GPU resources(if any).

audio.destroy();

Effects

We provide the following effect now.

  • GPU Particles

GPU Particles

When creating GPU particle effects, we should use a WASM to compile shader chunks. For more informations, see https://observablehq.com/@antv/compute-toys#cell-712

const shaderCompilerPath = new URL(
  '/public/glsl_wgsl_compiler_bg.wasm',
  import.meta.url,
).href;
const effect = new Stardust(shaderCompilerPath, {});

Let me briefly describe the implementation. The whole process inside compute shaders can be divided into four stages:

  • Simulate particles
  • Clear
  • Rasterize
  • Output to storage buffer
  • Blit to screen

The particle structure is really simple, it consists of 2 properties: position and velocity. We will load/store particles from/to storage textures later.

struct Particle {
  position: float4,
  velocity: float4,
}

fn LoadParticle(pix: int2) -> Particle {
  var p: Particle;
  p.position = textureLoad(pass_in, pix, 0, 0);
  p.velocity = textureLoad(pass_in, pix, 1, 0);
  return p;
}

fn SaveParticle(pix: int2, p: Particle) {
  textureStore(pass_out, pix, 0, p.position);
  textureStore(pass_out, pix, 1, p.velocity);
}

At the first frame, we assign the initial position & velocity for each particle.

@compute @workgroup_size(16, 16)
fn SimulateParticles(@builtin(global_invocation_id) id: uint3) {
  if (time.frame == 0u) {
    let rng = rand4();

    // Normalize from [0, 1] to [-1, 1].
    p.position = float4(2.0 * rng.xyz - 1.0, 0.0);
    p.velocity = float4(0.0, 0.0, 0.0, 0.0);
  }
}

And in each of the next frames, position will be updated with velocity.

let dt = custom.Speed * custom.TimeStep;
p.velocity += (ForceField(p.position.xyz, t) - custom.VelocityDecay * p.velocity) * dt;
p.position += p.velocity * dt;

GPU Sine

gpu sine

Online DEMO

  • radius number
  • sinea number
  • sineb number
  • speed number
  • blur number
  • samples number
  • mode number

GPU Stardust

gpu stardust

Online DEMO

  • radius number
  • timeStep number
  • samples number
  • blurRadius number
  • velocityDecay number
  • speed number
  • blurExponentA number
  • blurExponentB number
  • animatedNoise number
  • accumulation number
  • exposure number

GPU BlackHole

https://en.wikipedia.org/wiki/Kerr%E2%80%93Newman_metric

gpu blackhole

Online DEMO

  • radius number
  • timeStep number
  • samples number
  • animatedNoise number
  • accumulation number
  • exposure number
  • blurExponentA number
  • blurExponentB number
  • blurRadius number
  • kerrA number
  • kerrQ number
  • initSpeed number
  • initThick number
  • steps number
  • focalPlane number
  • motionBlur number
  • gamma number

Appendix

Readme

Keywords

Package Sidebar

Install

npm i @antv/a8

Weekly Downloads

3

Version

0.0.1

License

MIT

Unpacked Size

979 kB

Total Files

41

Last publish

Collaborators

  • xuying1027
  • lvisei
  • yisi.wang
  • basketduck
  • biupiubiupiu
  • flash1
  • dreammy23
  • laixingui.lxg
  • zhangjunjie-loki
  • rainy25ghz
  • zeyuwang
  • yanxiong
  • susiwen8
  • freestyle21
  • soundquiet
  • elaine.q.10
  • sturuby
  • sakuya223
  • serializedowen
  • xdzhao
  • yangzhanmei
  • wjgogogo
  • leungwensen
  • dori
  • iaaron
  • yard
  • simaq
  • dxq613
  • intchous
  • susan_ann
  • jinke.li
  • lzxue
  • army8735
  • atool
  • baizn
  • dengfuping
  • neoddish
  • jeffy2012
  • zqlu
  • afc163
  • pomelo-nwu
  • kopiluwaky
  • ccnuzindex
  • panyuqi
  • bubkoo
  • zengyue
  • kasmine
  • boyu.zlj
  • l1ud0ngq1
  • newbyvector
  • winniexing
  • chenluli
  • kn9117
  • xdddst
  • semious2020
  • esora
  • nadia_liu
  • bbsqq
  • mxz96102
  • openwayne
  • pearmini
  • pddpd
  • yiqianyao
  • zhanba
  • cxxxxxn