_2DNote: 2D Note Generator 
Purpose: An experiment in making visual interfaces more accessible to people who are blind. Screen readers linearize your page content, so they don't communicate 2-dimensional visuals very well. You can address that by mapping 2D positions to notes.
Potential Uses: Create 2D games and apps that people can use together, regardless of vision capabilities? Hear the contours of your signature as you write it? Enable people to "hear" what someone else is drawing? Locate elements on a page? Convert graphs into sound? Etc.
Some Design Choices: _2DNote does not rely on binaural audio, so devices with only 1 speaker can still communicate the 2D position of one or more elements simultaneously. It does that by relying on volume and pitch. Brain plasticity will take care of the rest.
You can think of the y dimension as the "proximity dimension" (louder = closer to you), and the x dimension as the "piano dimension" (left-to-right = low-to-high notes).
_2DNote.js is made from pure "vanilla" JavaScript. So no extra dependencies.
Further reading
Live Demo
https://codepen.io/hchiam/full/eYYdVeX
(You can live edit the demo code in real time here: https://codepen.io/hchiam/pen/eYYdVeX)
Can I include it in my own HTML code?
Yes:
<script
src="https://cdn.jsdelivr.net/gh/hchiam/_2DNote@1.9.0/_2DNote.min.js"
integrity="sha384-w9KzgKC5k/00txI3ng7+D2iZ/1HiglYG7YpmTdR8NnwbCr+0qcGiaZC1JGzgOWgD"
crossorigin="anonymous"
></script>While not recommended, you can auto-update to the latest by linking the src to:
-
https://cdn.jsdelivr.net/gh/hchiam/_2DNote@master/_2DNote.min.jsor https://cdn.jsdelivr.net/gh/hchiam/_2DNote@latest/_2DNote.min.js
By including _2DNote.js, you get an object _2DNote, which you can use in your HTML or JS code.
For example:
<div onmousemove="_2DNote.update(event,colourSoundIcon);">...</div>For quick setup on the body tag:
<body>
<script
src="https://cdn.jsdelivr.net/gh/hchiam/_2DNote@1.9.0/_2DNote.min.js"
integrity="sha384-w9KzgKC5k/00txI3ng7+D2iZ/1HiglYG7YpmTdR8NnwbCr+0qcGiaZC1JGzgOWgD"
crossorigin="anonymous"
></script>
<script>
_2DNote.setAs2DArea(document.body, callbackUponUpdate);
function callbackUponUpdate() {
// do something whenever _2DNote.update() is triggered;
}
</script>
</body>For quick setup of a custom 2D click/touch area:
<body>
<div id="2d-area" style="width: 100vw; height: 100vh;">...</div>
...
<script
src="https://cdn.jsdelivr.net/gh/hchiam/_2DNote@1.9.0/_2DNote.min.js"
integrity="sha384-w9KzgKC5k/00txI3ng7+D2iZ/1HiglYG7YpmTdR8NnwbCr+0qcGiaZC1JGzgOWgD"
crossorigin="anonymous"
></script>
<script>
_2DNote.setAs2DArea(document.getElementById("2d-area"), callbackUponUpdate);
function callbackUponUpdate() {
// do something whenever _2DNote.update() is triggered;
}
</script>
</body>For a full working code example, see example-include.html
Here's an example of 2 notes playing simultaneously: example-two-notes.html
API
The things you're most likely to use are: .play(e), .update(e), .stop(), and .copy().
(Full details in _2DNote.js.)
_2DNote.audioContext:
- This is the AudioContext used by the
_2DNoteinstance to do the work of actually creating the note.
_2DNote.note:
- This holds the information about the note (oscillator, volumeSetup) that the
_2DNoteinstance can play.
_2DNote.play(e):
- This plays a note based on the position of the element or mouse click event, which it figures out for you. (
eis for event or element.) - Note: The note automatically stops playing if the cursor moves outside the window. This avoids having users click something outside the window without realizing it.
-
UX reminder: If you plan to use
_2DNotein only one area of the page, you should let users know when the cursor moves outside of that area. To do that, try something like<tag-name onmouseleave="_2DNote.stop();">on the enclosing HTML element tag. Seeexample-include.htmlorexample-two-notes.htmlfor examples placed on thebodytag. If you'd like to disable thewarnExitedViewwhen the mouse leaves thebody, then you can rundocument.body.removeEventListener("mouseleave",_2DNote.warnExitedView);
_2DNote.update(e) or _2DNote.update(e, callback):
- This causes the note that the
_2DNoteinstance is playing to update to the current position of the element or mouse click event, which it figures out for you. (eis for event or element.) -
callbackis an optional parameter, and is a function that will be run from withinupdate()with this signature:callback(volume, frequency, pan). See full details in_2DNote.js. See usage example inexample-include.htmlorexample-two-notes.html.
_2DNote.stop():
- This stops the note that the
_2DNoteinstance is currently playing.
_2DNote.copy():
- This returns a deep clone of the
_2DNoteinstance. Extra instances make it more convenient to play multiple notes at the same time.
_2DNote.getFrequency(e):
- This returns the note frequency based on the x coordinate of the element or mouse click event, which it figures out for you. (
eis for event or element.)
_2DNote.getVolume(e):
- This returns the note volume based on the y coordinate of the element or mouse click event, which it figures out for you. (
eis for event or element.)
_2DNote.getPan(e):
- This returns the note panning based on the x coordinate of the element or mouse click event, which it figures out for you. (
eis for event or element.) Panning creates binaural audio: if you use headphones, it'll sound like the audio is coming from the left or right side of you.
_2DNote.normalize(value, inputRange, outputRange):
- This maps the value found within the input range (
[inputRangeMin, inputRangeMax]) to be in the chosen output range ([outputRangeMin, outputRangeMax]). It returns the re-mapped value.
Also available on NPM
https://www.npmjs.com/package/2dnote
npm i 2dnoteIf you're interested in general accessibility
https://github.com/hchiam/web-accessibility-course-notes#web-accessibility-a11y-course-notes
Other accessibility-related repos
https://github.com/hchiam/flying-focus
https://github.com/hchiam/keyboard-focus-trap
Local development
bash package.sh