Fullfører oppgave 16.5: Sound pads med 4×2 pad-grid

Implementerer lydpads (inspirert av RødeCaster Pro II) i mixeren:

- mixer.ts: Nytt pad-system med AudioBuffer-caching, GainNode per pad,
  og one-shot AudioBufferSourceNode-avspilling. Funksjoner for load,
  play, stop, og gain-kontroll.

- livekit.ts: Data message-støtte (sendDataMessage, onDataMessage) for
  synkronisert pad-avspilling på tvers av LiveKit-deltakere. Bruker
  reliable delivery med topic-filtrering.

- SoundPadGrid.svelte: 4×2 responsivt pad-grid med fargede knapper.
  Forhåndslaster lydfiler fra CAS til AudioBuffer. Visuell feedback
  ved avspilling (scale-animasjon). Konfigurasjonsmodus for å sette
  label, farge og laste opp lydfil per pad. Pad-konfig lagres i
  metadata.mixer.pads på samlingsnoden.

- MixerTrait.svelte: Integrerer SoundPadGrid mellom kanalstriper og
  master-seksjon. Sender isViewer-prop for tilgangskontroll.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
vegard 2026-03-18 05:17:30 +00:00
parent 00d4df6f53
commit 8acb5a8731
6 changed files with 492 additions and 7 deletions

View file

@ -219,11 +219,11 @@ Lydmixeren aktiveres via `mixer`-traitet på en samlings-node. Krever at
- [x] Tilgangskontroll: eier/admin kan sette deltaker til "viewer" (kun observere) - [x] Tilgangskontroll: eier/admin kan sette deltaker til "viewer" (kun observere)
### Fase C: Sound Pads ### Fase C: Sound Pads
- [ ] Pad-grid UI (4×2 grid med fargede knapper) - [x] Pad-grid UI (4×2 grid med fargede knapper)
- [ ] Last lydfiler fra CAS → AudioBuffer - [x] Last lydfiler fra CAS → AudioBuffer
- [ ] Avspilling ved trykk (AudioBufferSourceNode) - [x] Avspilling ved trykk (AudioBufferSourceNode)
- [ ] Pad-konfigurasjon: velg lydfil, farge, label (lagres i node metadata) - [x] Pad-konfigurasjon: velg lydfil, farge, label (lagres i node metadata)
- [ ] LiveKit Data Message for synkronisert avspilling på tvers av deltakere - [x] LiveKit Data Message for synkronisert avspilling på tvers av deltakere
### Fase D: Lydbehandling (EQ) ### Fase D: Lydbehandling (EQ)
- [ ] Fat bottom (lowshelf filter) - [ ] Fat bottom (lowshelf filter)

View file

@ -3,6 +3,7 @@
import { edgeStore, nodeStore, mixerChannelStore, stdb } from '$lib/spacetime'; import { edgeStore, nodeStore, mixerChannelStore, stdb } from '$lib/spacetime';
import type { MixerChannel } from '$lib/spacetime'; import type { MixerChannel } from '$lib/spacetime';
import TraitPanel from './TraitPanel.svelte'; import TraitPanel from './TraitPanel.svelte';
import SoundPadGrid from './SoundPadGrid.svelte';
import { import {
getStatus, getStatus,
getParticipants, getParticipants,
@ -328,6 +329,11 @@
{/each} {/each}
</div> </div>
<!-- Sound Pads -->
<div class="mt-4">
<SoundPadGrid {collection} {accessToken} {isViewer} />
</div>
<!-- Master section --> <!-- Master section -->
<div class="mt-4 rounded-lg border-2 border-gray-200 bg-white p-3"> <div class="mt-4 rounded-lg border-2 border-gray-200 bg-white p-3">
<div class="flex flex-col gap-2 sm:flex-row sm:items-center sm:gap-3"> <div class="flex flex-col gap-2 sm:flex-row sm:items-center sm:gap-3">

View file

@ -0,0 +1,327 @@
<script lang="ts">
/**
* Sound Pad Grid (4×2) for the mixer.
*
* Each pad plays a preloaded AudioBuffer from CAS on press.
* Pad triggers are synchronized to other participants via LiveKit Data Messages.
* Pad configuration (label, color, cas_hash) is stored in the collection node's
* metadata.mixer.pads array and persisted via the API.
*/
import type { Node } from '$lib/spacetime';
import { casUrl, updateNode, uploadMedia } from '$lib/api';
import {
loadPadAudio,
isPadLoaded,
playPad,
stopPad,
isPadPlaying,
unloadAllPads,
ensureAudioContext,
} from '$lib/mixer';
import { onDataMessage, sendDataMessage, getLocalIdentity } from '$lib/livekit';
interface PadConfig {
label: string;
cas_hash: string;
color: string;
}
interface Props {
collection: Node;
accessToken?: string;
isViewer?: boolean;
}
let { collection, accessToken, isViewer = false }: Props = $props();
// Pad state
let padConfigs: PadConfig[] = $state([]);
let loadingPads = $state(new Set<number>());
let playingPads = $state(new Set<number>());
let configMode = $state(false);
// Edit state for config mode
let editLabel = $state('');
let editColor = $state('#6366f1');
let editIndex: number | null = $state(null);
const GRID_SIZE = 8; // 4×2
const DEFAULT_COLORS = [
'#EF4444', '#F97316', '#EAB308', '#22C55E',
'#06B6D4', '#3B82F6', '#8B5CF6', '#EC4899',
];
// Parse pad config from collection metadata
$effect(() => {
const meta = collection.metadata ? JSON.parse(collection.metadata) : {};
const mixerMeta = meta?.mixer ?? {};
const rawPads: PadConfig[] = mixerMeta?.pads ?? [];
// Ensure we always have GRID_SIZE slots
const configs: PadConfig[] = [];
for (let i = 0; i < GRID_SIZE; i++) {
configs.push(rawPads[i] ?? { label: '', cas_hash: '', color: DEFAULT_COLORS[i] });
}
padConfigs = configs;
});
// Load audio buffers when pad configs change
$effect(() => {
for (let i = 0; i < padConfigs.length; i++) {
const pad = padConfigs[i];
if (pad.cas_hash && !isPadLoaded(`pad_${i}`)) {
loadPadBuffer(i, pad.cas_hash);
}
}
return () => {
unloadAllPads();
};
});
// Listen for remote pad triggers via LiveKit Data Messages
$effect(() => {
const unsub = onDataMessage((payload, senderIdentity, topic) => {
if (topic !== 'pad_trigger') return;
// Don't replay our own triggers
if (senderIdentity === getLocalIdentity()) return;
try {
const msg = JSON.parse(new TextDecoder().decode(payload));
if (msg.action === 'play' && typeof msg.padIndex === 'number') {
triggerPadLocally(msg.padIndex);
}
} catch {
// Invalid message — ignore
}
});
return unsub;
});
async function loadPadBuffer(index: number, casHash: string) {
const padId = `pad_${index}`;
loadingPads = new Set([...loadingPads, index]);
try {
const url = casUrl(casHash);
await loadPadAudio(padId, url);
} catch (err) {
console.error(`Failed to load pad ${index}:`, err);
} finally {
const next = new Set(loadingPads);
next.delete(index);
loadingPads = next;
}
}
function triggerPadLocally(index: number) {
const padId = `pad_${index}`;
if (!isPadLoaded(padId)) return;
ensureAudioContext();
playPad(padId);
// Visual feedback
playingPads = new Set([...playingPads, index]);
setTimeout(() => {
const next = new Set(playingPads);
next.delete(index);
playingPads = next;
}, 300);
}
async function handlePadPress(index: number) {
if (isViewer) return;
const pad = padConfigs[index];
if (!pad.cas_hash) return;
triggerPadLocally(index);
// Broadcast to other participants
const msg = JSON.stringify({ action: 'play', padIndex: index });
const data = new TextEncoder().encode(msg);
await sendDataMessage(data, 'pad_trigger');
}
// Config mode functions
function startEditPad(index: number) {
editIndex = index;
editLabel = padConfigs[index].label;
editColor = padConfigs[index].color;
}
function cancelEdit() {
editIndex = null;
editLabel = '';
editColor = '#6366f1';
}
async function handleFileUpload(event: Event) {
if (editIndex === null || !accessToken) return;
const input = event.target as HTMLInputElement;
const file = input.files?.[0];
if (!file) return;
try {
const result = await uploadMedia(accessToken, { file, visibility: 'team' });
const newConfigs = [...padConfigs];
newConfigs[editIndex] = {
label: editLabel || file.name.replace(/\.[^/.]+$/, ''),
cas_hash: result.cas_hash,
color: editColor,
};
await savePadConfigs(newConfigs);
padConfigs = newConfigs;
// Load the new audio immediately
await loadPadBuffer(editIndex, result.cas_hash);
cancelEdit();
} catch (err) {
console.error('Failed to upload pad audio:', err);
}
}
async function saveEditedPad() {
if (editIndex === null || !accessToken) return;
const newConfigs = [...padConfigs];
newConfigs[editIndex] = {
...newConfigs[editIndex],
label: editLabel,
color: editColor,
};
await savePadConfigs(newConfigs);
padConfigs = newConfigs;
cancelEdit();
}
async function clearPad(index: number) {
if (!accessToken) return;
const newConfigs = [...padConfigs];
newConfigs[index] = { label: '', cas_hash: '', color: DEFAULT_COLORS[index] };
await savePadConfigs(newConfigs);
padConfigs = newConfigs;
stopPad(`pad_${index}`);
}
async function savePadConfigs(configs: PadConfig[]) {
if (!accessToken) return;
// Filter out empty pads for clean storage
const padsToSave = configs.filter(p => p.cas_hash);
const meta = collection.metadata ? JSON.parse(collection.metadata) : {};
const mixer = meta.mixer ?? {};
mixer.pads = padsToSave.length > 0 ? configs : undefined;
meta.mixer = mixer;
await updateNode(accessToken, {
node_id: collection.id,
metadata: meta,
});
}
</script>
<div class="space-y-2">
<div class="flex items-center justify-between">
<h4 class="text-xs font-semibold text-gray-600 uppercase tracking-wide">Sound Pads</h4>
{#if !isViewer}
<button
onclick={() => { configMode = !configMode; cancelEdit(); }}
class="text-xs text-gray-400 hover:text-gray-600 transition-colors"
>
{configMode ? 'Ferdig' : 'Rediger'}
</button>
{/if}
</div>
<!-- 4×2 pad grid -->
<div class="grid grid-cols-4 gap-1.5 sm:gap-2">
{#each padConfigs as pad, index (index)}
{@const isPlaying = playingPads.has(index)}
{@const isLoading = loadingPads.has(index)}
{@const isEmpty = !pad.cas_hash}
{@const isEditing = configMode && editIndex === index}
{#if isEditing}
<!-- Edit mode for this pad -->
<div class="col-span-4 rounded-lg border border-indigo-200 bg-indigo-50 p-3 space-y-2">
<div class="flex items-center gap-2">
<input
type="text"
bind:value={editLabel}
placeholder="Label"
class="flex-1 rounded border border-gray-300 px-2 py-1 text-xs"
/>
<input
type="color"
bind:value={editColor}
class="h-7 w-7 rounded border border-gray-300 cursor-pointer"
/>
</div>
<div class="flex items-center gap-2">
<label class="flex-1 cursor-pointer rounded bg-indigo-600 px-3 py-1.5 text-center text-xs font-medium text-white hover:bg-indigo-700 transition-colors">
{pad.cas_hash ? 'Bytt lydfil' : 'Last opp lydfil'}
<input type="file" accept="audio/*" onchange={handleFileUpload} class="hidden" />
</label>
{#if pad.cas_hash}
<button
onclick={saveEditedPad}
class="rounded bg-green-600 px-3 py-1.5 text-xs font-medium text-white hover:bg-green-700 transition-colors"
>
Lagre
</button>
{/if}
<button
onclick={cancelEdit}
class="rounded bg-gray-200 px-3 py-1.5 text-xs font-medium text-gray-600 hover:bg-gray-300 transition-colors"
>
Avbryt
</button>
</div>
</div>
{:else}
<!-- Pad button with optional clear badge -->
<div class="relative">
<button
onclick={() => configMode ? startEditPad(index) : handlePadPress(index)}
disabled={!configMode && (isEmpty || isLoading || isViewer)}
class="w-full aspect-square rounded-lg border-2 transition-all duration-100 flex flex-col items-center justify-center gap-0.5
{isEmpty
? 'border-dashed border-gray-300 bg-gray-50 text-gray-400'
: isPlaying
? 'border-white shadow-lg scale-95'
: 'border-gray-200 hover:border-gray-300 shadow-sm hover:shadow'}
{isLoading ? 'animate-pulse' : ''}
disabled:opacity-40 disabled:cursor-not-allowed"
style={isEmpty ? '' : `background-color: ${pad.color}; border-color: ${pad.color}`}
title={isEmpty ? (configMode ? 'Klikk for å legge til lyd' : 'Tom pad') : pad.label}
>
{#if isEmpty}
{#if configMode}
<span class="text-lg leading-none">+</span>
<span class="text-[10px]">Legg til</span>
{:else}
<span class="text-[10px]">Tom</span>
{/if}
{:else}
<span class="text-xs font-bold text-white drop-shadow-sm truncate px-1 max-w-full leading-tight">
{pad.label || '?'}
</span>
{#if isLoading}
<span class="text-[9px] text-white/70">Laster...</span>
{/if}
{/if}
</button>
{#if configMode && !isEmpty}
<button
onclick={() => clearPad(index)}
class="absolute -top-1 -right-1 h-4 w-4 rounded-full bg-red-500 text-white text-[10px] leading-none flex items-center justify-center shadow hover:bg-red-600 z-10"
title="Fjern pad"
>
×
</button>
{/if}
</div>
{/if}
{/each}
</div>
</div>

View file

@ -186,6 +186,12 @@ export async function connect(wsUrl: string, token: string): Promise<void> {
}) })
.on(RoomEvent.ActiveSpeakersChanged, () => { .on(RoomEvent.ActiveSpeakersChanged, () => {
refreshParticipants(); refreshParticipants();
})
.on(RoomEvent.DataReceived, (payload: Uint8Array, participant?: RemoteParticipant, kind?: DataPacket_Kind, topic?: string) => {
const senderIdentity = participant?.identity;
for (const handler of dataListeners) {
handler(payload, senderIdentity, topic);
}
}); });
room = newRoom; room = newRoom;
@ -232,3 +238,30 @@ export async function toggleMute(): Promise<boolean> {
export function isConnected(): boolean { export function isConnected(): boolean {
return room?.state === ConnectionState.Connected; return room?.state === ConnectionState.Connected;
} }
// ─── Data Messages (for sound pad sync) ────────────────────────────────────
export type DataMessageHandler = (payload: Uint8Array, senderIdentity: string | undefined, topic: string | undefined) => void;
const dataListeners = new Set<DataMessageHandler>();
/**
* Subscribe to incoming data messages from other participants.
* Returns an unsubscribe function.
*/
export function onDataMessage(handler: DataMessageHandler): () => void {
dataListeners.add(handler);
return () => { dataListeners.delete(handler); };
}
/**
* Send a reliable data message to all participants in the room.
* Uses DataPacket_Kind.RELIABLE for guaranteed delivery.
*/
export async function sendDataMessage(payload: Uint8Array, topic?: string): Promise<void> {
if (!room || room.state !== ConnectionState.Connected) return;
await room.localParticipant.publishData(payload, {
reliable: true,
topic,
});
}

View file

@ -18,6 +18,12 @@ export interface MixerChannel {
gain: GainNode; gain: GainNode;
} }
export interface PadState {
buffer: AudioBuffer;
gain: GainNode;
activeSource: AudioBufferSourceNode | null;
}
export interface ChannelLevels { export interface ChannelLevels {
identity: string; identity: string;
peak: number; // 0.01.0, peak amplitude peak: number; // 0.01.0, peak amplitude
@ -31,6 +37,7 @@ let masterGain: GainNode | null = null;
let masterAnalyser: AnalyserNode | null = null; let masterAnalyser: AnalyserNode | null = null;
const channels = new Map<string, MixerChannel>(); const channels = new Map<string, MixerChannel>();
const pads = new Map<string, PadState>();
// Reusable buffer for analyser readings (allocated once per context) // Reusable buffer for analyser readings (allocated once per context)
let analyserBuffer: Float32Array | null = null; let analyserBuffer: Float32Array | null = null;
@ -257,6 +264,117 @@ function readAnalyserLevels(identity: string, analyser: AnalyserNode): ChannelLe
}; };
} }
// ─── Sound Pads ─────────────────────────────────────────────────────────────
/**
* Load an audio file from a URL into an AudioBuffer for a pad.
* Caches the buffer so subsequent plays are instant.
*/
export async function loadPadAudio(padId: string, url: string): Promise<void> {
const ctx = ensureAudioContext();
const response = await fetch(url);
if (!response.ok) throw new Error(`Failed to fetch pad audio: ${response.status}`);
const arrayBuffer = await response.arrayBuffer();
const audioBuffer = await ctx.decodeAudioData(arrayBuffer);
// Create a dedicated gain node for this pad, connected to master
const existing = pads.get(padId);
const gain = existing?.gain ?? ctx.createGain();
if (!existing) {
gain.gain.value = 1.0;
gain.connect(masterGain!);
}
pads.set(padId, { buffer: audioBuffer, gain, activeSource: existing?.activeSource ?? null });
}
/**
* Check if a pad's audio buffer is loaded and ready to play.
*/
export function isPadLoaded(padId: string): boolean {
return pads.has(padId);
}
/**
* Play a sound pad. Creates a new AudioBufferSourceNode each time
* (they are one-shot cannot be restarted after stopping).
* If the pad is already playing, stops the current playback first.
*/
export function playPad(padId: string): void {
const pad = pads.get(padId);
if (!pad || !audioContext) return;
// Stop any currently playing instance
stopPad(padId);
const source = audioContext.createBufferSource();
source.buffer = pad.buffer;
source.connect(pad.gain);
// Clean up reference when playback ends naturally
source.onended = () => {
if (pad.activeSource === source) {
pad.activeSource = null;
}
};
pad.activeSource = source;
source.start(0);
}
/**
* Stop a currently playing pad.
*/
export function stopPad(padId: string): void {
const pad = pads.get(padId);
if (!pad?.activeSource) return;
try {
pad.activeSource.stop();
} catch {
// Already stopped — ignore
}
pad.activeSource = null;
}
/**
* Set gain for a specific pad (0.01.5).
*/
export function setPadGain(padId: string, value: number): void {
const pad = pads.get(padId);
if (!pad) return;
pad.gain.gain.value = Math.max(0, Math.min(1.5, value));
}
/**
* Check if a pad is currently playing.
*/
export function isPadPlaying(padId: string): boolean {
const pad = pads.get(padId);
return pad?.activeSource !== null && pad?.activeSource !== undefined;
}
/**
* Unload a pad's audio buffer and disconnect its gain node.
*/
export function unloadPad(padId: string): void {
const pad = pads.get(padId);
if (!pad) return;
stopPad(padId);
pad.gain.disconnect();
pads.delete(padId);
}
/**
* Unload all pads.
*/
export function unloadAllPads(): void {
for (const [padId] of pads) {
unloadPad(padId);
}
}
// ─── Cleanup ─────────────────────────────────────────────────────────────── // ─── Cleanup ───────────────────────────────────────────────────────────────
/** /**
@ -267,6 +385,8 @@ export function destroyMixer(): void {
removeChannel(identity); removeChannel(identity);
} }
unloadAllPads();
if (masterAnalyser) { if (masterAnalyser) {
masterAnalyser.disconnect(); masterAnalyser.disconnect();
masterAnalyser = null; masterAnalyser = null;

View file

@ -181,8 +181,7 @@ Ref: `docs/features/lydmixer.md`
- [x] 16.2 Web Audio mixer-graf: opprett `AudioContext`, `MediaStreamSourceNode` per remote track → per-kanal `GainNode` → master `GainNode``destination`. `AnalyserNode` per kanal for VU-meter. - [x] 16.2 Web Audio mixer-graf: opprett `AudioContext`, `MediaStreamSourceNode` per remote track → per-kanal `GainNode` → master `GainNode``destination`. `AnalyserNode` per kanal for VU-meter.
- [x] 16.3 Mixer-UI (MixerTrait-komponent): kanalstripe per deltaker med volumslider (0150%), nød-mute-knapp (stor, rød), VU-meter (canvas/CSS), navnelabel. Master-fader og master-mute. Responsivt design (mobil: kompakt fader-modus). - [x] 16.3 Mixer-UI (MixerTrait-komponent): kanalstripe per deltaker med volumslider (0150%), nød-mute-knapp (stor, rød), VU-meter (canvas/CSS), navnelabel. Master-fader og master-mute. Responsivt design (mobil: kompakt fader-modus).
- [x] 16.4 Delt mixer-kontroll via SpacetimeDB: `MixerChannel`-tabell + reducers (`set_gain`, `set_mute`, `toggle_effect`). Frontend abonnerer og oppdaterer Web Audio-graf ved endring fra andre deltakere. Visuell feedback (sliders beveger seg i sanntid). Tilgangskontroll: eier/admin kan sette deltaker til viewer-modus. - [x] 16.4 Delt mixer-kontroll via SpacetimeDB: `MixerChannel`-tabell + reducers (`set_gain`, `set_mute`, `toggle_effect`). Frontend abonnerer og oppdaterer Web Audio-graf ved endring fra andre deltakere. Visuell feedback (sliders beveger seg i sanntid). Tilgangskontroll: eier/admin kan sette deltaker til viewer-modus.
- [~] 16.5 Sound pads: pad-grid UI (4×2), forhåndslast lydfiler fra CAS til `AudioBuffer`. Avspilling ved trykk (`AudioBufferSourceNode`). Pad-konfig i `metadata.mixer.pads` (label, farge, cas_hash). Synkronisert avspilling via LiveKit Data Message. - [x] 16.5 Sound pads: pad-grid UI (4×2), forhåndslast lydfiler fra CAS til `AudioBuffer`. Avspilling ved trykk (`AudioBufferSourceNode`). Pad-konfig i `metadata.mixer.pads` (label, farge, cas_hash). Synkronisert avspilling via LiveKit Data Message.
> Påbegynt: 2026-03-18T05:09
- [ ] 16.6 EQ-effektkjede: fat bottom (`BiquadFilterNode` lowshelf ~200Hz), sparkle (`BiquadFilterNode` highshelf ~10kHz), exciter (`WaveShaperNode` + highshelf). Per-kanal toggles, synkronisert via STDB. Presets (podcast-stemme, radio-stemme). - [ ] 16.6 EQ-effektkjede: fat bottom (`BiquadFilterNode` lowshelf ~200Hz), sparkle (`BiquadFilterNode` highshelf ~10kHz), exciter (`WaveShaperNode` + highshelf). Per-kanal toggles, synkronisert via STDB. Presets (podcast-stemme, radio-stemme).
- [ ] 16.7 Stemmeeffekter: robotstemme (ring-modulasjon: `OscillatorNode``GainNode.gain`), monsterstemme (egenutviklet `AudioWorkletProcessor` med phase vocoder for pitch shift). Effektvelger-UI per kanal. Parameterjustering (pitch-faktor, oscillator-frekvens). - [ ] 16.7 Stemmeeffekter: robotstemme (ring-modulasjon: `OscillatorNode``GainNode.gain`), monsterstemme (egenutviklet `AudioWorkletProcessor` med phase vocoder for pitch shift). Effektvelger-UI per kanal. Parameterjustering (pitch-faktor, oscillator-frekvens).