Init

README.md

@@ -8,76 +8,85 @@ pinned: false
 app_build_command: npm run build
 app_file: build/index.html
 license: mit
-short_description: NVIDIA Parakeet speech recognition for the browser (WebGPU)
+short_description: NVIDIA Parakeet speech recognition for the browser (WebGPU/WASM)
+models:
+- ysdede/parakeet-tdt-0.6b-v2-onnx
+tags:
+- parakeet
+- speech
+- onnx
+- webgpu
+- wasm
+- transcription
+- nvidia
+- speech-recognition
+- browser
 ---
 
-# Getting Started with Create React App
+# 🐠 Parakeet.js - HF Spaces Demo
 
-This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
+> **NVIDIA Parakeet speech recognition for the browser using WebGPU/WASM**
 
-## Available Scripts
+This demo showcases the **[parakeet.js](https://www.npmjs.com/package/parakeet.js)** library, which brings NVIDIA's Parakeet speech recognition models to the browser using ONNX Runtime Web with WebGPU and WASM backends.
 
-In the project directory, you can run:
+## 🚀 Features
 
-### `npm start`
+- **🖥️ Browser-based**: Runs entirely in your browser - no server required
+- **⚡ WebGPU acceleration**: Fast inference using WebGPU when available
+- **🔧 WASM fallback**: CPU-based inference using WebAssembly
+- **📱 Multiple formats**: Supports various audio formats (WAV, MP3, etc.)
+- **🎯 Real-time performance**: Optimized for fast transcription
+- **📊 Performance metrics**: Shows detailed timing information
+- **🎛️ Configurable**: Adjustable quantization, preprocessing, and backend settings
 
-Runs the app in the development mode.\
-Open [http://localhost:3000](http://localhost:3000) to view it in your browser.
+## 🔧 How to Use
 
-The page will reload when you make changes.\
-You may also see any lint errors in the console.
+1. **Click "Load Model"** to download and initialize the speech recognition model
+2. **Select your preferences**:
+   - **Backend**: Choose WebGPU (faster) or WASM (more compatible)
+   - **Quantization**: fp32 (higher quality) or int8 (faster)
+   - **Preprocessor**: Different audio processing options
+3. **Upload an audio file** using the file input
+4. **View the transcription** in real-time with performance metrics
 
-### `npm test`
+## 📦 Integration
 
-Launches the test runner in the interactive watch mode.\
-See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
+You can use parakeet.js in your own projects:
 
-### `npm run build`
+```bash
+npm install parakeet.js onnxruntime-web
+```
 
-Builds the app for production to the `build` folder.\
-It correctly bundles React in production mode and optimizes the build for the best performance.
+```javascript
+import { ParakeetModel, getParakeetModel } from 'parakeet.js';
 
-The build is minified and the filenames include the hashes.\
-Your app is ready to be deployed!
+// Load model from HuggingFace Hub
+const modelUrls = await getParakeetModel('ysdede/parakeet-tdt-0.6b-v2-onnx');
+const model = await ParakeetModel.fromUrls(modelUrls);
 
-See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
+// Transcribe audio
+const result = await model.transcribe(audioData, sampleRate);
+console.log(result.utterance_text);
+```
 
-### `npm run eject`
+## 🔗 Links
 
-**Note: this is a one-way operation. Once you `eject`, you can't go back!**
+- **📚 [GitHub Repository](https://github.com/ysdede/parakeet.js)** - Source code and documentation
+- **📦 [npm Package](https://www.npmjs.com/package/parakeet.js)** - Install via npm
+- **🤖 [NVIDIA Parakeet Model](https://huggingface.co/nvidia/parakeet-tdt-1.1b)** - Original model on HuggingFace
 
-If you aren't satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
+## 🧠 Model Information
 
-Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you're on your own.
+This demo uses the **ysdede/parakeet-tdt-0.6b-v2-onnx** model, which is an ONNX-converted version of NVIDIA's Parakeet speech recognition model optimized for browser deployment.
 
-You don't have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn't feel obligated to use this feature. However we understand that this tool wouldn't be useful if you couldn't customize it when you are ready for it.
+## 💡 Technical Details
 
-## Learn More
+- **Model Format**: ONNX for cross-platform compatibility
+- **Backends**: WebGPU (GPU acceleration) and WASM (CPU fallback)
+- **Quantization**: Support for both fp32 and int8 precision
+- **Audio Processing**: Built-in preprocessing for various audio formats
+- **Performance**: Real-time factor (RTF) typically < 1.0x for fast transcription
 
-You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
-
-To learn React, check out the [React documentation](https://reactjs.org/).
-
-### Code Splitting
-
-This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting)
-
-### Analyzing the Bundle Size
-
-This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size)
-
-### Making a Progressive Web App
-
-This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app)
-
-### Advanced Configuration
-
-This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration)
-
-### Deployment
-
-This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment)
-
-### `npm run build` fails to minify
+---
 
-This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify)
+*Built with ❤️ using React and deployed on Hugging Face Spaces*

package.json

@@ -1,5 +1,5 @@
 {
-  "name": "react-template",
+  "name": "parakeet-js-hf-spaces-demo",
   "version": "0.1.0",
   "private": true,
   "dependencies": {
@@ -7,6 +7,8 @@
     "@testing-library/jest-dom": "^6.6.3",
     "@testing-library/react": "^16.3.0",
     "@testing-library/user-event": "^13.5.0",
+    "parakeet.js": "^0.0.1",
+    "onnxruntime-web": "1.22.0-dev.20250409-89f8206ba4",
     "react": "^19.1.0",
     "react-dom": "^19.1.0",
     "react-scripts": "5.0.1",

src/App.css

@@ -1,38 +1,170 @@
-.App {
-  text-align: center;
+:root {
+  font-family: Inter, system-ui, sans-serif;
+  line-height: 1.4;
+  color: #222;
+  background: #f3f6f8;
 }
 
-.App-logo {
-  height: 40vmin;
-  pointer-events: none;
+.app {
+  max-width: 760px;
+  margin: 2rem auto;
+  background: #ffffff;
+  border-radius: 8px;
+  padding: 1.5rem 2rem;
+  box-shadow: 0 4px 14px rgba(0, 0, 0, 0.06);
 }
 
-@media (prefers-reduced-motion: no-preference) {
-  .App-logo {
-    animation: App-logo-spin infinite 20s linear;
-  }
+.controls {
+  display: flex;
+  flex-wrap: wrap;
+  gap: 0.75rem;
+  align-items: center;
+  margin-bottom: 1rem;
 }
 
-.App-header {
-  background-color: #282c34;
-  min-height: 100vh;
+.controls label {
+  font-size: 0.9rem;
   display: flex;
-  flex-direction: column;
   align-items: center;
-  justify-content: center;
-  font-size: calc(10px + 2vmin);
-  color: white;
+  gap: 0.35rem;
+}
+
+.controls select,
+.controls input[type="number"] {
+  padding: 0.25rem 0.5rem;
+  border: 1px solid #d1d5db;
+  border-radius: 4px;
+  background: #fff;
+}
+
+button.primary {
+  padding: 0.4rem 0.9rem;
+  background: #3b82f6;
+  color: #ffffff;
+  border: none;
+  border-radius: 4px;
+  cursor: pointer;
+}
+
+button.primary:hover {
+  background: #2563eb;
+}
+
+.status {
+  margin-top: 0.5rem;
+  font-weight: 500;
+}
+
+.progress-wrapper {
+  margin: 0.5rem 0;
+}
+
+.progress-bar {
+  height: 8px;
+  background: #e2e8f0;
+  border-radius: 4px;
+  overflow: hidden;
+}
+
+.progress-bar > div {
+  height: 100%;
+  background: #10b981;
+  transition: width 0.2s;
+}
+
+.progress-text {
+  font-size: 0.8rem;
+  color: #555;
+  margin-top: 0.25rem;
+}
+
+.textarea {
+  width: 100%;
+  height: 6rem;
+  resize: vertical;
+  padding: 0.75rem;
+  border: 1px solid #d1d5db;
+  border-radius: 4px;
+  font-family: inherit;
+  font-size: 0.9rem;
+}
+
+.performance {
+  font-size: 0.85rem;
+  background: #ecfdf5;
+  padding: 0.5rem 0.75rem;
+  border-radius: 6px;
+  border: 1px solid #d1fae5;
+  margin-bottom: 1rem;
+}
+
+.history {
+  margin-top: 1rem;
+}
+
+.history h3 {
+  margin-bottom: 0.5rem;
+  color: #333;
+}
+
+.history-item {
+  padding: 1rem;
+  border-bottom: 1px solid #f1f5f9;
+  background: #ffffff;
+}
+
+.history-item:last-child {
+  border-bottom: none;
+}
+
+.history-meta {
+  display: flex;
+  justify-content: space-between;
+  font-size: 0.9rem;
+  color: #666;
+  margin-bottom: 0.5rem;
+}
+
+.history-stats {
+  font-size: 0.75rem;
+  color: #666;
+  margin-bottom: 0.5rem;
+}
+
+.history-text {
+  background: #f9fafb;
+  padding: 0.5rem 0.75rem;
+  border-radius: 4px;
+  border: 1px solid #e5e7eb;
+  font-size: 0.9rem;
+}
+
+/* HF Spaces specific styles */
+.app h2 {
+  margin-top: 0;
+  color: #1f2937;
+}
+
+.app p {
+  margin-bottom: 1rem;
+  color: #6b7280;
+}
+
+.app h3 {
+  color: #374151;
+  margin-bottom: 0.5rem;
+}
+
+.app h4 {
+  color: #374151;
+  margin-bottom: 0.5rem;
 }
 
-.App-link {
-  color: #61dafb;
+.app a {
+  color: #3b82f6;
+  text-decoration: none;
 }
 
-@keyframes App-logo-spin {
-  from {
-    transform: rotate(0deg);
-  }
-  to {
-    transform: rotate(360deg);
-  }
+.app a:hover {
+  text-decoration: underline;
 }

src/App.js

@@ -1,25 +1,320 @@
-import logo from './logo.svg';
+import React, { useState, useRef, useEffect } from 'react';
+import { ParakeetModel, getParakeetModel } from 'parakeet.js';
 import './App.css';
 
-function App() {
+export default function App() {
+  const repoId = 'ysdede/parakeet-tdt-0.6b-v2-onnx';
+  const [backend, setBackend] = useState('webgpu-hybrid');
+  const [quant, setQuant] = useState('fp32');
+  const [preprocessor, setPreprocessor] = useState('nemo128');
+  const [status, setStatus] = useState('Idle');
+  const [progress, setProgress] = useState('');
+  const [progressText, setProgressText] = useState('');
+  const [progressPct, setProgressPct] = useState(null);
+  const [text, setText] = useState('');
+  const [latestMetrics, setLatestMetrics] = useState(null);
+  const [transcriptions, setTranscriptions] = useState([]);
+  const [isTranscribing, setIsTranscribing] = useState(false);
+  const [verboseLog, setVerboseLog] = useState(false);
+  const [decoderInt8, setDecoderInt8] = useState(true);
+  const [frameStride, setFrameStride] = useState(1);
+  const [dumpDetail, setDumpDetail] = useState(false);
+  const maxCores = navigator.hardwareConcurrency || 8;
+  const [cpuThreads, setCpuThreads] = useState(Math.max(1, maxCores - 2));
+  const modelRef = useRef(null);
+  const fileInputRef = useRef(null);
+
+  // Auto-adjust quant preset when backend changes
+  useEffect(() => {
+    if (backend.startsWith('webgpu')) {
+      setQuant('fp32');
+    } else if (backend === 'wasm') {
+      setQuant('int8');
+    }
+  }, [backend]);
+
+  async function loadModel() {
+    setStatus('Loading model…');
+    setProgress('');
+    setProgressText('');
+    setProgressPct(0);
+    console.time('LoadModel');
+
+    try {
+      const progressCallback = ({ loaded, total, file }) => {
+        const pct = total > 0 ? Math.round((loaded / total) * 100) : 0;
+        setProgressText(`${file}: ${pct}%`);
+        setProgressPct(pct);
+      };
+
+      // 1. Download all model files from HuggingFace Hub
+      const modelUrls = await getParakeetModel(repoId, { 
+        quantization: quant, 
+        preprocessor,
+        backend, // Pass backend to enable automatic fp32 selection for WebGPU
+        decoderInt8,
+        progress: progressCallback 
+      });
+
+      // Show compiling sessions stage
+      setStatus('Creating sessions…');
+      setProgressText('Compiling model (this may take ~10 s)…');
+      setProgressPct(null);
+
+      // 2. Create the model instance with all file URLs
+      modelRef.current = await ParakeetModel.fromUrls({ 
+        ...modelUrls.urls,
+        filenames: modelUrls.filenames,
+        backend, 
+        verbose: verboseLog,
+        decoderOnWasm: decoderInt8, // if we selected int8 decoder, keep it on WASM
+        decoderInt8,
+        cpuThreads,
+      });
+
+      // 3. Warm-up and verify
+      setStatus('Warming up & verifying…');
+      setProgressText('Model ready! Upload an audio file to transcribe.');
+      setProgressPct(null);
+
+      console.timeEnd('LoadModel');
+      setStatus('Model ready ✔');
+      setProgressText('');
+    } catch (e) {
+      console.error(e);
+      setStatus(`Failed: ${e.message}`);
+      setProgress('');
+    }
+  }
+
+  async function transcribeFile(e) {
+    if (!modelRef.current) return alert('Load model first');
+    const file = e.target.files?.[0];
+    if (!file) return;
+
+    setIsTranscribing(true);
+    setStatus(`Transcribing "${file.name}"…`);
+
+    try {
+      const buf = await file.arrayBuffer();
+      const audioCtx = new AudioContext({ sampleRate: 16000 });
+      const decoded = await audioCtx.decodeAudioData(buf);
+      const pcm = decoded.getChannelData(0);
+
+      console.time(`Transcribe-${file.name}`);
+      const res = await modelRef.current.transcribe(pcm, 16_000, { 
+        returnTimestamps: true, 
+        returnConfidences: true, 
+        frameStride
+      });
+      console.timeEnd(`Transcribe-${file.name}`);
+
+      if (dumpDetail) {
+        console.log('[Parakeet] Detailed transcription output', res);
+      }
+      setLatestMetrics(res.metrics);
+      // Add to transcriptions list
+      const newTranscription = {
+        id: Date.now(),
+        filename: file.name,
+        text: res.utterance_text,
+        timestamp: new Date().toLocaleTimeString(),
+        duration: pcm.length / 16000, // duration in seconds
+        wordCount: res.words?.length || 0,
+        confidence: res.confidence_scores?.overall_log_prob || null,
+        metrics: res.metrics
+      };
+
+      setTranscriptions(prev => [newTranscription, ...prev]);
+      setText(res.utterance_text); // Show latest transcription
+      setStatus('Model ready ✔'); // Ready for next file
+      
+    } catch (error) {
+      console.error('Transcription failed:', error);
+      setStatus('Transcription failed');
+      alert(`Failed to transcribe "${file.name}": ${error.message}`);
+    } finally {
+      setIsTranscribing(false);
+      // Clear the file input so the same file can be selected again
+      if (fileInputRef.current) {
+        fileInputRef.current.value = '';
+      }
+    }
+  }
+
+  function clearTranscriptions() {
+    setTranscriptions([]);
+    setText('');
+  }
+
   return (
-    <div className="App">
-      <header className="App-header">
-        <img src={logo} className="App-logo" alt="logo" />
+    <div className="app">
+      <h2>🐠 Parakeet.js - HF Spaces Demo</h2>
+      <p>NVIDIA Parakeet speech recognition for the browser using WebGPU/WASM</p>
+
+      <div className="controls">
         <p>
-          Edit <code>src/App.js</code> and save to reload.
+          <strong>Model:</strong> {repoId}
         </p>
-        <a
-          className="App-link"
-          href="https://reactjs.org"
-          target="_blank"
-          rel="noopener noreferrer"
+      </div>
+
+      <div className="controls">
+        <label>
+          Backend:
+          <select value={backend} onChange={e=>setBackend(e.target.value)}>
+            <option value="webgpu-hybrid">WebGPU (Hybrid)</option>
+            <option value="webgpu-strict">WebGPU (Strict)</option>
+            <option value="wasm">WASM (CPU)</option>
+          </select>
+        </label>
+        {' '}
+        <label>
+          Quant:
+          <select value={quant} onChange={e=>setQuant(e.target.value)}>
+            <option value="int8">int8 (faster)</option>
+            <option value="fp32">fp32 (higher quality)</option>
+          </select>
+        </label>
+        {' '}
+        {backend.startsWith('webgpu') && (
+          <label style={{ fontSize:'0.9em' }}>
+            <input type="checkbox" checked={decoderInt8} onChange={e=>setDecoderInt8(e.target.checked)} />
+            Decoder INT8 on CPU
+          </label>
+        )}
+        {' '}
+        <label>
+          Preprocessor:
+          <select value={preprocessor} onChange={e=>setPreprocessor(e.target.value)}>
+            <option value="nemo80">nemo80 (smaller)</option>
+            <option value="nemo128">nemo128 (default)</option>
+          </select>
+        </label>
+        {' '}
+        <label>
+          Stride:
+          <select value={frameStride} onChange={e=>setFrameStride(Number(e.target.value))}>
+            <option value={1}>1</option>
+            <option value={2}>2</option>
+            <option value={4}>4</option>
+          </select>
+        </label>
+        {' '}
+        <label>
+          <input type="checkbox" checked={verboseLog} onChange={e => setVerboseLog(e.target.checked)} />
+          Verbose Log
+        </label>
+        {' '}
+        <label style={{fontSize:'0.9em'}}>
+          <input type="checkbox" checked={dumpDetail} onChange={e=>setDumpDetail(e.target.checked)} />
+          Dump result to console
+        </label>
+        {(backend === 'wasm' || decoderInt8) && (
+          <label style={{fontSize:'0.9em'}}>
+            Threads:
+            <input type="number" min="1" max={maxCores} value={cpuThreads} onChange={e=>setCpuThreads(Number(e.target.value))} style={{width:'4rem'}} />
+          </label>
+        )}
+        <button 
+          onClick={loadModel} 
+          disabled={!status.toLowerCase().includes('fail') && status !== 'Idle'}
+          className="primary"
         >
-          Learn React
-        </a>
-      </header>
+          {status === 'Model ready ✔' ? 'Model Loaded' : 'Load Model'}
+        </button>
+      </div>
+
+      {typeof SharedArrayBuffer === 'undefined' && backend === 'wasm' && (
+        <div style={{ 
+          marginBottom: '1rem', 
+          padding: '0.5rem', 
+          backgroundColor: '#fff3cd', 
+          border: '1px solid #ffeaa7',
+          borderRadius: '4px',
+          fontSize: '0.9em'
+        }}>
+          ⚠️ <strong>Performance Note:</strong> SharedArrayBuffer is not available. 
+          WASM will run single-threaded. For better performance, use WebGPU.
+        </div>
+      )}
+
+      <div className="controls">
+        <input 
+          ref={fileInputRef}
+          type="file" 
+          accept="audio/*" 
+          onChange={transcribeFile} 
+          disabled={status !== 'Model ready ✔' || isTranscribing} 
+        />
+        {transcriptions.length > 0 && (
+          <button 
+            onClick={clearTranscriptions} 
+            style={{ marginLeft: '1rem', padding: '0.25rem 0.5rem' }}
+          >
+            Clear History
+          </button>
+        )}
+      </div>
+
+      <p>Status: {status}</p>
+      {progressPct!==null && (
+        <div className="progress-wrapper">
+          <div className="progress-bar"><div style={{ width: `${progressPct}%` }} /></div>
+          <p className="progress-text">{progressText}</p>
+        </div>
+      )}
+
+      {/* Latest transcription */}
+      <div className="controls">
+        <h3>Latest Transcription:</h3>
+        <textarea 
+          value={text} 
+          readOnly 
+          className="textarea"
+          placeholder="Transcribed text will appear here..."
+        />
+      </div>
+
+      {/* Latest transcription performace info */}
+      {latestMetrics && (
+        <div className="performance">
+          <strong>RTF:</strong> {latestMetrics.rtf?.toFixed(2)}x &nbsp;|&nbsp; Total: {latestMetrics.total_ms} ms<br/>
+          Preprocess {latestMetrics.preprocess_ms} ms · Encode {latestMetrics.encode_ms} ms · Decode {latestMetrics.decode_ms} ms · Tokenize {latestMetrics.tokenize_ms} ms
+        </div>
+      )}
+
+      {/* Transcription history */}
+      {transcriptions.length > 0 && (
+        <div className="history">
+          <h3>Transcription History ({transcriptions.length} files):</h3>
+          <div style={{ maxHeight: '400px', overflowY: 'auto', border: '1px solid #ddd', borderRadius: '4px' }}>
+            {transcriptions.map((trans) => (
+              <div className="history-item" key={trans.id}>
+                <div className="history-meta"><strong>{trans.filename}</strong><span>{trans.timestamp}</span></div>
+                <div className="history-stats">Duration: {trans.duration.toFixed(1)}s | Words: {trans.wordCount}{trans.confidence && ` | Confidence: ${trans.confidence.toFixed(2)}`}{trans.metrics && ` | RTF: ${trans.metrics.rtf?.toFixed(2)}x`}</div>
+                <div className="history-text">{trans.text}</div>
+              </div>
+            ))}
+          </div>
+        </div>
+      )}
+
+      <div style={{ marginTop: '2rem', padding: '1rem', backgroundColor: '#f8f9fa', borderRadius: '4px', fontSize: '0.9em' }}>
+        <h4>🔗 Links:</h4>
+        <p>
+          <a href="https://github.com/ysdede/parakeet.js" target="_blank" rel="noopener noreferrer">
+            GitHub Repository
+          </a>
+          {' | '}
+          <a href="https://www.npmjs.com/package/parakeet.js" target="_blank" rel="noopener noreferrer">
+            npm Package
+          </a>
+          {' | '}
+          <a href="https://huggingface.co/nvidia/parakeet-tdt-1.1b" target="_blank" rel="noopener noreferrer">
+            NVIDIA Parakeet Model
+          </a>
+        </p>
+      </div>
     </div>
   );
 }
-
-export default App;