What it is
RepWise is a fitness web app built for a hackathon by a team of four. It analyzes exercise form from a webcam or an uploaded video, generates a workout plan from an onboarding profile, and logs nutrition by scanning barcodes or photographing nutrition labels. Users can also compete on rep scores in live video rooms.
The problem
HackByte 3.0 asked teams to build free alternatives to premium services behind paywalls and red tape. The team chose fitness. Form coaching, personalized programming, and nutrition logging usually sit behind subscriptions. RepWise combines all three in one app.
How it works
RepWise is a SvelteKit app. Pages and API routes live in one codebase, and Prisma talks to PostgreSQL. The table lists the main modules.
| Module | What it does | Where |
|---|---|---|
| Form correction | Runs MediaPipe pose detection in the browser, scores each rep, and speaks feedback | src/lib/detection-component.svelte, src/lib/exercise-feedback.ts |
| Voice feedback | Converts a feedback sentence to speech with ElevenLabs | src/routes/api/voice |
| Workout plans | Asks Gemini for a structured plan that matches the onboarding profile | src/routes/api/workout-plan, src/lib/toolspecs/fitness-planner.ts |
| Macro tracking | Reads barcodes in the browser, looks products up in Open Food Facts, and stores macros | src/routes/(app)/macro-tracking, src/routes/api/macros |
| Label OCR (optical character recognition) | Sends a photo of a nutrition label to Mindee and stores the parsed result | src/routes/api/ocr |
| Competitions | Streams video through Agora and stores scores per user and room | src/routes/(app)/compete, src/routes/api/competition |
| Heart rate | Accepts heart-rate samples from an authenticated client | src/routes/api/heartrate |
| Auth | Email and password plus Google sign-in through Better Auth | src/lib/auth.ts |
The data flow for form correction stays on the device until feedback needs speech:
- The page loads the MediaPipe
pose_landmarker_heavymodel fromstatic/and runs it on the video with the GPU delegate. - Each frame appends a landmark result to a sequence.
- Every 10 frames, the exercise's feedback function reads the sequence and returns messages, a score, and a rep count.
- The component speaks the latest message through
/api/voice, at most once every 5 seconds.
What I built
I built the pose-based squat and push-up feedback with rep counting, the spoken corrections, and the competition room, scoring and status APIs and interface. Jay built authentication and onboarding, the dashboard, the workout planner and chat interface, barcode enhancements and the move to Agora. Sundaram built the Prisma schema, macro tracking, nutrition-label OCR, the routine and heart-rate APIs, and the theme and sidebar.
Design decisions
Rule-based form scoring. The code computes joint angles from landmarks and compares them with fixed thresholds. A squat scores 25 points each for depth, knee alignment, hip-to-knee balance, and tempo. A push-up uses depth, body alignment, arm position, and tempo. This approach needs no training data and produces a specific message for each failed check. The cost is coverage: only two exercises exist, and the thresholds apply to every body type.
Pose inference in the browser. Landmark detection runs client-side, so the server never receives video for form analysis. The trade-off is that performance depends on the user's device and GPU.
Structured output from Gemini. The workout route declares a create_workout_plan function with a typed schema (days, exercises, sets, reps, load, rest). It reads the arguments of the model's function call instead of parsing free text. The plan then has the same fields (sets, reps, load, rest) that the Routines table stores.
Managed video for competitions. The commit history shows that on 2025-04-05 the team added its own room and signaling API, then removed it and integrated the Agora RTC SDK the same day. Agora carries the video. The app stores only room status and per-user scores in PostgreSQL.
Run it locally
You need Node.js 20, a PostgreSQL database, and API keys for the services above.
-
Clone the repository and install dependencies. The install step also runs
prisma generate.git clone https://github.com/Crew-object-Object/RepWise.git cd RepWise npm install -
Create a
.envfile with these variables:DATABASE_URL,PUBLIC_BASE_URL(for examplehttp://localhost:5173),GOOGLE_CLIENT_ID,GOOGLE_CLIENT_SECRET,GEMINI_API_KEY,ELEVENLABS_API_KEY,OCR_API_KEY, andPUBLIC_AGORA_APP_ID. -
Create the tables.
npx prisma db push -
Start the dev server.
npm run dev
Reference
| Area | Choice |
|---|---|
| Framework | SvelteKit 2, Svelte 5, TypeScript |
| UI | Tailwind CSS, shadcn-svelte components, bits-ui |
| Database | PostgreSQL through Prisma |
| Auth | Better Auth |
| Pose detection | @mediapipe/tasks-vision |
| Text to speech | ElevenLabs |
| Workout plans | Google Gemini (gemini-2.0-flash) |
| Barcodes and nutrition data | barqode, Open Food Facts |
| Label OCR | Mindee |
| Live video | Agora RTC |
| Deployment | @sveltejs/adapter-vercel |
| Tests | Playwright end-to-end test, run in GitHub Actions against a Postgres service |
The Prisma schema defines User, Onboarding, Routines, Macros, HeartRate, CompetitionRoom, and CompetitionScore, plus the Better Auth tables.
Outcome
The team built RepWise at HackByte 3.0 at IIITDM Jabalpur on 2025-04-04. The hackathon page records a track placement and the GitHub award for issue and pull request tracking and documentation. The repository has 163 commits across three contributors. The first commit is dated 2025-04-04 and the last 2025-04-06. A deployment is listed at rep-wise.vercel.app.