TongFlow Moved to the Cloud — And Everything Else Since v0.1.8

Our last roundup ended at v0.1.8. Since then the biggest change isn’t a model — it’s how you run TongFlow. The desktop app became a thin shell around a hosted studio, and the catalog kept growing underneath it: Meta’s SAM and Sapiens2 suites, a full music workbench, and a per-node model picker that turns one API key into dozens of models.

New to TongFlow? It’s an open-source multi-modal GenAI workflow studio: every AI model is a node on an infinite canvas. Start with the introduction for the full story.

TongFlow now runs in the cloud

v0.2.0 replaced the desktop app with a cloud studio. Sign in with Google or WeChat at app.tongflow.com and the canvas is there — plugins and execution are managed for you, so there’s no Python runtime to provision and no weights to download on your own machine.

The installers are still on the Releases page, but they’re different now: a ~10 MB Pake (Tauri) shell that loads the cloud studio, shipping as TongFlow-mac-universal.dmg (Apple Silicon + Intel in one build) and TongFlow-win-x64.msi. The old Electron app — which bundled a Next.js server, a SQLite database, and a Python plugin runtime into ~200 MB — is no longer shipped. If you already have it installed it keeps working; we push no update to it.

The fully local, account-free TongFlow didn’t go away — it’s self-hosting. The core is still AGPL-3.0 on GitHub, still runs with pnpm start:prod or Docker, and still costs nothing.

On the cloud side, the price is $2.99 a month. Every new account gets 7 days of the full studio with no card; after that you can start a 7-day trial with a card, and it’s $2.99/month afterwards, cancel anytime. We don’t sell compute or credits — you bring your own provider keys and pay providers directly, at their prices. Modal’s free tier covers real GPU time each month, so compute often starts at $0.

Meta’s SAM suite

Four new official Modal plugins, all built on Meta’s Segment Anything family. They need a Hugging Face token for the gated checkpoints:

• SAM 3 (tongflow-modal-sam3) — text-guided matting. Describe a concept and it cuts out every instance of it in an image, or tracks it through a video.
• SAM Audio (tongflow-modal-sam-audio) — text-prompted sound separation: noise reduction, vocal isolation, free-text stem extraction. First official plugins for the noise-reduction and track-separation nodes.
• SAM 3D Objects (tongflow-modal-sam-3d-objects) — single image → 3D Gaussian splat.
• SAM 3D Body (tongflow-modal-sam-3d-body) — single image → full-body human mesh.

Meta Sapiens2, plus video motion capture

v0.2.1 added five nodes backed by the Sapiens2-1B checkpoints, all served from one Modal L40S:

• Pose detection — a 308-keypoint whole-body skeleton (body, hands, face) on a clean black background.
• Body-part segmentation — 29-class human parsing as a pure class-color map.
• Surface normals — a per-pixel normal map with the background masked out.
• Human matting — a straight-alpha transparent PNG.
• Image → 3D gained a Sapiens2 pointmap implementation: a colored human point cloud as GLB.

The headline is video motion capture: feed a monocular video, get back an animated 3D human GLB on Meta’s MHR rig (Momentum Human Rig, Apache-2.0). It plays back right in the model node and imports into Blender as a skinned armature. Two plugins implement the same node from opposite directions — sam-3d-body regresses MHR per frame with a learned prior (body, hands, and experimental face channels), while sapiens2 runs a geometric pipeline (308-keypoint pose plus a pointmap 3D lift, One-Euro smoothing, rest-pose hold for body parts that leave the frame). Same node, two engines, pick per job.

Animated models needed a viewer to match, so the model node now auto-plays animation clips (capped at 30 fps, paused off-screen and in background tabs) with a play/pause control, and the initial camera homes onto the model’s front and fits it to the frame.

Music became a workbench

The reworked tongflow-modal-ace-step plugin (ACE-Step 1.5) turned music from a single generate button into six nodes:

• Music repaint — regenerate a chosen time range.
• Music cover — restyle a song via caption and/or a reference track.
• Stem extraction — isolate one of 12 stems (vocals, drums, bass, guitar, …).
• Add track — generate a new stem over an existing mix.
• Complete arrangement — fill in the missing tracks.
• Music brief — one sentence of intent → lyrics, style tags, BPM, key, and duration.

The default model moved to xl-sft (the official best-quality variant), and each node has a model dropdown if you’d rather run xl-base or xl-turbo. The music node also gained an optional reference audio input back in v0.1.13 — connect an audio node and ACE-Step uses it for style conditioning, LeVo as a melody prompt.

Two adjacent capabilities landed alongside it: audio understanding (audio-describe) — select a clip, hit Describe, get genre, mood, instruments, vocals, and events in plain language, implemented by Gemini, OpenAI, Agnes, and Gemma 4 — and open-vocabulary sound separation, a new node where you describe a sound in free text and get both the target and the residual back from one inference, backed by SAM Audio.

One key, many models

• APIMart (tongflow-api-apimart) — one key routes 46 models across 7 nodes: image generation and editing (Z-Image-Turbo, Seedream, Nano Banana Pro, GPT-Image, Imagen 4.0, Qwen Image 2.0, Grok Imagine), text and image → video (Kling v3, VEO3.1, Sora 2, Seedance 2.0), text generation, Whisper transcription, and TTS.
• Agnes AI (tongflow-api-agnes) — one key covers 12 nodes: text generation, splitting and combining, image understanding, image generation and editing, multi-image fusion, and async text / image / first-last-frame → video.
• SenseNova-Vision (tongflow-modal-sensenova-vision) — a unified vision model across 5 nodes, and the first implementation that takes surface normals and matting beyond humans: full-scene normals and salient-object matting.
• LeVo and Boogu-Image joined the official catalog too.

Making that work took a contract change: a per-node model picker. Router-style plugins declare their model lists, the node shows a Model dropdown next to the plugin selector, and the choice travels top-level — through task creation, the database, the plugin envelope, and workflow export. Plugins that declare no models are unchanged.

Smaller things you’ll notice

• The 50 MB upload limit is gone — large media goes straight onto the canvas.
• Upload failures are no longer silent — per-file errors surface instead of the file quietly vanishing.
• Plugin cards in the picker show each plugin’s real name, description, and icon instead of a bare id.
• A Korean UI, joining English, Chinese, and Japanese.
• Node prompt boxes scroll at a fixed height instead of growing the whole node forever.
• Cancel a running node from its loading overlay, and uninstall plugins from the manager.
• Plugin installs work behind corporate proxies and private CAs — the OS trust store is honored, so no more unable to verify the first certificate.
• The Gemini plugin works out of the box again after Google retired gemini-2.0-flash upstream.
• Images → Video accepts a single image, and loading a workflow re-fits the view.

Where that leaves us

37 official plugins against 61 capability nodes in the contract — text, image, video, audio, music, 3D, documents, and the web. Every one of them is a node you can drop on the canvas and wire to the next.

Sign in at app.tongflow.com to try it without installing anything, or self-host from GitHub if you’d rather keep everything on your own machine. A star helps others find the project, and Discord is where bug reports, ideas, and finished work all land.

More models, more modalities, more ways to combine them — keep building.