I work on OpenSubs, a free, open source (AGPL-3.0) subtitle tool that runs entirely in the browser tab.

You drop in a video file, and Whisper transcribes it on your own machine, using transformers.js with WebGPU where available and WebAssembly otherwise. The model (40–250 MB) downloads once and is cached. There is no upload endpoint in the product, so the video has nowhere to go.

After that you can:

  • fix lines by typing over them (click a timestamp to jump to that moment)
  • translate into 20 languages with Chrome’s built-in on-device translator
  • pick one of 12 caption styles, including word-by-word highlighting
  • export SRT / VTT / ASS, or burn the subtitles into an MP4 (libass compiled to WebAssembly, encoded with WebCodecs)

A few things I learned building it:

  • Whisper hallucinates on silence and music (“Thanks for watching!”, or the Japanese equivalent). A Silero VAD pass runs before Whisper, and a cleanup step drops the known stock phrases.
  • Singing doesn’t count as speech for the VAD, so a music video gets a “no speech found” warning. You can still force it.

Honest limits: it only takes video files, not audio-only files. Cue timings can’t be edited yet. Builds are release candidates. Everything that runs locally is free with no account; the only paid part is optional cloud translation on our backend (US$5 for 1000 credits), and you can bring your own Claude / OpenAI / DeepL key instead.

Site: https://opensubs.app/ Code: https://github.com/open-subs/opensubs

Feedback welcome, especially on languages where the transcription goes wrong.

    • idriss@lemmy.ml
      link
      fedilink
      arrow-up
      5
      ·
      11 hours ago

      yep, I would rather generate one with Claude to my liking and make it more efficient (target the CLI for example)

      • antianarchist@sopuli.xyz
        link
        fedilink
        arrow-up
        4
        ·
        8 hours ago

        Which is absolutely legit. I only criticize the people releasing a “production ready” product, which is nothing but a PoC and then also claim that it is AGPL-3, while abandoning it after 2 months. This hurts the open source community imho.

        • Knock_Knock_Lemmy_In@lemmy.world
          link
          fedilink
          arrow-up
          1
          ·
          4 hours ago

          This hurts the open source community imho.

          Nah. At worst the open source community is indifferent. If someone can pick up where this left off then it benefits the community.

  • zurohki@aussie.zone
    link
    fedilink
    English
    arrow-up
    24
    ·
    1 day ago

    Whisper hallucinates on silence and music (“Thanks for watching!”, or the Japanese equivalent).

    I’ve had Whisper ask me to donate to anime fansubbing groups. I wonder what it was trained on?

  • Daniel Quinn@lemmy.ca
    link
    fedilink
    English
    arrow-up
    7
    ·
    1 day ago

    It sounds like a pretty cool project, thanks for sharing! As a browser-based project designed to run offline, you might want to consider shipping it as an electron app.

  • Kalothar@lemmy.ca
    link
    fedilink
    arrow-up
    1
    ·
    1 day ago

    What’s the max file size?

    I’ve been looking for the fan edit of marvels Infinity saga.

    It’s about 50 hours long divided into many many files ranging from 3-10 gb each. Would it be able to handle this? Or what’s the best way to split them up

    • ghost_laptop@lemmy.ml
      link
      fedilink
      arrow-up
      3
      ·
      1 day ago

      i get ya, but also a lot of people don’t have a dedicated self hosted setup. recently my hdd died and all my films were there, so now i’m downloading and deleting, which just feels like a drag to start bazarr for one film. this is handy for some people.

  • magnue@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    1 day ago

    I’m a lazy reader so don’t know if this is implemented but it would be cool if it could be done live with the current audio out of the system. I know this was done for streamers etc using OBS/localvocal but it’s a pain to get working in my experience - having to fiddle around with OBS filter settings.

          • Sammirr@aussie.zone
            link
            fedilink
            arrow-up
            2
            arrow-down
            1
            ·
            13 hours ago

            Yes and no. As an author of open source projects, they were made available for people to fork, improve and conrribute under permissive licenses. Then come LLMs that uses code like mine to generate code or slop. My choice about license was made before LLMs. Now they use derivatives of my work without attribution as required by the license.

            Even if I change the license, it won’t be respected, and nothing else will change. To me, that’s copyright infringement on me, ignoring the hours I put in.

            On a separate note, LLMs only produce more of the same, ignoring the environmental and economic impacts. Often, more of the same is not what we need. It leads to the same mistakes being made. I’ve never used LLM to code, never will.

            • ghost_laptop@lemmy.ml
              link
              fedilink
              arrow-up
              1
              ·
              4 hours ago

              as a pirate, i shit on copyright, culture should be free. the issue with llm is not that they used copyrighted shit, if the llm would be open source and not owned by a corpo, i’d be contributing to knowledge. here, you’re just contributing to some gringo westoid billionaire’s pockets. everything is a remix, i dont have an issue with that. so again, the issue is labour theft in marxists terms.