fp.

YTNT is a simple web-app for manually typing notes while watching a YouTube video. Simultaneously type and control playback with keyboard shortcuts. Export to Word (.docx) when you're done.

I'm seeking general feedback.

NOTES:

- The only way to "save" (for now) is to export to a Word .docx

- Speech-to-text only works in Chrome

GitHub: https://github.com/kavinaidoo/ytnt

Blog Post: http://archive.today/FRqwp

Thanks, Kavi

Comments

Leftium•1y ago

I forked oTranscribe and added some features to solve a similar problem.

- demo: https://otranscribe.netlify.app/?vsl=definedefine

- source code: https://github.com/Leftium/oTranscribe

- CLI tool to generate OTR (oTranscribe) files from (YouTube) SBV/TTML files: https://github.com/Leftium/otrgen

In my case, I wanted to start with the (auto-generated) YouTube transcript and get clickable timestamps. This makes it much faster to search through the content of a video: I can read/search much faster than watching a video, even on 2X speed.

I could also add my own notes to the transcript.

If you add support for loading transcripts like this, it could work cross-browser without the microphone/speech-to-text.

kavinaidoo•1y ago

Aha, starting with the YouTube transcript is a great idea. I wanted to start with it but I couldn't find a way to get it from the YouTube iFrame API so I went with the Web Speech API to listen and convert.

Forgive me for the confusion, the demo link implies that the transcript is loaded from the video but I see in the code that there's a pre-existing "/txt/definedefine.md" that is loaded. How are these SBV/TTML files downloaded from YouTube in the first place? I assume that it is a separate process? I see you are then using otrgen to presumably convert these so they can be used by oTranscribe.

If I could load the transcripts dynamically when loading the YouTube video that would be a great feature.

Leftium•1y ago

The demo simply demonstrates how I used my tool. It requires some manual set up:

- TTML files are downloaded via CLI: `yt-dlp.exe --skip-download --write-auto-sub --sub-format ttml`

- TTML files are converted to the OTR .MD format via my CLI tool

- The MD file can be dragged & dropped onto the web app.

---

I think it is possible to download SBV/TTML files, but the download must be done from the server due to CORS restrictions.

My app didn't go this far due to limitations of the (serverless) platform it is hosted on. Also it was faster to just do the steps manually vs. developing a server that does it.

I have seen many services that download the transcript. Here are a few:

- https://youtubetranscript.com

- https://kagi.com/summarizer

- https://www.tubepen.com

However, note YouTube may block your server if you download too many transcripts: https://kagifeedback.org/d/4451-universal-summarizer-cant-fi...

kavinaidoo

Leftium•1y ago

I'm planning a beat-aware YouTube player. Unfortunately, it is not possible to access the audio stream data across the YouTube embed. (For beat-detection analysis.)

I considered using the microphone like this. It's nice to see it works. Although there seems to be a time limit to how long the microphone can record?

My plan was to download video (youtube-dlp), then make a CLI tool that analyzes and uploads the beat-detection data. An advantage of the CLI tool is it can complete the analysis faster than playing the video at 1X.

kavinaidoo•1y ago

Yeah, I also wanted to "pipe" the audio directly to the Web Speech API but had to resort to using the mic. Another feature I wanted to add was to have a keyboard shortcut insert a screenshot of the video into the notes. Handy for diagrams etc but I hit some roadblocks there too and the workarounds were getting crazy.

Regarding the time-limit, I'm entirely accessing the audio stream using the Web Speech API and it "decides" (usually when there's a sufficient pause) when to finalize a recognition result. It was also firing a "recognition.onend" event after a certain amount of time (some old details here: https://stackoverflow.com/questions/38213580/chromes-webkits...) so I have a workaround where if the user did not turn off speech-to-text, it is immediately restarted. You'll see console warnings "userStopped == false - trying to restart" when this happens.

I assume you'll have no such issues because you'll be handling the "raw" audio data and not working through this API. Also, I've noticed that when using the Web Speech API on Safari, it does not "hear" what the tab is playing. It only hears external audio. I'm not sure whether this will be an issue for you but for me it means my app is Chrome only (suboptimal). Forgive me for my naïveté in this arena, It's the first time I'm using any browser audio API.

Would your tool run on a server and work with a frontend for YT link ingestion or will you just use it yourself from the cli?

Leftium•1y ago