Our docloader strips out unimportant content such as additional markup and image tags so that chunking can be as efficient as possible. This helps avoid unnecessary token usage when developing with AI, and also maximizes efficiency of the searches.
Convert from HTML, Markdown, reStructuredText, and SGML/DocBook.