My scenario involves a website hosting 100 online courses, each with various types of content such as transcripts, mp4s, titles, course descriptions, and modules (including descriptions, audio, and video).
I’m wondering if it’s more advisable to create a separate index for each content type or if it’s better to store all data in a single index to allow for cross-type queries. Any insights on how to approach this decision would be greatly appreciated.
The end goal is to create a recommendation system, search, chatbot, etc.
Hi @richard1, thanks for the post. The decision to partition the data for each online course depends on whether you can or want to allow queries to search across multiple courses. If it would be unacceptable for a query about one course to yield information from a different course, then you should partition your data into namespaces.
You should not create indexes for each course, as this will require significantly more resource management, infrastructure design, and costs.
With namespaces, queries and other operations are limited to one namespace, so different requests can search different subsets of your index. Additionally, you can easily create or delete namespaces with simple data plane operations.