Is there a way to determine operation-level Write Unit (WU) usage?
For read operations, it’s possible to get the Read Unit (RU) usage. However, for write operations, there doesn’t seem to be a direct way to obtain the WU usage at the operation level.
Since WUs affect our costs, I believe it should be possible to calculate the WU usage based on factors like the number of insertions and the size of the metadata, among other variables.
The reason I’m asking for operation-level WU usage, rather than index-level, is that I have clients sharing the same index, and I need to charge them based on their individual usage.
Hi @tomo, thanks for your post. Due to how write units (WUs) are calculated and our decision to prioritize upsert speed, you cannot currently retrieve the WUs in the operation response.
When you perform a write operation, the data size targeted for changes or deletions is unknown. We calculate his asynchronously to enable faster write operations.
That said, I have brought your feedback to our product team, and we are discussing what we can offer to help you measure WUs independently.
Alternatively — given that it sounds like you want to apply usage-based pricing on top of our usage-based model — your feedback essentially raises a request for namespace-based usage metrics. While this would be computationally expensive, and we cannot make any assurances, it’s an interesting thought we’ll consider.
Thank you for your response! The namespace-based approach is exactly what I had in mind too, and I’m excited about the potential updates ahead. Additionally, if there’s a formula to calculate expected WU usage based on variables like dimentions, metadata size, etc., that can also serve as an alternative solution.
We have the same problem.
Recently we saw our WU usage double after a code update and I’m struggling to understand what happened as it doesn’t seem like we’re upserting any more vectors than before the update.
The metrics tab only gives me the rate of WU’s which isn’t precise enough and the usage tab only updates after more than 24h.
Based on the above, I understand the calculation of WU’s happens only afterwards and it would be counterproductive to return it as part of the response. However, would it be possible to log the exact number of consumed WU’s e.g. in a table with timestamps. Not necessarily every separate request but aggregated to a say a minute or so. That would already help a lot.
Thanks, @tomo and @antti, for the feedback. We are exploring options to enhance WU visibility and/or simplify the calculations so that users can make more accurate projections.
Currently, you can use the following options for WU visibility:
View the WUs over time for up to the previous week in the Metrics tab for each index.
View the WUs per serverless region per day in the Usage page of the organization settings.
Use the Monitoring tools we offer to Standard and Enterprise tier customers to generate time-series performance metrics, including WUs, for each index.