Section 2 of 8 Storage & performance
Don't load all data
The embedded UI and TieredStorage APIs are built for reverse-chronological paging. Loading “everything” defeats the design and can exhaust browser memory or API timeouts.
- Use time windows (last 15 minutes, last hour) before widening the range.
- Combine KQL filters — level, category, traceId, userId, url — to narrow results.
- Keep MaxEntriesPerRequest modest (100–500). The UI defaults to 150 rows per refresh.
- Prefer offset/limit paging when exporting or building custom tools — do not read multi-gigabyte JSONL files into a single List<T>.
- Disable Live mode when diagnosing large backlogs; poll manually after applying filters.
Cap UI page size
MaxEntriesPerRequest guards the middleware before ReverseLogFileReader or SQLite query execution.
app.UseArgusUI(options =>
{
options.RoutePrefix = "/argus-logs";
options.MaxEntriesPerRequest = 150; // default UI page size — keep modest
});Anti-patterns vs recommended approach
| Pattern | Recommendation |
|---|---|
| Open /argus-logs with Live mode on a 2 GB JSONL file | Enable TieredStorage; filter by time + level; page with limit/offset |
| Download entire log file daily for search | Query SQLite hot tier or Parquet archives with retention; export only filtered slices |
| SELECT * FROM logs with no WHERE clause | Start with traceId: or userId: from an incident ticket, then widen |
| Load full request/response bodies for every HTTP call | Log metadata + status; enable bodies only in staging or for sampled routes |