Load pending state attributes and event data ids at startup (#88444)

* Load pending state attributes and event data ids at startup

Since we queue all events to be processed after startup
we can have a thundering herd of queries to prime the
LRUs of event data and state attributes ids. Since we
know we are about to process a chunk of events we can
fetch all the ids in two queries

* lru

* fix hang

* Fix recorder LRU being destroyed if event session is reopened

We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened

* Fix recorder LRU being destroyed if event session is reopened

We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened

* cleanup
This commit is contained in:
J. Nick Koston
2023-02-19 20:26:38 -06:00
committed by GitHub
parent c4f92f5ad4
commit c2b770bcb9
9 changed files with 190 additions and 74 deletions

View File

@@ -1,10 +1,12 @@
"""SQLAlchemy util functions."""
from __future__ import annotations
from collections.abc import Callable, Generator, Sequence
from collections.abc import Callable, Generator, Iterable, Sequence
from contextlib import contextmanager
from datetime import date, datetime, timedelta
import functools
from functools import partial
from itertools import islice
import logging
import os
import time
@@ -761,3 +763,19 @@ def resolve_period(
end_time += offset
return (start_time, end_time)
def take(take_num: int, iterable: Iterable) -> list[Any]:
"""Return first n items of the iterable as a list.
From itertools recipes
"""
return list(islice(iterable, take_num))
def chunked(iterable: Iterable, chunked_num: int) -> Iterable[Any]:
"""Break *iterable* into lists of length *n*.
From more-itertools
"""
return iter(partial(take, chunked_num, iter(iterable)), [])