Commit Graph

129 Commits

Author SHA1 Message Date
2c13c70e12 Update ruff to 0.12.0 (#147106) 2025-06-19 20:39:09 +02:00
f1975d9dbf Elevate Recorder "Error executing ..." from warning to error (#142816) 2025-04-24 11:36:39 +01:00
c9abe76023 Use multiple indexed group-by queries to get start time states for MySQL (#138786)
* tweaks

* mysql

* mysql

* Update homeassistant/components/recorder/history/modern.py

* Update homeassistant/components/recorder/history/modern.py

* Update homeassistant/components/recorder/const.py

* Update homeassistant/components/recorder/statistics.py

* Apply suggestions from code review

* mysql

* mysql

* cover

* make sure db is fully init on old schema

* fixes

* fixes

* coverage

* coverage

* coverage

* s/slow_dependant_subquery/slow_dependent_subquery/g

* reword

* comment that callers are responsible for staying under the limit

* comment that callers are responsible for staying under the limit

* switch to kwargs

* reduce branching complexity

* split stats query

* preen

* split tests

* split tests
2025-03-02 15:13:06 +01:00
d5568ff955 Standardize homeassistant imports in full-CI components (#136731)
Standardize homeassistant imports in components
2025-01-28 13:11:19 -05:00
aa741a9207 Combine short strings in components (#135265) 2025-01-10 11:07:51 +01:00
3c6113e37c Remove per engine max bind vars (#135153) 2025-01-09 22:50:13 +00:00
64752af4c2 Change minimum SQLite version to 3.40.1 (#135042)
Co-authored-by: J. Nick Koston <nick@koston.org>
2025-01-08 16:34:36 -10:00
43fab48d4e Improve purge performance for PostgreSQL with large databases (#133699) 2024-12-21 10:53:15 +01:00
c8480627ca Add comment motivating magic number for MySQL error codes (#133516)
* Add comment motivating magic number for MySQL error codes

* Pick nits
2024-12-19 09:56:32 +01:00
d26d483a2f Improve recorder util resolve_period (#132264) 2024-12-06 21:06:56 +01:00
a417d3dcf8 Fix recorder "year" period in leap year (#132167)
* FIX: make "year" period work in leap year

* Add test

* Set second and microsecond to non-zero in test start times

* FIX: better fix for leap year problem

* Revert "FIX: better fix for leap year problem"

This reverts commit 06aba46ec6.

---------

Co-authored-by: Erik <erik@montnemery.com>
2024-12-04 13:21:10 +01:00
d8b618f7c3 Remove support for live recorder data migration of context ids (#125309) 2024-10-26 07:19:03 +02:00
86ae70780c Refactor recorder retryable_database_job decorator (#125306) 2024-09-05 13:09:27 +02:00
984eba809c Simplify generic decorators in recorder (#125301)
* Simplify generic decorators in recorder

* Remove additional case
2024-09-05 10:16:44 +02:00
a8f2204f4f Teach recorder data migrator base class to update MigrationChanges (#125214)
* Teach recorder data migrator base class to update MigrationChanges

* Bump migration version

* Improve test coverage

* Update migration.py

* Revert migrator version bump

* Remove unneeded change
2024-09-05 08:56:18 +02:00
a35fa0e95a Warn that the minimum SQLite version will change to 3.40.1 as of 2025.2 (#104298)
Co-authored-by: Robert Resch <robert@resch.dev>
2024-07-31 08:13:04 -05:00
5dbd7684ce Fail tests if recorder creates nested sessions (#122579)
* Fail tests if recorder creates nested sessions

* Adjust import order

* Move get_instance
2024-07-25 21:18:55 +02:00
ca55986057 Import Generator from collections.abc (1) (#120914) 2024-07-01 11:51:51 +02:00
fe21e2b8ba Import Generator from typing_extensions (1) (#118986) 2024-06-06 17:02:13 +02:00
588380392d Small speed up to read-only database sessions (#118674) 2024-06-03 19:50:05 +02:00
099ad77078 Migrate recorder instance to use HassKey (#118673) 2024-06-03 11:19:00 -05:00
c616fc036e Move recorder chunk utils to shared collection utils (#118065) 2024-05-24 12:49:39 -10:00
907b9c42e5 Use PEP 695 for decorator typing with type aliases (2) (#117663) 2024-05-18 11:41:46 +02:00
Sid
2cc916db6d Replace pylint broad-except with Ruff BLE001 (#116250) 2024-05-07 14:00:27 +02:00
Sid
6587ee20db Enable Ruff TRY300 (#114437)
* Enable Ruff TRY300

* Update validation.py

* Address review comments
2024-03-30 10:37:59 +01:00
Sid
a5b609f081 Enable ruff TRY401 (#114395)
* Enable ruff TRY401

* fix tests
2024-03-29 07:20:36 +01:00
Sid
727581eea3 Add first batch of Ruff PT rules (#113665)
* Add first batch of Ruff PT rules

* fix weather test

* Fix pilight test

* Update test_intent.py

* Update pilight test_init.py

* Update test_init.py
2024-03-18 13:26:14 +01:00
Sid
d5fd005db8 Add ruff rule PIE808 (#113621) 2024-03-17 09:56:26 +01:00
af6f2a516e Guard against db corruption when renaming entities (#112718) 2024-03-08 11:34:07 -10:00
a033574ee2 Add empty line after module docstring [r] (#112703) 2024-03-08 15:05:07 +01:00
a22244707b Create an issue when database backups fail because the system runs out of resources (#109020) 2024-01-30 10:23:58 -10:00
24b1e01d71 Update Ruff to 0.1.8, avoid linter/formatter conflicts (#106080)
* Disable Ruff rules that may conflict with the formatter

* Upgrade Ruff to 0.1.8

- https://github.com/astral-sh/ruff/releases/tag/v0.1.7
- https://github.com/astral-sh/ruff/releases/tag/v0.1.8

* Format with Ruff 0.1.8
2023-12-20 23:55:09 +01:00
9ed745638d Chunk purging attributes and data ids for old SQLite versions (#104296) 2023-11-24 11:46:02 +01:00
da1d5fc862 Increase max bind vars based on database version (#101464) 2023-10-06 07:59:38 +02:00
34b47a2597 Remove unnnecessary pylint configs from components [m-r]* (#98924) 2023-08-24 00:56:50 +02:00
3094991236 Upgrade ruff to 0.0.285 (#98647) 2023-08-19 08:17:17 -04:00
3e85a29b86 Move overlapping pylint rules to ruff, disable mypy overlap (#94359) 2023-06-27 17:42:46 +02:00
ca936d0b38 Replace bandit with ruff (#93200) 2023-06-08 22:46:04 +02:00
95fcdc5684 Speed up sql queries where ORM rows are not needed (#91839)
* Speed up logbook and history queries where ORM rows are not needed

This avoids having sqlalchemy wrap Result in ChunkedIteratorResult
which has additional overhead we do not need for these cases

* more places

* anything that uses _sorted_statistics_to_dict does not need orm rows either
2023-04-21 23:28:07 -04:00
e40a373c4b Call sqlite pragma optimize during periodic cleanup task (#91245)
https://www.sqlite.org/pragma.html#pragma_optimize

> To achieve the best long-term query performance without the need to do a detailed engineering analysis of the application schema and SQL, it is recommended that applications run "PRAGMA optimize" (with no arguments) just before closing each database connection. Long-running applications might also benefit from setting a timer to run "PRAGMA optimize" every few hours.

> This pragma is usually a no-op or nearly so and is very fast.

Since we keep the recorder connection open for the entire time HA
is running we fall into the long-running application bucket
2023-04-11 22:39:19 -04:00
49079691d4 Reduce overhead of legacy database columns on new installs (#90246)
* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* not working as expected

* override the type compiler

* override the type compiler

* override the type compiler

* override the type compiler

* Apply suggestions from code review

* pgsql char1

* make entity filter test setup with old schema

* fix some more tests that were mutating state

* fix some more tests that were mutating state

* fix some more tests that were mutating state

* fix more dbstate mutations

* add shim for older tests

* split migration tests

* add coverage for purging legacy data

* tweak

* more fixes

* drop some legacy

* fix another test

* fix a few more

* add casts for postgresql in case someone deletes the schema changes table

* dry

* dry

* dry
2023-04-10 10:08:46 -04:00
b1f64de6ce Remove the old ix_states_event_id index if its no longer being used (#89901)
* Remove the old ix_states_event_id index if its no longer being used

* cover it

* fixes

* fixup

* Update homeassistant/components/recorder/tasks.py
2023-03-17 20:27:33 -04:00
85ca94e9d4 Mark database sessions that do not write data as read_only (#89600)
* Mark sessions that do not write data as read_only

* Mark sessions that do not write data as read_only
2023-03-12 21:33:28 -04:00
1bd9767d8c Handle InnoDB deadlocks during migration (#89073)
* Handle slow InnoDB rollback when encountering duplicates during migration

fixes #89069

* adjust

* fix mock

* tests

* return on success
2023-03-03 22:00:13 -05:00
69a3738bdb Fix migration failing when existing data has duplicates (#88712) 2023-02-24 11:41:44 -06:00
c2b770bcb9 Load pending state attributes and event data ids at startup (#88444)
* Load pending state attributes and event data ids at startup

Since we queue all events to be processed after startup
we can have a thundering herd of queries to prime the
LRUs of event data and state attributes ids. Since we
know we are about to process a chunk of events we can
fetch all the ids in two queries

* lru

* fix hang

* Fix recorder LRU being destroyed if event session is reopened

We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened

* Fix recorder LRU being destroyed if event session is reopened

We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened

* cleanup
2023-02-19 21:26:38 -05:00
3533e77ec9 Fix using MariaDB slow range select workaround with Postgresql (#88459)
* Mark PostgreSQL range select as fast

Currently we were using the slow range select workaround for
PostgreSQL that was original developed for MariaDB but
its actually slower on PostgreSQ

fixes #83253

* Mark PostgreSQL range select as fast

Currently we were using the slow range select workaround for
PostgreSQL that was original developed for MariaDB but
its actually slower on PostgreSQ

fixes #83253
2023-02-19 21:08:43 -05:00
b9f29c27ab Cleanup stale recorder code (#88275)
- Removes dead commit function
- Small cleanup to debug handling in execute
2023-02-16 23:12:11 -05:00
db503c0daa Always use UTC time for the MariaDB/MySQL session (#87583) 2023-02-09 11:39:45 -06:00
94519de8dd Upgrade SQLAlchemy to 2.0.2 (#86436)
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-08 08:17:32 -06:00