Compare commits

...

35 Commits

Author SHA1 Message Date
Wietse Wind
8b1805b792 Merge 1c9e2ef147 into 7439ac9bda 2025-01-25 13:43:24 -08:00
Sebastian Goscik
7439ac9bda Bump version: 0.11.0 → 0.12.0 2025-01-18 18:23:33 +00:00
Sebastian Goscik
e3cbcc819e git github action python version parsing 2025-01-18 18:23:33 +00:00
Sebastian Goscik
ccb816ddbc fix bump2version config 2025-01-18 17:19:47 +00:00
Sebastian Goscik
9d2d6558a6 Changelog 2025-01-18 17:18:05 +00:00
Sebastian Goscik
3c5056614c Monkey patch in experimental downloader 2025-01-18 17:07:44 +00:00
Sebastian Goscik
1f18c06e17 Bump dependency versions 2025-01-18 17:07:44 +00:00
Sebastian Goscik
3181080bca Fix issue when --camera isnt specified
Click defaults options with multiple=true to an empty list not None if they are not provided
2025-01-18 16:43:02 +00:00
Wietse Wind
6e5d90a9f5 Add ability to INCLUDE specific cameras instead of EXCLUDE (#179)
Co-authored-by: Sebastian Goscik <sebastian.goscik@live.co.uk>
2025-01-18 15:12:55 +00:00
dependabot[bot]
475beaee3d Bump aiohttp from 3.10.10 to 3.10.11 in the pip group across 1 directory
Bumps the pip group with 1 update in the / directory: [aiohttp](https://github.com/aio-libs/aiohttp).


Updates `aiohttp` from 3.10.10 to 3.10.11
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.10.10...v3.10.11)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: indirect
  dependency-group: pip
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-06 21:02:05 +00:00
Wietse Wind
75cd1207b4 Fix iterating over empty events 2025-01-06 20:41:11 +00:00
Wietse Wind
1c9e2ef147 Allow postprocessing binary/script post download 2024-12-23 17:45:33 +01:00
Sebastian Goscik
c067dbd9f7 Filter out on-going events
Unifi Protect has started to return events that have not ended. These are now explicitly filtered out
2024-10-26 22:12:50 +01:00
Sebastian Goscik
2c43149c99 ruff formatting 2024-10-26 22:12:50 +01:00
Sebastian Goscik
78a2c3034d Bump uiprotect 2024-10-26 22:12:50 +01:00
jimmydoh
1bb8496b30 Adding support for SMART_DETECT_LINE events 2024-10-26 22:12:50 +01:00
Sebastian Goscik
80ad55d0d0 Simplified websocket reconnection logic
This is now handled automatically by uiprotect internally. We do not need to worry about this, greatly simplifying the logic here to just logging messages
2024-10-26 21:27:19 +01:00
jimmydoh
0b2c46888c Replace check_ws with subcription to websocket state 2024-10-26 21:27:19 +01:00
Jonathan Laliberte
0026eaa2ca #171 - Use exponential backoff when logging into Unifi API (#172) 2024-10-10 20:55:00 +00:00
Sebastian Goscik
c3290a223a Update 30-config
Fixed path in error message
2024-09-10 12:32:50 +01:00
Sebastian Goscik
4265643806 Update contribution guide setup steps 2024-08-10 00:38:42 +01:00
Sebastian Goscik
78be4808d9 mypy fixes 2024-08-10 00:17:55 +01:00
Sebastian Goscik
0a6a259120 remove twine dev dependency 2024-08-09 23:53:18 +01:00
Sebastian Goscik
de4f69dcb5 switch pre-commit to ruff 2024-08-09 23:49:11 +01:00
Sebastian Goscik
a7c4eb8dae remove editor config 2024-08-09 23:46:50 +01:00
Sebastian Goscik
129d89480e update git ignore 2024-08-09 23:46:09 +01:00
Sebastian Goscik
a7ccef7f1d ruff check 2024-08-09 23:45:21 +01:00
Sebastian Goscik
bbd70f49bf ruff format 2024-08-09 23:43:03 +01:00
Sebastian Goscik
f9d74c27f9 change linter to ruff 2024-08-09 23:39:54 +01:00
Sebastian Goscik
9d79890eff Update poetry lock 2024-08-09 23:38:46 +01:00
Lloyd Pickering
ccf2cde272 Switch to using UIProtect library (#160)
* Updated poetry dependencies to remove optional flags on dev/test

* file fixups from running poetry run tox

* Updated to Python 3.10

* Switched to UI Protect library

* Updated changelog

* Fix docker permissions

- Make scripts executable by everyone
- Correct XDG variable name to fix incorrect config path being used

* Revert "Updated poetry dependencies to remove optional flags on dev/test" and regenerated lock file
This reverts commit 432d0d3df7.

---------

Co-authored-by: Sebastian Goscik <sebastian.goscik@live.co.uk>
2024-08-09 22:16:19 +00:00
Sebastian Goscik
a8328fd09e Bump version: 0.10.7 → 0.11.0 2024-06-08 01:31:58 +01:00
Sebastian Goscik
28d241610b changelog 2024-06-08 01:31:21 +01:00
Sebastian Goscik
aa1335e73b Fix typos and add experimental downloader to README 2024-06-08 01:29:06 +01:00
Sebastian Goscik
9cb2ccf8b2 Update pyunifiprotect to point to my fork
This is done to accept in features that have not been merged into the upstream repo yet. This also allows for stability in the future.
2024-06-08 01:18:14 +01:00
28 changed files with 1942 additions and 2128 deletions

View File

@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.10.7
current_version = 0.12.0
commit = True
tag = True
@@ -8,8 +8,8 @@ search = version = "{current_version}"
replace = version = "{new_version}"
[bumpversion:file:unifi_protect_backup/__init__.py]
search = __version__ = '{current_version}'
replace = __version__ = '{new_version}'
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"
[bumpversion:file:Dockerfile]
search = COPY dist/unifi_protect_backup-{current_version}.tar.gz sdist.tar.gz

View File

@@ -1,24 +0,0 @@
# http://editorconfig.org
root = true
[*]
indent_style = space
indent_size = 4
trim_trailing_whitespace = true
insert_final_newline = true
charset = utf-8
end_of_line = lf
[*.bat]
indent_style = tab
end_of_line = crlf
[LICENSE]
insert_final_newline = false
[Makefile]
indent_style = tab
[*.{yml, yaml}]
indent_size = 2

View File

@@ -59,7 +59,7 @@ jobs:
- uses: actions/setup-python@v2
with:
python-version: 3.9
python-version: 3.10
- name: Install dependencies
run: |

View File

@@ -2,24 +2,17 @@
name: release & publish workflow
# Controls when the action will run.
on:
# Triggers the workflow on push events but only for the master branch
push:
tags:
- 'v*'
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "release"
release:
name: Create Release
runs-on: ubuntu-20.04
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Get version from tag
id: tag_name
@@ -27,7 +20,6 @@ jobs:
echo ::set-output name=current_version::${GITHUB_REF#refs/tags/v}
shell: bash
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Get Changelog Entry
@@ -39,7 +31,7 @@ jobs:
- uses: actions/setup-python@v2
with:
python-version: 3.9
python-version: "3.10"
- name: Install dependencies
run: |

4
.gitignore vendored
View File

@@ -119,4 +119,6 @@ data/
.envrc
clips/
*.sqlite
*.sqlite
.tool-versions
docker-compose.yml

View File

@@ -7,30 +7,24 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-merge-conflict
- id: check-yaml
args: [ --unsafe ]
- repo: https://github.com/pre-commit/mirrors-isort
rev: v5.8.0
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.5.7
hooks:
- id: isort
args: [ "--filter-files" ]
- repo: https://github.com/ambv/black
rev: 21.5b1
hooks:
- id: black
language_version: python3.9
- repo: https://github.com/pycqa/flake8
rev: 3.9.2
hooks:
- id: flake8
additional_dependencies: [ flake8-typing-imports==1.10.0 ]
# Run the linter.
- id: ruff
# Run the formatter.
- id: ruff-format
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.901
rev: v1.14.1
hooks:
- id: mypy
exclude: tests/
additional_dependencies:
- types-click
- types-pytz
- types-cryptography
- types-python-dateutil
- types-aiofiles

View File

@@ -4,12 +4,30 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.12.0] - 2025-01-18
### Added
- Tool now targets UIProtect instead of pyunifiprotect which should help any lingering auth issues with Unifi OS 4.X
- Python Version bumped to 3.10 (based on UIProtect need)
- The ability to specify only specific cameras to backup
- Re-enabled the experimental downloader after adding a monkey patch for UIProtect to include the unmerged code
- Switched linter to `ruff`
- Added support for SMART_DETECT_LINE events
-
### Fixed
- Unifi now returns unfinished events, this is now handled correctly
- Login attempts now use an exponentially increasing delay to try work around aggressive rate limiting on logins
## [0.11.0] - 2024-06-08
### Added
- A new experimental downloader that uses the same mechanism the web ui does. Enable with
`--experimental-downloader`
### Fixed
- Support for UniFi OS 4.x.x
## [0.10.7] - 2024-03-22
### Fixed
- Set pyunifiprotect to a minimum version of 5.0.0
## [0.10.6] - 2024-03-22
### Fixed
- Bumped `pyunifiprotect` version to fix with versions of Unifi Protect after 3.0.10

View File

@@ -59,7 +59,7 @@ Ready to contribute? Here's how to set up `unifi-protect-backup` for local devel
4. Install dependencies and start your virtualenv:
```
$ poetry install -E test -E dev
$ poetry install --with dev,test
$ poetry shell
```
@@ -78,11 +78,18 @@ Ready to contribute? Here's how to set up `unifi-protect-backup` for local devel
$ poetry run unifi-protect-backup {args}
```
7. When you're done making changes, check that your changes pass the
tests, including testing other Python versions, with tox:
7. Install pre-commit git hooks to ensure all code commit to the repository
is formatted correctly and meets coding standards:
```
$ poetry run pre-commit install
```
8. When you're done making changes, check that your changes pass the
tests:
```
$ poetry run tox
$ poetry run pytest
```
8. Commit your changes and push your branch to GitHub:
@@ -103,7 +110,7 @@ Before you submit a pull request, check that it meets these guidelines:
2. If the pull request adds functionality, the docs should be updated. Put
your new functionality into a function with a docstring. If adding a CLI
option, you should update the "usage" in README.md.
3. The pull request should work for Python 3.9. Check
3. The pull request should work for Python 3.10. Check
https://github.com/ep1cman/unifi-protect-backup/actions
and make sure that the tests pass for all supported Python versions.

View File

@@ -7,10 +7,10 @@ LABEL maintainer="ep1cman"
WORKDIR /app
COPY dist/unifi_protect_backup-0.10.7.tar.gz sdist.tar.gz
COPY dist/unifi_protect_backup-0.12.0.tar.gz sdist.tar.gz
# https://github.com/rust-lang/cargo/issues/2808
ENV CARGO_NET_GIT_FETCH_WITH_CLI=true
ENV CARGO_NET_GIT_FETCH_WITH_CLI=true
RUN \
echo "**** install build packages ****" && \
@@ -51,7 +51,7 @@ ENV IGNORE_CAMERAS=""
ENV SQLITE_PATH=/config/database/events.sqlite
# Fixes issue where `platformdirs` is unable to properly detect the user directory
ENV XDG_CACHE_HOME=/config
ENV XDG_CONFIG_HOME=/config
COPY docker_root/ /

109
README.md
View File

@@ -28,8 +28,8 @@ retention period.
- Automatic pruning of old clips
## Requirements
- Python 3.9+
- Unifi Protect version 1.20 or higher (as per [`pyunifiprotect`](https://github.com/briis/pyunifiprotect))
- Python 3.10+
- Unifi Protect version 1.20 or higher (as per [`uiprotect`](https://github.com/uilibs/uiprotect))
- `rclone` installed with at least one remote configured.
# Setup
@@ -48,7 +48,7 @@ In order to connect to your unifi protect instance, you will first need to setup
## Installation
*The prefered way to run this tool is using a container*
*The preferred way to run this tool is using a container*
### Docker Container
You can run this tool as a container if you prefer with the following command.
@@ -129,8 +129,13 @@ Options:
example.
--rclone-purge-args TEXT Optional extra arguments to pass to `rclone delete` directly.
Common usage for this would be to execute a permanent delete
instead of using the recycle bin on a destination.
Google Drive example: `--drive-use-trash=false`
instead of using the recycle bin on a destination. Google Drive
example: `--drive-use-trash=false`
--postprocess-binary TEXT Optional binary or executable script to run after having
downloaded a video. This can e.g. be a bash script with a CURL
command to post-process the video (detection, move, ...). The
script / binary receives the path where the video is persisted
as first and only argument.
--detection-types TEXT A comma separated list of which types of detections to backup.
Valid options are: `motion`, `person`, `vehicle`, `ring`
[default: motion,person,vehicle,ring]
@@ -138,7 +143,12 @@ Options:
multiple times to ignore multiple IDs. If being set as an
environment variable the IDs should be separated by whitespace.
Alternatively, use a Unifi user with a role which has access
retricted to the subset of cameras that you wish to backup.
restricted to the subset of cameras that you wish to backup.
--camera TEXT IDs of *ONLY* cameras for which events should be backed up. Use
multiple times to include multiple IDs. If being set as an
environment variable the IDs should be separated by whitespace.
Alternatively, use a Unifi user with a role which has access
restricted to the subset of cameras that you wish to backup.
--file-structure-format TEXT A Python format string used to generate the file structure/name
on the rclone remote.For details of the fields available, see
the projects `README.md` file. [default: {camera_name}/{event.s
@@ -189,12 +199,15 @@ Options:
More details about supported platforms can be found here:
https://github.com/caronc/apprise
--skip-missing If set, events which are 'missing' at the start will be ignored.
--skip-missing If set, events which are 'missing' at the start will be ignored.
Subsequent missing events will be downloaded (e.g. a missed event) [default: False]
--download-rate-limit FLOAT Limit how events can be downloaded in one minute. Disabled by
default
--max-event-length INTEGER Only download events shorter than this maximum length, in
seconds [default: 7200]
--experimental-downloader If set, a new experimental download mechanism will be used to match
what the web UI does. This might be more stable if you are experiencing
a lot of failed downloads with the default downloader. [default: False]
--help Show this message and exit.
```
@@ -209,7 +222,9 @@ always take priority over environment variables):
- `RCLONE_DESTINATION`
- `RCLONE_ARGS`
- `RCLONE_PURGE_ARGS`
- `POSTPROCESS_BINARY`
- `IGNORE_CAMERAS`
- `CAMERAS`
- `DETECTION_TYPES`
- `FILE_STRUCTURE_FORMAT`
- `SQLITE_PATH`
@@ -220,6 +235,7 @@ always take priority over environment variables):
- `SKIP_MISSING`
- `DOWNLOAD_RATELIMIT`
- `MAX_EVENT_LENGTH`
- `EXPERIMENTAL_DOWNLOADER`
## File path formatting
@@ -231,7 +247,7 @@ If you wish for the clips to be structured differently you can do this using the
option. It uses standard [python format string syntax](https://docs.python.org/3/library/string.html#formatstrings).
The following fields are provided to the format string:
- *event:* The `Event` object as per https://github.com/briis/pyunifiprotect/blob/master/pyunifiprotect/data/nvr.py
- *event:* The `Event` object as per https://github.com/uilibs/uiprotect/blob/main/src/uiprotect/data/nvr.py
- *duration_seconds:* The duration of the event in seconds
- *detection_type:* A nicely formatted list of the event detection type and the smart detection types (if any)
- *camera_name:* The name of the camera that generated this event
@@ -244,13 +260,45 @@ now on, you can use the `--skip-missing` flag. This does not enable the periodic
If you use this feature it is advised that your run the tool once with this flag, then stop it once the database has been created and the events are ignored. Keeping this flag set permanently could cause events to be missed if the tool crashes and is restarted etc.
## Ignoring cameras
## Selecting cameras
Cameras can be excluded from backups by either:
- Using `--ignore-camera`, see [usage](#usage)
- IDs can be obtained by scanning the logs, starting at `Found cameras:` up to the next log line (currently `NVR TZ`). You can find this section of the logs by piping the logs in to this `sed` command
By default unifi-protect-backup backs up clips from all cameras.
If you want to limit the backups to certain cameras you can do that in one of two ways.
Note: Camera IDs can be obtained by scanning the logs, by looking for `Found cameras:`. You can find this section of the logs by piping the logs in to this `sed` command
`sed -n '/Found cameras:/,/NVR TZ/p'`
- Using a Unifi user with a role which has access retricted to the subset of cameras that you wish to backup.
### Back-up only specific cameras
By using the `--camera` argument, you can specify the ID of the cameras you want to backup. If you want to backup more than one camera you can specify this argument more than once. If this argument is specified all other cameras will be ignored.
#### Example:
If you have three cameras:
- `CAMERA_ID_1`
- `CAMERA_ID_2`
- `CAMERA_ID_3`
and run the following command:
```
$ unifi-protect-backup [...] --camera CAMERA_ID_1 --camera CAMERA_ID_2
```
Only `CAMERA_ID_1` and `CAMERA_ID_2` will be backed up.
### Ignoring cameras
By using the `--ignore-camera` argument, you can specify the ID of the cameras you *do not* want to backup. If you want to ignore more than one camera you can specify this argument more than once. If this argument is specified all cameras will be backed up except the ones specified
#### Example:
If you have three cameras:
- `CAMERA_ID_1`
- `CAMERA_ID_2`
- `CAMERA_ID_3`
and run the following command:
```
$ unifi-protect-backup [...] --ignore-camera CAMERA_ID_1 --ignore-camera CAMERA_ID_2
```
Only `CAMERA_ID_3` will be backed up.
### Note about unifi protect accounts
It is possible to limit what cameras a unifi protect accounts can see. If an account does not have access to a camera this tool will never see it as available so it will not be impacted by the above arguments.
# A note about `rclone` backends and disk wear
This tool attempts to not write the downloaded files to disk to minimise disk wear, and instead streams them directly to
@@ -261,6 +309,37 @@ such backends.
If you are running on a linux host you can setup `rclone` to use `tmpfs` (which is in RAM) to store its temp files, but this will significantly increase memory usage of the tool.
## Prostprocessing
To perform additional detection / cleaning / moving / ... on a video post downloading:
- Use `--postprocess-binary` or env. var: `POSTPROCESS_BINARY`
The binary / executable script receives a first argument with the storage location for the downloaded video. You can easily mount a script from a local filesystem to the container:
```bash
rm -r /tmp/unifi ; docker rmi ghcr.io/ep1cman/unifi-protect-backup ; poetry build && docker buildx build . -t ghcr.io/ep1cman/unifi-protect-backup ;
docker run --rm \
-e POSTPROCESS_BINARY='/postprocess.sh' \
-v '/My/Local/Folder/postprocess.sh':/postprocess.sh \
ghcr.io/ep1cman/unifi-protect-backup
```
The script can be as simple as this (to display the upload path inside the container):
```bash
#!/bin/bash
echo "$1"
```
The logging output will show the stdout and stderr for the postprocess script/binary:
```
Uploaded
-- Postprocessing: 'local:/data/camname/date/vidname.pm4' returned status code: '0'
> STDOUT: /data/camname/date/vidname.pm4
> STDERR:
```
### Running Docker Container (LINUX ONLY)
Add the following arguments to your docker run command:
```
@@ -281,7 +360,7 @@ tmpfs /mnt/tmpfs tmpfs nosuid,nodev,noatime 0 0
```
# Running Backup Tool as a Service (LINUX ONLY)
You can create a service that will run the docker or local version of this backup tool. The service can be configured to launch on boot. This is likely the preferred way you want to execute the tool once you have it completely configured and tested so it is continiously running.
You can create a service that will run the docker or local version of this backup tool. The service can be configured to launch on boot. This is likely the preferred way you want to execute the tool once you have it completely configured and tested so it is continuously running.
First create a service configuration file. You can replace `protectbackup` in the filename below with the name you wish to use for your service, if you change it remember to change the other locations in the following scripts as well.
@@ -351,7 +430,7 @@ docker run \
</a>
- Heavily utilises [`pyunifiprotect`](https://github.com/briis/pyunifiprotect) by [@briis](https://github.com/briis/)
- Heavily utilises [`uiprotect`](https://github.com/uilibs/uiprotect)
- All the cloud functionality is provided by [`rclone`](https://rclone.org/)
- This package was created with [Cookiecutter](https://github.com/audreyr/cookiecutter) and the [waynerv/cookiecutter-pypackage](https://github.com/waynerv/cookiecutter-pypackage) project template.

2
docker_root/etc/cont-init.d/30-config Normal file → Executable file
View File

@@ -4,7 +4,7 @@ mkdir -p /config/rclone
# For backwards compatibility
[[ -f "/root/.config/rclone/rclone.conf" ]] && \
echo "DEPRECATED: Copying rclone conf from /root/.config/rclone/rclone.conf, please change your mount to /config/rclone.conf"
echo "DEPRECATED: Copying rclone conf from /root/.config/rclone/rclone.conf, please change your mount to /config/rclone/rclone.conf"
cp \
/root/.config/rclone/rclone.conf \
/config/rclone/rclone.conf

18
docker_root/etc/services.d/unifi-protect-backup/run Normal file → Executable file
View File

@@ -1,9 +1,21 @@
#!/usr/bin/with-contenv bash
export RCLONE_CONFIG=/config/rclone/rclone.conf
export XDG_CACHE_HOME=/config
echo $VERBOSITY
[[ -n "$VERBOSITY" ]] && export VERBOSITY_ARG=-$VERBOSITY || export VERBOSITY_ARG=""
exec \
s6-setuidgid abc unifi-protect-backup ${VERBOSITY_ARG}
# Run without exec to catch the exit code
s6-setuidgid abc unifi-protect-backup ${VERBOSITY_ARG}
exit_code=$?
# If exit code is 200 (arg error), exit the container
if [ $exit_code -eq 200 ]; then
# Send shutdown signal to s6
/run/s6/basedir/bin/halt
exit $exit_code
fi
# Otherwise, let s6 handle potential restart
exit $exit_code

View File

@@ -30,4 +30,4 @@ clean:
docker:
poetry build
docker buildx build . --platform $(container_arches) -t $(container_name) --push
docker buildx build . --platform $(container_arches) -t $(container_name) --push

3055
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,7 @@
[tool]
[tool.poetry]
name = "unifi_protect_backup"
version = "0.10.7"
version = "0.12.0"
homepage = "https://github.com/ep1cman/unifi-protect-backup"
description = "Python tool to backup unifi event clips in realtime."
authors = ["sebastian.goscik <sebastian@goscik.com>"]
@@ -13,7 +13,7 @@ classifiers=[
'License :: OSI Approved :: MIT License',
'Natural Language :: English',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
]
packages = [
{ include = "unifi_protect_backup" },
@@ -21,7 +21,7 @@ packages = [
]
[tool.poetry.dependencies]
python = ">=3.9.0,<4.0"
python = ">=3.10.0,<4.0"
click = "8.0.1"
aiorun = "^2023.7.2"
aiosqlite = "^0.17.0"
@@ -30,24 +30,20 @@ apprise = "^1.5.0"
expiring-dict = "^1.1.0"
async-lru = "^2.0.4"
aiolimiter = "^1.1.0"
pyunifiprotect = {git = "https://github.com/cyberpower678/pyunifiprotect.git"}
uiprotect = "^6.3.1"
[tool.poetry.group.dev]
optional = true
[tool.poetry.group.dev.dependencies]
black = "^22.10.0"
isort = "^5.8.0"
flake8 = "^3.9.2"
flake8-docstrings = "^1.6.0"
virtualenv = "^20.2.2"
mypy = "^0.900"
mypy = "^1.11.1"
types-pytz = "^2021.3.5"
types-cryptography = "^3.3.18"
twine = "^3.3.0"
types-python-dateutil = "^2.8.19.10"
bump2version = "^1.0.1"
pre-commit = "^2.12.0"
types-python-dateutil = "^2.8.19.10"
ruff = "^0.5.7"
types-aiofiles = "^24.1.0.20241221"
[tool.poetry.group.test]
optional = true
@@ -61,39 +57,19 @@ tox-asdf = "^0.1.0"
[tool.poetry.scripts]
unifi-protect-backup = 'unifi_protect_backup.cli:main'
[tool.black]
[tool.ruff]
line-length = 120
skip-string-normalization = true
target-version = ['py39']
include = '\.pyi?$'
exclude = '''
/(
\.eggs
| \.git
| \.hg
| \.mypy_cache
| \.tox
| \.venv
| _build
| buck-out
| build
| dist
)/
'''
target-version = "py310"
[tool.isort]
multi_line_output = 3
include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
ensure_newline_before_comments = true
line_length = 120
skip_gitignore = true
# you can skip files as below
#skip_glob = docs/conf.py
[tool.ruff.lint]
[tool.ruff.format]
[tool.mypy]
allow_redefinition=true
exclude = [
'unifi_protect_backup/uiprotect_patch.py'
]
[build-system]
requires = ["poetry-core>=1.0.0"]

View File

@@ -1,88 +0,0 @@
[flake8]
max-line-length = 120
max-complexity = 18
ignore = E203, E266, W503
docstring-convention = google
per-file-ignores = __init__.py:F401
exclude = .git,
__pycache__,
setup.py,
build,
dist,
docs,
releases,
.venv,
.tox,
.mypy_cache,
.pytest_cache,
.vscode,
.github,
# By default test codes will be linted.
# tests
[mypy]
ignore_missing_imports = True
[coverage:run]
# uncomment the following to omit files during running
#omit =
[coverage:report]
exclude_lines =
pragma: no cover
def __repr__
if self.debug:
if settings.DEBUG
raise AssertionError
raise NotImplementedError
if 0:
if __name__ == .__main__.:
def main
[tox:tox]
isolated_build = true
envlist = py39, format, lint, build
[gh-actions]
python =
3.9: py39, format, lint, build
[testenv]
allowlist_externals = pytest
extras =
test
passenv = *
setenv =
PYTHONPATH = {toxinidir}
PYTHONWARNINGS = ignore
commands =
pytest --cov=unifi_protect_backup --cov-branch --cov-report=xml --cov-report=term-missing tests
[testenv:format]
allowlist_externals =
isort
black
extras =
test
commands =
isort unifi_protect_backup
black unifi_protect_backup tests
[testenv:lint]
allowlist_externals =
flake8
mypy
extras =
test
commands =
flake8 unifi_protect_backup tests
mypy unifi_protect_backup tests
[testenv:build]
allowlist_externals =
poetry
twine
extras =
dev
commands =
poetry build
twine check dist/*

View File

@@ -1,13 +1,21 @@
"""Top-level package for Unifi Protect Backup."""
__author__ = """sebastian.goscik"""
__email__ = 'sebastian@goscik.com'
__version__ = '0.10.7'
__email__ = "sebastian@goscik.com"
__version__ = "0.12.0"
from .downloader import VideoDownloader
from .downloader_experimental import VideoDownloaderExperimental
from .event_listener import EventListener
from .purge import Purge
from .uploader import VideoUploader
from .missing_event_checker import MissingEventChecker
from .missing_event_checker import MissingEventChecker # isort: skip
__all__ = [
"VideoDownloader",
"VideoDownloaderExperimental",
"EventListener",
"Purge",
"VideoUploader",
"MissingEventChecker",
]

View File

@@ -1,5 +1,6 @@
"""Console script for unifi_protect_backup."""
import sys
import re
import click
@@ -10,12 +11,12 @@ from unifi_protect_backup import __version__
from unifi_protect_backup.unifi_protect_backup_core import UnifiProtectBackup
from unifi_protect_backup.utils import human_readable_to_float
DETECTION_TYPES = ["motion", "person", "vehicle", "ring"]
DETECTION_TYPES = ["motion", "person", "vehicle", "ring", "line"]
def _parse_detection_types(ctx, param, value):
# split columns by ',' and remove whitespace
types = [t.strip() for t in value.split(',')]
types = [t.strip() for t in value.split(",")]
# validate passed columns
for t in types:
@@ -30,7 +31,7 @@ def parse_rclone_retention(ctx, param, retention) -> relativedelta:
matches = {k: int(v) for v, k in re.findall(r"([\d]+)(ms|s|m|h|d|w|M|y)", retention)}
# Check that we matched the whole string
if len(retention) != len(''.join([f"{v}{k}" for k, v in matches.items()])):
if len(retention) != len("".join([f"{v}{k}" for k, v in matches.items()])):
raise click.BadParameter("See here for expected format: https://rclone.org/docs/#time-option")
return relativedelta(
@@ -47,76 +48,94 @@ def parse_rclone_retention(ctx, param, retention) -> relativedelta:
@click.command(context_settings=dict(max_content_width=100))
@click.version_option(__version__)
@click.option('--address', required=True, envvar='UFP_ADDRESS', help='Address of Unifi Protect instance')
@click.option('--port', default=443, envvar='UFP_PORT', show_default=True, help='Port of Unifi Protect instance')
@click.option('--username', required=True, envvar='UFP_USERNAME', help='Username to login to Unifi Protect instance')
@click.option('--password', required=True, envvar='UFP_PASSWORD', help='Password for Unifi Protect user')
@click.option("--address", required=True, envvar="UFP_ADDRESS", help="Address of Unifi Protect instance")
@click.option("--port", default=443, envvar="UFP_PORT", show_default=True, help="Port of Unifi Protect instance")
@click.option("--username", required=True, envvar="UFP_USERNAME", help="Username to login to Unifi Protect instance")
@click.option("--password", required=True, envvar="UFP_PASSWORD", help="Password for Unifi Protect user")
@click.option(
'--verify-ssl/--no-verify-ssl',
"--verify-ssl/--no-verify-ssl",
default=True,
show_default=True,
envvar='UFP_SSL_VERIFY',
envvar="UFP_SSL_VERIFY",
help="Set if you do not have a valid HTTPS Certificate for your instance",
)
@click.option(
'--rclone-destination',
"--rclone-destination",
required=True,
envvar='RCLONE_DESTINATION',
envvar="RCLONE_DESTINATION",
help="`rclone` destination path in the format {rclone remote}:{path on remote}."
" E.g. `gdrive:/backups/unifi_protect`",
)
@click.option(
'--retention',
default='7d',
"--retention",
default="7d",
show_default=True,
envvar='RCLONE_RETENTION',
help="How long should event clips be backed up for. Format as per the `rclone1 time option format "
"(https://rclone.org/docs/#time-option)",
envvar="RCLONE_RETENTION",
help="How long should event clips be backed up for. Format as per the `--max-age` argument of `rclone` "
"(https://rclone.org/filtering/#max-age-don-t-transfer-any-file-older-than-this)",
callback=parse_rclone_retention,
)
@click.option(
'--rclone-args',
default='',
envvar='RCLONE_ARGS',
"--rclone-args",
default="",
envvar="RCLONE_ARGS",
help="Optional extra arguments to pass to `rclone rcat` directly. Common usage for this would "
"be to set a bandwidth limit, for example.",
)
@click.option(
'--rclone-purge-args',
default='',
envvar='RCLONE_PURGE_ARGS',
"--rclone-purge-args",
default="",
envvar="RCLONE_PURGE_ARGS",
help="Optional extra arguments to pass to `rclone delete` directly. Common usage for this would "
"be to execute a permanent delete instead of using the recycle bin on a destination. "
"Google Drive example: `--drive-use-trash=false`",
)
@click.option(
'--detection-types',
envvar='DETECTION_TYPES',
default=','.join(DETECTION_TYPES),
"--postprocess-binary",
default="",
envvar="POSTPROCESS_BINARY",
help="Optional path to binary to postprocess the processed video, gets video destination path as argument."
)
@click.option(
"--detection-types",
envvar="DETECTION_TYPES",
default=",".join(DETECTION_TYPES),
show_default=True,
help="A comma separated list of which types of detections to backup. "
f"Valid options are: {', '.join([f'`{t}`' for t in DETECTION_TYPES])}",
callback=_parse_detection_types,
)
@click.option(
'--ignore-camera',
'ignore_cameras',
"--ignore-camera",
"ignore_cameras",
multiple=True,
envvar="IGNORE_CAMERAS",
help="IDs of cameras for which events should not be backed up. Use multiple times to ignore "
"multiple IDs. If being set as an environment variable the IDs should be separated by whitespace.",
"multiple IDs. If being set as an environment variable the IDs should be separated by whitespace. "
"Alternatively, use a Unifi user with a role which has access restricted to the subset of cameras "
"that you wish to backup.",
)
@click.option(
'--file-structure-format',
envvar='FILE_STRUCTURE_FORMAT',
default="{camera_name}/{event.start:%Y-%m-%d}/{event.start:%Y-%m-%dT%H-%M-%S} {detection_type}.mp4",
"--camera",
"cameras",
multiple=True,
envvar="ONLY_CAMERAS",
help="IDs of *ONLY* cameras for which events should be backed up. Use multiple times to include "
"multiple IDs. If being set as an environment variable the IDs should be separated by whitespace. "
"Alternatively, use a Unifi user with a role which has access restricted to the subset of cameras "
"that you wish to backup.",
)
@click.option(
"--file-structure-format",
envvar="FILE_STRUCTURE_FORMAT",
default="{camera_name}/{event.start:%Y-%m-%d}/{event.end:%Y-%m-%dT%H-%M-%S} {detection_type}.mp4",
show_default=True,
help="A Python format string used to generate the file structure/name on the rclone remote."
"For details of the fields available, see the projects `README.md` file.",
)
@click.option(
'-v',
'--verbose',
"-v",
"--verbose",
count=True,
help="How verbose the logging output should be."
"""
@@ -136,38 +155,38 @@ all warnings, and websocket data
""",
)
@click.option(
'--sqlite_path',
default='events.sqlite',
envvar='SQLITE_PATH',
"--sqlite_path",
default="events.sqlite",
envvar="SQLITE_PATH",
help="Path to the SQLite database to use/create",
)
@click.option(
'--color-logging/--plain-logging',
"--color-logging/--plain-logging",
default=False,
show_default=True,
envvar='COLOR_LOGGING',
envvar="COLOR_LOGGING",
help="Set if you want to use color in logging output",
)
@click.option(
'--download-buffer-size',
default='512MiB',
"--download-buffer-size",
default="512MiB",
show_default=True,
envvar='DOWNLOAD_BUFFER_SIZE',
envvar="DOWNLOAD_BUFFER_SIZE",
help='How big the download buffer should be (you can use suffixes like "B", "KiB", "MiB", "GiB")',
callback=lambda ctx, param, value: human_readable_to_float(value),
)
@click.option(
'--purge_interval',
default='1d',
"--purge_interval",
default="1d",
show_default=True,
envvar='PURGE_INTERVAL',
envvar="PURGE_INTERVAL",
help="How frequently to check for file to purge.\n\nNOTE: Can create a lot of API calls, so be careful if "
"your cloud provider charges you per api call",
callback=parse_rclone_retention,
)
@click.option(
'--apprise-notifier',
'apprise_notifiers',
"--apprise-notifier",
"apprise_notifiers",
multiple=True,
envvar="APPRISE_NOTIFIERS",
help="""\b
@@ -185,39 +204,39 @@ If no tags are specified, it defaults to ERROR
More details about supported platforms can be found here: https://github.com/caronc/apprise""",
)
@click.option(
'--skip-missing',
"--skip-missing",
default=False,
show_default=True,
is_flag=True,
envvar='SKIP_MISSING',
envvar="SKIP_MISSING",
help="""\b
If set, events which are 'missing' at the start will be ignored.
Subsequent missing events will be downloaded (e.g. a missed event)
""",
)
@click.option(
'--download-rate-limit',
"--download-rate-limit",
default=None,
show_default=True,
envvar='DOWNLOAD_RATELIMIT',
envvar="DOWNLOAD_RATELIMIT",
type=float,
help="Limit how events can be downloaded in one minute. Disabled by default",
)
@click.option(
'--max-event-length',
"--max-event-length",
default=2 * 60 * 60,
show_default=True,
envvar='MAX_EVENT_LENGTH',
envvar="MAX_EVENT_LENGTH",
type=int,
help="Only download events shorter than this maximum length, in seconds",
)
@click.option(
'--experimental-downloader',
'use_experimental_downloader',
"--experimental-downloader",
"use_experimental_downloader",
default=False,
show_default=True,
is_flag=True,
envvar='EXPERIMENTAL_DOWNLOADER',
envvar="EXPERIMENTAL_DOWNLOADER",
help="""\b
If set, a new experimental download mechanism will be used to match
what the web UI does. This might be more stable if you are experiencing
@@ -226,8 +245,25 @@ a lot of failed downloads with the default downloader.
)
def main(**kwargs):
"""A Python based tool for backing up Unifi Protect event clips as they occur."""
event_listener = UnifiProtectBackup(**kwargs)
run(event_listener.start(), stop_on_unhandled_errors=True)
try:
# Validate only one of the camera select arguments was given
if kwargs.get("cameras") and kwargs.get("ignore_cameras"):
click.echo(
"Error: --camera and --ignore-camera options are mutually exclusive. "
"Please use only one of these options.",
err=True,
)
raise SystemExit(200) # throw 200 = arg error, service will not be restarted (docker)
# Only create the event listener and run if validation passes
event_listener = UnifiProtectBackup(**kwargs)
run(event_listener.start(), stop_on_unhandled_errors=True)
except SystemExit as e:
sys.exit(e.code)
except Exception as e:
click.echo(f"Error: {str(e)}", err=True)
sys.exit(1)
if __name__ == "__main__":

View File

@@ -10,11 +10,11 @@ from typing import Optional
import aiosqlite
import pytz
from aiohttp.client_exceptions import ClientPayloadError
from expiring_dict import ExpiringDict # type: ignore
from aiolimiter import AsyncLimiter
from pyunifiprotect import ProtectApiClient
from pyunifiprotect.data.nvr import Event
from pyunifiprotect.data.types import EventType
from expiring_dict import ExpiringDict # type: ignore
from uiprotect import ProtectApiClient
from uiprotect.data.nvr import Event
from uiprotect.data.types import EventType
from unifi_protect_backup.utils import (
SubprocessException,
@@ -29,14 +29,14 @@ from unifi_protect_backup.utils import (
async def get_video_length(video: bytes) -> float:
"""Uses ffprobe to get the length of the video file passed in as a byte stream."""
returncode, stdout, stderr = await run_command(
'ffprobe -v quiet -show_streams -select_streams v:0 -of json -', video
"ffprobe -v quiet -show_streams -select_streams v:0 -of json -", video
)
if returncode != 0:
raise SubprocessException(stdout, stderr, returncode)
json_data = json.loads(stdout)
return float(json_data['streams'][0]['duration'])
return float(json_data["streams"][0]["duration"])
class VideoDownloader:
@@ -75,10 +75,10 @@ class VideoDownloader:
self.base_logger = logging.getLogger(__name__)
setup_event_logger(self.base_logger, color_logging)
self.logger = logging.LoggerAdapter(self.base_logger, {'event': ''})
self.logger = logging.LoggerAdapter(self.base_logger, {"event": ""})
# Check if `ffprobe` is available
ffprobe = shutil.which('ffprobe')
ffprobe = shutil.which("ffprobe")
if ffprobe is not None:
self.logger.debug(f"ffprobe found: {ffprobe}")
self._has_ffprobe = True
@@ -100,9 +100,9 @@ class VideoDownloader:
event = await self.download_queue.get()
self.current_event = event
self.logger = logging.LoggerAdapter(self.base_logger, {'event': f' [{event.id}]'})
self.logger = logging.LoggerAdapter(self.base_logger, {"event": f" [{event.id}]"})
# Fix timezones since pyunifiprotect sets all timestamps to UTC. Instead localize them to
# Fix timezones since uiprotect sets all timestamps to UTC. Instead localize them to
# the timezone of the unifi protect NVR.
event.start = event.start.replace(tzinfo=pytz.utc).astimezone(self._protect.bootstrap.nvr.timezone)
event.end = event.end.replace(tzinfo=pytz.utc).astimezone(self._protect.bootstrap.nvr.timezone)

View File

@@ -10,11 +10,11 @@ from typing import Optional
import aiosqlite
import pytz
from aiohttp.client_exceptions import ClientPayloadError
from expiring_dict import ExpiringDict # type: ignore
from aiolimiter import AsyncLimiter
from pyunifiprotect import ProtectApiClient
from pyunifiprotect.data.nvr import Event
from pyunifiprotect.data.types import EventType
from expiring_dict import ExpiringDict # type: ignore
from uiprotect import ProtectApiClient
from uiprotect.data.nvr import Event
from uiprotect.data.types import EventType
from unifi_protect_backup.utils import (
SubprocessException,
@@ -29,14 +29,14 @@ from unifi_protect_backup.utils import (
async def get_video_length(video: bytes) -> float:
"""Uses ffprobe to get the length of the video file passed in as a byte stream."""
returncode, stdout, stderr = await run_command(
'ffprobe -v quiet -show_streams -select_streams v:0 -of json -', video
"ffprobe -v quiet -show_streams -select_streams v:0 -of json -", video
)
if returncode != 0:
raise SubprocessException(stdout, stderr, returncode)
json_data = json.loads(stdout)
return float(json_data['streams'][0]['duration'])
return float(json_data["streams"][0]["duration"])
class VideoDownloaderExperimental:
@@ -75,10 +75,10 @@ class VideoDownloaderExperimental:
self.base_logger = logging.getLogger(__name__)
setup_event_logger(self.base_logger, color_logging)
self.logger = logging.LoggerAdapter(self.base_logger, {'event': ''})
self.logger = logging.LoggerAdapter(self.base_logger, {"event": ""})
# Check if `ffprobe` is available
ffprobe = shutil.which('ffprobe')
ffprobe = shutil.which("ffprobe")
if ffprobe is not None:
self.logger.debug(f"ffprobe found: {ffprobe}")
self._has_ffprobe = True
@@ -100,9 +100,9 @@ class VideoDownloaderExperimental:
event = await self.download_queue.get()
self.current_event = event
self.logger = logging.LoggerAdapter(self.base_logger, {'event': f' [{event.id}]'})
self.logger = logging.LoggerAdapter(self.base_logger, {"event": f" [{event.id}]"})
# Fix timezones since pyunifiprotect sets all timestamps to UTC. Instead localize them to
# Fix timezones since uiprotect sets all timestamps to UTC. Instead localize them to
# the timezone of the unifi protect NVR.
event.start = event.start.replace(tzinfo=pytz.utc).astimezone(self._protect.bootstrap.nvr.timezone)
event.end = event.end.replace(tzinfo=pytz.utc).astimezone(self._protect.bootstrap.nvr.timezone)
@@ -149,7 +149,10 @@ class VideoDownloaderExperimental:
self._failures[event.id] = 1
else:
self._failures[event.id] += 1
self.logger.warning(f"Event failed download attempt {self._failures[event.id]}", exc_info=e)
self.logger.warning(
f"Event failed download attempt {self._failures[event.id]}",
exc_info=e,
)
if self._failures[event.id] >= 10:
self.logger.error(
@@ -171,7 +174,10 @@ class VideoDownloaderExperimental:
self.current_event = None
except Exception as e:
self.logger.error(f"Unexpected exception occurred, abandoning event {event.id}:", exc_info=e)
self.logger.error(
f"Unexpected exception occurred, abandoning event {event.id}:",
exc_info=e,
)
async def _download(self, event: Event) -> Optional[bytes]:
"""Downloads the video clip for the given event."""
@@ -181,8 +187,12 @@ class VideoDownloaderExperimental:
assert isinstance(event.start, datetime)
assert isinstance(event.end, datetime)
try:
prepared_video_file = await self._protect.prepare_camera_video(event.camera_id, event.start, event.end)
video = await self._protect.download_camera_video(event.camera_id, prepared_video_file['fileName'])
prepared_video_file = await self._protect.prepare_camera_video( # type: ignore
event.camera_id, event.start, event.end
)
video = await self._protect.download_camera_video( # type: ignore
event.camera_id, prepared_video_file["fileName"]
)
assert isinstance(video, bytes)
break
except (AssertionError, ClientPayloadError, TimeoutError) as e:

View File

@@ -5,10 +5,11 @@ import logging
from time import sleep
from typing import List
from pyunifiprotect.api import ProtectApiClient
from pyunifiprotect.data.nvr import Event
from pyunifiprotect.data.types import EventType
from pyunifiprotect.data.websocket import WSAction, WSSubscriptionMessage
from uiprotect.api import ProtectApiClient
from uiprotect.websocket import WebsocketState
from uiprotect.data.nvr import Event
from uiprotect.data.types import EventType
from uiprotect.data.websocket import WSAction, WSSubscriptionMessage
logger = logging.getLogger(__name__)
@@ -22,6 +23,7 @@ class EventListener:
protect: ProtectApiClient,
detection_types: List[str],
ignore_cameras: List[str],
cameras: List[str],
):
"""Init.
@@ -30,22 +32,22 @@ class EventListener:
protect (ProtectApiClient): UniFI Protect API client to use
detection_types (List[str]): Desired Event detection types to look for
ignore_cameras (List[str]): Cameras IDs to ignore events from
cameras (List[str]): Cameras IDs to ONLY include events from
"""
self._event_queue: asyncio.Queue = event_queue
self._protect: ProtectApiClient = protect
self._unsub = None
self._unsub_websocketstate = None
self.detection_types: List[str] = detection_types
self.ignore_cameras: List[str] = ignore_cameras
self.cameras: List[str] = cameras
async def start(self):
"""Main Loop."""
logger.debug("Subscribed to websocket")
self._unsub_websocket_state = self._protect.subscribe_websocket_state(self._websocket_state_callback)
self._unsub = self._protect.subscribe_websocket(self._websocket_callback)
while True:
await asyncio.sleep(60)
await self._check_websocket_and_reconnect()
def _websocket_callback(self, msg: WSSubscriptionMessage) -> None:
"""Callback for "EVENT" websocket messages.
@@ -61,9 +63,16 @@ class EventListener:
return
if msg.new_obj.camera_id in self.ignore_cameras:
return
if 'end' not in msg.changed_data:
if self.cameras and msg.new_obj.camera_id not in self.cameras:
return
if msg.new_obj.type not in [EventType.MOTION, EventType.SMART_DETECT, EventType.RING]:
if "end" not in msg.changed_data:
return
if msg.new_obj.type not in [
EventType.MOTION,
EventType.SMART_DETECT,
EventType.RING,
EventType.SMART_DETECT_LINE,
]:
return
if msg.new_obj.type is EventType.MOTION and "motion" not in self.detection_types:
logger.extra_debug(f"Skipping unwanted motion detection event: {msg.new_obj.id}") # type: ignore
@@ -71,6 +80,9 @@ class EventListener:
if msg.new_obj.type is EventType.RING and "ring" not in self.detection_types:
logger.extra_debug(f"Skipping unwanted ring event: {msg.new_obj.id}") # type: ignore
return
if msg.new_obj.type is EventType.SMART_DETECT_LINE and "line" not in self.detection_types:
logger.extra_debug(f"Skipping unwanted line event: {msg.new_obj.id}") # type: ignore
return
elif msg.new_obj.type is EventType.SMART_DETECT:
for event_smart_detection_type in msg.new_obj.smart_detect_types:
if event_smart_detection_type not in self.detection_types:
@@ -89,42 +101,20 @@ class EventListener:
# Unifi protect has started sending the event id in the websocket as a {event_id}-{camera_id} but when the
# API is queried they only have {event_id}. Keeping track of these both of these would be complicated so
# instead we fudge the ID here to match what the API returns
if '-' in msg.new_obj.id:
msg.new_obj.id = msg.new_obj.id.split('-')[0]
if "-" in msg.new_obj.id:
msg.new_obj.id = msg.new_obj.id.split("-")[0]
logger.debug(f"Adding event {msg.new_obj.id} to queue (Current download queue={self._event_queue.qsize()})")
async def _check_websocket_and_reconnect(self):
"""Checks for websocket disconnect and triggers a reconnect."""
logger.extra_debug("Checking the status of the websocket...")
if self._protect.check_ws():
logger.extra_debug("Websocket is connected.")
else:
self._protect.connect_event.clear()
logger.warning("Lost connection to Unifi Protect.")
def _websocket_state_callback(self, state: WebsocketState) -> None:
"""Callback for websocket state messages.
# Unsubscribe, close the session.
self._unsub()
await self._protect.close_session()
Flags the websocket for reconnection
while True:
logger.warning("Attempting reconnect...")
try:
# Start the pyunifiprotect connection by calling `update`
await self._protect.close_session()
self._protect._bootstrap = None
await self._protect.update(force=True)
if self._protect.check_ws():
self._unsub = self._protect.subscribe_websocket(self._websocket_callback)
break
else:
logger.error("Unable to establish connection to Unifi Protect")
except Exception as e:
logger.error("Unexpected exception occurred while trying to reconnect:", exc_info=e)
# Back off for a little while
await asyncio.sleep(10)
self._protect.connect_event.set()
logger.info("Re-established connection to Unifi Protect and to the websocket.")
Args:
msg (WebsocketState): new state of the websocket
"""
if state == WebsocketState.DISCONNECTED:
logger.error("Unifi Protect Websocket lost connection. Reconnecting...")
elif state == WebsocketState.CONNECTED:
logger.info("Unifi Protect Websocket connection restored")

View File

@@ -3,13 +3,13 @@
import asyncio
import logging
from datetime import datetime
from typing import List
from typing import AsyncIterator, List
import aiosqlite
from dateutil.relativedelta import relativedelta
from pyunifiprotect import ProtectApiClient
from pyunifiprotect.data.nvr import Event
from pyunifiprotect.data.types import EventType
from uiprotect import ProtectApiClient
from uiprotect.data.nvr import Event
from uiprotect.data.types import EventType
from unifi_protect_backup import VideoDownloader, VideoUploader
@@ -29,6 +29,7 @@ class MissingEventChecker:
retention: relativedelta,
detection_types: List[str],
ignore_cameras: List[str],
cameras: List[str],
interval: int = 60 * 5,
) -> None:
"""Init.
@@ -42,6 +43,7 @@ class MissingEventChecker:
retention (relativedelta): Retention period to limit search window
detection_types (List[str]): Detection types wanted to limit search
ignore_cameras (List[str]): Ignored camera IDs to limit search
cameras (List[str]): Included (ONLY) camera IDs to limit search
interval (int): How frequently, in seconds, to check for missing events,
"""
self._protect: ProtectApiClient = protect
@@ -52,28 +54,40 @@ class MissingEventChecker:
self.retention: relativedelta = retention
self.detection_types: List[str] = detection_types
self.ignore_cameras: List[str] = ignore_cameras
self.cameras: List[str] = cameras
self.interval: int = interval
async def _get_missing_events(self) -> List[Event]:
async def _get_missing_events(self) -> AsyncIterator[Event]:
start_time = datetime.now() - self.retention
end_time = datetime.now()
chunk_size = 500
while True:
# Get list of events that need to be backed up from unifi protect
logger.extra_debug(f"Fetching events for interval: {start_time} - {end_time}")
logger.extra_debug(f"Fetching events for interval: {start_time} - {end_time}") # type: ignore
events_chunk = await self._protect.get_events(
start=start_time,
end=end_time,
types=[EventType.MOTION, EventType.SMART_DETECT, EventType.RING],
types=[
EventType.MOTION,
EventType.SMART_DETECT,
EventType.RING,
EventType.SMART_DETECT_LINE,
],
limit=chunk_size,
)
if not events_chunk:
break # There were no events to backup
start_time = events_chunk[-1].end
unifi_events = {event.id: event for event in events_chunk}
# Filter out on-going events
unifi_events = {event.id: event for event in events_chunk if event.end is not None}
if not unifi_events:
break # No completed events to process
# Next chunks start time should be the end of the oldest complete event in the current chunk
start_time = max([event.end for event in unifi_events.values() if event.end is not None])
# Get list of events that have been backed up from the database
@@ -102,10 +116,14 @@ class MissingEventChecker:
return False # This event is still on-going
if event.camera_id in self.ignore_cameras:
return False
if self.cameras and event.camera_id not in self.cameras:
return False
if event.type is EventType.MOTION and "motion" not in self.detection_types:
return False
if event.type is EventType.RING and "ring" not in self.detection_types:
return False
if event.type is EventType.SMART_DETECT_LINE and "line" not in self.detection_types:
return False
elif event.type is EventType.SMART_DETECT:
for event_smart_detection_type in event.smart_detect_types:
if event_smart_detection_type not in self.detection_types:
@@ -125,7 +143,7 @@ class MissingEventChecker:
async def ignore_missing(self):
"""Ignore missing events by adding them to the event table."""
logger.info(f" Ignoring missing events")
logger.info(" Ignoring missing events")
async for event in self._get_missing_events():
logger.extra_debug(f"Ignoring event '{event.id}'")
@@ -150,7 +168,7 @@ class MissingEventChecker:
async for event in self._get_missing_events():
if not shown_warning:
logger.warning(f" Found missing events, adding to backup queue")
logger.warning(" Found missing events, adding to backup queue")
shown_warning = True
if event.type != EventType.SMART_DETECT:
@@ -166,6 +184,9 @@ class MissingEventChecker:
await self._download_queue.put(event)
except Exception as e:
logger.error("Unexpected exception occurred during missing event check:", exc_info=e)
logger.error(
"Unexpected exception occurred during missing event check:",
exc_info=e,
)
await asyncio.sleep(self.interval)

View File

@@ -8,11 +8,11 @@ notifier = apprise.Apprise()
def add_notification_service(url):
"""Add apprise URI with support for tags e.g. TAG1,TAG2=PROTOCOL://settings."""
config = apprise.AppriseConfig()
config.add_config(url, format='text')
config.add_config(url, format="text")
# If not tags are specified, default to errors otherwise ALL logging will
# be spammed to the notification service
if not config.servers()[0].tags:
config.servers()[0].tags = {'ERROR'}
config.servers()[0].tags = {"ERROR"}
notifier.add(config)

View File

@@ -64,7 +64,6 @@ class Purge:
f"SELECT * FROM events WHERE end < {retention_oldest_time}"
) as event_cursor:
async for event_id, event_type, camera_id, event_start, event_end in event_cursor:
logger.info(f"Purging event: {event_id}.")
# For every backup for this event
@@ -86,5 +85,5 @@ class Purge:
logger.error("Unexpected exception occurred during purge:", exc_info=e)
next_purge_time = datetime.now() + self.interval
logger.extra_debug(f'sleeping until {next_purge_time}')
logger.extra_debug(f"sleeping until {next_purge_time}")
await wait_until(next_purge_time)

View File

@@ -0,0 +1,135 @@
import enum
from datetime import datetime
from pathlib import Path
from typing import Any, Optional
import aiofiles
from uiprotect.data import Version
from uiprotect.exceptions import BadRequest
from uiprotect.utils import to_js_time
# First, let's add the new VideoExportType enum
class VideoExportType(str, enum.Enum):
TIMELAPSE = "timelapse"
ROTATING = "rotating"
def monkey_patch_experimental_downloader():
from uiprotect.api import ProtectApiClient
# Add the version constant
ProtectApiClient.NEW_DOWNLOAD_VERSION = Version("4.0.0") # You'll need to import Version from uiprotect
async def _validate_channel_id(self, camera_id: str, channel_index: int) -> None:
if self._bootstrap is None:
await self.update()
try:
camera = self._bootstrap.cameras[camera_id]
camera.channels[channel_index]
except (IndexError, AttributeError, KeyError) as e:
raise BadRequest(f"Invalid input: {e}") from e
async def prepare_camera_video(
self,
camera_id: str,
start: datetime,
end: datetime,
channel_index: int = 0,
validate_channel_id: bool = True,
fps: Optional[int] = None,
filename: Optional[str] = None,
) -> Optional[dict[str, Any]]:
if self.bootstrap.nvr.version < self.NEW_DOWNLOAD_VERSION:
raise ValueError("This method is only support from Unifi Protect version >= 4.0.0.")
if validate_channel_id:
await self._validate_channel_id(camera_id, channel_index)
params = {
"camera": camera_id,
"start": to_js_time(start),
"end": to_js_time(end),
}
if channel_index == 3:
params.update({"lens": 2})
else:
params.update({"channel": channel_index})
if fps is not None and fps > 0:
params["fps"] = fps
params["type"] = VideoExportType.TIMELAPSE.value
else:
params["type"] = VideoExportType.ROTATING.value
if not filename:
start_str = start.strftime("%m-%d-%Y, %H.%M.%S %Z")
end_str = end.strftime("%m-%d-%Y, %H.%M.%S %Z")
filename = f"{camera_id} {start_str} - {end_str}.mp4"
params["filename"] = filename
return await self.api_request(
"video/prepare",
params=params,
raise_exception=True,
)
async def download_camera_video(
self,
camera_id: str,
filename: str,
output_file: Optional[Path] = None,
iterator_callback: Optional[callable] = None,
progress_callback: Optional[callable] = None,
chunk_size: int = 65536,
) -> Optional[bytes]:
if self.bootstrap.nvr.version < self.NEW_DOWNLOAD_VERSION:
raise ValueError("This method is only support from Unifi Protect version >= 4.0.0.")
params = {
"camera": camera_id,
"filename": filename,
}
if iterator_callback is None and progress_callback is None and output_file is None:
return await self.api_request_raw(
"video/download",
params=params,
raise_exception=False,
)
r = await self.request(
"get",
f"{self.api_path}video/download",
auto_close=False,
timeout=0,
params=params,
)
if output_file is not None:
async with aiofiles.open(output_file, "wb") as output:
async def callback(total: int, chunk: Optional[bytes]) -> None:
if iterator_callback is not None:
await iterator_callback(total, chunk)
if chunk is not None:
await output.write(chunk)
await self._stream_response(r, chunk_size, callback, progress_callback)
else:
await self._stream_response(
r,
chunk_size,
iterator_callback,
progress_callback,
)
r.close()
return None
# Patch the methods into the class
ProtectApiClient._validate_channel_id = _validate_channel_id
ProtectApiClient.prepare_camera_video = prepare_camera_video
ProtectApiClient.download_camera_video = download_camera_video

View File

@@ -4,13 +4,13 @@ import asyncio
import logging
import os
import shutil
from datetime import datetime, timezone, timedelta
from datetime import datetime, timedelta, timezone
from typing import Callable, List
import aiosqlite
from dateutil.relativedelta import relativedelta
from pyunifiprotect import ProtectApiClient
from pyunifiprotect.data.types import ModelType
from uiprotect import ProtectApiClient
from uiprotect.data.types import ModelType
from unifi_protect_backup import (
EventListener,
@@ -21,13 +21,27 @@ from unifi_protect_backup import (
VideoUploader,
notifications,
)
from unifi_protect_backup.utils import SubprocessException, VideoQueue, human_readable_size, run_command, setup_logging
from unifi_protect_backup.utils import (
SubprocessException,
VideoQueue,
human_readable_size,
run_command,
setup_logging,
)
from unifi_protect_backup.uiprotect_patch import monkey_patch_experimental_downloader
logger = logging.getLogger(__name__)
# TODO: https://github.com/cjrh/aiorun#id6 (smart shield)
# We have been waiting for a long time for this PR to get merged
# https://github.com/uilibs/uiprotect/pull/249
# Since it has not progressed, we will for now patch in the functionality ourselves
monkey_patch_experimental_downloader()
async def create_database(path: str):
"""Creates sqlite database and creates the events abd backups tables."""
db = await aiosqlite.connect(path)
@@ -56,8 +70,10 @@ class UnifiProtectBackup:
retention: relativedelta,
rclone_args: str,
rclone_purge_args: str,
postprocess_binary: str,
detection_types: List[str],
ignore_cameras: List[str],
cameras: List[str],
file_structure_format: str,
verbose: int,
download_buffer_size: int,
@@ -67,7 +83,7 @@ class UnifiProtectBackup:
max_event_length: int,
sqlite_path: str = "events.sqlite",
color_logging: bool = False,
download_rate_limit: float = None,
download_rate_limit: float | None = None,
port: int = 443,
use_experimental_downloader: bool = False,
):
@@ -88,8 +104,10 @@ class UnifiProtectBackup:
rclone_args (str): A bandwidth limit which is passed to the `--bwlimit` argument of
`rclone` (https://rclone.org/docs/#bwlimit-bandwidth-spec)
rclone_purge_args (str): Optional extra arguments to pass to `rclone delete` directly.
postprocess_binary (str): Optional path to a binary that gets called to postprocess, with download location as argument.
detection_types (List[str]): List of which detection types to backup.
ignore_cameras (List[str]): List of camera IDs for which to not backup events.
cameras (List[str]): List of ONLY camera IDs for which to backup events.
file_structure_format (str): A Python format string for output file path.
verbose (int): How verbose to setup logging, see :func:`setup_logging` for details.
download_buffer_size (int): How many bytes big the download buffer should be
@@ -126,7 +144,9 @@ class UnifiProtectBackup:
logger.debug(f" {retention=}")
logger.debug(f" {rclone_args=}")
logger.debug(f" {rclone_purge_args=}")
logger.debug(f" {postprocess_binary=}")
logger.debug(f" {ignore_cameras=}")
logger.debug(f" {cameras=}")
logger.debug(f" {verbose=}")
logger.debug(f" {detection_types=}")
logger.debug(f" {file_structure_format=}")
@@ -143,6 +163,7 @@ class UnifiProtectBackup:
self.retention = retention
self.rclone_args = rclone_args
self.rclone_purge_args = rclone_purge_args
self.postprocess_binary = postprocess_binary
self.file_structure_format = file_structure_format
self.address = address
@@ -160,6 +181,7 @@ class UnifiProtectBackup:
subscribed_models={ModelType.EVENT},
)
self.ignore_cameras = ignore_cameras
self.cameras = cameras
self._download_queue: asyncio.Queue = asyncio.Queue()
self._unsub: Callable[[], None]
self.detection_types = detection_types
@@ -188,18 +210,25 @@ class UnifiProtectBackup:
logger.info("Checking rclone configuration...")
await self._check_rclone()
# Start the pyunifiprotect connection by calling `update`
# Start the uiprotect connection by calling `update`
logger.info("Connecting to Unifi Protect...")
for attempts in range(10):
delay = 5 # Start with a 5 second delay
max_delay = 3600 # 1 hour in seconds
for attempts in range(20):
try:
await self._protect.update()
break
except Exception as e:
logger.warning(f"Failed to connect to UniFi Protect, retrying in {attempts}s...", exc_info=e)
await asyncio.sleep(attempts)
logger.warning(
f"Failed to connect to UniFi Protect, retrying in {delay}s...",
exc_info=e,
)
await asyncio.sleep(delay)
delay = min(max_delay, delay * 2) # Double the delay but do not exceed max_delay
else:
raise ConnectionError("Failed to connect to UniFi Protect after 10 attempts")
raise ConnectionError("Failed to connect to UniFi Protect after 20 attempts")
# Add a lock to the protect client that can be used to prevent code accessing the client when it has
# lost connection
@@ -212,8 +241,8 @@ class UnifiProtectBackup:
logger.info(f" - {camera.id}: {camera.name}")
# Print timezone info for debugging
logger.debug(f'NVR TZ: {self._protect.bootstrap.nvr.timezone}')
logger.debug(f'Local TZ: {datetime.now(timezone.utc).astimezone().tzinfo}')
logger.debug(f"NVR TZ: {self._protect.bootstrap.nvr.timezone}")
logger.debug(f"Local TZ: {datetime.now(timezone.utc).astimezone().tzinfo}")
tasks = []
@@ -257,19 +286,26 @@ class UnifiProtectBackup:
self.file_structure_format,
self._db,
self.color_logging,
self.postprocess_binary,
)
tasks.append(uploader.start())
# Create event listener task
# This will connect to the unifi protect websocket and listen for events. When one is detected it will
# be added to the queue of events to download
event_listener = EventListener(download_queue, self._protect, self.detection_types, self.ignore_cameras)
event_listener = EventListener(
download_queue, self._protect, self.detection_types, self.ignore_cameras, self.cameras
)
tasks.append(event_listener.start())
# Create purge task
# This will, every midnight, purge old backups from the rclone remotes and database
purge = Purge(
self._db, self.retention, self.rclone_destination, self._purge_interval, self.rclone_purge_args
self._db,
self.retention,
self.rclone_destination,
self._purge_interval,
self.rclone_purge_args,
)
tasks.append(purge.start())
@@ -285,6 +321,7 @@ class UnifiProtectBackup:
self.retention,
self.detection_types,
self.ignore_cameras,
self.cameras,
)
if self._skip_missing:
logger.info("Ignoring missing events")
@@ -313,7 +350,7 @@ class UnifiProtectBackup:
ValueError: The given rclone destination is for a remote that is not configured
"""
rclone = shutil.which('rclone')
rclone = shutil.which("rclone")
if not rclone:
raise RuntimeError("`rclone` is not installed on this system")
logger.debug(f"rclone found: {rclone}")

View File

@@ -6,16 +6,16 @@ import re
from datetime import datetime
import aiosqlite
from pyunifiprotect import ProtectApiClient
from pyunifiprotect.data.nvr import Event
from uiprotect import ProtectApiClient
from uiprotect.data.nvr import Event
from unifi_protect_backup.utils import (
SubprocessException,
VideoQueue,
get_camera_name,
human_readable_size,
run_command,
setup_event_logger,
SubprocessException,
)
@@ -34,6 +34,7 @@ class VideoUploader:
file_structure_format: str,
db: aiosqlite.Connection,
color_logging: bool,
postprocess_binary: str,
):
"""Init.
@@ -45,18 +46,20 @@ class VideoUploader:
file_structure_format (str): format string for how to structure the uploaded files
db (aiosqlite.Connection): Async SQlite database connection
color_logging (bool): Whether or not to add color to logging output
postprocess_binary (str): Optional postprocess binary path (output location as arg)
"""
self._protect: ProtectApiClient = protect
self.upload_queue: VideoQueue = upload_queue
self._rclone_destination: str = rclone_destination
self._rclone_args: str = rclone_args
self._postprocess_binary: str = postprocess_binary
self._file_structure_format: str = file_structure_format
self._db: aiosqlite.Connection = db
self.current_event = None
self.base_logger = logging.getLogger(__name__)
setup_event_logger(self.base_logger, color_logging)
self.logger = logging.LoggerAdapter(self.base_logger, {'event': ''})
self.logger = logging.LoggerAdapter(self.base_logger, {"event": ""})
async def start(self):
"""Main loop.
@@ -70,7 +73,7 @@ class VideoUploader:
event, video = await self.upload_queue.get()
self.current_event = event
self.logger = logging.LoggerAdapter(self.base_logger, {'event': f' [{event.id}]'})
self.logger = logging.LoggerAdapter(self.base_logger, {"event": f" [{event.id}]"})
self.logger.info(f"Uploading event: {event.id}")
self.logger.debug(
@@ -82,9 +85,16 @@ class VideoUploader:
self.logger.debug(f" Destination: {destination}")
try:
await self._upload_video(video, destination, self._rclone_args)
await self._upload_video(video, destination, self._rclone_args, self._postprocess_binary)
await self._update_database(event, destination)
self.logger.debug("Uploaded")
# Postprocess
if self._postprocess_binary:
returncode_postprocess, stdout_postprocess, stderr_postprocess = await run_command(f'"{self._postprocess_binary}" "{destination}"')
self.logger.debug(f" -- Postprocessing: '{destination}' returned status code: '{returncode_postprocess}'")
self.logger.debug(f" > STDOUT: {stdout_postprocess.strip()}")
self.logger.debug(f" > STDERR: {stderr_postprocess.strip()}")
except SubprocessException:
self.logger.error(f" Failed to upload file: '{destination}'")
@@ -93,7 +103,7 @@ class VideoUploader:
except Exception as e:
self.logger.error(f"Unexpected exception occurred, abandoning event {event.id}:", exc_info=e)
async def _upload_video(self, video: bytes, destination: pathlib.Path, rclone_args: str):
async def _upload_video(self, video: bytes, destination: pathlib.Path, rclone_args: str, postprocess_binary: str):
"""Upload video using rclone.
In order to avoid writing to disk, the video file data is piped directly
@@ -103,6 +113,7 @@ class VideoUploader:
video (bytes): The data to be written to the file
destination (pathlib.Path): Where rclone should write the file
rclone_args (str): Optional extra arguments to pass to `rclone`
postprocess_binary (str): Optional extra path to postprocessing binary
Raises:
RuntimeError: If rclone returns a non-zero exit code
@@ -138,7 +149,7 @@ class VideoUploader:
Provides the following fields to the format string:
event: The `Event` object as per
https://github.com/briis/pyunifiprotect/blob/master/pyunifiprotect/data/nvr.py
https://github.com/briis/uiprotect/blob/master/uiprotect/data/nvr.py
duration_seconds: The duration of the event in seconds
detection_type: A nicely formatted list of the event detection type and the smart detection types (if any)
camera_name: The name of the camera that generated this event
@@ -164,6 +175,6 @@ class VideoUploader:
}
file_path = self._file_structure_format.format(**format_context)
file_path = re.sub(r'[^\w\-_\.\(\)/ ]', '', file_path) # Sanitize any invalid chars
file_path = re.sub(r"[^\w\-_\.\(\)/ ]", "", file_path) # Sanitize any invalid chars
return pathlib.Path(f"{self._rclone_destination}/{file_path}")

View File

@@ -8,8 +8,8 @@ from typing import List, Optional
from apprise import NotifyType
from async_lru import alru_cache
from pyunifiprotect import ProtectApiClient
from pyunifiprotect.data.nvr import Event
from uiprotect import ProtectApiClient
from uiprotect.data.nvr import Event
from unifi_protect_backup import notifications
@@ -50,11 +50,11 @@ def add_logging_level(levelName: str, levelNum: int, methodName: Optional[str] =
methodName = levelName.lower()
if hasattr(logging, levelName):
raise AttributeError('{} already defined in logging module'.format(levelName))
raise AttributeError("{} already defined in logging module".format(levelName))
if hasattr(logging, methodName):
raise AttributeError('{} already defined in logging module'.format(methodName))
raise AttributeError("{} already defined in logging module".format(methodName))
if hasattr(logging.getLoggerClass(), methodName):
raise AttributeError('{} already defined in logger class'.format(methodName))
raise AttributeError("{} already defined in logger class".format(methodName))
# This method was inspired by the answers to Stack Overflow post
# http://stackoverflow.com/q/2183233/2988730, especially
@@ -84,19 +84,19 @@ def add_color_to_record_levelname(record):
"""Colorizes logging level names."""
levelno = record.levelno
if levelno >= logging.CRITICAL:
color = '\x1b[31;1m' # RED
color = "\x1b[31;1m" # RED
elif levelno >= logging.ERROR:
color = '\x1b[31;1m' # RED
color = "\x1b[31;1m" # RED
elif levelno >= logging.WARNING:
color = '\x1b[33;1m' # YELLOW
color = "\x1b[33;1m" # YELLOW
elif levelno >= logging.INFO:
color = '\x1b[32;1m' # GREEN
color = "\x1b[32;1m" # GREEN
elif levelno >= logging.DEBUG:
color = '\x1b[36;1m' # CYAN
color = "\x1b[36;1m" # CYAN
elif levelno >= logging.EXTRA_DEBUG:
color = '\x1b[35;1m' # MAGENTA
color = "\x1b[35;1m" # MAGENTA
else:
color = '\x1b[0m'
color = "\x1b[0m"
return f"{color}{record.levelname}\x1b[0m"
@@ -174,7 +174,7 @@ class AppriseStreamHandler(logging.StreamHandler):
def create_logging_handler(format, color_logging):
"""Constructs apprise logging handler for the given format."""
date_format = "%Y-%m-%d %H:%M:%S"
style = '{'
style = "{"
sh = AppriseStreamHandler(color_logging)
formatter = logging.Formatter(format, date_format, style)
@@ -203,11 +203,11 @@ def setup_logging(verbosity: int, color_logging: bool = False, apprise_notifiers
"""
add_logging_level(
'EXTRA_DEBUG',
"EXTRA_DEBUG",
logging.DEBUG - 1,
)
add_logging_level(
'WEBSOCKET_DATA',
"WEBSOCKET_DATA",
logging.DEBUG - 2,
)
@@ -294,7 +294,7 @@ async def get_camera_name(protect: ProtectApiClient, id: str):
# Refresh cameras
logger.debug(f"Unknown camera id: '{id}', checking API")
await protect.update(force=True)
await protect.update()
try:
name = protect.bootstrap.cameras[id].name
@@ -337,9 +337,9 @@ async def run_command(cmd: str, data=None):
)
stdout, stderr = await proc.communicate(data)
stdout = stdout.decode()
stdout_indented = '\t' + stdout.replace('\n', '\n\t').strip()
stdout_indented = "\t" + stdout.replace("\n", "\n\t").strip()
stderr = stderr.decode()
stderr_indented = '\t' + stderr.replace('\n', '\n\t').strip()
stderr_indented = "\t" + stderr.replace("\n", "\n\t").strip()
if proc.returncode != 0:
logger.error(f"Failed to run: '{cmd}")
@@ -377,7 +377,7 @@ class VideoQueue(asyncio.Queue):
self._queue.append(item) # type: ignore
self._bytes_sum += len(item[1])
def full(self, item: tuple[Event, bytes] = None):
def full(self, item: tuple[Event, bytes] | None = None):
"""Return True if there are maxsize bytes in the queue.
optionally if `item` is provided, it will return False if there is enough space to