extern/apprise-api
1
0
Fork 0
mirror of https://github.com/caronc/apprise-api.git synced 2024-10-05 01:31:58 +02:00
Code Issues Packages Projects Releases Wiki Activity

Persistent Storage Support (Apprise v1.9.0+) Added

Browse Source
This commit is contained in:
Chris Caron 2024-09-02 22:11:30 -04:00
parent e50b2dbad9
commit f87f4c3471
15 changed files with 367 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
Dockerfile
View File

@ -47,17 +47,17 @@ FROM base AS runtime
COPY ./requirements.txt /etc/requirements.txt
COPY --from=builder /build/*.whl ./
RUN set -eux && \
echo "Installing cryptography" && \
pip3 install *.whl && \
echo "Installing python requirements" && \
pip3 install --no-cache-dir -q -r /etc/requirements.txt gunicorn supervisor && \
echo "Installing nginx" && \
apt-get update -qq && \
apt-get install -y -qq \
nginx && \
echo "Installing cryptography" && \
pip3 install *.whl && \
echo "Installing tools" && \
apt-get install -y -qq \
curl sed && \
curl sed git && \
echo "Installing python requirements" && \
pip3 install --no-cache-dir -q -r /etc/requirements.txt gunicorn supervisor && \
echo "Cleaning up" && \
apt-get --yes autoremove --purge && \
apt-get clean --yes && \

20
README.md
View File

@ -43,8 +43,10 @@ Using [dockerhub](https://hub.docker.com/r/caronc/apprise) you can do the follow
docker pull caronc/apprise:latest
# Start it up:
# /config is used for a persistent store, you do not have to mount
# /config/store is used for a persistent store, you do not have to mount
# this if you don't intend to use it.
# /config is used for a spot to write all of the configuration files
# generated through the API
# /plugin is used for a location you can add your own custom apprise plugins.
# You do not have to mount this if you don't intend to use it.
# /attach is used for file attachments
@ -90,6 +92,18 @@ docker run --name apprise \
-e APPRISE_WORKER_COUNT=1 \
-v /etc/apprise:/config \
-d apprise/local:latest
# Change your paths to what you want them to be, you may also wish to
# just do the following:
mkdir -p config store
docker run --name apprise \
-p 8000:8000 \
-e PUID=$(id -u) \
-e PGID=$(id -g) \
-e APPRISE_STATEFUL_MODE=simple \
-e APPRISE_WORKER_COUNT=1 \
-v ./config:/config \
-d apprise/local:latest
```
A `docker-compose.yml` file is already set up to grant you an instant production ready simulated environment:
@ -125,6 +139,7 @@ Below is a sample of just a simple text response:
# one or more of the following separated by a comma:
# - ATTACH_PERMISSION_ISSUE: Can not write attachments (likely a permission issue)
# - CONFIG_PERMISSION_ISSUE: Can not write configuration (likely a permission issue)
# - STORE_PERMISSION_ISSUE: Can not write to persistent storage (likely a permission issue)
curl -X GET http://localhost:8000/status
```
@ -383,6 +398,9 @@ The use of environment variables allow you to provide over-rides to default sett
| `APPRISE_DEFAULT_THEME` | Can be set to `light` or `dark`; it defaults to `light` if not otherwise provided. The theme can be toggled from within the website as well.
| `APPRISE_DEFAULT_CONFIG_ID` | Defaults to `apprise`. This is the presumed configuration ID you always default to when accessing the configuration manager via the website.
| `APPRISE_CONFIG_DIR` | Defines an (optional) persistent store location of all configuration files saved. By default:<br/> - Configuration is written to the `apprise_api/var/config` directory when just using the _Django_ `manage runserver` script. However for the path for the container is `/config`.
| `APPRISE_STORAGE_DIR` | Defines an (optional) persistent store location of all cache files saved. By default persistent storage is written into the `<APPRISE_CONFIG_DIR>/store`.
| `APPRISE_STORAGE_MODE` | Defines the storage mode to use. If no `APPRISE_STORGE_DIR` is identified, then this is set to `memory` in all circumtances reguardless what it might otherwise be set to. The possible options are:<br/>📌 **auto**: This is also the default. Writes cache files on demand only. <br/>📌 **memory**: Persistent storage is disabled; local memory is used for simple internal references. This is effectively the behavior of Apprise of versions 1.8.1 and earlier.<br/>📌 **flush**: A bit more i/o intensive then `auto`. Content is written to disk constantly if changed in anyway. This mode is still experimental.
| `APPRISE_STORAGE_UID_LENGTH` | Defines the unique key lengths used to identify an Apprise URL. By default this is set to `8`. Value can not be set to a smaller value then `2` or larger then `64`.
| `APPRISE_ATTACH_DIR` | The directory the uploaded attachments are placed in. By default:<br/> - Attachments are written to the `apprise_api/var/attach` directory when just using the _Django_ `manage runserver` script. However for the path for the container is `/attach`.
| `APPRISE_ATTACH_SIZE` | Over-ride the attachment size (defined in MB). By default it is set to `200` (Megabytes). You can set this up to a maximum value of `500` which is the restriction in place for NginX (internal hosting ervice) at this time. If you set this to zero (`0`) then attachments will not be passed along even if provided.
| `APPRISE_UPLOAD_MAX_MEMORY_SIZE` | Over-ride the in-memory accepted payload size (defined in MB). By default it is set to `3` (Megabytes). There is no reason the HTTP payload (excluding attachments) should exceed this limit. This value is only configurable for those who have edge cases where there are exceptions to this rule.

0
apprise_api/api/management/__init__.py Normal file
View File

0
apprise_api/api/management/commands/__init__.py Normal file
View File

45
apprise_api/api/management/commands/storeprune.py Normal file
View File

@ -0,0 +1,45 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2023 Chris Caron <lead2gold@gmail.com>
# All rights reserved.
#
# This code is licensed under the MIT License.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files(the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions :
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from django.core.management.base import BaseCommand
from django.conf import settings
import apprise
class Command(BaseCommand):
help = f"Prune all persistent content older then {settings.APPRISE_STORAGE_PRUNE_DAYS} days()"
def add_arguments(self, parser):
parser.add_argument("-d", "--days", type=int, default=settings.APPRISE_STORAGE_PRUNE_DAYS)
def handle(self, *args, **options):
# Persistent Storage cleanup
apprise.PersistentStore.disk_prune(
path=settings.APPRISE_STORAGE_DIR,
expires=options["days"] * 86400, action=True,
)
self.stdout.write(
self.style.SUCCESS('Successfully pruned persistent storeage (days: %d)' % options["days"])
)

8
apprise_api/api/templates/config.html
View File

@ -351,8 +351,14 @@ async function main_init(){
let code = document.createElement('code');
let li = document.createElement('li');
code.textContent = entry.url;
li.setAttribute('class', 'card-panel');
li.appendChild(code);
li.setAttribute('class', 'card-panel');
if (entry.id) {
let url_id = document.createElement('code');
url_id.setAttribute('class', 'url-id');
url_id.textContent = entry.id;
li.appendChild(url_id);
}
urlList.appendChild(li);
// Store `all` tag

35
apprise_api/api/tests/test_cli.py Normal file
View File

@ -0,0 +1,35 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2023 Chris Caron <lead2gold@gmail.com>
# All rights reserved.
#
# This code is licensed under the MIT License.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files(the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions :
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import io
from django.test import SimpleTestCase
from django.core import management
class CommandTests(SimpleTestCase):
def test_command_style(self):
out = io.StringIO()
management.call_command('storeprune', days=40, stdout=out)

56
apprise_api/api/tests/test_healthecheck.py
View File

@ -66,6 +66,7 @@ class HealthCheckTests(SimpleTestCase):
'config_lock': False,
'attach_lock': False,
'status': {
'persistent_storage': True,
'can_write_config': True,
'can_write_attach': True,
'details': ['OK']
@ -90,6 +91,7 @@ class HealthCheckTests(SimpleTestCase):
'config_lock': True,
'attach_lock': False,
'status': {
'persistent_storage': True,
'can_write_config': False,
'can_write_attach': True,
'details': ['OK']
@ -113,6 +115,7 @@ class HealthCheckTests(SimpleTestCase):
'config_lock': False,
'attach_lock': False,
'status': {
'persistent_storage': True,
'can_write_config': False,
'can_write_attach': True,
'details': ['OK']
@ -136,6 +139,7 @@ class HealthCheckTests(SimpleTestCase):
'config_lock': False,
'attach_lock': True,
'status': {
'persistent_storage': True,
'can_write_config': True,
'can_write_attach': False,
'details': ['OK']
@ -159,6 +163,7 @@ class HealthCheckTests(SimpleTestCase):
'config_lock': False,
'attach_lock': False,
'status': {
'persistent_storage': True,
'can_write_config': True,
'can_write_attach': True,
'details': ['OK']
@ -172,6 +177,7 @@ class HealthCheckTests(SimpleTestCase):
result = healthcheck(lazy=True)
assert result == {
'persistent_storage': True,
'can_write_config': True,
'can_write_attach': True,
'details': ['OK']
@ -180,6 +186,7 @@ class HealthCheckTests(SimpleTestCase):
# A Double lazy check
result = healthcheck(lazy=True)
assert result == {
'persistent_storage': True,
'can_write_config': True,
'can_write_attach': True,
'details': ['OK']
@ -192,16 +199,48 @@ class HealthCheckTests(SimpleTestCase):
# We still succeed; we just don't leverage our lazy check
# which prevents addition (unnessisary) writes
assert result == {
'persistent_storage': True,
'can_write_config': True,
'can_write_attach': True,
'details': ['OK'],
}
# Force a lazy check where we can't acquire the modify time
with mock.patch('os.path.getmtime') as mock_getmtime:
mock_getmtime.side_effect = OSError()
result = healthcheck(lazy=True)
# We still succeed; we just don't leverage our lazy check
# which prevents addition (unnessisary) writes
assert result == {
'persistent_storage': True,
'can_write_config': False,
'can_write_attach': False,
'details': [
'CONFIG_PERMISSION_ISSUE',
'ATTACH_PERMISSION_ISSUE',
]}
# Force a non-lazy check
with mock.patch('os.makedirs') as mock_makedirs:
mock_makedirs.side_effect = OSError()
result = healthcheck(lazy=False)
assert result == {
'persistent_storage': False,
'can_write_config': False,
'can_write_attach': False,
'details': [
'CONFIG_PERMISSION_ISSUE',
'ATTACH_PERMISSION_ISSUE',
'STORE_PERMISSION_ISSUE',
]}
with mock.patch('os.path.getmtime') as mock_getmtime:
with mock.patch('os.fdopen', side_effect=OSError()):
mock_getmtime.side_effect = OSError()
mock_makedirs.side_effect = None
result = healthcheck(lazy=False)
assert result == {
'persistent_storage': True,
'can_write_config': False,
'can_write_attach': False,
'details': [
@ -209,18 +248,31 @@ class HealthCheckTests(SimpleTestCase):
'ATTACH_PERMISSION_ISSUE',
]}
mock_makedirs.side_effect = (None, OSError())
with mock.patch('apprise.PersistentStore.flush', return_value=False):
result = healthcheck(lazy=False)
assert result == {
'persistent_storage': False,
'can_write_config': True,
'can_write_attach': True,
'details': [
'STORE_PERMISSION_ISSUE',
]}
mock_makedirs.side_effect = (OSError(), OSError(), None, None, None, None)
result = healthcheck(lazy=False)
assert result == {
'persistent_storage': True,
'can_write_config': False,
'can_write_attach': False,
'details': [
'CONFIG_PERMISSION_ISSUE',
'ATTACH_PERMISSION_ISSUE',
]}
mock_makedirs.side_effect = (OSError(), None)
mock_makedirs.side_effect = (OSError(), None, None, None, None)
result = healthcheck(lazy=False)
assert result == {
'persistent_storage': True,
'can_write_config': False,
'can_write_attach': True,
'details': [

63
apprise_api/api/tests/test_utils.py Normal file
View File

@ -0,0 +1,63 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2024 Chris Caron <lead2gold@gmail.com>
# All rights reserved.
#
# This code is licensed under the MIT License.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files(the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions :
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import os
import mock
import tempfile
from django.test import SimpleTestCase
from .. import utils
class UtilsTests(SimpleTestCase):
def test_touchdir(self):
"""
Test touchdir()
"""
with tempfile.TemporaryDirectory() as tmpdir:
with mock.patch('os.makedirs', side_effect=OSError()):
assert utils.touchdir(os.path.join(tmpdir, 'tmp-file')) is False
with mock.patch('os.makedirs', side_effect=FileExistsError()):
# Dir doesn't exist
assert utils.touchdir(os.path.join(tmpdir, 'tmp-file')) is False
assert utils.touchdir(os.path.join(tmpdir, 'tmp-file')) is True
# Date is updated
assert utils.touchdir(os.path.join(tmpdir, 'tmp-file')) is True
with mock.patch('os.utime', side_effect=OSError()):
# Fails to update file
assert utils.touchdir(os.path.join(tmpdir, 'tmp-file')) is False
def test_touch(self):
"""
Test touch()
"""
with tempfile.TemporaryDirectory() as tmpdir:
with mock.patch('os.fdopen', side_effect=OSError()):
assert utils.touch(os.path.join(tmpdir, 'tmp-file')) is False

60
apprise_api/api/utils.py
View File

@ -207,6 +207,26 @@ class HTTPAttachment(A_MGR['http']):
pass
def touchdir(path, mode=0o770, **kwargs):
"""
Acts like a Linux touch and updates a dir with a current timestamp
"""
try:
os.makedirs(path, mode=mode, exist_ok=False)
except FileExistsError:
# Update the mtime of the directory
try:
os.utime(path, None)
except OSError:
return False
except OSError:
return False
return True
def touch(fname, mode=0o666, dir_fd=None, **kwargs):
"""
Acts like a Linux touch and updates a file with a current timestamp
@ -778,6 +798,7 @@ def healthcheck(lazy=True):
# Some status variables we can flip
response = {
'persistent_storage': False,
'can_write_config': False,
'can_write_attach': False,
'details': [],
@ -853,6 +874,45 @@ def healthcheck(lazy=True):
# We can take an early exit
response['details'].append('ATTACH_PERMISSION_ISSUE')
if settings.APPRISE_STORAGE_DIR:
#
# Persistent Storage Check
#
store = apprise.PersistentStore(
path=settings.APPRISE_STORAGE_DIR,
namespace='tmp_hc',
mode=settings.APPRISE_STORAGE_MODE,
)
if store.mode != settings.APPRISE_STORAGE_MODE:
# Persistent storage not as configured
response['details'].append('STORE_PERMISSION_ISSUE')
elif store.mode != apprise.PersistentStoreMode.MEMORY:
# G
path = settings.APPRISE_STORAGE_DIR
if lazy:
try:
modify_date = datetime.fromtimestamp(os.path.getmtime(path))
delta = (datetime.now() - modify_date).total_seconds()
if delta <= 30.00: # 30s
response['persistent_storage'] = True
except OSError:
# No worries... continue with below testing
pass
if not (store.set('foo', 'bar') and store.flush()):
# No persistent store
response['details'].append('STORE_PERMISSION_ISSUE')
else:
# Toggle our status
response['persistent_storage'] = True
# Clear our test
store.clear('foo')
if not response['details']:
response['details'].append('OK')

22
apprise_api/api/views.py
View File

@ -992,10 +992,16 @@ class NotifyView(View):
status=status,
)
# Prepare our keyword arguments (to be passed into an AppriseAsset
# object)
# Prepare our keyword arguments (to be passed into an AppriseAsset object)
kwargs = {
# Load our dynamic plugin path
'plugin_paths': settings.APPRISE_PLUGIN_PATHS,
# Load our persistent storage path
'storage_path': settings.APPRISE_STORAGE_DIR,
# Our storage URL ID Length
'storage_idlen': settings.APPRISE_STORAGE_UID_LENGTH,
# Define if we flush to disk as soon as possible or not when required
'storage_mode': settings.APPRISE_STORAGE_MODE,
}
if body_format:
@ -1353,10 +1359,16 @@ class StatelessNotifyView(View):
'error': msg,
}, encoder=JSONEncoder, safe=False, status=status)
# Prepare our keyword arguments (to be passed into an AppriseAsset
# object)
# Prepare our keyword arguments (to be passed into an AppriseAsset object)
kwargs = {
# Load our dynamic plugin path
'plugin_paths': settings.APPRISE_PLUGIN_PATHS,
# Load our persistent storage path
'storage_path': settings.APPRISE_STORAGE_DIR,
# Our storage URL ID Length
'storage_idlen': settings.APPRISE_STORAGE_UID_LENGTH,
# Define if we flush to disk as soon as possible or not when required
'storage_mode': settings.APPRISE_STORAGE_MODE,
}
if body_format:
@ -1576,6 +1588,7 @@ class JsonUrlView(View):
# "tags": ["tag1', "tag2", "tag3"],
# "urls": [
# {
# "uid": "efa313ab",
# "url": "windows://",
# "tags": [],
# },
@ -1640,6 +1653,7 @@ class JsonUrlView(View):
for notification in a_obj.find(tag):
# Set Notification
response['urls'].append({
'id': notification.url_id(),
'url': notification.url(privacy=privacy),
'tags': notification.tags,
})

22
apprise_api/core/settings/__init__.py
View File

@ -143,6 +143,28 @@ APPRISE_WEBHOOK_URL = os.environ.get('APPRISE_WEBHOOK_URL', '')
APPRISE_CONFIG_DIR = os.environ.get(
'APPRISE_CONFIG_DIR', os.path.join(BASE_DIR, 'var', 'config'))
# The location to store Apprise Persistent Storage files
APPRISE_STORAGE_DIR = os.environ.get(
'APPRISE_STORAGE_DIR', os.path.join(APPRISE_CONFIG_DIR, 'store'))
# Default number of days to prune persistent storage
APPRISE_STORAGE_PRUNE_DAYS = \
int(os.environ.get('APPRISE_STORAGE_PRUNE_DAYS', 30))
# The default URL ID Length
APPRISE_STORAGE_UID_LENGTH = \
int(os.environ.get('APPRISE_STORAGE_UID_LENGTH', 8))
# The default storage mode; options are:
# - memory : Disables persistent storage (this is also automatically set
# if the APPRISE_STORAGE_DIR is empty reguardless of what is
# defined below.
# - auto : Writes to storage after each notifications execution (default)
# - flush : Writes to storage constantly (as much as possible). This
# produces more i/o but can allow multiple calls to the same
# notification to be in sync more
APPRISE_STORAGE_MODE = os.environ.get('APPRISE_STORAGE_MODE', 'auto').lower()
# The location to place file attachments
APPRISE_ATTACH_DIR = os.environ.get(
'APPRISE_ATTACH_DIR', os.path.join(BASE_DIR, 'var', 'attach'))

19
apprise_api/static/css/base.css
View File

@ -122,8 +122,9 @@ textarea {
border-radius: 12px;
width: 50%;
min-width: 35rem;
min-height: 12em;
min-height: 4em;
float: left;
position: relative;
}
.chip {
@ -134,6 +135,7 @@ textarea {
cursor: pointer;
}
#url-list code {
overflow-x: hidden;
overflow-y: hidden;
@ -141,9 +143,24 @@ textarea {
text-wrap: wrap;
overflow-wrap: break-word;
border-radius: 5px;
margin-top: 0.8rem;
display: block;
}
#url-list .card-panel .url-id {
width: auto;
margin: 0.3rem;
background-color: inherit;
color: #aaa;
padding: 0.2em;
font-size: 0.7em;
border: 0px;
/* Top Justified */
position: absolute;
top: -0.5em;
right: 0.3em;
}
/* Notification Details */
ul.logs {
font-family: monospace, monospace;

3
apprise_api/supervisord-startup
View File

@ -57,8 +57,9 @@ chmod o+w /dev/stdout /dev/stderr
[ ! -d /attach ] && mkdir -p /attach
chown -R $USER:$GROUP /attach
[ ! -d /config ] && mkdir -p /config
[ ! -d /config ] && mkdir -p /config /config/store
chown $USER:$GROUP /config
chown -R $USER:$GROUP /config/store
[ ! -d /plugin ] && mkdir -p /plugin
[ ! -d /run/apprise ] && mkdir -p /run/apprise

2
requirements.txt
View File

@ -10,7 +10,7 @@
# apprise @ git+https://github.com/caronc/apprise@custom-tag-or-version
## 3. The below grabs our stable version (generally the best choice):
apprise == 1.8.1
apprise == 1.9.0
## Apprise API Minimum Requirements
django