Merge branch 'master' into master

.travis.yml

@@ -1,5 +1,5 @@
 language: python
-dist: bionic # use LTS 18.04
+dist: focal # use LTS 20.04
 
 python:
     - "3.6"
@@ -7,7 +7,7 @@ python:
     - "3.8"
 
 env:
-    - DJANGO=2.2.8
+    - DJANGO=3.1.2
 
 install:
     - pip install -q Django==$DJANGO

demo/demodesk/config/settings.py

@@ -56,7 +56,7 @@ MIDDLEWARE = [
     'django.middleware.clickjacking.XFrameOptionsMiddleware',
 ]
 
-ROOT_URLCONF = 'demodesk.config.urls'
+ROOT_URLCONF = 'demo.demodesk.config.urls'
 
 TEMPLATES = [
     {
@@ -75,7 +75,7 @@ TEMPLATES = [
     },
 ]
 
-WSGI_APPLICATION = 'demodesk.config.wsgi.application'
+WSGI_APPLICATION = 'demo.demodesk.config.wsgi.application'
 
 
 # django-helpdesk configuration settings

docs/configuration.rst

@@ -1,12 +1,12 @@
 Configuration
 =============
 
+   **IMPORTANT NOTE**: Any tickets created via POP3 or IMAP mailboxes will DELETE the original e-mail from the mail server.
+
 Before django-helpdesk will be much use, you need to do some basic configuration. Most of this is done via the Django admin screens.
 
 1. Visit ``http://yoursite/admin/`` and add a Helpdesk Queue. If you wish, enter your POP3 or IMAP server details.
 
-   **IMPORTANT NOTE**: Any tickets created via POP3 or IMAP mailboxes will DELETE the original e-mail from the mail server.
-
 2. Visit ``http://yoursite/helpdesk/`` (or whatever path as defined in your ``urls.py``)
 
 3. If you wish to automatically create tickets from the contents of an e-mail inbox, set up a cronjob to run the management command on a regular basis. (Or use Celery, see below)
@@ -17,7 +17,7 @@ Before django-helpdesk will be much use, you need to do some basic configuration
 
    This will run the e-mail import every 5 minutes
 
-   **IMPORTANT NOTE**: Any tickets created via POP3 or IMAP mailboxes will DELETE the original e-mail from the mail server.
+   You will need to create a support queue, and associated login/host values, in the Django admin interface, in order for mail to be picked-up from the mail server and placed in the tickets table of your database. The values in the settings file alone, will not create the necessary values to trigger the get_email function.
 
  If you wish to use `celery` instead of cron, you must add 'django_celery_beat' to `INSTALLED_APPS` and add a periodic celery task through the Django admin.
 
@@ -46,3 +46,31 @@ Before django-helpdesk will be much use, you need to do some basic configuration
 8. If you wish to use SOCKS4/5 proxy with Helpdesk Queue email operations, install PySocks manually. Please note that mixing both SOCKS and non-SOCKS email sources for different queues is only supported under Python 2; on Python 3, SOCKS proxy support is all-or-nothing: either all queue email sources must use SOCKS or none may use it. If you need this functionality on Python 3 please `let us know <https://github.com/django-helpdesk/django-helpdesk/issues/new>`_.
 
 You're now up and running! Happy ticketing.
+
+Queue settings via admin interface
+----------------------------------
+Locale
+^^^^^^
+The *Locale* value must match the value in the ``locale`` column in the ``helpdesk_emailtemplate`` table if you wish to use those templates. For default installations/templates those values are ``cs``, ``de``, ``en``, ``es``, ``fi``, ``fr``, ``it``, ``pl``, ``ru`` and ``zh``.
+
+If you want to use a different *Local* then you will need to generate/edit the necessary templates (and set the value in the ``locale`` column) for those locales. This includes when using language variants, such as ``de-CH``, ``en-GB`` or ``fr-CA`` for example. 
+
+E-Mail Check Interval
+^^^^^^^^^^^^^^^^^^^^^
+This setting does not trigger e-mail collection, it merely throttles it. In order to trigger e-mail collection you must run a crontab to trigger ``manage.py get_email``. The setting in *E-Mail Check Interval* prevents your crontab from running the e-mail trigger more often than the interval set.
+
+For example, setting *E-Mail Check Interval* to ``5`` will limit the collection of e-mail to once every five minutes, even if your crontab is firing every five seconds. If your cron job is set to fire once every hour, then e-mail will only be collected once every hour.
+
+The cron job triggers the collection of e-mail, *E-Mail Check Interval* restricts how often the trigger is effective.
+
+To remove this limit, set *E-Mail Check Interval* to ``0``.
+
+Potential problems
+""""""""""""""""""
+There is potential for a timing clash to prevent triggering of mail collection if *E-Mail Check Interval* and your crontab interval are identical. Because the crontab runs fractionally before, or at exactly the same time as *E-Mail Check Interval* is run, if the timings for both are identical then every second call by the crontab will be ignored by *E-Mail Check Interval* because its interval has yet to expire.
+
+The result is that if both crontab and *E-Mail Check Interval* are set to run at five minute intervals, then mail may actually only be collected every ten minutes. You will see the evidence of this in the helpdesk mail log, or in the logs of your mail server.
+
+To avoid this problem set the crontab and *E-Mail Check Interval* to marginally different values (or set *E-Mail Check Interval* to ``0``). *E-Mail Check Interval* will only take an integer value, in minutes, so if you want a five minute interval between mail checks, then you will either have to set *E-Mail Check Interval* to ``4`` and the crontab interval to ``300 seconds``, or the *E-Mail Check Interval* to ``5`` and the crontab interval to ``305 seconds``.
+
+The crontab interval overrides the *E-Mail Check Interval*, and resets the *E-Mail Check Interval* each time it fires, as long as the crontab interval is greater than *E-Mail Check Interval*.

helpdesk/migrations/0018_fix_migrations.py

@@ -0,0 +1,58 @@
+# Generated by Django 2.2.16 on 2020-10-10 12:26
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('helpdesk', '0017_default_owner_on_delete_null'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='followup',
+            name='public',
+            field=models.BooleanField(blank=True, default=False, help_text='Public tickets are viewable by the submitter and all staff, but non-public tickets can only be seen by staff.', verbose_name='Public'),
+        ),
+        migrations.AlterField(
+            model_name='ignoreemail',
+            name='keep_in_mailbox',
+            field=models.BooleanField(blank=True, default=False, help_text='Do you want to save emails from this address in the mailbox? If this is unticked, emails from this address will be deleted.', verbose_name='Save Emails in Mailbox?'),
+        ),
+        migrations.AlterField(
+            model_name='queue',
+            name='allow_email_submission',
+            field=models.BooleanField(blank=True, default=False, help_text='Do you want to poll the e-mail box below for new tickets?', verbose_name='Allow E-Mail Submission?'),
+        ),
+        migrations.AlterField(
+            model_name='queue',
+            name='allow_public_submission',
+            field=models.BooleanField(blank=True, default=False, help_text='Should this queue be listed on the public submission form?', verbose_name='Allow Public Submission?'),
+        ),
+        migrations.AlterField(
+            model_name='queue',
+            name='email_box_ssl',
+            field=models.BooleanField(blank=True, default=False, help_text='Whether to use SSL for IMAP or POP3 - the default ports when using SSL are 993 for IMAP and 995 for POP3.', verbose_name='Use SSL for E-Mail?'),
+        ),
+        migrations.AlterField(
+            model_name='savedsearch',
+            name='shared',
+            field=models.BooleanField(blank=True, default=False, help_text='Should other users see this query?', verbose_name='Shared With Other Users?'),
+        ),
+        migrations.AlterField(
+            model_name='ticket',
+            name='on_hold',
+            field=models.BooleanField(blank=True, default=False, help_text='If a ticket is on hold, it will not automatically be escalated.', verbose_name='On Hold'),
+        ),
+        migrations.AlterField(
+            model_name='ticketcc',
+            name='can_update',
+            field=models.BooleanField(blank=True, default=False, help_text='Can this CC login and update the ticket?', verbose_name='Can Update Ticket?'),
+        ),
+        migrations.AlterField(
+            model_name='ticketcc',
+            name='can_view',
+            field=models.BooleanField(blank=True, default=False, help_text='Can this CC login to view the ticket details?', verbose_name='Can View Ticket?'),
+        ),
+    ]

helpdesk/migrations/0018_ticket_secret_key.py

@@ -1,27 +0,0 @@
-# Generated by Django 2.0.1 on 2018-09-07 21:22
-from django.db import migrations, models
-import helpdesk.models
-
-
-def clear_secret_keys(apps, schema_editor):
-    Ticket = apps.get_model("helpdesk", "Ticket")
-    db_alias = schema_editor.connection.alias
-
-    for ticket in Ticket.objects.using(db_alias).all():
-         ticket.secret_key=''
-         ticket.save()
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('helpdesk', '0017_default_owner_on_delete_null'),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name='ticket',
-            name='secret_key',
-            field=models.CharField(default=helpdesk.models.mk_secret, max_length=36, null=True, verbose_name='Secret key needed for viewing/editing ticket by non-logged in users'),
-        ),
-        migrations.RunPython(clear_secret_keys),
-    ]

helpdesk/migrations/0019_ticket_secret_key.py

@@ -3,16 +3,25 @@ from django.db import migrations, models
 import helpdesk.models
 
 
+def clear_secret_keys(apps, schema_editor):
+    Ticket = apps.get_model("helpdesk", "Ticket")
+    db_alias = schema_editor.connection.alias
+
+    for ticket in Ticket.objects.using(db_alias).all():
+         ticket.secret_key=''
+         ticket.save()
+
 class Migration(migrations.Migration):
 
     dependencies = [
-        ('helpdesk', '0018_ticket_secret_key'),
+        ('helpdesk', '0018_fix_migrations'),
     ]
 
     operations = [
-        migrations.AlterField(
+        migrations.AddField(
             model_name='ticket',
             name='secret_key',
             field=models.CharField(default=helpdesk.models.mk_secret, max_length=36, verbose_name='Secret key needed for viewing/editing ticket by non-logged in users'),
         ),
+        migrations.RunPython(clear_secret_keys),
     ]

helpdesk/models.py

@@ -752,7 +752,7 @@ class Ticket(models.Model):
         User = get_user_model()
         try:
             return User.objects.get(email=self.submitter_email)
-        except User.DoesNotExist:
+        except (User.DoesNotExist, User.MultipleObjectsReturned):
             return None
 
     class Meta:

helpdesk/query.py

@@ -219,5 +219,4 @@ class __Query__:
             'hour': date.hour,
             'minute': date.minute,
             'second': date.second,
-            'second': date.second,
         }

helpdesk/serializers.py

@@ -32,28 +32,30 @@ class DatatablesTicketSerializer(serializers.ModelSerializer):
                   'time_spent', 'kbitem')
 
     def get_queue(self, obj):
-        return ({"title": obj.queue.title, "id": obj.queue.id})
+        return {"title": obj.queue.title, "id": obj.queue.id}
 
     def get_ticket(self, obj):
-        return (str(obj.id) + " " + obj.ticket)
+        return str(obj.id) + " " + obj.ticket
 
     def get_status(self, obj):
-        return (obj.get_status)
+        return obj.get_status
 
     def get_created(self, obj):
-        return (humanize.naturaltime(obj.created))
+        return humanize.naturaltime(obj.created)
 
     def get_due_date(self, obj):
-        return (humanize.naturaltime(obj.due_date))
+        return humanize.naturaltime(obj.due_date)
 
     def get_assigned_to(self, obj):
         if obj.assigned_to:
             if obj.assigned_to.get_full_name():
-                return (obj.assigned_to.get_full_name())
+                return obj.assigned_to.get_full_name()
+            elif obj.assigned_to.email:
+                return obj.assigned_to.email
             else:
-                return (obj.assigned_to.email)
+                return obj.assigned_to.username
         else:
-            return ("None")
+            return "None"
 
     def get_submitter(self, obj):
         return obj.submitter_email
@@ -62,7 +64,7 @@ class DatatablesTicketSerializer(serializers.ModelSerializer):
         return format_time_spent(obj.time_spent)
 
     def get_row_class(self, obj):
-        return (obj.get_priority_css_class)
+        return obj.get_priority_css_class
 
     def get_kbitem(self, obj):
         return obj.kbitem.title if obj.kbitem else ""

helpdesk/templates/helpdesk/report_output.html

@@ -60,6 +60,9 @@
                         {% for f in d %}<td class='report'>{{ f }}</td>{% endfor %}
                     </tr>
                     {% endfor %}
+                    <tr>
+                        {% for f in total_data %}<td class='report'>{{ f }}</td>{% endfor %}
+                    </tr>
                 </tbody>
             </table>
         </div>

helpdesk/views/staff.py

@@ -9,7 +9,6 @@ views/staff.py - The bulk of the application - provides most business logic and
 from copy import deepcopy
 import json
 
-from django import VERSION as DJANGO_VERSION
 from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.decorators import user_passes_test
@@ -20,16 +19,13 @@ from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger
 from django.db.models import Q
 from django.http import HttpResponseRedirect, Http404, HttpResponse, JsonResponse
 from django.shortcuts import render, get_object_or_404
-from django.utils.dates import MONTHS_3
 from django.utils.translation import ugettext as _
 from django.utils.html import escape
-from django import forms
 from django.utils import timezone
 from django.views.generic.edit import FormView, UpdateView
 
 from helpdesk.query import (
     get_query_class,
-    query_to_dict,
     query_to_base64,
     query_from_base64,
 )
@@ -44,12 +40,11 @@ from helpdesk.forms import (
     TicketForm, UserSettingsForm, EmailIgnoreForm, EditTicketForm, TicketCCForm,
     TicketCCEmailForm, TicketCCUserForm, EditFollowUpForm, TicketDependencyForm
 )
-from helpdesk.decorators import staff_member_required, superuser_required
+from helpdesk.decorators import superuser_required
 from helpdesk.lib import (
     safe_template_context,
     process_attachments,
     queue_template_context,
-    format_time_spent,
 )
 from helpdesk.models import (
     Ticket, Queue, FollowUp, TicketChange, PreSetReply, FollowUpAttachment, SavedSearch,
@@ -60,8 +55,7 @@ import helpdesk.views.abstract_views as abstract_views
 from helpdesk.views.permissions import MustBeStaffMixin
 from ..lib import format_time_spent
 
-from rest_framework import viewsets, status
+from rest_framework import status
-from rest_framework.response import Response
 from rest_framework.decorators import api_view
 
 from datetime import date, datetime, timedelta
@@ -104,6 +98,14 @@ def dashboard(request):
     showing ticket counts by queue/status, and a list of unassigned tickets
     with options for them to 'Take' ownership of said tickets.
     """
+    # user settings num tickets per page
+    tickets_per_page = request.user.usersettings_helpdesk.tickets_per_page or 25
+
+    # page vars for the three ticket tables
+    user_tickets_page = request.GET.get(_('ut_page'), 1)
+    user_tickets_closed_resolved_page = request.GET.get(_('utcr_page'), 1)
+    all_tickets_reported_by_current_user_page = request.GET.get(_('atrbcu_page'), 1)
+
     huser = HelpdeskUser(request.user)
     active_tickets = Ticket.objects.select_related('queue').exclude(
         status__in=[Ticket.CLOSED_STATUS, Ticket.RESOLVED_STATUS],
@@ -447,7 +449,7 @@ def subscribe_to_ticket_updates(ticket, user=None, email=None, can_view=True, ca
         return ticketcc
 
 
-def subscribe_staff_member_to_ticket(ticket, user, email=''):
+def subscribe_staff_member_to_ticket(ticket, user, email='', can_view=True, can_update=False):
     """used in view_ticket() and update_ticket()"""
     return subscribe_to_ticket_updates(ticket=ticket, user=user, email=email, can_view=can_view, can_update=can_update)
 
@@ -827,7 +829,7 @@ def mass_update(request):
                 'submitter': ('closed_submitter', context),
                 'ticket_cc': ('closed_cc', context),
             }
-            if ticket.assigned_to and ticket.assigned_to.usersettings_helpdesk.email_on_ticket_change:
+            if t.assigned_to and t.assigned_to.usersettings_helpdesk.email_on_ticket_change:
                 roles['assigned_to'] = ('closed_owner', context),
 
             messages_sent_to.update(t.send(
@@ -1009,10 +1011,10 @@ def load_saved_query(request, query_params=None):
 
     if request.GET.get('saved_query', None):
         try:
-            saved_query = SavedSearch.objects.get(pk=request.GET.get('saved_query'))
+            saved_query = SavedSearch.objects.get(
-        except SavedSearch.DoesNotExist:
+                Q(pk=request.GET.get('saved_query')) & (Q(shared=True) | Q(user=request.user))
-            raise QueryLoadError()
+            )
-        if not (saved_query.shared or saved_query.user == request.user):
+        except (SavedSearch.DoesNotExist, ValueError):
             raise QueryLoadError()
 
         try:
@@ -1224,9 +1226,6 @@ def run_report(request, report):
     # a second table for more complex queries
     summarytable2 = defaultdict(int)
 
-    def month_name(m):
-        MONTHS_3[m].title()
-
     first_ticket = Ticket.objects.all().order_by('created')[0]
     first_month = first_ticket.created.month
     first_year = first_ticket.created.year
@@ -1349,11 +1348,17 @@ def run_report(request, report):
 
     column_headings = [col1heading] + possible_options
 
+    # Prepare a dict to store totals for each possible option
+    totals = {}
     # Pivot the data so that 'header1' fields are always first column
     # in the row, and 'possible_options' are always the 2nd - nth columns.
     for item in header1:
         data = []
         for hdr in possible_options:
+            if hdr not in totals.keys():
+                totals[hdr] = summarytable[item, hdr]
+            else:
+                totals[hdr] += summarytable[item, hdr]
             data.append(summarytable[item, hdr])
         table.append([item] + data)
 
@@ -1372,10 +1377,16 @@ def run_report(request, report):
     for series in table:
         series_names.append(series[0])
 
+    # Add total row to table
+    total_data = ['Total']
+    for hdr in possible_options:
+        total_data.append(str(totals[hdr]))
+
     return render(request, 'helpdesk/report_output.html', {
         'title': title,
         'charttype': charttype,
         'data': table,
+        'total_data': total_data,
         'headings': column_headings,
         'series_names': series_names,
         'morrisjs_data': morrisjs_data,