yo/pymailq
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

First commit

Browse Source
This commit is contained in:
yo 2020-12-01 17:43:04 +01:00
commit 243c04c51b
10 changed files with 2586 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
*.pyc
__pycache__

339
LICENSE Normal file
View File

@ -0,0 +1,339 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc., <http://fsf.org/>
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
{description}
Copyright (C) {year} {fullname}
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
{signature of Ty Coon}, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.

180
README.rst Normal file
View File

@ -0,0 +1,180 @@
|PythonPIP|_ |PythonSupport|_ |License|_ |Codacy|_ |Coverage|_ |RTFD|_ |Travis|_
pymailq - Simple Postfix queue management
=========================================
| **Contact:** Denis 'jawa' Pompilio <denis.pompilio@gmail.com>
| **Sources:** https://github.com/outini/pymailq/
|
| A full content documentation, is online at https://pymailq.readthedocs.io/en/latest/
|
| The pymailq module makes it easy to view and control Postfix mails queue. It
| provide several classes to store, view and interact with mail queue using
| Postfix command line tools. This module is provided for automation and
| monitoring developments.
|
| This project also provides a shell-like to interact with Postfix mails queue.
| It provide simple means to view the queue content, filter mails on criterias
| like Sender or delivery errors and lead administrative operations.
Installation
------------
Install pymailq module from https://pypi.python.org::
pip install pymailq
Install pymailq module from sources::
python setup.py install
A SPEC file is also provided for RPM builds (currently tested only on Fedora),
thanks to Nils Ratusznik (https://github.com/ahpnils). Debian binary packages
are also available.
Requirements
------------
This module actually support the following Python versions:
* *Python 2.7*
* *Python 3+*
A shell is provided for interactive administration. Based on Python *cmd*
module, using Python compiled with *readline* support is highly recommended
to access shell's full features.
Using the shell
---------------
Mails queue summary::
~$ pqshell --summary
====================== Mail queue summary ========================
Total mails in queue: 1773
Total queue size: 40.2 MB
Mails by accepted date:
last 24h: 939
1 to 4 days ago: 326
older than 4 days: 173
----- Mails by status ---------- ----- Mails by size ----------
Active 2 Average size 23.239 KB
Hold 896 Maximum size 1305.029 KB
Deferred 875 Minimum size 0.517 KB
----- Unique senders ----------- ----- Unique recipients ------
Senders 156 Recipients 1003
Domains 141 Domains 240
----- Top senders ------------------------------------------------
228 sender-3@domain-1.tld
195 sender-1@domain-4.tld
116 MAILER-DAEMON
105 sender-2@domain-2.tld
61 sender-7@domain-3.tld
----- Top sender domains -----------------------------------------
228 domain-1.tld
195 domain-4.tld
105 domain-2.tld
75 domain-0.tld
61 domain-3.tld
----- Top recipients ---------------------------------------------
29 user-1@domain-5.tld
28 user-5@domain-9.tld
23 user-2@domain-8.tld
20 user-3@domain-6.tld
20 user-4@domain-7.tld
----- Top recipient domains --------------------------------------
697 domain-7.tld
455 domain-5.tld
37 domain-6.tld
37 domain-9.tld
34 domain-8.tld
Using the shell in interactive mode::
~$ pqshell
Welcome to PyMailq shell.
PyMailq (sel:0)> store load
500 mails loaded from queue
PyMailq (sel:500)> show selected limit 5
2017-09-02 17:54:34 B04C91183774 [deferred] sender-6@test-domain.tld (425B)
2017-09-02 17:54:34 B21D71183681 [deferred] sender-2@test-domain.tld (435B)
2017-09-02 17:54:34 B422D11836AB [deferred] sender-7@test-domain.tld (2416B)
2017-09-02 17:54:34 B21631183753 [deferred] sender-6@test-domain.tld (425B)
2017-09-02 17:54:34 F2A7E1183789 [deferred] sender-2@test-domain.tld (2416B)
...Preview of first 5 (495 more)...
PyMailq (sel:500)> show selected limit 5 long
2017-09-02 17:54:34 B04C91183774 [deferred] sender-6@test-domain.tld (425B)
Rcpt: user-3@test-domain.tld
Err: Test error message
2017-09-02 17:54:34 B21D71183681 [deferred] sender-2@test-domain.tld (435B)
Rcpt: user-3@test-domain.tld
Err: Test error message
2017-09-02 17:54:34 B422D11836AB [deferred] sender-7@test-domain.tld (2416B)
Rcpt: user-2@test-domain.tld
Err: mail transport unavailable
2017-09-02 17:54:34 B21631183753 [deferred] sender-6@test-domain.tld (425B)
Rcpt: user-3@test-domain.tld
Err: mail transport unavailable
2017-09-02 17:54:34 F2A7E1183789 [deferred] sender-2@test-domain.tld (2416B)
Rcpt: user-1@test-domain.tld
Err: mail transport unavailable
...Preview of first 5 (495 more)...
PyMailq (sel:500)> select error "Test error message"
PyMailq (sel:16)> show selected rankby sender
sender count
================================================
sender-2@test-domain.tld 7
sender-4@test-domain.tld 3
sender-6@test-domain.tld 2
sender-5@test-domain.tld 1
sender-8@test-domain.tld 1
sender-3@test-domain.tld 1
sender-1@test-domain.tld 1
PyMailq (sel:16)> select sender sender-2@test-domain.tld
PyMailq (sel:7)> super hold
postsuper: Placed on hold: 7 messages
PyMailq (sel:7)> select reset
Selector resetted with store content (500 mails)
PyMailq (sel:500)> show selected rankby status
status count
================================================
deferred 493
hold 7
PyMailq (sel:500)> exit
Exiting shell... Bye.
Packaging
---------
Binary packages for some linux distribution are available. See the *packaging*
directory for more information.
License
-------
"GNU GENERAL PUBLIC LICENSE" (Version 2) *(see LICENSE file)*
.. |PythonPIP| image:: https://img.shields.io/pypi/v/pymailq.svg
.. _PythonPIP: https://pypi.python.org/pypi/pymailq/
.. |PythonSupport| image:: https://img.shields.io/badge/python-2.7,%203.4,%203.5,%203.6-blue.svg
.. _PythonSupport: https://github.com/outini/pymailq/
.. |License| image:: https://img.shields.io/badge/license-GPLv2-blue.svg
.. _License: https://github.com/outini/pymailq/
.. |Codacy| image:: https://api.codacy.com/project/badge/Grade/8444a0f124fe463d86a91d80a2a52e7c
.. _Codacy: https://www.codacy.com/app/outini/pymailq
.. |Coverage| image:: https://api.codacy.com/project/badge/Coverage/8444a0f124fe463d86a91d80a2a52e7c
.. _Coverage: https://www.codacy.com/app/outini/pymailq
.. |RTFD| image:: https://readthedocs.org/projects/pymailq/badge/?version=latest
.. _RTFD: http://pymailq.readthedocs.io/en/latest/?badge=latest
.. |Travis| image:: https://travis-ci.org/outini/pymailq.svg?branch=master
.. _Travis: https://travis-ci.org/outini/pymailq

1
VERSION Normal file
View File

@ -0,0 +1 @@
0.9.0

116
__init__.py Normal file
View File

@ -0,0 +1,116 @@
# coding: utf-8
#
# Postfix queue control python tool (pymailq)
#
# Copyright (C) 2014 Denis Pompilio (jawa) <denis.pompilio@gmail.com>
#
# This file is part of pymailq
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, see <http://www.gnu.org/licenses/>.
import sys
import shlex
from functools import wraps
from datetime import datetime
try:
import configparser
except ImportError:
import ConfigParser as configparser
#: Boolean to control activation of the :func:`debug` decorator.
DEBUG = False
#: Current version of the package as :class:`str`.
VERSION = "0.9.0"
#: Module configuration as :class:`dict`.
CONFIG = {
"core": {
"postfix_spool": "/var/spool/postfix"
},
"commands": {
"use_sudo": False,
"list_queue": ["mailq"],
"cat_message": ["postcat", "-qv"],
"hold_message": ["postsuper", "-h"],
"release_message": ["postsuper", "-H"],
"requeue_message": ["postsuper", "-r"],
"delete_message": ["postsuper", "-d"]
}
}
def debug(function):
"""
Decorator to print some call informations and timing debug on stderr.
Function's name, passed args and kwargs are printed to stderr. Elapsed time
is also print at the end of call. This decorator is based on the value of
:data:`DEBUG`. If ``True``, the debug informations will be displayed.
"""
@wraps(function)
def run(*args, **kwargs):
name = function.__name__
if DEBUG is True:
sys.stderr.write("[DEBUG] Running {0}\n".format(name))
sys.stderr.write("[DEBUG] args: {0}\n".format(args))
sys.stderr.write("[DEBUG] kwargs: {0}\n".format(kwargs))
start = datetime.now()
ret = function(*args, **kwargs)
if DEBUG is True:
stop = datetime.now()
sys.stderr.write("[DEBUG] Exectime of {0}: {1} seconds\n".format(
name, (stop - start).total_seconds()))
return ret
return run
def load_config(cfg_file):
"""
Load module configuration from .ini file
Information from this file are directly used to override values stored in
:attr:`pymailq.CONFIG`.
Commands from configuration file are treated using :func:`shlex.split` to
properly transform command string to list of arguments.
:param str cfg_file: Configuration file
.. seealso::
:ref:`pymailq-configuration`
"""
global CONFIG
cfg = configparser.ConfigParser()
cfg.read(cfg_file)
if "core" in cfg.sections():
if cfg.has_option("core", "postfix_spool"):
CONFIG["core"]["postfix_spool"] = cfg.get("core", "postfix_spool")
if "commands" in cfg.sections():
for key in cfg.options("commands"):
if key == "use_sudo":
if cfg.get("commands", key) == "yes":
CONFIG["commands"]["use_sudo"] = True
else:
command = shlex.split(cfg.get("commands", key))
CONFIG["commands"][key] = command

172
control.py Normal file
View File

@ -0,0 +1,172 @@
# coding: utf-8
#
# Postfix queue control python tool (pymailq)
#
# Copyright (C) 2014 Denis Pompilio (jawa) <denis.pompilio@gmail.com>
#
# This file is part of pymailq
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, see <http://www.gnu.org/licenses/>.
import time
import subprocess
from pymailq import CONFIG, debug
class QueueControl(object):
"""
Postfix queue control using postsuper command.
The :class:`~control.QueueControl` instance defines the following
attributes:
.. attribute:: use_sudo
Boolean to control the use of `sudo` to invoke Postfix command.
Default is ``False``
.. attribute:: postsuper_cmd
Postfix command and arguments :func:`list` for mails queue
administrative operations. Default is ``["postsuper"]``
.. attribute:: known_operations
Known Postfix administrative operations :class:`dict` to associate
operations to command arguments. Known associations are::
delete: -d
hold: -h
release: -H
requeue: -r
.. warning::
Default known associations are provided for the default mails
queue administrative command `postsuper`_.
.. seealso::
Postfix manual:
`postsuper`_ -- Postfix superintendent
"""
@staticmethod
def get_operation_cmd(operation):
"""Get operation related command from configuration
This method use Postfix administrative commands defined
in :attr:`pymailq.CONFIG` attribute under the key 'list_queue'.
Command and arguments list is build on call with the configuration data.
Command keys are built with the ``operation`` argument suffixed with
``_message``. Example: ``hold_message`` for the hold command.
:param str operation: Operation name
:return: Command and arguments as :class:`list`
:raise KeyError: Operation is unknown
.. seealso::
:ref:`pymailq-configuration`
"""
cmd = CONFIG['commands'][operation + '_message']
if CONFIG['commands']['use_sudo']:
cmd.insert(0, 'sudo')
return cmd
@debug
def _operate(self, operation, messages):
"""
Generic method to lead operations messages from postfix mail queue.
Operations can be one of Postfix known operations stored in
PyMailq module configuration.
:param str operation: Known operation from :attr:`pymailq.CONFIG`.
:param list messages: List of :class:`~store.Mail` objects targetted
for operation.
:return: Command's *stderr* output lines
:rtype: :func:`list`
"""
# validate that object's attribute "qid" exist. Raise AttributeError.
for msg in messages:
getattr(msg, "qid")
# We may modify this part to improve security.
# It should not be possible to inject commands, but who knows...
# https://www.kevinlondon.com/2015/07/26/dangerous-python-functions.html
# And consider the use of sh module: https://amoffat.github.io/sh/
operation_cmd = self.get_operation_cmd(operation) + ['-']
try:
child = subprocess.Popen(operation_cmd,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE)
except EnvironmentError as exc:
command_str = " ".join(operation_cmd)
error_msg = "Unable to call '%s': %s" % (command_str, str(exc))
raise RuntimeError(error_msg)
# If permissions error, the postsuper process takes ~1s to teardown.
# Wait this delay and raise error message if process has stopped.
time.sleep(1.1)
child.poll()
if child.returncode:
raise RuntimeError(child.communicate()[1].strip().decode())
try:
for msg in messages:
child.stdin.write((msg.qid+'\n').encode())
stderr = child.communicate()[1].strip()
except BrokenPipeError:
raise RuntimeError("Unexpected error: child process has crashed")
return [line.strip() for line in stderr.decode().split('\n')]
def delete_messages(self, messages):
"""
Delete several messages from postfix mail queue.
This method is a :func:`~functools.partial` wrapper on
:meth:`~control.QueueControl._operate`. Passed operation is ``delete``
"""
return self._operate('delete', messages)
def hold_messages(self, messages):
"""
Hold several messages from postfix mail queue.
This method is a :func:`~functools.partial` wrapper on
:meth:`~control.QueueControl._operate`. Passed operation is ``hold``
"""
return self._operate('hold', messages)
def release_messages(self, messages):
"""
Release several messages from postfix mail queue.
This method is a :func:`~functools.partial` wrapper on
:meth:`~control.QueueControl._operate`. Passed operation is ``release``
"""
return self._operate('release', messages)
def requeue_messages(self, messages):
"""
Requeue several messages from postfix mail queue.
This method is a :func:`~functools.partial` wrapper on
:meth:`~control.QueueControl._operate`. Passed operation is ``requeue``
"""
return self._operate('requeue', messages)

315
selector.py Normal file
View File

@ -0,0 +1,315 @@
# coding: utf-8
#
# Postfix queue control python tool (pymailq)
#
# Copyright (C) 2014 Denis Pompilio (jawa) <denis.pompilio@gmail.com>
#
# This file is part of pymailq
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, see <http://www.gnu.org/licenses/>.
import gc
from functools import wraps
from datetime import datetime
from pymailq import debug
class MailSelector(object):
"""
Mail selector class to request mails from store matching criterias.
The :class:`~selector.MailSelector` instance provides the following
attributes:
.. attribute:: mails
Currently selected :class:`~store.Mail` objects :func:`list`
.. attribute:: store
Linked :class:`~store.PostqueueStore` at the
:class:`~selector.MailSelector` instance initialization.
.. attribute:: filters
Applied filters :func:`list` on current selection. Filters list
entries are tuples containing ``(function.__name__, args, kwargs)``
for each applied filters. This list is filled by the
:meth:`~selector.MailSelector.filter_registration` decorator while
calling filtering methods. It is possible to replay registered
filter using :meth:`~selector.MailSelector.replay_filters` method.
"""
def __init__(self, store):
"""Init method"""
self.mails = []
self.store = store
self.filters = []
self.reset()
def filter_registration(function):
"""
Decorator to register applied filter.
This decorated is used to wrap selection methods ``lookup_*``. It
registers a ``(function.__name__, args, kwargs)`` :func:`tuple` in
the :attr:`~MailSelector.filters` attribute.
"""
@wraps(function)
def wrapper(self, *args, **kwargs):
filterinfo = (function.__name__, args, kwargs)
self.filters.append(filterinfo)
return function(self, *args, **kwargs)
return wrapper
def reset(self):
"""
Reset mail selector with initial store mails list.
Selected :class:`~store.Mail` objects are deleted and the
:attr:`~MailSelector.mails` attribute is removed for memory releasing
purpose (with help of :func:`gc.collect`). Attribute
:attr:`~MailSelector.mails` is then reinitialized a copy of
:attr:`~MailSelector.store`'s :attr:`~PostqueueStore.mails` attribute.
Registered :attr:`~MailSelector.filters` are also emptied.
"""
del self.mails
gc.collect()
self.mails = [mail for mail in self.store.mails]
self.filters = []
def replay_filters(self):
"""
Reset selection with store content and replay registered filters.
Like with the :meth:`~selector.MailSelector.reset` method, selected
:class:`~store.Mail` objects are deleted and reinitialized with a copy
of :attr:`~MailSelector.store`'s :attr:`~PostqueueStore.mails`
attribute.
However, registered :attr:`~MailSelector.filters` are kept and replayed
on resetted selection. Use this method to refresh your store content
while keeping your filters.
"""
del self.mails
gc.collect()
self.mails = [mail for mail in self.store.mails]
filters = [entry for entry in self.filters]
for filterinfo in filters:
name, args, kwargs = filterinfo
getattr(self, name)(*args, **kwargs)
self.filters = filters
def get_mails_by_qids(self, qids):
"""
Get mails with specified IDs.
This function is not registered as filter.
:param list qids: List of mail IDs.
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
return [mail for mail in self.mails
if mail.qid in qids]
@debug
@filter_registration
def lookup_qids(self, qids):
"""
Lookup mails with specified IDs.
:param list qids: List of mail IDs.
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
self.mails = self.get_mails_by_qids(qids)
return self.mails
@debug
@filter_registration
def lookup_header(self, header, value, exact=True):
"""
Lookup mail headers with specified value.
:param str header: Header name to filter on.
:param str value: Header value to filter on.
:param bool exact: Allow lookup with partial or exact match
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
matches = []
for mail in self.mails:
header_value = getattr(mail.head, header, None)
if not header_value:
continue
if not isinstance(header_value, list):
header_value = [header_value]
if exact and value in header_value:
matches.append(mail)
elif not exact:
for entry in header_value:
if value in entry:
matches.append(mail)
break
self.mails = matches
return self.mails
@debug
@filter_registration
def lookup_status(self, status):
"""
Lookup mails with specified postqueue status.
:param list status: List of matching status to filter on.
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
self.mails = [mail for mail in self.mails
if mail.status in status]
return self.mails
@debug
@filter_registration
def lookup_sender(self, sender, exact=True):
"""
Lookup mails send from a specific sender.
Optionnal parameter ``partial`` allow lookup of partial sender like
``@domain.com`` or ``sender@``. By default, ``partial`` is ``False``
and selection is made on exact sender.
.. note::
Matches are made against :attr:`Mail.sender` attribute instead of
real mail header :mailheader:`Sender`.
:param str sender: Sender address to lookup in :class:`~store.Mail`
objects selection.
:param bool exact: Allow lookup with partial or exact match
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
if exact is False:
self.mails = [mail for mail in self.mails
if sender in mail.sender]
else:
self.mails = [mail for mail in self.mails
if sender == mail.sender]
return self.mails
@debug
@filter_registration
def lookup_recipient(self, recipient, exact=True):
"""
Lookup mails send to a specific recipient.
Optionnal parameter ``partial`` allow lookup of partial sender like
``@domain.com`` or ``sender@``. By default, ``partial`` is ``False``
and selection is made on exact sender.
.. note::
Matches are made against :attr:`Mail.recipients` attribute instead
of real mail header :mailheader:`To`.
:param str recipient: Recipient address to lookup in
:class:`~store.Mail` objects selection.
:param bool exact: Allow lookup with partial or exact match
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
if exact is False:
selected = []
for mail in self.mails:
for value in mail.recipients:
if recipient in value:
selected += [mail]
self.mails = selected
else:
self.mails = [mail for mail in self.mails
if recipient in mail.recipients]
return self.mails
@debug
@filter_registration
def lookup_error(self, error_msg):
"""
Lookup mails with specific error message (message may be partial).
:param str error_msg: Error message to filter on
:return: List of newly selected :class:`~store.Mail` objects`
:rtype: :func:`list`
"""
self.mails = [mail for mail in self.mails
if True in [True for err in mail.errors
if error_msg in err]]
return self.mails
@debug
@filter_registration
def lookup_date(self, start=None, stop=None):
"""
Lookup mails send on specific date range(s).
:param datetime.date start: Start date (Default: None)
:param datetime.date stop: Stop date (Default: None)
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
if start is None:
start = datetime(1970, 1, 1)
if stop is None:
stop = datetime.now()
self.mails = [mail for mail in self.mails
if start <= mail.date <= stop]
return self.mails
@debug
@filter_registration
def lookup_size(self, smin=0, smax=0): # TODO: documentation
"""
Lookup mails send with specific size.
Both arguments ``smin`` and ``smax`` are optionnal and default is set
to ``0``. Maximum size is ignored if setted to ``0``. If both ``smin``
and ``smax`` are setted to ``0``, no filtering is done and the entire
:class:`~store.Mail` objects selection is returned.
:param int smin: Minimum size (Default: ``0``)
:param int smax: Maximum size (Default: ``0``)
:return: List of newly selected :class:`~store.Mail` objects
:rtype: :func:`list`
"""
if smin == 0 and smax == 0:
return self.mails
if smax > 0:
self.mails = [mail for mail in self.mails if mail.size <= smax]
self.mails = [mail for mail in self.mails if mail.size >= smin]
return self.mails

539
shell.py Normal file
View File

@ -0,0 +1,539 @@
# coding: utf-8
#
# Postfix queue control python tool (pymailq)
#
# Copyright (C) 2014 Denis Pompilio (jawa) <denis.pompilio@gmail.com>
# Copyright (C) 2014 Jocelyn Delalande <jdelalande@oasiswork.fr>
#
# This file is part of pymailq
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, see <http://www.gnu.org/licenses/>.
import cmd
from functools import partial
from datetime import datetime, timedelta
from subprocess import CalledProcessError
import shlex
import inspect
from pymailq import store, control, selector, utils
class PyMailqShell(cmd.Cmd):
"""PyMailq shell for interactive mode"""
# Automatic building of supported methods and documentation
commands_info = {
'store': 'Control of Postfix queue content storage',
'select': 'Select mails from Postfix queue content',
'inspect': 'Mail content inspector',
'super': 'Call postsuper commands'
}
# XXX: do_* methods are parsed before init and must be declared here
do_inspect = None
do_store = None
do_select = None
do_super = None
def __init__(self, completekey='tab', stdin=None, stdout=None,
store_auto_load=False):
"""Init method"""
cmd.Cmd.__init__(self, completekey, stdin, stdout)
# EOF action is registered here to hide it from user
self.do_EOF = self.do_exit
for command in self.commands_info:
setattr(self, "help_%s" % (command,), partial(self._help_, command))
setattr(self, "do_%s" % (command,), partial(self.__do, command))
# show command is specific and cannot be build dynamically
setattr(self, "help_show", partial(self._help_, "show"))
self.pstore = store.PostqueueStore()
self.selector = selector.MailSelector(self.pstore)
self.qcontrol = control.QueueControl()
if store_auto_load:
self.respond("Loading mails queue content to store")
self._store_load()
def respond(self, answer):
"""Send response"""
if not isinstance(answer, str):
answer = answer.encode('utf-8')
self.stdout.write('%s\n' % answer)
# Internal functions
def emptyline(self):
"""Action on empty lines"""
pass
def help_help(self):
"""Help of command help"""
self.respond("Show available commands")
@staticmethod
def do_exit(arg):
"""Action on exit"""
return True
def help_exit(self):
"""Help of command exit"""
self.respond("Exit PyMailq shell (or use Ctrl-D)")
def cmdloop_nointerrupt(self):
"""Specific cmdloop to handle KeyboardInterrupt"""
can_exit = False
# intro message is not in self.intro not to display it each time
# cmdloop is restarted
self.respond("Welcome to PyMailq shell.")
while can_exit is not True:
try:
self.cmdloop()
can_exit = True
except KeyboardInterrupt:
self.respond("^C")
def postloop(self):
cmd.Cmd.postloop(self)
self.respond("\nExiting shell... Bye.")
def _help_(self, command):
docstr = self.commands_info.get(
command, getattr(self, "do_%s" % (command,)).__doc__)
self.respond(inspect.cleandoc(docstr))
self.respond("Subcommands:")
for method in dir(self):
if method.startswith("_%s_" % (command,)):
docstr = getattr(self, method).__doc__
doclines = inspect.cleandoc(docstr).split('\n')
self.respond(" %-10s %s" % (method[len(command)+2:],
doclines.pop(0)))
for line in doclines:
self.respond(" %-10s %s" % ("", line))
#
# PyMailq methods
#
@property
def prompt(self):
"""Dynamic prompt with usefull informations"""
prompt = ['PyMailq']
if self.selector is not None:
prompt.append(' (sel:%d)' % (len(self.selector.mails)))
prompt.append('> ')
return "".join(prompt)
def __do(self, cmd_category, str_arg):
"""Generic do_* method to call cmd categories"""
args = shlex.split(str_arg)
if not len(args):
getattr(self, "help_%s" % (cmd_category,))()
return None
command = args.pop(0)
method = "_%s_%s" % (cmd_category, command)
try:
lines = getattr(self, method)(*args)
if lines is not None and len(lines):
self.respond('\n'.join(lines))
except AttributeError:
self.respond("%s has no subcommand: %s" % (cmd_category, command))
except (SyntaxError, TypeError) as exc:
# Rewording Python TypeError message for cli display
msg = str(exc)
if "%s()" % (method,) in msg:
msg = "%s command %s" % (cmd_category, msg[len(method)+3:])
self.respond("*** Syntax error: " + msg)
@staticmethod
def get_modifiers(match, excludes=()):
"""Get modifiers from match
:param str match: String to match in modifiers
:param list excludes: Excluded modifiers
:return: Matched modifiers as :func:`list`
"""
modifiers = {
'limit': ['<n>'],
'rankby': ['<field>'],
'sortby': ['<field> [asc|desc]']
}
if match in modifiers and match not in excludes:
return modifiers[match]
return [mod for mod in modifiers
if mod not in excludes and mod.startswith(match)]
def completenames(self, text, *ignored):
"""Complete known commands"""
dotext = 'do_'+text
suggests = [a[3:] for a in self.get_names() if a.startswith(dotext)]
if len(suggests) == 1:
# Only one suggest, return it with a space
suggests[0] += " "
return suggests
def completedefault(self, text, line, *ignored):
"""Generic command completion method"""
# we may consider the use of re.match for params in completion
completion = {
'show': {
'__allow_mods__': True,
},
'inspect': {
'mails': ['<qid>[,<qid>,...]']
},
'select': {
'date': ['<datespec>'],
'error': ['<error_msg>'],
'rmfilter': ['<filterid>'],
'sender': ['<sender> [exact]'],
'recipient': ['<recipient> [exact]'],
'size': ['<-n|n|+n> [-n]'],
'status': ['<status>']
}
}
args = shlex.split(line)
command = args.pop(0)
sub_command = ""
if len(args):
sub_command = args.pop(0)
match = "_%s_" % (command,)
suggests = [name[len(match):] for name in dir(self)
if name.startswith(match + sub_command)]
# No suggests, return None
if not len(suggests):
return None
# Return multiple suggests for sub-command
if len(suggests) > 1:
return suggests
suggest = suggests.pop(0)
exact_match = True if suggest == sub_command else False
if suggest in completion.get(command, {}):
if not exact_match:
# Sub-command takes params, suffix it with a space
return [suggest + " "]
elif not len(args):
# Return sub-command params
return completion[command][sub_command]
elif not exact_match:
# Sub-command doesn't take params, return as is
return [suggest]
# Command allows modifiers
if completion[command].get('__allow_mods__'):
if len(args) or not len(text):
match = args[-1] if len(args) else ""
mods = self.get_modifiers(match, excludes=args[:-1])
if not len(mods):
mods = self.get_modifiers("", excludes=args)
if len(mods):
mods[0] += " " if len(mods) == 1 else ""
suggests = mods
if not len(suggests):
return None
return suggests
def _store_load(self, filename=None):
"""Load Postfix queue content"""
try:
self.pstore.load(filename=filename)
# Automatic load of selector if it is empty and never used.
if not len(self.selector.mails) and not len(self.selector.filters):
self.selector.reset()
return ["%d mails loaded from queue" % (len(self.pstore.mails))]
except (OSError, IOError, CalledProcessError) as exc:
return ["*** Error: unable to load store", " %s" % (exc,)]
def _store_status(self):
"""Show store status"""
if self.pstore is None or self.pstore.loaded_at is None:
return ["store is not loaded"]
return ["store loaded with %d mails at %s" % (
len(self.pstore.mails), self.pstore.loaded_at)]
def _select_reset(self):
"""Reset content of selector with store content"""
self.selector.reset()
return ["Selector resetted with store content (%s mails)" % (
len(self.selector.mails))]
def _select_replay(self):
"""Reset content of selector with store content and replay filters"""
self.selector.replay_filters()
return ["Selector resetted and filters replayed"]
def _select_rmfilter(self, filterid):
"""
Remove filter previously applied
Filters ids are used to specify filter to remove
Usage: select rmfilter <filterid>
"""
try:
idx = int(filterid)
self.selector.filters.pop(idx)
self.selector.replay_filters()
# TODO: except should be more accurate
except:
raise SyntaxError("invalid filter ID: %s" % filterid)
def _select_qids(self, *qids):
"""
Select mails by ID
Usage: select qids <qid>[,<qid>,...]
"""
self.selector.lookup_qids(qids)
def _select_status(self, status):
"""
Select mails with specific postfix status
Usage: select status <status>
"""
self.selector.lookup_status(status=status)
def _select_sender(self, sender, exact=False):
"""
Select mails from sender
Usage: select sender <sender> [exact]
"""
if exact is not False: # received from command line
if exact != "exact":
raise SyntaxError("invalid keyword: %s" % exact)
exact = True
self.selector.lookup_sender(sender=sender, exact=exact)
def _select_recipient(self, recipient, exact=False):
"""
Select mails to recipient
Usage: select recipient <recipient> [exact]
"""
if exact is not False: # received from command line
if exact != "exact":
raise SyntaxError("invalid keyword: %s" % exact)
exact = True
self.selector.lookup_recipient(recipient=recipient, exact=exact)
def _select_size(self, size_a, size_b=None):
"""
Select mails by size in Bytes
- and + are supported, if not specified, search for exact size
Size range is allowed by using - (lesser than) and + (greater than)
Usage: select size <-n|n|+n> [-n]
"""
smin = None
smax = None
exact = None
try:
for size in size_a, size_b:
if size is None:
continue
if exact is not None:
raise SyntaxError("exact size must be used alone")
if size.startswith("-"):
if smax is not None:
raise SyntaxError("multiple max sizes specified")
smax = int(size[1:])
elif size.startswith("+"):
if smin is not None:
raise SyntaxError("multiple min sizes specified")
smin = int(size[1:])
else:
exact = int(size)
except ValueError:
raise SyntaxError("specified sizes must be valid numbers")
if exact is not None:
smin = exact
smax = exact
if smax is None:
smax = 0
if smin is None:
smin = 0
if smin > smax > 0:
raise SyntaxError("minimum size is greater than maximum size")
self.selector.lookup_size(smin=smin, smax=smax)
def _select_date(self, date_spec):
"""
Select mails by date.
Usage:
select date <DATESPEC>
Where <DATESPEC> can be
YYYY-MM-DD (exact date)
YYYY-MM-DD..YYYY-MM-DD (within a date range (included))
+YYYY-MM-DD (after a date (included))
-YYYY-MM-DD (before a date (included))
"""
try:
if ".." in date_spec:
(str_start, str_stop) = date_spec.split("..", 1)
start = datetime.strptime(str_start, "%Y-%m-%d")
stop = datetime.strptime(str_stop, "%Y-%m-%d")
elif date_spec.startswith("+"):
start = datetime.strptime(date_spec[1:], "%Y-%m-%d")
stop = datetime.now()
elif date_spec.startswith("-"):
start = datetime(1970, 1, 1)
stop = datetime.strptime(date_spec[1:], "%Y-%m-%d")
else:
start = datetime.strptime(date_spec, "%Y-%m-%d")
stop = start + timedelta(1)
self.selector.lookup_date(start, stop)
except ValueError as exc:
raise SyntaxError(str(exc))
def _select_error(self, error_msg):
"""
Select mails by error message
Specified error message can be partial
Usage: select error <error_msg>
"""
self.selector.lookup_error(str(error_msg))
def _inspect_mails(self, *qids):
"""
Show mails content
Usage: inspect mails <qid> [qid] ...
"""
mails = self.selector.get_mails_by_qids(qids)
if not len(mails):
return ['Mail IDs not found']
response = []
for mail in mails:
mail.parse()
if len(mail.parse_error):
return [mail.parse_error]
response.append(mail.show())
return response
def do_show(self, str_arg):
"""
Generic viewer utility
Optionnal modifiers can be provided to alter output:
limit <n> display the first n entries
sortby <field> [asc|desc] sort output by field asc or desc
rankby <field> Produce mails ranking by field
Known fields:
qid Postqueue mail ID
date Mail date
sender Mail sender
recipients Mail recipients (list, no sort)
size Mail size
errors Postqueue deferred error messages (list, no sort)
"""
args = shlex.split(str_arg)
if not len(args):
return self.help_show()
sub_cmd = args.pop(0)
try:
lines = getattr(self, "_show_%s" % sub_cmd)(*args)
except (TypeError, AttributeError):
self.respond("*** Syntax error: show {0}".format(str_arg))
return self.help_show()
except SyntaxError as error:
# Rewording Python TypeError message for cli display
msg = str(error)
if "%s()" % sub_cmd in msg:
msg = "show command %s" % msg
self.respond("*** Syntax error: " + msg)
return self.help_show()
self.respond("\n".join(lines))
@utils.viewer
@utils.ranker
@utils.sorter
def _show_selected(self):
"""
Show selected mails
Usage: show selected [modifiers]
"""
return self.selector.mails
def _show_filters(self):
"""
Show filters applied on current mails selection
Usage: show filters
"""
if not len(self.selector.filters):
return ["No filters applied on current selection"]
lines = []
for idx, pqfilter in enumerate(self.selector.filters):
name, _args, _kwargs = pqfilter
# name should always be prefixed with lookup_
lines.append('%d: select %s:' % (idx, name[7:]))
for key in sorted(_kwargs):
lines.append(" %s: %s" % (key, _kwargs[key]))
return lines
# Postsuper generic command
def __do_super(self, operation):
"""Postsuper generic command"""
if not self.pstore.loaded_at:
return ["The store is not loaded"]
if not len(self.selector.mails):
return ["No mail selected"]
else:
func = getattr(self.qcontrol, '%s_messages' % operation)
try:
resp = func(self.selector.mails)
except RuntimeError as exc:
return [str(exc)]
# reloads the data
self._store_load()
self._select_replay()
return [resp[-1]]
def _super_delete(self):
"""Deletes the mails in current selection
Usage: super delete
"""
return self.__do_super('delete')
def _super_hold(self):
"""Put on hold the mails in current selection
Usage: super hold
"""
return self.__do_super('hold')
def _super_release(self):
"""Releases from hold the mails in current selection
Usage: super release
"""
return self.__do_super('release')
def _super_requeue(self):
"""requeue the mails in current selection
Usage: super requeue
"""
return self.__do_super('requeue')

733
store.py Normal file
View File

@ -0,0 +1,733 @@
# coding: utf-8
#
# Postfix queue control python tool (pymailq)
#
# Copyright (C) 2014 Denis Pompilio (jawa) <denis.pompilio@gmail.com>
#
# This file is part of pymailq
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, see <http://www.gnu.org/licenses/>.
from __future__ import unicode_literals
import sys
import os
import gc
import re
import subprocess
import email
from email import header
from collections import Counter
from datetime import datetime, timedelta
from pymailq import CONFIG, debug
class MailHeaders(object):
"""
Simple object to store mail headers.
Object's attributes are dynamically created when parent :class:`~store.Mail`
object's method :meth:`~store.Mail.parse` is called. Those attributes are
retrieved with help of :func:`~email.message_from_string` method provided
by the :mod:`email` module.
Standard RFC *822-style* mail headers becomes attributes including but not
limited to:
- :mailheader:`Received`
- :mailheader:`From`
- :mailheader:`To`
- :mailheader:`Cc`
- :mailheader:`Bcc`
- :mailheader:`Sender`
- :mailheader:`Reply-To`
- :mailheader:`Subject`
Case is kept while creating attribute and access will be made with
:attr:`Mail.From` or :attr:`Mail.Received` for example. All those
attributes will return *list* of values.
.. seealso::
Python modules:
:mod:`email` -- An email and MIME handling package
:class:`email.message.Message` -- Representing an email message
:rfc:`822` -- Standard for ARPA Internet Text Messages
"""
class Mail(object):
"""
Simple object to manipulate email messages.
This class provides the necessary methods to load and inspect mails
content. This object functionnalities are mainly based on :mod:`email`
module's provided class and methods. However,
:class:`email.message.Message` instance's stored informations are
extracted to extend :class:`~store.Mail` instances attributes.
Initialization of :class:`~store.Mail` instances are made the following
way:
:param str mail_id: Mail's queue ID string
:param int size: Mail size in Bytes (Default: ``0``)
:param datetime.datetime date: Acceptance date and time in mails queue.
(Default: :data:`None`)
:param str sender: Mail sender string as seen in mails queue.
(Default: empty :func:`str`)
The :class:`~pymailq.Mail` class defines the following attributes:
.. attribute:: qid
Mail Postfix queue ID string, validated by
:meth:`~store.PostqueueStore._is_mail_id` method.
.. attribute:: size
Mail size in bytes. Expected type is :func:`int`.
.. attribute:: parsed
:func:`bool` value to track if mail's content has been loaded from
corresponding spool file.
.. attribute:: parse_error
Last encountered parse error message :func:`str`.
.. attribute:: date
:class:`~datetime.datetime` object of acceptance date and time in
mails queue.
.. attribute:: status
Mail's queue status :func:`str`.
.. attribute:: sender
Mail's sender :func:`str` as seen in mails queue.
.. attribute:: recipients
Recipients :func:`list` as seen in mails queue.
.. attribute:: errors
Mail deliver errors :func:`list` as seen in mails queue.
.. attribute:: head
Mail's headers :class:`~store.MailHeaders` structure.
.. attribute:: raw_content
Mail's raw content :func:`str`.
.. attribute:: postcat_cmd
This property use Postfix mails content parsing command defined in
:attr:`pymailq.CONFIG` attribute under the key 'cat_message'.
Command and arguments list is build on call with the configuration
data.
.. seealso::
:ref:`pymailq-configuration`
"""
def __init__(self, mail_id, size=0, date=None, sender=""):
"""Init method"""
self.parsed = False
self.parse_error = ""
self.qid = mail_id
self.date = date
self.status = ""
self.size = int(size)
self.sender = sender
self.recipients = []
self.errors = []
self.head = MailHeaders()
# Getting optionnal status from postqueue mail_id
postqueue_status = {'*': "active", '!': "hold"}
if mail_id[-1] in postqueue_status:
self.qid = mail_id[:-1]
self.status = postqueue_status.get(mail_id[-1], "deferred")
@property
def postcat_cmd(self):
"""
Get the cat_message command from configuration
:return: Command as :class:`list`
"""
postcat_cmd = CONFIG['commands']['cat_message'] + [self.qid]
if CONFIG['commands']['use_sudo']:
postcat_cmd.insert(0, 'sudo')
return postcat_cmd
def show(self):
"""
Return mail detailled representation for printing
:return: Representation as :class:`str`
"""
output = "=== Mail %s ===\n" % (self.qid,)
for attr in sorted(dir(self.head)):
if attr.startswith("_"):
continue
value = getattr(self.head, attr)
if not isinstance(value, str):
value = ", ".join(value)
if attr == "Subject":
#print(attr, value)
value, enc = header.decode_header(value)[0]
#print(enc, attr, value)
if sys.version_info[0] == 2:
value = value.decode(enc) if enc else unicode(value)
output += "%s: %s\n" % (attr, value)
return output
@debug
def parse(self):
"""
Parse message content.
This method use Postfix mails content parsing command defined in
:attr:`~Mail.postcat_cmd` attribute.
This command is runned using :class:`subprocess.Popen` instance.
Parsed headers become attributes and are retrieved with help of
:func:`~email.message_from_string` function provided by the
:mod:`email` module.
.. seealso::
Postfix manual:
`postcat`_ -- Show Postfix queue file contents
"""
# Reset parsing error message
self.parse_error = ""
child = subprocess.Popen(self.postcat_cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, stderr = child.communicate()
if not len(stdout):
# Ignore first 3 line on stderr which are:
# postcat: name_mask: all
# postcat: inet_addr_local: configured 3 IPv4 addresses
# postcat: inet_addr_local: configured 3 IPv6 addresses
self.parse_error = "\n".join(stderr.decode().split('\n')[3:])
return
raw_content = ""
for line in stdout.decode('utf-8', errors='replace').split('\n'):
if self.size == 0 and line.startswith("message_size: "):
self.size = int(line[14:].strip().split()[0])
elif self.date is None and line.startswith("create_time: "):
self.date = datetime.strptime(line[13:].strip(),
"%a %b %d %H:%M:%S %Y")
elif not len(self.sender) and line.startswith("sender: "):
self.sender = line[8:].strip()
elif line.startswith("regular_text: "):
raw_content += "%s\n" % (line[14:],)
# For python2.7 compatibility, encode unicode to str
if not isinstance(raw_content, str):
raw_content = raw_content.encode('utf-8')
message = email.message_from_string(raw_content)
for mailheader in set(message.keys()):
value = message.get_all(mailheader)
setattr(self.head, mailheader, value)
self.raw_content = raw_content
self.parsed = True
@debug
def dump(self):
"""
Dump mail's gathered informations to a :class:`dict` object.
Mails informations are splitted in two parts in dictionnary.
``postqueue`` key regroups every informations directly gathered from
Postfix queue, while ``headers`` regroups :class:`~store.MailHeaders`
attributes converted from mail content with the
:meth:`~store.Mail.parse` method.
If mail has not been parsed with the :meth:`~store.Mail.parse` method,
informations under the ``headers`` key will be empty.
:return: Mail gathered informations
:rtype: :class:`dict`
"""
datas = {'postqueue': {},
'headers': {}}
for attr in self.__dict__:
if attr[0] != "_" and attr != 'head':
datas['postqueue'].update({attr: getattr(self, attr)})
for mailheader in self.head.__dict__:
if mailheader[0] != "_":
datas['headers'].update(
{mailheader: getattr(self.head, mailheader)}
)
return datas
class PostqueueStore(object):
"""
Postfix mails queue informations storage.
The :class:`~store.PostqueueStore` provides methods to load Postfix
queued mails informations into Python structures. Thoses structures are
based on :class:`~store.Mail` and :class:`~store.MailHeaders` classes
which can be processed by a :class:`~selector.MailSelector` instance.
The :class:`~store.PostqueueStore` class defines the following attributes:
.. attribute:: mails
Loaded :class:`MailClass` objects :func:`list`.
.. attribute:: loaded_at
:class:`datetime.datetime` instance to store load date and time
informations, useful for datas deprecation tracking. Updated on
:meth:`~store.PostqueueStore.load` call with
:meth:`datetime.datetime.now` method.
.. attribute:: postqueue_cmd
:obj:`list` object to store Postfix command and arguments to view
the mails queue content. This property use Postfix mails content
parsing command defined in :attr:`pymailq.CONFIG` attribute under
the key 'list_queue'. Command and arguments list is build on call
with the configuration data.
.. attribute:: spool_path
Postfix spool path string.
Default is ``"/var/spool/postfix"``.
.. attribute:: postqueue_mailstatus
Postfix known queued mail status list.
Default is ``['active', 'deferred', 'hold']``.
.. attribute:: mail_id_re
Python compiled regular expression object (:class:`re.RegexObject`)
provided by :func:`re.compile` method to match postfix IDs.
Recognized IDs are either:
- hexadecimals, 8 to 12 chars length (regular queue IDs)
- encoded in a 52-character alphabet, 11 to 16 chars length
(long queue IDs)
They can be followed with ``*`` or ``!``.
Default used regular expression is:
``r"^([A-F0-9]{8,12}|[B-Zb-z0-9]{11,16})[*!]?$"``.
.. attribute:: mail_addr_re
Python compiled regular expression object (:class:`re.RegexObject`)
provided by :func:`re.compile` method to match email addresses.
Default used regular expression is:
``r"^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]+$"``
.. attribute:: MailClass
The class used to manipulate/parse mails individually.
Default is :class:`~store.Mail`.
.. seealso::
Python modules:
:mod:`datetime` -- Basic date and time types
:mod:`re` -- Regular expression operations
Postfix manual:
`postqueue`_ -- Postfix queue control
:rfc:`3696` -- Checking and Transformation of Names
"""
postqueue_cmd = None
spool_path = None
postqueue_mailstatus = ['active', 'deferred', 'hold']
mail_id_re = re.compile(r"^([A-F0-9]{8,12}|[B-Zb-z0-9]{11,16})[*!]?$")
mail_addr_re = re.compile(r"^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]+$")
MailClass = Mail
def __init__(self):
"""Init method"""
self.spool_path = CONFIG['core']['postfix_spool']
self.postqueue_cmd = CONFIG['commands']['list_queue']
if CONFIG['commands']['use_sudo']:
self.postqueue_cmd.insert(0, 'sudo')
self.loaded_at = None
self.mails = []
@property
@debug
def known_headers(self):
"""Return known headers from loaded mails
:return: headers as :func:`set`
"""
headers = set()
for mail in self.mails:
for mailheader in dir(mail.head):
if not mailheader.startswith("_"):
headers.add(mailheader)
return headers
@debug
def _get_postqueue_output(self):
"""
Get Postfix postqueue command output.
This method used the postfix command defined in
:attr:`~PostqueueStore.postqueue_cmd` attribute to view the mails queue
content.
Command defined in :attr:`~PostqueueStore.postqueue_cmd` attribute is
runned using a :class:`subprocess.Popen` instance.
:return: Command's output lines.
:rtype: :func:`list`
.. seealso::
Python module:
:mod:`subprocess` -- Subprocess management
"""
child = subprocess.Popen(self.postqueue_cmd,
stdout=subprocess.PIPE)
stdout = child.communicate()[0]
# return lines list without the headers and footers
return [line.strip() for line in stdout.decode().split('\n')][1:-2]
def _is_mail_id(self, mail_id):
"""
Check mail_id for a valid postfix queued mail ID.
Validation is made using a :class:`re.RegexObject` stored in
the :attr:`~PostqueueStore.mail_id_re` attribute of the
:class:`~store.PostqueueStore` instance.
:param str mail_id: Mail Postfix queue ID string
:return: True or false
:rtype: :func:`bool`
"""
if self.mail_id_re.match(mail_id) is None:
return False
return True
@debug
def _load_from_postqueue(self, filename=None, parse=False):
"""
Load content from postfix queue using postqueue command output.
Output lines from :attr:`~store.PostqueueStore._get_postqueue_output`
are parsed to build :class:`~store.Mail` objects. Sample Postfix queue
control tool (`postqueue`_) output::
C0004979687 4769 Tue Apr 29 06:35:05 sender@domain.com
(error message from mx.remote1.org with parenthesis)
first.rcpt@remote1.org
(error message from mx.remote2.org with parenthesis)
second.rcpt@remote2.org
third.rcpt@remote2.org
Parsing rules are pretty simple:
- Line starts with a valid :attr:`Mail.qid`: create new
:class:`~store.Mail` object with :attr:`~Mail.qid`,
:attr:`~Mail.size`, :attr:`~Mail.date` and :attr:`~Mail.sender`
informations from line.
+-------------+------+---------------------------+-----------------+
| Queue ID | Size | Reception date and time | Sender |
+-------------+------+-----+-----+----+----------+-----------------+
| C0004979687 | 4769 | Tue | Apr | 29 | 06:35:05 | user@domain.com |
+-------------+------+-----+-----+----+----------+-----------------+
- Line starts with a parenthesis: store error messages to last created
:class:`~store.Mail` object's :attr:`~Mail.errors` attribute.
- Any other matches: add new recipient to the :attr:`~Mail.recipients`
attribute of the last created :class:`~store.Mail` object.
Optionnal argument ``filename`` can be set with a file containing
output of the `postqueue`_ command. In this case, output lines of
`postqueue`_ command are directly read from ``filename`` and parsed,
the `postqueue`_ command is never used.
Optionnal argument ``parse`` controls whether mails are parsed or not.
This is useful to load every known mail headers for later filtering.
:param str filename: File to load mails from
:param bool parse: Controls whether loaded mails are parsed or not.
"""
if filename is None:
postqueue_output = self._get_postqueue_output()
else:
postqueue_output = open(filename).readlines()
mail = None
for line in postqueue_output:
line = line.strip()
# Headers and footers start with dash (-)
if line.startswith('-'):
continue
# Mails are blank line separated
if not len(line):
continue
fields = line.split()
if "(" == fields[0][0]:
# Store error message without parenthesis: [1:-1]
# gathered errors must be associated with specific recipients
# TODO: change recipients or errors structures to link these
# objects together.
mail.errors.append(" ".join(fields)[1:-1])
else:
if self._is_mail_id(fields[0]):
# postfix does not precise year in mails timestamps so
# we consider mails have been sent this year.
# If gathered date is in the future:
# mail has been received last year (or NTP problem).
now = datetime.now()
datestr = "{0} {1}".format(" ".join(fields[2:-1]), now.year)
date = datetime.strptime(datestr, "%a %b %d %H:%M:%S %Y")
if date > now:
date = date - timedelta(days=365)
mail = self.MailClass(fields[0], size=fields[1],
date=date,
sender=fields[-1])
self.mails.append(mail)
else:
# Email address validity check can be tricky. RFC3696 talks
# about. Fow now, we use a simple regular expression to
# match most of email addresses.
rcpt_email_addr = " ".join(fields)
if self.mail_addr_re.match(rcpt_email_addr):
mail.recipients.append(rcpt_email_addr)
if parse:
#print("parsing mails")
[mail.parse() for mail in self.mails]
@debug
def _load_from_spool(self, parse=True):
"""
Load content from postfix queue using files from spool.
Mails are loaded using the command defined in
:attr:`~PostqueueStore.postqueue_cmd` attribute. Some informations may
be missing using the :meth:`~store.PostqueueStore._load_from_spool`
method, including at least :attr:`Mail.status` field.
Optionnal argument ``parse`` controls whether mails are parsed or not.
This is useful to load every known mail headers for later filtering.
Loaded mails are stored as :class:`~store.Mail` objects in
:attr:`~PostqueueStore.mails` attribute.
:param bool parse: Controls whether loaded mails are parsed or not.
.. warning::
Be aware that parsing mails on disk is slow and can lead to
high load usage on system with large mails queue.
"""
for status in self.postqueue_mailstatus:
for fs_data in os.walk("%s/%s" % (self.spool_path, status)):
for mail_id in fs_data[2]:
mail = self.MailClass(mail_id)
mail.status = status
mail.parse()
self.mails.append(mail)
@debug
def _load_from_file(self, filename):
"""Unimplemented method"""
@debug
def load(self, method="postqueue", filename=None, parse=False):
"""
Load content from postfix mails queue.
Mails are loaded using postqueue command line tool or reading directly
from spool. The optionnal argument, if present, is a method string and
specifies the method used to gather mails informations. By default,
method is set to ``"postqueue"`` and the standard Postfix queue
control tool: `postqueue`_ is used.
Optionnal argument ``parse`` controls whether mails are parsed or not.
This is useful to load every known mail headers for later filtering.
:param str method: Method used to load mails from Postfix queue
:param str filename: File to load mails from
:param bool parse: Controls whether loaded mails are parsed or not.
Provided method :func:`str` name is directly used with :func:`getattr`
to find a *self._load_from_<method>* method.
"""
# releasing memory
del self.mails
gc.collect()
self.mails = []
if filename is None:
getattr(self, "_load_from_{0}".format(method))(parse=parse)
else:
getattr(self, "_load_from_{0}".format(method))(filename, parse)
self.loaded_at = datetime.now()
@debug
def summary(self):
"""
Summarize the mails queue content.
:return: Mail queue summary as :class:`dict`
Sizes are in bytes.
Example response::
{
'total_mails': 500,
'total_mails_size': 709750,
'average_mail_size': 1419.5,
'max_mail_size': 2414,
'min_mail_size': 423,
'top_errors': [
('mail transport unavailable', 484),
('Test error message', 16)
],
'top_recipient_domains': [
('test-domain.tld', 500)
],
'top_recipients': [
('user-3@test-domain.tld', 200),
('user-2@test-domain.tld', 200),
('user-1@test-domain.tld', 100)
],
'top_sender_domains': [
('test-domain.tld', 500)
],
'top_senders': [
('sender-1@test-domain.tld', 100),
('sender-2@test-domain.tld', 100),
('sender-7@test-domain.tld', 50),
('sender-4@test-domain.tld', 50),
('sender-5@test-domain.tld', 50)
],
'top_status': [
('deferred', 500),
('active', 0),
('hold', 0)
],
'unique_recipient_domains': 1,
'unique_recipients': 3,
'unique_sender_domains': 1,
'unique_senders': 8
}
"""
senders = Counter()
sender_domains = Counter()
recipients = Counter()
recipient_domains = Counter()
status = Counter(active=0, hold=0, deferred=0)
errors = Counter()
total_mails_size = 0
average_mail_size = 0
max_mail_size = 0
min_mail_size = 0
mails_by_age = {
'last_24h': 0,
'1_to_4_days_ago': 0,
'older_than_4_days': 0
}
for mail in self.mails:
status[mail.status] += 1
senders[mail.sender] += 1
if '@' in mail.sender:
sender_domains[mail.sender.split('@', 1)[1]] += 1
for recipient in mail.recipients:
recipients[recipient] += 1
if '@' in recipient:
recipient_domains[recipient.split('@', 1)[1]] += 1
for error in mail.errors:
errors[error] += 1
total_mails_size += mail.size
if mail.size > max_mail_size:
max_mail_size = mail.size
if min_mail_size == 0:
min_mail_size = mail.size
elif mail.size < min_mail_size:
min_mail_size = mail.size
mail_age = datetime.now() - mail.date
if mail_age.days >= 4:
mails_by_age['older_than_4_days'] += 1
elif mail_age.days == 1:
mails_by_age['1_to_4_days_ago'] += 1
elif mail_age.days == 0:
mails_by_age['last_24h'] += 1
if len(self.mails):
average_mail_size = total_mails_size / len(self.mails)
summary = {
'total_mails': len(self.mails),
'mails_by_age': mails_by_age,
'total_mails_size': total_mails_size,
'average_mail_size': average_mail_size,
'max_mail_size': max_mail_size,
'min_mail_size': min_mail_size,
'top_status': status.most_common()[:5],
'unique_senders': len(list(senders)),
'unique_sender_domains': len(list(sender_domains)),
'unique_recipients': len(list(recipients)),
'unique_recipient_domains': len(list(recipient_domains)),
'top_senders': senders.most_common()[:5],
'top_sender_domains': sender_domains.most_common()[:5],
'top_recipients': recipients.most_common()[:5],
'top_recipient_domains': recipient_domains.most_common()[:5],
'top_errors': errors.most_common()[:5]
}
return summary

189
utils.py Normal file
View File

@ -0,0 +1,189 @@
# coding: utf-8
#
# Postfix queue control python tool (pymailq)
#
# Copyright (C) 2014 Denis Pompilio (jawa) <denis.pompilio@gmail.com>
#
# This file is part of pymailq
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, see <http://www.gnu.org/licenses/>.
import re
from functools import wraps
from collections import Counter
FORMAT_PARSER = re.compile(r'\{[^{}]+\}')
FORMATS = {
'brief': "{date} {qid} [{status}] {sender} ({size}B)",
'long': ("{date} {qid} [{status}] {sender} ({size}B)\n"
" Rcpt: {recipients}\n"
" Err: {errors}")
}
def viewer(function):
"""Result viewer decorator
:param func function: Function to decorate
"""
def wrapper(*args, **kwargs):
args = list(args) # conversion need for arguments cleaning
limit = None
overhead = 0
try:
if "limit" in args:
limit_idx = args.index('limit')
args.pop(limit_idx) # pop option, next arg is value
limit = int(args.pop(limit_idx))
except (IndexError, TypeError, ValueError):
raise SyntaxError("limit modifier needs a valid number")
output = "brief"
for known in FORMATS:
if known in args:
output = args.pop(args.index(known))
break
out_format = FORMATS[output]
elements = function(*args, **kwargs)
total_elements = len(elements)
if not total_elements:
return ["No element to display"]
# Check for headers and increase limit accordingly
headers = 0
if total_elements > 1 and "========" in str(elements[1]):
headers = 2
if limit is not None:
if total_elements > (limit + headers):
overhead = total_elements - (limit + headers)
else:
limit = total_elements
else:
limit = total_elements
out_format_attrs = FORMAT_PARSER.findall(out_format)
formatted = []
for element in elements[:limit + headers]:
# if attr qid exists, assume this is a mail
if hasattr(element, "qid"):
attrs = {}
for att in out_format_attrs:
if att == "{recipients}":
rcpts = getattr(element, att[1:-1], ["-"])
attrs[att[1:-1]] = ", ".join(rcpts)
elif att == "{errors}":
errors = getattr(element, att[1:-1], ["-"])
attrs[att[1:-1]] = "\n".join(errors)
else:
attrs[att[1:-1]] = getattr(element, att[1:-1], "-")
formatted.append(out_format.format(**attrs))
else:
formatted.append(element)
if overhead > 0:
msg = "...Preview of first %d (%d more)..." % (limit, overhead)
formatted.append(msg)
return formatted
wrapper.__doc__ = function.__doc__
return wrapper
def sorter(function):
"""Result sorter decorator.
This decorator inspect decorated function arguments and search for
known keyword to sort decorated function result.
"""
@wraps(function)
def wrapper(*args, **kwargs):
args = list(args) # conversion need for arguments cleaning
sortkey = "date" # default sort by date
reverse = True # default sorting is desc
if "sortby" in args:
sortby_idx = args.index('sortby')
args.pop(sortby_idx) # pop option, next arg is value
try:
sortkey = args.pop(sortby_idx)
except IndexError:
raise SyntaxError("sortby requires a field")
# third param may be asc or desc, ignore unknown values
try:
if "asc" == args[sortby_idx]:
args.pop(sortby_idx)
reverse = False
elif "desc" == args[sortby_idx]:
args.pop(sortby_idx)
except IndexError:
pass
elements = function(*args, **kwargs)
try:
sorted_elements = sorted(elements,
key=lambda x: getattr(x, sortkey),
reverse=reverse)
except AttributeError:
msg = "elements cannot be sorted by %s" % sortkey
raise SyntaxError(msg)
return sorted_elements
wrapper.__doc__ = function.__doc__
return wrapper
def ranker(function):
"""Result ranker decorator
"""
@wraps(function)
def wrapper(*args, **kwargs):
args = list(args) # conversion need for arguments cleaning
rankkey = None
if "rankby" in args:
rankby_idx = args.index('rankby')
args.pop(rankby_idx) # pop option, next arg is value
try:
rankkey = args.pop(rankby_idx)
except IndexError:
raise SyntaxError("rankby requires a field")
elements = function(*args, **kwargs)
if rankkey is not None:
try:
rank = Counter()
for element in elements:
rank[getattr(element, rankkey)] += 1
# XXX: headers are taken in elements display limit :(
ranked_elements = ['%-40s count' % rankkey, '='*48]
for entry in rank.most_common():
key, value = entry
ranked_elements.append('%-40s %s' % (key, value))
return ranked_elements
except AttributeError:
msg = "elements cannot be ranked by %s" % rankkey
raise SyntaxError(msg)
return elements
wrapper.__doc__ = function.__doc__
return wrapper