Queues That Won't Load

[benne238]

We are aware that the following queues are not loading and are looking into them. If the queue you are having issues with is not listed here please add it in the comment below.

• me
• webmaster
• machelp

[benne238]

webmaster Queue Fix

The issue stemmed from a lack of error handling for empty header information from additional info supplied from user sections. That has now been fixed with #140

Additional info supplied by user structure:

=== Additional information supplied by user ===

Subject:  Subject Line Here
From:  "From_Name" <email@aol.com>
Cc:  "epicsite@ecn.purdue.edu" <epicsite@ecn.purdue.edu>,
            "CC Name One" <ccone@purdue.edu>,
            "CC Name Two" <cctwo@purdue.edu>,
            "CC Name Three" <ccthree@aol.com>
Date:  Mon, 19 Jan 2015 11:30:44 -0500

content...


===============================================

The structure consists of a beginning delimiter, a newline indicating header information, header information, a new line indicating the end of the header information and the beginning of the content, two new lines and the ending delimiter. However, the below queue resulted in an error during the date sort within the api:

=== Additional information supplied by user ===


content


===============================================

The problem with the section above (taken from webmaster 137 in the live queue) was with the handling of the header information. The api does not expect a fixed number of lines within the header, meaning that 0 lines of header information can be included within the reply-from-user section and still result in a successful parse for the rest of the section. However, a line indicating the date of the section must be provided in the header so that the script can sort the section by date, something that was not checked.
In order to successfully parse correctly, the api now looks for a date in the header info and returns a parse_error if none is found.

[benne238]

root-mail 798

Contents of the root-mail 798 item in the live queue:

From root@ecn.purdue.edu Tue Nov 17 23:00:05 2020
Received: from mx02.ecn.purdue.edu (mx02.ecn.purdue.edu [128.46.154.91])
        by pier.ecn.purdue.edu (8.14.7/8.14.7) with ESMTP id 0AI405bJ041236
        (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
        Tue, 17 Nov 2020 23:00:05 -0500
Received: from marina.ecn.purdue.edu (marina.ecn.purdue.edu [128.46.154.70])
        by mx02.ecn.purdue.edu (8.15.2/8.15.2/Debian-10) with ESMTPS id 0AI405e4004219
        (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT)
        for <rdistmail-lname@mx.ecn.purdue.edu>; Tue, 17 Nov 2020 23:00:05 -0500
Received: from marina.ecn.purdue.edu (localhost [127.0.0.1])
        by marina.ecn.purdue.edu (8.14.7/8.14.7) with ESMTP id 0AI404tY006463
        for <rdistmail-lname@marina.ecn.purdue.edu>; Tue, 17 Nov 2020 23:00:04 -0500
Received: (from root@localhost)
        by marina.ecn.purdue.edu (8.14.7/8.14.7/Submit) id 0AI404ka006458
        for rdistmail-lname; Tue, 17 Nov 2020 23:00:04 -0500
From: System Administration <root@ecn.purdue.edu>
Message-Id: <202011180400.0AI404ka006458@marina.ecn.purdue.edu>
Date: Tue, 17 Nov 2020 23:00:04 -0500
To: rdistmail-lname@ecn.purdue.edu
Subject: Package lname rdist: no errors
User-Agent: Heirloom mailx 12.5 7/5/10
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by pier.ecn.purdue.edu id 0AI405bJ041236

Two things are wrong with this item:

1. There is no initial message, which is currently not expected behavior and it throws an error when trying to parse content that doesn't exist. Fix: implement a check before trying to parse content to ensure there is an initial message
2. The first line of the header information is malformed and will likely cause an error if this item had an initial message. Fix: check for malformed header information.

Fixed with this comit in Bugfix-Queues-That-Won't-Load

[benne238]

me 14

Header info from the me 14 item in the live queue:

...
Date: Tue, 10 Nov 2020 16:31:09 -0500
From: abuganza
Message-ID: <202011102131.0AALVAgd008651@smtp.ecn.purdue.edu>
Subject: Re: Computer in 3009 is down, ordering new computer
To: abuganza@purdue.edu, mesite@ecn.purdue.edu
Content-Type: text/plain; charset="utf-8"
X-ECN-Queue-Original-Path: /home/pier/e/queue/Attachments/inbox/2020-11-10/300-original.txt
X-ECN-Queue-Original-URL: https://engineering.purdue.edu/webqueue/Attachments/inbox/2020-11-10/300-original.txt
...

From: abuganza is a malformed line in which a header line starting with From: needs to contain an email address with the @ symbol. This line doesn't contan a full emai, resulting in an error when attempting to parse. Fix: Check for malformed header info.

[benne238]

machelp 46

Issue with additional-information-from-user section header-date:

same problem as webmaster Queue fix from above

[campb303]

Parsing issues should be moved from custom logic to a proper parsing grammar.

[campb303]

Not sure if this can be closed. Asked @benne238 in Slack for an update.

[benne238]

dave 42

Some header information doesn't get parsed, this might be due to the formatting of long strings within the header information.

The current method of extracting header information involves the use of the python email package that automatically attempts to separate key value pairs into dictionaries. However, due to the formatting of strings in excess of a certain length, I think this is causing the email parser to omit certain key value pairs if a value to a specific key takes up multiple lines.

Continuing to use the email parser might still be possible, but a method for accommodating for multi length header values will need to be implemented

[campb303]

These changes can be merged in for now but the parsing of an item should be moved to a formal grammar. See #159