Bug #8167: net/imap failing to parse IMAP response containing one BODYSTRUCTURE

• Status: Closed
• Assignee: shugo (Shugo Maeda)
• Target version: -
• ruby -v: 2.0.0p0
• Backport: 2.0.0: DONE, 2.1: DONTNEED

[nlestat (Nathan Mackinnon)]

Here is debugging output from Net::IMAP::ResponseParse::parse method:

@str: "* 29021 FETCH (RFC822.SIZE 3162 UID 113622 RFC822.HEADER {1155}\r\nReturn-path: <>\r\nEnvelope-to: [e-mail]\r\nDelivery-date: Tue, 26 Mar 2013 12:42:58 +0100\r\nReceived: from mail by xxxx.xxxxxxxxxxx.net with spam-scanned (Exim 4.76)\r\n\tid 1UKSHI-000Cwl-AR\r\n\tfor [e-mail]; Tue, 26 Mar 2013 12:42:58 +0100\r\nX-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on xxxx.xxxxxxxxxxx.net\r\nX-Spam-Level: **\r\nX-Spam-Status: No, score=2.1 required=7.0 tests=DKIM_ADSP_NXDOMAIN,RDNS_NONE\r\n\tautolearn=no version=3.3.1\r\nReceived: from [xx.xxx.xxx.xx] (port=56890 helo=xxxxxx.localdomain)\r\n\tby xxxx.xxxxxxxxxxx.net with esmtp (Exim 4.76)\r\n\tid 1UKSHI-000Cwi-9j\r\n\tfor [e-mail]; Tue, 26 Mar 2013 12:42:56 +0100\r\nReceived: by xxxxxx.localdomain (Postfix)\r\n\tid 72725BEA64A; Tue, 26 Mar 2013 12:42:55 +0100 (CET)\r\nDate: Tue, 26 Mar 2013 12:42:55 +0100 (CET)\r\nFrom: [e-mail] (Mail Delivery System)\r\nSubject: Undelivered Mail Returned to Sender\r\nTo: [e-mail]\r\nAuto-Submitted: auto-replied\r\nMIME-Version: 1.0\r\nContent-Type: multipart/report; report-type=delivery-status;\r\n\tboundary="27797BEA649.1364298175/xxxxxx.localdomain"\r\nMessage-Id: [e-mail]\r\n\r\n BODYSTRUCTURE (("text" "plain" ("charset" "us-ascii") NIL "Notification" "7bit" 510 14 NIL NIL NIL NIL)("message" "delivery-status" NIL NIL "Delivery report" "7bit" 410 NIL NIL NIL NIL)("text" "rfc822-headers" ("charset" "us-ascii") NIL "Undelivered Message Headers" "7bit" 612 15 NIL NIL NIL NIL) "report" ("report-type" "delivery-status" "boundary" "27797BEA649.1364298175/xxxxxx.localdomain") NIL NIL NIL))\r\n"
@pos: 1390
@lex_state: EXPR_DATA
@token.symbol: SPACE
@token.value: " "
Net::IMAP::ResponseParseError: unexpected token SPACE (expected RPAR)

This error is observed on development machine with ruby 2.0.0-p0. Never reproduced on production installation running 1.9.3-p374.

I do not even know enough about IMAP standard to tell if message is actually invalid, or this is net/imap parser bug.

[bpot (Bob Potter)]

It looks like the response is invalid at least according the RFC3501. The additional fields that the parser is blowing up on should only be included if the type is 'message/rfc822', in this case it is 'message/delivery-status'. It looks like java's parser also has issues with this type of message: https://social.technet.microsoft.com/forums/en-US/exchangesvrsecuremessaginglegacy/thread/a2c7df45-2768-4a67-9c61-b912b56281bf

It would be possible to make the imap parser more liberal to handle these cases but I'm not sure if that's desirable.

It looks like the changed that changed this behavior is r37240.

[shugo (Shugo Maeda)]

• Status changed from Open to Assigned
• Assignee set to shugo (Shugo Maeda)

bpot (Bob Potter) wrote:

It looks like the response is invalid at least according the RFC3501. The additional fields that the parser is blowing up on should only be included if the type is 'message/rfc822', in this case it is 'message/delivery-status'. It looks like java's parser also has issues with this type of message: https://social.technet.microsoft.com/forums/en-US/exchangesvrsecuremessaginglegacy/thread/a2c7df45-2768-4a67-9c61-b912b56281bf

It would be possible to make the imap parser more liberal to handle these cases but I'm not sure if that's desirable.

I believe Net::IMAP should accept non-conformant input as long as the meaning is clear.

[shugo (Shugo Maeda)]

• Status changed from Assigned to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r40372.
Nathan, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

• lib/net/imap.rb (body_type_msg): should accept
  message/delivery-status with extra data.
  [ruby-core:53741] [Bug #8167]

• test/net/imap/test_imap_response_parser.rb: related test.

[usa (Usaku NAKAMURA)]

• Backport set to 2.0.0: DONE, 2.1: DONTNEED

backported into ruby_2_0_0 at r46733.