Scheduled Report shows successful status; but Mail server never Receives... Where's the problem?

[Luke_D]

System Overview:

Using SAP Business Objects 4.3 SP3

We recently switched to using a mail forwarding service SendGrid which has a 20mg limit on SMTP emails.  For the most part this has worked fine but there are a few schedules which have reports generated greater than 20mg in size.  Such schedules are failing to make it to their recipient; yet the schedule shows success.  It appears this is a hard limit within SendGrid that we can't configure.  I wouldn't mind but I would have expected the BO schedule to report failed; instead the schedule returns success!  Why?

I found KBA 1778797 Email recipient does not receive report but the instance status is "Success"

This KBA indicates because the relay/mail server "accepted responsibility for delivery of the message" BOBJ's responsibility has ended; thus job status is success. 

However, SendGrid testing using telnet shows it's not accepting responsibility.  it immediately returns an error:  "Mailbox unavailable.  The server response was: error reading data, max message size exceeded."  

I'm confident there's a gap here I'm missing but I'm not technical enough to see it. 

Could it be that SendGrid receives the email message.  sends an Acknowledgement back to BOBJ which BOBJ interprets as "I got it" and BOBJ moves on?  But then SendGrid does a process sees it violates rules and sends a message back which BOBJ ignores? 

To prove this out I created two reports one which is under 20mg (barely) one over 20mg (23).  These reports run daily.  I always receive the one less than 20mg.  The 23mg one has yet to get to me.  Both show status "Success"

When I use telnet to connect to the mail relay and generate a message to send, it returns an error message indicating the message is not accepted by the Mail server because it's too big.  I don't get any other acknowledgement..  So to me there's only one message from SendGrid... I reject your mail... so why does BOBJ show success? 

Does KBA 1778797 apply here? Do I need to open a ticket with SAP for investigation or is this a influence request to better handle errors/warnings from the mail servers when sending messages?   This appears to be a GAP in the process of handling email destinations I wasn't expecting to encounter.

Note: I have no access to our Server logs (they are administrated by a 3rd party) I have no access to mail server or relay (they are administrated by a different group).  

Update: 4/29/2024

SAP KBA 177879 provides a link to RFC 5321 SMTP protocol.  It references 4.2.4 Reply Code 502 indicating the SMTP server has "Accepted" responsivity for the message.  Accepted is in quotes because the 4.2.4 specifically says, 

   Questions have been raised as to when reply code 502 (Command not
   implemented) SHOULD be returned in preference to other codes. 502
   SHOULD be used when the command is actually recognized by the SMTP
   server, but not implemented.  If the command is not recognized, code
   500 SHOULD be returned.  Extended SMTP systems MUST NOT list
   capabilities in response to EHLO for which they will return 502 (or
   500) replies.

I Take this to mean since BOBJ's message was well formed SMTP; and because the issue arose because of a mail server rule, BOBJ is saying "not my problem" and closing out job as successful (not failed, not partial success)

If we look earlier at section 4.2.1 it reads:

5yz  Permanent Negative Completion reply
      The command was not accepted and the requested action did not
      occur.  The SMTP client SHOULD NOT repeat the exact request (in
      the same sequence).  Even some "permanent" error conditions can be
      corrected, so the human user may want to direct the SMTP client to

---

      reinitiate the command sequence by direct action at some point in
      the future (e.g., after the spelling has been changed, or the user
      has altered the account status).

Thus implying emails for which a status codes in the 500 range are, as far as the mail server is concerned, "requested action did not occur.".

So I go back to I think this is a GAP.  It appears SAPs/BOBJ interpretation is: since the error code is 502; and the issue isn't with the SAP formatted message; but rather a rule within the SMTP mail client, it can disregard the error. 

If someone wants to help diagnose and have access to email logs & BOBJ Logs... 

1. Use a schedule to send a SMTP email using BOBJ which violates a SMTP rule to obtain message 502. (File too big, or some other mail rule) 
2. Confirm the job achieves success.
3. Confirm the recipient never gets the email.
4. If you have access to either the SMTP logs or the BOBJ logs provide evidence via logs what status code  you encountered.  One might be expecting 500 or 502 or 5xx; but RFC5321 indicates otherwise.

So, my guess is you can't. 

Per RFC5321 section 4.2.1, "the requested action did not occur" so why would the mail server log it?  (it doesn't per my mail admin).  I would have hoped BOBJ logs it since it was an error; but an incident with SAP resulted in, "it is quite hard to validate if the mail was delivered to SMTP server with the help of any logs."  To get the information related to this, you can surely contact your SMTP Administrator."

(I did; it's not logged since it wasn't a success per 4.2.1.)

Per KBA 1778797: 

This must be investigated by the SMTP administrator, as the SMTP server has accepted responsibility for delivery of the message.

Per RFC5321 4.2.1:

The command was not accepted, and the requested action did not occur.

But that's only true if the error code is 500+... which I can't prove since nowhere is it logged.  All I can prove is it's not logged by mail server as the mail server only logs success; therefor the message failed and would have provided a 500 message status; which isn't proof; it's an assumption.  

Like I said at the start; this is getting into the technical weeds. in my opinion it all boils down to how does BOBJ define "Accepted" vs how does RFC5321 define "Accepted".  RFC5321 says all 500 status codes are not accepted.  a telnet session with the SMTP port sending a file > 20mg to the mail server in a well formatted message resulted in an error; but no number was provided.  How can I prove its error code 500 range; and is the logic behind the "Successful" job faulty... next step: Influencer ticket.  Everything I have says error code in 500 range is returned.  I'd expect SAP job to return "Failed" or "Partial Success"

[nscheaffer]

If your recipients are internal and have access to view the report, you could send a link to the results rather than attaching them.

[Luke_D]

NScheaffer, Sadly the recipients are non-BO users. I appreciate the work around but there's other factors making sending the file somewhat important. So sending them the PDF/Excel report(s) is required. We discussed offloading the results to a SharePoint Site (or other) which the users have access but it changes the paradigm a bit from a push to a pull... and found out not all the users have access to the site due to how MSFT does licensing and what they've signed up for.  Granted we could still send an email once new content is available with the link to the file making it both a "push and pull" and this may be the approach we end up taking (albeit some users may be left out then so maybe this will not work) It just seemed odd to me Webi shows success when it wasn't. The issue here is we have other reports / schedules which may be getting close to the 20mg limit. We'd have to monitor this and find out when we exceed this value and switch over approaches since there's no way for us to know when the SendGrid will not forward the message since BO doesn't show a failed status.  

Perhaps I need to look more at SendGrid about when it "fails" to send on how we obtain such information...

Changing from a push to pull would be a somewhat sizeable paradigm shift on how we send communications as anything close to the 20mg should likely be converted.  And given the data in each report is dynamic based on parameters and data at the time, we won't always know when that 20mg limit is hit unless users tell us they didn't get the communication.  (I'd rather avoid that as its poor customer service) 

The question remains: Why does BO show success when the email/relay clearly shows it sends back a unable to send message? Are the next steps ticket iwth SAP/Influence? 

[amitrathi239]

I think  it's a correct behaviour. The problem is from SendGrid side.BO generated the report and  then SMTP server responsibilities to send the file..it is not delivered  because of SendGrid size restriction. 

You can try with SAP support ticket but more likely you will get the similar response.

[Luke_D]

I'd agree except for KBA 1778797. It states, 'The email was dropped after the SMTP server accepted the email from the Adaptive Job Server".... I show no evidence the SMTP server "Accepted" the message. A telenet to the SMTP port on SendGrid and manually generating a message results in an immediate error from the SendGrid Server. "Send-MailMesssage : Mailbox unavailable. The server response was: Error reading data, max message size exceeded" So I ask what evidence is there the BOBJ server received "Acceptance" from the SendGrid mail server? Or is BOBJ sending a "well formed email" and going with a "Fire and forget" approach? If it's this how is this an "Acceptance?" I know this is getting into the weeds; but it's what my interpertation of the text is and I want to better understand the meaning of the KBA.

[Luke_D]

Out of curiosity, Why do you think this is correct behavior or what makes this correct behavior? SMTP Status codes in 500 range according to RFC5321 4.2.1 explicity states, "The command was not accepted, and the requested action did not occur." I agree SendGrid has the rule preventing the message from being sent. But if SendGrid responds with a status code of in the 500 range with a "known error" why should BOBJ move on? if SendGrid provided a 250 or 200 range code, I'd expect it to move on no issue. But the SMTP server is sending back a "I do not accept this" with the 500 error. Putting it in terms of teh customer who has the schedule, "Why is my schedule repotting success when it never sent an SMTP "Accepted" email? Sure it sent a well formed one; and yes the mail server indicated there were no problems with the message itself; but it still reported a 500 status which says, "The command was not accepted"