SharePoint 2016 Outbound SMTP Failures

Recently I was configuring a SharePoint 2016 farm, and encountered some peculiar issues with outbound email.

SharePoint 2016 is the first version of SharePoint to include built-in support for TLS. In any previous version of SharePoint, TLS requirements were fulfilled by setting up a SMTP relay capable of authenticating to the desired target SMTP server.

Interestingly, It seems that SharePoint 2016 also responds to SMTP authentication challenges despite not having an explicit configuration option in Central Administration for which credentials to use for SMTP.

The issue I recently experienced is as follows:

  • List / Library “initial” alert subscription messages are delivered to the appropriate address
  • Actual alerts from a list / library are not delivered
  • Workflow Task emails are not delivered

Digging into the ULS logs of the SharePoint server, I noticed the following:

  • Messages send by w3wp (running under the web app pool service account) were delivered
  • Messages sent by OWSTIMER (running under the farm account) were not delivered.  The timer job in question is “job-immediate-alerts.”

So, despite having outbound email configured in Central Administration, it seems that SharePoint is not treating different classes of outbound email equally.

I tried many of the “well known fixes” to no avail:

  • Re-starting the server
  • Re-starting the timer service
  • Manually starting the job-immediate-alerts timer job with PowerShell
  • Altering the alerts properties of the site with stsadm

I finally broke out WireShark on my SharePoint server to observe the SMTP traffic.  What I found was interesting:

  • Messages sent by w3wp.exe had these characteristics:
    • SharePoint sends the message immediately upon request from the browser to subscribe to alerts on a library
    • SharePoint opens a SMTP session to the configured server
    • The Exchange 2013 server responds with an SMTP ntlm authentication challenge
    • The SharePoint server provides the credentials of the web app service account!
    • Exchange returns with smtp 5.7.1 client was not authenticated. 
    • SharePoint ignores the 5.7.1 error message, and delivers the message anyway
  • Message sent by OWSTIMER.exe had these characteristics:
    • SharePoint attempts to send the message with each execution of the job-immediate-alerts timer job.
    • SharePoint opens a SMTP session to the configured server
    • The Exchange 2013 server responds with an SMTP ntlm authentication challenge
    • The SharePoint server provides the credentials of the farm service account!
    • Exchange returns with smtp 5.7.1 client was not authenticated. 
    • SharePoint stops attempting to deliver the message because of the error!

In both of these scenarios, neither the farm service account, nor the web app service account are configured with Exchange mailboxes, so the authentication fails.

The receive connector in Exchange is configured to allow TLS, Exchange Authentication, and Anonymous authentication.

The unexpected behavior is this: SharePoint reacts to an SMTP 5.7.1. unauthenticated message differently depending on the context from which the SMTP session was initiated.  SMTP sessions initiated directly in the web app context succeed, but SMTP sessions initiated from timer jobs fail.

My temporary solution was to create a separate receive connector in Exchange on a separate port scoped so to only the SharePoint server’s IP that allows only anonymous authentication (it seems that by having Exchange Authentication checked, SharePoint fails).  This causes the Exchange server to never prompt the SharePoint server for STMP authentication, and therefore messages are delivered.

I’ll update this post as I discover more.

Display Approval Tasks on InfoPath Form

I’be been working through building InfoPath forms to streamline the approval process of some internal documents, and one of the project requirements is to display the date / time as well as comments of each person who approves a document.

I built a SharePoint Designer workflow which first computes the approval routing (which varies between 8 and 10 approvers depending on the value of some fields),  then collects the approvals via the “Start Approval Process” task, and then emails up to 10 SharePoint groups based on a different set of criteria on each document.

SharePoint Designer Workflows store these “Assigned Tasks” in a Task List, which the developer is able to specify.  Each Task in the Task List contains a HIDDEN COLUMN called WorkflowItemId which associates the Task with the Item against which the workflow is running.   This column is a pesky little bugger for reasons explained below.

There is a blog post which describes one method for displaying all approvals tasks on the actual InfoPath form which goes roughly as follows:

  1. Create a new custom list containing all of the columns you need to reference
  2. Edit the “Behavior of a Single Task” for the Approval Process in question so that if the outcome is approved, add a new item to the custom list
  3. Add a Data Connection on the InfoPath form to pull data from the new custom list and display it on the form.

I didn’t want to go through the hassle of creating a separate list for each workflow I’m running, just to store data that’s already being stored in the associated Task List.

So, the big question: Why don’t you just add the Task List as an InfoPath Datasource and call it a day?

Well, the answer to that question may infuriate you: you are unable to filter the list according to the ID of the item in question because the attribute that stores the item id (WorkflowItemId) is forcibly hidden! 

  • InfoPath does not provide WorkflowItemId as an option in the Data Connection query path.
  • CSOM CAML queries error out when you attempt to use WorkflowItemId as a query field, so the SOAP / REST Data Connections in InfoPath also fail.

Other than the solution above, there are really only two other options:

I went the second route, and created such a web service, which is available here: https://github.com/bciu22/ApprovalTaskListService.

The result is that you can add an InfoPath Data Connection that looks something like this:

So that you can have a repeating table on your form with all approvals that looks something like this: