ENS Basic Troubleshooting

Warning

Database for ENSv2 must be named “ENS”.

Alive Check

Cloud-hosted ENS: https://ens-eu.getboxer.com/api/ens https://ens-eu.getboxer.com/api/ens/alive

Steps:

  1. publickey request
    a. The device requests a public key to encrypt the account credentials with. It sends a hash of the email address as the userid. This helps identify the user and link together all user devices

  2. subscribe
    a. device sends encrypted creds, user id (server created) and device apns token so ENS server has all the necessary pieces to subscribe and get notifications of new emails

  3. push subscription
    a. ENS discovers endpoint based on creds and subscribes to exchange using a webhook link that contains the encrypted credentials ie: ens.airwatch.com/notify?id=&creds=<Base64(RSAEncrypted(username:password))>

  4. new email notification
    a. Exchange sends notification of changes to the provided url.
    b. ENS extracts and decrypt creds and prepares call to fetch email

  5. email fetch
    a. ENS performs a fetch for the email

  6. push email
    a. ENS finds user devices with the user id and pushes email details to CNS for delivery to all user devices

On-prem ENS: https://_ENS_HOST/MailNotifications/api/ens https://_ENS_HOST/MailNotifications/api/ens/alive

Registration Interaction

The following diagram shows in more detail a registration/new email interaction between the client, ENSv2 and the exchange server. This diagram shows in more detail how we can use credentials without keeping them saved inside the ENSv2 environment.

ENSv2 Database Details - see Article

Locale

Warning

ENS may cause errors in case of date mismatch Ensure that server, DB, etc have US English locate!

Boxer - Add Assingment

Network

ENSLINKAddress On-Premise, should point to the externally accessible hostname pointed to ENS service. A support ticket has to be made with VMware Airwatch to request API token (internally the support reaches out to Boxer product manager and requests API key).

MS Exchange Server

User agent is configured for ENSv2 on MS Exchange Server CAS role. User agent must have access to receive data from MS Exchange, or ENS will not be able to receive PUSH notifications.

Troubleshooting

ENSLINKAddress for On-Premise installation should point it correctly to the customer’s externally accessible hostname pointed to ENS service.

Autodiscovery errors showing on logs.

  • Make sure that the EWSUrl key is configured in the console with a correct value for the EWS url for their exchange environments.
    Test it out by opening it on the browser and making sure you are prompted for credentials.
    Tip

    Alternatively they can just turn ON autodiscovery on their environment.

Authentication errors (401s)

  • Check what type of authentication is enabled in EWS? Make sure it has parity with whatever they are using for ActiveSync (Basic, OAuth, CBA) as Boxer will be the one to pass whatever type of credentials it has to ENS to use against EWS.
  • Verify the EWSUrl is correct and resolves to Exchange environment and test the credentials used by navigating to the EWS URL and testing them there.
  • Run “Outlook Connectivity Tests” on https://testconnectivity.microsoft.com
    Tip

    Try the Exchange Server or Office 365 tabs accordingly

ENSv2 not sending notifications but no logs found on help portal.

  • Verify the ENSAPIToken key is correct in the console configuration.

  • Verify the ENSLinkAddress key is correct in the console configuration

    Tip

    Try appending the “alive” endpoint for the environment and make sure it responds.

  • Verify the EWSUrl is correctly configured with a valid EWS value.

  • Verify that ENSv2 servers have inbound access to their EWS environment (firewall may be blocking access, they need to open the corresponding IPs)

  • Verify that EWS can send outbound traffic to the corresponding ENSv2 domain (https://.getboxer.com/api/ens)

General FAQ

  • **How are credentials or authentications tokens handled?
    **
    • Although the client does share the credentials/tokens with the ENSv2 environment upon registration, they are not kept (saved anywhere) by AirWatch servers. Rather, the Exchange server passes them back to AirWatch, encrypted, as part of a notification it sends whenever a new email is available. From that notification (Exchange -> ENSv2), ENS decrypts the credentials and uses them to make any requests necessary to the Exchange server. After performing any necessary requests, the credentials are once again discarded.
Warning

Bug in MS Exchange detected: Exchange returns the email information even though the user is not the owner. This results in the notification payload being created for the wrong user and ultimately another user seeing the notification. 

With ENSv2 1.2+, a new service object is created for each EWS request. This will prevent the application from making a request to the EWS endpoint with different credentials.

Patch for Exchange needed from Microsoft on this, since this is unexpected Exchange behavior.

  • If credentials are not persisted, is there any data persistent at all by ENS? How is it secured?
    • There is a secure database that keeps a list of devices and a list of public private key pairs used to unencrypt the credentials when they come from Exchange;
    • Logs are also kept to aide in debugging issues and monitoring the system. These don’t contain any customer’s private information and access to them is also tightly secured via account permissions.
  • What data is transmitted through the ENS server without being persisted? How is it secured?
    • User credentials (encrypted with RSA encryption)
    • Email subject and sender (sent via HTTPS)
    • All communication is done via HTTPS
  • What additional cloud services does ENS depend on?
    • AWS Simple Notification Service (SNS) for push notification handling.
    • Apple Push Notification Service (APNS) as it is the only way to pass notifications to Apple devices.
    • AWS Relational database service (RDS) for data persistence.
  • What is the user agent used by ENSv2 when sending requests to Exchange?
    • MailNotificationService/v2 (ExchangeServicesClient/15.00.0913.015+ (will change as new libraries from Microsoft are released)
  • What email folders does ENSv2 monitor for incoming messages and actions?
    • Currently, ENSv2 only monitors each user’s Inbox folder.

Load Balancing ENSv2

For HA, it is recommended to load balance several ENS web servers as needed following the Hardware Requirements. All web servers should point to the same database server as this will be their shared source of state for each of the clients.

Since the ENS web application itself is stateless there are no requirements to configure any session handling (stickiness) in the loadbalancer so a straightforward configuration should suffice.