Disclaimer
My postings are my own and don’t necessarily represent VMware’s positions, strategies or opinions.
External Links:
- Replace Realm in vIDM - https://docs.vmware.com/en/VMware-Identity-Manager/19.03/vidm-install/GUID-F8BDC75E-150B-4FC3-AA70-B330E26551D3.html
- Using the Built-in KDC in vIDM - https://docs.vmware.com/en/VMware-Identity-Manager/3.3/vidm-install/GUID-B5ADF0FA-7DAF-44FD-A864-0B8A04D0F07A.html
- MobileSSO for iOS/Android with Cloud KDC - https://darrylmiles.blog/2019/03/16/setting-up-workspace-one-single-sign-on-sso-and-conditional-access
Do NOT use UAG in front of vIDM for MobileSSO for iOS scenario!
Built-In KDC Schema
iOS Login Flow
DNS Entries
Given L4 Load Balancer at 1.2.3.4 and a DNS domain of example.com and a realm of EXAMPLE.COM:
kdc.example.com. 1800 IN AAAA ::ffff:1.2.3.4
kdc.example.com. 1800 IN A 1.2.3.4
_kerberos._tcp.example.com. IN SRV 10 0 88 kdc.example.com.
_kerberos._udp.example.com. IN SRV 10 0 88 kdc.example.com.
There MUST be an IPv6 AAAA entry - iPhone requires it to work correctly.
AAAA entry may have to be converted to IPv6 format: :::::ffff:102:304
Use web-tool: https://www.ultratools.com/tools/ipv4toipv6
Troubleshooting DNS records
To test the DNS settings, you can use the DIG command (built-in to Mac and Linux) or NSLOOKUP on Windows.
Here is what the dig command looks like for DNS records:
dig SRV _kerberos._tcp.example.com dig SRV _kerberos._udp.example.com # You may wish to define the name server to use with dig by using the following command: dig @ns1.no-ip.com SRV _kerberos._tcp.example.com
Checking DNS entries with tools
Use Google Toolbox to check the SRV entries:
https://toolbox.googleapps.com/apps/dig/#SRV/_kerberos._tcp.example.com
The result must contain string of type:
;ANSWER
_kerberos._tcp.example.com. 3599 IN SRV 10 0 88 krb.example.com.
Use NC to check UDP/Kerberos services:
nc -u -z kdc.example.com 88 # Answer: #> Connection to kdc.example.com port 88 [udp/kerberos] succeeded!
Use nslookup to check the SRV entries:
nslookup -q=srv _kerberos._tcp.vmwareidentity.eu Server: 10.26.28.233 Address: 10.26.28.233#53 Non-authoritative answer: _kerberos._tcp.vmwareidentity.eu service = 10 0 88 kdc.vmwareidentity.eu.
Certificates
Mobile Device Profile
The mobile device profile for the Cloud KDC feature must include the following:
- Add KDC server root certificate from previous step to credentials
- Add credential for Certificate Authority issued certificate
- Configure the SSO profile
- A user principal name that does not contain an "@" symbol, i.e., it is is typically set to "{EnrollmentUser}".
- The realm name that identifies the site where the tenant is deployed. This is the domain name for the site in upper case, e.g., VMWAREIDENTITY.COM.
- Specify the Kerberos principal name: {EnrollmentUser}
- Specify the certificate credential
- Specify the URL List: https://tenant.example.com/
- The URLPrefixMatches value in the Kerberos dictionary in the com.apple.sso payload type must include https://tenant-vidm-url/ where "tenant-vidm-url" is the FQDN of the tenant's vIDM address. Note that this FQDN may be different than the one that is used for the realm, e.g., workspaceair.com vs. vmwareidentity.com.
Specify a list of app bundle ids:
- com.apple.mobilesafari (enables access from the Safari browser)
- com.apple.SafariViewController (enables apps that use Safari View Controller)
- com.air-watch.appcenter (enables the Workspace ONE native app)
OCSP Requirements
The CA OCSP responder must be accessible on the Internet so that it can be accessed by the KDC. The URL for the OCSP responder is provided in the client certificate in the usual manner.
AirWatch has confirmed that the AW CA will meet this requirement.
vIDM/Access Appliance (SUSE Ent. Linux)
Ensure all Prereqs are completed such as ports 88 are open, and DNS is configured.
- Information on predecisions (vIDM2.8): Link
- Port 88 must be open TCP+UDP. Use packet sniffer to check packets from device.
- DNS is configured to match the upcoming initialization tasks: Link
Enable KDC
For a single Workspace ONE environment or primary domain:
/etc/init.d/vmware-kdc init --realm MY-IDM.EXAMPLE.COM # Restart horizon-workspace service: service horizon-workspace start # Start KDC service: service vmware-kdc start
Full instructions link: Initialize the KDC in the Appliance
Reinitialize the KDC
Login to the console (or SSH and SU to root) and do the following:
a. Stop the vmware-kdc service
b. Force reinitialization of the KDC service (note the use of the '--force' switch at the end)
c. Restart the vmware-kdc service
d. Restart the horizon-workspace service
service vmware-kdc stop /etc/init.d/vmware-kdc init --realm EXAMPLE.COM --subdomain idm.example.com --force service vmware-kdc start service horizon-workspace restart #Or just type reboot to restart the appliance.
Enable KDC using subdomain
This is useful for multiple Workspace ONE environments (Prod, UAT, DEV). Remember that realm must be all CAPs and subdomain is all lowercase.
- /etc/init.d/vmware-kdc init --force --realm SUBDOMAIN.IDM.DOMAIN.COM --subdomain subdomain.idm.customer.com
- Restart horizon-workspace service: service horizon-workspace start
- Start KDC service: service vmware-kdc start
Flags:
--force (force init erasing previous initialized KDC)
--realm (by convention, this is all caps, this is the Kerberos realm and is usually the DNS domain of the customer. It can also be a DNS sub domain if the customer needs to have multiple Kerberos realms for multiple systems
--subdomain (this is not related to subdomain in the conventional sense. This should be your IDM's FQDN that the end users use)
Change KDC using subdomain
Please be aware that this procedure is unnecessary if the "vmware-kdc init" command is run with the correct values in the first place. If a reinitialization is necessary, then there are two better options.
If the KDC CA cert doesn't have to be maintained, then just run "vmware-kdc init --force" with the right arguments, or
If the KDC CA cert does need to be maintained (so that the AirWatch profile doesn't have to be updated), then copy the kdc-cacert.pem, kdc-cert.pem, and kdc-certkey.pem files from /opt/vmware/kdc/conf to a temporary area, make them readable by the horizon user, and then run "vmware-kdc init --force" but use the arguments that allow the certs to be passed in (--kdc-cacert, --kdc-cert, and --kdc-certkey).
How to update kdc subdomain without regenerating certificates
Navigate to the kdc-init.input file and change the subdomain there to the new value
- service vmware-kdc update
- service vmware-kdc restart
- Enter command /etc/init.d/vmware-kdc kadmin
- Next command: addprinc -clearpolicy -randkey - requires preauth HTTP/(insertnewsubdomain)
- Next command: ktadd -k kdc-adapter.keytab HTTP/(insertnewsubdomain)
- Next command: listprincs
- You should see the HTTP/(newsubdomain) after the listprincs command. This confirms a successful update of the subdomain.
Windows-based IDM
Windows VIDM uses the cloud KDC which only requires the appliance to reach outbound on 443 and 88. The clients also need to be able to reach outbound on 443 and 88.
Full directions are listed on https://docs.vmware.com/en/VMware-Workspace-ONE/services/WS1-IDM-deploymentguide.pdf
One item not clear is that for Windows and cloud KDC is the REALM used in AirWatch and in Windows VIDM is: OP.VMWAREIDENTITY.COM
Troubleshooting MobileSSO for iOS
Reasons for failure
- The device isn't enrolled
- The device is enrolled but cannot access port 88 on the VIDM service
- Response from port 88 on the VIDM service cannot reach device
- DNS entries not configured correctly
- The device is enrolled but the profile has the wrong realm
- The device is enrolled but doesn't have a certificate in the profile
- The device is enrolled but has the wrong certificate in the profile
- The device is enrolled but the Kerberos principal is wrong
- The certificate SAN values cannot be matched with the Kerberos principal
- The device is enrolled but the KdcKerberosAuthAdapter has not been configured with the issuer certificate
- The device is enrolled but the user's certificate has been revoked (via OCSP)
- The device is enrolled but doesn't send the certificate during login
- The device is enrolled but the app bundle id for the app isn't in the profile
- Time is not synced between device and KDC (this will cause the TGT to be issued and a TGS_REQ to be made, but the device will reject the service ticket and asking for an "http/fqdn" ticket)
- Time is not synced between KDC nodes (this will cause save of adapter settings to fail)
- KDC subdomain value is set incorrectly (this causes the TGS_REQ to be rejected)
General checks
- What is not working?
- Unable to save Mobile SSO for iOS adapter settings
- Is the error message "Checksum failed"?
- Is this a cluster?
- How was the data copied from one node to another? / Was "vmware-kdc init" run on multiple nodes?
- Use dump/load to replicate the data to all nodes rather than running init on all nodes.
- How was the data copied from one node to another? / Was "vmware-kdc init" run on multiple nodes?
- Is this a cluster?
- Is the error message "Checksum failed"?
- Device cannot login via Safari
- Device cannot login via Workspace ONE app
- Device cannot login via some other native app
- Is the app bundle id included in the profile
- Does the app use Safari View Controller?
- Is com.apple.SafariViewController included in the list of app bundle ids in the profile?
- Unable to save Mobile SSO for iOS adapter settings
- If the device cannot login, why is the login failing?
- Device makes no request at all to the KDC
- vIDM policy not setup correctly
- Device cannot reach the KDC
- DNS lookup issue
- Networking issue with reaching the KDC
- AS_REQ received but certificate isn't validated
- Does the certificate have acceptable SAN values?
- Is OCSP used?
- Is the OCSP server reachable?
- Is the OCSP server validating the certificate?
- TGT issued but TGS_REQ not issued
- Is the subdomain set correctly
- Is the service ticket ignored by the device
- Is the time on the KDC service sync'd with the device?
- Device makes no request at all to the KDC
- Is it failing for all devices or just a subset or one device?
- What version of iOS is being used?
- iOS 9.3.5 is known not to work for Safari
- iOS 10.3.3 has a known issue with native apps that use SafariViewController
- Can iOS device reach the KDC? (use sniffer to check)
- There may be a networking issue. Sometime firewalls or load balancers do not pass through UDP and TCP traffic on port 88
- Are the DNS SRV entries set up correctly?
- Is the subdomain value for the on-premises KDC set correctly?
- If using Hybrid KDC, does registration fails to save the adapter configuration?
- Is the vIDM service node able to resolve the _mtkadmin._tcp.<realm> SRV entry?
- Is the vIDM service node able to reach the https://hybrid-kdc-admin.<realm>/mtkadmin/rest/health URL?
- Is the vIDM service node able to reach UDP port 88 for the KDC?
- Are responses from the KDC with source UDP port 88 able to reach the vIDM service node?
- Is the AirWatch tunnel being used?
- Is the tunnel bypassing port 88 traffic to allow it to reach the KDC?
- Is REALM domain in KDC and DNS all CAPITAL LETTERS, and subdomain all lower letters?
- If you are using {Enrollment User} as the Kerberos principal in the device profile and {Enrollment User} is mapped to the sAMAccountName, and the sAMAccountname does not match the first portion of the user’s UPN – which is everything before the @ sign – then mobile SSO will fail.
- For certificate authentication to work (Android SSO included), UserPrincipalName value in SAN needs to be set to user's UPN or Email, in user's certificate or SAN must have user's email if UPN is missing.
- When creating the certificate template in ADCS - use the KerberosAuthentication template with the following modifications: change the template name, change Subject Name to accept it from the request, add a new "Kerberos Client Authentication" auth policy (oid=1.3.6.1.5.2.3.4), add the AirWatch service account to the list of users that can use the certificate.
The authentication policy oid must be exactly as specified
vIDM Logs
Built-in Kerberos adapter log: /opt/vmware/horizon/workspace/logs/horizon.log
KDC logs: /var/log/messages log and grep for 'pkinit' to see the errors returned.
grep -i 'pkinit' /var/log/messages
This may return errors like vmw_service_matches_map: “HTTP/idm.test.ws@TEST.WS" is not a match for the service_regex and Server not found in Kerberos database. Basically, either DNS is not correct or built-in KDC Server is not properly initialized.
How to Enable Kerberos iOS Device Logs to troubleshoot Mobile SSO
When having issues troubleshooting Mobile SSO for iOS, install the GSS Debug Profile on the device to retrieve more verbose Kerberos logs.
- Copy this config file to your desktop.
- Email the config file to the device.
- From the device, tap on the attachment, tap on Install. Enter the device passcode when prompted.
- Tap on Install in the upper right on the Apple Consent notification.