Users and groups can be added to and removed from the Team Server manually, but also using an external identity provider. If all users and groups haven been added manually, removing them can also be done manually. When users and groups have been added to the Team Server using an identity provider, removing them may work differently, depending on the identity provider you are using.
To remove a user or group from the Team Server you only need to remove it from your Azure AD environment. After a new sychronization, the remove command is passed onto the Team Server. After the removal delay period has passed, the user or group will be removed from the Team Server. This period is set at the Azure AD authentication settings in the Team Server and is to prevent data loss in case of mistakenly removed users and groups. The period has a default value of 72 hours, but can be different in your authentication settings.
However, if you want the respective user or group to be removed from the Team Server before the delay period has passed, you can manually remove that user or group from the Team Server.
If you want to remove a user or group from the Team Server, you first remove it from your SAML-based identity provider environment. After that you need to manually remove that user or group from the Team Server, because SAML cannot actively remove users and groups from the Team Server.
The SAML authentication settings in the Team Server do have a removal delay period (with a default value of 24 hours), but it only applies to group memberships. If a user was member of a specific group and has been removed from that group in the identity provider environment, then this user will on the next login in the Team Server nog longer be a member of that group. This will take effect after the delay period has passed.
If you want to remove a user from the Team Server, you first remove it from Active Directory. After that, you synchronize the users in the Team Server to remove the user from the Team Server. This is the best way of working, for example, because it will clear up any licenses this user might have taken. It is also better for maintenance reasons. If you choose to not remove the user from Active Directory, and then adjust the LDAP query in your Team Server authentication settings to exclude the user. That way you would remove the user only from the Team Server, but if the LDAP query would change again, you may risk re-importing the user again.
With Active Directory only users can be imported in the Team Server. Active Directory supports the use of groups, but they cannot be imported and synched in the Team Server. Groups are manually added to the Team Server. There is no link between groups defined in the Team Server and groups defined in Active Directory. For removing groups this means that you can just manually remove them from the Team Server.