...
If you are not assigning users specifically to the enterprise application, please make sure that ‘user assignment required’ field is set to ‘no’ otherwise users won’t be able to use the application. This field can be found by navigating to the enterprise application in Azure and should be under the settings blade.
App Federation MetaData URLs
...